News

April 26th, 2017
The rules and challenges of CASMI 2017 are public now !

Jan 20th, 2017
Organisation of CASMI 2017 is underway, stay tuned!


Evaluation Scipts
After the submission closes, we are going to drop the submissions into a folder for each participant , and run the evaluation script on all of them. For the first leaderboard evaluation, we will anonymise the participants and create a table showing only the Formula 1 score. Before the final evaluation and public ranking, participants will get a sneak preview of the evaluation results a few days before they go public to make sure all is working well. The script will read the table of correct solutions, preprocess the submissions (normalising the scores to sum(score)=1) and calculate the evaluation measures. Finally, a set of HTML tables and graphics are produced to give an overview of the results.