News
Oct 31st, 2017
The results are now available. Oct 30th, 2017
The solutions are now available. Sept 8th, 2017
Update for Challenge 15 available, but will not count in evaluation. Sept 4th, 2017
Updated mailling list and submission information. Aug 23rd, 2017
The preliminary results have been sent out to participants, and are now available. July 09th, 2017
We fixed the intensities in the TSV archive for challenges 046-243. June 22nd, 2017
We added the Category 4 on a subset of the data files. May 22nd, 2017
We have improved challenges 29, 42, 71, 89, 105, 106 and 144. April 26th, 2017
The rules and challenges of CASMI 2017 are public now ! Jan 20th, 2017
Organisation of CASMI 2017 is underway, stay tuned!
Oct 31st, 2017
The results are now available. Oct 30th, 2017
The solutions are now available. Sept 8th, 2017
Update for Challenge 15 available, but will not count in evaluation. Sept 4th, 2017
Updated mailling list and submission information. Aug 23rd, 2017
The preliminary results have been sent out to participants, and are now available. July 09th, 2017
We fixed the intensities in the TSV archive for challenges 046-243. June 22nd, 2017
We added the Category 4 on a subset of the data files. May 22nd, 2017
We have improved challenges 29, 42, 71, 89, 105, 106 and 144. April 26th, 2017
The rules and challenges of CASMI 2017 are public now ! Jan 20th, 2017
Organisation of CASMI 2017 is underway, stay tuned!
Evaluation Scipts
After the submission closes, we are going to drop the submissions into a folder for
each
participant
, and run the
evaluation script
on all of them. For the first leaderboard evaluation, we will anonymise the participants and create a table showing only the Formula 1 score. Before the
final evaluation and public ranking,
participants will get a sneak preview of the evaluation results a few days before they go public
to make sure all is working well.
The script will read the table of correct solutions,
preprocess the submissions (normalising the scores
to sum(score)=1) and calculate the evaluation
measures. Finally, a set of HTML tables and graphics are
produced to give an overview of the results.