News

April 26th, 2017
The rules and challenges of CASMI 2017 are public now !

Jan 20th, 2017
Organisation of CASMI 2017 is underway, stay tuned!


Submission format

Submission to CASMI requires an abstract file plus results files for each challenge.

The abstract should be provided as a plain text file containing details on the method and, where appropriate, metadata used. The file should be named in the form: <participant>-<category>.txt. A template is available here and examples from 2014 are available here. Participant names must contain only letters and numbers, no dashes, spaces nor special characters.

For each challenge, the results should be submitted as a plain text, tab separated file with at least two columns. The first column should contain the representation of the structure as the (standard) InChI or the SMILES code. The second column should contain the score. The score should be non-negative with a higher score representing a better candidate for the evaluation script to work properly. Any further information can be contained in additional columns (but will be ignored for the automatic evaluation). Do not include headers or row names. Report the small molecules themselves, not the adduct species like [M+H]+. Molecules that you consider to be charged in the natural state should be reported as such. The file name must be in the format <participant>-<category>-<challenge>.txt, see here for an example. Only the two dashes shown are allowed in the file names. If you enter a category with different submissions, use different participant names, e.g. MetFragWithCitations and MetFragWithRT. Each submission requires its own abstract.

Send your submissions before the deadline (see deadlines) in a ZIP archive to casmi-team2017, see email at the bottom.

Evaluation and declaration of winners

To be considered, your submissions will have to pass the automatic evaluation without manual intervention. The evaluation pipeline filters and remove all stereo-isomers, i.e. we will calculate the InChIkey for each submission, and for all compounds with identical InChIkey first block, only retain the one with the highest score for the following ranking.

For each unknown, the absolute rank of the correct solution (ordered by score) will be determined and, if the score is equal among several candidates, the average rank over all these candidates will be used. This year, we will use the "Formula 1 Score", based on the method used in Formula 1 racing, which is the sum of the Top 1 to 10 ranks of the correct candidates weighted by the points 25 (Top1), 18, 15, 12, 10, 8, 6, 4, 2 and 1 for correct solution ranked 10th. See also the CASMI 2016 paper.

The role of the organisers will be to resolve any issues, such as (but not limited to): 1) Re-categorising an entry (submitter will be notified as soon as possible after submission) 2) Including a valid tautomer of the correct solution, if this was not considered initially in the automatic evaluation. The tautomer InChIKey will be added and all entries re-evaluated. 3) Fixing any bugs in the evaluation script - once fixed, all entries will be re-evaluated. Please report any issues to the CASMI team, see email link below. 4) Reporting format issues to participants for them to fix and re-submit immediately.