We have defined four categories for challenge data from high-resolution LC-MS and low resolution (nominal mass) GC-MS data:

1. Best molecular formula for high resolution LC/MS data
2. Best identification for high resolution LC/MS data
3. Best molecular formula for low resolution GC/MS data
4. Best identification for low resolution GC/MS data

All contributions have to follow the principles of reproducible research, and need to accurately describe how the results were achieved.

To be considered, your submissions will have to pass the automatic evaluation without manual intervention. We can assist you with feedback if you send us an early version of your submission so we can run it through the evaluation.

For each of the challenges, we are looking for a list of identifications with non-negative scores, where a higher score corresponds to a more likely candidate formula or structure. We only allow one submission per challenge and category. As shown in the example submissions, your predictions need to be in one file per category and challenge, with the naming scheme <participant>-<category>-<challenge>.txt (please avoid white space, special characters including “./” in the participant description and also “-”, except as delimiter as shown above) e.g. sneumann-category1-challenge0.txt. Each file must have at least two columns (see the category-specific format description for details) separated by tab stop, without column or row names. Additional columns will be ignored (which means you could add any information).

For each challenge, we will use the absolute rank of the correct solution (we sort the candidates by score) to compare the participants and declare the winner for each challenge. In case of tied scores, we use the maximum (i.e. worst-case) rank. The overall winner will then be based on the number of victories.

In addition to the absolute ranks, we will also calculate score-based ranks. Within each submission file, we normalize the scores such that sum(scores)=1. Then, instead of the rank, we add up all scores an experimentalist has to look at before reaching the correct solution. This evaluation will shed light on how useful the scores are in practice.

Finally, for each participant and category, we need one metadata file that cotains the "bibliographic" information and description of the submission. This includes information on whether the results can be obtained with an automated pipeline or integrated system (i.e. where the identification works without user interaction, except for the CASMI related formatting) and whether spectral libraries were used.

We have created an example submission for Challenge0, which includes the MetFrag results, and the corresponding MetaData. Please model your submissions on this example.