News
July 31st, 2014
The first articles appeared in the proceedings. Feb 3rd, 2014
The organising committee has declared the winners. Jan 15th, 2014
The automatic evaluation results are out. Jan 15th, 2014
Check the authors information on the proceedings page. Jan 1st, 2014
The solutions are online. Dec. 3, 2013
The submission deadline is approaching fast! Please email submissions to casmi2013@massbank.jp Oct. 1, 2013
Details about the CASMI 2013 Special Issue and dates are now available! Sept. 24, 2013
The rules and challenge data pages have been updated. Sept. 2, 2013
The CASMI 2013 Challenges have been officially released! August 29, 2013
The challenges for CASMI2013 will be released on Monday, September 2nd!
July 31st, 2014
The first articles appeared in the proceedings. Feb 3rd, 2014
The organising committee has declared the winners. Jan 15th, 2014
The automatic evaluation results are out. Jan 15th, 2014
Check the authors information on the proceedings page. Jan 1st, 2014
The solutions are online. Dec. 3, 2013
The submission deadline is approaching fast! Please email submissions to casmi2013@massbank.jp Oct. 1, 2013
Details about the CASMI 2013 Special Issue and dates are now available! Sept. 24, 2013
The rules and challenge data pages have been updated. Sept. 2, 2013
The CASMI 2013 Challenges have been officially released! August 29, 2013
The challenges for CASMI2013 will be released on Monday, September 2nd!
Evaluation Details
To be considered, the submissions will have to pass
the automatic
evaluation without manual intervention.
Participants can check their submissions prior to the deadline
by emailing their files for a test run through
the evaluation script (this has no influence on the final results).
Only one submission
per challenge and category is allowed.
The submissions need to be
in one file per category and challenge, named:
<participant>-<category>-<challenge>.txt
Please choose a participant name that is closely related to
the name of the contact author, but is likely to be unique,
e.g. Jane Smith (participant name = jsmith)
and Michael Mustermann (participant name = mmuster).
Please avoid white space, special characters
including “./” in the participant description and also
“-”, except as delimiter as shown above,
e.g. sneumann-category2-challenge0.txt.
Each file must have at least two columns separated by
tab stop, without column or row names. Additional
columns will be ignored (which means you could add any
information).
Please indicate in the submission metadata file whether the entries were
generated using essentially automated methods (where manual intervention is
limited to converting input/output of the programs) or manual methods which
include expert knowledge.
All participants will be assessed with the automatic evaluation, which
calculates the (worst case) absolute ranks and score-based criteria as
described in the summary paper of CASMI 2012,
CASMI: And the Winner Is...
For Category 1 (molecular formula), the absolute rank will be used to
determine the winner for each challenge for both automatic and manual
entries.
For Category 2 (molecular structure), the absolute rank of the correct
molecular structure will be used to determine the winner for each challenge
for automated entries. How well the participants’ programs work will be
discussed in the summary paper following the competition.
For manual entries in Category 2, the organisers will consider how rational
the metadata is and then how close the proposed molecular structure is to
the correct answer using the Tanimoto similarity concept. Please include
sufficient details of the manual judgements in the “metadata” file
accompanying manual submissions.
The participant who wins the most challenges per category will be declared
the overall winner. Score-based ranks and additional considerations (how
well the program worked for automated entries, or the quality of the
metadata for manual entries) will be used to rank participants with the
equal number of “wins”.