News
March 29th, 2017
The CASMI 2016 Cat 2+3 paper is out! Jan 20th, 2017
Organisation of CASMI 2017 is underway, stay tuned! Dec 4th, 2016
The MS1 peak lists for Category 2+3 have been added for completeness. May 6th, 2016
The winners and full results are available. April 25th, 2016
The solutions are public now. April 18th, 2016
The contest is closed now, the results are fantastic and will be opened soon! April 9th, 2016
All teams who submit before the deadline April 11th will be allowed to update the submission until Friday 15th. February 12th, 2016
New categories 2 and 3 and data for automatic methods released. 10 new challenges in category 1. January 25th, 2016
E. Schymanski and S. Neumann joined the organising team, additional contest data coming soon. January 11th, 2016
New CASMI 2016 raw data files are available.
March 29th, 2017
The CASMI 2016 Cat 2+3 paper is out! Jan 20th, 2017
Organisation of CASMI 2017 is underway, stay tuned! Dec 4th, 2016
The MS1 peak lists for Category 2+3 have been added for completeness. May 6th, 2016
The winners and full results are available. April 25th, 2016
The solutions are public now. April 18th, 2016
The contest is closed now, the results are fantastic and will be opened soon! April 9th, 2016
All teams who submit before the deadline April 11th will be allowed to update the submission until Friday 15th. February 12th, 2016
New categories 2 and 3 and data for automatic methods released. 10 new challenges in category 1. January 25th, 2016
E. Schymanski and S. Neumann joined the organising team, additional contest data coming soon. January 11th, 2016
New CASMI 2016 raw data files are available.
Results in Category 3
Summary of Challenge wins
Allen (retrained) | Kind | |
---|---|---|
Gold | 156 | 159 |
Silver | 52 | 38 |
Bronze | 0 | 0 |
Gold (neg) | 61 | 64 |
Gold (pos) | 95 | 95 |
Summary statistics per participant
Mean rank | Median rank | Top | Top3 | Top10 | Mean RRP | Median RRP | |
---|---|---|---|---|---|---|---|
Allen (retrained) | 13.62 | 1.0 | 120 | 160 | 182 | 0.971 | 1.000 |
Kind | 6.40 | 1.0 | 146 | 162 | 174 | 0.904 | 1.000 |
Summary of Rank by Challenge and Participant
For each challenge, the rank of the winner(s) is highlighted in bold. If the submission did not contain the correct candidate this is denoted as "-". If someone did not participate in a challenge, nothing is shown. The tables are sortable if you click into the column header. This summary is also available as CSV download.Allen (retrained) | Kind | |
---|---|---|
challenge-001 | 1.5 | 2.0 |
challenge-002 | 2.0 | 4.0 |
challenge-003 | 7.5 | 16.5 |
challenge-004 | 1.5 | 2.0 |
challenge-005 | 1.0 | 1.0 |
challenge-006 | 14.0 | 18.0 |
challenge-007 | 1.0 | 1.0 |
challenge-008 | 1.0 | 1.0 |
challenge-009 | 1.0 | 1.0 |
challenge-010 | 6.0 | 27.5 |
challenge-011 | 1.0 | 1.0 |
challenge-012 | 1.0 | 1.0 |
challenge-013 | 2.0 | 1.0 |
challenge-014 | 19.5 | 1.0 |
challenge-015 | 1.0 | 1.0 |
challenge-016 | 1.0 | 1.0 |
challenge-017 | 1.0 | 4.5 |
challenge-018 | 1.0 | 1.0 |
challenge-019 | 1.0 | 1.0 |
challenge-020 | 3.0 | 2.0 |
challenge-021 | 1.0 | 1.0 |
challenge-022 | 7.0 | 4.0 |
challenge-023 | 1.5 | 2.0 |
challenge-024 | 1.0 | 1.0 |
challenge-025 | 1.0 | 1.0 |
challenge-026 | 1.5 | 44.0 |
challenge-027 | 94.5 | 94.5 |
challenge-028 | 7.0 | 1.0 |
challenge-029 | 1.0 | 1.0 |
challenge-030 | 1.0 | 1.0 |
challenge-031 | 1.0 | 1.0 |
challenge-032 | 42.0 | 58.0 |
challenge-033 | 4.0 | 1.0 |
challenge-034 | 3.0 | 1.0 |
challenge-035 | 1.0 | 6.5 |
challenge-036 | 1170.5 | 1.0 |
challenge-037 | 1.0 | 1.0 |
challenge-038 | 1.0 | 1.0 |
challenge-039 | 5.0 | 9.5 |
challenge-040 | 2.0 | 1.0 |
challenge-041 | 437.0 | 65.5 |
challenge-042 | 1.5 | 2.0 |
challenge-043 | 1.5 | 3.0 |
challenge-044 | 1.0 | 1.0 |
challenge-045 | 3.0 | 1.0 |
challenge-046 | 8.5 | 94.0 |
challenge-047 | 136.0 | 3.0 |
challenge-048 | 1.0 | 1.0 |
challenge-049 | 1.5 | 1.0 |
challenge-050 | 1.0 | 1.0 |
challenge-051 | 1.0 | 1.0 |
challenge-052 | 1.0 | 18.0 |
challenge-053 | 1.0 | 1.0 |
challenge-054 | 3.0 | 1.0 |
challenge-055 | 1.0 | 1.0 |
challenge-056 | 1.0 | 1.0 |
challenge-057 | 1.0 | 1.0 |
challenge-058 | 1.0 | 1.0 |
challenge-059 | 1.0 | 1.0 |
challenge-060 | 2.0 | 1.0 |
challenge-061 | 1.0 | 1.0 |
challenge-062 | 1.0 | 1.0 |
challenge-063 | 1.0 | 1.0 |
challenge-064 | 1.0 | 1.0 |
challenge-065 | 2.0 | 1.0 |
challenge-066 | 4.5 | 1.0 |
challenge-067 | 1.0 | 1.0 |
challenge-068 | 1.0 | 1.0 |
challenge-069 | 1.0 | 1.0 |
challenge-070 | 1.0 | 1.0 |
challenge-071 | 1.0 | 1.0 |
challenge-072 | 1.0 | 1.0 |
challenge-073 | 1.0 | 1.0 |
challenge-074 | 1.0 | 1.0 |
challenge-075 | 1.0 | 1.0 |
challenge-076 | 1.0 | 2.0 |
challenge-077 | 2.0 | 1.0 |
challenge-078 | 7.0 | 1.0 |
challenge-079 | 1.0 | 1.0 |
challenge-080 | 1.0 | 1.0 |
challenge-081 | 2.0 | 1.0 |
challenge-082 | 1.0 | 1.0 |
challenge-083 | 19.0 | 50.0 |
challenge-084 | 1.0 | 1.0 |
challenge-085 | 4.0 | 22.0 |
challenge-086 | 1.0 | 1.0 |
challenge-087 | 85.0 | - |
challenge-088 | 1.0 | 1.0 |
challenge-089 | 123.0 | - |
challenge-090 | 13.0 | 1.0 |
challenge-091 | 14.0 | 4.5 |
challenge-092 | 23.0 | 13.0 |
challenge-093 | 1.0 | 1.0 |
challenge-094 | 1.0 | 1.0 |
challenge-095 | 3.0 | 1.0 |
challenge-096 | 1.0 | 1.0 |
challenge-097 | 1.0 | 1.0 |
challenge-098 | 1.0 | 1.0 |
challenge-099 | 1.0 | 1.0 |
challenge-100 | 7.0 | 14.0 |
challenge-101 | 10.0 | - |
challenge-102 | 2.0 | 1.0 |
challenge-103 | 6.0 | 13.0 |
challenge-104 | 3.0 | 1.0 |
challenge-105 | 1.0 | 1.0 |
challenge-106 | 2.0 | 1.0 |
challenge-107 | 1.0 | 77.0 |
challenge-108 | 1.0 | 1.0 |
challenge-109 | 1.0 | 1.0 |
challenge-110 | 1.0 | 1.0 |
challenge-111 | 1.0 | 1.0 |
challenge-112 | 1.0 | 1.0 |
challenge-113 | 4.0 | 1.5 |
challenge-114 | 1.0 | 1.0 |
challenge-115 | 1.0 | 1.0 |
challenge-116 | 2.0 | 35.0 |
challenge-117 | 1.0 | 1.0 |
challenge-118 | 1.0 | 1.0 |
challenge-119 | 45.0 | - |
challenge-120 | 2.0 | 1.0 |
challenge-121 | 3.0 | 1.0 |
challenge-122 | 1.0 | 1.0 |
challenge-123 | 1.0 | 1.0 |
challenge-124 | 1.0 | 1.0 |
challenge-125 | 1.0 | 25.0 |
challenge-126 | 6.0 | - |
challenge-127 | 1.0 | - |
challenge-128 | 1.0 | 3.0 |
challenge-129 | 28.0 | 3.5 |
challenge-130 | 1.0 | 1.0 |
challenge-131 | 28.0 | - |
challenge-132 | 1.0 | 1.0 |
challenge-133 | 1.0 | 1.0 |
challenge-134 | 1.0 | 2.0 |
challenge-135 | 2.0 | 22.0 |
challenge-136 | 21.0 | 47.0 |
challenge-137 | 1.0 | 1.0 |
challenge-138 | 2.0 | 1.0 |
challenge-139 | 1.0 | 1.0 |
challenge-140 | 8.0 | 1.0 |
challenge-141 | 6.0 | 1.0 |
challenge-142 | 1.0 | 1.0 |
challenge-143 | 3.0 | 1.0 |
challenge-144 | 2.0 | 1.0 |
challenge-145 | 1.0 | 1.0 |
challenge-146 | 2.0 | 1.0 |
challenge-147 | 1.0 | 1.0 |
challenge-148 | 2.0 | 1.0 |
challenge-149 | 1.0 | 1.0 |
challenge-150 | 1.0 | 1.0 |
challenge-151 | 1.0 | 1.0 |
challenge-152 | 1.0 | 1.0 |
challenge-153 | 3.0 | 6.0 |
challenge-154 | 19.0 | 23.0 |
challenge-155 | 1.0 | 1.0 |
challenge-156 | 1.0 | 1.0 |
challenge-157 | 15.0 | 8.0 |
challenge-158 | 1.0 | 1.0 |
challenge-159 | 32.0 | 6.0 |
challenge-160 | 4.0 | 1.0 |
challenge-161 | 1.0 | 3.0 |
challenge-162 | 7.0 | 3.0 |
challenge-163 | 3.0 | 2.0 |
challenge-164 | 1.0 | 1.0 |
challenge-165 | 1.0 | 1.0 |
challenge-166 | 1.0 | 1.0 |
challenge-167 | 1.0 | 1.0 |
challenge-168 | 2.0 | 1.0 |
challenge-169 | 1.0 | 1.0 |
challenge-170 | 1.0 | 1.0 |
challenge-171 | 34.0 | - |
challenge-172 | 1.0 | 1.0 |
challenge-173 | 1.0 | 1.0 |
challenge-174 | 1.0 | 1.0 |
challenge-175 | 13.0 | 66.0 |
challenge-176 | 1.0 | 1.0 |
challenge-177 | 1.0 | 1.0 |
challenge-178 | 2.0 | 1.0 |
challenge-179 | 1.0 | 1.0 |
challenge-180 | 22.0 | 39.0 |
challenge-181 | 1.0 | 1.0 |
challenge-182 | 1.0 | 1.0 |
challenge-183 | 1.0 | 9.0 |
challenge-184 | 1.0 | 1.0 |
challenge-185 | 1.0 | 1.0 |
challenge-186 | 2.0 | 1.0 |
challenge-187 | 6.0 | 1.0 |
challenge-188 | 14.0 | 1.0 |
challenge-189 | 1.0 | 1.0 |
challenge-190 | 1.0 | 1.0 |
challenge-191 | 1.0 | 1.0 |
challenge-192 | 2.0 | 1.0 |
challenge-193 | 1.0 | 7.0 |
challenge-194 | 1.0 | 1.0 |
challenge-195 | 1.0 | 1.0 |
challenge-196 | 1.0 | 2.0 |
challenge-197 | 2.0 | 1.0 |
challenge-198 | 13.0 | - |
challenge-199 | 1.0 | - |
challenge-200 | 2.0 | 3.0 |
challenge-201 | 1.0 | 1.0 |
challenge-202 | 2.0 | 1.0 |
challenge-203 | 1.0 | 1.0 |
challenge-204 | 5.0 | - |
challenge-205 | 4.0 | 1.0 |
challenge-206 | 1.0 | 1.0 |
challenge-207 | 19.0 | 123.0 |
challenge-208 | 1.0 | 1.0 |
Participant information and abstracts
Participant: Allen Authors: Felicity Allen, Tanvir Sajed, Russ Greiner, David Wishart Affiliations: Department of Computing Science University of Alberta, Canada ParticipantID: FelicityAllenCFMRetrained Category: category3 Automatic pipeline: yes Spectral libraries: no Abstract We processed the list of molecules and provided candidates using cfm-id. We combined two scores: CFM_SCORE + DB_SCORE CFM SCORE A new CFM model trained on data from Metlin and NIST MS/MS was used for the positive mode spectra. This new model also incorporated altered chemical features and a neural network within the transition function. Mass tolerances of 10ppm were used, and the DotProduct score was applied for spectral comparisons. This model only combined the spectra across energies before training, so only one energy exists in the output. For the Negative model we have not yet trained a new model, so the original negative CFM model was used, as for the CFMOrig entry. DB_SCORE We checked for membership of each candidate in HMDB, ChEBI, a local database of plant derived compounds, FOODB and DRUGBANK and assigned +10 to the score for each database the compound was found to be a member of.
Participant: Kind Authors: Tobias Kind(1), Hiroshi Tsugawa(2) Affiliations: (1) UC Davis Genome Center - Metabolomics, Davis CA, USA (2) RIKEN Center for Sustainable Resource Science (CSRS), Wako, Japan ParticipantID: tkind Category: category3 Automatic methods: semi-automatic Abstract This is a submission for the http://www.casmi-contest.org/2016/ Category 3: Best Automatic Structural Identification - Full Information This third category uses MS/MS spectra of 208 unknown compounds (validation set). All MS/MS spectra were obtained on a Q Exactive Plus Orbitrap from Thermo Scientific, with <5 ppm mass accuracy and MS/MS resolution of 35,000 using electrospray ionization and stepped 20/35/50 HCD nominal collision energies. The [M+H]+ (positive) and [M-H]- ion masses were recorded. A reversed phase C18 column was used (2.6 uM, 2.1x50 mm with a 2.1x5 mm precolumn) with a gradient of (A/B): 95/5 at 0 min, 95/5 at 1 min, 0/100 at 13 min, 0/100 at 24 min (A = water, B = methanol, both with 0.1% formic acid) at a flow of 300 uL/min. In Category 3 any form of additional information can be used (retention time information, mass spectral libraries, patents, reference count. This allows to demonstrate whether/how much additional information can improve the results of the unknown annotation. Approach: Here we used a two-step procedure, first MS-Finder search and then MS/MS search for confirmation whenever possible. (1) Molecular formulas and structures were determined with the MS-Finder software [http://prime.psc.riken.jp/Metabolomics_Software/] by querying an internal structure databases and all CASMI provided structures. Result data was exported as text file and results were formatted for CASMI submission. First the molecular formulas were determined with Lewis and Senior check, 97% element ratio check and 20% isotopic abundance ratio and 5 ppm mass accuracy for MS1 and 20 ppm for MS2. Elements CHNOPSFClBrI were included (Si was excluded). The top 5 formula were regarded for structure queries. However no MS1 information was provided for this contest, only precursor mass and product ion information. Each formula was queried in an internal structure database and the CASMI compounds for the validation set category 3. An tree-depth of 2 and relative abundance cutoff of 1% as well as up to 100 possible structures were reported with MS-Finder. The score was calculated by the in-silico fragmenter that simulates the alpha-cleavage of linear chains up to three chemical bonds with consideration of the bond-dissociation energy. Multiple bonds (double-, triple-, or cycles) are modeled as penalized single bonds in which hydrogens are lost. The final score also includes mass accuracy, isotopic ratio, product ion assignment, neutral loss assignment and existence of the compound in the internal MS-Finder structure databases. (2) MS/MS search was used for further confirmation and the NIST MS Search GUI [http://chemdata.nist.gov/] together with major MS/MS databases such as NIST, MONA, ReSpect and MassBank was utilized. The precursor was set to 5 ppm and product ion search tolerance to 200 ppm. For some of the searches that gave no MS/MS results also simple similarity search without precursor info was used. Results that gave overall low hit scores were also cross-referenced with the STOFF-IDENT database of environmentally-relevant substances, to obtain information on potential hit candidates.