News
July 31st, 2014
The first articles appeared in the proceedings. Feb 3rd, 2014
The organising committee has declared the winners. Jan 15th, 2014
The automatic evaluation results are out. Jan 15th, 2014
Check the authors information on the proceedings page. Jan 1st, 2014
The solutions are online. Dec. 3, 2013
The submission deadline is approaching fast! Please email submissions to casmi2013@massbank.jp Oct. 1, 2013
Details about the CASMI 2013 Special Issue and dates are now available! Sept. 24, 2013
The rules and challenge data pages have been updated. Sept. 2, 2013
The CASMI 2013 Challenges have been officially released! August 29, 2013
The challenges for CASMI2013 will be released on Monday, September 2nd!
July 31st, 2014
The first articles appeared in the proceedings. Feb 3rd, 2014
The organising committee has declared the winners. Jan 15th, 2014
The automatic evaluation results are out. Jan 15th, 2014
Check the authors information on the proceedings page. Jan 1st, 2014
The solutions are online. Dec. 3, 2013
The submission deadline is approaching fast! Please email submissions to casmi2013@massbank.jp Oct. 1, 2013
Details about the CASMI 2013 Special Issue and dates are now available! Sept. 24, 2013
The rules and challenge data pages have been updated. Sept. 2, 2013
The CASMI 2013 Challenges have been officially released! August 29, 2013
The challenges for CASMI2013 will be released on Monday, September 2nd!
Results in Category 2
Summary of Rank by Challenge and Participant
For each challenge, the rank of the winner(s) is highlighted in bold. If the submission did not contain the correct candidate this is denoted as "-". If someone did not participate in a challenge, nothing is shown.an | ds | es | fa | lr | tm | |
---|---|---|---|---|---|---|
challenge1 | 1 | 1 | 9 | 12 | 1 | - |
challenge2 | 1 | 1 | 44 | - | 3 | - |
challenge3 | - | 21 | - | 17 | - | |
challenge4 | 1 | - | 238 | 18 | 78 | - |
challenge5 | 1 | 1 | 4 | 9 | 2 | - |
challenge6 | 1 | 1 | 1 | 1 | - | |
challenge7 | 1 | 1 | 17 | 23 | 1 | - |
challenge8 | 1 | 2 | 1 | 1 | 1 | - |
challenge9 | 1 | 1 | 1 | 2 | 1 | - |
challenge10 | 1 | 1 | 1 | 1 | 1 | - |
challenge11 | 2 | 6 | 21 | |||
challenge11(tautomer1) | 3 | 5 | 1 | |||
challenge11(tautomer2) | 1 | - | 22 | |||
challenge12 | 1 | 3 | 35 | |||
challenge13 | 12 | 24 | 42 | |||
challenge14 | 1 | 2 | 1 | 761 | 5 | |
challenge15 | 1 | 1 | - | |||
challenge16 | 1 | 1 | - | 100 |
Participant information and abstracts
ParticipantID: AN Category: category2 Authors: Andrew Newsome and Dejan Nikolic Affiliations: University of Illinois at Chicago, College of Pharmacy, Department of Medicinal Chemistry & Pharmacognosy, Chicago Mass Spectrometry Laboratory Automatic methods: No Abstract Structure candidates were determined on a case by case basis using manual methods. The manual methods used to arrive at structural candidates typically involved a combination of monoisotopic fragment ion and neutral loss formula analysis, database searching on molecular ion and fragment formulas and monoisotopic masses, literature consultation, deductive reasoning, and tacit knowledge and experience. The search databases most often employed were Chemspider, SciFinder Scholar, Reaxys, and Google Scholar. For ranking candidate structures, a subjective confidence scale from 0.60 to 1.00 was used. Structures were placed on the scale based upon how "confident" we felt about the proposed structure from our overall assessment of the fit of the candidates to the challenge data. The confidence scale ranking brackets are defined as follows: 1.00: Full confidence that the single candidate is the correct structure. 0.90 to 0.99: High confidence that candidate is the correct structure. 0.80 to 0.89: Good confidence that candidate is the correct structure. 0.70 to 0.79: Fair confidence that candidate is the correct structure. 0.60 to 0.69: Poor confidence that candidate is the correct structure. Where several possible structural isomers existed that matched the challenge data, isomers that were thought to be more likely were placed in a higher ranking bracket. In cases where many other possible structures existed that could potentially match the challenge data, we noted this in the respective abstract and lowered the confidence score for the submission accordingly. Structures placed in the same ranking bracket were regarded as equally likely. Structure candidates were submitted for all challenges except for challenge 13.
ParticipantID: FA Category: category1 and category2 Authors: Allen, Felicity and Greiner, Russ Affiliations: Department of Computing Science University of Alberta, Canada Automatic methods: yes Abstract A list of candidate structures was obtained by querying PubChem for all structures within 10ppm of the precursor mass (or with the given molecular formula if this was provided). For cases where the precursor mass was not provided, this value was deduced manually by considering the MS2 and MS1 data. Where further specific information was provided, the candidate lists were filtered using that information e.g. aromaticity, amide bonds. The candidate lists were then processed with the input spectra by the program cfm-id (http://sourceforge.net/projects/cfm-id/) to produce a ranked list of structures for Category 2. A Single-Energy CFM model was used, for which parameters were trained using non-peptide metabolite data from Metlin, as described in http://arxiv.org/abs/1312.0264 and stored in the supplementary data section of the above sourceforge project. Since the model expects a low, medium and high energy spectrum, whereas the challenge data (except 16) only has one spectrum, we repeated the provided spectrum for all three energy levels. For Challenge 16, we repeated the CE20 spectrum for low and medium and used the CE40 spectrum for high. All spectra were pre-processed - peaks below 1% intensity relative to the highest peaks were removed. For Category 1, the molecular formula was computed for each structure from Category 2 and kept in the same order. The list was then processed to remove duplicate entries, keeping only the highest ranked listing for each unique molecular formula. Submission is only made for positive ion mode, since cfm-id does not currently support negative mode.
ParticipantID: ES Category: category2 Authors: Emma Schymanski(1), Michael Gerlich(2), Christoph Ruttkies(2), Steffen Neumann(2) Affiliations: (1) Eawag: Swiss Federal Institute of Aquatic Science and Technology, Überlandstrasse 133, CH-8600 Dübendorf, Switzerland (2) IPB: Leibniz Institute of Plant Biochemistry, Department of Stress and Development Biology, Weinberg 3, DE-06120 Halle (Saale), Germany. Automatic pipeline: yes Abstract: Category 2 challenges were processed using MetFrag and MetFusion using compound database queries. Three databases (PubChem, ChemSpider and KEGG) were queried and the results were merged to create one candidate list, taking the maximum score for entries in more than one database. MetFusion also used massbank.jp to retrieve spectral information, with default parameters. Information from the Analytical Methods files and Category 1 results (ES) were used to adjust input parameters for the automatic pipeline. The formula was used for candidate retrieval where this was given or clear from Category 1, otherwise the exact mass was used. We enabled the element filter CHNOPS unless we obtained high scoring non-CHNOPS candidates without it. The exact mass database retrieval and fragmentation parameters were adjusted according to the expected or quoted instrument accuracy. Manual checking of the automatic calculations were performed to detect any anomolies. Where the MetFusion scores were poor (few or no matching spectra from MassBank), MetFrag results were submitted.
ParticipantID: DS Category: 2 Author: Daniel L. Sweeney Affiliation: MathSpec, Inc., Arlington Heights, IL Automated Process: yes Spectral Libraries: No Abstract: Used Rational Numbers Search software and searched the mass spectral peak lists against a database of approximately 200000 molecules that had been partitioned for rapid mass spectral searching. The InChi structures were copied from the PubChem entry for the corresponding compound. In challenges where the precursor ion was not present in the MS/MS data file, the precursor ion was copied from the MS data file. All ions greater in mass than the precursor ion, if present in the MS/MS data file, were removed prior to the analysis. The isotope data was not used. Challenges 7 and 8 were done manually. Challenge 10 was done manually with the aid of an Excel Add-In. Attempts were made to identify all sixteen compounds, but no possibilities were found for challenges 3 or 13.
ParticipantID: TM Category: category2 Authors: Miyazaki, Tsubasa and Hisayuki, Horai Affiliations: Ibaraki National College of Technology, Dept. of Electronic and Computer Engineering, Hitachinaka, Japan Automatic methods: no Abstract CH1: 1. We mapped spectra in MassBank and the challenge spectrum data SP to a vector space and found a compound C whose spectrum SP1 is the most similar to SP. 2. Because the distance between spectra of SP and SP1 is relatively small and the highest m/z values in SP1 and SP is equivalent, C was decided to be the result of the challenge. CH5: 1. We mapped spectra in MassBank and the challenge spectrum data SP to a vector space and found a compound C whose spectrum SP1 is the most similar to SP. 2. Because the distance between spectra of SP and SP1 is relatively small and the highest m/z value in SP is lesser than the one of SP1, the result of the challenge is decided to be a substructure of C. CH3, CH6: 1. We mapped spectra in MassBank and the challenge spectrum data SP to a vector space and found a compound C whose spectrum SP1 is the most similar to SP. 2. Because the distance between spectra of SP and SP1 is relatively small and the highest m/z value in SP is greater than the one of SP1, C is decided to be a substructure of the result of the challenge. CH2, CH4, CH7, CH8, CH9, CH10: 1. We mapped spectra in MassBank and the challenge spectrum data SP to a vector space and found a compound whose spectrum SP1 is the most similar to SP. 2. Because the distance between spectra of SP and SP1 is quite large, we choses important peaks P and search spectra which have peaks whose m/z values are equal to Ps'. The choice of PP is based on the intuition the analyst. 3. The score indicates the confidence of the analyst.
ParticipantID: LR Category: category2 Authors: Ridder, Lars(1) and Hooft, Justin J.J. van der(2) Affiliations: (1) Wageningen University, Laboratory of Biochemistry, Wageningen, The Netherlands (2) University of Glasgow, College of Medical, Veterinary, and Life Sciences, United Kingdom Automatic methods: yes Abstract The challenge peak lists were converted to MAGMa input files, and processed with MAGMa using candidate molecules from PubChem (Ridder et al. 2012, Ridder et al. 2013) This method is available here: http://www.emetabolomics.org/magma. It does not make use of searches in spectral libraries. By default MAGMa is restricted to candidate molecules from PubChem <1200 Da and consisting of the elements C,H,N,O,P and S. For challenges 7, 8 and 9 candidates molecules were retrieved from PubChem outside the default restrictions. Candidates for 7 and 8 were >1200 Da, and challenge 9 was recognised to contain chloro atoms, based on the isotope pattern, so candidate molecules with halogens were included. The reported score represents the "refined ranking" as described in Ridder et al. (2013). For challenges 1, 2 and 14 de large numbers of PubChem candidates obtained initially were reduced based on a threshold of 5 on the number of related PubChem references. No submissions are made for challenges 11,12,15 and 16 for which none of the retrieved pubchem candidates (based on default restrictions) provided a satisfactory match in MAGMa between fragment ions and in silico substructures. References: Ridder, L.; van der Hooft, J. J. J.; Verhoeven, S.; de Vos, R. C. H.; van Schaik, R.; Vervoort (2012) J. Rapid Commun. Mass Spectrom. 26, 2461-2471. Ridder, L.; van der Hooft, J. J. J.; Verhoeven, S.; de Vos, R. C. H.; Bino, R. J.; Vervoort (2013) J. Anal. Chem. 85, 6033-6040.
Details per Challenge and Participant. See legend at bottom for more details
The details table is also available as CSV download. The individual submissions are also available for download.participant | challenge | rank | tc | bc | wc | ec | rrp | p | wbc | wwc | wec | wrrp |
---|---|---|---|---|---|---|---|---|---|---|---|---|
an | challenge1 | 1 | 2 | 0 | 1 | 1 | 1.00 | 0.60 | 0.00 | 0.40 | 0.00 | 1.00 |
ds | challenge1 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
es | challenge1 | 9 | 5631 | 8 | 5622 | 1 | 1.00 | 0.00 | 0.00 | 1.00 | 0.00 | 1.00 |
fa | challenge1 | 12 | 6767 | 9 | 6755 | 3 | 1.00 | 0.01 | 0.06 | 0.92 | 0.01 | 0.92 |
lr | challenge1 | 1 | 1084 | 0 | 1083 | 1 | 1.00 | 0.00 | 0.00 | 1.00 | 0.00 | 1.00 |
tm | challenge1 | - | 1 | - | - | - | - | - | - | - | - | - |
an | challenge2 | 1 | 2 | 0 | 1 | 1 | 1.00 | 0.60 | 0.00 | 0.40 | 0.00 | 1.00 |
ds | challenge2 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
es | challenge2 | 44 | 12702 | 43 | 12658 | 1 | 1.00 | 0.00 | 0.00 | 1.00 | 0.00 | 1.00 |
fa | challenge2 | - | 131 | - | - | - | - | - | - | - | - | - |
lr | challenge2 | 3 | 631 | 2 | 628 | 1 | 1.00 | 0.00 | 0.01 | 0.99 | 0.00 | 0.99 |
tm | challenge2 | - | 1 | - | - | - | - | - | - | - | - | - |
an | challenge3 | - | 1 | - | - | - | - | - | - | - | - | - |
es | challenge3 | 21 | 335 | 0 | 314 | 21 | 0.97 | 0.00 | 0.00 | 0.90 | 0.10 | 0.90 |
fa | challenge3 | - | 18 | - | - | - | - | - | - | - | - | - |
lr | challenge3 | 17 | 370 | 2 | 353 | 15 | 0.98 | 0.01 | 0.01 | 0.91 | 0.07 | 0.92 |
tm | challenge3 | - | 1 | - | - | - | - | - | - | - | - | - |
an | challenge4 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
ds | challenge4 | - | 10 | - | - | - | - | - | - | - | - | - |
es | challenge4 | 238 | 721 | 236 | 483 | 2 | 0.67 | 0.00 | 0.37 | 0.63 | 0.00 | 0.63 |
fa | challenge4 | 18 | 1622 | 16 | 1604 | 2 | 0.99 | 0.00 | 0.04 | 0.95 | 0.00 | 0.96 |
lr | challenge4 | 78 | 825 | 77 | 747 | 1 | 0.91 | 0.00 | 0.18 | 0.82 | 0.00 | 0.82 |
tm | challenge4 | - | 1 | - | - | - | - | - | - | - | - | - |
an | challenge5 | 1 | 3 | 0 | 2 | 1 | 1.00 | 0.42 | 0.00 | 0.58 | 0.00 | 1.00 |
ds | challenge5 | 1 | 2 | 0 | 1 | 1 | 1.00 | 0.60 | 0.00 | 0.40 | 0.00 | 1.00 |
es | challenge5 | 4 | 366 | 3 | 362 | 1 | 0.99 | 0.00 | 0.01 | 0.98 | 0.00 | 0.99 |
fa | challenge5 | 9 | 2725 | 8 | 2716 | 1 | 1.00 | 0.00 | 0.03 | 0.97 | 0.00 | 0.97 |
lr | challenge5 | 2 | 350 | 1 | 348 | 1 | 1.00 | 0.01 | 0.01 | 0.99 | 0.00 | 0.99 |
tm | challenge5 | - | 1 | - | - | - | - | - | - | - | - | - |
an | challenge6 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
ds | challenge6 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
es | challenge6 | 1 | 6 | 0 | 5 | 1 | 1.00 | 0.72 | 0.00 | 0.28 | 0.00 | 1.00 |
lr | challenge6 | 1 | 2 | 0 | 1 | 1 | 1.00 | 0.67 | 0.00 | 0.33 | 0.00 | 1.00 |
tm | challenge6 | - | 1 | - | - | - | - | - | - | - | - | - |
an | challenge7 | 1 | 7 | 0 | 6 | 1 | 1.00 | 0.15 | 0.00 | 0.85 | 0.00 | 1.00 |
ds | challenge7 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
es | challenge7 | 17 | 17 | 0 | 0 | 17 | 0.50 | 0.06 | 0.00 | 0.00 | 0.94 | 0.06 |
fa | challenge7 | 23 | 24 | 14 | 1 | 9 | 0.22 | 0.04 | 0.62 | 0.01 | 0.33 | 0.05 |
lr | challenge7 | 1 | 17 | 0 | 16 | 1 | 1.00 | 0.11 | 0.00 | 0.89 | 0.00 | 1.00 |
tm | challenge7 | - | 1 | - | - | - | - | - | - | - | - | - |
an | challenge8 | 1 | 3 | 0 | 2 | 1 | 1.00 | 0.42 | 0.00 | 0.58 | 0.00 | 1.00 |
ds | challenge8 | 2 | 2 | 0 | 0 | 2 | 0.50 | 0.50 | 0.00 | 0.00 | 0.50 | 0.50 |
es | challenge8 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
fa | challenge8 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
lr | challenge8 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
tm | challenge8 | - | 2 | - | - | - | - | - | - | - | - | - |
an | challenge9 | 1 | 6 | 0 | 5 | 1 | 1.00 | 0.23 | 0.00 | 0.77 | 0.00 | 1.00 |
ds | challenge9 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
es | challenge9 | 1 | 4 | 0 | 3 | 1 | 1.00 | 0.43 | 0.00 | 0.57 | 0.00 | 1.00 |
fa | challenge9 | 2 | 150 | 1 | 148 | 1 | 0.99 | 0.02 | 0.03 | 0.95 | 0.00 | 0.97 |
lr | challenge9 | 1 | 113 | 0 | 112 | 1 | 1.00 | 0.02 | 0.00 | 0.98 | 0.00 | 1.00 |
tm | challenge9 | - | 1 | - | - | - | - | - | - | - | - | - |
an | challenge10 | 1 | 2 | 0 | 1 | 1 | 1.00 | 0.56 | 0.00 | 0.44 | 0.00 | 1.00 |
ds | challenge10 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
es | challenge10 | 1 | 9 | 0 | 8 | 1 | 1.00 | 0.13 | 0.00 | 0.87 | 0.00 | 1.00 |
fa | challenge10 | 1 | 20 | 0 | 19 | 1 | 1.00 | 0.33 | 0.00 | 0.67 | 0.00 | 1.00 |
lr | challenge10 | 1 | 20 | 0 | 19 | 1 | 1.00 | 0.10 | 0.00 | 0.90 | 0.00 | 1.00 |
tm | challenge10 | - | 3 | - | - | - | - | - | - | - | - | - |
an | challenge11 | 2 | 3 | 1 | 1 | 1 | 0.50 | 0.36 | 0.36 | 0.28 | 0.00 | 0.64 |
ds | challenge11 | 6 | 17 | 5 | 11 | 1 | 0.69 | 0.06 | 0.31 | 0.63 | 0.00 | 0.69 |
es | challenge11 | 21 | 2392 | 20 | 2371 | 1 | 0.99 | 0.00 | 0.01 | 0.98 | 0.00 | 0.99 |
an | challenge11(tautomer1) | 3 | 3 | 2 | 0 | 1 | 0.00 | 0.28 | 0.72 | 0.00 | 0.00 | 0.28 |
ds | challenge11(tautomer1) | 5 | 17 | 4 | 12 | 1 | 0.75 | 0.06 | 0.25 | 0.69 | 0.00 | 0.75 |
es | challenge11(tautomer1) | 1 | 2392 | 0 | 2391 | 1 | 1.00 | 0.00 | 0.00 | 1.00 | 0.00 | 1.00 |
an | challenge11(tautomer2) | 1 | 3 | 0 | 2 | 1 | 1.00 | 0.36 | 0.00 | 0.64 | 0.00 | 1.00 |
ds | challenge11(tautomer2) | - | 17 | - | - | - | - | - | - | - | - | - |
es | challenge11(tautomer2) | 22 | 2392 | 21 | 2370 | 1 | 0.99 | 0.00 | 0.02 | 0.98 | 0.00 | 0.98 |
an | challenge12 | 1 | 3 | 0 | 2 | 1 | 1.00 | 0.38 | 0.00 | 0.62 | 0.00 | 1.00 |
ds | challenge12 | 3 | 21 | 2 | 18 | 1 | 0.90 | 0.05 | 0.10 | 0.85 | 0.00 | 0.90 |
es | challenge12 | 35 | 902 | 34 | 867 | 1 | 0.96 | 0.00 | 0.06 | 0.93 | 0.00 | 0.94 |
es | challenge13 | 12 | 227 | 11 | 215 | 1 | 0.95 | 0.01 | 0.07 | 0.93 | 0.00 | 0.93 |
fa | challenge13 | 24 | 284 | 18 | 260 | 6 | 0.93 | 0.01 | 0.17 | 0.80 | 0.03 | 0.80 |
lr | challenge13 | 42 | 206 | 41 | 164 | 1 | 0.80 | 0.01 | 0.36 | 0.63 | 0.00 | 0.64 |
an | challenge14 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
ds | challenge14 | 2 | 5 | 1 | 3 | 1 | 0.75 | 0.21 | 0.21 | 0.58 | 0.00 | 0.79 |
es | challenge14 | 1 | 8219 | 0 | 8218 | 1 | 1.00 | 0.00 | 0.00 | 1.00 | 0.00 | 1.00 |
fa | challenge14 | 761 | 9708 | 732 | 8947 | 29 | 0.92 | 0.00 | 0.16 | 0.83 | 0.01 | 0.83 |
lr | challenge14 | 5 | 1583 | 4 | 1578 | 1 | 1.00 | 0.00 | 0.01 | 0.99 | 0.00 | 0.99 |
an | challenge15 | 1 | 3 | 0 | 2 | 1 | 1.00 | 0.43 | 0.00 | 0.57 | 0.00 | 1.00 |
ds | challenge15 | 1 | 4 | 0 | 3 | 1 | 1.00 | 0.33 | 0.00 | 0.67 | 0.00 | 1.00 |
es | challenge15 | - | 6 | - | - | - | - | - | - | - | - | - |
an | challenge16 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
ds | challenge16 | 1 | 4 | 0 | 3 | 1 | 1.00 | 0.28 | 0.00 | 0.72 | 0.00 | 1.00 |
es | challenge16 | - | 3976 | - | - | - | - | - | - | - | - | - |
fa | challenge16 | 100 | 10637 | 97 | 10537 | 3 | 0.99 | 0.00 | 0.02 | 0.98 | 0.00 | 0.98 |
Table legend:
- rank
- Absolute rank of correct solution
- tc
- Total number of candidates
- bc
- Number of candidates with a score better than correct solution
- wc
- Number of candidates with a score worse than correct solution
- ec
- Number of candidates with same score as the correct solution
- rrp
- Relative ranking position (1.0 is good, 0.0 is not)
- p
- Score of correct solution
- wbc
- Sum of scores better than correct solution
- wwc
- Sum of scores worse than correct solution
- wec
- Sum of scores equal to correct solution
- wrrp
- RRP weighted by the scores (1 is good)