News
Oct. 1, 2013
Details about the CASMI 2013 Special Issue and dates are now available! Sept. 24, 2013
The rules and challenge data pages have been updated. Sept. 2, 2013
The CASMI 2013 Challenges have been officially released! August 29, 2013
The challenges for CASMI2013 will be released on Monday, September 2nd! August 29, 2013
The CASMI 2012 poster will be presented in Langenau in November 2013
Oct. 1, 2013
Details about the CASMI 2013 Special Issue and dates are now available! Sept. 24, 2013
The rules and challenge data pages have been updated. Sept. 2, 2013
The CASMI 2013 Challenges have been officially released! August 29, 2013
The challenges for CASMI2013 will be released on Monday, September 2nd! August 29, 2013
The CASMI 2012 poster will be presented in Langenau in November 2013
Results in Category 1
Summary of Rank by Challenge and Participant
For each challenge, the rank of the winner(s) is highlighted in bold. If the submission did not contain the correct candidate this is denoted as "-". If someone did not participate in a challenge, nothing is shown.birmingham | hshen | kduehrkop | rdisop | schymane | |
---|---|---|---|---|---|
challenge1 | 1 | 4 | 3 | 18 | 23 |
challenge2 | - | 1 | - | - | - |
challenge3 | 1 | - | 8 | - | 1 |
challenge4 | 1 | 3 | - | 5 | - |
challenge5 | 1 | 4 | 2 | 1 | 8 |
challenge6 | 2 | 4 | - | 9 | - |
challenge10 | 1 | 1 | 1 | 1 | 1 |
challenge11 | - | - | - | 4 | |
challenge12 | - | - | - | 4 | |
challenge13 | - | - | 1 | 1 | 1 |
challenge14 | 1 | 1 | 1 | 1 | 1 |
challenge15 | 1 | 5 | 1 | 2 | 1 |
challenge16 | - | - | 4 | ||
challenge17 | 1 | - | 1 | 1 | 1 |
Participant information and abstracts
ParticipantID: Dunn and Birmingham Category: Category1 and category 2 Authors:Members of Dunn and Viant groups at University of Birmingham, UK Affiliations:University of Birmingham, UK Automatic pipeline:No Spectral libraries:No Abstract The group automatically applied workflow 2 of PUTMEDID-LCMS to calculate one or multiple molecular formula that matched the accurate mass of the neutral metabolite. The group then automatically or manually searched MMD, KEGG and ChemSpider in this order to define potential metabolite structures, which were manually filtered in relation to isotopes present or absent and 12C/13C ratios for instruments where an accurate ratio can be calculated. The structures were applied in MetFrag to calculate matches between in-silico fragmentation and experimental data; these data were manually assessed to remove biologically unreasonable metabolites. We processed only the LC-MS challenges as follows: 1,2,3,4,5,6,10,13,14,15,17. The challenge data were converted to molecular formula(s), searched against MMD, KEGG and Chemspider and comparison of experimental and in-silico fragmentation data were compared in MetFrag v0.9.
ParticipantID: hshen Category: category 1 and 2 Authors: Huibin, Shen(1) and Nicola, Zamboni(2) and Markus, Heinonen(3) and Juho, Rousu(1) Affiliations: (1) Helsinki Institute for Information Technology; Department of Information and Computer Science, Aalto University, Finland (2) Institute of Molecular Systems Biology, ETH Zurich, Switzerland. (3) IBISC, Université d’Evry-Val d’Essonne, France Automatic pipeline: yes Spectral libraries: yes (MassBank) Abstract We processed only the LC-MS challenges. We predict the molecular fingerprints of the challenge data using FingerID and use them to search the Kegg compound database.
ParticipantID: kduehrkop Category: category1 Authors: Kai Dührkop, Kerstin Scheubert, Sebastian Böcker Affiliations: Chair for Bioinformatics, Friedrich-Schiller-University, Jena, Germany Automatic pipeline: yes Spectral libraries: no Abstract We processed only the LC-MS challenges. The spectral data was analyzed by the sirius command line tool. We choosed the parameters of our tool according to the challenge description. We combined the scores of the isotopic patterns and the fragmentation trees computed from ms/ms spectra. The pubchem molecular formula search was used to assign adducts to the ions. If no formula was found in pubchem, we assumed a protonation. For intrinsic charged formulas, we added a protonated variant with a slightly reduced score.
ParticipantID: rdisop Category: category1 Authors: Neumann, Steffen Affiliations: Leibniz Institute for Plant Biochemistry, Dept. of Stress- and Developmental Biology, Halle, Germany Automatic pipeline: yes Spectral libraries: no Abstract: I create a small script extracting the isotope pattern from the MS peaklist files, and processed them with the Bioconductor package rdisop (which in turn uses the disop library developed by the Boecker group). No efforts were made to cleverly detect the [M+H]+ or adducts.
ParticipantID: schymane Category: category1 Authors: Schymanski, Emma(1) and Meringer, Markus (2) Affiliations: (1) Eawag: Swiss Federal Institute of Aquatic Science and Technology, Überlandstrasse 133, CH-8600 Dübendorf, Switzerland (2) DLR: German Aerospace Centre, Münchnerstrasse 20, D-82234 Oberpfaffenhofen-Wessling, Germany Automatic pipeline: yes Spectral libraries: no Abstract: We processed Category 1 challenges with MOLGEN-MS/MS. Where no evidence of halogens was present, we chose the elements CHNOPS. Default parameters were MS accuracy 5 ppm, MS/MS accuracy 10 ppm, using the existence filter and allowing "OEI" ions to explain MS/MS peaks. The option to use positive or negative ionisation was taken from the information on the web, using +H for positive and -H for negative, except for the tricky cases. We submitted the formulas generated with these settings, using the combined match value as the score. Where multiple MS/MS files were available we combined them to get the maximum number of peaks, using the peak of highest intensity where the same peak occurred more than once. As the score was not weighted by intensity, this should make no difference to the outcome. Deviations from these default settings were: Challenge 11: additional setting m=232.088 to set the M+ ion mass and ion mode "-e" corresponding to an M+ ion. Challenge 16: additional setting m=359.1481 to set the M+ ion mass and ion mode "-e" corresponding to the M+ ion. Could also have added a H+ to the mass and run mode +H. MSMS match values taken as score, as MS values were 0.
Details per Challenge and Participant. See legend at bottom for more details
The table is also available as CSV downloadparticipant | category | challenge | rank | tc | bc | wc | ec | rrp | p | wbc | wwc | wec | wrrp |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
birmingham | category1 | challenge1 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
hshen | category1 | challenge1 | 4 | 5 | 3 | 1 | 1 | 0.25 | 0.08 | 0.85 | 0.06 | 0.00 | 0.15 |
kduehrkop | category1 | challenge1 | 3 | 136 | 2 | 133 | 1 | 0.99 | 0.00 | 1.00 | 0.00 | 0.00 | 0.00 |
rdisop | category1 | challenge1 | 18 | 1466 | 17 | 1448 | 1 | 0.99 | 0.00 | 1.00 | 0.00 | 0.00 | 0.00 |
schymane | category1 | challenge1 | 23 | 54 | 22 | 31 | 1 | 0.58 | 0.02 | 0.61 | 0.36 | 0.00 | 0.39 |
birmingham | category1 | challenge2 | - | 3 | - | - | - | - | - | - | - | - | - |
hshen | category1 | challenge2 | 1 | 4 | 0 | 3 | 1 | 1.00 | 0.40 | 0.00 | 0.60 | 0.00 | 1.00 |
kduehrkop | category1 | challenge2 | - | 926 | - | - | - | - | - | - | - | - | - |
rdisop | category1 | challenge2 | - | 4430 | - | - | - | - | - | - | - | - | - |
schymane | category1 | challenge2 | - | 226 | - | - | - | - | - | - | - | - | - |
birmingham | category1 | challenge3 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
hshen | category1 | challenge3 | - | 6 | - | - | - | - | - | - | - | - | - |
kduehrkop | category1 | challenge3 | 8 | 425 | 7 | 417 | 1 | 0.98 | 0.00 | 1.00 | 0.00 | 0.00 | 0.00 |
rdisop | category1 | challenge3 | - | 1180 | - | - | - | - | - | - | - | - | - |
schymane | category1 | challenge3 | 1 | 90 | 0 | 89 | 1 | 1.00 | 0.01 | 0.00 | 0.99 | 0.00 | 1.00 |
birmingham | category1 | challenge4 | 1 | 2 | 0 | 1 | 1 | 1.00 | 0.67 | 0.00 | 0.33 | 0.00 | 1.00 |
hshen | category1 | challenge4 | 3 | 12 | 2 | 9 | 1 | 0.82 | 0.12 | 0.28 | 0.60 | 0.00 | 0.72 |
kduehrkop | category1 | challenge4 | - | 39 | - | - | - | - | - | - | - | - | - |
rdisop | category1 | challenge4 | 5 | 217 | 4 | 212 | 1 | 0.98 | 0.00 | 1.00 | 0.00 | 0.00 | 0.00 |
schymane | category1 | challenge4 | - | 13 | - | - | - | - | - | - | - | - | - |
birmingham | category1 | challenge5 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
hshen | category1 | challenge5 | 4 | 11 | 3 | 7 | 1 | 0.70 | 0.14 | 0.47 | 0.39 | 0.00 | 0.53 |
kduehrkop | category1 | challenge5 | 2 | 30 | 1 | 28 | 1 | 0.97 | 0.00 | 1.00 | 0.00 | 0.00 | 0.00 |
rdisop | category1 | challenge5 | 1 | 226 | 0 | 225 | 1 | 1.00 | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
schymane | category1 | challenge5 | 8 | 10 | 7 | 2 | 1 | 0.22 | 0.08 | 0.82 | 0.10 | 0.00 | 0.18 |
birmingham | category1 | challenge6 | 2 | 2 | 1 | 0 | 1 | 0.00 | 0.33 | 0.67 | 0.00 | 0.00 | 0.33 |
hshen | category1 | challenge6 | 4 | 15 | 3 | 11 | 1 | 0.79 | 0.10 | 0.32 | 0.58 | 0.00 | 0.68 |
kduehrkop | category1 | challenge6 | - | 78 | - | - | - | - | - | - | - | - | - |
rdisop | category1 | challenge6 | 9 | 498 | 8 | 489 | 1 | 0.98 | 0.00 | 1.00 | 0.00 | 0.00 | 0.00 |
schymane | category1 | challenge6 | - | 25 | - | - | - | - | - | - | - | - | - |
birmingham | category1 | challenge10 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
hshen | category1 | challenge10 | 1 | 11 | 0 | 10 | 1 | 1.00 | 0.18 | 0.00 | 0.82 | 0.00 | 1.00 |
kduehrkop | category1 | challenge10 | 1 | 7 | 0 | 6 | 1 | 1.00 | 0.93 | 0.00 | 0.07 | 0.00 | 1.00 |
rdisop | category1 | challenge10 | 1 | 44 | 0 | 43 | 1 | 1.00 | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
schymane | category1 | challenge10 | 1 | 3 | 0 | 2 | 1 | 1.00 | 0.48 | 0.00 | 0.52 | 0.00 | 1.00 |
hshen | category1 | challenge11 | - | 15 | - | - | - | - | - | - | - | - | - |
kduehrkop | category1 | challenge11 | - | 15 | - | - | - | - | - | - | - | - | - |
rdisop | category1 | challenge11 | - | 29 | - | - | - | - | - | - | - | - | - |
schymane | category1 | challenge11 | 4 | 6 | 0 | 2 | 4 | 0.70 | 0.20 | 0.00 | 0.20 | 0.60 | 0.40 |
hshen | category1 | challenge12 | - | 21 | - | - | - | - | - | - | - | - | - |
kduehrkop | category1 | challenge12 | - | 68 | - | - | - | - | - | - | - | - | - |
rdisop | category1 | challenge12 | - | 8 | - | - | - | - | - | - | - | - | - |
schymane | category1 | challenge12 | 4 | 18 | 0 | 14 | 4 | 0.91 | 0.12 | 0.00 | 0.51 | 0.37 | 0.63 |
birmingham | category1 | challenge13 | - | 1 | - | - | - | - | - | - | - | - | - |
hshen | category1 | challenge13 | - | 15 | - | - | - | - | - | - | - | - | - |
kduehrkop | category1 | challenge13 | 1 | 20 | 0 | 19 | 1 | 1.00 | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
rdisop | category1 | challenge13 | 1 | 141 | 0 | 140 | 1 | 1.00 | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
schymane | category1 | challenge13 | 1 | 10 | 0 | 9 | 1 | 1.00 | 0.14 | 0.00 | 0.86 | 0.00 | 1.00 |
birmingham | category1 | challenge14 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
hshen | category1 | challenge14 | 1 | 12 | 0 | 11 | 1 | 1.00 | 0.17 | 0.00 | 0.83 | 0.00 | 1.00 |
kduehrkop | category1 | challenge14 | 1 | 4 | 0 | 3 | 1 | 1.00 | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
rdisop | category1 | challenge14 | 1 | 12 | 0 | 11 | 1 | 1.00 | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
schymane | category1 | challenge14 | 1 | 2 | 0 | 1 | 1 | 1.00 | 0.83 | 0.00 | 0.17 | 0.00 | 1.00 |
birmingham | category1 | challenge15 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
hshen | category1 | challenge15 | 5 | 6 | 4 | 1 | 1 | 0.20 | 0.15 | 0.71 | 0.14 | 0.00 | 0.29 |
kduehrkop | category1 | challenge15 | 1 | 4 | 0 | 3 | 1 | 1.00 | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
rdisop | category1 | challenge15 | 2 | 27 | 1 | 25 | 1 | 0.96 | 0.05 | 0.92 | 0.03 | 0.00 | 0.08 |
schymane | category1 | challenge15 | 1 | 2 | 0 | 1 | 1 | 1.00 | 0.50 | 0.00 | 0.50 | 0.00 | 1.00 |
hshen | category1 | challenge16 | - | 12 | - | - | - | - | - | - | - | - | - |
kduehrkop | category1 | challenge16 | - | 46 | - | - | - | - | - | - | - | - | - |
schymane | category1 | challenge16 | 4 | 20 | 1 | 16 | 3 | 0.89 | 0.08 | 0.09 | 0.67 | 0.16 | 0.75 |
birmingham | category1 | challenge17 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
hshen | category1 | challenge17 | - | 11 | - | - | - | - | - | - | - | - | - |
kduehrkop | category1 | challenge17 | 1 | 5 | 0 | 4 | 1 | 1.00 | 0.89 | 0.00 | 0.11 | 0.00 | 1.00 |
rdisop | category1 | challenge17 | 1 | 30 | 0 | 29 | 1 | 1.00 | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
schymane | category1 | challenge17 | 1 | 3 | 0 | 2 | 1 | 1.00 | 0.54 | 0.00 | 0.46 | 0.00 | 1.00 |
Table legend:
- rank
- Absolute rank of correct solution
- tc
- Total number of candidates
- bc
- Number of candidates with a score better than correct solution
- wc
- Number of candidates with a score worse than correct solution
- ec
- Number of candidates with same score as the correct solution
- rrp
- Relative ranking position (1.0 is good, 0.0 is not)
- p
- Score of correct solution
- wbc
- Sum of scores better than correct solution
- wwc
- Sum of scores worse than correct solution
- wec
- Sum of scores equal to correct solution
- wrrp
- RRP weighted by the scores (1 is good)