News
Oct. 1, 2013
Details about the CASMI 2013 Special Issue and dates are now available! Sept. 24, 2013
The rules and challenge data pages have been updated. Sept. 2, 2013
The CASMI 2013 Challenges have been officially released! August 29, 2013
The challenges for CASMI2013 will be released on Monday, September 2nd! August 29, 2013
The CASMI 2012 poster will be presented in Langenau in November 2013
Oct. 1, 2013
Details about the CASMI 2013 Special Issue and dates are now available! Sept. 24, 2013
The rules and challenge data pages have been updated. Sept. 2, 2013
The CASMI 2013 Challenges have been officially released! August 29, 2013
The challenges for CASMI2013 will be released on Monday, September 2nd! August 29, 2013
The CASMI 2012 poster will be presented in Langenau in November 2013
Results in Category 2
Summary of Rank by Challenge and Participant
For each challenge, the rank of the winner(s) is highlighted in bold. If the submission did not contain the correct candidate this is denoted as "-". If someone did not participate in a challenge, nothing is shown.| birmingham | cruttkies | hshen | metfusion | oberacher | schymane | |
|---|---|---|---|---|---|---|
| challenge1 | 1 | 45 | 5 | 1 | 1 | |
| challenge2 | - | - | 1 | 30 | ||
| challenge3 | - | 21 | - | - | ||
| challenge4 | - | - | - | 195 | ||
| challenge5 | 4 | 275 | 5 | 386 | ||
| challenge6 | - | - | 11 | 25 | ||
| challenge10 | - | 302 | - | 307 | - | 63 |
| challenge11 | - | - | - | 3 | ||
| challenge12 | - | - | - | |||
| challenge13 | - | 5 | - | 64 | 1 | 3 |
| challenge14 | 12 | 39 | - | 1 | - | 22 |
| challenge15 | - | 316 | - | 1 | 1 | 26 |
| challenge16 | 2585 | - | 1562 | |||
| challenge17 | - | 21 | - | 40 | - |
Participant information and abstracts
ParticipantID: Dunn and Birmingham Category: Category1 and category 2 Authors:Members of Dunn and Viant groups at University of Birmingham, UK Affiliations:University of Birmingham, UK Automatic pipeline:No Spectral libraries:No Abstract The group automatically applied workflow 2 of PUTMEDID-LCMS to calculate one or multiple molecular formula that matched the accurate mass of the neutral metabolite. The group then automatically or manually searched MMD, KEGG and ChemSpider in this order to define potential metabolite structures, which were manually filtered in relation to isotopes present or absent and 12C/13C ratios for instruments where an accurate ratio can be calculated. The structures were applied in MetFrag to calculate matches between in-silico fragmentation and experimental data; these data were manually assessed to remove biologically unreasonable metabolites. We processed only the LC-MS challenges as follows: 1,2,3,4,5,6,10,13,14,15,17. The challenge data were converted to molecular formula(s), searched against MMD, KEGG and Chemspider and comparison of experimental and in-silico fragmentation data were compared in MetFrag v0.9.
ParticipantID: cruttkie
Category: category2
Authors: Christoph ruttkies
Affiliations: (1) IPB Halle, Dept. of Stress and
Developmental Biology, Halle, Germany
Automatic pipeline: yes
Spectral libraries: no
Abstract
The challenge data was converted to MetFrag query files, and processed with
MetFrag v0.9. Afterwards an Inchi-Key filter was applied which removed all
duplicates out of the candidate lists. To receive the final score for
challenge 1-6 the metabolite likeness score (Peironcely JE et al. 2011) was
added to the MetFrag score. The resulting candidate SDF was converted to TXT
as submission.
ParticipantID: hshen
Category: category 1 and 2
Authors: Huibin, Shen(1) and Nicola, Zamboni(2) and Markus,
Heinonen(3) and Juho, Rousu(1)
Affiliations: (1) Helsinki Institute for Information Technology;
Department of Information and Computer Science,
Aalto University, Finland (2) Institute of
Molecular Systems Biology, ETH Zurich, Switzerland.
(3) IBISC, Université d’Evry-Val d’Essonne, France
Automatic pipeline: yes
Spectral libraries: yes (MassBank)
Abstract
We processed only the LC-MS challenges. We predict the molecular
fingerprints of the challenge data using FingerID and use them to
search the Kegg compound database.
ParticipantID: mgerlich Category: category2 Authors: Michael Gerlich Affiliations: IPB Halle, Dept. of Stress and Developmental Biology, Halle, Germany Automatic pipeline: yes Spectral libraries: yes Abstract All category2 challenges were converted into MetFusion specific query files, containing the exact mass of the precursor ion as well as a merged peaklist from the spectra with varying collision energies (where applicable). The use of merged spectra is recommended by the MassBank spectral library, which was used as reference library for spectra. The instrument filter was set to use only ESI instruments, thus retrieving no spectra from EI ionization or other ionization types. The resulting candidate lists were treated with an InChIKey-based filter which removes duplicate structures based on connectivity information. The newly ranked candidates were stored in an SDF file which was converted to a text file containing the corresponding SMILES and scoring information as submission. The challenge data was processed with the command line version of MetFusion.
ParticipantID: oberacher
Category: category2
Authors: Oberacher, Herbert
Affiliations: Institute of Legal Medicine and Core Facility
Metabolomics, Innsbruck Medical University
Automatic pipeline: yes
Spectral libraries: yes
Abstract
We processed only the LC-MS challenges. The challenge data was used as
input for automated library search in 4 libraries.
(a) MassBank (Date: 7.1.2013, Spectrum Search, Tolerance m/z: 0.3
units, Cutoff Threshold: 5, MS Type: All, Positive and Negative)
(b) Metlin (Date: 9.1.2013, MSMS Spectrum Search, Tolerance MSMS
0.01-0.1 Da, Tolerance precursor: 100 ppm, Positive and Negative)
(c) NIST (NIST 2012 - May 2012, NIST MS Search Program 2.0g - MSMS
Search - Identity Search, m/z tolerance: 1.6 for precursor ions and
0.8 for product ions, Positive and Negative)
(d) Wiley Registry of Tandem Mass Spectral Data, MSforID (Ed. 1,
MSforID Search, m/z tolerance: 0.01-0.1, Intensity threshold: 0.05,
Positive and Negative)
ParticipantID: schymane
Category: category2
Authors: Schymanski, Emma(1) and Meringer, Markus (2)
Affiliations: (1) Eawag: Swiss Federal Institute of Aquatic Science and
Technology, Überlandstrasse 133, CH-8600 Dübendorf,
Switzerland (2) DLR: German Aerospace Centre,
Münchnerstrasse 20, D-82234 Oberpfaffenhofen-Wessling,
Germany
Automatic pipeline: no
Spectral libraries: no (or very limited)
Abstract:
There is no specific MOLGEN for LC-MS/MS yet.
Thus, Category 2 challenges only have answers where the formula and
specific substructure information was clear. There, we used structure
generation with either MOLGEN 3.5 or 5.0, adding the substructures
from spectral interpretation manually. We then came up with consensus
scores using the steric energy of the candidates calculated with
MOLGEN-QSPR and scores from in silico fragmentation with MetFrag.
The MOLGEN 5.0 file went through an additional conversion with OpenBabel
so that MetFrag could handle the aromaticity properly. Challenge 16
MetFrag results were calculated with the command line version and has
slightly different scoring to the rest. For the same challenge, the
minimum of 3 QSPR calculations was also used.
Challenges 1-6, 12, 16: no submissions.
MOLGEN 3.5: Challenges 10, 11, 13, 15, 17.
MOLGEN 5.0: Challenge 14.
Details per Challenge and Participant. See legend at bottom for more details
The table is also available as CSV download| participant | category | challenge | rank | tc | bc | wc | ec | rrp | p | wbc | wwc | wec | wrrp |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| birmingham | category2 | challenge1 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
| cruttkies | category2 | challenge1 | 45 | 1423 | 21 | 1378 | 24 | 0.98 | 0.01 | 0.15 | 0.69 | 0.16 | 0.70 |
| hshen | category2 | challenge1 | 5 | 6 | 4 | 1 | 1 | 0.20 | 0.07 | 0.87 | 0.06 | 0.00 | 0.13 |
| metfusion | category2 | challenge1 | 1 | 1356 | 0 | 1355 | 1 | 1.00 | 0.00 | 0.00 | 1.00 | 0.00 | 1.00 |
| oberacher | category2 | challenge1 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
| birmingham | category2 | challenge2 | - | 7 | - | - | - | - | - | - | - | - | - |
| cruttkies | category2 | challenge2 | - | 250 | - | - | - | - | - | - | - | - | - |
| hshen | category2 | challenge2 | 1 | 6 | 0 | 5 | 1 | 1.00 | 0.28 | 0.00 | 0.72 | 0.00 | 1.00 |
| metfusion | category2 | challenge2 | 30 | 543 | 29 | 513 | 1 | 0.95 | 0.00 | 0.09 | 0.91 | 0.00 | 0.91 |
| birmingham | category2 | challenge3 | - | 1 | - | - | - | - | - | - | - | - | - |
| cruttkies | category2 | challenge3 | 21 | 1312 | 20 | 1291 | 1 | 0.98 | 0.00 | 0.20 | 0.80 | 0.00 | 0.80 |
| hshen | category2 | challenge3 | - | 8 | - | - | - | - | - | - | - | - | - |
| metfusion | category2 | challenge3 | - | 1246 | - | - | - | - | - | - | - | - | - |
| birmingham | category2 | challenge4 | - | 2 | - | - | - | - | - | - | - | - | - |
| cruttkies | category2 | challenge4 | - | 1092 | - | - | - | - | - | - | - | - | - |
| hshen | category2 | challenge4 | - | 14 | - | - | - | - | - | - | - | - | - |
| metfusion | category2 | challenge4 | 195 | 3023 | 194 | 2828 | 1 | 0.94 | 0.00 | 0.08 | 0.92 | 0.00 | 0.92 |
| birmingham | category2 | challenge5 | 4 | 4 | 0 | 0 | 4 | 0.50 | 0.25 | 0.00 | 0.00 | 0.75 | 0.25 |
| cruttkies | category2 | challenge5 | 275 | 3978 | 270 | 3703 | 5 | 0.93 | 0.00 | 0.16 | 0.83 | 0.00 | 0.83 |
| hshen | category2 | challenge5 | 5 | 17 | 3 | 12 | 2 | 0.78 | 0.09 | 0.29 | 0.54 | 0.09 | 0.62 |
| metfusion | category2 | challenge5 | 386 | 3760 | 385 | 3374 | 1 | 0.90 | 0.00 | 0.12 | 0.88 | 0.00 | 0.88 |
| birmingham | category2 | challenge6 | - | 3 | - | - | - | - | - | - | - | - | - |
| cruttkies | category2 | challenge6 | - | 4566 | - | - | - | - | - | - | - | - | - |
| hshen | category2 | challenge6 | 11 | 20 | 10 | 9 | 1 | 0.47 | 0.06 | 0.67 | 0.28 | 0.00 | 0.33 |
| metfusion | category2 | challenge6 | 25 | 3254 | 24 | 3229 | 1 | 0.99 | 0.00 | 0.01 | 0.99 | 0.00 | 0.99 |
| birmingham | category2 | challenge10 | - | 3 | - | - | - | - | - | - | - | - | - |
| cruttkies | category2 | challenge10 | 302 | 536 | 282 | 234 | 20 | 0.46 | 0.00 | 0.98 | 0.01 | 0.01 | 0.01 |
| hshen | category2 | challenge10 | - | 15 | - | - | - | - | - | - | - | - | - |
| metfusion | category2 | challenge10 | 307 | 515 | 306 | 208 | 1 | 0.40 | 0.00 | 0.65 | 0.35 | 0.00 | 0.35 |
| oberacher | category2 | challenge10 | - | 2 | - | - | - | - | - | - | - | - | - |
| schymane | category2 | challenge10 | 63 | 171 | 62 | 108 | 1 | 0.64 | 0.01 | 0.51 | 0.48 | 0.00 | 0.49 |
| cruttkies | category2 | challenge11 | - | 2156 | - | - | - | - | - | - | - | - | - |
| hshen | category2 | challenge11 | - | 32 | - | - | - | - | - | - | - | - | - |
| metfusion | category2 | challenge11 | - | 346 | - | - | - | - | - | - | - | - | - |
| schymane | category2 | challenge11 | 3 | 8 | 2 | 5 | 1 | 0.71 | 0.13 | 0.26 | 0.61 | 0.00 | 0.74 |
| cruttkies | category2 | challenge12 | - | 386 | - | - | - | - | - | - | - | - | - |
| hshen | category2 | challenge12 | - | 47 | - | - | - | - | - | - | - | - | - |
| metfusion | category2 | challenge12 | - | 699 | - | - | - | - | - | - | - | - | - |
| birmingham | category2 | challenge13 | - | 1 | - | - | - | - | - | - | - | - | - |
| cruttkies | category2 | challenge13 | 5 | 1307 | 1 | 1302 | 4 | 1.00 | 0.01 | 0.01 | 0.97 | 0.02 | 0.98 |
| hshen | category2 | challenge13 | - | 38 | - | - | - | - | - | - | - | - | - |
| metfusion | category2 | challenge13 | 64 | 1119 | 63 | 1055 | 1 | 0.94 | 0.00 | 0.08 | 0.92 | 0.00 | 0.92 |
| oberacher | category2 | challenge13 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
| schymane | category2 | challenge13 | 3 | 4 | 2 | 1 | 1 | 0.33 | 0.25 | 0.50 | 0.25 | 0.00 | 0.50 |
| birmingham | category2 | challenge14 | 12 | 28 | 10 | 16 | 2 | 0.61 | 0.04 | 0.39 | 0.53 | 0.04 | 0.57 |
| cruttkies | category2 | challenge14 | 39 | 123 | 35 | 84 | 4 | 0.70 | 0.01 | 0.74 | 0.22 | 0.03 | 0.23 |
| hshen | category2 | challenge14 | - | 28 | - | - | - | - | - | - | - | - | - |
| metfusion | category2 | challenge14 | 1 | 243 | 0 | 242 | 1 | 1.00 | 0.01 | 0.00 | 0.99 | 0.00 | 1.00 |
| oberacher | category2 | challenge14 | - | 1 | - | - | - | - | - | - | - | - | - |
| schymane | category2 | challenge14 | 22 | 41 | 21 | 19 | 1 | 0.47 | 0.03 | 0.64 | 0.34 | 0.00 | 0.36 |
| birmingham | category2 | challenge15 | - | 1 | - | - | - | - | - | - | - | - | - |
| cruttkies | category2 | challenge15 | 316 | 2053 | 312 | 1737 | 4 | 0.85 | 0.00 | 0.50 | 0.49 | 0.00 | 0.49 |
| hshen | category2 | challenge15 | - | 10 | - | - | - | - | - | - | - | - | - |
| metfusion | category2 | challenge15 | 1 | 1757 | 0 | 1756 | 1 | 1.00 | 0.00 | 0.00 | 1.00 | 0.00 | 1.00 |
| oberacher | category2 | challenge15 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
| schymane | category2 | challenge15 | 26 | 32 | 25 | 6 | 1 | 0.19 | 0.03 | 0.80 | 0.17 | 0.00 | 0.20 |
| cruttkies | category2 | challenge16 | 2585 | 2585 | 286 | 0 | 2299 | 0.44 | 0.00 | 1.00 | 0.00 | 0.00 | 0.00 |
| hshen | category2 | challenge16 | - | 13 | - | - | - | - | - | - | - | - | - |
| metfusion | category2 | challenge16 | 1562 | 4351 | 1561 | 2789 | 1 | 0.64 | 0.00 | 0.38 | 0.62 | 0.00 | 0.62 |
| birmingham | category2 | challenge17 | - | 1 | - | - | - | - | - | - | - | - | - |
| cruttkies | category2 | challenge17 | 21 | 898 | 19 | 877 | 2 | 0.98 | 0.01 | 0.12 | 0.87 | 0.01 | 0.87 |
| hshen | category2 | challenge17 | - | 18 | - | - | - | - | - | - | - | - | - |
| metfusion | category2 | challenge17 | 40 | 625 | 39 | 585 | 1 | 0.94 | 0.00 | 0.08 | 0.92 | 0.00 | 0.92 |
| schymane | category2 | challenge17 | - | 1590 | - | - | - | - | - | - | - | - | - |
Table legend:
- rank
- Absolute rank of correct solution
- tc
- Total number of candidates
- bc
- Number of candidates with a score better than correct solution
- wc
- Number of candidates with a score worse than correct solution
- ec
- Number of candidates with same score as the correct solution
- rrp
- Relative ranking position (1.0 is good, 0.0 is not)
- p
- Score of correct solution
- wbc
- Sum of scores better than correct solution
- wwc
- Sum of scores worse than correct solution
- wec
- Sum of scores equal to correct solution
- wrrp
- RRP weighted by the scores (1 is good)