News
March 29th, 2017
The CASMI 2016 Cat 2+3 paper is out! Jan 20th, 2017
Organisation of CASMI 2017 is underway, stay tuned! Dec 4th, 2016
The MS1 peak lists for Category 2+3 have been added for completeness. May 6th, 2016
The winners and full results are available. April 25th, 2016
The solutions are public now. April 18th, 2016
The contest is closed now, the results are fantastic and will be opened soon! April 9th, 2016
All teams who submit before the deadline April 11th will be allowed to update the submission until Friday 15th. February 12th, 2016
New categories 2 and 3 and data for automatic methods released. 10 new challenges in category 1. January 25th, 2016
E. Schymanski and S. Neumann joined the organising team, additional contest data coming soon. January 11th, 2016
New CASMI 2016 raw data files are available.
March 29th, 2017
The CASMI 2016 Cat 2+3 paper is out! Jan 20th, 2017
Organisation of CASMI 2017 is underway, stay tuned! Dec 4th, 2016
The MS1 peak lists for Category 2+3 have been added for completeness. May 6th, 2016
The winners and full results are available. April 25th, 2016
The solutions are public now. April 18th, 2016
The contest is closed now, the results are fantastic and will be opened soon! April 9th, 2016
All teams who submit before the deadline April 11th will be allowed to update the submission until Friday 15th. February 12th, 2016
New categories 2 and 3 and data for automatic methods released. 10 new challenges in category 1. January 25th, 2016
E. Schymanski and S. Neumann joined the organising team, additional contest data coming soon. January 11th, 2016
New CASMI 2016 raw data files are available.
Results in Category 1
Details per Challenge and Participant. See legend at bottom for more details
The details table is also available as CSV download. The InChIKey block1 filtered submissions used in the automatic evaluation are available for download, as well as the original submissions.participant | challenge | rank | rankmax | tc | bc | wc | ec | rrp | p | wbc | wwc | wec | wrrp |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
avaniya002-challenge-010 | challenge-010 | 1.0 | 1 | 2 | 0 | 1 | 1 | 1.00 | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
avaniya002-challenge-011 | challenge-011 | 1.0 | 1 | 13 | 0 | 12 | 1 | 1.00 | 0.08 | 0.00 | 0.92 | 0.00 | 1.00 |
avaniya002-challenge-012 | challenge-012 | 1.0 | 1 | 24 | 0 | 23 | 1 | 1.00 | 0.91 | 0.00 | 0.09 | 0.00 | 1.00 |
avaniya002-challenge-013 | challenge-013 | 1.0 | 1 | 108 | 0 | 107 | 1 | 1.00 | 0.63 | 0.00 | 0.37 | 0.00 | 1.00 |
avaniya002-challenge-014 | challenge-014 | 68.0 | 68 | 389 | 67 | 321 | 1 | 0.83 | 0.00 | 0.57 | 0.43 | 0.00 | 0.43 |
avaniya002-challenge-015 | challenge-015 | 1.0 | 1 | 87 | 0 | 86 | 1 | 1.00 | 0.31 | 0.00 | 0.69 | 0.00 | 1.00 |
avaniya002-challenge-016 | challenge-016 | 2.0 | 2 | 144 | 1 | 142 | 1 | 0.99 | 0.35 | 0.36 | 0.29 | 0.00 | 0.64 |
avaniya002-challenge-017 | challenge-017 | 1.0 | 1 | 16 | 0 | 15 | 1 | 1.00 | 0.78 | 0.00 | 0.22 | 0.00 | 1.00 |
avaniya002-challenge-018 | challenge-018 | 1.0 | 1 | 56 | 0 | 55 | 1 | 1.00 | 0.73 | 0.00 | 0.27 | 0.00 | 1.00 |
avaniya002-challenge-019 | challenge-019 | 1.0 | 1 | 17 | 0 | 16 | 1 | 1.00 | 0.85 | 0.00 | 0.15 | 0.00 | 1.00 |
avaniya002-challenge-001 | challenge-001 | 1.0 | 1 | 43 | 0 | 42 | 1 | 1.00 | 0.15 | 0.00 | 0.85 | 0.00 | 1.00 |
avaniya002-challenge-002 | challenge-002 | 1.0 | 1 | 44 | 0 | 43 | 1 | 1.00 | 0.13 | 0.00 | 0.87 | 0.00 | 1.00 |
avaniya002-challenge-004 | challenge-004 | 1.0 | 1 | 299 | 0 | 298 | 1 | 1.00 | 0.01 | 0.00 | 0.99 | 0.00 | 1.00 |
avaniya002-challenge-005 | challenge-005 | - | - | 5 | - | - | - | - | - | - | - | - | - |
avaniya002-challenge-006 | challenge-006 | - | - | 35 | - | - | - | - | - | - | - | - | - |
avaniya002-challenge-007 | challenge-007 | 1.0 | 1 | 46 | 0 | 45 | 1 | 1.00 | 0.12 | 0.00 | 0.88 | 0.00 | 1.00 |
avaniya002-challenge-008 | challenge-008 | 1.0 | 1 | 285 | 0 | 284 | 1 | 1.00 | 0.01 | 0.00 | 0.99 | 0.00 | 1.00 |
avaniya002-challenge-009 | challenge-009 | 1.0 | 1 | 189 | 0 | 188 | 1 | 1.00 | 0.56 | 0.00 | 0.44 | 0.00 | 1.00 |
felicityallencfmorig-challenge-010 | challenge-010 | 1.0 | 1 | 44 | 0 | 43 | 1 | 1.00 | 0.09 | 0.00 | 0.91 | 0.00 | 1.00 |
felicityallencfmorig-challenge-011 | challenge-011 | 19.0 | 19 | 297 | 18 | 278 | 1 | 0.94 | 0.01 | 0.21 | 0.78 | 0.00 | 0.79 |
felicityallencfmorig-challenge-012 | challenge-012 | 2.0 | 2 | 243 | 1 | 241 | 1 | 1.00 | 0.03 | 0.03 | 0.94 | 0.00 | 0.97 |
felicityallencfmorig-challenge-013 | challenge-013 | 3.0 | 3 | 290 | 2 | 287 | 1 | 0.99 | 0.01 | 0.03 | 0.96 | 0.00 | 0.97 |
felicityallencfmorig-challenge-014 | challenge-014 | 8.0 | 8 | 495 | 7 | 487 | 1 | 0.99 | 0.00 | 0.04 | 0.96 | 0.00 | 0.96 |
felicityallencfmorig-challenge-015 | challenge-015 | 1.0 | 1 | 359 | 0 | 358 | 1 | 1.00 | 0.02 | 0.00 | 0.98 | 0.00 | 1.00 |
felicityallencfmorig-challenge-016 | challenge-016 | 2.0 | 2 | 141 | 1 | 139 | 1 | 0.99 | 0.05 | 0.07 | 0.88 | 0.00 | 0.93 |
felicityallencfmorig-challenge-017 | challenge-017 | 1.0 | 1 | 30 | 0 | 29 | 1 | 1.00 | 0.10 | 0.00 | 0.90 | 0.00 | 1.00 |
felicityallencfmorig-challenge-018 | challenge-018 | 4.0 | 4 | 157 | 3 | 153 | 1 | 0.98 | 0.03 | 0.11 | 0.86 | 0.00 | 0.89 |
felicityallencfmorig-challenge-019 | challenge-019 | 3.0 | 3 | 105 | 2 | 102 | 1 | 0.98 | 0.02 | 0.06 | 0.91 | 0.00 | 0.94 |
felicityallencfmorig-challenge-001 | challenge-001 | 4.5 | 5 | 22 | 3 | 17 | 2 | 0.83 | 0.08 | 0.37 | 0.46 | 0.08 | 0.55 |
felicityallencfmorig-challenge-002 | challenge-002 | 1.0 | 1 | 12 | 0 | 11 | 1 | 1.00 | 0.18 | 0.00 | 0.82 | 0.00 | 1.00 |
felicityallencfmorig-challenge-004 | challenge-004 | 1.0 | 1 | 63 | 0 | 62 | 1 | 1.00 | 0.05 | 0.00 | 0.95 | 0.00 | 1.00 |
felicityallencfmorig-challenge-005 | challenge-005 | - | - | 196 | - | - | - | - | - | - | - | - | - |
felicityallencfmorig-challenge-006 | challenge-006 | 2.5 | 3 | 11 | 1 | 8 | 2 | 0.85 | 0.13 | 0.18 | 0.56 | 0.13 | 0.69 |
felicityallencfmorig-challenge-007 | challenge-007 | 1.0 | 1 | 537 | 0 | 536 | 1 | 1.00 | 0.03 | 0.00 | 0.97 | 0.00 | 1.00 |
felicityallencfmorig-challenge-008 | challenge-008 | 4.0 | 4 | 63 | 3 | 59 | 1 | 0.95 | 0.05 | 0.19 | 0.76 | 0.00 | 0.81 |
felicityallencfmorig-challenge-009 | challenge-009 | 1.0 | 1 | 558 | 0 | 557 | 1 | 1.00 | 0.02 | 0.00 | 0.98 | 0.00 | 1.00 |
gnpscsifingerid-challenge-010 | challenge-010 | 1.0 | 1 | 7 | 0 | 6 | 1 | 1.00 | 0.19 | 0.00 | 0.81 | 0.00 | 1.00 |
gnpscsifingerid-challenge-011 | challenge-011 | - | - | 10 | - | - | - | - | - | - | - | - | - |
gnpscsifingerid-challenge-012 | challenge-012 | 1.0 | 1 | 10 | 0 | 9 | 1 | 1.00 | 0.19 | 0.00 | 0.81 | 0.00 | 1.00 |
gnpscsifingerid-challenge-013 | challenge-013 | 1.0 | 1 | 10 | 0 | 9 | 1 | 1.00 | 0.16 | 0.00 | 0.84 | 0.00 | 1.00 |
gnpscsifingerid-challenge-014 | challenge-014 | - | - | 10 | - | - | - | - | - | - | - | - | - |
gnpscsifingerid-challenge-015 | challenge-015 | 4.0 | 4 | 43 | 3 | 39 | 1 | 0.93 | 0.04 | 0.15 | 0.81 | 0.00 | 0.85 |
gnpscsifingerid-challenge-016 | challenge-016 | 1.0 | 1 | 8 | 0 | 7 | 1 | 1.00 | 0.19 | 0.00 | 0.81 | 0.00 | 1.00 |
gnpscsifingerid-challenge-017 | challenge-017 | 1.0 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
gnpscsifingerid-challenge-018 | challenge-018 | - | - | 13 | - | - | - | - | - | - | - | - | - |
gnpscsifingerid-challenge-019 | challenge-019 | 1.0 | 1 | 8 | 0 | 7 | 1 | 1.00 | 0.31 | 0.00 | 0.69 | 0.00 | 1.00 |
gnpscsifingerid-challenge-001 | challenge-001 | - | - | 10 | - | - | - | - | - | - | - | - | - |
gnpscsifingerid-challenge-002 | challenge-002 | 1.0 | 1 | 10 | 0 | 9 | 1 | 1.00 | 0.19 | 0.00 | 0.81 | 0.00 | 1.00 |
gnpscsifingerid-challenge-004 | challenge-004 | 1.0 | 1 | 10 | 0 | 9 | 1 | 1.00 | 0.18 | 0.00 | 0.82 | 0.00 | 1.00 |
gnpscsifingerid-challenge-005 | challenge-005 | - | - | 10 | - | - | - | - | - | - | - | - | - |
gnpscsifingerid-challenge-007 | challenge-007 | 1.0 | 1 | 10 | 0 | 9 | 1 | 1.00 | 0.20 | 0.00 | 0.80 | 0.00 | 1.00 |
gnpscsifingerid-challenge-008 | challenge-008 | 1.0 | 1 | 10 | 0 | 9 | 1 | 1.00 | 0.19 | 0.00 | 0.81 | 0.00 | 1.00 |
gnpscsifingerid-challenge-009 | challenge-009 | 1.0 | 1 | 10 | 0 | 9 | 1 | 1.00 | 0.20 | 0.00 | 0.80 | 0.00 | 1.00 |
nikolic-challenge-010 | challenge-010 | 1.0 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
nikolic-challenge-011 | challenge-011 | 1.0 | 1 | 2 | 0 | 1 | 1 | 1.00 | 0.50 | 0.00 | 0.50 | 0.00 | 1.00 |
nikolic-challenge-012 | challenge-012 | 1.0 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
nikolic-challenge-013 | challenge-013 | 1.0 | 1 | 2 | 0 | 1 | 1 | 1.00 | 0.55 | 0.00 | 0.45 | 0.00 | 1.00 |
nikolic-challenge-014 | challenge-014 | 2.0 | 2 | 5 | 1 | 3 | 1 | 0.75 | 0.23 | 0.25 | 0.52 | 0.00 | 0.75 |
nikolic-challenge-015 | challenge-015 | 1.0 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
nikolic-challenge-016 | challenge-016 | 1.0 | 1 | 2 | 0 | 1 | 1 | 1.00 | 0.60 | 0.00 | 0.40 | 0.00 | 1.00 |
nikolic-challenge-017 | challenge-017 | 1.0 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
nikolic-challenge-018 | challenge-018 | 1.0 | 1 | 2 | 0 | 1 | 1 | 1.00 | 0.58 | 0.00 | 0.42 | 0.00 | 1.00 |
nikolic-challenge-019 | challenge-019 | 1.0 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
nikolic-challenge-001 | challenge-001 | 2.0 | 2 | 4 | 1 | 2 | 1 | 0.67 | 0.25 | 0.29 | 0.47 | 0.00 | 0.71 |
nikolic-challenge-002 | challenge-002 | 2.0 | 2 | 2 | 1 | 0 | 1 | 0.00 | 0.48 | 0.52 | 0.00 | 0.00 | 0.48 |
nikolic-challenge-004 | challenge-004 | 1.0 | 1 | 9 | 0 | 8 | 1 | 1.00 | 0.12 | 0.00 | 0.88 | 0.00 | 1.00 |
nikolic-challenge-005 | challenge-005 | 1.0 | 1 | 3 | 0 | 2 | 1 | 1.00 | 0.36 | 0.00 | 0.64 | 0.00 | 1.00 |
nikolic-challenge-006 | challenge-006 | 1.0 | 1 | 3 | 0 | 2 | 1 | 1.00 | 0.36 | 0.00 | 0.64 | 0.00 | 1.00 |
nikolic-challenge-007 | challenge-007 | 1.0 | 1 | 2 | 0 | 1 | 1 | 1.00 | 0.57 | 0.00 | 0.43 | 0.00 | 1.00 |
nikolic-challenge-008 | challenge-008 | 2.0 | 2 | 2 | 1 | 0 | 1 | 0.00 | 0.46 | 0.54 | 0.00 | 0.00 | 0.46 |
nikolic-challenge-009 | challenge-009 | 1.0 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
pma-challenge-010 | challenge-010 | 1.0 | 1 | 1 | 0 | 0 | 1 | - | 1.00 | 0.00 | 0.00 | 0.00 | 1.00 |
pma-challenge-011 | challenge-011 | 8.0 | 8 | 9 | 7 | 1 | 1 | 0.12 | 0.09 | 0.82 | 0.09 | 0.00 | 0.18 |
pma-challenge-012 | challenge-012 | 4.0 | 4 | 9 | 3 | 5 | 1 | 0.62 | 0.12 | 0.41 | 0.47 | 0.00 | 0.59 |
pma-challenge-013 | challenge-013 | - | - | 13 | - | - | - | - | - | - | - | - | - |
pma-challenge-014 | challenge-014 | - | - | 12 | - | - | - | - | - | - | - | - | - |
pma-challenge-001 | challenge-001 | 3.0 | 3 | 8 | 2 | 5 | 1 | 0.71 | 0.15 | 0.37 | 0.48 | 0.00 | 0.63 |
pma-challenge-002 | challenge-002 | 2.0 | 2 | 8 | 1 | 6 | 1 | 0.86 | 0.19 | 0.22 | 0.59 | 0.00 | 0.78 |
pma-challenge-004 | challenge-004 | 2.0 | 2 | 11 | 1 | 9 | 1 | 0.90 | 0.10 | 0.11 | 0.79 | 0.00 | 0.89 |
pma-challenge-005 | challenge-005 | - | - | 11 | - | - | - | - | - | - | - | - | - |
pma-challenge-006 | challenge-006 | 7.0 | 7 | 8 | 6 | 1 | 1 | 0.14 | 0.10 | 0.81 | 0.09 | 0.00 | 0.19 |
pma-challenge-007 | challenge-007 | 2.0 | 2 | 8 | 1 | 6 | 1 | 0.86 | 0.17 | 0.19 | 0.64 | 0.00 | 0.81 |
pma-challenge-008 | challenge-008 | 4.0 | 4 | 12 | 3 | 8 | 1 | 0.73 | 0.09 | 0.27 | 0.64 | 0.00 | 0.73 |
pma-challenge-009 | challenge-009 | 1.0 | 1 | 12 | 0 | 11 | 1 | 1.00 | 0.11 | 0.00 | 0.89 | 0.00 | 1.00 |
samuelblcms-challenge-010 | challenge-010 | 1.0 | 1 | 2 | 0 | 1 | 1 | 1.00 | 0.50 | 0.00 | 0.50 | 0.00 | 1.00 |
samuelblcms-challenge-011 | challenge-011 | 4.0 | 4 | 7 | 3 | 3 | 1 | 0.50 | 0.14 | 0.43 | 0.42 | 0.00 | 0.57 |
samuelblcms-challenge-012 | challenge-012 | 1.0 | 1 | 17 | 0 | 16 | 1 | 1.00 | 0.06 | 0.00 | 0.94 | 0.00 | 1.00 |
samuelblcms-challenge-013 | challenge-013 | - | - | 11 | - | - | - | - | - | - | - | - | - |
samuelblcms-challenge-014 | challenge-014 | 29.0 | 29 | 443 | 28 | 414 | 1 | 0.94 | 0.00 | 0.07 | 0.93 | 0.00 | 0.93 |
samuelblcms-challenge-015 | challenge-015 | 1.0 | 1 | 32 | 0 | 31 | 1 | 1.00 | 0.03 | 0.00 | 0.97 | 0.00 | 1.00 |
samuelblcms-challenge-016 | challenge-016 | 2.0 | 2 | 12 | 1 | 10 | 1 | 0.91 | 0.10 | 0.10 | 0.81 | 0.00 | 0.90 |
samuelblcms-challenge-017 | challenge-017 | - | - | 14 | - | - | - | - | - | - | - | - | - |
samuelblcms-challenge-018 | challenge-018 | 12.0 | 12 | 19 | 11 | 7 | 1 | 0.39 | 0.05 | 0.58 | 0.37 | 0.00 | 0.42 |
samuelblcms-challenge-019 | challenge-019 | 1.0 | 1 | 11 | 0 | 10 | 1 | 1.00 | 0.10 | 0.00 | 0.90 | 0.00 | 1.00 |
samuelblcms-challenge-001 | challenge-001 | 7.0 | 7 | 10 | 6 | 3 | 1 | 0.33 | 0.10 | 0.63 | 0.27 | 0.00 | 0.37 |
samuelblcms-challenge-002 | challenge-002 | 1.0 | 1 | 10 | 0 | 9 | 1 | 1.00 | 0.12 | 0.00 | 0.88 | 0.00 | 1.00 |
samuelblcms-challenge-004 | challenge-004 | 6.0 | 11 | 11 | 0 | 0 | 11 | 0.50 | 0.09 | 0.00 | 0.00 | 0.91 | 0.09 |
samuelblcms-challenge-005 | challenge-005 | - | - | 12 | - | - | - | - | - | - | - | - | - |
samuelblcms-challenge-006 | challenge-006 | 6.0 | 6 | 10 | 5 | 4 | 1 | 0.44 | 0.12 | 0.62 | 0.25 | 0.00 | 0.38 |
samuelblcms-challenge-007 | challenge-007 | 1.0 | 1 | 10 | 0 | 9 | 1 | 1.00 | 0.22 | 0.00 | 0.78 | 0.00 | 1.00 |
samuelblcms-challenge-009 | challenge-009 | 2.0 | 2 | 15 | 1 | 13 | 1 | 0.93 | 0.07 | 0.08 | 0.86 | 0.00 | 0.92 |
tkind-challenge-010 | challenge-010 | 1.0 | 1 | 150 | 0 | 149 | 1 | 1.00 | 0.01 | 0.00 | 0.99 | 0.00 | 1.00 |
tkind-challenge-011 | challenge-011 | 1.0 | 1 | 224 | 0 | 223 | 1 | 1.00 | 0.01 | 0.00 | 0.99 | 0.00 | 1.00 |
tkind-challenge-012 | challenge-012 | 1.0 | 1 | 992 | 0 | 991 | 1 | 1.00 | 0.00 | 0.00 | 1.00 | 0.00 | 1.00 |
tkind-challenge-013 | challenge-013 | 1.0 | 1 | 360 | 0 | 359 | 1 | 1.00 | 0.01 | 0.00 | 0.99 | 0.00 | 1.00 |
tkind-challenge-014 | challenge-014 | 292.0 | 582 | 582 | 1 | 0 | 581 | 0.50 | 0.00 | 0.00 | 0.00 | 1.00 | 0.00 |
tkind-challenge-015 | challenge-015 | 1.0 | 1 | 2006 | 0 | 2005 | 1 | 1.00 | 0.00 | 0.00 | 1.00 | 0.00 | 1.00 |
tkind-challenge-016 | challenge-016 | 1.0 | 1 | 409 | 0 | 408 | 1 | 1.00 | 0.00 | 0.00 | 1.00 | 0.00 | 1.00 |
tkind-challenge-017 | challenge-017 | 1.0 | 1 | 22 | 0 | 21 | 1 | 1.00 | 0.08 | 0.00 | 0.92 | 0.00 | 1.00 |
tkind-challenge-018 | challenge-018 | 1.0 | 1 | 69 | 0 | 68 | 1 | 1.00 | 0.02 | 0.00 | 0.98 | 0.00 | 1.00 |
tkind-challenge-019 | challenge-019 | 1.0 | 1 | 20 | 0 | 19 | 1 | 1.00 | 0.24 | 0.00 | 0.76 | 0.00 | 1.00 |
tkind-challenge-001 | challenge-001 | 5.0 | 5 | 9 | 4 | 4 | 1 | 0.50 | 0.10 | 0.70 | 0.20 | 0.00 | 0.30 |
tkind-challenge-002 | challenge-002 | 1.0 | 1 | 13 | 0 | 12 | 1 | 1.00 | 0.17 | 0.00 | 0.83 | 0.00 | 1.00 |
tkind-challenge-004 | challenge-004 | 1.0 | 1 | 11 | 0 | 10 | 1 | 1.00 | 0.10 | 0.00 | 0.90 | 0.00 | 1.00 |
tkind-challenge-005 | challenge-005 | - | - | 32 | - | - | - | - | - | - | - | - | - |
tkind-challenge-006 | challenge-006 | - | - | 20 | - | - | - | - | - | - | - | - | - |
tkind-challenge-007 | challenge-007 | 3.0 | 5 | 5 | 0 | 0 | 5 | 0.50 | 0.20 | 0.00 | 0.00 | 0.80 | 0.20 |
tkind-challenge-008 | challenge-008 | 2.0 | 3 | 3 | 0 | 0 | 3 | 0.50 | 0.33 | 0.00 | 0.00 | 0.67 | 0.33 |
tkind-challenge-009 | challenge-009 | 1.0 | 1 | 3 | 0 | 2 | 1 | 1.00 | 0.36 | 0.00 | 0.64 | 0.00 | 1.00 |
Table legend:
- rank
- Absolute rank of correct solution (average of tied ranks)
- rankmax
- Absolute rank of correct solution (max of tied ranks, worst case)
- tc
- Total number of candidates
- bc
- Number of candidates with a score better than correct solution
- wc
- Number of candidates with a score worse than correct solution
- ec
- Number of candidates with same score as the correct solution
- rrp
- Relative ranking position (1.0 is good, 0.0 is not)
- p
- Score of correct solution
- wbc
- Sum of scores better than correct solution
- wwc
- Sum of scores worse than correct solution
- wec
- Sum of scores equal to correct solution
- wrrp
- RRP weighted by the scores (1 is good)