Benchmarks

Benchmarks are used in the project to validate the models and compare them to similar tools.

List of Benchmarks

McIntyre et al (2017) mcintyre2017: https://ftp-private.ncbi.nlm.nih.gov/nist-immsa/IMMSA/
Meyer et al (2022) meyer2022: https://cami-challenge.org/ - The “marine” dataset (https://cami-challenge.org/datasets/Marine/) includes a gold standard.

References

[1]

Alexander Fritz, Patrick Hofmann, Stefan Majda, Elisabeth Dahms, Helene Marie Draese, Till R Lesker, Peter Belmann, Matthew Z DeMaere, Aaron E Darling, Alexander Sczyrba, Andreas Bremges, and Alice C McHardy. Camisim: simulating metagenomes and microbial communities. Microbiome, 7(1):17, 2019. URL: https://doi.org/10.1186/s40168-019-0633-6, doi:10.1186/s40168-019-0633-6.

[2]

Dirk Hackenberger, Hamna Imtiaz, Amogelang R. Raphenya, Brian P. Alcock, Hendrik N. Poinar, Gerard D. Wright, and Andrew G. McArthur. Carpdm: cost-effective antibiotic resistome profiling of metagenomic samples using targeted enrichment. bioRxiv, 2024. URL: https://www.biorxiv.org/content/early/2024/03/30/2024.03.27.587061, arXiv:https://www.biorxiv.org/content/early/2024/03/30/2024.03.27.587061.full.pdf, doi:10.1101/2024.03.27.587061.

[3]

Alison B.R. McIntyre, Rachid Ounit, Ebrahim Afshinnekoo, and others. Comprehensive benchmarking and ensemble approaches for metagenomic classifiers. Genome Biology, 18(1):182, 2017. URL: https://doi.org/10.1186/s13059-017-1299-7, doi:10.1186/s13059-017-1299-7.