Field evaluation of the diagnostic performance of EasyScan GO: A digital malaria microscopy device based on machine-learning

Debashish Das; Ranitha Vongpromek; Thanawat Assawariyathipat; Ketsanee Srinamon; Kalynn Kennon; Kasia Stepniewska; Aniruddha Ghose; Abdullah Abu Sayeed; M. Abul Faiz; Rebeca Linhares Abreu Netto; Andre Siqueira; Serge R. Yerbanga; Jean Bosco Ouédraogo; James J. Callery; Thomas J. Peto; Rupam Tripura; Felix Koukouikila-Koussounda; Francine Ntoumi; John Michael Ong’echa; Bernhards Ogutu; Prakash Ghimire; Jutta Marfurt; Benedikt Ley; Amadou Seck; Magatte Ndiaye; Bhavani Moodley; Lisa Ming Sun; Laypaw Archasuksan; Stephane Proux; Sam L. Nsobya; Philip J. Rosenthal; Matthew P. Horning; Shawn K. McGuire; Courosh Mehanian; Stephen Burkot; Charles B. Delahunt; Christine Bachman; Ric N. Price; Arjen M. Dondorp; François Chappuis; Philippe J. Guérin; Mehul Dhorda

doi:10.1186/s12936-022-04146-1

Field evaluation of the diagnostic performance of EasyScan GO: A digital malaria microscopy device based on machine-learning

Debashish Das, Ranitha Vongpromek, Thanawat Assawariyathipat, Ketsanee Srinamon, Kalynn Kennon, Kasia Stepniewska, Aniruddha Ghose, Abdullah Abu Sayeed, M. Abul Faiz, Rebeca Linhares Abreu Netto, Andre Siqueira, Serge R. Yerbanga, Jean Bosco Ouédraogo, James J. Callery, Thomas J. Peto, Rupam Tripura, Felix Koukouikila-Koussounda, Francine Ntoumi, John Michael Ong’echa, Bernhards OgutuPrakash Ghimire, Jutta Marfurt, Benedikt Ley, Amadou Seck, Magatte Ndiaye, Bhavani Moodley, Lisa Ming Sun, Laypaw Archasuksan, Stephane Proux, Sam L. Nsobya, Philip J. Rosenthal, Matthew P. Horning, Shawn K. McGuire, Courosh Mehanian, Stephen Burkot, Charles B. Delahunt, Christine Bachman, Ric N. Price, Arjen M. Dondorp, François Chappuis, Philippe J. Guérin, Mehul Dhorda

Research output: Contribution to journal › Article › peer-review

25 Citations (Scopus)

69 Downloads (Pure)

Abstract

Background: Microscopic examination of Giemsa-stained blood films remains the reference standard for malaria parasite detection and quantification, but is undermined by difficulties in ensuring high-quality manual reading and inter-reader reliability. Automated parasite detection and quantification may address this issue.

Methods: A multi-centre, observational study was conducted during 2018 and 2019 at 11 sites to assess the performance of the EasyScan Go, a microscopy device employing machine-learning-based image analysis. Sensitivity, specificity, accuracy of species detection and parasite density estimation were assessed with expert microscopy as the reference. Intra- and inter-device reliability of the device was also evaluated by comparing results from repeat reads on the same and two different devices. This study has been reported in accordance with the Standards for Reporting Diagnostic accuracy studies (STARD) checklist.

Results: In total, 2250 Giemsa-stained blood films were prepared and read independently by expert microscopists and the EasyScan Go device. The diagnostic sensitivity of EasyScan Go was 91.1% (95% CI 88.9–92.7), and specificity 75.6% (95% CI 73.1–78.0). With good quality slides sensitivity was similar (89.1%, 95%CI 86.2–91.5), but specificity increased to 85.1% (95%CI 82.6–87.4). Sensitivity increased with parasitaemia rising from 57% at < 200 parasite/µL, to ≥ 90% at > 200–200,000 parasite/µL. Species were identified accurately in 93% of Plasmodium falciparum samples (kappa = 0.76, 95% CI 0.69–0.83), and in 92% of Plasmodium vivax samples (kappa = 0.73, 95% CI 0.66–0.80). Parasite density estimates by the EasyScan Go were within ± 25% of the microscopic reference counts in 23% of slides.

Conclusions: The performance of the EasyScan Go in parasite detection and species identification accuracy fulfil WHO-TDR Research Malaria Microscopy competence level 2 criteria. In terms of parasite quantification and false positive rate, it meets the level 4 WHO-TDR Research Malaria Microscopy criteria. All performance parameters were significantly affected by slide quality. Further software improvement is required to improve sensitivity at low parasitaemia and parasite density estimations. Trial registration ClinicalTrials.gov number NCT03512678.

Original language	English
Article number	122
Pages (from-to)	1-12
Number of pages	12
Journal	Malaria Journal
Volume	21
Issue number	1
DOIs	https://doi.org/10.1186/s12936-022-04146-1
Publication status	Published - Dec 2022

Bibliographical note

Funding Information:
The study was funded by the Intellectual Ventures' Global Good Fund. The sub-study sites were funded by the Wellcome Trust of Great Britain (Bangladesh), the Organização Pan-Americana da Saúde and Conselho Nacional de Desenvolvimento Científico e Tecnológico (Brazil), the Bill & Melinda Gates Foundation—OPP1132628 (Cambodia), the Asia Pacific Malaria Elimination Network supported by the Bill & Melinda Gates Foundation, INV-010504 (Nepal), the President’s Malaria Initiative (Senegal) and the South African Medical Research Council Collaborating Centre Grant (South Africa).

Publisher Copyright:
© 2022, The Author(s).

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1186/s12936-022-04146-1

57112127-oaFinal published version, 1.85 MBLicence: CC BY

Cite this

Das, D., Vongpromek, R., Assawariyathipat, T., Srinamon, K., Kennon, K., Stepniewska, K., Ghose, A., Sayeed, A. A., Faiz, M. A., Netto, R. L. A., Siqueira, A., Yerbanga, S. R., Ouédraogo, J. B., Callery, J. J., Peto, T. J., Tripura, R., Koukouikila-Koussounda, F., Ntoumi, F., Ong’echa, J. M., ... Dhorda, M. (2022). Field evaluation of the diagnostic performance of EasyScan GO: A digital malaria microscopy device based on machine-learning. Malaria Journal, 21(1), 1-12. Article 122. https://doi.org/10.1186/s12936-022-04146-1

@article{c7c7f8dcf5144449bc0254828a1fe71d,

title = "Field evaluation of the diagnostic performance of EasyScan GO: A digital malaria microscopy device based on machine-learning",

abstract = "Background: Microscopic examination of Giemsa-stained blood films remains the reference standard for malaria parasite detection and quantification, but is undermined by difficulties in ensuring high-quality manual reading and inter-reader reliability. Automated parasite detection and quantification may address this issue. Methods: A multi-centre, observational study was conducted during 2018 and 2019 at 11 sites to assess the performance of the EasyScan Go, a microscopy device employing machine-learning-based image analysis. Sensitivity, specificity, accuracy of species detection and parasite density estimation were assessed with expert microscopy as the reference. Intra- and inter-device reliability of the device was also evaluated by comparing results from repeat reads on the same and two different devices. This study has been reported in accordance with the Standards for Reporting Diagnostic accuracy studies (STARD) checklist. Results: In total, 2250 Giemsa-stained blood films were prepared and read independently by expert microscopists and the EasyScan Go device. The diagnostic sensitivity of EasyScan Go was 91.1% (95% CI 88.9–92.7), and specificity 75.6% (95% CI 73.1–78.0). With good quality slides sensitivity was similar (89.1%, 95%CI 86.2–91.5), but specificity increased to 85.1% (95%CI 82.6–87.4). Sensitivity increased with parasitaemia rising from 57% at < 200 parasite/µL, to ≥ 90% at > 200–200,000 parasite/µL. Species were identified accurately in 93% of Plasmodium falciparum samples (kappa = 0.76, 95% CI 0.69–0.83), and in 92% of Plasmodium vivax samples (kappa = 0.73, 95% CI 0.66–0.80). Parasite density estimates by the EasyScan Go were within ± 25% of the microscopic reference counts in 23% of slides. Conclusions: The performance of the EasyScan Go in parasite detection and species identification accuracy fulfil WHO-TDR Research Malaria Microscopy competence level 2 criteria. In terms of parasite quantification and false positive rate, it meets the level 4 WHO-TDR Research Malaria Microscopy criteria. All performance parameters were significantly affected by slide quality. Further software improvement is required to improve sensitivity at low parasitaemia and parasite density estimations. Trial registration ClinicalTrials.gov number NCT03512678.",

keywords = "Artificial intelligence, Diagnostic accuracy, Digital microscopy, Light microscopy, Machine-learning, Malaria",

author = "Debashish Das and Ranitha Vongpromek and Thanawat Assawariyathipat and Ketsanee Srinamon and Kalynn Kennon and Kasia Stepniewska and Aniruddha Ghose and Sayeed, {Abdullah Abu} and Faiz, {M. Abul} and Netto, {Rebeca Linhares Abreu} and Andre Siqueira and Yerbanga, {Serge R.} and Ou{\'e}draogo, {Jean Bosco} and Callery, {James J.} and Peto, {Thomas J.} and Rupam Tripura and Felix Koukouikila-Koussounda and Francine Ntoumi and Ong{\textquoteright}echa, {John Michael} and Bernhards Ogutu and Prakash Ghimire and Jutta Marfurt and Benedikt Ley and Amadou Seck and Magatte Ndiaye and Bhavani Moodley and Sun, {Lisa Ming} and Laypaw Archasuksan and Stephane Proux and Nsobya, {Sam L.} and Rosenthal, {Philip J.} and Horning, {Matthew P.} and McGuire, {Shawn K.} and Courosh Mehanian and Stephen Burkot and Delahunt, {Charles B.} and Christine Bachman and Price, {Ric N.} and Dondorp, {Arjen M.} and Fran{\c c}ois Chappuis and Gu{\'e}rin, {Philippe J.} and Mehul Dhorda",

note = "Funding Information: The study was funded by the Intellectual Ventures' Global Good Fund. The sub-study sites were funded by the Wellcome Trust of Great Britain (Bangladesh), the Organiza{\c c}{\~a}o Pan-Americana da Sa{\'u}de and Conselho Nacional de Desenvolvimento Cient{\'i}fico e Tecnol{\'o}gico (Brazil), the Bill & Melinda Gates Foundation—OPP1132628 (Cambodia), the Asia Pacific Malaria Elimination Network supported by the Bill & Melinda Gates Foundation, INV-010504 (Nepal), the President{\textquoteright}s Malaria Initiative (Senegal) and the South African Medical Research Council Collaborating Centre Grant (South Africa). Publisher Copyright: {\textcopyright} 2022, The Author(s).",

year = "2022",

month = dec,

doi = "10.1186/s12936-022-04146-1",

language = "English",

volume = "21",

pages = "1--12",

journal = "Malaria Journal",

issn = "1475-2875",

publisher = "BioMed Central",

number = "1",

}

Das, D, Vongpromek, R, Assawariyathipat, T, Srinamon, K, Kennon, K, Stepniewska, K, Ghose, A, Sayeed, AA, Faiz, MA, Netto, RLA, Siqueira, A, Yerbanga, SR, Ouédraogo, JB, Callery, JJ, Peto, TJ, Tripura, R, Koukouikila-Koussounda, F, Ntoumi, F, Ong’echa, JM, Ogutu, B, Ghimire, P, Marfurt, J, Ley, B, Seck, A, Ndiaye, M, Moodley, B, Sun, LM, Archasuksan, L, Proux, S, Nsobya, SL, Rosenthal, PJ, Horning, MP, McGuire, SK, Mehanian, C, Burkot, S, Delahunt, CB, Bachman, C, Price, RN, Dondorp, AM, Chappuis, F, Guérin, PJ & Dhorda, M 2022, 'Field evaluation of the diagnostic performance of EasyScan GO: A digital malaria microscopy device based on machine-learning', Malaria Journal, vol. 21, no. 1, 122, pp. 1-12. https://doi.org/10.1186/s12936-022-04146-1

TY - JOUR

T1 - Field evaluation of the diagnostic performance of EasyScan GO

T2 - A digital malaria microscopy device based on machine-learning

AU - Das, Debashish

AU - Vongpromek, Ranitha

AU - Assawariyathipat, Thanawat

AU - Srinamon, Ketsanee

AU - Kennon, Kalynn

AU - Stepniewska, Kasia

AU - Ghose, Aniruddha

AU - Sayeed, Abdullah Abu

AU - Faiz, M. Abul

AU - Netto, Rebeca Linhares Abreu

AU - Siqueira, Andre

AU - Yerbanga, Serge R.

AU - Ouédraogo, Jean Bosco

AU - Callery, James J.

AU - Peto, Thomas J.

AU - Tripura, Rupam

AU - Koukouikila-Koussounda, Felix

AU - Ntoumi, Francine

AU - Ong’echa, John Michael

AU - Ogutu, Bernhards

AU - Ghimire, Prakash

AU - Marfurt, Jutta

AU - Ley, Benedikt

AU - Seck, Amadou

AU - Ndiaye, Magatte

AU - Moodley, Bhavani

AU - Sun, Lisa Ming

AU - Archasuksan, Laypaw

AU - Proux, Stephane

AU - Nsobya, Sam L.

AU - Rosenthal, Philip J.

AU - Horning, Matthew P.

AU - McGuire, Shawn K.

AU - Mehanian, Courosh

AU - Burkot, Stephen

AU - Delahunt, Charles B.

AU - Bachman, Christine

AU - Price, Ric N.

AU - Dondorp, Arjen M.

AU - Chappuis, François

AU - Guérin, Philippe J.

AU - Dhorda, Mehul

N1 - Funding Information: The study was funded by the Intellectual Ventures' Global Good Fund. The sub-study sites were funded by the Wellcome Trust of Great Britain (Bangladesh), the Organização Pan-Americana da Saúde and Conselho Nacional de Desenvolvimento Científico e Tecnológico (Brazil), the Bill & Melinda Gates Foundation—OPP1132628 (Cambodia), the Asia Pacific Malaria Elimination Network supported by the Bill & Melinda Gates Foundation, INV-010504 (Nepal), the President’s Malaria Initiative (Senegal) and the South African Medical Research Council Collaborating Centre Grant (South Africa). Publisher Copyright: © 2022, The Author(s).

PY - 2022/12

Y1 - 2022/12

N2 - Background: Microscopic examination of Giemsa-stained blood films remains the reference standard for malaria parasite detection and quantification, but is undermined by difficulties in ensuring high-quality manual reading and inter-reader reliability. Automated parasite detection and quantification may address this issue. Methods: A multi-centre, observational study was conducted during 2018 and 2019 at 11 sites to assess the performance of the EasyScan Go, a microscopy device employing machine-learning-based image analysis. Sensitivity, specificity, accuracy of species detection and parasite density estimation were assessed with expert microscopy as the reference. Intra- and inter-device reliability of the device was also evaluated by comparing results from repeat reads on the same and two different devices. This study has been reported in accordance with the Standards for Reporting Diagnostic accuracy studies (STARD) checklist. Results: In total, 2250 Giemsa-stained blood films were prepared and read independently by expert microscopists and the EasyScan Go device. The diagnostic sensitivity of EasyScan Go was 91.1% (95% CI 88.9–92.7), and specificity 75.6% (95% CI 73.1–78.0). With good quality slides sensitivity was similar (89.1%, 95%CI 86.2–91.5), but specificity increased to 85.1% (95%CI 82.6–87.4). Sensitivity increased with parasitaemia rising from 57% at < 200 parasite/µL, to ≥ 90% at > 200–200,000 parasite/µL. Species were identified accurately in 93% of Plasmodium falciparum samples (kappa = 0.76, 95% CI 0.69–0.83), and in 92% of Plasmodium vivax samples (kappa = 0.73, 95% CI 0.66–0.80). Parasite density estimates by the EasyScan Go were within ± 25% of the microscopic reference counts in 23% of slides. Conclusions: The performance of the EasyScan Go in parasite detection and species identification accuracy fulfil WHO-TDR Research Malaria Microscopy competence level 2 criteria. In terms of parasite quantification and false positive rate, it meets the level 4 WHO-TDR Research Malaria Microscopy criteria. All performance parameters were significantly affected by slide quality. Further software improvement is required to improve sensitivity at low parasitaemia and parasite density estimations. Trial registration ClinicalTrials.gov number NCT03512678.

AB - Background: Microscopic examination of Giemsa-stained blood films remains the reference standard for malaria parasite detection and quantification, but is undermined by difficulties in ensuring high-quality manual reading and inter-reader reliability. Automated parasite detection and quantification may address this issue. Methods: A multi-centre, observational study was conducted during 2018 and 2019 at 11 sites to assess the performance of the EasyScan Go, a microscopy device employing machine-learning-based image analysis. Sensitivity, specificity, accuracy of species detection and parasite density estimation were assessed with expert microscopy as the reference. Intra- and inter-device reliability of the device was also evaluated by comparing results from repeat reads on the same and two different devices. This study has been reported in accordance with the Standards for Reporting Diagnostic accuracy studies (STARD) checklist. Results: In total, 2250 Giemsa-stained blood films were prepared and read independently by expert microscopists and the EasyScan Go device. The diagnostic sensitivity of EasyScan Go was 91.1% (95% CI 88.9–92.7), and specificity 75.6% (95% CI 73.1–78.0). With good quality slides sensitivity was similar (89.1%, 95%CI 86.2–91.5), but specificity increased to 85.1% (95%CI 82.6–87.4). Sensitivity increased with parasitaemia rising from 57% at < 200 parasite/µL, to ≥ 90% at > 200–200,000 parasite/µL. Species were identified accurately in 93% of Plasmodium falciparum samples (kappa = 0.76, 95% CI 0.69–0.83), and in 92% of Plasmodium vivax samples (kappa = 0.73, 95% CI 0.66–0.80). Parasite density estimates by the EasyScan Go were within ± 25% of the microscopic reference counts in 23% of slides. Conclusions: The performance of the EasyScan Go in parasite detection and species identification accuracy fulfil WHO-TDR Research Malaria Microscopy competence level 2 criteria. In terms of parasite quantification and false positive rate, it meets the level 4 WHO-TDR Research Malaria Microscopy criteria. All performance parameters were significantly affected by slide quality. Further software improvement is required to improve sensitivity at low parasitaemia and parasite density estimations. Trial registration ClinicalTrials.gov number NCT03512678.

KW - Artificial intelligence

KW - Diagnostic accuracy

KW - Digital microscopy

KW - Light microscopy

KW - Machine-learning

KW - Malaria

UR - http://www.scopus.com/inward/record.url?scp=85128011870&partnerID=8YFLogxK

U2 - 10.1186/s12936-022-04146-1

DO - 10.1186/s12936-022-04146-1

M3 - Article

C2 - 35413904

AN - SCOPUS:85128011870

SN - 1475-2875

VL - 21

SP - 1

EP - 12

JO - Malaria Journal

JF - Malaria Journal

IS - 1

M1 - 122

ER -

Field evaluation of the diagnostic performance of EasyScan GO: A digital malaria microscopy device based on machine-learning

Abstract

Bibliographical note

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this