A novel Data and Model Centric artificial intelligence based approach in developing high-performance Named Entity Recognition for Bengali Language

Khadija Akter Lima, Khan Md Hasib, Sami Azam, Asif Karim, Sidratul Montaha, Sheak Rashed Haider Noori, Mirjam Jonkman

Research output: Contribution to journalArticlepeer-review

66 Downloads (Pure)

Abstract

Named Entity Recognition (NER) plays a significant role in enhancing the performance of all types of domain specific applications in Natural Language Processing (NLP). According to the type of application, the goal of NER is to identify target entities based on the context of other existing entities in a sentence. Numerous architectures have demonstrated good performance for high-resource languages such as English and Chinese NER. However, currently existing NER models for Bengali could not achieve reliable accuracy due to morphological richness of Bengali and limited availability of resources. This work integrates both Data and Model Centric AI concepts to achieve a state-of-the-art performance. A unique dataset was created for this study demonstrating the impact of a good quality dataset on accuracy. We proposed a method for developing a high quality NER dataset for any language. We have used our dataset to evaluate the performance of various Deep Learning models. A hybrid model performed with the exact match F1 score of 87.50%, partial match F1 score of 92.31%, and micro F1 score of 98.32%. Our proposed model reduces the need for feature engineering and utilizes minimal resources.

Original languageEnglish
Article numbere0287818
Pages (from-to)1-36
Number of pages36
JournalPLoS One
Volume18
Issue number9 September
DOIs
Publication statusPublished - 22 Sept 2023

Fingerprint

Dive into the research topics of 'A novel Data and Model Centric artificial intelligence based approach in developing high-performance Named Entity Recognition for Bengali Language'. Together they form a unique fingerprint.

Cite this