Lexical and Morphological Statistics of an Arabic POS-Tagged Corpus | ||||
| The Egyptian Journal of Language Engineering | ||||
| Article 3, Volume 1, Issue 1, January 2014, Page 24-41 PDF (790.42 K) | ||||
| Document Type: Original Article | ||||
| DOI: 10.21608/ejle.2014.59857 | ||||
 View on SCiNiTO | ||||
| Authors | ||||
Hamdy Mubarak  1; Kareem Shaban1; Mohamed Forat2
| ||||
| 1Arabic NLP Researche and Development Department, Sakhr Software Co. | ||||
| 2Arabic NLP Research and Development Department, Sakhr Software Co. | ||||
| Abstract | ||||
| Part-Of-Speech (POS) tagging is a basic component necessary for many Natural Language Processing (NLP) applications. Building a manually tagged corpus helps in studying key statistics of a given language which form the basis for POS tagging systems. In this paper, we present both lexical and morphological statistics for Arabic that are derived from the Sakhr’s POS manually tagged corpus. It covers text (7 M words) from a wide range of Arab countries in different domains over the years 2002-2004. The derived statistics are used as heuristics and preferential rules within a statistical Diacritizer which achieves a high accuracy in stem diacritization and POS disambiguation. Statistics includes information related to sentence and word lengths, punctuation marks, distribution of Arabic letters and diacritics, in addition to lexical and morphological information for POS distribution, stems, prefixes, suffixes, roots, morphological patterns, and morphosyntactic features like gender, number, person, and case ending. Modern Standard Arabic (MSA) is studied by analyzing the coverage of stems, roots, morphological patterns, prefixes, and suffixes. Comparisons with an arbitrary English corpus are shown in applicable cases. | ||||
| Keywords | ||||
| Corpus Statistics; Arabic NLP; POS Tagging; Diacritization; MSA | ||||
|
Statistics Article View: 108 PDF Download: 326 |
||||
