*Improving Natural Language Queries Search and Retrieval through Semantic Image Annotation Understanding
Samih, H., Rady, S., Ismail, M., Gharib, T. (2021). *Improving Natural Language Queries Search and Retrieval through Semantic Image Annotation Understanding. EKB Journal Management System, 20(2), 67-78. doi: 10.21608/ijicis.2021.50161.1041
Haitham Samih; Sherine Rady; Manal Ismail; Tarek Gharib. "*Improving Natural Language Queries Search and Retrieval through Semantic Image Annotation Understanding". EKB Journal Management System, 20, 2, 2021, 67-78. doi: 10.21608/ijicis.2021.50161.1041
Samih, H., Rady, S., Ismail, M., Gharib, T. (2021). '*Improving Natural Language Queries Search and Retrieval through Semantic Image Annotation Understanding', EKB Journal Management System, 20(2), pp. 67-78. doi: 10.21608/ijicis.2021.50161.1041
Samih, H., Rady, S., Ismail, M., Gharib, T. *Improving Natural Language Queries Search and Retrieval through Semantic Image Annotation Understanding. EKB Journal Management System, 2021; 20(2): 67-78. doi: 10.21608/ijicis.2021.50161.1041

*Improving Natural Language Queries Search and Retrieval through Semantic Image Annotation Understanding

International Journal of Intelligent Computing and Information Sciences
Article 12, Volume 20, Issue 2, December 2020, Page 67-78  XML PDF (627.58 K)
Document Type: Original Article
DOI: 10.21608/ijicis.2021.50161.1041
View on SCiNiTO View on SCiNiTO
Authors
Haitham Samih email 1; Sherine Radyorcid 2; Manal Ismail3; Tarek Ghariborcid 4
1Faculty of Computer and Information Technology, Egyptian E-Learning University, Cairo, Egypt
2Ain Shams University
3Computer Engineering and Systems Department, Faculty of Engineering, Helwan University, Cairo, Egypt
4Information Systems Department, Faculty of Computer and Information Sciences, Ain Shams University, Cairo, Egypt
Abstract
Retrieving images using detailed natural language queries remains a difficult challenge. Traditional annotation-based image retrieval systems using word matching techniques cannot efficiently support such query types. Significant improvements for this problem can be achieved with a semantic understanding for those query sentences and image annotations. This paper presents a two-stage semantic understanding approach for natural language query sentences. At the first stage, the Stanford parser and a designed rule-based relation extraction tool are used in triple extraction process to efficiently extract the objects attributes, instances and natural language annotations relationships involving these objects. The second stage integrates the extracted relations with external commonsense knowledge source in a mapping process to provide high-level semantic meanings to the extracted triples. Experiments are conducted for evaluating the benefit of the proposed semantic understanding against a testing set of natural language sentences from the Flickr8k dataset. The results show that the proposed approach succeeds to extract relational triples with average accuracy value of 97% for the different types of annotations relationships: attributes and instance relations, multiword dependence relations, and semantic relations.
Keywords
Image retrieval; natural language queries; semantic understanding; commonsense knowledge sources
Statistics
Article View: 331
PDF Download: 355