A Distant Supervision Method based on Paradigmatic Relations for Learning Word Embeddings

Li, Jianquan, Hu, Renfen, Liu, Xiaokang, Tiwari, Prayag, Pandey, Hari, Chen, Wei, Jing, Yaohong and Yang, Kaicheng (2019) A Distant Supervision Method based on Paradigmatic Relations for Learning Word Embeddings. Neural Computing and Applications. ISSN 0941-0643 (In Press)

[img] Text
Paper.pdf - Accepted Version
Restricted to Repository staff only until 30 January 2020.
Available under License Creative Commons Attribution No Derivatives.

Download (22MB) | Request a copy

Abstract

Word embeddings learned on external resources have succeeded in improving many NLP tasks. However, existing embedding models still face challenges in situations where fine-gained semantic information is required, e.g. distinguishing antonyms from synonyms. In this paper, a distant supervision method is proposed to guide the training process by introducing semantic knowledge in a thesaurus. Specifically, the proposed model shortens the distance between target word and its synonyms by controlling the movements of them in both unidirectional and bidirectional, yielding three different models, namely, Unidirectional Movement of Target Model (UMT), Unidirectional Movement of Synonyms Model (UMS) and Bidirectional Movement of Target and Synonyms Model (BMTS). Extensive computational experiments have been conducted and results are collected for analysis purpose. The results show that the proposed models not only efficiently capture semantic information of antonyms but also achieve significant improvements in both intrinsic and extrinsic evaluation tasks. To validate the performance of the proposed models (UMT, UMS and BMTS), results are compared against well known models, namely, Skip-gram, JointRCM, WE-TD and dict2vec. The performances of the proposed models are evaluated on 4-tasks (benchmarks): word analogy (intrinsic), synonym-antonym detection (intrinsic), sentence matching (extrinsic) and text classification (extrinsic). A case study is provided to illustrate the working of the proposed models in an effective manner. Overall, a distant supervision method based on paradigmatic relations is proposed for learning word embeddings and it outperformed when compared against other existing models.

Item Type: Article
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: Computing and Information Systems
Date Deposited: 14 Feb 2019 14:26
URI: http://repository.edgehill.ac.uk/id/eprint/11034

Archive staff only

Item control page Item control page