NSL OpenIR  > 中国科学院文献情报中心(北京)  > 信息系统部
Masked Sentence Model Based on BERT for Move Recognition in Medical Scientific Abstracts
Yu GH(于改红)
2019-11
Source PublicationJournal of Data and Information Science
Volume4Issue:4Pages:42-55
Abstract

Purpose: Move recognition in scientific abstracts is an NLP task of classifying sentences of the abstracts into different types of language units. To improve the performance of move recognition in scientific abstracts, a novel model of move recognition is proposed that outperforms the BERT-based method.

Design/methodology/approach: Prevalent models based on BERT for sentence classification often classify sentences without considering the context of the sentences. In this paper, inspired by the BERT masked language model (MLM), we propose a novel model called the masked sentence model that integrates the content and contextual information of the sentences in move recognition. Experiments are conducted on the benchmark dataset PubMed 20K RCT in three steps. Then, we compare our model with HSLN-RNN, BERT-based and SciBERT using the same dataset.

Findings: Compared with the BERT-based and SciBERT models, the F1 score of our model outperforms them by 4.96% and 4.34%, respectively, which shows the feasibility and effectiveness of the novel model and the result of our model comes closest to the state-of-the-art results of HSLN-RNN at present.

Research limitations: The sequential features of move labels are not considered, which might be one of the reasons why HSLN-RNN has better performance. Our model is restricted to dealing with biomedical English literature because we use a dataset from PubMed, which is a typical biomedical database, to fine-tune our model.

Practical implications: The proposed model is better and simpler in identifying move structures in scientific abstracts and is worthy of text classification experiments for capturing contextual features of sentences.

Originality/value: The study proposes a masked sentence model based on BERT that considers the contextual features of the sentences in abstracts in a new way. The performance of this classification model is significantly improved by rebuilding the input layer without changing the structure of neural networks.

KeywordMove Recognition Bert Masked Sentence Model Scientific Abstracts
MOST Discipline Catalogue管理学::图书情报与档案管理
DOI10.2478/jdis-2019-0020
URL查看原文
Indexed ByCSSCI
Language英语
Citation statistics
Document Type期刊论文
Identifierhttp://ir.las.ac.cn/handle/12502/11536
Collection中国科学院文献情报中心(北京)_信息系统部
Affiliation1.中国科学院文献情报中心
2.中国科学院大学
Recommended Citation
GB/T 7714
Yu GH. Masked Sentence Model Based on BERT for Move Recognition in Medical Scientific Abstracts[J]. Journal of Data and Information Science,2019,4(4):42-55.
APA Yu GH.(2019).Masked Sentence Model Based on BERT for Move Recognition in Medical Scientific Abstracts.Journal of Data and Information Science,4(4),42-55.
MLA Yu GH."Masked Sentence Model Based on BERT for Move Recognition in Medical Scientific Abstracts".Journal of Data and Information Science 4.4(2019):42-55.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Yu GH(于改红)]'s Articles
Baidu academic
Similar articles in Baidu academic
[Yu GH(于改红)]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Yu GH(于改红)]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.