• 基于文本生成技术的历史古籍事件识别模型构建研究

    分类: 图书馆学、情报学 >> 情报过程自动化的方法和设备 提交时间: 2022-08-31

    摘要: 目的 对比序列标注方法和文本生成方法在历史古籍事件识别上的表现,以构建历史古籍事件识别模型。 方法 本文选取《三国志》为原始语料,序列标注实验对《三国志》事件数据集进行BMES标注,构建BBCN-SG模型,文本生成实验构建T5-SG模型,对比两种方法的表现。又构建RoBERTa-SG、NEZHA-SG模型展开生成模型的对比实验。结合三个文本生成模型,融入Stacking集成学习的思想,构建Stacking-TRN-SG模型。 结果 在历史古籍事件识别建模问题上,文本生成方法的表现明显优于序列标注方法。而在文本生成方法中,三个模型表现则是RoBERTa-SG > T5-SG > NEZHA-SG。Stacking集成学习大大提高了生成模型的识别效果。 局限 本文计算资源有限,Stacking-TRN-SG模型缺少在其他历史古籍语料中的应用研究。 结论 本文构建的Stacking-TRN-SG模型初步实现历史古籍的自动事件识别。

  • Masked Sentence Model based on BERT for Move Recognition in Medical Scientific Abstracts

    分类: 计算机科学 >> 自然语言理解与机器翻译 分类: 图书馆学、情报学 >> 情报过程自动化的方法和设备 提交时间: 2019-10-29

    摘要:Purpose: Move recognition in scientific abstracts is an NLP task of classifying sentences of the abstracts into different types of language unit. To improve the performance of move recognition in scientific abstracts, a novel model of move recognition is proposed that outperforms BERT-Base method. Design: Prevalent models based on BERT for sentence classification often classify sentences without considering the context of the sentences. In this paper, inspired by the BERT's Masked Language Model (MLM), we propose a novel model called Masked Sentence Model that integrates the content and contextual information of the sentences in move recognition. Experiments are conducted on the benchmark dataset PubMed 20K RCT in three steps. And then compare our model with HSLN-RNN, BERT-Base and SciBERT using the same dataset. Findings: Compared with BERT-Base and SciBERT model, the F1 score of our model outperforms them by 4.96% and 4.34% respectively, which shows the feasibility and effectiveness of the novel model and the result of our model comes closest to the state-of-the-art results of HSLN-RNN at present. Research Limitations: The sequential features of move labels are not considered, which might be one of the reasons why HSLN-RNN has better performance. And our model is restricted to dealing with bio-medical English literature because we use dataset from PubMed which is a typical bio-medical database to fine-tune our model. Practical implications: The proposed model is better and simpler in identifying move structure in scientific abstracts, and is worthy for text classification experiments to capture contextual features of sentences. Originality: The study proposes a Masked Sentence Model based on BERT which takes account of the contextual features of the sentences in abstracts in a new way. And the performance of this classification model is significantly improved by rebuilding the input layer without changing the structure of neural networks.