Graph neural networks: A review of methods and applications
2020
Lots of learning tasks require dealing with graph data which contains rich relation information among elements. Modeling physics systems, learning molecular fingerprints, predicting protein interface,...
Share article
Pre-Trained Models: Past, Present and Future
Available online 26 August 2021
Large-scale pre-trained models (PTMs) such as BERT and GPT have recently achieved great success and become a milestone in the field of artificial intelligence (AI). Owing to sophisticated pre-training...
Share article
Neural machine translation: A review of methods, resources, and tools
2020
Machine translation (MT) is an important sub-field of natural language processing that aims to translate natural languages using computers. In recent years, end-to-end neural machine translation (NMT)...
Share article
A comprehensive survey of entity alignment for knowledge graphs
2021
Knowledge Graphs (KGs), as a structured human knowledge, manage data in an ease-of-store, recognizable, and understandable way for machines and provide a rich knowledge base for different artificial...
Share article
Extracting Events and Their Relations from Texts: A Survey on Recent Research Progress and Challenges
2020
Event is a common but non-negligible knowledge type. How to identify events from texts, extract their arguments, even analyze the relations between different events are important for many applications....
Share article
Heterogeneous graph knowledge enhanced stock market prediction
2021
We focus on the task of stock market prediction based on financial text which contains information that could influence the movement of stock market. Previous works mainly utilize a single semantic...
Share article
Neural, symbolic and neural-symbolic reasoning on knowledge graphs
2021
Knowledge graph reasoning is the fundamental component to support machine learning applications such as information extraction, information retrieval, and recommendation. Since knowledge graphs can...
Share article
Advances and challenges in conversational recommender systems: A survey
2021
Recommender systems exploit interaction history to estimate user preference, having been heavily used in a wide range of industry applications. However, static recommendation models are difficult to...
Share article
Lawformer: A pre-trained language model for Chinese legal long documents
2021
Legal artificial intelligence (LegalAI) aims to benefit legal systems with the technology of artificial intelligence, especially natural language processing (NLP). Recently, inspired by the success...
Share article
CPM: A large-scale generative Chinese Pre-trained language model
2021
Pre-trained Language Models (PLMs) have proven to be beneficial for various downstream NLP tasks. Recently, GPT-3, with 175 billion parameters and 570 GB training data, drew a lot of attention due to...
Share article
Discrete and continuous representations and processing in deep learning: Looking forward
2021
Discrete and continuous representations of content (e.g., of language or images) have interesting properties to be explored for the understanding of or reasoning with this content by machines. This...
Share article
A comprehensive review on resolving ambiguities in natural language processing
2021
Natural language processing is a known technology behind the development of some widely known AI assistants such as: SIRI, Natasha, and Watson. However, NLP is a diverse technology used for numerous...
Share article
WuDaoCorpora: A super large-scale Chinese corpora for pre-training language models
2021
Using large-scale training data to build a pre-trained language model (PLM) with a larger volume of parameters can significantly improve downstream tasks. For example, OpenAI trained the GPT3 model...
Share article
AI-driven drug discovery: A boon against COVID-19?
2020
The COVID-19 is an issue of international concern and threat to public health and there is an urgent need of drug/vaccine design. There is no vaccine or specific drug yet made as of July 23, 2020, for...
Share article
Robustness of deep learning models on graphs: A survey
2021
Machine learning (ML) technologies have achieved significant success in various downstream tasks, e.g., node classification, link prediction, community detection, graph classification and graph clustering....
Share article
CokeBERT: Contextual knowledge selection and embedding towards enhanced pre-trained language models
2021
Several recent efforts have been devoted to enhancing pre-trained language models (PLMs) by utilizing extra heterogeneous knowledge in knowledge graphs (KGs), and achieved consistent improvements on...
Share article
Rule-based data augmentation for knowledge graph embedding
2021
Knowledge graph (KG) embedding models suffer from the incompleteness issue of observed facts. Different from existing solutions that incorporate additional information or employ expressive and complex...
Share article
Share article
Know what you don't need: Single-Shot Meta-Pruning for attention heads
2021
Deep pre-trained Transformer models have achieved state-of-the-art results over a variety of natural language processing (NLP) tasks. By learning rich language knowledge with millions of parameters,...
Share article
The road from MLE to EM to VAE: A brief tutorial
Available online 28 October 2021
Variational Auto-Encoders (VAEs) have emerged as one of the most popular genres of generative models, which are learned to characterize the data distribution. The classic Expectation Maximization (EM)...
Share article
A review of deep learning in question answering over knowledge bases
2021
Question answering over knowledge bases (KBQA) is a challenging task in natural language processing. It requires machines to answer natural language questions based on large-scale knowledge bases. Recent...
Share article
The information propagation model of Weibo network based on spiking neural p systems
2021
Information propagation models in the Weibo network play a primary role in analyzing user behaviors, obtaining the propagation paths, determining the opinion leaders, and discovering the hot spots of...
Share article
User behavior modeling for Web search evaluation
2020
Search engines are widely used in our daily life. Batch evaluation of the performance of search systems to their users has always been an essential issue in the field of information retrieval. However,...
Share article
Incorporating bidirectional interactive information and regional features for relational facts extraction
2021
Extracting entity and relation jointly is often complicated since the relational triplets may be overlapped. In this paper, we propose a novel unified joint extraction model that considers the significant...
Share article
Structure-enhanced meta-learning for few-shot graph classification
2021
Graph classification is a highly impactful task that plays a crucial role in a myriad of real-world applications such as molecular property prediction and protein function prediction. Aiming to handle...
Share article