00057-读的 KGE 论文--PDF版本-windows10

前言

知识图谱嵌入（Knowledge Graph Embedding, KGE）是一种将实体和关系表示成低维稠密向量的技术，进而 KG 被建模成低维向量空间，在这个向量空间内，头实体 $h$ 和尾实体 $t$ 是一个单独的向量，每个关系 $r$ 是头实体 $h$ 和尾实体 $t$ 之间的一个运算操作。

记录读过的 PDF 版本的 KGE 论文.

操作系统：Windows 10 专业版

RESCAL

论文: A Three-Way Model for Collective Learning on Multi-Relational Data .

Abstract

Relational learning is becoming increasingly important in many areas of application. Here, we present a novel approach to relational learning based on the factorization of a three-way tensor. We show that unlike other tensor approaches, our method is able to perform collective learning via the latent components of the model and provide an efficient algorithm to compute the factorization. We substantiate our theoretical considerations regarding the collective learning capabilities of our model by the means of experiments on both a new dataset and a dataset commonly used in entity resolution. Furthermore, we show on common benchmark datasets that our approach achieves better or on-par results, if compared to current state-of-the-art relational learning solutions, while it is significantly faster to compute.

论文地址: http://www.icml-2011.org/papers.php .

PDF 地址:

@InProceedings{ICML2011Nickel_438,
  author =    {Maximilian Nickel and Volker Tresp and Hans-Peter Kriegel},
  title =     {A Three-Way Model for Collective Learning on Multi-Relational Data},
  booktitle = {Proceedings of the 28th International Conference on Machine Learning (ICML-11)},
  series =    {ICML '11},
  year =      {2011},
  editor =    {Lise Getoor and Tobias Scheffer},
  location =  {Bellevue, Washington, USA},
  isbn =      {978-1-4503-0619-5},
  month =     {June},
  publisher = {ACM},
  address =   {New York, NY, USA},
  pages=      {809--816},
}

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/09-RESCAL.pdf .

TransE

论文: Translating Embeddings for Modeling Multi-relational Data .

Abstract

We consider the problem of embedding entities and relationships of multi-relational data in low-dimensional vector spaces. Our objective is to propose a canonical model which is easy to train, contains a reduced number of parameters and can scale up to very large databases. Hence, we propose, TransE, a method which models relationships by interpreting them as translations operating on the low-dimensional embeddings of the entities. Despite its simplicity, this assumption proves to be powerful since extensive experiments show that TransE significantly outperforms state-of-the-art methods in link prediction on two knowledge bases. Besides, it can be successfully trained on a large scale data set with 1M entities, 25k relationships and more than 17M training samples.

论文地址: https://proceedings.neurips.cc/paper_files/paper/2013/hash/1cecc7a77928ca8133fa24680a88d2f9-Abstract.html .

PDF 地址: https://proceedings.neurips.cc/paper_files/paper/2013/file/1cecc7a77928ca8133fa24680a88d2f9-Paper.pdf .

@inproceedings{NIPS2013_1cecc7a7,
 author = {Bordes, Antoine and Usunier, Nicolas and Garcia-Duran, Alberto and Weston, Jason and Yakhnenko, Oksana},
 booktitle = {Advances in Neural Information Processing Systems},
 editor = {C.J. Burges and L. Bottou and M. Welling and Z. Ghahramani and K.Q. Weinberger},
 pages = {},
 publisher = {Curran Associates, Inc.},
 title = {Translating Embeddings for Modeling Multi-relational Data},
 url = {https://proceedings.neurips.cc/paper_files/paper/2013/file/1cecc7a77928ca8133fa24680a88d2f9-Paper.pdf},
 volume = {26},
 year = {2013}
}

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/06-TransE.pdf .

TransH

论文: Knowledge Graph Embedding by Translating on Hyperplanes .

Abstract

We deal with embedding a large scale knowledge graph composed of entities and relations into a continuous vector space. TransE is a promising method proposed recently, which is very efficient while achieving state-of-the-art predictive performance. We discuss some mapping properties of relations which should be considered in embedding, such as reflexive, one-to-many, many-to-one, and many-to-many. We note that TransE does not do well in dealing with these properties. Some complex models are capable of preserving these mapping properties but sacrifice efficiency in the process. To make a good trade-off between model capacity and efficiency, in this paper we propose TransH which models a relation as a hyperplane together with a translation operation on it. In this way, we can well preserve the above mapping properties of relations with almost the same model complexity of TransE. Additionally, as a practical knowledge graph is often far from completed, how to construct negative examples to reduce false negative labels in training is very important. Utilizing the one-to-many/many-to-one mapping property of a relation, we propose a simple trick to reduce the possibility of false negative labeling. We conduct extensive experiments on link prediction, triplet classification and fact extraction on benchmark datasets like WordNet and Freebase. Experiments show TransH delivers significant improvements over TransE on predictive accuracy with comparable capability to scale up.

论文地址: https://ojs.aaai.org/index.php/AAAI/article/view/8870 .

PDF 地址: https://ojs.aaai.org/index.php/AAAI/article/view/8870/8729 .

Wang, Z., Zhang, J., Feng, J., & Chen, Z. (2014). Knowledge Graph Embedding by Translating on Hyperplanes. Proceedings of the AAAI Conference on Artificial Intelligence, 28(1). https://doi.org/10.1609/aaai.v28i1.8870

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/07-TransH.pdf .

DistMult

论文: Embedding Entities and Relations for Learning and Inference in Knowledge Bases .

Abstract

We consider learning representations of entities and relations in KBs using the neural-embedding approach. We show that most existing models, including NTN (Socher et al., 2013) and TransE (Bordes et al., 2013b), can be generalized under a unified learning framework, where entities are low-dimensional vectors learned from a neural network and relations are bilinear and/or linear mapping functions. Under this framework, we compare a variety of embedding models on the link prediction task. We show that a simple bilinear formulation achieves new state-of-the-art results for the task (achieving a top-10 accuracy of 73.2% vs. 54.7% by TransE on Freebase). Furthermore, we introduce a novel approach that utilizes the learned relation embeddings to mine logical rules such as “BornInCity(a,b) and CityInCountry(b,c) => Nationality(a,c)”. We find that embeddings learned from the bilinear objective are particularly good at capturing relational semantics and that the composition of relations is characterized by matrix multiplication. More interestingly, we demonstrate that our embedding-based rule extraction approach successfully outperforms a state-of-the-art confidence-based rule mining approach in mining Horn rules that involve compositional reasoning.

论文地址: https://arxiv.org/abs/1412.6575 .

PDF 地址: https://arxiv.org/pdf/1412.6575.pdf .

@misc{yang2015embedding,
      title={Embedding Entities and Relations for Learning and Inference in Knowledge Bases}, 
      author={Bishan Yang and Wen-tau Yih and Xiaodong He and Jianfeng Gao and Li Deng},
      year={2015},
      eprint={1412.6575},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Yang BS，Yih S，He XD，et al．Embedding entities and relations for learning and inference in knowledge bases[C]//Proceedings of ICLR．2015．http://arxiv.org/abs/1412.6575．

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/04-DistMult.pdf .

TransR

论文: Learning Entity and Relation Embeddings for Knowledge Graph Completion .

Abstract

Knowledge graph completion aims to perform link prediction between entities. In this paper, we consider the approach of knowledge graph embeddings. Recently, models such as TransE and TransH build entity and relation embeddings by regarding a relation as translation from head entity to tail entity. We note that these models simply put both entities and relations within the same semantic space. In fact, an entity may have multiple aspects and various relations may focus on different aspects of entities, which makes a common space insufficient for modeling. In this paper, we propose TransR to build entity and relation embeddings in separate entity space and relation spaces. Afterwards, we learn embeddings by first projecting entities from entity space to corresponding relation space and then building translations between projected entities. In experiments, we evaluate our models on three tasks including link prediction, triple classification and relational fact extraction. Experimental results show significant and consistent improvements compared to state-of-the-art baselines including TransE and TransH. The source code of this paper can be obtained from https://github.com/mrlyk423/relation_extraction.

论文地址: https://ojs.aaai.org/index.php/AAAI/article/view/9491 .

PDF 地址: https://ojs.aaai.org/index.php/AAAI/article/view/9491/9350 .

Lin, Y., Liu, Z., Sun, M., Liu, Y., & Zhu, X. (2015). Learning Entity and Relation Embeddings for Knowledge Graph Completion. Proceedings of the AAAI Conference on Artificial Intelligence, 29(1). https://doi.org/10.1609/aaai.v29i1.9491

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/03-TransR.pdf .

TransD

论文: Knowledge Graph Embedding via Dynamic Mapping Matrix .

Abstract

Knowledge graphs are useful resources for numerous AI applications, but they are far from completeness. Previous work such as TransE, TransH and TransR/CTransR regard a relation as translation from head entity to tail entity and the CTransR achieves state-of-the-art performance. In this paper, we propose a more fine-grained model named TransD, which is an improvement of TransR/CTransR. In TransD, we use two vectors to represent a named symbol object (entity and relation). The first one represents the meaning of a(n) entity (relation), the other one is used to construct mapping matrix dynamically. Compared with TransR/CTransR, TransD not only considers the diversity of relations, but also entities. TransD has less parameters and has no matrix-vector multiplication operations, which makes it can be applied on large scale graphs. In Experiments, we evaluate our model on two typical tasks including triplets classification and link prediction. Evaluation results show that our approach outperforms state-of-the-art methods.

论文地址: https://aclanthology.org/P15-1067/ .

PDF 地址: https://aclanthology.org/P15-1067.pdf .

@inproceedings{ji-etal-2015-knowledge,
    title = "Knowledge Graph Embedding via Dynamic Mapping Matrix",
    author = "Ji, Guoliang  and
      He, Shizhu  and
      Xu, Liheng  and
      Liu, Kang  and
      Zhao, Jun",
    booktitle = "Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)",
    month = jul,
    year = "2015",
    address = "Beijing, China",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/P15-1067",
    doi = "10.3115/v1/P15-1067",
    pages = "687--696",
}

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/08-TransD.pdf .

HolE

论文: Holographic Embeddings of Knowledge Graphs .

Abstract

Learning embeddings of entities and relations is an efficient and versatile method to perform machine learning on relational data such as knowledge graphs. In this work, we propose holographic embeddings (HolE) to learn compositional vector space representations of entire knowledge graphs. The proposed method is related to holographic models of associative memory in that it employs circular correlation to create compositional representations. By using correlation as the compositional operator, HolE can capture rich interactions but simultaneously remains efficient to compute, easy to train, and scalable to very large datasets. Experimentally, we show that holographic embeddings are able to outperform state-of-the-art methods for link prediction on knowledge graphs and relational learning benchmark datasets.

论文地址: https://ojs.aaai.org/index.php/AAAI/article/view/10314 .

PDF 地址: https://ojs.aaai.org/index.php/AAAI/article/view/10314/10173 .

Nickel, M., Rosasco, L., & Poggio, T. (2016). Holographic Embeddings of Knowledge Graphs. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1). https://doi.org/10.1609/aaai.v30i1.10314

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/12-HolE.pdf .

ComplEx

论文: Complex Embeddings for Simple Link Prediction .

Abstract

In statistical relational learning, the link prediction problem is key to automatically understand the structure of large knowledge bases. As in previous studies, we propose to solve this problem through latent factorization. However, here we make use of complex valued embeddings. The composition of complex embeddings can handle a large variety of binary relations, among them symmetric and antisymmetric relations. Compared to state-of-the-art models such as Neural Tensor Network and Holographic Embeddings, our approach based on complex embeddings is arguably simpler, as it only uses the Hermitian dot product, the complex counterpart of the standard dot product between real vectors. Our approach is scalable to large datasets as it remains linear in both space and time, while consistently outperforming alternative approaches on standard link prediction benchmarks.

论文地址: https://arxiv.org/abs/1606.06357 .

PDF 地址: https://arxiv.org/pdf/1606.06357.pdf .

Complex Embeddings for Simple Link Prediction, Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier and Guillaume Bouchard, ICML 2016.

@inproceedings{trouillon2016complex,
	title = {{Complex embeddings for simple link prediction}},
	author = {Trouillon, Th\'eo and Welbl, Johannes and Riedel, Sebastian and Gaussier, \'Eric and Bouchard, Guillaume},
	booktitle = {International Conference on Machine Learning (ICML)},
	volume={48},
	pages={2071--2080},
	year = {2016}
}

@misc{trouillon2016complex,
      title={Complex Embeddings for Simple Link Prediction}, 
      author={Théo Trouillon and Johannes Welbl and Sebastian Riedel and Éric Gaussier and Guillaume Bouchard},
      year={2016},
      eprint={1606.06357},
      archivePrefix={arXiv},
      primaryClass={cs.AI}
}

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/05-ComplEx.pdf .

R-GCN

论文: Modeling Relational Data with Graph Convolutional Networks .

Abstract

Knowledge graphs enable a wide variety of applications, including question answering and information retrieval. Despite the great effort invested in their creation and maintenance, even the largest (e.g., Yago, DBPedia or Wikidata) remain incomplete. We introduce Relational Graph Convolutional Networks (R-GCNs) and apply them to two standard knowledge base completion tasks: Link prediction (recovery of missing facts, i.e. subject-predicate-object triples) and entity classification (recovery of missing entity attributes). R-GCNs are related to a recent class of neural networks operating on graphs, and are developed specifically to deal with the highly multi-relational data characteristic of realistic knowledge bases. We demonstrate the effectiveness of R-GCNs as a stand-alone model for entity classification. We further show that factorization models for link prediction such as DistMult can be significantly improved by enriching them with an encoder model to accumulate evidence over multiple inference steps in the relational graph, demonstrating a large improvement of 29.8% on FB15k-237 over a decoder-only baseline.

论文地址:

https://arxiv.org/abs/1703.06103 .

PDF 地址:

https://arxiv.org/pdf/1703.06103.pdf .

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/23-R-GCN.pdf .

ANALOGY

论文: Analogical Inference for Multi-relational Embeddings .

Abstract

Large-scale multi-relational embedding refers to the task of learning the latent representations for entities and relations in large knowledge graphs. An effective and scalable solution for this problem is crucial for the true success of knowledge-based inference in a broad range of applications. This paper proposes a novel framework for optimizing the latent representations with respect to the analogical properties of the embedded entities and relations. By formulating the objective function in a differentiable fashion, our model enjoys both its theoretical power and computational scalability, and significantly outperformed a large number of representative baseline methods on benchmark datasets. Furthermore, the model offers an elegant unification of several well-known methods in multi-relational embedding, which can be proven to be special instantiations of our framework.

论文地址:

PDF 地址:

Hanxiao Liu, Yuexin Wu, Yiming Yang Proceedings of the 34th International Conference on Machine Learning, PMLR 70:2168-2178, 2017.


@InProceedings{pmlr-v70-liu17d,
  title = 	 {Analogical Inference for Multi-relational Embeddings},
  author =       {Hanxiao Liu and Yuexin Wu and Yiming Yang},
  booktitle = 	 {Proceedings of the 34th International Conference on Machine Learning},
  pages = 	 {2168--2178},
  year = 	 {2017},
  editor = 	 {Precup, Doina and Teh, Yee Whye},
  volume = 	 {70},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {06--11 Aug},
  publisher =    {PMLR},
  pdf = 	 {http://proceedings.mlr.press/v70/liu17d/liu17d.pdf},
  url = 	 {https://proceedings.mlr.press/v70/liu17d.html},
  abstract = 	 {Large-scale multi-relational embedding refers to the task of learning the latent representations for entities and relations in large knowledge graphs. An effective and scalable solution for this problem is crucial for the true success of knowledge-based inference in a broad range of applications. This paper proposes a novel framework for optimizing the latent representations with respect to the <em>analogical</em> properties of the embedded entities and relations. By formulating the objective function in a differentiable fashion, our model enjoys both its theoretical power and computational scalability, and significantly outperformed a large number of representative baseline methods on benchmark datasets. Furthermore, the model offers an elegant unification of several well-known methods in multi-relational embedding, which can be proven to be special instantiations of our framework.}
}

@misc{liu2017analogical,
      title={Analogical Inference for Multi-Relational Embeddings}, 
      author={Hanxiao Liu and Yuexin Wu and Yiming Yang},
      year={2017},
      eprint={1705.02426},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/11-ANALOGY.pdf .

SimplE

论文: SimplE Embedding for Link Prediction in Knowledge Graphs .

Abstract

Knowledge graphs contain knowledge about the world and provide a structured representation of this knowledge. Current knowledge graphs contain only a small subset of what is true in the world. Link prediction approaches aim at predicting new links for a knowledge graph given the existing links among the entities. Tensor factorization approaches have proved promising for such link prediction problems. Proposed in 1927, Canonical Polyadic (CP) decomposition is among the first tensor factorization approaches. CP generally performs poorly for link prediction as it learns two independent embedding vectors for each entity, whereas they are really tied. We present a simple enhancement of CP (which we call SimplE) to allow the two embeddings of each entity to be learned dependently. The complexity of SimplE grows linearly with the size of embeddings. The embeddings learned through SimplE are interpretable, and certain types of background knowledge can be incorporated into these embeddings through weight tying. We prove SimplE is fully expressive and derive a bound on the size of its embeddings for full expressivity. We show empirically that, despite its simplicity, SimplE outperforms several state-of-the-art tensor factorization techniques. SimplE’s code is available on GitHub at https://github.com/Mehran-k/SimplE.

论文地址: https://proceedings.neurips.cc/paper_files/paper/2018/hash/b2ab001909a8a6f04b51920306046ce5-Abstract.html .

PDF 地址:

@inproceedings{NEURIPS2018_b2ab0019,
 author = {Kazemi, Seyed Mehran and Poole, David},
 booktitle = {Advances in Neural Information Processing Systems},
 editor = {S. Bengio and H. Wallach and H. Larochelle and K. Grauman and N. Cesa-Bianchi and R. Garnett},
 pages = {},
 publisher = {Curran Associates, Inc.},
 title = {SimplE Embedding for Link Prediction in Knowledge Graphs},
 url = {https://proceedings.neurips.cc/paper_files/paper/2018/file/b2ab001909a8a6f04b51920306046ce5-Paper.pdf},
 volume = {31},
 year = {2018}
}

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/10-SimplE.pdf .

RotatE

论文: RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space .

Abstract

We study the problem of learning representations of entities and relations in knowledge graphs for predicting missing links. The success of such a task heavily relies on the ability of modeling and inferring the patterns of (or between) the relations. In this paper, we present a new approach for knowledge graph embedding called RotatE, which is able to model and infer various relation patterns including: symmetry/antisymmetry, inversion, and composition. Specifically, the RotatE model defines each relation as a rotation from the source entity to the target entity in the complex vector space. In addition, we propose a novel self-adversarial negative sampling technique for efficiently and effectively training the RotatE model. Experimental results on multiple benchmark knowledge graphs show that the proposed RotatE model is not only scalable, but also able to infer and model various relation patterns and significantly outperform existing state-of-the-art models for link prediction.

论文地址: https://openreview.net/forum?id=HkgEQnRqYQ .

PDF 地址: https://openreview.net/pdf?id=HkgEQnRqYQ .

@inproceedings{
sun2018rotate,
title={RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space},
author={Zhiqing Sun and Zhi-Hong Deng and Jian-Yun Nie and Jian Tang},
booktitle={International Conference on Learning Representations},
year={2019},
url={https://openreview.net/forum?id=HkgEQnRqYQ},
}

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/02-RotatE.pdf .

KBGAT

论文: Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs .

Abstract

The recent proliferation of knowledge graphs (KGs) coupled with incomplete or partial information, in the form of missing relations (links) between entities, has fueled a lot of research on knowledge base completion (also known as relation prediction). Several recent works suggest that convolutional neural network (CNN) based models generate richer and more expressive feature embeddings and hence also perform well on relation prediction. However, we observe that these KG embeddings treat triples independently and thus fail to cover the complex and hidden information that is inherently implicit in the local neighborhood surrounding a triple. To this effect, our paper proposes a novel attention-based feature embedding that captures both entity and relation features in any given entity’s neighborhood. Additionally, we also encapsulate relation clusters and multi-hop relations in our model. Our empirical study offers insights into the efficacy of our attention-based model and we show marked performance gains in comparison to state-of-the-art methods on all datasets.

论文地址: https://aclanthology.org/P19-1466/ .

PDF 地址: https://aclanthology.org/P19-1466.pdf .

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/24-KBGAT.pdf .

CompGCN

论文: Composition-based Multi-Relational Graph Convolutional Networks .

Abstract

Graph Convolutional Networks (GCNs) have recently been shown to be quite successful in modeling graph-structured data. However, the primary focus has been on handling simple undirected graphs. Multi-relational graphs are a more general and prevalent form of graphs where each edge has a label and direction associated with it. Most of the existing approaches to handle such graphs suffer from over-parameterization and are restricted to learning representations of nodes only. In this paper, we propose CompGCN, a novel Graph Convolutional framework which jointly embeds both nodes and relations in a relational graph. CompGCN leverages a variety of entity-relation composition operations from Knowledge Graph Embedding techniques and scales with the number of relations. It also generalizes several of the existing multi-relational GCN methods. We evaluate our proposed method on multiple tasks such as node classification, link prediction, and graph classification, and achieve demonstrably superior results. We make the source code of CompGCN available to foster reproducible research.

论文地址: https://openreview.net/forum?id=BylA_C4tPr .

PDF 地址: https://openreview.net/pdf?id=BylA_C4tPr .

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/25-CompGCN.pdf .

OpenKE

论文: OpenKE: An Open Toolkit for Knowledge Embedding .

Abstract

We release an open toolkit for knowledge embedding (OpenKE), which provides a unified framework and various fundamental models to embed knowledge graphs into a continuous low-dimensional space. OpenKE prioritizes operational efficiency to support quick model validation and large-scale knowledge representation learning. Meanwhile, OpenKE maintains sufficient modularity and extensibility to easily incorporate new models into the framework. Besides the toolkit, the embeddings of some existing large-scale knowledge graphs pre-trained by OpenKE are also available, which can be directly applied for many applications including information retrieval, personalized recommendation and question answering. The toolkit, documentation, and pre-trained embeddings are all released on http://openke.thunlp.org/.

论文地址: https://aclanthology.org/D18-2024/ .

PDF 地址: https://aclanthology.org/D18-2024.pdf .

@inproceedings{han-etal-2018-openke,
    title = "{O}pen{KE}: An Open Toolkit for Knowledge Embedding",
    author = "Han, Xu  and
      Cao, Shulin  and
      Lv, Xin  and
      Lin, Yankai  and
      Liu, Zhiyuan  and
      Sun, Maosong  and
      Li, Juanzi",
    booktitle = "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
    month = nov,
    year = "2018",
    address = "Brussels, Belgium",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/D18-2024",
    doi = "10.18653/v1/D18-2024",
    pages = "139--144",
    abstract = "We release an open toolkit for knowledge embedding (OpenKE), which provides a unified framework and various fundamental models to embed knowledge graphs into a continuous low-dimensional space. OpenKE prioritizes operational efficiency to support quick model validation and large-scale knowledge representation learning. Meanwhile, OpenKE maintains sufficient modularity and extensibility to easily incorporate new models into the framework. Besides the toolkit, the embeddings of some existing large-scale knowledge graphs pre-trained by OpenKE are also available, which can be directly applied for many applications including information retrieval, personalized recommendation and question answering. The toolkit, documentation, and pre-trained embeddings are all released on \url{http://openke.thunlp.org/}.",
}

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/01-OpenKE.pdf .

Pykg2vec

论文: Pykg2vec: A Python Library for Knowledge Graph Embedding .

Abstract

Pykg2vec is a Python library for learning the representations of the entities and relations in knowledge graphs. Pykg2vec’s flexible and modular software architecture currently implements 25 state-of-the-art knowledge graph embedding algorithms, and is designed to easily incorporate new algorithms.The goal of pykg2vec is to provide a practical and educational platform to accelerate research in knowledge graph representation learning. Pykg2vec is built on top of PyTorch and Python’s multiprocessing framework and provides modules for batch generation, Bayesian hyperparameter optimization, evaluation of KGE tasks, embedding, and result visualization. Pykg2vec is released under the MIT License and is also available in the Python Package Index (PyPI). The source code of pykg2vec is available at https://github.com/Sujit-O/pykg2vec.

论文地址: https://jmlr.org/papers/v22/19-433.html .

PDF 地址: https://jmlr.org/papers/volume22/19-433/19-433.pdf .

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/16-Pykg2vec.pdf .

PyKEEN 1.0

论文: PyKEEN 1.0: A Python Library for Training and Evaluating Knowledge Graph Embeddings .

Abstract

Recently, knowledge graph embeddings (KGEs) have received significant attention, and several software libraries have been developed for training and evaluation. While each of them addresses specific needs, we report on a community effort to a re-design and re-implementation of PyKEEN, one of the early KGE libraries. PyKEEN 1.0 enables users to compose knowledge graph embedding models based on a wide range of interaction models, training approaches, loss functions, and permits the explicit modeling of inverse relations. It allows users to measure each component’s influence individually on the model’s performance. Besides, an automatic memory optimization has been realized in order to optimally exploit the provided hardware. Through the integration of Optuna, extensive hyper-parameter optimization (HPO) functionalities are provided.

论文地址: https://jmlr.org/papers/v22/20-825.html .

PDF 地址: https://jmlr.org/papers/volume22/20-825/20-825.pdf .

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/15-PyKEEN.pdf .

CogKGE

论文: CogKGE: A Knowledge Graph Embedding Toolkit and Benchmark for Representing Multi-source and Heterogeneous Knowledge .

Abstract

In this paper, we propose CogKGE, a knowledge graph embedding (KGE) toolkit, which aims to represent multi-source and heterogeneous knowledge. For multi-source knowledge, unlike existing methods that mainly focus on entity-centric knowledge, CogKGE also supports the representations of event-centric, commonsense and linguistic knowledge. For heterogeneous knowledge, besides structured triple facts, CogKGE leverages additional unstructured information, such as text descriptions, node types and temporal information, to enhance the meaning of embeddings. Designing CogKGE aims to provide a unified programming framework for KGE tasks and a series of knowledge representations for downstream tasks. As a research framework, CogKGE consists of five parts, including core, data, model, knowledge and adapter module. As a knowledge discovery toolkit, CogKGE provides pre-trained embedders to discover new facts, cluster entities and check facts. Furthermore, we construct two benchmark datasets for further research on multi-source heterogeneous KGE tasks: EventKG240K and CogNet360K. We also release an online system to discover knowledge visually. Source code, datasets and pre-trained embeddings are publicly available at GitHub, with a short instruction video.

论文地址: https://aclanthology.org/2022.acl-demo.16/ .

PDF 地址: https://aclanthology.org/2022.acl-demo.16.pdf .

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/13-CogKGE.pdf .

NeuralKG

论文: NeuralKG: An Open Source Library for Diverse Representation Learning of Knowledge Graphs .

Abstract

NeuralKG is an open-source Python-based library for diverse representation learning of knowledge graphs. It implements three different series of Knowledge Graph Embedding (KGE) methods, including conventional KGEs, GNN-based KGEs, and Rule-based KGEs. With a unified framework, NeuralKG successfully reproduces link prediction results of these methods on benchmarks, freeing users from the laborious task of reimplementing them, especially for some methods originally written in non-python programming languages. Besides, NeuralKG is highly configurable and extensible. It provides various decoupled modules that can be mixed and adapted to each other. Thus with NeuralKG, developers and researchers can quickly implement their own designed models and obtain the optimal training methods to achieve the best performance efficiently. We built an website in this http URL to organize an open and shared KG representation learning community. The source code is all publicly released at this https URL.

论文地址: https://arxiv.org/abs/2202.12571 .

PDF 地址: https://arxiv.org/pdf/2202.12571.pdf .

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/14-NeuralKG.pdf .

中文综述-哈尔滨理工大学

论文: 知识图谱嵌入技术研究进展 .

Abstract

知识图谱嵌入（KGE）是知识图谱领域一个新的研究热点，旨在利用词向量的平移不变性将知识图谱中实体和关系嵌入到低维向量空间，进而完成知识表示。以解决实际问题的类型为划分依据，首先，阐述了四类主要的知识图谱嵌入方法，包括基于深度学习的方法、基于图形特征的方法、基于翻译模型的方法以及基于其他模型的方法，对每种模型的算法思想进行详细阐述，总结了每种模型的优缺点；其次，从常用数据集、评价指标、算法、实验四方面对知识图谱嵌入算法实验进行分析与归纳，对嵌入方法做了横纵向对比；最后，从解决实际问题的角度出发，给出了知识图谱嵌入技术未来的发展方向。通过研究，发现在基于深度学习的方法中，LCPE模型的效果最好；在基于图形特征的方法中，TCE模型的效果最好；在基于翻译模型的方法中，NTransGH模型的效果最好。今后的研究可以在LCPE、TCE、NTransGH的基础上进行拓展，不断提高链接预测和三元组分类的实验效果。

论文地址: http://fcst.ceaj.org/CN/10.3778/j.issn.1673-9418.2103086 .

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/17-zh-review.pdf .

中文综述-哈尔滨工业大学

论文: 面向知识图谱的图嵌入学习研究进展 .

Abstract

知识图谱是一种用网络结构存储知识的知识库, 在知识图谱中, 单条知识被表示成三元组的形式, 即(头实体, 关系, 尾实体). 得力于知识图谱在各个领域的广泛应用, 面向知识图谱的图嵌入学习也得到越来越多研究人员的关注. 面向知识图谱的图嵌入学习任务旨在为图谱中的实体与关系学习低维且稠密的向量, 通过图嵌入向量表达实体与关系的语义信息以及度量实体之间、关系之间、实体与关系之间的语义联系, 已有许多研究证明图嵌入模型在下游任务中的有效性. 近年来, 越来越多研究人员开始关注知识图谱的图嵌入学习, 并取得大量的研究成果, 尝试将图嵌入算法分成了基于转移思想、基于张量分解、基于传统深度学习模型、基于图神经网络以及融入额外信息的图嵌入学习共5大类, 梳理、介绍各类图嵌入算法的设计思路、算法特征以及优缺点, 以帮助指导初步接触该领域的研究人员快速学习了解该研究领域的相关模型和算法.

论文地址: http://www.jos.org.cn/jos/article/abstract/6426 .

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/18-zh-survey.pdf .

Review-University-of-North-Dakota

论文: A Review of Knowledge Graph Completion .

Abstract

Information extraction methods proved to be effective at triple extraction from structured or unstructured data. The organization of such triples in the form of (head entity, relation, tail entity) is called the construction of Knowledge Graphs (KGs). Most of the current knowledge graphs are incomplete. In order to use KGs in downstream tasks, it is desirable to predict missing links in KGs. Different approaches have been recently proposed for representation learning of KGs by embedding both entities and relations into a low-dimensional vector space aiming to predict unknown triples based on previously visited triples. According to how the triples will be treated independently or dependently, we divided the task of knowledge graph completion into conventional and graph neural network representation learning and we discuss them in more detail. In conventional approaches, each triple will be processed independently and in GNN-based approaches, triples also consider their local neighborhood.

论文地址: https://www.mdpi.com/2078-2489/13/8/396 .

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/19-en-review.pdf .

Review-GNN

论文: A Comprehensive Survey of Graph Neural Networks for Knowledge Graphs .

Abstract

The Knowledge graph, a multi-relational graph that represents rich factual information among entities of diverse classifications, has gradually become one of the critical tools for knowledge management. However, the existing knowledge graph still has some problems which form hot research topics in recent years. Numerous methods have been proposed based on various representation techniques. Graph Neural Network, a framework that uses deep learning to process graph-structured data directly, has significantly advanced the state-of-the-art in the past few years. This study firstly is aimed at providing a broad, complete as well as comprehensive overview of GNN-based technologies for solving four different KG tasks, including link prediction, knowledge graph alignment, knowledge graph reasoning, and node classification. Further, we also investigated the related artificial intelligence applications of knowledge graphs based on advanced GNN methods, such as recommender systems, question answering, and drug-drug interaction. This review will provide new insights for further study of KG and GNN.

论文地址: https://ieeexplore.ieee.org/abstract/document/9831453 .

论文 PDF 地址: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9831453 .

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/20-en-gnn-review.pdf .

Review-Northeastern-University

论文: A comprehensive overview of knowledge graph completion .

Abstract

Knowledge Graph (KG) provides high-quality structured knowledge for various downstream knowledge-aware tasks (such as recommendation and intelligent question-answering) with its unique advantages of representing and managing massive knowledge. The quality and completeness of KGs largely determine the effectiveness of the downstream tasks. But in view of the incomplete characteristics of KGs, there is still a large amount of valuable knowledge is missing from the KGs. Therefore, it is necessary to improve the existing KGs to supplement the missed knowledge. Knowledge Graph Completion (KGC) is one of the popular technologies for knowledge supplement. Accordingly, there has a growing concern over the KGC technologies. Recently, there have been lots of studies focusing on the KGC field. To investigate and serve as a helpful resource for researchers to grasp the main ideas and results of KGC studies, and further highlight ongoing research in KGC, in this paper, we provide a all-round up-to-date overview of the current state-of-the-art in KGC.

According to the information sources used in KGC methods, we divide the existing KGC methods into two main categories: the KGC methods relying on structural information and the KGC methods using other additional information. Further, each category is subdivided into different granularity for summarizing and comparing them. Besides, the other KGC methods for KGs of special fields (including temporal KGC, commonsense KGC, and hyper-relational KGC) are also introduced. In particular, we discuss comparisons and analyses for each category in our overview. Finally, some discussions and directions for future research are provided.

论文地址: https://www.sciencedirect.com/science/article/pii/S095070512200805X .

论文 PDF 地址: https://www.sciencedirect.com/science/article/pii/S095070512200805X/pdfft?md5=d0a6889250719442926d4f38a7cb2899&pid=1-s2.0-S095070512200805X-main.pdf .

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/21-en-review.pdf .

中文综述-东北大学

论文: 知识图谱嵌入技术研究综述 .

Abstract

知识图谱(KG)是一种用图模型来描述知识和建模事物之间关联关系的技术. 知识图谱嵌入(KGE)作为一种被广泛采用的知识表示方法, 其主要思想是将知识图谱中的实体和关系嵌入到连续的向量空间中, 用来简化操作, 同时保留KG的固有结构. 可以使得多种下游任务受益, 例如KG补全和关系提取等. 首先对现有的知识图谱嵌入技术进行全面回顾, 不仅包括使用KG中观察到的事实进行嵌入的技术, 还包括添加时间维度的动态KG嵌入方法, 以及融合多源信息的KG嵌入技术. 对相关模型从实体嵌入、关系嵌入、评分函数等方面进行分析、对比与总结. 然后简要介绍KG嵌入技术在下游任务中的典型应用, 包括问答系统、推荐系统和关系提取等. 最后阐述知识图谱嵌入面临的挑战, 对未来的研究方向进行展望.

论文地址: http://www.jos.org.cn/jos/article/abstract/6429 .

笔记 PDF 地址: https://cdn.jsdelivr.net/gh/LuYF-Lemon-love/susu-kge-papers/read/22-zh-review.pdf .