German English

FAst Multi-source Entity Resolution system (FAMER)

FAMER (FAst Multi-source Entity Resolution system) is a scalable framework for distributed multi-source entity resolution. It can construct similarity graphs for entities of multiple sources based on different linking schemes; existing links from the Web of Data could also be used to build the similarity graph. FAMER also provides several entity clustering schemes. They use the similarity graph to determine groups of matching entities aiming at maximizing the similarity between entities within a cluster and minimizing the similarity between entities of different clusters. Moreover, FAMER is able to repair clusters, e.g. that are overlapping and/or source-inconsistent.

FAMER is also able to perform the incremental matching process. The approach uses a so-called clustered similarity graph, i.e., a similarity graph reflecting already determined clusters. The input of the workflow is a stream of new entities from existing sources or from a new source plus the already determined clustered similarity graph from previous iterations. The incremental Linking and Clustering/Repairing part supports two general approaches for integrating the group of new entities into clusters. In the base (non-repairing) approach the new entities are either added to a similar existing cluster or they form a new cluster. A more sophisticated approach is able to repair existing clusters to achieve a better cluster assignment for new entities by reclustering a portion of the existing clustered graph. The output of incremental clustering is a fully clustered graph. The clusters can optionally be fused in the Fusion component so that all entities are represented by a single entity called cluster representative.

FAMER is implemented using Apache Flink™ so that the calculation of similarity graphs and the clustering approaches can be executed in parallel on clusters of variable size. We have also developed a visualization tool, SIMG-Viz to visually analyze the similarity graphs and clusters determined by FAMER.

Main Aims

  • Efficient parallel execution of match workflows in the cloud
  • Efficient application of clustering schemes for entity matching
  • Efficient methods for entity matching repairing
  • Efficient parallel execution for incremental linking and clustering.

Awards

Initial relaease and documentation

We provided an initial release and documentation of FAMER as well as a test dataset for a entity resolution/clustering challenge at the EDBT 2019 summer school on data integration. It is also a good starting point for using FAMER in other projects.

Source Code

Acknowledgements

This work is partially funded by the German Federal Ministry of Education and Research under project ScaDS Dresden/Leipzig (BMBF 01IS14014B).


Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI)

People

Research

Scientific / Student Assistants

  • Stefan Lerm

Tech and Developement Support Team

Former Colleagues and Assistants

  • Dr. M. Ali Rostami
  • Volodymyr Moroz

Selected Talks

DateTalkEventLanguage
Jun. 2020Incremental Multi-source Entity Resolution for Knowledge Graph Completion17th ESWC, Summer 2020en
Sep. 2019Entity Resolution for Large Scale Data Integration14th EDBT Summer School, Summer 2019en
Jun. 2018FAst Multi‐source Entity Resolution System6. Leipziger Semantic Web Tag, Summer 2018de

Publications

PDF

Google Scholar
publication iconSaeedi, Alieh; Peukert, Eric; Rahm, Erhard
Incremental Multi-source Entity Resolution for Knowledge Graph Completion
Proc. ESWC
2020-06
PDF
further information
Google Scholar
publication iconObraczka, D.; Saeedi, A.; Rahm, E.
Knowledge Graph Completion with FAMER
Proc. KDD workshop on Data Integration to Knowledge Graphs (DI2KG) (DI2KG Challenge Winner)
2019-08
PDF

Google Scholar
publication iconNentwig, Markus; Rahm, Erhard
Incremental Clustering on Linked Data
Proc. IEEE International Conference on Data Mining Workshop, ICDMW 2018, Singapore
2018-11
PDF
further information
Google Scholar
publication iconSaeedi, Alieh; Nentwig, Markus; Peukert, Eric; Rahm, Erhard
Scalable Matching and Clustering of Entities with FAMER
Complex Systems Informatics and Modeling Quarterly (CSIMQ), Issue 16, Sep./Oct. 2018, pp 61–83
2018-11
PDF

Google Scholar
Saeedi, Alieh; Peukert, Eric; Rahm, Erhard
Using Link Features for Entity Clustering in Knowledge Graphs
Proc. ESWC 2018 (Best research paper award)
2018-06
PDF

Google Scholar
Rostami, M. Ali; Saeedi, Alieh; Peukert, Eric ; Rahm, Erhard
Interactive Visualization of Large Similarity Graphs and Entity Resolution Clusters
Proc. EDBT 2018
2018-03
PDF

Google Scholar
Saeedi, Alieh; Peukert, Eric; Rahm, Erhard
Comparative Evaluation of Distributed Clustering Schemes for Multi-source Entity Resolution
Proc. ADBIS, LNCS 10509, pp 278-293
2017-09

Related Datasets

DS1- Geographical Settlements (read more)
DS2- Music Brainz (read more)
DS3- North Carolina Voters (read more)

AnhangGröße
famerIncremental.png158.66 KB
LSWT2018-Rahm.pdf1.06 MB
scadsAI.png4.95 KB