Record linkage
linkage dedup <json_rows> <json_rules>
Description
Duplicates in a table: the pairs above the threshold (0.85) with the similarity of each field, and the clusters of rows that describe the same entity
Duplicates in a table: the pairs above the threshold (0.85) with the similarity of each field, and the clusters of rows that describe the same entity
Parameters
json_rows: The rows - String - required
json_rules: The rules: [{"field", "method", "weight"}] - String - required
json_rows: The rows - String - required
json_rules: The rules: [{"field", "method", "weight"}] - String - required
admin
linkage dedup "[{\"id\": 1, \"name\": \"Jean Dupont\", \"city\": \"Paris\", \"zip\": \"75011\", \"born\": \"1980-04-12\"}, {\"id\": 2, \"name\": \"Jean Dupond\", \"city\": \"Paris\", \"zip\": \"75011\", \"born\": \"1980-04-12\"}, {\"id\": 3, \"name\": \"Marie Martin\", \"city\": \"Lyon\", \"zip\": \"69003\", \"born\": \"1975-09-30\"}, {\"id\": 4, \"name\": \"MARTIN Marie\", \"city\": \"Lyon\", \"zip\": \"69003\", \"born\": \"1975-09-30\"}, {\"id\": 5, \"name\": \"Paul Durand\", \"city\": \"Nice\", \"zip\": \"06000\", \"born\": \"1990-01-02\"}]" "[{\"field\": \"name\", \"method\": \"tokens\", \"weight\": 2}, {\"field\": \"name\", \"method\": \"jaro_winkler\", \"weight\": 1}, {\"field\": \"born\", \"method\": \"exact\", \"weight\": 2}, {\"field\": \"zip\", \"method\": \"exact\"}]"
mentdb
{"rows": 5, "compared": 10, "pairs": [{"a": 0, "b": 1, "score": 0.9939, "fields": {"name": 0.964, "born": 1, "zip": 1}}, {"a": 2, "b": 3, "score": 0.9806, "fields": {"name": 0.883, "born": 1, "zip": 1}}], "clusters": [[0, 1], [2, 3]], "unique_entities": 3}
linkage dedup <json_rows> <json_rules> <json_params>
Description
Duplicates with parameters: threshold, block (field), block_prefix (first characters of the block field), id (field shown instead of the row index)
Duplicates with parameters: threshold, block (field), block_prefix (first characters of the block field), id (field shown instead of the row index)
Parameters
json_rows: The rows - String - required
json_rules: The rules - String - required
json_params: The parameters - String - required
json_rows: The rows - String - required
json_rules: The rules - String - required
json_params: The parameters - String - required
admin
linkage dedup "[{\"id\": 1, \"name\": \"Jean Dupont\", \"city\": \"Paris\", \"zip\": \"75011\", \"born\": \"1980-04-12\"}, {\"id\": 2, \"name\": \"Jean Dupond\", \"city\": \"Paris\", \"zip\": \"75011\", \"born\": \"1980-04-12\"}, {\"id\": 3, \"name\": \"Marie Martin\", \"city\": \"Lyon\", \"zip\": \"69003\", \"born\": \"1975-09-30\"}, {\"id\": 4, \"name\": \"MARTIN Marie\", \"city\": \"Lyon\", \"zip\": \"69003\", \"born\": \"1975-09-30\"}, {\"id\": 5, \"name\": \"Paul Durand\", \"city\": \"Nice\", \"zip\": \"06000\", \"born\": \"1990-01-02\"}]" "[{\"field\": \"name\", \"method\": \"tokens\", \"weight\": 2}, {\"field\": \"name\", \"method\": \"jaro_winkler\", \"weight\": 1}, {\"field\": \"born\", \"method\": \"exact\", \"weight\": 2}, {\"field\": \"zip\", \"method\": \"exact\"}]" "{\"threshold\": 0.8, \"block\": \"zip\", \"block_prefix\": 2, \"id\": \"id\"}"
mentdb
{"rows": 5, "compared": 2, "pairs": [{"a": 1, "b": 2, "score": 0.9939, "fields": {"name": 0.964, "born": 1, "zip": 1}}, {"a": 3, "b": 4, "score": 0.9806, "fields": {"name": 0.883, "born": 1, "zip": 1}}], "clusters": [[1, 2], [3, 4]], "unique_entities": 3}
linkage match <json_left> <json_right> <json_rules>
Description
Match two tables: for each row of the left table, the best row of the right table above the threshold; the rows without a match
Match two tables: for each row of the left table, the best row of the right table above the threshold; the rows without a match
Parameters
json_left: The left rows - String - required
json_right: The right rows - String - required
json_rules: The rules - String - required
json_left: The left rows - String - required
json_right: The right rows - String - required
json_rules: The rules - String - required
admin
linkage match "[{\"ref\": \"C1\", \"name\": \"Dupont Jean\", \"zip\": \"75011\"}, {\"ref\": \"C2\", \"name\": \"Paul Durant\", \"zip\": \"06000\"}, {\"ref\": \"C3\", \"name\": \"Luc Petit\", \"zip\": \"33000\"}]" "[{\"id\": 1, \"name\": \"Jean Dupont\", \"city\": \"Paris\", \"zip\": \"75011\", \"born\": \"1980-04-12\"}, {\"id\": 2, \"name\": \"Jean Dupond\", \"city\": \"Paris\", \"zip\": \"75011\", \"born\": \"1980-04-12\"}, {\"id\": 3, \"name\": \"Marie Martin\", \"city\": \"Lyon\", \"zip\": \"69003\", \"born\": \"1975-09-30\"}, {\"id\": 4, \"name\": \"MARTIN Marie\", \"city\": \"Lyon\", \"zip\": \"69003\", \"born\": \"1975-09-30\"}, {\"id\": 5, \"name\": \"Paul Durand\", \"city\": \"Nice\", \"zip\": \"06000\", \"born\": \"1990-01-02\"}]" "[{\"field\": \"name\", \"method\": \"tokens\"}, {\"field\": \"name\", \"method\": \"jaro_winkler\"}, {\"field\": \"zip\", \"method\": \"exact\"}]"
mentdb
{"matched": 1, "matches": [{"left": 1, "right": 4, "score": 0.9879}], "unmatched": [0, 2]}
linkage match <json_left> <json_right> <json_rules> <json_params>
Description
Match with parameters: threshold, block, block_prefix, left_id, right_id
Match with parameters: threshold, block, block_prefix, left_id, right_id
Parameters
json_left: The left rows - String - required
json_right: The right rows - String - required
json_rules: The rules - String - required
json_params: The parameters - String - required
json_left: The left rows - String - required
json_right: The right rows - String - required
json_rules: The rules - String - required
json_params: The parameters - String - required
admin
linkage match "[{\"ref\": \"C1\", \"name\": \"Dupont Jean\", \"zip\": \"75011\"}, {\"ref\": \"C2\", \"name\": \"Paul Durant\", \"zip\": \"06000\"}, {\"ref\": \"C3\", \"name\": \"Luc Petit\", \"zip\": \"33000\"}]" "[{\"id\": 1, \"name\": \"Jean Dupont\", \"city\": \"Paris\", \"zip\": \"75011\", \"born\": \"1980-04-12\"}, {\"id\": 2, \"name\": \"Jean Dupond\", \"city\": \"Paris\", \"zip\": \"75011\", \"born\": \"1980-04-12\"}, {\"id\": 3, \"name\": \"Marie Martin\", \"city\": \"Lyon\", \"zip\": \"69003\", \"born\": \"1975-09-30\"}, {\"id\": 4, \"name\": \"MARTIN Marie\", \"city\": \"Lyon\", \"zip\": \"69003\", \"born\": \"1975-09-30\"}, {\"id\": 5, \"name\": \"Paul Durand\", \"city\": \"Nice\", \"zip\": \"06000\", \"born\": \"1990-01-02\"}]" "[{\"field\": \"name\", \"method\": \"tokens\"}, {\"field\": \"name\", \"method\": \"jaro_winkler\"}]" "{\"threshold\": 0.7, \"block\": \"zip\", \"left_id\": \"ref\", \"right_id\": \"id\"}"
mentdb
{"matched": 2, "matches": [{"left": "C1", "right": 1, "score": 0.7576}, {"left": "C2", "right": 5, "score": 0.9818}], "unmatched": ["C3"]}