Data profiling
profile columns <json_rows>
Description
Profile of each column: main type (integer, decimal, date, timestamp, email, boolean, string) and type conformity, completeness, distinct values, unique, lengths, min, max, mean, std, median, quartiles, IQR outliers, top 5 values, top 3 patterns (A = upper letter, a = lower letter, 9 = digit)
Profile of each column: main type (integer, decimal, date, timestamp, email, boolean, string) and type conformity, completeness, distinct values, unique, lengths, min, max, mean, std, median, quartiles, IQR outliers, top 5 values, top 3 patterns (A = upper letter, a = lower letter, 9 = digit)
Parameters
json_rows: The rows: JSON array of objects - String - required
json_rows: The rows: JSON array of objects - String - required
admin
profile columns "[{\"id\": 1, \"name\": \"Ann\", \"email\": \"ann@mail.com\", \"age\": 34, \"city\": \"Paris\", \"zip\": \"75001\"}, {\"id\": 2, \"name\": \"Bob\", \"email\": \"bob@mail\", \"age\": 41, \"city\": \"Lyon\", \"zip\": \"69002\"}, {\"id\": 3, \"name\": \"Cid\", \"email\": null, \"age\": 29, \"city\": \"Paris\", \"zip\": \"75001\"}, {\"id\": 4, \"name\": \"Dan\", \"email\": \"dan@mail.fr\", \"age\": 230, \"city\": \"Lyon\", \"zip\": \"69002\"}, {\"id\": 5, \"name\": \"Eve\", \"email\": \"eve@mail.com\", \"age\": null, \"city\": \"Nice\", \"zip\": \"06000\"}]"
mentdb
{"rows": 5, "columns": 6, "profile": [{"field": "id", "type": "integer", "type_conformity": 1, "count": 5, "nulls": 0, "completeness": 1, "distinct": 5, "unique": true, "min_length": 1, "max_length": 1, "min": 1, "max": 5, "mean": 3, "std": 1.5811388300841898, "median": 3, "q1": 2, "q3": 4, "outliers": 0, "top_values": [{"value": "1", "count": 1}, {"value": "2", "count": 1}, {"value": "3", "count": 1}, {"value": "4", "count": 1}, {"value": "5", "count": 1}], "patterns": [{"pattern": "9", "count": 5}]}, {"field": "name", "type": "string", "type_conformity": 1, "count": 5, "nulls": 0, "completeness": 1, "distinct": 5, "unique": true, "min_length": 3, "max_length": 3, "top_values": [{"value": "Ann", "count": 1}, {"value": "Bob", "count": 1}, {"value": "Cid", "count": 1}, {"value": "Dan", "count": 1}, {"value": "Eve", "count": 1}], "patterns": [{"pattern": "Aa2", "count": 5}]}, {"field": "email", "type": "email", "type_conformity": 0.75, "count": 5, "nulls": 1, "completeness": 0.8, "distinct": 4, "unique": false, "min_length": 8, "max_length": 12, "top_values": [{"value": "ann@mail.com", "count": 1}, {"value": "bob@mail", "count": 1}, {"value": "dan@mail.fr", "count": 1}, {"value": "eve@mail.com", "count": 1}], "patterns": [{"pattern": "a3@a4.a3", "count": 2}, {"pattern": "a3@a4", "count": 1}, {"pattern": "a3@a4.a2", "count": 1}]}, {"field": "age", "type": "integer", "type_conformity": 1, "count": 5, "nulls": 1, "completeness": 0.8, "distinct": 4, "unique": false, "min_length": 2, "max_length": 3, "min": 29, "max": 230, "mean": 83.5, "std": 97.79059259458448, "median": 37.5, "q1": 32.75, "q3": 88.25, "outliers": 1, "top_values": [{"value": "230", "count": 1}, {"value": "29", "count": 1}, {"value": "34", "count": 1}, {"value": "41", "count": 1}], "patterns": [{"pattern": "92", "count": 3}, {"pattern": "93", "count": 1}]}, {"field": "city", "type": "string", "type_conformity": 1, "count": 5, "nulls": 0, "completeness": 1, "distinct": 3, "unique": false, "min_length": 4, "max_length": 5, "top_values": [{"value": "Lyon", "count": 2}, {"value": "Paris", "count": 2}, {"value": "Nice", "count": 1}], "patterns": [{"pattern": "Aa3", "count": 3}, {"pattern": "Aa4", "count": 2}]}, {"field": "zip", "type": "integer", "type_conformity": 0.8, "count": 5, "nulls": 0, "completeness": 1, "distinct": 3, "unique": false, "min_length": 5, "max_length": 5, "min": 69002, "max": 75001, "mean": 72001.5, "std": 3463.524264868565, "median": 72001.5, "q1": 69002, "q3": 75001, "outliers": 0, "top_values": [{"value": "69002", "count": 2}, {"value": "75001", "count": 2}, {"value": "06000", "count": 1}], "patterns": [{"pattern": "95", "count": 5}]}]}
profile columns <json_rows> <json_params>
Description
Profile with parameters: top (number of frequent values, 5), patterns (number of patterns, 3)
Profile with parameters: top (number of frequent values, 5), patterns (number of patterns, 3)
Parameters
json_rows: The rows - String - required
json_params: The parameters - String - required
json_rows: The rows - String - required
json_params: The parameters - String - required
admin
profile columns "[{\"id\": 1, \"name\": \"Ann\", \"email\": \"ann@mail.com\", \"age\": 34, \"city\": \"Paris\", \"zip\": \"75001\"}, {\"id\": 2, \"name\": \"Bob\", \"email\": \"bob@mail\", \"age\": 41, \"city\": \"Lyon\", \"zip\": \"69002\"}, {\"id\": 3, \"name\": \"Cid\", \"email\": null, \"age\": 29, \"city\": \"Paris\", \"zip\": \"75001\"}, {\"id\": 4, \"name\": \"Dan\", \"email\": \"dan@mail.fr\", \"age\": 230, \"city\": \"Lyon\", \"zip\": \"69002\"}, {\"id\": 5, \"name\": \"Eve\", \"email\": \"eve@mail.com\", \"age\": null, \"city\": \"Nice\", \"zip\": \"06000\"}]" "{\"top\": 2, \"patterns\": 1}"
mentdb
{"rows": 5, "columns": 6, "profile": [{"field": "id", "type": "integer", "type_conformity": 1, "count": 5, "nulls": 0, "completeness": 1, "distinct": 5, "unique": true, "min_length": 1, "max_length": 1, "min": 1, "max": 5, "mean": 3, "std": 1.5811388300841898, "median": 3, "q1": 2, "q3": 4, "outliers": 0, "top_values": [{"value": "1", "count": 1}, {"value": "2", "count": 1}], "patterns": [{"pattern": "9", "count": 5}]}, {"field": "name", "type": "string", "type_conformity": 1, "count": 5, "nulls": 0, "completeness": 1, "distinct": 5, "unique": true, "min_length": 3, "max_length": 3, "top_values": [{"value": "Ann", "count": 1}, {"value": "Bob", "count": 1}], "patterns": [{"pattern": "Aa2", "count": 5}]}, {"field": "email", "type": "email", "type_conformity": 0.75, "count": 5, "nulls": 1, "completeness": 0.8, "distinct": 4, "unique": false, "min_length": 8, "max_length": 12, "top_values": [{"value": "ann@mail.com", "count": 1}, {"value": "bob@mail", "count": 1}], "patterns": [{"pattern": "a3@a4.a3", "count": 2}]}, {"field": "age", "type": "integer", "type_conformity": 1, "count": 5, "nulls": 1, "completeness": 0.8, "distinct": 4, "unique": false, "min_length": 2, "max_length": 3, "min": 29, "max": 230, "mean": 83.5, "std": 97.79059259458448, "median": 37.5, "q1": 32.75, "q3": 88.25, "outliers": 1, "top_values": [{"value": "230", "count": 1}, {"value": "29", "count": 1}], "patterns": [{"pattern": "92", "count": 3}]}, {"field": "city", "type": "string", "type_conformity": 1, "count": 5, "nulls": 0, "completeness": 1, "distinct": 3, "unique": false, "min_length": 4, "max_length": 5, "top_values": [{"value": "Lyon", "count": 2}, {"value": "Paris", "count": 2}], "patterns": [{"pattern": "Aa3", "count": 3}]}, {"field": "zip", "type": "integer", "type_conformity": 0.8, "count": 5, "nulls": 0, "completeness": 1, "distinct": 3, "unique": false, "min_length": 5, "max_length": 5, "min": 69002, "max": 75001, "mean": 72001.5, "std": 3463.524264868565, "median": 72001.5, "q1": 69002, "q3": 75001, "outliers": 0, "top_values": [{"value": "69002", "count": 2}, {"value": "75001", "count": 2}], "patterns": [{"pattern": "95", "count": 5}]}]}
profile keys <json_rows>
Description
Candidate keys (columns, then pairs of columns, without null and without duplicate) and functional dependencies a -> b (each value of a has one value of b)
Candidate keys (columns, then pairs of columns, without null and without duplicate) and functional dependencies a -> b (each value of a has one value of b)
Parameters
json_rows: The rows - String - required
json_rows: The rows - String - required
admin
profile keys "[{\"id\": 1, \"name\": \"Ann\", \"email\": \"ann@mail.com\", \"age\": 34, \"city\": \"Paris\", \"zip\": \"75001\"}, {\"id\": 2, \"name\": \"Bob\", \"email\": \"bob@mail\", \"age\": 41, \"city\": \"Lyon\", \"zip\": \"69002\"}, {\"id\": 3, \"name\": \"Cid\", \"email\": null, \"age\": 29, \"city\": \"Paris\", \"zip\": \"75001\"}, {\"id\": 4, \"name\": \"Dan\", \"email\": \"dan@mail.fr\", \"age\": 230, \"city\": \"Lyon\", \"zip\": \"69002\"}, {\"id\": 5, \"name\": \"Eve\", \"email\": \"eve@mail.com\", \"age\": null, \"city\": \"Nice\", \"zip\": \"06000\"}]"
mentdb
{"rows": 5, "keys": ["id", "name"], "pair_keys": [], "dependencies": [{"from": "city", "to": "zip"}, {"from": "zip", "to": "city"}]}