ML Metrics | MQL Functions

ML Metrics

ml metrics accuracy ml metrics confusion ml metrics report ml metrics precision ml metrics recall ml metrics f1 ml metrics kappa ml metrics mcc ml metrics roc ml metrics auc ml metrics log_loss ml metrics regression ml metrics mae ml metrics mse ml metrics rmse ml metrics mape ml metrics r2 ml metrics silhouette ml metrics cost

ml metrics accuracy <json_y_true> <json_y_pred>

Description

Fraction of correct predictions

Parameters

json_y_true: The true labels - String - required
json_y_pred: The predicted labels - String - required

admin
ml metrics accuracy "[0, 1, 1, 0, 1]" "[0, 1, 0, 0, 1]"

mentdb
0.8

ml metrics confusion <json_y_true> <json_y_pred>

Description

Confusion matrix (rows = true labels, columns = predicted labels)

Parameters

json_y_true: The true labels - String - required
json_y_pred: The predicted labels - String - required

admin
ml metrics confusion "[\"cat\", \"dog\", \"dog\", \"cat\"]" "[\"cat\", \"dog\", \"cat\", \"cat\"]"

mentdb
{"labels": ["cat", "dog"], "matrix": [[2, 0], [1, 1]]}

ml metrics report <json_y_true> <json_y_pred>

Description

Classification report: precision, recall, f1 and support of each class, macro and weighted averages, accuracy, Cohen kappa, Matthews correlation, confusion matrix

Parameters

json_y_true: The true labels - String - required
json_y_pred: The predicted labels - String - required

admin
ml metrics report "[0, 0, 1, 1, 2, 2]" "[0, 1, 1, 1, 2, 0]"

mentdb
{"accuracy": 0.6666666666666666, "classes": [{"label": 0, "precision": 0.5, "recall": 0.5, "f1": 0.5, "support": 2}, {"label": 1, "precision": 0.6666666666666666, "recall": 1, "f1": 0.8, "support": 2}, {"label": 2, "precision": 1, "recall": 0.5, "f1": 0.6666666666666666, "support": 2}], "macro": {"precision": 0.7222222222222221, "recall": 0.6666666666666666, "f1": 0.6555555555555556}, "weighted": {"precision": 0.7222222222222221, "recall": 0.6666666666666666, "f1": 0.6555555555555556}, "kappa": 0.4999999999999999, "mcc": 0.5222329678670935, "confusion": [[1, 1, 0], [0, 2, 0], [1, 0, 1]], "labels": [0, 1, 2]}

Used in use cases (1 recipe)

Deep learning engineer › Confusion matrix of an intent classifier

Often used with

dl n_bayesian create_model · dl n_bayesian delete · dl n_bayesian exist · dl n_bayesian load · dl n_bayesian predict · file create · file delete · json doc

ml metrics precision <json_y_true> <json_y_pred> <average>

Description

Precision: macro, weighted, micro average or for one positive label

Parameters

json_y_true: The true labels - String - required
json_y_pred: The predicted labels - String - required
average: macro, weighted, micro or a label - String - required

admin
ml metrics precision "[0, 1, 1, 0, 1]" "[0, 1, 0, 0, 1]" "1"

mentdb
1

ml metrics recall <json_y_true> <json_y_pred> <average>

Description

Recall (sensitivity): macro, weighted, micro average or for one positive label

Parameters

json_y_true: The true labels - String - required
json_y_pred: The predicted labels - String - required
average: macro, weighted, micro or a label - String - required

admin
ml metrics recall "[0, 1, 1, 0, 1]" "[0, 1, 0, 0, 1]" "1"

mentdb
0.6666666666666666

Used in use cases (1 recipe)

Epidemiologist › Detect a rare disease with SMOTE

Often used with

ml model delete · ml model predict · ml prep smote · ml train

ml metrics f1 <json_y_true> <json_y_pred> <average>

Description

F1 score: macro, weighted, micro average or for one positive label

Parameters

json_y_true: The true labels - String - required
json_y_pred: The predicted labels - String - required
average: macro, weighted, micro or a label - String - required

admin
ml metrics f1 "[0, 0, 1, 1, 2, 2]" "[0, 1, 1, 1, 2, 0]" "macro"

mentdb
0.6555555555555556

ml metrics kappa <json_y_true> <json_y_pred>

Description

Cohen kappa (agreement corrected by chance)

Parameters

json_y_true: The true labels - String - required
json_y_pred: The predicted labels - String - required

admin
ml metrics kappa "[0, 1, 1, 0, 1, 0]" "[0, 1, 0, 0, 1, 1]"

mentdb
0.33333333333333326

ml metrics mcc <json_y_true> <json_y_pred>

Description

Matthews correlation coefficient (multiclass)

Parameters

json_y_true: The true labels - String - required
json_y_pred: The predicted labels - String - required

admin
ml metrics mcc "[0, 1, 1, 0, 1, 0]" "[0, 1, 0, 0, 1, 1]"

mentdb
0.3333333333333333

ml metrics roc <json_y_true> <json_y_score> <positive>

Description

ROC curve and area under the curve from scores (higher = positive)

Parameters

json_y_true: The true labels - String - required
json_y_score: The scores or probabilities - String - required
positive: The positive label - String - required

admin
ml metrics roc "[0, 0, 1, 1]" "[0.1, 0.4, 0.35, 0.8]" "1"

mentdb
{"auc": 0.75, "fpr": [0, 0, 0.5, 0.5, 1], "tpr": [0, 0.5, 0.5, 1, 1], "thresholds": ["Infinity", 0.8, 0.4, 0.35, 0.1]}

Used in use cases (1 recipe)

Statistician & biostatistician › Diagnostic test: AUC and Youden threshold

ml metrics auc <json_y_true> <json_y_score> <positive>

Description

Area under the ROC curve

Parameters

json_y_true: The true labels - String - required
json_y_score: The scores or probabilities - String - required
positive: The positive label - String - required

admin
ml metrics auc "[0, 0, 1, 1, 1]" "[0.2, 0.3, 0.6, 0.9, 0.25]" "1"

mentdb
0.8333333333333333

ml metrics log_loss <json_y_true> <json_y_proba> <positive>

Description

Logarithmic loss of the probabilities of the positive label

Parameters

json_y_true: The true labels - String - required
json_y_proba: The probability of the positive label - String - required
positive: The positive label - String - required

admin
ml metrics log_loss "[1, 0, 1]" "[0.9, 0.2, 0.6]" "1"

mentdb
0.2797765635793423

ml metrics regression <json_y_true> <json_y_pred>

Description

All the regression metrics: MAE, MSE, RMSE, MAPE (%), R2, explained variance, max error, median absolute error, bias

Parameters

json_y_true: The true values - String - required
json_y_pred: The predicted values - String - required

admin
ml metrics regression "[3, -0.5, 2, 7]" "[2.5, 0, 2, 8]"

mentdb
{"mae": 0.5, "mse": 0.375, "rmse": 0.6123724356957945, "mape": 32.73809523809524, "r2": 0.9486081370449679, "explained_variance": 0.9571734475374732, "max_error": 1, "median_ae": 0.5, "bias": -0.25}

ml metrics mae <json_y_true> <json_y_pred>

Description

Mean absolute error

Parameters

json_y_true: The true values - String - required
json_y_pred: The predicted values - String - required

admin
ml metrics mae "[3, -0.5, 2, 7]" "[2.5, 0, 2, 8]"

mentdb
0.5

Used in use cases (1 recipe)

Forecaster & econometrician › Choose the forecasting model automatically: ARIMA or exponential smoothing?

Often used with

ml ts auto_arima · ml ts ets

ml metrics mse <json_y_true> <json_y_pred>

Description

Mean squared error

Parameters

json_y_true: The true values - String - required
json_y_pred: The predicted values - String - required

admin
ml metrics mse "[3, -0.5, 2, 7]" "[2.5, 0, 2, 8]"

mentdb
0.375

ml metrics rmse <json_y_true> <json_y_pred>

Description

Root mean squared error

Parameters

json_y_true: The true values - String - required
json_y_pred: The predicted values - String - required

admin
ml metrics rmse "[3, -0.5, 2, 7]" "[2.5, 0, 2, 8]"

mentdb
0.6123724356957945

Used in use cases (1 recipe)

Chemist › Near infrared calibration: protein content by PLS

Often used with

dsp spectra_prep · stat pls

ml metrics mape <json_y_true> <json_y_pred>

Description

Mean absolute percentage error in % (the zero true values are ignored)

Parameters

json_y_true: The true values - String - required
json_y_pred: The predicted values - String - required

admin
ml metrics mape "[100, 200, 300]" "[110, 190, 330]"

mentdb
8.333333333333334

ml metrics r2 <json_y_true> <json_y_pred>

Description

Coefficient of determination R2

Parameters

json_y_true: The true values - String - required
json_y_pred: The predicted values - String - required

admin
ml metrics r2 "[3, -0.5, 2, 7]" "[2.5, 0, 2, 8]"

mentdb
0.9486081370449679

ml metrics silhouette <json_X> <json_labels>

Description

Silhouette score of a clustering (euclidean), global and per point; the label -1 (noise) is ignored

Parameters

json_X: The rows - String - required
json_labels: The cluster of each row - String - required

admin
ml metrics silhouette "[[0, 0], [0, 1], [10, 10], [10, 11]]" "[0, 0, 1, 1]"

mentdb
{"score": 0.9292895427118657, "samples": [0.9310539882297567, 0.9275250971939747, 0.9275250971939747, 0.9310539882297567]}

ml metrics cost <json_y_true> <json_y_pred> <json_cost>

Description

Total and mean cost of predictions with a cost matrix cost[true class][predicted class], the classes sorted (numbers in increasing order)

Parameters

json_y_true: The true labels - String - required
json_y_pred: The predicted labels - String - required
json_cost: The cost matrix - String - required

admin
ml metrics cost "[0, 1, 1, 0, 0]" "[0, 0, 1, 1, 0]" "[[0, 1], [10, 0]]"

mentdb
{"classes": [0, 1], "total_cost": 11, "mean_cost": 2.2, "confusion": [[2, 1], [1, 1]]}

Used in use cases (1 recipe)

Industrial quality engineer › Inspection driven by the cost of errors

Often used with

ml model delete · ml model predict · ml train