Profiles / Engineering & data
SB

Statistician & biostatistician

Confidence intervals, sample size, ANOVA, chi-square and non-parametric tests.

27 recipes on this page · open in the interactive view

Recipes
  1. Confidence interval and sample size
  2. ANOVA and Kruskal-Wallis on three diets
  3. Clinical trial 2x2 table
  4. Bayesian A/B test with Beta posteriors
  5. Before / after treatment: paired t-test or Wilcoxon
  6. z-test with known sigma, power and sample size
  7. Summary of a 5-point Likert survey item
  8. Which mean? Arithmetic, geometric and harmonic in practice
  9. The Student t density from the Gamma and Beta functions
  10. Mincer wage equation from a survey table in SQL
  11. Diagnostic test: AUC and Youden threshold
  12. How many populations? Gaussian mixture and BIC
  13. Correspondence analysis of a survey table
  14. Prediction interval of a calibration
  15. Is the best model really better?
  16. How many subjects for the study?
  17. Decide an A/B test
  18. Bayesian logistic regression by MCMC
  19. Repeated measures: each subject has its own slope
  20. Missing values: multiple imputation instead of deleting the rows
  21. Repeated measures: time, group and their interaction
  22. Small sample: Bayesian logistic regression and the probability of an effect
  23. Size of a survival trial: events, patients and power
  24. A/B test of a landing page: probability to win and expected loss
  25. Multi-center trial: partial pooling of the effects of 8 centers
  26. Effect of a speed limit on road accidents: interrupted time series
  27. Opinion poll: raking on sex and age, weighted estimate and margin of error
executed on MentDB

Confidence interval and sample size

A pilot study measures the drop of systolic pressure on 10 patients. The script gives the 95 % confidence interval of the mean and the number of patients needed for a margin of 1.5 mmHg.

Pipeline
Script
admin
#1. Pilot study: drop of systolic pressure in mmHg after 8 weeks, 10 patients;
-> "[d]" "[8, 12, 5, 15, 10, 7, 13, 9, 11, 6]";
json load "rx_st_ci" (stat conf_interval_mean [d] 0.95);
#2. Sample size for a margin of 1.5 mmHg at 95 percent: n = z s / E squared;
-> "[z]" (stat dist_inv "normal" "[0, 1]" 0.975);
-> "[n]" (math ceil (math calc (concat "([z]*" (stat stddev [d]) "/1.5)^2")));
concat "mean = " (json select "rx_st_ci" /mean) " mmHg, 95% CI [" (math round (json select "rx_st_ci" /lower) 2) ", " (math round (json select "rx_st_ci" /upper) 2) "], margin " (math round (json select "rx_st_ci" /margin) 2) " -> " [n] " patients needed for a margin of 1.5 mmHg"
Result
mentdb
mean = 9.6 mmHg, 95% CI [7.31, 11.89], margin 2.29 -> 18 patients needed for a margin of 1.5 mmHg
Go further
  • Use the Student quantile with stat dist_inv "student" and iterate on n
  • Add the power of a test: n = ((z_alpha + z_beta) s / delta)^2
  • Check the hypothesis of the paired difference with stat ttest_1
executed on MentDB

ANOVA and Kruskal-Wallis on three diets

Weight loss is measured on three diets of six subjects. The script runs a one-way ANOVA with its effect size and the non-parametric Kruskal-Wallis test as a check.

Pipeline
Script
admin
#1. Weight loss in kg with three diets;
-> "[g]" "[[3.1, 4.2, 2.8, 3.9, 3.5, 4.0], [5.2, 4.8, 6.1, 5.5, 4.9, 5.8], [3.8, 4.5, 4.1, 3.6, 4.9, 4.4]]";
#2. Parametric one-way ANOVA;
json load "rx_st_a" (stat anova [g]);
#3. Non-parametric check: Kruskal-Wallis on the ranks;
json load "rx_st_k" (stat kruskal [g]);
concat "ANOVA F(" (json select "rx_st_a" /df_between) ", " (json select "rx_st_a" /df_within) ") = " (math round (json select "rx_st_a" /f) 2) ", p = " (math round_sig (json select "rx_st_a" /p_value) 3) ", eta2 = " (math round (json select "rx_st_a" /eta2) 3) " ; Kruskal-Wallis H = " (math round (json select "rx_st_k" /h) 2) ", p = " (math round_sig (json select "rx_st_k" /p_value) 3)
Result
mentdb
ANOVA F(2, 15) = 18.92, p = 0.0000791, eta2 = 0.716 ; Kruskal-Wallis H = 12.07, p = 0.00240
Go further
executed on MentDB

Clinical trial 2x2 table

A trial counts the events in a treatment group and a placebo group of 100 patients each. The script runs the chi-square and Fisher exact tests and computes the relative risk, the absolute risk reduction and the number needed to treat.

Pipeline
Script
admin
#1. Trial: events / no event, treatment 12 / 88, placebo 27 / 73;
-> "[t]" "[[12, 88], [27, 73]]";
#2. Chi-square test of independence and Fisher exact test;
json load "rx_st_c" (stat chi2_independence [t]);
json load "rx_st_f" (stat fisher_exact [t]);
#3. Relative risk, absolute risk reduction and number needed to treat;
-> "[rr]" (math calc "(12/100)/(27/100)");
-> "[arr]" (math calc "27/100-12/100");
concat "chi2 = " (math round (json select "rx_st_c" /chi2) 2) ", p = " (math round_sig (json select "rx_st_c" /p_value) 3) " ; Fisher p = " (math round_sig (json select "rx_st_f" /p_value) 3) ", OR = " (math round (json select "rx_st_f" /odds_ratio) 3) " ; RR = " (math round [rr] 3) ", ARR = " (math round (math calc "100*[arr]") 1) " %, NNT = " (math ceil (math calc "1/[arr]"))
Result
mentdb
chi2 = 7.17, p = 0.00743 ; Fisher p = 0.0118, OR = 0.369 ; RR = 0.444, ARR = 15 %, NNT = 7
Go further
executed on MentDB

Bayesian A/B test with Beta posteriors

Conversions 120/1000 against 145/1000 with uniform priors give Beta posteriors. The script computes the probability that B is better than A by a double integral of the joint posterior (normalised with lgamma) and checks it in one dimension with the incomplete beta function.

Pipeline
Script
admin
#1. A/B test: A 120 conversions out of 1000, B 145 out of 1000, uniform priors: Beta posteriors;
-> "[a1]" 121; -> "[b1]" 881; -> "[a2]" 146; -> "[b2]" 856;
#2. Log normalising constants with lgamma: ln B of a,b = lgamma a + lgamma b - lgamma a+b;
-> "[lB1]" (math calc "lgamma([a1])+lgamma([b1])-lgamma([a1]+[b1])");
-> "[lB2]" (math calc "lgamma([a2])+lgamma([b2])-lgamma([a2]+[b2])");
#3. P of pB > pA: double integral of the joint posterior over y > x;
-> "[P]" (num integrate2 (concat "exp(120*ln(x)+880*ln(1-x)-(" [lB1] ")+145*ln(y)+855*ln(1-y)-(" [lB2] "))") "x" 0.05 0.25 "y" "x" 0.3);
#4. Check in one dimension with the incomplete beta function;
-> "[P1]" (num integrate (concat "exp(120*ln(x)+880*ln(1-x)-(" [lB1] "))*(1-beta_inc(146, 856, x))") "x" 0.05 0.25);
concat "P(B better than A) = " (math round [P] 4) " (1D check " (math round [P1] 4) "), posterior means " (math round (math calc "[a1]/([a1]+[b1])") 4) " and " (math round (math calc "[a2]/([a2]+[b2])") 4)
Result
mentdb
P(B better than A) = 0.9502 (1D check 0.9502), posterior means 0.1208 and 0.1457
Go further
executed on MentDB

Before / after treatment: paired t-test or Wilcoxon

The systolic pressure of 12 patients is measured before and after 8 weeks of treatment. Test the normality of the differences (Shapiro-Wilk), run the paired t-test and the Wilcoxon signed-rank test, and give the effect size Cohen dz. Here the differences are not normal (p < 0.05), so the Wilcoxon test is the one to report.

Pipeline
Script
admin
#1. Systolic blood pressure in mmHg of 12 patients before and after 8 weeks of treatment;
-> "[before]" "[152, 148, 160, 145, 158, 151, 163, 149, 155, 147, 159, 150]";
-> "[after]" "[141, 145, 148, 140, 150, 147, 151, 146, 144, 145, 149, 148]";
#2. Differences and their normality, which the paired t-test assumes;
-> "[d]" (vector sub [before] [after]);
json load "rx_sw" (stat shapiro [d]);
#3. Paired t-test and its non-parametric counterpart;
json load "rx_tp" (stat ttest_paired [before] [after]);
json load "rx_wx" (stat wilcoxon [before] [after]);
#4. Effect size: Cohen dz = mean of d / sd of d;
-> "[dz]" (/ (stat mean [d]) (stat stddev [d]));
concat "mean drop " (math round (stat mean [d]) 2) " mmHg | Shapiro W = " (math round (json select "rx_sw" "/w") 3) ", p = " (math round (json select "rx_sw" "/p_value") 3) " | paired t = " (math round (json select "rx_tp" "/t") 3) ", df = " (json select "rx_tp" "/df") ", p = " (math round_sig (json select "rx_tp" "/p_value") 3) " | Wilcoxon W = " (json select "rx_wx" "/w") ", p = " (math round_sig (json select "rx_wx" "/p_value") 3) " (" (json select "rx_wx" "/method") ") | Cohen dz = " (math round [dz] 2)
Result
mentdb
mean drop 6.92 mmHg | Shapiro W = 0.848, p = 0.035 | paired t = 5.812, df = 11, p = 0.000117 | Wilcoxon W = 0, p = 0.000488 (permutation) | Cohen dz = 1.68
Go further
executed on MentDB

z-test with known sigma, power and sample size

A standardised reading test has a known sigma of 15 and a national mean of 100. A school tested 12 pupils. The statistician runs the z-test, computes the power to detect a 5-point shift with this n, and the n needed for 80 % power, all from the error function.

Pipeline
Script
admin
#1. Scores of the class, H0: mu = 100, sigma = 15;
-> "[x]" "[112, 98, 105, 121, 93, 108, 117, 101, 99, 110, 125, 104]";
json load "rx_p4g_z" (stat ztest [x] 100 15);
-> "[t]" (concat "mean = " (math round (stat mean [x]) 2) ", z = " (math round (json select "rx_p4g_z" /z) 3) ", p = " (math round (json select "rx_p4g_z" /p_value) 4));
json unload "rx_p4g_z";
#2. Critical values: Phi^-1 of p = sqrt2 . erf_inv of 2p - 1;
-> "[za]" (* (math sqrt 2) (math erf_inv 0.95));
-> "[zb]" (* (math sqrt 2) (math erf_inv 0.6));
#3. Power for delta = 5 with n = 12, Phi of x = 0.5 . erfc of -x / sqrt2;
-> "[k]" (- (/ (* 5 (math sqrt 12)) 15) [za]);
-> "[pw]" (* 0.5 (math erfc (/ (- 0 [k]) (math sqrt 2))));
#4. n for 80 % power;
-> "[n]" (math ceil (math power (/ (* (+ [za] [zb]) 15) 5) 2));
concat [t] [_n_] "z 0.975 = " (math round [za] 4) ", z 0.80 = " (math round [zb] 4) [_n_] "power for +5 points with n = 12: " (math round (* 100 [pw]) 1) " %" [_n_] "n for 80 % power: " [n]
Result
mentdb
mean = 107.75, z = 1.79, p = 0.0735
z 0.975 = 1.96, z 0.80 = 0.8416
power for +5 points with n = 12: 21 %
n for 80 % power: 71
Go further
  • Compare with stat ttest_1 when sigma is estimated from the sample.
  • Plot the power curve against n with plot chart.
executed on MentDB

Summary of a 5-point Likert survey item

An item of a satisfaction survey is ordinal (1 = strongly disagree to 5 = strongly agree). The statistician reports what fits ordinal data: the frequency table, the mode, the median, the top-2-box and the bottom-2-box, with the mean only for information.

Pipeline
Script
admin
#1. Answers of 24 respondents;
-> "[a]" "[4, 5, 3, 4, 4, 2, 5, 4, 3, 1, 4, 5, 4, 3, 2, 4, 5, 4, 3, 4, 5, 2, 4, 3]";
json load "rx_p4g_lk" (stat frequency [a]);
#2. Frequency per answer;
-> "[tab]" "";
for (-> "[i]" 0) (< [i] (json count "rx_p4g_lk" /)) (++ "[i]") {
  -> "[tab]" (concat [tab] (json select "rx_p4g_lk" (concat "/[" [i] "]/value")) ": " (math round (* 100 (json select "rx_p4g_lk" (concat "/[" [i] "]/frequency"))) 1) " %  ");
};
#3. Bottom-2 box = cumulative frequency of answer 2, top-2 = 1 - cumulative of 3;
-> "[b2]" (json select "rx_p4g_lk" "/[1]/cumulative_frequency");
-> "[t2]" (- 1 (json select "rx_p4g_lk" "/[2]/cumulative_frequency"));
json unload "rx_p4g_lk";
concat [tab] [_n_] "mode = " (stat mode [a]) ", median = " (stat median [a]) ", mean = " (math round (math avg [a]) 2) [_n_] "top-2-box = " (math round (* 100 [t2]) 1) " %, bottom-2-box = " (math round (* 100 [b2]) 1) " %, net score = " (math round (* 100 (- [t2] [b2])) 1)
Result
mentdb
1: 4.2 %  2: 12.5 %  3: 20.8 %  4: 41.7 %  5: 20.8 %
mode = [4], median = 4, mean = 3.63
top-2-box = 62.5 %, bottom-2-box = 16.7 %, net score = 45.8
Go further
executed on MentDB

Which mean? Arithmetic, geometric and harmonic in practice

Three classic cases where the arithmetic mean misleads: the average yearly growth of an investment (geometric mean), the P/E ratio of an equal-amount portfolio (harmonic mean), and the population versus sample variance of a full census (divide by n).

Pipeline
Script
admin
#1. Yearly growth factors: +25 %, -20 %, +15 %, +5 %;
-> "[g]" "[1.25, 0.80, 1.15, 1.05]";
-> "[ga]" (math avg [g]);
-> "[gg]" (stat geo_mean [g]);
#2. Same amount invested in 4 shares with these P/E ratios;
-> "[pe]" "[12, 18, 30, 45]";
#3. Census of the 6 branches of a company: staff per branch;
-> "[s]" "[14, 22, 9, 31, 18, 26]";
concat "growth: arithmetic " (math round (* 100 (- [ga] 1)) 2) " %/yr is wrong, geometric " (math round (* 100 (- [gg] 1)) 2) " %/yr, check final value " (math round (math power [gg] 4) 4) " = " (math round (* 1.25 (* 0.8 (* 1.15 1.05))) 4) [_n_] "portfolio P/E: harmonic " (math round (stat harm_mean [pe]) 2) " (arithmetic would say " (math avg [pe]) ")" [_n_] "census variance /n = " (math round (stat pvariance [s]) 3) ", sample variance /(n-1) = " (math round (stat variance [s]) 3)
Result
mentdb
growth: arithmetic 6.25 %/yr is wrong, geometric 4.83 %/yr, check final value 1.2075 = 1.2075
portfolio P/E: harmonic 20.57 (arithmetic would say 26.25)
census variance /n = 53.667, sample variance /(n-1) = 64.4
Go further
  • Show the inequality harmonic <= geometric <= arithmetic on any positive series.
  • Use stat weighted_mean when the amounts invested differ.
executed on MentDB

The Student t density from the Gamma and Beta functions

To teach where the t density comes from, the script evaluates f(t) = Gamma((v+1)/2) / (sqrt(v pi) Gamma(v/2)) (1 + t^2/v)^(-(v+1)/2) with the Gamma function, then with the Beta form, then with lngamma for v = 200 (where Gamma overflows), and compares with the built-in law.

Pipeline
Script
admin
#1. v = 5 degrees of freedom, t = 1.5;
-> "[v]" 5; -> "[t]" 1.5;
-> "[k]" (math power (+ 1 (/ (* [t] [t]) [v])) (- 0 (/ (+ [v] 1) 2)));
-> "[f1]" (* (/ (math gamma (/ (+ [v] 1) 2)) (* (math sqrt (* [v] (math pi))) (math gamma (/ [v] 2)))) [k]);
#2. Same with the Beta function;
-> "[f2]" (/ [k] (* (math sqrt [v]) (math beta 0.5 (/ [v] 2))));
#3. v = 200: Gamma of 100.5 is huge, use lngamma;
-> "[c200]" (math exp (- (math lngamma 100.5) (math lngamma 100)));
-> "[f200]" (* (/ [c200] (math sqrt (* 200 (math pi)))) (math power (+ 1 (/ 2.25 200)) -100.5));
concat "v = 5: Gamma form " (math round [f1] 8) ", Beta form " (math round [f2] 8) ", stat dist_pdf " (math round (stat dist_pdf "student" "[5]" 1.5) 8) [_n_] "v = 200: lngamma form " (math round [f200] 6) ", stat dist_pdf " (math round (stat dist_pdf "student" "[200]" 1.5) 6) ", normal " (math round (stat dist_pdf "normal" "[0, 1]" 1.5) 6)
Result
mentdb
v = 5: Gamma form 0.12451734, Beta form 0.12451734, stat dist_pdf 0.12451734
v = 200: lngamma form 0.129445, stat dist_pdf 0.129445, normal 0.129518
Go further
executed on MentDB

Mincer wage equation from a survey table in SQL

Survey records sit in a database table (embedded H2 here). The statistician loads log hourly wage against years of education and experience straight from a SELECT into a multiple regression, and reports the coefficients, their standard errors and the adjusted R2.

Pipeline
Script
admin
#1. Survey database, embedded H2 file under tmp;
cm set "tuto_p4g_svy" {execute "db.h2.embedded.config.get"
	"[database]" "./tmp/tuto_p4g_svy/survey"
	"[user]" "tuto"
	"[password]" "tuto_pwd"
;};
sql connect "tuto_p4g_s2" {cm get "tuto_p4g_svy";};
sql dml "tuto_p4g_s2" "create table resp (educ double, exper double, lnwage double)";
sql dml "tuto_p4g_s2" "insert into resp values (10, 5, 2.45), (12, 3, 2.52), (12, 10, 2.78), (14, 2, 2.70), (16, 4, 2.98), (16, 12, 3.21), (11, 20, 2.80), (18, 6, 3.25), (13, 8, 2.80), (15, 15, 3.12), (12, 25, 2.95), (17, 9, 3.20)";
sql disconnect "tuto_p4g_s2";
#2. Multiple regression straight from the query, unused X fields empty;
pa rm load "rx_p4g_wage" "tuto_p4g_svy" "educ" "exper" "" "" "" "lnwage" "select educ, exper, lnwage from resp";
json load "rx_p4g_b" (pa rm estimate_regression_parameters "rx_p4g_wage");
json load "rx_p4g_se" (pa rm estimate_regression_parameters_standard_errors "rx_p4g_wage");
#3. Return to one year of schooling = exp of b1 - 1;
-> "[b1]" (json select "rx_p4g_b" "/[1]");
-> "[r]" (concat "ln w = " (math round (json select "rx_p4g_b" "/[0]") 3) " + " (math round [b1] 4) " educ + " (math round (json select "rx_p4g_b" "/[2]") 4) " exper" [_n_] "SE: " (math round (json select "rx_p4g_se" "/[0]") 3) ", " (math round (json select "rx_p4g_se" "/[1]") 4) ", " (math round (json select "rx_p4g_se" "/[2]") 4) ", t educ = " (math round (/ [b1] (json select "rx_p4g_se" "/[1]")) 2) [_n_] "adjusted R2 = " (math round (pa rm calculate_adjusted_r_squared "rx_p4g_wage") 4) ", return to schooling = " (math round (* 100 (- (math exp [b1]) 1)) 2) " % per year");
#4. Close and remove the temporary database;
json unload "rx_p4g_b"; json unload "rx_p4g_se";
pa rm close "rx_p4g_wage";
cm remove "tuto_p4g_svy";
file delete "tmp/tuto_p4g_svy";
[r]
Result
mentdb
ln w = 1.296 + 0.1018 educ + 0.0195 exper
SE: 0.087, 0.0057, 0.0021, t educ = 17.93
adjusted R2 = 0.9695, return to schooling = 10.71 % per year
Go further
executed on MentDB

Diagnostic test: AUC and Youden threshold

From the biomarker level of 12 patients with or without the disease, compute the ROC curve and its area, then the cut-off that maximizes the Youden index J = sensitivity + specificity - 1.

Pipeline
Script
admin
#1. Biomarker in ng/mL - disease 1 or 0;
-> "[y]" "[0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1]";
-> "[score]" "[1.2, 2.3, 1.8, 3.1, 2.7, 4.2, 3.9, 5.5, 4.8, 6.1, 2.9, 7.0]";
json load "rx_roc" (ml metrics roc [y] [score] "1");
#2. Youden index on each threshold;
-> "[best]" -1; -> "[cut]" 0; -> "[se]" 0; -> "[sp]" 0;
for (-> "[i]" 1) (< [i] (json count "rx_roc" "/thresholds")) (++ "[i]") {
  -> "[tpr]" (json select "rx_roc" (concat "/tpr/[" [i] "]"));
  -> "[fpr]" (json select "rx_roc" (concat "/fpr/[" [i] "]"));
  -> "[j]" (- [tpr] [fpr]);
  if (> [j] [best]) {-> "[best]" [j]; -> "[cut]" (json select "rx_roc" (concat "/thresholds/[" [i] "]")); -> "[se]" [tpr]; -> "[sp]" (- 1 [fpr]);};
};
-> "[res]" (concat "AUC " (math round (json select "rx_roc" "/auc") 3) " | cut-off >= " [cut] " ng/mL | sensitivity " (math round [se] 3) " | specificity " (math round [sp] 3) " | Youden J " (math round [best] 3));
json unload "rx_roc";
[res]
Result
mentdb
AUC 0.917 | cut-off >= 3.9 ng/mL | sensitivity 0.833 | specificity 0.833 | Youden J 0.667
Go further
executed on MentDB

How many populations? Gaussian mixture and BIC

Fit gaussian mixtures with 1, 2 and 3 components on two biomarkers of 15 subjects and keep the number of populations with the lowest BIC, with the weight and mean of each population.

Pipeline
Script
admin
#1. Two biomarkers per subject;
-> "[X]" "[[1.0, 2.1], [1.2, 1.9], [0.8, 2.3], [1.1, 2.0], [0.9, 1.8], [1.3, 2.2], [1.0, 2.4], [4.1, 6.0], [3.6, 6.4], [4.5, 5.7], [4.0, 6.3], [3.7, 5.8], [4.4, 6.2], [4.2, 5.6], [3.9, 6.1]]";
-> "[best]" 0; -> "[bic]" 1e300; -> "[report]" "";
for (-> "[k]" 1) (<= [k] 3) (++ "[k]") {
  json load "rx_g" (ml fit "rx_gmm" "gmm" [X] (concat "{\"k\": " [k] ", \"seed\": 1}"));
  -> "[b]" (json select "rx_g" "/bic");
  -> "[report]" (concat [report] "k=" [k] " BIC " (math round [b] 1) "; ");
  if (< [b] [bic]) {-> "[bic]" [b]; -> "[best]" [k];};
};
json load "rx_g" (ml fit "rx_gmm" "gmm" [X] (concat "{\"k\": " [best] ", \"seed\": 1}"));
-> "[res]" (concat [report] "populations: " [best] " | weights " (json select "rx_g" "/weights") " | means " (json select "rx_g" "/means"));
json unload "rx_g"; ml model delete "rx_gmm";
[res]
Result
mentdb
k=1 BIC 87.4; k=2 BIC 45.4; k=3 BIC 60.4; populations: 2 | weights [
  0.4666666666666667,
  0.5333333333333333
] | means [
  [
    1.042857142857143,
    2.1
  ],
  [
    4.05,
    6.012500000000001
  ]
]
Go further
  • ml model transform gives the probability of each subject to belong to each population.
  • The AIC is also returned; the BIC penalizes more the complex models.
executed on MentDB

Correspondence analysis of a survey table

Cross the hair color and the eye color of 592 students, test the independence and place the categories on the first axis of a correspondence analysis.

Pipeline
Script
admin
#1. Rows: black, brown, red, blond hair - columns: brown, hazel, green, blue eyes;
-> "[t]" "[[68, 15, 5, 20], [119, 54, 29, 84], [26, 14, 14, 17], [7, 10, 16, 94]]";
json load "rx_ca" (ml ca [t] 2);
-> "[res]" (concat "chi2 " (math round (json select "rx_ca" "/chi2") 1) " p = " (json select "rx_ca" "/p_value") " | axis 1: " (math round (* 100 (json select "rx_ca" "/explained_ratio/[0]")) 1) " % of the inertia | black hair " (math round (json select "rx_ca" "/row_coordinates/[0]/[0]") 3) ", blond hair " (math round (json select "rx_ca" "/row_coordinates/[3]/[0]") 3) " | brown eyes " (math round (json select "rx_ca" "/column_coordinates/[0]/[0]") 3) ", blue eyes " (math round (json select "rx_ca" "/column_coordinates/[3]/[0]") 3));
json unload "rx_ca";
[res]
Result
mentdb
chi2 138.3 p = 0 | axis 1: 89.4 % of the inertia | black hair -0.505, blond hair 0.835 | brown eyes -0.492, blue eyes 0.547
Go further
  • The categories close on an axis are associated: blond hair with blue eyes, black hair with brown eyes.
  • stat chi2_independence gives the same chi2 test with the expected counts.
executed on MentDB

Prediction interval of a calibration

Fit the linear calibration of an instrument, then give the 95 % prediction interval of a new measure, exact Student interval, and compare with the Laplace interval of a support vector regression.

Pipeline
Script
admin
#1. Reference concentration in mg/L - signal;
-> "[X]" "[[0], [5], [10], [15], [20], [25], [30], [35], [40], [45]]";
-> "[y]" "[0.02, 0.51, 0.98, 1.52, 2.05, 2.48, 3.01, 3.55, 3.98, 4.51]";
ml train "rx_cal" "linear" [X] [y];
ml train "rx_svr" "svm_regressor" [X] [y] "{\"kernel\": \"linear\", \"C\": 10, \"epsilon\": 0.01, \"probability\": true}";
json load "rx_i" (ml model predict_interval "rx_cal" "[[22]]" 0.95);
json load "rx_j" (ml model predict_interval "rx_svr" "[[22]]" 0.95);
-> "[res]" (concat "OLS at 22 mg/L: " (math round (json select "rx_i" "/[0]/prediction") 4) " [" (math round (json select "rx_i" "/[0]/lower") 4) " - " (math round (json select "rx_i" "/[0]/upper") 4) "] | SVR: " (math round (json select "rx_j" "/[0]/prediction") 4) " [" (math round (json select "rx_j" "/[0]/lower") 4) " - " (math round (json select "rx_j" "/[0]/upper") 4) "]");
json unload "rx_i"; json unload "rx_j"; ml model delete "rx_cal"; ml model delete "rx_svr";
[res]
Result
mentdb
OLS at 22 mg/L: 2.211 [2.1443 - 2.2777] | SVR: 2.2033 [2.1163 - 2.2904]
Go further
  • The prediction interval is wider than the confidence interval of the line: it includes the noise of one new measure.
  • Invert the calibration with ml train linear on [signal] -> concentration to read unknown samples.
executed on MentDB

Is the best model really better?

Like the Weka Experimenter: the same 10 x 10-fold cross validation for five algorithms, and a corrected resampled paired t-test of each one against naive Bayes, to know which differences are significant and which are noise.

Pipeline
Script
admin
#1. Measures and species;
-> "[X]" "[[-0.08, 1.37, -0.34, -1.78], [2.26, 1.1, 2.61, 0.57], [0.95, 1.53, 1.25, -0.14], [-0.26, 0.92, -0.03, -1.17], [2.72, 0.57, 1.91, 1.61], [1.6, 0.48, 1.0, -1.47], [1.52, -0.5, -0.79, 1.95], [0.5, 0.63, 0.02, -0.42], [0.37, 2.72, 0.95, 0.46], [0.68, -1.46, -0.54, -0.14], [0.59, 0.0, 0.3, 1.13], [0.47, 2.2, 1.15, 1.16], [-0.02, 1.0, -0.26, -0.62], [2.71, 1.27, 3.45, 0.73], [1.29, 2.81, 4.02, 0.64], [0.86, -0.26, -0.36, -2.28], [2.48, 0.63, 1.77, -1.79], [0.66, 1.59, 0.53, 1.04], [1.23, -1.06, -1.13, 1.88], [0.99, 1.51, 2.02, 2.02], [-0.15, 1.13, -0.07, 1.35], [-1.05, -0.59, 0.31, 1.25], [1.59, 1.13, 2.03, 1.3], [1.49, 2.42, 3.99, 0.6], [-1.06, -0.39, 0.73, 0.26], [1.78, 0.83, 2.53, 0.07], [1.55, 2.53, 3.87, 1.34], [0.9, 0.47, 0.04, 1.41], [2.81, 0.53, 0.69, 0.06], [-0.01, 1.89, -0.15, -0.73], [1.04, 0.21, 0.63, -0.87], [1.93, 1.86, 3.73, -0.63], [0.28, 1.56, 0.45, 0.82], [0.24, 0.8, 0.49, 0.24], [3.3, 1.29, 4.22, -0.05], [1.09, 1.42, 1.17, 0.15], [-0.37, 0.65, -0.39, 0.22], [2.29, 0.17, 0.48, -0.63], [0.71, 2.84, 2.16, 0.11], [-0.62, 0.74, 0.18, -2.12], [3.07, 0.8, 2.83, 1.31], [0.16, 3.71, 0.45, -0.15], [0.68, 0.02, 0.13, -0.42], [1.14, 0.61, 0.4, 1.49], [1.32, 3.4, 4.14, -2.03], [-2.01, -0.25, 0.14, 0.48], [0.75, -0.68, -0.39, -0.3], [1.72, 2.84, 5.04, -1.14], [-0.7, 2.19, -1.58, 0.41], [2.21, 0.57, 1.24, -0.47], [0.62, 0.24, 0.69, -0.0], [-1.12, 0.57, -0.4, -1.37], [1.62, 1.98, 3.39, 0.55], [2.02, 1.98, 3.92, 1.74], [-0.07, 0.02, 0.23, 0.14], [0.96, 2.34, 2.25, 1.95], [1.57, 1.26, 2.48, -0.07], [-0.7, 0.07, -0.68, 0.52], [1.42, 0.36, 0.28, 0.8], [1.43, 3.09, 4.96, 0.4], [0.18, 0.38, -0.15, -0.45], [2.79, -0.2, -1.11, -0.25], [1.33, 3.45, 4.41, -0.85], [0.18, -0.05, 0.1, -0.42], [2.28, 0.06, 0.47, 0.39], [1.62, 1.69, 2.33, 0.12], [0.04, 1.11, 0.38, 0.07], [2.05, -0.32, -0.64, 0.32], [0.71, 1.1, 1.2, -0.84], [1.61, 1.18, 2.4, -0.77], [2.67, -0.69, -1.91, -1.46], [0.42, 1.63, 1.0, 1.2], [-0.25, -0.17, -0.21, -1.07], [0.45, 0.39, 0.38, -0.69], [1.42, 1.03, 1.18, -0.45], [0.89, -0.4, -0.09, -0.27], [1.07, -1.26, -1.46, -0.89], [1.91, 2.48, 4.26, 0.94], [0.56, -2.57, -1.74, 0.33], [0.62, -0.09, 0.13, 0.34], [1.49, 2.63, 3.67, 2.5], [0.55, 2.02, 1.06, 0.65], [2.38, 0.44, 1.16, 0.34], [1.81, 2.36, 4.32, 0.57], [1.39, 0.14, 0.4, 0.22], [0.29, -0.25, -0.33, -1.32], [0.51, 0.47, 0.34, -0.46], [1.18, 0.42, 0.13, -0.14], [2.11, -0.05, -0.18, -1.01], [0.41, 2.32, 0.86, 1.29]]";
-> "[y]" "[\"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\", \"setosa\", \"versi\", \"virgi\"]";
json load "rx_c" (ml compare [X] [y] "[\"naive_bayes\", \"logistic\", {\"algo\": \"knn_classifier\", \"params\": {\"k\": 7}}, \"bayes_net\", \"one_r\"]");
-> "[res]" "";
for (-> "[i]" 0) (< [i] (json count "rx_c" "/results")) (++ "[i]") {
  -> "[p]" (concat "/results/[" [i] "]");
  -> "[res]" (concat [res] (json select "rx_c" (concat [p] "/algo")) " " (math round (json select "rx_c" (concat [p] "/mean")) 3) (if (> [i] 0) {concat " " (json select "rx_c" (concat [p] "/vs_baseline")) " p=" (math round (json select "rx_c" (concat [p] "/p_value")) 3)} {" baseline"}) " | ");
};
json unload "rx_c";
[res]
Result
mentdb
naive_bayes 0.731 baseline | logistic 0.702 same p=0.394 | knn_classifier 0.719 same p=0.756 | bayes_net 0.603 worse p=0.004 | one_r 0.572 worse p=0.017 |
Go further
  • The corrected t-test (Nadeau and Bengio) takes into account that the training sets overlap: the naive t-test finds too many differences.
  • Give {"k": 5, "repeats": 3} for the slow algorithms.
executed on MentDB

How many subjects for the study?

Plan a study before collecting the data: the number of subjects per group to detect a medium effect (d = 0.5) with a t test, a difference of proportions 65 % vs 45 %, and an ANOVA on 4 groups, with 80 % and 90 % power.

Pipeline
Script
admin
json load "rx_a" (stat sample_size "t_two" "{\"d\": 0.5}");
json load "rx_b" (stat sample_size "t_two" "{\"d\": 0.5, \"power\": 0.9}");
json load "rx_c" (stat sample_size "prop_two" "{\"p1\": 0.65, \"p2\": 0.45}");
json load "rx_d" (stat sample_size "anova" "{\"f\": 0.25, \"k\": 4}");
json load "rx_e" (stat power "t_two" "{\"d\": 0.5, \"n\": 30}");
-> "[res]" (concat "t test: " (json select "rx_a" "/n") " per group at 80 %, " (json select "rx_b" "/n") " at 90 % | proportions: " (json select "rx_c" "/n") " per group | ANOVA 4 groups: " (json select "rx_d" "/n") " per group | with only 30 per group, power = " (math round (json select "rx_e" "/power") 2));
json unload "rx_a"; json unload "rx_b"; json unload "rx_c"; json unload "rx_d"; json unload "rx_e";
[res]
Result
mentdb
t test: 64 per group at 80 %, 86 at 90 % | proportions: 96 per group | ANOVA 4 groups: 45 per group | with only 30 per group, power = 0.48
Go further
  • d = difference of the means / standard deviation: 0.2 small, 0.5 medium, 0.8 large.
  • Add 10 to 20 % for the subjects lost during the study.
executed on MentDB

Decide an A/B test

A new landing page (B) is tested against the current one (A): 120 conversions on 2400 visits against 156 on 2380. The z test, the 95 % interval of the difference and the Bayesian probability that B is better support the decision.

Pipeline
Script
admin
json load "rx_t" (analytics abtest 120 2400 156 2380);
json load "rx_p" (stat power "prop_two" "{\"p1\": 0.05, \"p2\": 0.065, \"n\": 2400}");
-> "[res]" (concat "A " (math round (* 100 (json select "rx_t" "/rate_a")) 2) " %, B " (math round (* 100 (json select "rx_t" "/rate_b")) 2) " % | lift " (math round (* 100 (json select "rx_t" "/lift")) 1) " % [" (math round (* 100 (json select "rx_t" "/lower")) 2) ", " (math round (* 100 (json select "rx_t" "/upper")) 2) "] points | p = " (math round (json select "rx_t" "/p_value") 4) " | P(B > A) = " (math round (json select "rx_t" "/probability_b_better") 3) " | power " (math round (json select "rx_p" "/power") 2));
json unload "rx_t"; json unload "rx_p";
[res]
Result
mentdb
A 5 %, B 6.55 % | lift 31.1 % [0.23, 2.88] points | p = 0.0212 | P(B > A) = 0.989 | power 0.61
Go further
  • Fix the sample size before the test with stat sample_size "prop_two": stopping when p < 0.05 inflates the false positives.
  • The Bayesian probability is easier to explain than the p-value.
executed on MentDB

Bayesian logistic regression by MCMC

Three doses of a product are given to 20 animals each, with 2, 8 and 17 deaths: the log posterior of the logistic model is written directly, then sampled by MCMC (4 chains); the dose that kills half of the animals (LD50 = -a / b) is computed from the posterior means.

Pipeline
01 · mcmcAdaptive Metropolis-Hastingsml mcmc→02 · ld50LD50 from the posteriormath
Script
admin
#1. Log likelihood of the binomial logistic model, flat priors;
-> "[lp]" "2*(a+b*1) - 20*ln(1+exp(a+b*1)) + 8*(a+b*2) - 20*ln(1+exp(a+b*2)) + 17*(a+b*3) - 20*ln(1+exp(a+b*3))";
json load "rx_m" (ml mcmc [lp] "{\"a\": 0, \"b\": 0}" "{\"iterations\": 4000}");
-> "[a]" (json select "rx_m" "/parameters/[0]/mean");
-> "[b]" (json select "rx_m" "/parameters/[1]/mean");
-> "[res]" (concat "b = " (math round [b] 2) " [" (math round (json select "rx_m" "/parameters/[1]/lower") 2) ", " (math round (json select "rx_m" "/parameters/[1]/upper") 2) "] | LD50 about dose " (math round (/ (* -1 [a]) [b]) 2) " | R-hat " (math round (json select "rx_m" "/parameters/[1]/rhat") 3) ", ESS " (math round (json select "rx_m" "/parameters/[1]/ess") 0) " | converged " (json select "rx_m" "/converged"));
json unload "rx_m";
[res]
Result
mentdb
b = 2.11 [1.21, 3.19] | LD50 about dose 2.16 | R-hat 1.002, ESS 2228 | converged true
Go further
  • ln is the natural logarithm in the expressions (log is the base 10 logarithm).
  • Add {"samples": 1000} to get draws and compute the posterior of any function of the parameters, like the LD50 itself.
executed on MentDB

Repeated measures: each subject has its own slope

Eight patients are measured every week for 6 weeks. A random intercept only lets each patient start at its own level; the random slope also lets each one progress at its own speed. The AIC of the two REML models tells whether the slopes differ between patients.

Pipeline
Script
admin
#1. Week, score and patient;
-> "[X]" "[[0], [1], [2], [3], [4], [5], [0], [1], [2], [3], [4], [5], [0], [1], [2], [3], [4], [5], [0], [1], [2], [3], [4], [5], [0], [1], [2], [3], [4], [5], [0], [1], [2], [3], [4], [5], [0], [1], [2], [3], [4], [5], [0], [1], [2], [3], [4], [5]]";
-> "[y]" "[46.3, 43.5, 48.3, 46.8, 53.4, 53.7, 56.0, 57.4, 59.1, 62.8, 63.1, 66.9, 52.2, 59.1, 59.0, 61.1, 65.3, 67.3, 51.5, 51.1, 53.9, 57.2, 61.8, 61.0, 45.3, 52.9, 60.4, 68.8, 70.0, 73.7, 52.1, 55.7, 57.7, 60.2, 61.5, 66.5, 55.3, 60.2, 62.8, 63.6, 67.5, 72.8, 49.9, 52.2, 54.8, 54.5, 62.7, 62.9]";
-> "[g]" "[\"p1\", \"p1\", \"p1\", \"p1\", \"p1\", \"p1\", \"p2\", \"p2\", \"p2\", \"p2\", \"p2\", \"p2\", \"p3\", \"p3\", \"p3\", \"p3\", \"p3\", \"p3\", \"p4\", \"p4\", \"p4\", \"p4\", \"p4\", \"p4\", \"p5\", \"p5\", \"p5\", \"p5\", \"p5\", \"p5\", \"p6\", \"p6\", \"p6\", \"p6\", \"p6\", \"p6\", \"p7\", \"p7\", \"p7\", \"p7\", \"p7\", \"p7\", \"p8\", \"p8\", \"p8\", \"p8\", \"p8\", \"p8\"]";
json load "rx_1" (stat lmm [X] [y] [g]);
json load "rx_2" (stat lmm [X] [y] [g] "{\"random_slopes\": [0]}");
-> "[res]" (concat "mean progress " (math round (json select "rx_2" "/fixed_effects/[1]/estimate") 2) " per week (se " (math round (json select "rx_2" "/fixed_effects/[1]/se") 2) ", random intercept only: se " (math round (json select "rx_1" "/fixed_effects/[1]/se") 2) ") | sd between patients: level " (math round (json select "rx_2" "/random_effects/[0]/sd") 2) ", slope " (math round (json select "rx_2" "/random_effects/[1]/sd") 2) " | AIC " (math round (json select "rx_1" "/aic") 1) " -> " (math round (json select "rx_2" "/aic") 1) " | p1 slope " (math round (+ (json select "rx_2" "/fixed_effects/[1]/estimate") (json select "rx_2" "/group_effects/p1/x0")) 2));
json unload "rx_1"; json unload "rx_2";
[res]
Result
mentdb
mean progress 2.93 per week (se 0.43, random intercept only: se 0.23) | sd between patients: level 3.9, slope 1.13 | AIC 259 -> 243.7 | p1 slope 1.83
Go further
  • When the slopes really differ, the random intercept model underestimates the standard error of the mean slope: the random slope model is the honest one.
  • Both models are fitted by REML with the same fixed effects, so their AIC can be compared.
executed on MentDB

Missing values: multiple imputation instead of deleting the rows

The blood pressure is explained by the age and the body mass index, but the BMI is missing for a third of the patients, more often for the older ones. Deleting these rows loses power; multiple imputation (MICE) fills the BMI 20 times from the other columns, fits the regression on each completed table and pools the results by Rubin's rules.

Pipeline
Script
admin
#1. Systolic pressure, age, BMI (null = missing);
-> "[d]" "[[165.7, 76.0, 22.9], [148.4, 46.9, 21.5], [160.7, 54.6, null], [172.7, 63.0, null], [136.0, 30.5, 24.5], [141.1, 53.2, 21.2], [158.2, 61.2, 25.9], [135.5, 38.5, 23.6], [132.7, 31.2, 26.1], [152.4, 43.0, 26.7], [153.1, 53.2, null], [162.9, 53.6, 25.3], [149.9, 44.4, 29.1], [152.3, 59.9, null], [149.3, 54.0, 20.1], [146.4, 42.2, 30.3], [181.0, 66.2, null], [145.4, 54.3, 28.4], [123.6, 59.9, 17.3], [156.0, 41.4, 19.4], [148.9, 51.7, 27.6], [153.8, 51.7, null], [131.8, 42.8, 25.2], [146.3, 44.1, 18.1], [157.1, 36.1, 32.4], [135.8, 26.5, 19.6], [142.4, 44.3, 23.4], [149.8, 61.8, 21.9], [159.1, 63.2, 30.2], [179.8, 63.3, null], [140.7, 36.1, 23.4], [158.0, 65.3, null], [157.2, 34.9, 30.1], [157.9, 68.8, null], [146.4, 45.6, 26.6], [134.2, 37.0, 20.4], [170.0, 42.4, 33.8], [151.4, 62.7, 19.7], [140.4, 42.1, 23.6], [160.3, 60.1, null], [132.7, 30.7, 28.6], [155.6, 52.1, null], [155.3, 48.0, 33.7], [162.8, 49.9, 29.7], [155.7, 55.8, 23.9], [143.7, 42.7, 24.8], [163.7, 48.2, null], [154.9, 68.2, 26.6], [162.3, 48.7, 27.5], [153.2, 41.1, 34.3], [171.1, 66.4, null], [149.4, 64.9, 25.4], [144.0, 57.1, null], [163.9, 44.2, 32.3], [139.7, 40.6, 27.1], [143.3, 39.9, 22.0], [133.4, 49.1, 16.8], [168.5, 73.2, null], [152.3, 53.4, 18.7], [143.6, 53.0, null], [146.4, 46.5, 26.6], [150.3, 61.6, null], [148.4, 42.9, 31.1], [132.1, 52.5, 16.8], [127.7, 40.9, 22.6], [130.1, 33.1, 22.9], [149.4, 56.4, null], [156.7, 52.8, 17.8], [140.6, 45.4, 20.0], [166.0, 57.2, 31.3], [154.9, 44.1, 25.2], [146.9, 55.6, 24.1], [139.5, 50.0, 20.5], [153.1, 48.4, 29.1], [143.2, 34.9, 25.5], [155.2, 40.3, 32.5], [146.1, 55.6, 24.9], [153.3, 23.0, 31.5], [171.3, 67.3, 31.7], [130.8, 52.4, 24.2], [122.2, 40.5, 25.3], [138.8, 33.4, 27.1], [151.2, 49.5, 24.5], [157.7, 43.2, 31.1], [156.2, 50.1, 26.0], [124.8, 31.1, 18.6], [160.4, 63.7, 32.2], [158.9, 62.6, 30.0], [154.2, 59.3, 28.2], [157.8, 35.5, 27.2], [137.1, 54.1, 27.9], [139.4, 33.7, 20.9], [125.2, 35.3, 24.6], [137.7, 34.6, 26.9], [166.9, 58.5, null], [141.1, 42.1, 21.8], [128.8, 15.6, 24.2], [152.9, 44.6, 20.0], [161.7, 58.2, 30.7], [160.9, 53.4, 33.1], [153.3, 58.9, 30.6], [157.5, 34.7, 33.9], [146.5, 49.0, null], [143.2, 39.3, 21.3], [136.3, 35.4, 28.6], [155.5, 63.7, null], [152.1, 63.1, 21.3], [137.5, 44.7, 26.5], [163.6, 66.1, null], [141.6, 47.0, 21.1], [124.9, 39.0, 21.2], [126.8, 30.0, 24.6], [158.3, 46.3, 27.8], [161.0, 50.7, 30.1], [151.9, 68.7, null], [150.2, 30.0, 29.1], [163.6, 62.6, null], [133.3, 44.1, 21.4], [155.9, 78.9, 22.7], [145.6, 18.7, 28.3], [135.8, 48.8, 22.3], [155.2, 45.7, 29.5], [153.6, 51.5, null], [150.2, 52.2, 18.8], [159.8, 64.4, 30.3], [156.2, 63.6, 20.4], [163.5, 62.1, null], [134.9, 38.4, 18.5], [144.7, 47.2, 18.2], [150.1, 36.6, 29.7], [154.2, 69.9, 23.0], [147.2, 53.2, null], [173.1, 61.3, 22.9], [152.4, 73.1, null], [139.5, 42.7, 25.3], [151.8, 41.7, 27.6], [174.9, 62.3, null], [144.2, 44.4, 27.1], [144.9, 43.5, 23.1], [174.6, 54.8, null], [145.0, 42.0, 24.9], [148.7, 49.1, 25.9], [149.7, 43.7, 29.8], [161.5, 46.5, 25.0], [162.5, 56.5, null], [170.3, 54.6, 32.9], [142.8, 35.7, 25.4], [147.8, 46.6, 25.6], [141.8, 33.5, 19.7], [149.1, 50.1, 25.1], [151.6, 64.6, 25.4], [132.2, 30.3, 27.0], [144.4, 56.4, 18.9], [136.8, 40.1, 23.0], [157.9, 51.4, 29.0], [151.0, 47.7, 21.7], [161.8, 56.9, null], [156.1, 57.9, null], [147.4, 46.4, null], [146.4, 61.8, null], [149.8, 52.6, 23.8], [149.3, 45.8, 28.6], [151.6, 37.2, 32.0], [125.8, 43.1, 25.0], [169.8, 60.5, 31.9], [157.9, 51.2, 26.4], [127.9, 24.0, 27.0], [162.9, 42.5, 28.4], [146.7, 55.2, 22.2], [127.7, 48.8, 19.9], [163.6, 39.3, 31.2], [144.9, 25.9, 20.9], [149.3, 37.9, 23.6], [146.3, 38.8, 26.0], [149.9, 52.4, null], [155.6, 65.1, null], [140.6, 46.7, 23.0], [168.0, 63.5, null], [154.8, 44.5, 30.0], [145.3, 43.1, 20.9], [139.4, 44.0, 24.2], [137.2, 41.9, 22.2], [161.2, 50.9, null], [156.1, 62.8, null], [121.4, 48.2, 25.3], [166.8, 68.8, 20.2], [140.7, 54.1, 22.8], [154.3, 62.6, null], [161.9, 46.4, 28.5], [160.6, 63.3, 26.2], [172.3, 46.6, 24.7], [134.1, 38.0, null], [165.2, 48.7, 28.8], [157.7, 52.3, 21.8], [154.2, 48.9, 27.9], [149.7, 73.3, 26.5], [154.7, 29.8, 33.0], [146.5, 56.0, 24.8], [142.9, 51.0, 19.6], [158.1, 68.0, 29.5]]";
json load "rx_i" (stat mice [d] "{\"m\": 20, \"analysis\": {\"y\": 0}}");
-> "[res]" (concat "missing BMI: " (json select "rx_i" "/missing_by_column/[2]") " / " (json select "rx_i" "/n") " | pooled: age " (math round (json select "rx_i" "/pooled/[1]/estimate") 3) " (se " (math round (json select "rx_i" "/pooled/[1]/se") 3) "), BMI " (math round (json select "rx_i" "/pooled/[2]/estimate") 3) " (se " (math round (json select "rx_i" "/pooled/[2]/se") 3) ", fraction of missing information " (math round (json select "rx_i" "/pooled/[2]/fmi") 2) ")");
json unload "rx_i";
[res]
Result
mentdb
missing BMI: 42 / 200 | pooled: age 0.558 (se 0.054), BMI 1.332 (se 0.148, fraction of missing information 0.11)
Go further
  • For another analysis, ask {"return_data": true}, run it on each completed table and pool the m results with stat pool.
  • Predictive mean matching (pmm, the default) only imputes values seen in the data: no negative BMI, integers stay integers.
executed on MentDB

Repeated measures: time, group and their interaction

Sixteen subjects, half on a diet, have their weight measured 4 times. A mixed (split-plot) ANOVA tests the time, the group and the interaction time x group (do the groups evolve differently?); the Greenhouse-Geisser correction protects against a violation of sphericity.

Pipeline
Script
admin
#1. One row per subject: the 4 measures;
-> "[w]" "[[118.3, 116.1, 114.2, 104.1], [114.5, 112.3, 109.1, 103.3], [118.5, 119.1, 112.6, 111.6], [122.2, 117.7, 113.8, 109.6], [129.4, 128.6, 122.0, 118.4], [116.6, 110.4, 112.2, 108.7], [131.0, 122.3, 115.7, 119.0], [119.4, 116.1, 116.2, 114.8], [121.0, 123.4, 123.2, 117.6], [122.1, 123.3, 123.5, 121.8], [113.0, 108.5, 112.1, 105.8], [116.5, 115.2, 116.3, 115.6], [130.6, 131.2, 125.7, 129.5], [127.6, 126.9, 121.2, 119.5], [114.7, 118.1, 110.2, 113.7], [125.0, 121.6, 121.9, 124.1]]";
json load "rx_r" (stat rm_anova [w] "{\"groups\": [\"diet\", \"diet\", \"diet\", \"diet\", \"diet\", \"diet\", \"diet\", \"diet\", \"control\", \"control\", \"control\", \"control\", \"control\", \"control\", \"control\", \"control\"]}");
-> "[res]" (concat "time: F " (math round (json select "rx_r" "/anova/[2]/f") 1) ", p GG " (math round (json select "rx_r" "/sphericity/conditions_p_gg") 6) " | time x group: F " (math round (json select "rx_r" "/anova/[3]/f") 1) ", p GG " (math round (json select "rx_r" "/sphericity/interaction_p_gg") 6) " | group: p " (math round (json select "rx_r" "/anova/[0]/p_value") 4) " | Mauchly p " (math round (json select "rx_r" "/sphericity/p_value") 3) ", epsilon GG " (math round (json select "rx_r" "/sphericity/greenhouse_geisser_epsilon") 2));
json unload "rx_r";
[res]
Result
mentdb
time: F 21.3, p GG 0 | time x group: F 6, p GG 0.00222 | group: p 0.1959 | Mauchly p 0.895, epsilon GG 0.93
Go further
  • The interaction is the question of a trial: a significant time effect alone only says that the weights change.
  • With several outcomes measured once (weight, fat mass, waist), compare the groups with stat manova.
executed on MentDB

Small sample: Bayesian logistic regression and the probability of an effect

Thirty patients, a dose and a genotype: with so few events the maximum likelihood estimates are unstable and the p-value answers an indirect question. The Bayesian logistic regression gives the posterior of each coefficient, its credible interval and directly the probability that the dose increases the response.

Pipeline
Script
admin
#1. Dose and genotype, response (0 / 1);
-> "[X]" "[[0.15, 0], [1.48, 0], [1.99, 0], [2.31, 1], [0.23, 0], [1.5, 0], [3.34, 0], [2.39, 1], [1.51, 0], [0.75, 0], [1.18, 0], [0.41, 0], [2.49, 1], [2.66, 1], [3.39, 1], [2.37, 1], [2.69, 0], [2.21, 0], [3.6, 1], [0.17, 1], [0.33, 0], [3.49, 0], [1.23, 1], [0.15, 0], [3.17, 0], [0.07, 1], [2.13, 1], [2.81, 1], [1.54, 0], [0.82, 1]]";
-> "[y]" "[0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 1, 0, 0, 1, 1]";
json load "rx_b" (stat bayes_glm [X] [y] "binomial");
json load "rx_g" (stat glm [X] [y] "binomial");
-> "[res]" (concat "dose: posterior median " (math round (json select "rx_b" "/coefficients/[1]/median") 2) " [" (math round (json select "rx_b" "/coefficients/[1]/lower") 2) ", " (math round (json select "rx_b" "/coefficients/[1]/upper") 2) "], P(effect > 0) = " (math round (json select "rx_b" "/coefficients/[1]/probability_positive") 3) " | maximum likelihood " (math round (json select "rx_g" "/coefficients/[1]/estimate") 2) ", p = " (math round (json select "rx_g" "/coefficients/[1]/p_value") 3) " | R-hat " (math round (json select "rx_b" "/coefficients/[1]/rhat") 3));
json unload "rx_b"; json unload "rx_g";
[res]
Result
mentdb
dose: posterior median 0.43 [-0.27, 1.11], P(effect > 0) = 0.872 | maximum likelihood 0.37, p = 0.287 | R-hat 1.001
Go further
  • The prior normal(0, 2.5) on the standardized slopes pulls the extreme estimates toward 0; {"prior_sd": 1} pulls harder, {"prior_sd": 10} almost not.
  • Check R-hat < 1.05 and ess > 400 before reading the results; increase iterations otherwise.
executed on MentDB

Size of a survival trial: events, patients and power

A trial must detect a hazard ratio of 0.75 between two arms with a power of 80 % (two-sided 5 %). The log-rank test needs a number of events (Schoenfeld); the number of patients depends on the share of patients with an event during the follow-up. The power of the trial if only 500 patients can be included is also computed.

Pipeline
Script
admin
#1. 35 % of the patients have an event during the follow-up;
json load "rx_n" (stat sample_size "logrank" "{\"hr\": 0.75, \"p_event\": 0.35}");
json load "rx_n2" (stat sample_size "logrank" "{\"hr\": 0.75, \"p_event\": 0.35, \"ratio\": 0.667}");
json load "rx_w" (stat power "logrank" "{\"hr\": 0.75, \"p_event\": 0.35, \"n\": 500}");
-> "[res]" (concat "1:1 -> " (json select "rx_n" "/events") " events, " (json select "rx_n" "/n") " patients | 2:1 -> " (json select "rx_n2" "/n") " patients | power with 500 patients: " (math round (* 100 (json select "rx_w" "/power")) 1) " %");
json unload "rx_n"; json unload "rx_n2"; json unload "rx_w";
[res]
Result
mentdb
1:1 -> 380 events, 1084 patients | 2:1 -> 1220 patients | power with 500 patients: 47.7 %
Go further
  • Add 10 to 20 % for the patients lost to follow-up.
  • A longer follow-up raises p_event and lowers the number of patients for the same number of events.
executed on MentDB

A/B test of a landing page: probability to win and expected loss

Two versions of a landing page: A (current) converts 312 visitors out of 4 120, B (new) 368 out of 4 087. Instead of a p-value, the Bayesian A/B test gives the probability that B is better, the interval of the lift, and the expected loss of each decision: switch to B when its expected loss is below the threshold of caring (here 0.1 point of conversion).

Pipeline
Script
admin
json load "rx_b" (stat bayes_ab 312 4120 368 4087);
-> "[loss]" (json select "rx_b" "/expected_loss_choosing_b");
-> "[decision]" "keep collecting data";
if (< [loss] 0.001) {-> "[decision]" "switch to B";};
-> "[res]" (concat "conversion A " (math round (* 100 (json select "rx_b" "/rate_a")) 2) " %, B " (math round (* 100 (json select "rx_b" "/rate_b")) 2) " % | P(B > A) = " (math round (json select "rx_b" "/prob_b_beats_a") 4) " | lift " (math round (* 100 (json select "rx_b" "/lift_median")) 1) " % [" (math round (* 100 (json select "rx_b" "/lift_lower")) 1) ", " (math round (* 100 (json select "rx_b" "/lift_upper")) 1) "] | expected loss of B " (math round (* 100 [loss]) 3) " point -> " [decision]);
json unload "rx_b";
[res]
Result
mentdb
conversion A 7.59 %, B 9.02 % | P(B > A) = 0.9906 | lift 18.9 % [2.9, 37.5] | expected loss of B 0.002 point -> switch to B
Go further
  • The probability is exact (Evan Miller formula); prob_b_beats_a_mc is its Monte Carlo check.
  • With a prior from the past campaigns, give {"prior": [a, b]}; stat bayes_factor_t compares two means in the same spirit, stat prop_test gives the classical test.
executed on MentDB

Multi-center trial: partial pooling of the effects of 8 centers

Each of 8 hospitals estimated the effect of a treatment on the LDL cholesterol (mg/dL) with its standard error. The small centers give extreme and uncertain effects. The normal hierarchical model pools them partially: each center is pulled toward the common mean according to its precision, and tau measures the real heterogeneity between centers.

Pipeline
Script
admin
-> "[eff]" "[-6.2, -3.1, -9.8, -1.0, -4.5, -12.4, -2.7, -5.6]";
-> "[se]" "[2.8, 2.1, 4.5, 3.0, 2.4, 5.1, 2.6, 3.3]";
json load "rx_h" (stat bayes_hierarchical [eff] [se] (concat "{\"names\": " "[\"Paris\", \"Lyon\", \"Lille\", \"Nantes\", \"Bordeaux\", \"Toulouse\", \"Rennes\", \"Saint-Denis\"]" ", \"tau_prior\": \"half_cauchy\"}"));
-> "[res]" (concat "common effect mu " (math round (json select "rx_h" "/mu/mean") 2) " [" (math round (json select "rx_h" "/mu/lower") 2) ", " (math round (json select "rx_h" "/mu/upper") 2) "] | tau median " (math round (json select "rx_h" "/tau/median") 2) " | Toulouse " (json select "rx_h" "/groups/[5]/estimate") " -> " (math round (json select "rx_h" "/groups/[5]/posterior_mean") 2) " (pooling " (math round (json select "rx_h" "/groups/[5]/shrinkage") 2) ") | Lyon " (json select "rx_h" "/groups/[1]/estimate") " -> " (math round (json select "rx_h" "/groups/[1]/posterior_mean") 2) " (pooling " (math round (json select "rx_h" "/groups/[1]/shrinkage") 2) ")");
json unload "rx_h";
[res]
Result
mentdb
common effect mu -4.61 [-7.28, -2.21] | tau median 1.24 | Toulouse -12.4 -> -5.41 (pooling 0.89) | Lyon -3.1 -> -4.08 (pooling 0.69)
Go further
  • The half-Cauchy prior on tau (scale 5) is the recommendation of Gelman (2006) when there are few groups; stat meta gives the classical random effects meta-analysis.
  • To compare models fitted with ml mcmc, give their pointwise log-likelihood to stat waic (WAIC and PSIS-LOO).
executed on MentDB

Effect of a speed limit on road accidents: interrupted time series

Monthly road injuries on a network over 3 years; the speed limit dropped at month 18. The segmented regression separates the immediate level change from the change of trend, with Newey-West standard errors because the monthly counts are autocorrelated.

Pipeline
Script
admin
-> "[y]" "[51.7, 51.1, 49.1, 49.2, 52.5, 49.4, 49.9, 49.7, 50.0, 50.5, 49.5, 48.1, 49.9, 51.8, 49.7, 47.8, 54.2, 48.2, 42.7, 41.1, 42.6, 41.5, 38.5, 42.0, 40.4, 36.8, 36.8, 36.0, 39.5, 37.3, 36.7, 37.6, 37.9, 34.9, 34.9, 33.4]";
json load "rx_i" (stat its [y] 18);
-> "[res]" (concat "baseline trend " (math round (json select "rx_i" "/baseline_slope") 2) " per month | level change " (math round (json select "rx_i" "/level_change/estimate") 2) " [" (math round (json select "rx_i" "/level_change/lower") 2) ", " (math round (json select "rx_i" "/level_change/upper") 2) "], p = " (math round (json select "rx_i" "/level_change/p_value") 4) " | slope change " (math round (json select "rx_i" "/slope_change/estimate") 2) " per month, p = " (math round (json select "rx_i" "/slope_change/p_value") 4) " | Durbin-Watson " (math round (json select "rx_i" "/durbin_watson") 2));
json unload "rx_i";
[res]
Result
mentdb
baseline trend -0.04 per month | level change -7.51 [-8.85, -6.16], p = 0 | slope change -0.42 per month, p = 0 | Durbin-Watson 2.43
Go further
  • With a comparable network where nothing changed, stat did or stat synthetic_control (several donors) remove the common trends.
  • Seasonality: add a seasonal adjustment before (ts decompose) for monthly data with strong seasons.
executed on MentDB

Opinion poll: raking on sex and age, weighted estimate and margin of error

An online poll of 120 people over-represents the men and the young. Raking on the census margins (sex and age) corrects the sample; the weighted share of "yes" changes, and the margin of error must use the design effect of the weights (the effective sample size is smaller than 120).

Pipeline
Script
admin
-> "[rows]" "[{\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"18-34\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 1}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"18-34\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"18-34\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"18-34\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 1}, {\"sex\": \"F\", \"age\": \"18-34\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 1}, {\"sex\": \"F\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"18-34\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"F\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"18-34\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"18-34\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"18-34\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"18-34\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"18-34\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"18-34\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"18-34\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"18-34\", \"yes\": 1}, {\"sex\": \"F\", \"age\": \"18-34\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"18-34\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"18-34\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"18-34\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"18-34\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"F\", \"age\": \"18-34\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"18-34\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 1}, {\"sex\": \"F\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 1}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"18-34\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"F\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"F\", \"age\": \"35-64\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 0}, {\"sex\": \"M\", \"age\": \"65+\", \"yes\": 1}]";
json load "rx_w" (survey rake [rows] "{\"sex\": {\"M\": 0.48, \"F\": 0.52}, \"age\": {\"18-34\": 0.27, \"35-64\": 0.47, \"65+\": 0.26}}");
#The answers in the order of the rows;
json load "rx_r" [rows];
json load "rx_y" "[]";
for (-> "[i]" 0) (< [i] (json count "rx_r" "/")) (++ "[i]") {
  json iarray "rx_y" "/" (json select "rx_r" (concat "/[" [i] "]/yes")) NUM;
};
json load "rx_e" (survey estimate (json doc "rx_y") (json select "rx_w" "/weights"));
json load "rx_m" (survey margin (json select "rx_e" "/estimates/[0]/mean") 120 (concat "{\"deff\": " (json select "rx_w" "/design_effect_kish") "}"));
-> "[res]" (concat "raking converged in " (json select "rx_w" "/iterations") " iterations, weights " (math round (json select "rx_w" "/min_weight") 2) " to " (math round (json select "rx_w" "/max_weight") 2) ", effective n " (math round (json select "rx_w" "/effective_n") 1) " | yes: weighted " (math round (* 100 (json select "rx_e" "/estimates/[0]/mean")) 1) " % (se " (math round (* 100 (json select "rx_e" "/estimates/[0]/se")) 1) ") | margin of error with the design effect: +/- " (math round (* 100 (json select "rx_m" "/margin_of_error")) 1) " points");
json unload "rx_w"; json unload "rx_r"; json unload "rx_y"; json unload "rx_e"; json unload "rx_m";
[res]
Result
mentdb
raking converged in 5 iterations, weights 0.62 to 2.01, effective n 101.5 | yes: weighted 36.9 % (se 4.8) | margin of error with the design effect: +/- 9.4 points
Go further
  • survey sample_size {"deff": ...} plans the next wave with the design effect observed here.
  • With strata or clusters (face to face polls by area), give them to survey estimate {"strata": [...], "clusters": [...]}.

Also in Engineering & data

TR
Telecom & radar engineer
Link budgets in dB, radar equation, detection thresholds and spectra of modulated signals.
11 recipes
CE
Control engineer
Transfer functions, margins, PID and state feedback: complex numbers, CAS, ODE and matrices.
10 recipes
RE
Robotics engineer
Kinematics, Jacobians and trajectories: matrices, vectors, non-linear systems and splines.
10 recipes
EE
Electrical engineer
Impedances, three-phase power and cable sizing with complex numbers and units.
10 recipes
ES
Electronics & signal engineer
Filter design, spectra, peak detection and SNR on sampled signals.
12 recipes
IS
IoT & sensor engineer
Sensor calibration, smoothing, drift detection and measurement uncertainty.
18 recipes
MC
Mechanical & civil engineer
Deflection, stresses, safety factors and concrete strength statistics.
13 recipes
IQ
Industrial quality engineer
SPC control limits, process capability and tests on machines.
20 recipes
DS
Data scientist
Regression, correlation, feature scaling and drift tests.
22 recipes
DQ
Data quality analyst
Outliers, format checks, normalization and duplicate detection.
13 recipes