Application developer

Inventory of the web applications

Before adding an application, the developer lists the applications on the http and https ports, checks the admin application and its virtual hosts, and counts the remote IPs seen and blocked by the web server.

#1. Web applications on the http and https ports;
json load "rx_http" (app show "http");
json load "rx_https" (app show "https");
#2. The admin application and its virtual hosts;
-> "[admin]" (app exist "https" "admin");
json load "rx_vh" (app vhost show "https" "admin");
#3. Visitors and blocked IPs seen by the web server;
-> "[ips]" (app webserver show_all_remote_ips);
-> "[blocked]" (app webserver show_all_blocked_remote_ips);
concat "{\"http_apps\": " (json count "rx_http" /) ", \"https_apps\": " (json count "rx_https" /) ", \"https_names\": \"" (json select "rx_https" "/[0]") "\", \"admin_app\": " [admin] ", \"admin_vhosts\": " (json count "rx_vh" /) ", \"vhost_example_com\": " (app vhost exist "https" "admin" "admin.example.com") ", \"remote_ips\": " (string regexp_count [ips] "\]:[0-9]+:") ", \"blocked_ips\": " (string regexp_count [blocked] "\]:[0-9]+:") "}"

{"http_apps": 0, "https_apps": 1, "https_names": "admin", "admin_app": 1, "admin_vhosts": 0, "vhost_example_com": 0, "remote_ips": 1, "blocked_ips": 0}

Audit of users, groups and script rights

Rights review before a release: number of accounts and groups, the groups and scripts of admin, the members of the sys group, the scripts of the sample and public groups, and who can run the addition.post web service.

#1. Accounts and groups of the server;
json load "rx_users" (user show);
json load "rx_groups" (group show);
#2. Rights of the admin account;
json load "rx_ug" (user show groups "admin");
json load "rx_us" (user show scripts "admin");
#3. Members of the sys group and content of the sample group;
json load "rx_sys" (group get user "sys");
json load "rx_sample" (group get "sample");
json load "rx_pub" (group get script "public");
#4. Who can run the addition.post web service;
-> "[s]" "addition.post";
json load "rx_sg" (script show groups [s]);
json load "rx_su" (script show users [s]);
concat "{\"users\": " (json count "rx_users" /) ", \"groups\": " (json count "rx_groups" /) ", \"admin_exists\": " (user exist "admin") ", \"admin_groups\": " (json count "rx_ug" /) ", \"admin_scripts\": " (json count "rx_us" /) ", \"sys_members\": " (json count "rx_sys" /) ", \"admin_in_sys\": " (group is granted user "admin" "sys") ", \"sample_scripts\": " (json count "rx_sample" /scripts) ", \"public_scripts\": " (json count "rx_pub" /) ", \"cluster_group\": " (group exist "cluster") ", \"ws_groups\": " (json count "rx_sg" /) ", \"ws_users\": " (json count "rx_su" /) ", \"ws_in_mentdb\": " (group is granted script [s] "mentdb") ", \"ws_for_ai\": " (script is granted [s] "ai") ", \"ws_for_public\": " (script is granted [s] "public") "}"

{"users": 5, "groups": 16, "admin_exists": 1, "admin_groups": 16, "admin_scripts": 670, "sys_members": 3, "admin_in_sys": 1, "sample_scripts": 41, "public_scripts": 2, "cluster_group": 1, "ws_groups": 2, "ws_users": 3, "ws_in_mentdb": 1, "ws_for_ai": 1, "ws_for_public": 0}

New web application on its own domain

The shop team gets a group, a developer account with a random password, a new https application from the default template served on shop.example.com, and all the scripts of the app.

#1. A group for the shop team and a developer account, random password;
group add "shop_dev";
-> "[pwd]" (string generate_random_str 20);
user create "anna.dev" [pwd];
group grant user "anna.dev" "shop_dev";
#2. The web application from the default template, served on its own domain;
app create "https" "shop" "default" "100";
app vhost add "https" "shop" "shop.example.com";
#3. All the scripts of the app go to the team, then reload the web server;
group grant_all script "app.100.template.shop." "shop_dev";
app webserver restart;
#4. Checks;
concat "app=" (app exist "https" "shop") " vhost=" (app vhost exist "https" "shop" "shop.example.com") " member=" (group is granted user "anna.dev" "shop_dev") " login=" (user check password "anna.dev" [pwd])

CRUD web services generated from a MySQL table

The orders table of the shop database needs select, insert, update and delete web services. The password goes to a private parameter, the connection is declared, then SCRUD generates the scripts and eval installs them.

#1. The database password lives in a private parameter;
parameter add "SHOP_DB_PWD" "change_me" "Password of the shop_app MySQL account" true;
#2. The MySQL connection of the shop;
cm set "shop_mysql" {execute "db.mysql.config.get"
	"[hostname]" "db.example.com"
	"[port]" "3306"
	"[database]" "shop"
	"[user]" "shop_app"
	"[password]" (parameter get value "SHOP_DB_PWD")
;};
#3. The connection exists and SCRUD can use it;
cm exist "shop_mysql" "db";
cm show_scrud;
#4. Generate the select, insert, update and delete scripts of the orders table and install them;
eval (scrud select "shop_mysql" "orders");
eval (scrud insert "shop_mysql" "orders");
eval (scrud update "shop_mysql" "orders");
eval (scrud delete "shop_mysql" "orders");
eval (scrud merge "shop_mysql" "orders");
#5. The parameters of the generated select web service;
script show params "shop_mysql.orders.select.get"

Version the scripts of a group

Before changing the VAT rule, the developer checks what changed, commits the script, works on a copy, swaps the names, commits the whole group, exports it and reads the history and the code of the commit.

#1. What changed in the group since the last commit: I insert, U update, D delete;
version check_group "shop_dev";
#2. Commit the script before touching it;
version commit_script "shop_dev" "shop.orders.total.post" "Before the VAT change";
#3. Work on a copy, then make it the official version;
script copy "shop.orders.total.post" post "shop.orders.total_v2";
script rename "shop.orders.total.post" post "shop.orders.total_old";
script rename "shop.orders.total_v2.post" post "shop.orders.total";
group grant script "shop.orders.total.post" "shop_dev";
script delete "shop.orders.total_old.post";
#4. Commit the whole group, export it to disk and read the history;
-> "[cid]" (version commit_group "shop_dev" "Release 1.4 - VAT 20 percent");
script export_group "shop_dev";
version show_commits "shop_dev" ".*.*" 10;
version show_scriptnames "shop_dev" [cid];
#5. The code of the script in this commit, ready to restore;
version show_script "shop_dev" [cid] "shop.orders.total.post"

Onboard a partner account on a web service

A partner must call the price web service. The script creates the service, a partner account with a generated password (never printed) in its own group, grants the script to the group, runs the checks, makes a first HTTPS call (refused until the account joins the REST API group) and removes everything.

#1. The web service the partner will call;
script create post "tuto.p4h.price" false 1
	(param
		(var "[sku]" {type is_varchar [sku] 12} "Product code" is_null:false is_empty:false "BK-0042")
		(var "[qty]" {type is_integer [qty] 4} "Quantity" is_null:false is_empty:false "3")
	;)
	"Price of an order line"
{
	concat "{\"sku\": \"" [sku] "\", \"total\": " (math round (* [qty] 19.9) 2) "}";
} "JSON price";
#2. Partner account with a generated password, never printed, in its own group;
-> "[pwd]" (string generate_random_str 24);
user create "tuto_p4h_acme" [pwd];
group add "tuto_p4h_partners";
group grant user "tuto_p4h_acme" "tuto_p4h_partners";
group grant script "tuto.p4h.price.post" "tuto_p4h_partners";
#3. Checks before sending the access to the partner;
-> "[checks]" (concat "password ok: " (user check password "tuto_p4h_acme" [pwd]) " | granted on the script: " (script is granted "tuto.p4h.price.post" "tuto_p4h_acme") " | groups: " (string replace (string replace (user show groups "tuto_p4h_acme") " " "") [_n_] "") " | secret key: " (string length (user secret_key "tuto_p4h_acme")) " chars, masked");
#4. First HTTPS call: refused until the account joins the REST API group;
try {
	-> "[http]" (rest https "POST" "https://localhost:9999/api/tuto/p4h/price" "sku=BK-0042&qty=12" (concat "{\"x-user\": \"tuto_p4h_acme\", \"x-password\": \"" [pwd] "\"}") "[]" "utf-8");
} {
	-> "[http]" (string regexp_extract [err] "HTTP [0-9]+: [^(]+" 0);
} "[err]";
#5. Clean up;
group ungrant script "tuto.p4h.price.post" "tuto_p4h_partners";
group ungrant user "tuto_p4h_acme" "tuto_p4h_partners";
group remove "tuto_p4h_partners";
user delete "tuto_p4h_acme";
#user delete keeps the personal group of the user: remove it too;
group remove "tuto_p4h_acme";
script delete "tuto.p4h.price.post";
concat [checks] [_n_] "first call: " [http] [_n_] "left after cleanup: user " (user exist "tuto_p4h_acme") ", personal group " (group exist "tuto_p4h_acme")

password ok: 1 | granted on the script: 1 | groups: {"tuto_p4h_partners":0,"tuto_p4h_acme":0} | secret key: 24 chars, masked
first call: HTTP 403: Sorry, 'tuto_p4h_acme' is not in 'api-rest' group
left after cleanup: user 0, personal group 0

Contract of a web service for the front-end team

Before the front-end team codes against a new GET service, the developer gives them its contract: parameters, URL, MQL call, and a contract test with a valid call and a call refused by the enum check.

#1. A typed GET web service: stock of a product in a warehouse;
script create get "tuto.p4h.stock" false 1
	(param
		(var "[sku]" {type is_varchar [sku] 12} "Product code" is_null:false is_empty:false "BK-0042")
		(var "[site]" {type is_enum [site] "PAR,LYO,MRS"} "Warehouse code" is_null:false is_empty:false "LYO")
	;)
	"Stock of a product in a warehouse"
{
	-> "[stock]" (case (equal [site] "PAR") {120} (equal [site] "LYO") {45} {0});
	concat "{\"sku\": \"" [sku] "\", \"site\": \"" [site] "\", \"stock\": " [stock] "}";
} "JSON stock";
#2. The contract for the front-end team: parameters, URL, MQL calls;
json load "rx_p4h_params" (script show params "tuto.p4h.stock.get");
-> "[url]" (string regexp_extract (script generate url "tuto.p4h.stock.get") "https?://[^ ]+" 0);
-> "[exe]" (script generate execute "tuto.p4h.stock.get");
#3. Contract test: a valid call and a call with an unknown warehouse;
-> "[ok]" (execute "tuto.p4h.stock.get" "[sku]" "BK-0042" "[site]" "LYO");
try {execute "tuto.p4h.stock.get" "[sku]" "BK-0042" "[site]" "NYC";} {-> "[ko]" (string regexp_extract [err] "the variable [^;]+" 0);} "[err]";
script delete "tuto.p4h.stock.get";
concat (json count "rx_p4h_params" /) " parameters | URL " [url] [_n_] "MQL: " (string replace (string replace [exe] [_n_] " ") "	" "") [_n_] "valid call: " [ok] [_n_] "unknown site: " [ko]

2 parameters | URL https://localhost:9999/api/tuto/p4h/stock?x-token=&x-user=&sku=&site=
MQL: execute "tuto.p4h.stock.get" "[sku]" "BK-0042" "[site]" "LYO" ;
valid call: {"sku": "BK-0042", "site": "LYO", "stock": 45}
unknown site: the variable '[site]' in the script 'tuto.p4h.stock.get' is not valid (type is_enum [site] "PAR,LYO,MRS"

Feature flags of a web application

The new checkout page is released behind a flag. The script creates the documented flag and a private payment key, turns the flag on, lets the application choose the page, tests the private key without showing it, and keeps the rollback line before cleaning up.

#1. Feature flags of the web shop, documented, the payment key private;
parameter add "tuto_p4h_ff_new_checkout" "off" "Tutorial: new checkout page (on|off)" false;
parameter add "tuto_p4h_ff_pay_key" "sk_test_000" "Tutorial: payment sandbox key" true;
#2. Turn the flag on for the release and update its documentation;
parameter update "tuto_p4h_ff_new_checkout" "on";
parameter set_description "tuto_p4h_ff_new_checkout" "Tutorial: new checkout page, on since release 4.3";
#3. The application reads the flag, the private key is only tested and never shown;
-> "[page]" (if (equal (parameter get value "tuto_p4h_ff_new_checkout") "on") {"checkout_v2.html"} {"checkout.html"});
-> "[report]" (concat "page served: " [page] " | doc: " (parameter get description "tuto_p4h_ff_new_checkout") " | key private: " (parameter is_private "tuto_p4h_ff_pay_key") ", length " (string length (parameter get value "tuto_p4h_ff_pay_key")));
#4. Rollback line kept with the release notes, then clean up;
-> "[rollback]" (parameter generate merge "tuto_p4h_ff_new_checkout");
parameter remove "tuto_p4h_ff_new_checkout";
parameter remove "tuto_p4h_ff_pay_key";
concat [report] [_n_] "rollback: " [rollback] [_n_] "flags left: " (+ (parameter exist "tuto_p4h_ff_new_checkout") (parameter exist "tuto_p4h_ff_pay_key"))

page served: checkout_v2.html | doc: Tutorial: new checkout page, on since release 4.3 | key private: 1, length 11
rollback: parameter merge "tuto_p4h_ff_new_checkout" "on";
flags left: 0

Quarterly rotation of an API account password

The password of the partner API account is rotated every quarter. The script generates a new 32-character password, sets it, checks that the new one is accepted and the old one refused, stores it in the private vault parameter and traces the rotation.

#1. Quarterly rotation of the password of the partner API account;
-> "[old]" (parameter get value "PARTNER_ACME_PWD");
-> "[new]" (string generate_random_str 32);
#2. Set it, check it, and check the old one is refused;
user set password "partner_acme" [new];
-> "[ok]" (user check password "partner_acme" [new]);
-> "[old_refused]" (not (user check password "partner_acme" [old]));
#3. Store the new password in the private vault parameter and trace the rotation;
parameter update "PARTNER_ACME_PWD" [new];
log write "Password of partner_acme rotated" "OK" "rotation" (date sysdate);
concat "new password accepted: " [ok] ", old one refused: " [old_refused]

POST an order to a partner HTTP API

An order must be sent as JSON to the warehouse API of a partner inside the private network. The script builds the JSON order, the bearer header from a private parameter, posts it and reads the tracking number.

#1. The order to send to the partner warehouse API, inside the private network;
json load "rx_order" "{}";
json iobject "rx_order" / "order_id" "SO-2026-1182" STR;
json iobject "rx_order" / "lines" "[{\"sku\": \"BK-0042\", \"qty\": 12}]" ARRAY;
#2. Headers: the API token stays in a private parameter;
json load "rx_headers" "{}";
json iobject "rx_headers" / "Authorization" (concat "Bearer " (parameter get value "WAREHOUSE_API_TOKEN")) STR;
#3. POST the JSON and read the tracking number;
json load "rx_resp" (rest http_json_post "http://warehouse.internal:8080/api/v1/orders" (json doc "rx_headers") "[]" (json doc "rx_order"));
concat "order " (json select "rx_order" "/order_id") " accepted, tracking " (json select "rx_resp" "/tracking_number")