Profiles / Programming
WA

Web application developer

Create a web app, its host name, pages, ajax forms, menu and grants, then remove it all.

10 recipes on this page · open in the interactive view

Recipes
  1. Create a web application context
  2. Give the app its own host name
  3. Smoke test the login page over HTTP
  4. A page script that returns safe HTML
  5. A JSON endpoint for the page
  6. Generate an ajax form and test it
  7. The server action behind the Save button
  8. Build the menu and test who sees what
  9. Grant the page scripts to a group
  10. Tear the app down and prove it is gone
executed on MentDB

Create a web application context

A new customer portal needs its own web context. The script checks that the context is free, creates the http application tuto_app from the default template (version 100), then checks it exists and lists the http applications. app create copies web/template into web/http/tuto_app, writes its index.jsp and restarts the web contexts.

Pipeline
Script
admin
#1. The context must not exist yet;
-> "[before]" (app exist "http" "tuto_app");
#2. Create the app from the default template, version 100;
-> "[msg]" (app create "http" "tuto_app" "default" "100");
#3. Check the context and list the http apps;
-> "[after]" (app exist "http" "tuto_app");
-> "[apps]" (app show "http");
concat "Exists before: " [before] [_n_] [msg] [_n_] "Exists after: " [after] [_n_] "Apps on http: " [apps] [_n_] "Open: http://localhost:8080/tuto_app/index.jsp";
Result
mentdb
Exists before: 0
Application created with successful.
Exists after: 1
Apps on http: [
  "tuto_app"
]
Open: http://localhost:8080/tuto_app/index.jsp
Go further
  • Create the https variant with app create "https" ... for a portal behind TLS
  • Run app exist first in a deploy script so a second run does not fail
executed on MentDB

Give the app its own host name

The portal must answer on tuto-app.localhost instead of /tuto_app. The script maps the host name to the app, checks it from the app side, then reads the server-wide map of http virtual hosts as JSON.

Pipeline
Script
admin
#1. Map a host name to the app;
-> "[host]" "tuto-app.localhost";
-> "[added]" (app vhost add "http" "tuto_app" [host]);
#2. Check it from the app side;
-> "[exists]" (app vhost exist "http" "tuto_app" [host]);
-> "[of_app]" (app vhost show "http" "tuto_app");
#3. Read the server-wide map of http virtual hosts;
json load "tuto_app_vh" (app vhost show "http");
-> "[hosts]" (json fields "tuto_app_vh" "/");
-> "[nb]" (json count "tuto_app_vh" "/");
json unload "tuto_app_vh";
concat "Added: " [added] ", exists: " [exists] [_n_] "Vhosts of tuto_app: " [of_app] [_n_] "All http vhosts: " [nb] " " [hosts];
Result
mentdb
Added: 1, exists: 1
Vhosts of tuto_app: {
  "tuto-app.localhost": "tuto_app"
}
All http vhosts: 1 [
  "tuto-app.localhost"
]
Go further
executed on MentDB

Smoke test the login page over HTTP

After a deployment, check that the app really answers. The script loads the login page of tuto_app over HTTP into a DOM, reads the title, lists the names of the form inputs, reads one element by its id, then closes the DOM.

Pipeline
Script
admin
#1. Load the login page of the app over HTTP;
-> "[url]" "http://localhost:8080/tuto_app/index.jsp";
html load_from_url "tuto_app_dom" [url] "get" 5000 "{}" "{}" "{}";
#2. Read the page title;
-> "[title]" "";
html parse "tuto_app_dom" "tuto_app_el" "TAG" "title" "" {
	-> "[title]" (json select "tuto_app_el" "/text");
};
#3. List the names of the form fields;
-> "[fields]" "";
html parse "tuto_app_dom" "tuto_app_el" "TAG" "input" "" {
	concat_var "[fields]" (json select "tuto_app_el" "/attributes/name") " ";
};
#4. One element by its id;
html element "tuto_app_dom" "tuto_app_user" "x-user";
-> "[placeholder]" (json select "tuto_app_user" "/attributes/placeholder");
-> "[docs]" (html show);
html close "tuto_app_dom";
json unload "tuto_app_user";
concat "Title: " [title] [_n_] "Fields: " (string trim [fields]) [_n_] "User box placeholder: " [placeholder] [_n_] "Open DOMs before close: " [docs] ", still open: " (html exist "tuto_app_dom");
Result
mentdb
Title: Demonstration
Fields: x-lang x-user x-password
User box placeholder: User
Open DOMs before close: [
  "tuto_app_dom"
], still open: 0
Go further
  • Turn it into a monitoring job that logs an error when the title is missing
  • Use html parse with SELECT and a CSS selector such as form input[type=password]
executed on MentDB

A page script that returns safe HTML

The orders page shows a customer name typed by users. The script creates a typed exe script that builds a Bootstrap container, row and columns with the sajax skeleton helpers, escapes the name and rounds the total, then renders it with a hostile name to prove the escaping.

Pipeline
Script
admin
#1. A page script: Bootstrap grid from the sajax skeleton helpers;
script create exe "tuto.app.page.orders" false 1
  (param
  	(var "[customer]" {true} "Customer name typed by a user" is_null:false is_empty:false "Bob & Co")
  	(var "[total]" {type is_double [total]} "Order total" is_null:false is_empty:false "120.5")
  ;)
  "HTML block of the orders page"
{
	-> "[page]" "";
	include "app.100.obj.sajax.skeleton.container.exe" "[id]" "orders" "[name]" "" "[class]" "" "[style]" "";
		include "app.100.obj.sajax.skeleton.row.exe" "[id]" "" "[name]" "" "[class]" "" "[style]" "";
			include "app.100.obj.sajax.skeleton.col.exe" "[id]" "customer" "[name]" "" "[class]" "col-8" "[style]" "";
				#Never trust user input: escape it;
				concat_var "[page]" "<b>" (string html_escape [customer]) "</b>";
			include "app.100.obj.sajax.skeleton.div.end.exe";
			include "app.100.obj.sajax.skeleton.col.exe" "[id]" "total" "[name]" "" "[class]" "col-4 text-end" "[style]" "";
				concat_var "[page]" (math round [total] 2) " EUR";
			include "app.100.obj.sajax.skeleton.div.end.exe";
		include "app.100.obj.sajax.skeleton.div.end.exe";
	include "app.100.obj.sajax.skeleton.div.end.exe";
	[page];
} "html";
#2. Render it with a hostile customer name;
-> "[html]" (execute "tuto.app.page.orders.exe" "[customer]" "<script>alert(1)</script> & Co" "[total]" 120.456);
concat "Script exists: " (script exist "tuto.app.page.orders.exe") [_n_] [html];
Result
mentdb
Script exists: 1
<div id='orders' class='container-fluid'><div class='row'><div id='customer' class='col-8'><b>&lt;script&gt;alert(1)&lt;/script&gt; &amp; Co</b></div><div id='total' class='col-4 text-end'>120.46 EUR</div></div></div>
Go further
  • Add a column with include "app.100.obj.sajax.skeleton.col.exe" and a progress bar with app.100.obj.progress_bar.exe
  • The type check {type is_double [total]} already refuses a non numeric total
executed on MentDB

A JSON endpoint for the page

The page loads its orders as JSON. The script creates a get script with an enum parameter that builds, sorts and counts the orders with the json functions, calls it, shows that a bad status is refused, and prints its REST path with escaped values.

Pipeline
Script
admin
#1. A get script that returns the orders of a status as JSON;
script create get "tuto.app.api.orders" false 1
  (param
  	(var "[status]" {type is_enum [status] "open,late,closed"} "Order status" is_null:false is_empty:false "open")
  ;)
  "JSON data of the orders page"
{
	json load "tuto_app_out" "{}";
	json iobject "tuto_app_out" / status [status] STR;
	json iobject "tuto_app_out" / orders "[]" ARRAY;
	json iarray "tuto_app_out" /orders "{\"id\": 102, \"customer\": \"O'Neil\", \"total\": 89.9}" OBJ;
	json iarray "tuto_app_out" /orders "{\"id\": 101, \"customer\": \"Bob & Co\", \"total\": 120.5}" OBJ;
	json sort "tuto_app_out" /orders total DESC;
	json iobject "tuto_app_out" / count (json count "tuto_app_out" /orders) INT;
	-> "[doc]" (json doc "tuto_app_out");
	json unload "tuto_app_out";
	[doc];
} "JSON object: status, orders, count";
#2. Call it like the page does, then read the answer;
json load "tuto_app_rx" (execute "tuto.app.api.orders.get" "[status]" "open");
-> "[first]" (json select "tuto_app_rx" "/orders[0]/customer");
-> "[fields]" (json fields "tuto_app_rx" "/");
json unload "tuto_app_rx";
#3. A bad status is refused by the parameter check;
try {
	execute "tuto.app.api.orders.get" "[status]" "pending";
} {
	-> "[refused]" (string regexp_extract [err] "\[\[\[ (.*) \]\]\]" 1);
} "[err]";
#4. The REST path of the script, and values escaped for JSON and for a URL;
-> "[path]" (string regexp_extract (script generate url "tuto.app.api.orders.get") "/api/[a-z/]+" 0);
-> "[q]" (concat "{\"customer\": \"" (string json_escape "Bob \"the boss\"") "\"}");
concat "Fields: " [fields] [_n_] "Biggest order: " [first] [_n_] "Bad call: " [refused] [_n_] "REST path: " [path] "?status=" (string url_encode "open & late") [_n_] "JSON value: " [q];
Result
mentdb
Fields: [
  "count",
  "orders",
  "status"
]
Biggest order: Bob & Co
Bad call: Sorry, the variable '[status]' in the script 'tuto.app.api.orders.get' is not valid (type is_enum [status] "open,late,closed";).
REST path: /api/tuto/app/api/orders?status=open%20%26%20late
JSON value: {"customer": "Bob \"the boss\""}
Go further
  • Grant the script to the api-rest group to call it from outside
  • Show the parameters of the script with script show params
executed on MentDB

Generate an ajax form and test it

The new order form is built server side with the form helpers. The script builds a non-modal form with a text box and a number box, reads the REFRESH command that the helpers put in the ajax result, loads the generated HTML in a DOM to list the required inputs, and decodes which MQL script the Save button calls.

Pipeline
Script
admin
#1. The form helpers append to [object] and send a REFRESH into the ajax result;
-> "[app_lang]" "en";
-> "[object]" "";
json load "result" "[]";
include "app.100.obj.form.begin.exe" "[form_id]" "form_order" "[modal]" false
	"[action]" "index.jsp" "[enctype]" false "[title]" "New order" "[subTitle]" ""
	"[method]" "post" "[widthIfModal]" "modal-lg" "[html]" "";
include "app.100.obj.form.control.textbox.text.exe" "[control_id]" "customer" "[class]" "col-12"
	"[label]" "Customer" "[description]" "" "[maxlength]" "80" "[placeholder]" ""
	"[readonly]" false "[required]" true "[disabled]" false "[html]" "" "[value]" "";
include "app.100.obj.form.control.textbox.number.exe" "[control_id]" "total" "[class]" "col-6"
	"[label]" "Total" "[description]" "" "[maxlength]" ""
	"[min]" 0 "[max]" 10000 "[step]" 0.01 "[placeholder]" ""
	"[readonly]" false "[required]" true "[disabled]" false "[html]" "" "[value]" 0;
include "app.100.obj.form.end.exe" "[form_id]" "form_order" "[modal]" false
	"[OBJ_OVERWRITE_B64]" (string encode_b64 "{}") "[container_id]" "orders"
	"[mql_script]" "tuto.app.action.save_order.exe" "[data_eval]" "$('#form_order').serializeObject()"
	"[closeTitle]" "Close" "[submitTitle]" "Save" "[submitType]" "primary";
#2. Inspect the ajax answer: one REFRESH of the container;
-> "[type]" (json select "result" "/[0]/type");
-> "[target]" (json select "result" "/[0]/target");
#3. Parse the generated HTML like a browser would;
html load_from_str "tuto_app_form" (json select "result" "/[0]/innerHtml");
-> "[inputs]" "";
html parse "tuto_app_form" "tuto_app_el" "SELECT" "input[required]" "" {
	concat_var "[inputs]" (json select "tuto_app_el" "/attributes/name") ":" (json select "tuto_app_el" "/attributes/type") " ";
};
#4. The button calls execute_script with the script name in base 64;
html parse "tuto_app_form" "tuto_app_el" "TAG" "button" "" {
	-> "[onclick]" (json select "tuto_app_el" "/attributes/onclick");
};
-> "[called]" (string decode_b64 (string regexp_extract [onclick] "execute_script\('([^']+)'" 1));
html close "tuto_app_form";
json unload "result";
concat "Answer: " [type] " of #" [target] [_n_] "Required inputs: " (string trim [inputs]) [_n_] "Save button runs: " [called];
Result
mentdb
Answer: REFRESH of #orders
Required inputs: customer:text total:number
Save button runs: tuto.app.action.save_order.exe
Go further
  • Pass "[modal]" true to open the same form in a modal window
  • Add a list with app.100.obj.form.control.select_mono.exe
executed on MentDB

The server action behind the Save button

The Save button posts the form to a remote sajax script. The script creates that action: it decodes the base 64 parameters, then answers with an alert, a refresh of the orders block and a JavaScript reset, or an error alert. It simulates a valid and an invalid click, then generates the onclick code for a custom button.

Pipeline
Script
admin
#1. The server action of the Save button: a remote sajax script;
script create exe "tuto.app.action.save_order" false 1
  (param
  	(var "[OBJ_OVERWRITE_B64]" {true} "Parameters in base 64" is_null:false is_empty:false "{}")
  ;)
  "Save an order posted by the form"
{
	json load "tuto_app_param" (string decode_b64 [OBJ_OVERWRITE_B64]);
	-> "[customer]" (json select "tuto_app_param" "/customer");
	-> "[total]" (json select "tuto_app_param" "/total");
	json unload "tuto_app_param";
	if (> [total] 0) {
		include "app.100.obj.sajax.alert.exe" "[type]" "ALERT_SUCCESS" "[strong]" "Saved!"
			"[msg]" (concat "Order of " (string html_escape [customer]) ": " [total] " EUR");
		include "app.100.obj.sajax.refresh.exe" "[target]" "orders"
			"[innerHtml]" (execute "tuto.app.page.orders.exe" "[customer]" [customer] "[total]" [total]);
		include "app.100.obj.sajax.javascript.exe" "[javascript]" "$('#form_order')[0].reset();";
	} {
		include "app.100.obj.sajax.alert.exe" "[type]" "ALERT_DANGER" "[strong]" "Error"
			"[msg]" "The total must be positive.";
	};
} "Ajax commands in the result array";
#2. Simulate two clicks as the browser sends them - include shares variables, so use own names;
json load "result" "[]";
include "tuto.app.action.save_order.exe" "[OBJ_OVERWRITE_B64]" (string encode_b64 "{\"customer\": \"Bob & Co\", \"total\": 120.5}");
-> "[ok]" (concat (json count "result" "/") " commands: " (json select "result" "/[0]/type") ", " (json select "result" "/[1]/type") ", " (json select "result" "/[2]/type"));
-> "[alert1]" (json select "result" "/[0]/msg");
json load "result" "[]";
include "tuto.app.action.save_order.exe" "[OBJ_OVERWRITE_B64]" (string encode_b64 "{\"customer\": \"Bob\", \"total\": -3}");
-> "[ko]" (concat (json select "result" "/[0]/type") " " (json select "result" "/[0]/msg"));
json unload "result";
json unload "o";
#3. The JavaScript a custom button needs to call the same action;
-> "[js]" (include "app.100.obj.sajax.client.exe" "[scriptname]" "tuto.app.action.save_order.exe"
	"[OBJ_OVERWRITE_B64]" (string encode_b64 "{\"customer\": \"Ann\", \"total\": 50}")
	"[getElementById]" "" "[data]" "{}");
concat "Valid order: " [ok] [_n_] "  alert: " [alert1] [_n_] "Invalid order: " [ko] [_n_] "onclick=" (string left [js] 40) "...";
Result
mentdb
Valid order: 3 commands: ALERT_SUCCESS, REFRESH, JAVASCRIPT
  alert: Order of Bob &amp; Co: 120.5 EUR
Invalid order: ALERT_DANGER The total must be positive.
onclick="execute_script('dHV0by5hcHAuYWN0aW9uLnN...
Go further
  • include runs in the same environment: the helpers set [type], [msg], [target]... so give your own variables other names
  • Put a ready button in a container with app.100.obj.sajax.button.exe
executed on MentDB

Grant the page scripts to a group

Only the editors may run the scripts of the app. The script creates a tuto_app_editors group, grants all tuto.app. scripts by prefix, grants the MentDB user web that runs the app, checks the rights, reverts the user grant, then compares tag checks for an administrator and a super administrator inside a page.

Pipeline
Script
admin
#1. A group for the scripts of the app, granted by name prefix;
-> "[added]" (group add "tuto_app_editors");
group grant_all script "tuto.app." "tuto_app_editors";
json load "tuto_app_g" (group get script "tuto_app_editors");
-> "[scripts]" (json count "tuto_app_g" "/");
json unload "tuto_app_g";
#2. The app runs as the MentDB user web: grant it, check, then revert;
group grant user "web" "tuto_app_editors";
-> "[web_in]" (group is granted user "web" "tuto_app_editors");
-> "[web_api]" (script is granted "tuto.app.api.orders.get" "web");
-> "[group_page]" (group is granted script "tuto.app.page.orders.exe" "tuto_app_editors");
group ungrant user "web" "tuto_app_editors";
-> "[web_after]" (script is granted "tuto.app.api.orders.get" "web");
-> "[users]" (group get user "tuto_app_editors");
#3. Inside a page, tags decide: a user tagged admin *- versus super admin *+;
json load "app" "{}";
json iobject "app" / groups "{\"*-\": 0}" OBJ;
-> "[a]" (concat (app is_granted_a "tuto_app_orders_delete") "/" (app is_granted_sa "tuto_app_orders_delete"));
json uobject "app" / groups "{\"*+\": 0}" OBJ;
-> "[sa]" (concat (app is_granted_a "tuto_app_orders_delete") "/" (app is_granted_sa "tuto_app_orders_delete"));
json unload "app";
concat [added] " " [scripts] " scripts in the group" [_n_] "web in group: " [web_in] ", web may run the API: " [web_api] ", group has the page: " [group_page] [_n_] "After revert, web may run the API: " [web_after] ", users: " [users] [_n_] "Tag orders_delete a/sa for *- user: " [a] ", for *+ user: " [sa];
Result
mentdb
Group added with successful. 3 scripts in the group
web in group: 1, web may run the API: 1, group has the page: 1
After revert, web may run the API: 0, users: {
  "admin": 0
}
Tag orders_delete a/sa for *- user: 1/0, for *+ user: 1/1
Go further
  • See every group of a user with user show groups "web"
  • Grant a single script with group grant script
executed on MentDB

Tear the app down and prove it is gone

The demo is over. The script reads the web server counters (callers whose last request hit /tuto_app/, blocked IPs), deletes the virtual host, the group and the three scripts, deletes the app with app delete, which removes web/http/tuto_app, its record and its rows, then checks that no object is left.

Pipeline
Script
admin
#1. Before removing: did the app get traffic, is any caller blocked;
-> "[hits]" (string regexp_count (app webserver show_all_remote_ips) "/tuto_app/");
-> "[blocked]" (string regexp_count (app webserver show_all_blocked_remote_ips) "[0-9a-f]+[:.][0-9a-f:.]+");
#2. Remove the vhost, the group and the scripts;
app vhost delete "http" "tuto_app" "tuto-app.localhost";
group ungrant_all script "tuto.app." "tuto_app_editors";
group remove "tuto_app_editors";
script delete "tuto.app.page.orders.exe";
script delete "tuto.app.api.orders.get";
script delete "tuto.app.action.save_order.exe";
#3. Delete the app: folder, record and rows, then restart of the web contexts;
-> "[msg]" (app delete "http" "tuto_app");
#4. Check that nothing is left;
-> "[left]" (+ (app exist "http" "tuto_app") (group exist "tuto_app_editors") (script exist "tuto.app.page.orders.exe") (script exist "tuto.app.api.orders.get") (script exist "tuto.app.action.save_order.exe"));
concat "Requests seen on /tuto_app/: " [hits] ", blocked IPs: " [blocked] [_n_] [msg] [_n_] "Objects left: " [left] [_n_] "Apps on http: " (app show "http");
Result
mentdb
Requests seen on /tuto_app/: 1, blocked IPs: 0
Application deleted with successful.
Objects left: 0
Apps on http: []
Go further
  • show_all_remote_ips keeps the last request of each IP, so the count is per caller, not per request
  • Block an abusive caller with app webserver add_blocked_ip

Also in Programming

MB
MQL beginner
Variables, operators, if, loops and your first try/exception: learn MQL step by step.
10 recipes
MD
MQL developer
Atom lists, data types, constants, translations, session variables and error patterns.
10 recipes
BW
Backend & web service developer
Typed scripts, execute / call / include, generated calls and URLs, parameters and logs.
10 recipes
DT
Date & time developer
Calendars, business days, time zones, ISO and epoch, durations, quarters and week numbers.
10 recipes
SE
String & encoding developer
Cleaning, case styles, regex, URL/HTML/Base64 encodings, number bases and encryption.
10 recipes
DV
Data validation developer
IBAN, BIC, SIRET, Luhn, email, IP, UUID, JSON and date checks gathered in a validation report.
10 recipes
JD
JSON & document store developer
In-memory JSON documents, persistent nodes and sequences: build, walk, sort, clean up.
10 recipes
SD
Script developer
Full script lifecycle in the MQL editor: typed parameters, read back, generate, patch, deploy, copy, grant, delete.
10 recipes
EM
Execution modes
execute, call, include, sub_include, parallel, stack: which environment, which checks, which logs.
11 recipes
WS
Web service developer
A small REST resource with get, post, put and delete scripts: validation, JSON answers, codes, grants, URLs and API docs.
10 recipes
AD
Async developer
Asynchronous processing with the stack: push runs now or later, priorities and parallel limits, errors and replays, flows and cleanup.
10 recipes
SD
Scheduler developer
Cron jobs and delayed runs: add, resume, pause, update and remove jobs, prove they ran, and postpone stack runs with a delay condition.
10 recipes
ED
Event developer
Event-driven scripts: file watchers that push each file event to the stack, validation, archiving, errors and safe restarts.
10 recipes
CC
Configuration & connections
Parameters (private, described, locked, deployed), environment variables, sessions and connections used by your scripts.
12 recipes
VD
Versioning & deployment
Group your scripts, commit, compare, roll back, export, import and package a release.
10 recipes
DE
Debugging & error handling
Coded errors with try/exception, logs with correlation ids, timing, metrics and defensive checks.
10 recipes
SD
SQL developer in MQL
Connections, DDL and DML with sql encode, transactions, cursors, parse, and exports to JSON, CSV, Excel and PDF on a real local database. Run the recipes in order: the first one creates the tuto_db_h2 database.
10 recipes
CD
CRUD & data API developer
Generate SCRUD scripts from a table, patch and install them, write typed read and write APIs, check data quality and manage connections per environment. Uses the tuto_db_h2 database of the SQL developer recipes; the last recipe removes it.
10 recipes
AI
Algorithms in MQL
Sorting, searching, recursion, data structures and number algorithms: MQL is a real programming language.
10 recipes
CF
Control flow
if, if force, case, switch, mode, for, while, repeat, break, continue, parallel and synchronized on real cases.
10 recipes
TM
The MQL language in depth
Data types and null, booleans, comments, variables and scope, eval and mql blocks, coded errors and try.
10 recipes
SU
Screen & UI developer (sajax)
Build the screens of MentDB web apps in MQL: Bootstrap grids, forms and every control, ajax buttons, alerts, refreshes and charts.
10 recipes