Avro
Read and write Apache Avro container files and read their schema. Typical uses: exchange data with Kafka pipelines or Hadoop.
avro writeavro writeavro readavro readavro schemaavro schema

avro write <path> <json_schema> <json_rows>


Description

    Write JSON rows to an Avro file, without compression. Result: {file, rows, bytes, codec}

Parameters

    path:   The Avro file (same rights as file load) - String - required
    json_schema:   The Avro schema, as JSON - String - required
    json_rows:   The rows, as a JSON array - String - required
admin
avro write "tmp/doc_missing_dir/x.avro" "\"long\"" "[1, 2, 3]"
mentdb
ERROR: Sorry, an error appears (the folder of the parameter 'path' does not exist).
A file that exists is refused
admin
avro write "tools/mql_dev/README.md" "\"long\"" "[1, 2, 3]"
mentdb
ERROR: Sorry, an error appears (the file 'tools/mql_dev/README.md' already exists (set overwrite to true to replace it)).

avro write <path> <json_schema> <json_rows> <json_params>


Description

    Write with parameters: codec (null, deflate, snappy), level (6, for deflate, 0 to 9), block_bytes (64000, 1024 to 64 MB: size of a block before compression), overwrite (false), metadata (a JSON object of texts, stored in the file; keys must not start with avro.)

Parameters

    path:   The Avro file (same rights as file load) - String - required
    json_schema:   The Avro schema - String - required
    json_rows:   The rows - String - required
    json_params:   The parameters - String - required
admin
avro write "tmp/doc_orders.avro" "{\"type\": \"record\", \"name\": \"Order\", \"namespace\": \"com.acme\", \"fields\": [{\"name\": \"id\", \"type\": \"long\"}, {\"name\": \"shop\", \"type\": \"string\"}, {\"name\": \"state\", \"type\": {\"type\": \"enum\", \"name\": \"State\", \"symbols\": [\"NEW\", \"PAID\"]}}, {\"name\": \"day\", \"type\": {\"type\": \"int\", \"logicalType\": \"date\"}}, {\"name\": \"amount\", \"type\": {\"type\": \"bytes\", \"logicalType\": \"decimal\", \"precision\": 10, \"scale\": 2}}, {\"name\": \"tags\", \"type\": {\"type\": \"array\", \"items\": \"string\"}}, {\"name\": \"note\", \"type\": [\"null\", \"string\"], \"default\": null}]}" "[{\"id\": 1, \"shop\": \"Saint-Denis\", \"state\": \"PAID\", \"day\": \"2026-11-05\", \"amount\": \"120.50\", \"tags\": [\"web\", \"promo\"], \"note\": \"first\"}, {\"id\": 2, \"shop\": \"Le Tampon\", \"state\": \"NEW\", \"day\": \"2026-11-06\", \"amount\": \"7.00\", \"tags\": []}]" "{\"overwrite\": true}"
mentdb
{"file": "tmp/doc_orders.avro", "rows": 2, "bytes": 653, "codec": "null"}
Deflate and metadata
admin
avro write "tmp/doc_orders.avro" "{\"type\": \"record\", \"name\": \"Order\", \"namespace\": \"com.acme\", \"fields\": [{\"name\": \"id\", \"type\": \"long\"}, {\"name\": \"shop\", \"type\": \"string\"}, {\"name\": \"state\", \"type\": {\"type\": \"enum\", \"name\": \"State\", \"symbols\": [\"NEW\", \"PAID\"]}}, {\"name\": \"day\", \"type\": {\"type\": \"int\", \"logicalType\": \"date\"}}, {\"name\": \"amount\", \"type\": {\"type\": \"bytes\", \"logicalType\": \"decimal\", \"precision\": 10, \"scale\": 2}}, {\"name\": \"tags\", \"type\": {\"type\": \"array\", \"items\": \"string\"}}, {\"name\": \"note\", \"type\": [\"null\", \"string\"], \"default\": null}]}" "[{\"id\": 1, \"shop\": \"Saint-Denis\", \"state\": \"PAID\", \"day\": \"2026-11-05\", \"amount\": \"120.50\", \"tags\": [\"web\", \"promo\"], \"note\": \"first\"}, {\"id\": 2, \"shop\": \"Le Tampon\", \"state\": \"NEW\", \"day\": \"2026-11-06\", \"amount\": \"7.00\", \"tags\": []}]" "{\"overwrite\": true, \"codec\": \"deflate\", \"metadata\": {\"source\": \"erp\"}}"
mentdb
{"file": "tmp/doc_orders.avro", "rows": 2, "bytes": 668, "codec": "deflate"}
Snappy
admin
avro write "tmp/doc_orders.avro" "{\"type\": \"record\", \"name\": \"Order\", \"namespace\": \"com.acme\", \"fields\": [{\"name\": \"id\", \"type\": \"long\"}, {\"name\": \"shop\", \"type\": \"string\"}, {\"name\": \"state\", \"type\": {\"type\": \"enum\", \"name\": \"State\", \"symbols\": [\"NEW\", \"PAID\"]}}, {\"name\": \"day\", \"type\": {\"type\": \"int\", \"logicalType\": \"date\"}}, {\"name\": \"amount\", \"type\": {\"type\": \"bytes\", \"logicalType\": \"decimal\", \"precision\": 10, \"scale\": 2}}, {\"name\": \"tags\", \"type\": {\"type\": \"array\", \"items\": \"string\"}}, {\"name\": \"note\", \"type\": [\"null\", \"string\"], \"default\": null}]}" "[{\"id\": 1, \"shop\": \"Saint-Denis\", \"state\": \"PAID\", \"day\": \"2026-11-05\", \"amount\": \"120.50\", \"tags\": [\"web\", \"promo\"], \"note\": \"first\"}, {\"id\": 2, \"shop\": \"Le Tampon\", \"state\": \"NEW\", \"day\": \"2026-11-06\", \"amount\": \"7.00\", \"tags\": []}]" "{\"overwrite\": true, \"codec\": \"snappy\"}"
mentdb
{"file": "tmp/doc_orders.avro", "rows": 2, "bytes": 662, "codec": "snappy"}
A value that does not fit is refused, with its row and field
admin
avro write "tmp/doc_bad.avro" "{\"type\": \"record\", \"name\": \"Order\", \"namespace\": \"com.acme\", \"fields\": [{\"name\": \"id\", \"type\": \"long\"}, {\"name\": \"shop\", \"type\": \"string\"}, {\"name\": \"state\", \"type\": {\"type\": \"enum\", \"name\": \"State\", \"symbols\": [\"NEW\", \"PAID\"]}}, {\"name\": \"day\", \"type\": {\"type\": \"int\", \"logicalType\": \"date\"}}, {\"name\": \"amount\", \"type\": {\"type\": \"bytes\", \"logicalType\": \"decimal\", \"precision\": 10, \"scale\": 2}}, {\"name\": \"tags\", \"type\": {\"type\": \"array\", \"items\": \"string\"}}, {\"name\": \"note\", \"type\": [\"null\", \"string\"], \"default\": null}]}" "[{\"id\": \"x\"}]" "{\"overwrite\": true}"
mentdb
ERROR: Sorry, an error appears (the row 0 of 'json_rows': the field 'id' must be an integer).
Used in use cases (1 recipe)

    Integrator & ETL developer › Export orders to an Avro file for a data pipeline, then check the file
Often used with

    avro read  ·  avro schema  ·  file delete  ·  file exist  ·  file mkdir

avro read <path>


Description

    Read the rows of an Avro file as a JSON array, 100000 at most

Parameters

    path:   The Avro file (same rights as file load) - String - required
admin
avro write "tmp/doc_orders.avro" "{\"type\": \"record\", \"name\": \"Order\", \"namespace\": \"com.acme\", \"fields\": [{\"name\": \"id\", \"type\": \"long\"}, {\"name\": \"shop\", \"type\": \"string\"}, {\"name\": \"state\", \"type\": {\"type\": \"enum\", \"name\": \"State\", \"symbols\": [\"NEW\", \"PAID\"]}}, {\"name\": \"day\", \"type\": {\"type\": \"int\", \"logicalType\": \"date\"}}, {\"name\": \"amount\", \"type\": {\"type\": \"bytes\", \"logicalType\": \"decimal\", \"precision\": 10, \"scale\": 2}}, {\"name\": \"tags\", \"type\": {\"type\": \"array\", \"items\": \"string\"}}, {\"name\": \"note\", \"type\": [\"null\", \"string\"], \"default\": null}]}" "[{\"id\": 1, \"shop\": \"Saint-Denis\", \"state\": \"PAID\", \"day\": \"2026-11-05\", \"amount\": \"120.50\", \"tags\": [\"web\", \"promo\"], \"note\": \"first\"}, {\"id\": 2, \"shop\": \"Le Tampon\", \"state\": \"NEW\", \"day\": \"2026-11-06\", \"amount\": \"7.00\", \"tags\": []}]" "{\"overwrite\": true}"; avro read "tmp/doc_orders.avro"
mentdb
[{"id": 1, "shop": "Saint-Denis", "state": "PAID", "day": "2026-11-05", "amount": "120.50", "tags": ["web", "promo"], "note": "first"}, {"id": 2, "shop": "Le Tampon", "state": "NEW", "day": "2026-11-06", "amount": "7.00", "tags": [], "note": null}]

avro read <path> <json_params>


Description

    Read with parameters: limit (100000, up to 10000000), offset (0; whole blocks are skipped without being decompressed), max_bytes (256 MB), max_data_bytes (512 MB), max_items (3000000; see the limits at the top of the page)

Parameters

    path:   The Avro file (same rights as file load) - String - required
    json_params:   The parameters - String - required
admin
avro write "tmp/doc_orders.avro" "{\"type\": \"record\", \"name\": \"Order\", \"namespace\": \"com.acme\", \"fields\": [{\"name\": \"id\", \"type\": \"long\"}, {\"name\": \"shop\", \"type\": \"string\"}, {\"name\": \"state\", \"type\": {\"type\": \"enum\", \"name\": \"State\", \"symbols\": [\"NEW\", \"PAID\"]}}, {\"name\": \"day\", \"type\": {\"type\": \"int\", \"logicalType\": \"date\"}}, {\"name\": \"amount\", \"type\": {\"type\": \"bytes\", \"logicalType\": \"decimal\", \"precision\": 10, \"scale\": 2}}, {\"name\": \"tags\", \"type\": {\"type\": \"array\", \"items\": \"string\"}}, {\"name\": \"note\", \"type\": [\"null\", \"string\"], \"default\": null}]}" "[{\"id\": 1, \"shop\": \"Saint-Denis\", \"state\": \"PAID\", \"day\": \"2026-11-05\", \"amount\": \"120.50\", \"tags\": [\"web\", \"promo\"], \"note\": \"first\"}, {\"id\": 2, \"shop\": \"Le Tampon\", \"state\": \"NEW\", \"day\": \"2026-11-06\", \"amount\": \"7.00\", \"tags\": []}]" "{\"overwrite\": true, \"codec\": \"deflate\"}"; avro read "tmp/doc_orders.avro" "{\"offset\": 1, \"limit\": 1}"
mentdb
[{"id": 2, "shop": "Le Tampon", "state": "NEW", "day": "2026-11-06", "amount": "7.00", "tags": [], "note": null}]
Used in use cases (1 recipe)

    Integrator & ETL developer › Export orders to an Avro file for a data pipeline, then check the file
Often used with

    avro schema  ·  avro write  ·  file delete  ·  file exist  ·  file mkdir

avro schema <path>


Description

    Read the description of a file without decoding the rows. Result: {codec, rows, blocks, bytes, metadata, schema}

Parameters

    path:   The Avro file (same rights as file load) - String - required
admin
avro write "tmp/doc_orders.avro" "{\"type\": \"record\", \"name\": \"Order\", \"namespace\": \"com.acme\", \"fields\": [{\"name\": \"id\", \"type\": \"long\"}, {\"name\": \"shop\", \"type\": \"string\"}, {\"name\": \"state\", \"type\": {\"type\": \"enum\", \"name\": \"State\", \"symbols\": [\"NEW\", \"PAID\"]}}, {\"name\": \"day\", \"type\": {\"type\": \"int\", \"logicalType\": \"date\"}}, {\"name\": \"amount\", \"type\": {\"type\": \"bytes\", \"logicalType\": \"decimal\", \"precision\": 10, \"scale\": 2}}, {\"name\": \"tags\", \"type\": {\"type\": \"array\", \"items\": \"string\"}}, {\"name\": \"note\", \"type\": [\"null\", \"string\"], \"default\": null}]}" "[{\"id\": 1, \"shop\": \"Saint-Denis\", \"state\": \"PAID\", \"day\": \"2026-11-05\", \"amount\": \"120.50\", \"tags\": [\"web\", \"promo\"], \"note\": \"first\"}, {\"id\": 2, \"shop\": \"Le Tampon\", \"state\": \"NEW\", \"day\": \"2026-11-06\", \"amount\": \"7.00\", \"tags\": []}]" "{\"overwrite\": true, \"codec\": \"deflate\", \"metadata\": {\"source\": \"erp\"}}"; avro schema "tmp/doc_orders.avro"
mentdb
{"codec": "deflate", "rows": 2, "blocks": 1, "bytes": 668, "metadata": {"source": "erp"}, "schema": {"type": "record", "name": "Order", "namespace": "com.acme", "fields": [{"name": "id", "type": "long"}, {"name": "shop", "type": "string"}, {"name": "state", "type": {"type": "enum", "name": "State", "symbols": ["NEW", "PAID"]}}, {"name": "day", "type": {"type": "int", "logicalType": "date"}}, {"name": "amount", "type": {"type": "bytes", "logicalType": "decimal", "precision": 10, "scale": 2}}, {"name": "tags", "type": {"type": "array", "items": "string"}}, {"name": "note", "type": ["null", "string"], "default": null}]}}

avro schema <path> <json_params>


Description

    Read the description with parameters: max_bytes (256 MB)

Parameters

    path:   The Avro file (same rights as file load) - String - required
    json_params:   The parameters - String - required
admin
avro write "tmp/doc_orders.avro" "{\"type\": \"record\", \"name\": \"Order\", \"namespace\": \"com.acme\", \"fields\": [{\"name\": \"id\", \"type\": \"long\"}, {\"name\": \"shop\", \"type\": \"string\"}, {\"name\": \"state\", \"type\": {\"type\": \"enum\", \"name\": \"State\", \"symbols\": [\"NEW\", \"PAID\"]}}, {\"name\": \"day\", \"type\": {\"type\": \"int\", \"logicalType\": \"date\"}}, {\"name\": \"amount\", \"type\": {\"type\": \"bytes\", \"logicalType\": \"decimal\", \"precision\": 10, \"scale\": 2}}, {\"name\": \"tags\", \"type\": {\"type\": \"array\", \"items\": \"string\"}}, {\"name\": \"note\", \"type\": [\"null\", \"string\"], \"default\": null}]}" "[{\"id\": 1, \"shop\": \"Saint-Denis\", \"state\": \"PAID\", \"day\": \"2026-11-05\", \"amount\": \"120.50\", \"tags\": [\"web\", \"promo\"], \"note\": \"first\"}, {\"id\": 2, \"shop\": \"Le Tampon\", \"state\": \"NEW\", \"day\": \"2026-11-06\", \"amount\": \"7.00\", \"tags\": []}]" "{\"overwrite\": true}"; avro schema "tmp/doc_orders.avro" "{\"max_bytes\": 10}"
mentdb
ERROR: Sorry, an error appears (the Avro file is larger than max_bytes (10 bytes)).
Used in use cases (1 recipe)

    Integrator & ETL developer › Export orders to an Avro file for a data pipeline, then check the file
Often used with

    avro read  ·  avro write  ·  file delete  ·  file exist  ·  file mkdir