Machine language
The procedure document (bm-procedure/2): inputs, tools, values, steps, errors and lints.
Machines are in beta. The language can get new keys and new lints before general availability. A saved version never changes.
A procedure is a JSON document that says what a Machine does, step by step. The compiler checks it against the schemas of its tools and writes a program, the IR. The engine runs the IR. Read Machines for the run loop.
This page is the reference for a person or a model that writes a procedure. POST /v2/machines/lint checks a procedure and gives each problem with its path. Read Lints.
The examples on this page use an equipment rental company. Its tools are examples only.
The procedure document
Names
Step ids, input names, table names and as names match ^[a-z][a-z0-9_]{0,39}$. Each name is unique in the procedure. These names are reserved: input, tables, result, args, error, self, item, true, false and null.
Inputs
Each input has a type: "file", "string", "number", "boolean" or "object". An input can also give a JSON Schema in schema.
- A file input has the type
{file_id, name, content_type, bytes}. Read the id with@input.f1.file_id. A run gets a file as a URL or as the id of a file of the organization. "later": truemarks an input that arrives after the start, for example a second message. Read a later input only after anawaitstep that names it.- The message of turn 1 is the string input
user_message. The message of turn n isuser_message<n>(user_message2, and so on), declaredlater. No other name carries a user message. A step id cannot be a message name. - The exception model and a person get the messages that arrived, by these names.
Tools and types
The compiler gets a tool list in the MCP shape: a JSON array, or {"tools": [...]}.
A step can raise the effect of its tool, for example from write to irreversible. A step cannot lower it.
The compiler reads this subset of JSON Schema: type, properties, required, items, enum, const, anyOf, additionalProperties, pattern and format. It accepts other keys and ignores them.
An output schema can depend on an argument. The node {"x-from-arg": "schema"} takes the type of that literal argument. The output of file_extract is {file_id, value}, and value has the shape of the schema argument. So the compiler checks @form.value.contract_number against the extraction schema that the step sends.
Paperwork tools
The engine runs the file_* tools itself, with the Paperwork API of your organization. The tool list does not repeat them. Their effect is read, and a step cannot raise it (builtin-effect). A Paperwork step bills as a Paperwork call.
Values
An argument value (in args, retry_with, use, context, attach and the input of a code step) is one of the forms below.
Literal
Any JSON value is a literal. A string that starts with @ is a reference: write @@x for the text @x. An object with the key lookup, find, template or expr is a value form. To send such an object as data, wrap it: {"literal": {...}}.
Reference
A reference is "@<name><path>".
<name>is a step id,input, theasname of an enclosingforeach,error(in anon_errorcase),self(incompensate: the output of the step),tablesorsent.<path>is zero or more of.key,[n],["any key"]and[*](each item of a list).- The reference replaces the whole string. There is no interpolation in a reference: use a template.
- The compiler checks each key against the schema of its level. A key that the tool does not return does not compile. The message lists the keys that exist.
- A key that the schema does not list in
requiredgives the warningref-optional.
@sent.<step>.<arg> reads the arguments that a tool step sent. After a retry_with, it reads the last call. In an error case of that step, it reads the call that failed. Use it when a later step needs a value that the tool does not return. In an expression, write sent.<step>.<arg>.
Order of references
The referenced step must run before the value on every path (ref-order).
- A step inside a
whencase is visible in that case. After thewhen, an expression can read it: it isnullwhen its case did not run. An argument can read it only when each path that continues defines it. - A step is tested inside a
whencase whoseifreads the step, and after acheckstep that reads it. An argument can read a tested step, also when it ran only on some paths or when an error case can skip it. The engine still gives the errorref_errorwhen the value isnullat run time.
Scope of foreach
- In the body of a
foreach, theasname is the current item. A step of the body, read in the body, is the output for the current item. - Outside the body, a step of the body is a list: one entry per item, in item order. An item that
whereskipped, or awhenthat did not run the step, givesnull. Loop over such a list with awherethat tests!= null. - Nested loops apply the same rule at each level.
Table lookup
{"lookup": "fees", "key": <value>}gives the value of the key in the tablefees. Keys match after trim and lower case."default": <literal>gives the value when no key matches.{"lookup": {"a": 1, "b": 2}, "key": <value>}uses an inline table.{"find": <list>, "where": {"<key>": <value>}, "take": "<path>"}gives the first item where each key equals the value (strings after trim and lower case), then the path in that item. With notake, it gives the whole item.defaultapplies as above.- A miss with no
defaultis a step error with the codelookup_miss. Theon_errorof the step applies, and the default is a hold.
Template
{"template": "Charge {{@add.charge_id}} is on contract {{@contract.contract_id}}."} gives a string. Each {{@ref}} is a reference. A string shows as it is, and another value shows as JSON.
The report of finish is a template string. In a report, a placeholder can read a step that ran only on some paths, or a step in a loop ({{@add[*].charge_id}}). A value that is null or absent shows as -.
Expression
{"expr": "<expression>", "schema": <JSON Schema of the result>}. The expression language is RakeXL (@cloudraker/rakexl 1.4.6), a JEXL dialect.
- Names in an expression: step ids,
input,asnames andtables. By place, alsoresult(inexpect),args(inrequire) anderror(in anon_errorcaseif). A step that did not run isnull. - The compiler checks each identifier path as it checks a reference.
- In the expression string of
filter,maporfind, onlyvalue,indexandarrayexist. To compare with an outer value, use a code step. - A function that is not deterministic does not compile (
expr-nondeterministic):now,millis,uuid,uid,random,shuffleand the date functions with no input.
Steps
A step is an object with an id and exactly one kind key: tool, ask, code, check, when, foreach, await, hold or finish. Each step can also have a note, a text for the person who reads the procedure. Steps run in the order of the list.
Tool step
A check is an expression string, or {"check": expr, "message": text}. It passes only when its value is truthy and nothing throws.
Limit
limit counts the first call, each retry after a timeout or a network error, and each retry_with. A call that got a 429 or 503 answer did not run, so it does not count. The default is 1 + retry.max for a read or idempotent step, else 1. Add 1 when a case has retry_with. An irreversible step must have "limit": 1.
Ask step
An ask step gets one value that the procedure cannot compute: a label, a choice or a short text.
askis the question: a text with optional{{@ref}}parts, or{"template": text}.answer(required) is the JSON Schema of the answer, with atypeor anenum. An empty schema{}does not compile.{"enum": {"table": t, "of": "values"}}takes the enum from a table ("keys"takes its keys). The output of the step has this type.rungis the list of answerers, in order. The default is["model", "person"].decide: a decision model. It answers anenumor a boolean only (ask-decide).model: the exception model, with a grammar made fromanswer.person: the run stops with a hold. The client sends the answer. Read Answer a hold.
- The engine goes to the next rung when a rung gives no valid answer, or a confidence under
min_confidence. contextlists the values that the answerer sees. The exception model and a person also see the receipts of the run.- A string answer with no
enumand nopatterngives the warningask-enum.
An answer that can come from model is unchecked data, as an extraction is. A document can steer a model to a wrong value, also inside an enum. A closed answer (an enum or a boolean) counts as checked only when the rungs start with decide and min_confidence is set. Otherwise, a check that reads the answer must run before a write uses it (write-unchecked).
Code step
A code step maps data when an expression cannot: it merges two lists, removes duplicates or picks the row whose dates cover a date.
- The code is a JavaScript module with a default export. It gets
inputand gives a JSON value. - It runs in an isolated sandbox with no network. JSON goes in and JSON comes out.
returns(required) is the type of the output. An output that does not match is a step error with the codecode_failed.- The effect of a code step is always
read. - Limits: 64 KiB of source, 256 KiB of input and of output, and
timeout_msof at most 1000.
Check step
A false check runs else. The default else is a hold with the message as the reason. A check step that reads an extraction or a model answer marks those values as checked.
When block
The first case whose if is truthy runs. else is required (when-else): a list of steps, [] for “nothing to do”, or {"hold": "reason"}. An if that throws counts as false.
Foreach block
foreachis a list value: a reference, anexpror the output of a code step.as(required) is the name of the item.whereis a filter expression.distinctis an expression on the item. The loop skips an item whose value it saw before, and logs it.max_items: more items is a step error with the codetoo_many_items, and the default is a hold. With nomax_items, the compiler gives the warningforeach-max.- Items run in order, one at a time.
Hold
A hold step ends the run with the outcome held. A person gets the reason (a text or a template) and the attached values, and finishes the work outside the run. The reason is required (hold-reason). A hold step does not run the compensations.
An ask with the rung person also stops the run, but that hold can resume: the client sends the answer and the run continues.
Await
The run waits until each named later input arrives. Read a later input only after its await (await-order).
Finish
The status is done, refused or aborted. The run ends. aborted runs the compensations.
Order and reachability
A step after a hold or a finish in the same list never runs, so it does not compile (unreachable). The top-level finish applies when the run gets to the end of steps.
Invariants
The engine checks each invariant before each write or irreversible call. call is {tool, step, args}, and calls is the list of the done calls [{tool, step, args, output}]. A false invariant does not run the call: the run holds.
Errors and the exception ladder
on_error
Transient errors
retry is {"max": n, "codes": [...]}. The default max is 2. The engine knows two classes of transient error:
- Not run: HTTP
429and503, with the codesrate_limitedandunavailable. The answer proves that the call did not run. The engine retries it on every effect, at mostretry.maxtimes, with backoff. These retries do not count towardlimit. - Unknown outcome: a timeout, a network error, HTTP
500,502and504, with the codestimeoutandnetwork. The call can have run. The engine retries only areadstep or anidempotentstep, withinretry.maxandlimit. Another step holds at once with the codeunknown_outcome.
codes adds codes of the tool to the not-run class, for example a code that the tool documents as “nothing changed”. timeout or network in codes of an irreversible step does not compile (irreversible-retry).
Do not write cases for transient errors. The engine does that work.
Cases
The engine tries the cases in order. A case matches the error code, and its optional if. The steps of its do run, then its then applies:
The steps of a case do stay visible after the step, as the steps of a when case do.
A failed require or expect is an error with the code require_failed or check_failed, and error.message is the message of the check. It goes to cases and else, never to retry.
Retry of a write
On a write step, each value of retry_with must come from a source that you trust (retry-unverified):
@error.<key>of a tool of your list, when itsx-errorsschema for that code declares<key>. Your system names the replacement.- A tool output, a table lookup, a literal, or the answer of an ask with
"rung": ["person"].
A value that passed through a model answer, a decision model answer or a document extraction is not allowed. The compiler follows the value through code steps and expressions. A document value can select a row of a tool list (a find over the list): the value comes from the tool. When no value that you trust exists, hold with the reason, or ask a person in the case do.
else
else applies when no case matches:
"hold": the default.{"classify": "decide"}or{"classify": "model"}: the decision model or the exception model picks one of the case codes of the step, ornone. A pick goes back to the cases with that code.{"patch": "model", "max": n}: the exception model writes only the arguments to change, under a grammar made frominputSchema. A patch that changes nothing fails the rung.{"patch": "person", "max": n}: the run stops with a hold that can resume. A person sends the arguments to change.
The ladder
The engine applies the rungs in a fixed order:
- Transient retry, as above.
- Procedure branch: the first case that matches.
- Decide:
else: {"classify": "decide"}. - Model:
else: {"patch": "model"}or{"classify": "model"}. - Person: a hold with the error, the step and the receipts.
The run log marks each hold that a rung from 3 to 5 gives, each error that no case matched and each invariant hold. Read these marks to change the procedure.
When the exception model is off for a run (options.exception_model: "off"), or not available, each model rung goes to the next rung.
Unknown outcome
A call can start and give no result, for example after a crash. A read call and an idempotent call run again with the same idempotency key. The engine does not send a write call (with no idempotent) or an irreversible call again: the run holds with the code unknown_outcome.
Repeated writes
A write or irreversible step does not call the same tool with the same arguments two times in one run. The engine compares the arguments as JSON with sorted keys. A repeat does not run: the step fails with the code repeat_write, and the default is a hold. This catches a list that has the same item two times.
"allow_repeat": trueon the step turns off the rule. Two equal items can be two orders.distincton aforeachskips the repeated items before a step runs.retry_withand the transient retries are the same activation, not a repeat.
Compensation
compensate is allowed on write steps only (compensate-effect). When the run ends with aborted, the engine calls the compensations of the done steps, in the reverse order of completion. The log records each compensation. A failed compensation holds the run. A hold and a cancel do not compensate: a person decides.
Lints
The compiler gives one diagnostic per problem: {level, code, path, message}. The path is the place in the procedure, for example steps[3].do[1].args.amount. The message says what to change.
An error stops the compile. A warning does not, unless the request has "strict": true.
The API adds these checks of the tool list and of the size:
Write a procedure
Before you save: the checklist of the compiler
- Each key that you read exists in the output schema of its tool, or in the extraction schema.
- Each
whenhas anelse. - Each irreversible step (a payment, a submit, a release, an approval) has
"limit": 1, no retry, and a check before it. - Each write uses extracted values only after a check that reads the extraction. An
expecton the extraction is enough. - Each ask has an
answerschema. Each hold has a reason. - Each required argument of a tool has a value of the correct type.
- The report names the reference numbers that the process asks for:
{{@close.invoice_number}}.
When the process text does not settle something
A run must finish the work for a normal input. An ask that no answerer can get right stops every run. Before you write an ask, name who answers it on every run, and from what data.
- A step with no rule. The text says “check the open rentals of the customer”, but gives no rule that blocks the work. Call the read tool and show its result in the report. Do not add an ask or a check that decides from it.
- A required argument that the text does not give:
- A key of a record (an id, an account, a number): take it from a tool output, the document or the message. When nothing can give it, hold with a reason.
- An amount, a quantity or a date: take it from the document or a tool output only. Never from an ask.
- A free text label of a new record (a charge code, a category): ask the exception model with a
patternand the context that it needs. Then add a check step that reads the answer, before the write. Use"rung": ["model", "person"], not["person"]alone.
- Hold only where the text says to stop: a refusal, a mismatch or a limit that the text gives.
- A retry of a write takes a value that your system gave. Read Retry of a write.
Worked examples
A lookup that can find nothing
After skip, read the step in an argument only inside a when whose if reads it.
Error recovery with another tool
When the error output names the fix (replacement_unit), use @error.<key> in retry_with. No model is necessary. The x-errors schema of the code must declare the key.
A label that the model writes
A value that an earlier step sent
The tool rent_reserve_unit does not return the depot, but the step book sent it. Outside a loop, sent.book is a list.
Merge two lists, each person once
Frequent compiler errors
A full example
This procedure closes a rental contract from a return form. It compiles with the tool list below it.