Workflows over HTTP

Run several tools wired together in one request: a product handed downstream, a step that runs only if a condition holds, and a step that runs once per item of a list.

What it doesCopy Anchor Link

The HTTP API runs one tool per request. A workflow is a graph of tool calls, the same graph the workflow builder in the editor draws, and it adds the three things a single call cannot say:

• Wires. One step's output becomes another step's input. Wired into $subject, a step's product becomes the molecule the next step works on, so a digest runs on the amplicon rather than on the template.
• Conditions. A step carrying $whenExpression runs only when the expression is true. When it is false the step is reported as unmet, which is an answer, not a failure.
• For-each. A step carrying scatter: ["<input>"] runs once per element of that input, and every output comes back as a list with one entry per element.

Every step is a tool from the tool listing, with exactly the parameters, bounds and defaults that listing documents. There is no second set of workflow-only tools.

The requestCopy Anchor Link

{
  "graph": {
    "name": "a label for the run record",
    "nodes": [{ "id": "pcr", "tool": "pcr_amplify", "params": { "forward": "…", "reverse": "…" } }],
    "edges": [{ "id": "w1", "from": { "node": "pcr", "port": "record" }, "to": { "node": "cuts", "port": "$subject" } }]
  },
  "_subject": { "sequence": "…", "name": "…", "circular": true }
}

A node is { id, tool, params }. A wire is { id, from: { node, port }, to: { node, port } }: the from port is an output name from the tool listing, the to port an input name or $subject. _subject at the top level of the body (a wire still names the port $subject) is the molecule every step works on unless a wire hands it a different one, in the same shape the single-tool endpoints take.

A worked example: amplify an insert, then characterise the productCopy Anchor Link

The template is the bundled pUC57-LoxP-GSK3-LoxP plasmid, 8,767 bp, circular. We amplify between the two universal sequencing primers, M13 forward (−20) GTAAAACGACGGCCAGT and M13 reverse CAGGAAACAGCTATGAC, then count four enzymes' sites on the product and size it on a gel. Both downstream steps are wired to the PCR step's record output, so they see the amplicon, not the plasmid.

curl -s https://<host>/api/workflows/run \
  -H 'content-type: application/json' \
  -d '{
    "graph": {
      "nodes": [
        { "id": "pcr", "tool": "pcr_amplify",
          "params": { "forward": "GTAAAACGACGGCCAGT", "reverse": "CAGGAAACAGCTATGAC" } },
        { "id": "counts", "tool": "enzyme_cut_counts",
          "params": { "enzymes": ["EcoRI", "BamHI", "HindIII", "XhoI"] } },
        { "id": "gel", "tool": "run_gel_lane", "params": { "source": "plasmid" } }
      ],
      "edges": [
        { "id": "w1", "from": { "node": "pcr", "port": "record" }, "to": { "node": "counts", "port": "$subject" } },
        { "id": "w2", "from": { "node": "pcr", "port": "record" }, "to": { "node": "gel", "port": "$subject" } }
      ]
    },
    "_subject": { "name": "pUC57-LoxP-GSK3-LoxP", "circular": true, "sequence": "…8,767 bases…" }
  }'

The run record answers step by step, in the order the steps ran:

The product length is the arithmetic you can check by hand: the forward primer starts at base 379 and the reverse primer's site ends at base 6,562 of the plasmid, which is 6,184 bases inclusive. The site counts are the plasmid's own sites that fall inside that span (HindIII at 5,627, XhoI at 2,576 and 2,675). The gel lane reads the product as a linear molecule because it arrived as one: had the lane been run on the plasmid itself it would show supercoiled and nicked forms instead.

The same graph, with a conditionCopy Anchor Link

Add "$whenExpression": "subject.circular" to a step's params and it runs only on a circular molecule. Sent a map step that way, with the plasmid as _subject:

The run as a whole is still ok: true in both cases. A condition exists to leave a branch unrun, so an unrun branch is not an error.

A condition can read the subject (subject.length, subject.circular) and the step's own inputs, with == != < > <= >= and or not + - * / % ?? and the functions abs, ceil, contains, floor, isset, len, lower, max, min, number, round, upper. There is no method call and no ternary: a condition returns one verdict. The interactive reference at /api/openapi prints the same list from the same declaration.

The same check over three primers, with for-eachCopy Anchor Link

scatter runs one step once per element. Checking where three primers anneal on the plasmid is one step rather than three requests:

{ "id": "check", "tool": "primer_check",
  "params": { "primer": ["GTAAAACGACGGCCAGT", "CAGGAAACAGCTATGAC", "TAATACGACTCACTATAGGG"] },
  "scatter": ["primer"] }

The step reports three jobs, and each output is a list in the order the primers were given: siteCount is [1, 1, 0]. M13 forward anneals once on the top strand and M13 reverse once on the bottom strand, both as exact matches; the T7 promoter primer has no site on this plasmid. Each job's own input, result and status is kept under jobs, so "two of three bound" is visible without re-running anything.

Checking a graph before running itCopy Anchor Link

POST /api/workflows/validate takes the same body and runs nothing. It answers ok: true with no issues for a graph that will run, or ok: false with each blocking problem naming its step or wire: an unknown tool, a wire into a port that does not exist, a cycle. /run performs the same check first and refuses a graph that fails it with 400 and the same issue list, so a pipeline is never half run because step four was misspelled.

What the status codes meanCopy Anchor Link

• 200 — the run happened. Read nodes: each step is ok, error (the tool declined, with its reason), skipped (a step it depends on did not produce anything) or unmet (its condition was false). A step failing does not fail the request, because "step three of six declined, and here is why" is the answer a caller needs and a status code cannot carry it.
• 400 — the request itself is unusable: no graph, a _subject that cannot be read, or a graph that cannot run. The body says which.

Starting from a templateCopy Anchor Link

GET /api/workflows/templates lists the ready-made pipelines with what each does and which tools it chains (restriction map, diagnostic digest, colony PCR screen, align and build a tree, batch Sanger verification and others). ?id=starter_amplify_characterize returns one graph, which can be posted straight back to /run once its open parameters (the primers, the enzymes) are filled in.

Limits worth knowingCopy Anchor Link

• A run is one request, with a budget. There is no job to poll. A graph holds at most 100 steps, and a run may compute for 120 seconds: past that, the steps that finished keep their results, the rest are marked cancelled, and stoppedBy says why. The validation step is what stops a graph that cannot run from starting at all.
• Templates that read the project (the list of traces in a project, a named document) need an editor session. Over HTTP every step works on _subject or on what is wired into it.