Running flows

When you start a flow, its Controller hands it to a Host. The Host turns the flow into a .NET assembly, loads it and calls its start pins. From then on, records pass from component to component as method calls inside that process.

Compilation

1. Types become classes. Every type the flow uses is generated as a C# class, together with its JSON serialization code.
2. The flow becomes code. The components, their configuration and the connections between them are generated as C#, and so are the bindings in configuration fields. C# script components are part of the same code.
3. Roslyn compiles it. The result is one assembly per flow, loaded into its own isolated load context, so a flow is replaced or unloaded without touching the other flows in the Host.

A mistake in a script, a binding or a type mapping stops the flow at compilation with the component and the error, before any record is processed.

Execution

A component receives a record on an input pin, does its work and passes its result to an output pin. The record reaches the next component through a direct call, in the same process: there is no bus and no serialization between components. Records are serialized only where they leave the flow, for example to the data monitor, to a log entry or to another system.

Components can work synchronously or asynchronously. Listener components (Http listener, Kafka consumer, TCP server) process requests concurrently; Fork, the queue plugins and Scatter / Gather move work to other threads or split and join it.

How a flow starts, stays alive and stops is described in Concepts.

Scripting engines

Language	Component	Runs as
C#	C# script, C# shared script	Compiled into the flow's assembly, at the speed of the rest of the flow.
Python	Python script	CPython 3.11 or newer (3.12 by default), embedded in the Host. Packages installed into that Python are available to the script.
JavaScript	JavaScript script	The V8 engine, embedded in the Host.

All three see the same flow services: logging, storage, assets, repository access and batches, under the same names.

Hosting strategies

Each flow has a hosting strategy that decides which Host process it runs in.

Strategy	Runs the flow	Use it for
Private host	in a process of its own	Flows that must be isolated: a crash, a memory leak in a library or a restart affects that flow only.
Tag based host	in one process with the other flows of the same hosting tag	Groups of related flows. They share memory and can share listener ports, so many small flows stay cheap.
Shared host	in one process with all the environment's shared-host flows	Development, debugging and small installations. About 100 flows fit into a container with 1 GB of memory.

You change a flow's strategy in its settings; the next start uses the new one.

Flows calling flows

Need	Use
Reuse a flow as a component, with a request and a response	Publish it as a plugin and place it in other flows.
Start another flow	Start flow, optionally waiting until it finishes.
Send records to a long-running flow	Pipe client and Pipe listener.
Call a flow from outside Viaducta	Webhook receiver, Rest handler, Http listener, SOAP handler, or TCP, UDP and WebSocket listeners.

Sync call orders the steps inside one flow; it does not call another flow.