Concepts

These are the ideas every flow uses. Each section links to the pages that go deeper.

Flows

A flow is a set of connected components that does one job: poll a table every minute and push the new rows to an API, answer a REST call, decode a file of call records. A flow belongs to a namespace, lives in a folder of the repository, and is started, stopped and monitored on its own.

Viaducta compiles each flow into its own .NET assembly. Changing a flow and starting it again compiles it again; other flows keep running.

Plugins and components

A plugin is a kind of building block: Timer, Sql query, Http client, C# script. A component is one plugin placed in a flow, with its own configuration. A flow can hold several components of the same plugin, each configured differently.

Plugins ship with Viaducta and cover databases, files, web services, message buses, network protocols, SMS, codecs, scripting and flow control. Each has a reference page under Plugins. You can also publish a flow as a plugin and use it in other flows like any other component.

Pins and connections

Components exchange records through pins. You connect an output pin of one component to an input pin of the next; the connection is the path a record takes.

Pin kind	What it does
Input	Receives records.
Output	Sends the component's result on.
Continue	Called when the component has done its work and has no record to pass on, or to go on after a step, for example after a Timer tick or once a table has been read.
Start	Called once by the system when the flow starts, if left unconnected. Timers, listeners and consumers start here.
No data	Called when there was nothing to return, for example a query without rows.
Error, Timeout	Called when the component reports an error or a timeout instead of raising it.
Validate, Report, Register, On connect	Outputs with a specific purpose, used by a few plugins.

A record sent from an output reaches every connection whose input accepts the record's type. Connections can also carry a route id: the Decision router uses it to send a record down one branch only.

Calls along a connection are direct method calls inside the host process. A component waits for the components after it to finish before it returns, unless the plugin hands the work to another thread, as a Fork or a queue does.

Records and types

Every record has a type: a class with named, typed fields, nested types and lists. Types come from three places:

• The type editor, where you define them field by field.
• Imports from a JSON or YAML sample, or a JSON Schema or OpenAPI document.
• Components that know their data: a Table reader generates the type of its table, a CSV decoder the type of its columns, an ASN.1 decoder one type per definition in the module.

When the flow is compiled, every type becomes a C# class. Scripts work with real properties (order.Customer.Name), the Data mapper maps fields between types, and the same types can be exported as code for other applications.

Metadata

Next to the record, every message carries metadata: a correlation id and a session id that follow the message through the flow, plus entries that components add for the components after them. A Try catch puts the error details into the metadata of the record it sends to Catch. An Http listener takes the correlation and session id from the ViaCorrelationId and ViaSessionId request headers, so a caller can follow its request through the logs.

Configuration values

A component's configuration fields take one of these:

• A literal: the value as typed.
• An expression setting: =Package.Setting reads a value from an expression settings package of the environment. Use it for anything that changes between environments: URLs, connection strings, credentials, folder names. The flow stays the same from development to production; each environment holds its own values.
• A binding: text with placeholders in braces, compiled with the flow and evaluated for every message. {input.Customer.Name} reads the record, {meta.CorrelationId} the metadata, {config.BaseUrl} the flow configuration, and {settings("Api.Path")} an expression setting. Expressions such as {input.Price * 2} and formats such as Total: {input.Price:N2} EUR work too. Write {{ and }} for a literal brace.
• A pre-execution script in C#, Python or JavaScript, for a value that needs code to compute.

Starting and stopping

When a flow starts, the system calls every unconnected Start pin, and every unconnected input pin that accepts an empty record, once.

• A flow that only does work in those start calls, such as a one-off import, ends when they return.
• A flow stays up when one of its components keeps it alive (Timer, Http listener, Kafka consumer and other listeners), when its keep alive setting is on, or when it runs as a published plugin.

Flows marked auto start start with the environment. An auto-start flow can name other flows it depends on; it then starts only once those are running.

Errors

An exception in a component stops the processing of that record and travels back along the path to the component that started it: a Timer logs it and carries on with the next tick, an Http listener answers the request with a JSON error and status 500. To handle errors in the flow, put a Try catch in front of the components that can fail: the record goes to its Catch pin with the error in the metadata. Many components also have an Error or Timeout pin you connect instead. Retry runs a branch again, Error raises an error of your own, and Alarm raises an alarm for the operators.

Environments and promotion

An execution environment is a place where flows run: development, test, production, or one environment per business area. Each environment has its own controller, its own repository of flows, types and settings, and its own host processes. One designer works with all the environments of a tenant; you switch between them at the top of the explorer.

The repository of an environment is plain files, versioned with Git. You commit a flow in one environment and pull it in the next, so the same flow moves from development to test to production while each environment keeps its own expression settings, database links and certificates. Export and import is the alternative when Git is not used.