Viaducta Documentation

Appearance

AI in flows ​

Two plugins put a language model into a flow:

  • The LLM chat client sends text to a model and passes the answer on. Use it to classify messages, extract fields from free text, summarise, translate, or answer questions behind an HTTP endpoint.
  • The AI tool exposes a branch of your flow as a function the model can call while it answers: look up an order, check stock, create a ticket.

These plugins are configured per component and are independent of the designer's assistant: a flow can use a different provider and model.

A question-answering endpoint ​

text
Http listener → LLM chat client → Http response
                    ▲ Register
                    │
                AI tool "order_status" → (Output) Sql query → Data mapper → (Input) AI tool
  1. LLM chat client. Choose the model in ModelProvider: Google Gemini, OpenAI GPT, or oAI_Compatible for any OpenAI-compatible endpoint such as vLLM, with Endpoint and ModelName. Put the API key in an expression setting and reference it in ApiKey (=Secrets.LlmKey). Write the system instructions in Prompt: the model's role, its rules, the output format, and when to use which tool.
  2. AI tool. Give it a Name (order_status) and Instructions that tell the model what the tool does and what its arguments mean. Connect its Register pin to the chat client's Register pin.
  3. Arguments. The type on the AI tool's Output connection defines the arguments: the model fills an object of that type (for example an OrderStatusRequest with OrderId), and the record goes to the components behind Output.
  4. Result. Connect the end of the tool branch back to the AI tool's Input. That record is returned to the model as the tool result, and the model continues its answer.

The chat client takes its message from a TextRecord or from the body of an HTTP request, and sends the final answer on Output as a TextRecord.

Conversations ​

With SessionHandling set to keep sessions, messages with the same SessionKey continue one conversation, so the model remembers earlier questions. Bind the key to something that identifies the conversation, for example {meta.SessionId}. Conversations not used for IdleSessionTimeout seconds (one hour by default) are discarded. Without sessions, every message is answered on its own.

Things to know ​

  • Each tool call must produce exactly one record on the AI tool's Input, within Timeout (30 seconds by default).
  • Connect each AI tool to one chat client. Tool names must be unique per chat client.
  • The model's answer is free text. When the next component needs structured data, tell the model in Prompt to answer in JSON and use a Json deserializer after the chat client.

Viaducta documentation