---
id: ai-in-flows
title: "AI in flows"
summary: "Use a language model inside a flow with the LLM chat client, and let the model call flow logic through AI tool components - for classification, extraction, answering questions and chat front ends."
audience: user
keywords: "llm, ai, chat client, ai tool, function calling, tools, gemini, openai, gpt, vllm, openai-compatible, prompt, session"
---
# AI in flows

Two plugins put a language model into a flow:

- The [LLM chat client](/documentation/plugins/llm/llm-chat-client) sends text to a model and passes the answer on. Use it to
  classify messages, extract fields from free text, summarise, translate, or answer questions behind an HTTP endpoint.
- The [AI tool](/documentation/plugins/llm/llm-tool) exposes a branch of your flow as a function the model can call while it
  answers: look up an order, check stock, create a ticket.

These plugins are configured per component and are independent of the designer's assistant: a flow can use a
different provider and model.

## A question-answering endpoint

```text
Http listener → LLM chat client → Http response
                    ▲ Register
                    │
                AI tool "order_status" → (Output) Sql query → Data mapper → (Input) AI tool
```

1. **LLM chat client.** Choose the model in **ModelProvider**: Google Gemini, OpenAI GPT, or
   **oAI_Compatible** for any OpenAI-compatible endpoint such as vLLM, with **Endpoint** and **ModelName**. Put the API
   key in an expression setting and reference it in **ApiKey** (`=Secrets.LlmKey`). Write the system instructions in
   **Prompt**: the model's role, its rules, the output format, and when to use which tool.
2. **AI tool.** Give it a **Name** (`order_status`) and **Instructions** that tell the model what the tool does and
   what its arguments mean. Connect its **Register** pin to the chat client's **Register** pin.
3. **Arguments.** The type on the AI tool's **Output** connection defines the arguments: the model fills an object of
   that type (for example an `OrderStatusRequest` with `OrderId`), and the record goes to the components behind
   **Output**.
4. **Result.** Connect the end of the tool branch back to the AI tool's **Input**. That record is returned to the
   model as the tool result, and the model continues its answer.

The chat client takes its message from a `TextRecord` or from the body of an HTTP request, and sends the final answer
on **Output** as a `TextRecord`.

## Conversations

With **SessionHandling** set to keep sessions, messages with the same **SessionKey** continue one conversation, so
the model remembers earlier questions. Bind the key to something that identifies the conversation, for example
`{meta.SessionId}`. Conversations not used for **IdleSessionTimeout** seconds (one hour by default) are discarded.
Without sessions, every message is answered on its own.

## Things to know

- Each tool call must produce exactly one record on the AI tool's **Input**, within **Timeout** (30 seconds by
  default).
- Connect each AI tool to one chat client. Tool names must be unique per chat client.
- The model's answer is free text. When the next component needs structured data, tell the model in **Prompt** to
  answer in JSON and use a [Json deserializer](/documentation/plugins/text/json-de-serializer) after the chat client.