---
id: queues--de-duplicate-filter
title: "Duplicate check"
summary: "Passes a record to `Output` only the first time its key (the value of `FilterProperty`) is seen, optionally within a time window; repeats go to `Already exists`. Use it in front of anything that must not process the same item twice."
audience: user
keywords: "input, output, already exists, filterproperty, filterlifetimeinmsec"
---
# Duplicate check

Passes a record to `Output` only the first time its key (the value of `FilterProperty`) is seen, optionally within a time window; repeats go to `Already exists`. Use it in front of anything that must not process the same item twice.

## Pins

| Pin | Kind | Types | Description |
|---|---|---|---|
| Input | input | any | Record to check. |
| Output | output | same as input | Record whose key was not seen (or whose earlier key expired), with the original metadata. |
| Already exists | continue | any | Record whose key was already seen within the window, with the original metadata. |

## Configuration

| Field | Type | Default | Description |
|---|---|---|---|
| `FilterProperty` | dropdown (property of the input type) | | Required. Property whose value is the duplicate key. Options appear after the input is connected. |
| `FilterLifetimeInMsec` | integer (≥ 0) | `0` | How long a key is remembered, in milliseconds. `0` = forever (until the flow stops). |

## Behaviour

- At start the component builds its own key set for `FilterProperty` (the input type is taken from the `Input` connection). Without `FilterProperty` the flow fails to start.
- For each record the key is added atomically: a new key, or a key whose lifetime has passed, goes to `Output`; a key still remembered goes to `Already exists`.
- Keys keep the type of the property (a number stays a number, compared by value); a record whose key is null always goes to `Already exists`.
- Keys are never released by later components: only the lifetime ends them.

## Notes

- The key set lives in memory, belongs to this component and is empty after a flow restart. It is not shared with queues; a queue's own `FilterProperty` gives the same check inside the queue (and can be persisted).
- With `FilterLifetimeInMsec` = `0` the set grows with every distinct key for as long as the flow runs; set a lifetime for unbounded key spaces.
- Safe for parallel callers: two records with the same new key at the same time pass only once.

## Example

```json
{ "FilterProperty": "OrderId", "FilterLifetimeInMsec": 600000 }
```

Http listener → **Duplicate check** → Queue ; `Already exists` → Http response (409)