---
id: aws--s3-enumerator
title: "S3 enumerator"
summary: "Lists the objects of an Amazon S3 bucket (or of an S3-compatible store such as MinIO), optionally filtered by key prefix, folder level, name pattern, age and an object tag, and emits one file info record per object. Use it with *S3 reader* to process the files of a bucket."
audience: user
keywords: "input, output, continue, authentication, accesskeyid, secretaccesskey, sessiontoken, rolearn, externalid, stsserviceurl, region, serviceurl, forcepathstyle, bucketname, prefix, skiptag, toplevelonly, namepattern, minageminutes, orderby, maxobjects"
---
# S3 enumerator

Lists the objects of an Amazon S3 bucket (or of an S3-compatible store such as MinIO), optionally filtered by key prefix, folder level, name pattern, age and an object tag, and emits one file info record per object. Use it with *S3 reader* to process the files of a bucket.

## Pins

| Pin | Kind | Types | Description |
|---|---|---|---|
| Input | input | any | Starts the enumeration (for example a timer). |
| Output | output | `Viaducta.TypeSystem.Metadata.ViaFileInfo` | Called once per object. |
| Continue | continue output | same as input | Called with the input record after the last object. |

## Configuration

| Field | Type | Default | Description |
|---|---|---|---|
| `Authentication` | dropdown: `0` AccessKey, `1` DefaultChain, `2` AssumeRole | `0` | How to sign in. |
| `AccessKeyId` | string | | Access key id. Required for `0`. For `2` optional: with it the role is assumed with this key, without it with the default chain. |
| `SecretAccessKey` | password | | Secret of `AccessKeyId`. Required with `AccessKeyId`. |
| `SessionToken` | password | | Session token of temporary credentials; optional. |
| `RoleArn` | string | | `2`: ARN of the role to assume, for example `arn:aws:iam::123456789012:role/import`. Required for `2`. |
| `ExternalId` | string | | `2`: external id the trust policy of the role requires; optional. |
| `StsServiceUrl` | string | | `2`: STS endpoint. Empty: the regional AWS STS endpoint of `Region`. For an S3-compatible store with STS (MinIO) the same URL as `ServiceUrl`. |
| `Region` | string | | AWS region of the bucket, for example `eu-central-1`. Required when `ServiceUrl` is empty; with `ServiceUrl` the signing region (empty: `us-east-1`). |
| `ServiceUrl` | string | | Endpoint of an S3-compatible store or an S3 VPC endpoint, `http(s)` without query, for example `http://minio:9000`. Empty: AWS S3 in `Region`. |
| `ForcePathStyle` | boolean | `false` | `true` addresses `<endpoint>/<bucket>/<key>` instead of `<bucket>.<endpoint>`; most S3-compatible stores need it. |
| `BucketName` | string | | Required. Bucket to list. |
| `Prefix` | string | | Optional key prefix, for example `incoming/2026/`. Empty lists all objects. |
| `SkipTag` | string | | Object tag name; objects that have this tag are not emitted (for example `viaducta-processed`, set by *S3 reader* post-processing `3` Tag). Costs one request per object (see Notes). Empty lists all. |
| `TopLevelOnly` | boolean | `false` | `true` lists only the objects directly under `Prefix` (end it with `/`), not those in deeper folders. |
| `NamePattern` | string | `*` | Only objects whose key after the last `/` matches; `*` and `?` as wildcards, case-insensitive, for example `*.csv`. |
| `MinAgeMinutes` | integer | `0` | Only objects not modified for at least this many minutes, so objects still being written are skipped. `0` lists all. |
| `OrderBy` | dropdown: `0` Name, `1` LastModified | `0` | Emit order: by key (as S3 lists, UTF-8 binary order) or oldest modification first. |
| `MaxObjects` | integer | `0` | Emit at most this many objects per run, the first ones in `OrderBy` order. `0` emits all. |

## Behaviour

- The S3 client is created at flow start; missing or invalid sign-in fields fail the start. `1` DefaultChain uses the AWS default credential chain: environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`), the shared credentials file and profile (`AWS_PROFILE`), web identity (`AWS_WEB_IDENTITY_TOKEN_FILE` and `AWS_ROLE_ARN`, as on EKS), container credentials (ECS, EKS Pod Identity) and the EC2 instance profile. `2` AssumeRole calls STS `AssumeRole` (session name `viaducta`) and renews the temporary credentials five minutes before they expire. With `1` and `2` the credentials are fetched with the first S3 call. The identity needs `s3:ListBucket` (and `s3:GetObjectTagging` with `SkipTag`).
- Per input the whole listing is read first (`ListObjectsV2`; all levels, or with `TopLevelOnly` only the objects directly under `Prefix`), folder markers (keys ending with `/`) are dropped, then the objects are filtered by `NamePattern` and `MinAgeMinutes`, ordered by `OrderBy`, checked for `SkipTag` and cut to `MaxObjects`; then they are emitted. Changes downstream components make in the bucket during the run (for example *S3 reader* moving objects) don't change the list.
- Each object is emitted as `ViaFileInfo`: `FileName` = key (including folders), `FilePath` = `s3://<bucket>/<key>` (every key segment URL-escaped, for example `s3://imports/daily/my%20file.csv`), `Size` = size in bytes, `CreateDateTime` = last modification (UTC; S3 keeps no creation time), `LastAccessDateTime` = `DateTime.MinValue`.
- The progress bar shows `<emitted>/<total> objects` (while a large bucket is listed: `listing, <n> objects`, while tags are read: `reading tags, <n> objects`); the run ends with the status `<emitted>/<total> objects listed`.
- After the last object `Continue` is called with the input record. A missing bucket raises an error. No record positions are set on the emitted records.

## Notes

- Connect `Output` to *S3 reader* to read each listed object: the reader takes bucket and key from `FilePath` (leave its `BucketName`/`Key` empty). With post-processing in the reader (delete, move, tag + `SkipTag`) a timer-driven flow processes each object once.
- `SkipTag` cost: the listing carries no tags, so every candidate object costs one `GetObjectTagging` request (after the other filters, and only until `MaxObjects` objects are found). For large backlogs prefer reader post-processing `2` Move to a prefix outside `Prefix`, or a small `MaxObjects`.
- The listing is kept in memory for the run; use `Prefix` to narrow very large buckets. `MaxObjects` limits what is emitted, not what is listed.
- `MaxObjects` with `OrderBy` `1` and reader post-processing processes a backlog oldest first in slices, one slice per timer run.
- `1` DefaultChain in the platform: shared-profile entries that assume a role (`role_arn`) or use AWS IAM Identity Center (`sso_session`) are not supported; use `2` AssumeRole or environment credentials instead.
- Store secrets as expression settings and reference them (`=Secrets.AwsSecretKey`) instead of typing keys into the flow.

## Example

```json
{ "Authentication": 1, "Region": "eu-central-1", "BucketName": "imports", "Prefix": "daily/", "SkipTag": "viaducta-processed", "NamePattern": "*.csv", "MinAgeMinutes": 5, "OrderBy": 1, "MaxObjects": 100 }
```

Timer → **S3 enumerator** → S3 reader → CSV decoder → ...