---
id: aws--s3-writer
title: "S3 writer"
summary: "Uploads text or a file stream as an object to Amazon S3 (or an S3-compatible store such as MinIO) and passes the input on. Sets content type, metadata and storage class, can refuse to overwrite and can create the bucket. Large streams are uploaded in parts."
audience: user
keywords: "input, output, authentication, accesskeyid, secretaccesskey, sessiontoken, rolearn, externalid, stsserviceurl, region, serviceurl, forcepathstyle, bucketname, createbucket, key, overwrite, contenttype, storageclass, metadata"
---
# S3 writer

Uploads text or a file stream as an object to Amazon S3 (or an S3-compatible store such as MinIO) and passes the input on. Sets content type, metadata and storage class, can refuse to overwrite and can create the bucket. Large streams are uploaded in parts.

## Pins

| Pin | Kind | Types | Description |
|---|---|---|---|
| Input | input | `Viaducta.Plugins.TextPlugin.Types.TextRecord`, `Viaducta.Plugins.Files.Types.ViaFileStream` | Content to store: `TextRecord.Value` (UTF-8) or the stream. |
| Output | output | same as input | Called with the input record after a successful write. |

## Configuration

| Field | Type | Default | Description |
|---|---|---|---|
| `Authentication` | dropdown: `0` AccessKey, `1` DefaultChain, `2` AssumeRole | `0` | How to sign in. |
| `AccessKeyId` | string | | Access key id. Required for `0`. For `2` optional: with it the role is assumed with this key, without it with the default chain. |
| `SecretAccessKey` | password | | Secret of `AccessKeyId`. Required with `AccessKeyId`. |
| `SessionToken` | password | | Session token of temporary credentials; optional. |
| `RoleArn` | string | | `2`: ARN of the role to assume. Required for `2`. |
| `ExternalId` | string | | `2`: external id the trust policy of the role requires; optional. |
| `StsServiceUrl` | string | | `2`: STS endpoint. Empty: the regional AWS STS endpoint of `Region`. |
| `Region` | string | | AWS region of the bucket, for example `eu-central-1`. Required when `ServiceUrl` is empty; with `ServiceUrl` the signing region (empty: `us-east-1`). |
| `ServiceUrl` | string | | Endpoint of an S3-compatible store or an S3 VPC endpoint, `http(s)` without query. Empty: AWS S3 in `Region`. |
| `ForcePathStyle` | boolean | `false` | `true` addresses `<endpoint>/<bucket>/<key>`; most S3-compatible stores need it. |
| `BucketName` | string | | Target bucket. |
| `CreateBucket` | boolean | `false` | `true` creates the bucket (in `Region`, private) when it does not exist. |
| `Key` | string | | Target key including folders, for example `exports/daily.csv`. Ending with `/` it is a folder and the record's name is appended (see Behaviour). |
| `Overwrite` | boolean | `true` | `true` replaces an existing object; `false` fails the write (HTTP 412) when the object exists. |
| `ContentType` | string | | Content type of the object, for example `text/csv`. Empty: the `ContentType` of the `TextRecord`, otherwise from the extension of the key (`.csv`, `.json`, `.jsonl`, `.xml`, `.txt`, `.pdf`, `.zip`, `.gz`, `.parquet`, images, ...), otherwise `application/octet-stream`. |
| `StorageClass` | dropdown: `0` Default, `1` Standard, `2` StandardInfrequentAccess, `3` OneZoneInfrequentAccess, `4` IntelligentTiering, `5` GlacierInstantRetrieval, `6` GlacierFlexibleRetrieval, `7` GlacierDeepArchive, `8` ReducedRedundancy | `0` | Storage class of the object; `0` sends none (the bucket default, Standard). |
| `Metadata` | dictionary | | Name/value pairs stored as user metadata (`x-amz-meta-<name>`). |

## Behaviour

- The S3 client is created at flow start; missing sign-in fields or an unknown storage class fail the start. Sign-in modes as in *S3 enumerator*. The identity needs `s3:PutObject` (plus `s3:ListBucket` and `s3:CreateBucket` for `CreateBucket`).
- Key: for a `TextRecord`, its `Name` replaces `Key`, or is appended when `Key` ends with `/`; without a `Name`, `Key` is used. For a `ViaFileStream`, `Key` is used; when it is empty the stream's `Name` is used, and when it ends with `/` the stream's `Name` is appended. An empty result fails the record.
- With `CreateBucket` the bucket is created on the first write of the run if it is missing.
- Content up to 16 MB is sent in one PUT. A larger stream is uploaded in parts of 16 MB as it is read (no length or seeking needed); a failed multipart upload is aborted. Content type, metadata and storage class are set on the object.
- With `Overwrite` `false` the write is conditional (`If-None-Match: *`): an existing object fails it with HTTP 412 and stays unchanged.
- Then `Output` is called with the input record. Other input types fail with a "not supported" error. S3 errors (missing bucket, authentication, existing object with `Overwrite` `false`, a storage class the store doesn't offer) raise an error.

## Notes

- `ViaFileStream` input: the content of `Stream` is uploaded from its current position; the stream stays open (it belongs to the component that opened it).
- S3 objects can't be appended to, so there is no append mode as in *Blob writer*: every record writes a whole object. For logs or JSON lines write one object per record (`Key` a folder, the record name unique) or collect the records into one stream first.
- At most 10,000 parts: one object can hold up to about 156 GB.
- Use ASCII for metadata names and values; S3 returns metadata names in lower case.
- Glacier classes (`6`, `7`) take objects offline: they must be restored before they can be read. S3-compatible stores offer fewer classes (MinIO: Standard and Reduced Redundancy).
- Store secrets as expression settings (`=Secrets.AwsSecretKey`).

## Example

```json
{ "Authentication": 1, "Region": "eu-central-1", "BucketName": "exports", "CreateBucket": true, "Key": "daily/", "StorageClass": 2, "Metadata": { "source": "erp" } }
```

Timer → Sql query → Json serializer → **S3 writer** → Log (the `TextRecord.Name` of the serializer becomes the file in `daily/`)