---
id: aws--s3-operation
title: "S3 operation"
summary: "Deletes, copies, moves, tags or changes the storage class of an object in Amazon S3 (or an S3-compatible store such as MinIO). The object comes from an incoming object file info (*S3 enumerator*) or from the configuration; the file info of the resulting object is passed on. Use it for housekeeping (archive or delete old objects) or to move objects between buckets."
audience: user
keywords: "input, output, nodata, authentication, accesskeyid, secretaccesskey, sessiontoken, rolearn, externalid, stsserviceurl, region, serviceurl, forcepathstyle, operation, bucketname, key, targetbucket, targetnamepattern, existingobjectoption, tag, storageclass"
---
# S3 operation

Deletes, copies, moves, tags or changes the storage class of an object in Amazon S3 (or an S3-compatible store such as MinIO). The object comes from an incoming object file info (*S3 enumerator*) or from the configuration; the file info of the resulting object is passed on. Use it for housekeeping (archive or delete old objects) or to move objects between buckets.

## Pins

| Pin | Kind | Types | Description |
|---|---|---|---|
| Input | input | `Viaducta.TypeSystem.Metadata.ViaFileInfo` | Triggers the operation. A `ViaFileInfo` whose `FilePath` is an `s3://` URL selects the object; any other input uses `BucketName`/`Key`. |
| Output | output | `Viaducta.TypeSystem.Metadata.ViaFileInfo` | Called after the operation with the object's file info: the target of a copy or move, otherwise the object itself. |
| NoData | no-data output | same as input | Called with the input record when the bucket or object does not exist (HTTP 404). |

## Configuration

| Field | Type | Default | Description |
|---|---|---|---|
| `Authentication` | dropdown: `0` AccessKey, `1` DefaultChain, `2` AssumeRole | `0` | How to sign in. |
| `AccessKeyId` | string | | Access key id. Required for `0`. For `2` optional: with it the role is assumed with this key, without it with the default chain. |
| `SecretAccessKey` | password | | Secret of `AccessKeyId`. Required with `AccessKeyId`. |
| `SessionToken` | password | | Session token of temporary credentials; optional. |
| `RoleArn` | string | | `2`: ARN of the role to assume. Required for `2`. |
| `ExternalId` | string | | `2`: external id the trust policy of the role requires; optional. |
| `StsServiceUrl` | string | | `2`: STS endpoint. Empty: the regional AWS STS endpoint of `Region`. |
| `Region` | string | | AWS region of the bucket, for example `eu-central-1`. Required when `ServiceUrl` is empty; with `ServiceUrl` the signing region (empty: `us-east-1`). |
| `ServiceUrl` | string | | Endpoint of an S3-compatible store or an S3 VPC endpoint, `http(s)` without query. Empty: AWS S3 in `Region`. |
| `ForcePathStyle` | boolean | `false` | `true` addresses `<endpoint>/<bucket>/<key>`; most S3-compatible stores need it. |
| `Operation` | dropdown: `0` Delete, `1` Copy, `2` Move, `3` Tag, `4` ChangeStorageClass | `0` | What to do with the object. |
| `BucketName` | string | | Bucket of the object, used when the input is not an object `ViaFileInfo`. |
| `Key` | string | | Object key including folders, used when the input is not an object `ViaFileInfo`. |
| `TargetBucket` | string | | Copy/Move: target bucket (must exist); empty keeps the object's bucket. |
| `TargetNamePattern` | string | `%SN%` | Copy/Move: target key. `%SN%` source key with folders, `%FN%` its last part (after the last `/`), `%BN%` the last part without extension, `%TS%` `yyyyMMdd-HHmmss`, `%DATE%` `yyyyMMdd`, `%TIME%` `HHmmss` (UTC). |
| `ExistingObjectOption` | dropdown: `0` Fail, `1` Overwrite, `2` Rename | `0` | Copy/Move when the target exists: fail (HTTP 412), replace it, or write to `<name>_yyyyMMdd_HHmmss<ext>` (plus `_1`, `_2`, ... if that exists too). |
| `Tag` | string | `viaducta-processed` | Tag: object tag to add, `name` or `name=value`; without a value the UTC time is the value. Other tags are kept. |
| `StorageClass` | dropdown: `1` Standard, `2` StandardInfrequentAccess, `3` OneZoneInfrequentAccess, `4` IntelligentTiering, `5` GlacierInstantRetrieval, `6` GlacierFlexibleRetrieval, `7` GlacierDeepArchive, `8` ReducedRedundancy | `2` | ChangeStorageClass: the new storage class. |

## Behaviour

- The S3 client is created at flow start; missing sign-in fields, an unknown operation or existing-object option, Tag without a name or ChangeStorageClass without a class fail the start. Sign-in modes as in *S3 enumerator*. The identity needs the S3 permissions of the operation (`s3:DeleteObject`; `s3:GetObject`, `s3:PutObject`, `s3:GetObjectTagging`, `s3:PutObjectTagging` for copies; `s3:DeleteObject` too for Move).
- Per input the object is chosen (from the `ViaFileInfo` `s3://` URL or `BucketName`/`Key`; with neither the input fails) and its metadata is read (HEAD). A 404 calls `NoData` with the input record.
- `0` Delete deletes the object. `1` Copy copies it server-side to `<TargetBucket or the same bucket>/<TargetNamePattern>` with content type, metadata, tags and storage class. `2` Move copies, then deletes the source. `3` Tag adds `Tag` to the object's tags. `4` ChangeStorageClass copies the object onto itself with the new class (content, metadata and tags stay; the last modification time changes); an object that already has the class is left as it is.
- Delete, Copy, Move and ChangeStorageClass only act on the version whose metadata was read: an object changed meanwhile fails with HTTP 412.
- `Output` is called with a `ViaFileInfo` (`FileName` = key, `FilePath` = `s3://<bucket>/<key>`, size and last modification): of the target for Copy and Move, of the object otherwise (for Delete the object that was deleted). S3 errors raise an error.

## Notes

- Copy and move stay within the endpoint, region and sign-in; the target bucket must exist. Objects over 5 GB are copied in parts of 512 MB (content type, metadata and tags are carried over; other headers such as `Cache-Control` are not).
- AWS S3 checks `ExistingObjectOption` `0` and `2` atomically (`If-None-Match` on the copy) and the version on Delete (`If-Match`). Some S3-compatible stores (MinIO) ignore these headers; there a HEAD request right before does the check, which leaves a short window.
- Objects in Glacier Flexible Retrieval or Deep Archive must be restored before they can be copied, moved or changed to another class (S3 returns `InvalidObjectState`); restoring is not part of this plugin. S3 allows at most 10 tags per object.
- In a versioned bucket Delete and Move leave a delete marker, and ChangeStorageClass adds a version.
- Store secrets as expression settings (`=Secrets.AwsSecretKey`).

## Example

```json
{ "Authentication": 1, "Region": "eu-central-1", "Operation": 2, "TargetBucket": "archive", "TargetNamePattern": "%DATE%/%FN%", "ExistingObjectOption": 2 }
```

Timer → S3 enumerator (`OrderBy` `1`, `MinAgeMinutes` `1440`) → **S3 operation** → Log