AWS

Read and write objects in Amazon S3 and in S3-compatible stores such as MinIO. The S3 enumerator lists the objects of a bucket, the S3 reader opens one as a file stream (one batch per object, then delete, move or tag it), the S3 writer uploads text or a file stream, and S3 operation deletes, copies, moves, tags or changes the storage class of an object.

Timer → S3 enumerator → S3 reader → CSV decoder → ... your logic

The reader emits the same file stream the local file plugins use, so every decoder works on S3 objects as it does on local files. Sign in with an access key, the default AWS credential chain (IAM role, instance profile, environment) or an assumed role.

Plugins

Plugin	What it does	Pins
S3 enumerator	Lists the objects of an Amazon S3 bucket (or of an S3-compatible store such as MinIO), optionally filtered by key prefix, folder level, name pattern, age and an object tag, and emits one file info record per object. Use it with S3 reader to process the files of a bucket.	Input, Output, Continue
S3 operation	Deletes, copies, moves, tags or changes the storage class of an object in Amazon S3 (or an S3-compatible store such as MinIO). The object comes from an incoming object file info (S3 enumerator) or from the configuration; the file info of the resulting object is passed on. Use it for housekeeping (archive or delete old objects) or to move objects between buckets.	Input, Output, NoData
S3 reader	Reads an object from Amazon S3 (or an S3-compatible store such as MinIO) and emits it as a file stream, for example for a CSV or ASN.1 decoder. Every object is a batch: once it is processed and committed downstream, the object can be deleted, moved or tagged; a failed object can be moved to an error location. The object comes from an incoming object file info (S3 enumerator) or from the configuration. If the object does not exist, the input is passed to NoData.	Input, Output, NoData
S3 writer	Uploads text or a file stream as an object to Amazon S3 (or an S3-compatible store such as MinIO) and passes the input on. Sets content type, metadata and storage class, can refuse to overwrite and can create the bucket. Large streams are uploaded in parts.	Input, Output