Read file

Opens local files and passes them on as ViaFileStream (file info plus an open read stream). It either scans a folder of the environment data folder for files matching a mask (Start pin), or opens a single file given as ViaFileInfo (Input pin, e.g. from Collect files).

Pins

Pin	Kind	Types	Description
Start	start		Scans LocalFileGroup and emits every matching file. Typically connected to a Timer.
Input	input	Viaducta.TypeSystem.Metadata.ViaFileInfo	Opens the file at ViaFileInfo.FilePath (full path) and emits it.
Output	output	Viaducta.Plugins.Files.Types.ViaFileStream	Called once per file with the open stream.
NoFiles	no data	same as input	Called after a Start scan that found no files.

Configuration

Field	Type	Default	Description
FileNamePattern	string	*	File mask with * and ? wildcards (*.csv, ORD_??.xml). Empty = all files.
OrderBy	dropdown: 0 CreateTime, 1 FileName	0	Processing order: oldest creation time first, or by name ascending.
LocalFileGroup	folder (FolderExplorer)		Folder to scan, relative to the environment data folder, e.g. /inbound/orders. Required for Start; the folder must exist when the flow starts.
ErrorFileGroup	folder (FolderExplorer)		Error folder, relative to the data folder. Created if missing. A file that fails (open, downstream processing or post-processing) is moved here. Empty = failed files stay in place and the error stops the scan.
PostProcessing	dropdown: 0 None, 1 Delete, 2 MoveToGroup, 3 MoveToAchieve	0	What happens to a file after Output returned without error: None leaves it in place, Delete deletes it, MoveToGroup moves it to PostProcessFileGroup in the data folder, MoveToAchieve moves it to PostProcessFileGroup/yyyyMMdd in the archive folder.
PostProcessFileGroup	folder (FolderExplorer)		Target folder for MoveToGroup (relative to the data folder) or MoveToAchieve (relative to the archive folder). Required for those two options; created if missing.
ReprocessErrorsMin	integer	0	Every N minutes (checked on Start, first time on the first Start) all files in ErrorFileGroup are moved back to LocalFileGroup and read again. 0 = never.
ParallelReaderNo	integer	1	Number of files processed in parallel. 1 = one after another.
MaxConsecErrors	integer	10	With ErrorFileGroup set: the scan stops with FileReadTooManyErrorsException after this many files in a row failed. 0 = no limit.

Behaviour

• Start: lists the files in LocalFileGroup (top folder only, no subfolders) that match FileNamePattern, orders them by OrderBy and emits one ViaFileStream per file. If a scan is already running, the call returns immediately. If no file was found, NoFiles is called.
• Input: opens the single file given by ViaFileInfo.FilePath and emits it; NoFiles is not used.
• ViaFileStream contains Name (file name with extension), FilePath (full path), Extension (last extension, e.g. .csv), CreateDateTime, LastAccessTime, Size (bytes) and Stream (read-only FileStream, shared read).
• The stream is closed as soon as the downstream processing of Output returns, so consumers must read it synchronously.
• Errors with ErrorFileGroup set: the failed file is moved to ErrorFileGroup, the error is logged and the scan continues with the next file. After MaxConsecErrors failures in a row the scan stops and throws FileReadTooManyErrorsException (remaining files are read on the next Start). On the Input pin the file is moved and the error is rethrown.
• Errors without ErrorFileGroup: the error stops the scan and the file stays where it was (remaining files are read on the next Start).
• Every file is a batch (batch label = file name). Batch consumers downstream (e.g. Write file) commit their output when all records of the file have been processed; a failed record that reached Read file, or a failed commit, fails the file and the consumers roll back. Errors handled by a Try catch don't fail the file.
• Post-processing runs after the stream is closed, after the batch consumers committed, and only when Output returned without error. It applies to both Start and Input. If the target folder already contains a file with the same name, _yyyyMMdd_HHmmss is appended to the moved file's name.
• Progress: a Start run shows done/total files and N/M files read at the end (nothing when no file matched).

Notes

• With PostProcessing = None every Start reads the same files again unless a downstream component removes them.
• The folder picker for PostProcessFileGroup browses the data folder; with MoveToAchieve the path is used relative to the archive folder.
• A file moved back from ErrorFileGroup is skipped (left in the error folder) while a file with the same name is waiting in LocalFileGroup.
• All files are emitted with the same metadata. Components that collect several records into one result (e.g. Stream compress) therefore treat each file separately.
• With ParallelReaderNo > 1 the downstream components are called from several threads at the same time.
• Extension holds only the last extension (archive.tar.gz → .gz).

Example

{ "FileNamePattern": "*.csv", "OrderBy": 1, "LocalFileGroup": "/inbound/orders", "ParallelReaderNo": 1 }

Timer → Read file → Csv decoder → Database writer; NoFiles → Log