IMPORTANT: This documentation has been discontinued. Read the updated File Reader documentation on our new documentation portal.
File Reader reads a local file and converts it into a JSON structure that can be manipulated inside the pipeline. The component supports the reading of multi-line text files or binary files.
Take a look at the configuration parameters of the component:
File Name: local file to be read.
Charset: name of the characters code for the file reading (standard UTF-8).
Binary File: if “true”, the file is considered binary and the reading consists in a string with BASE64 representation of the file content; if "false", the file is read as text.
Maximum File Size: specifies the maximum size allowed (in bytes); if the file size is greater than the informed value, then a reading error will be thrown.
Read As A Single String: if "true", the text file will be read as an unique string; if "false", the text file will be read as an array of strings in which each item represents a line from the file.
Fail On Error: if the option is enabled, the execution of the pipeline with error will be interrupted; otherwise, the pipeline execution proceeds, but the result will show a false value for the “success” property.
How are text files read?
The text files are read line by line. A structure is formed at the component output, with all the read lines:
{
"data" : [
"line 1",
"line 2",
...
],
"fileName": "",
"lineCount": 0
}
data: has a JSON array of converted lines
filename: name of the file use as source for the component
lineCount: amount of line read from the file
If the Read As A Single String parameter goes as "true", then the returned structure will contain all the lines from the file read in one unique string:
{
"data" : [
"line 1\r\nline2\r\nline n\r\n"
],
"fileName": "",
"lineCount": 1
}
Notice that, in this case, the lines reading includes the line break character(s). Generally, if the file is created in Unix-based systems, only the line break with \n will be returned. On the other hand, if the file is created in Windows-based systems, the line break with \r\n will be returned.
Handling a characters set (Charset)
For text files to be read, it’s important to set the charset with the same value that was used during the creation of the file. If an incompatible characters set is used, the file might be read and misinterpreted. That takes to imprecisions with special characters, such as accented letters and others.
How are binary files read?
Binary files can't have their content naturally expressed in properties inside JSON messages, once many binary characters aren't "printable". That way, the content of binary files is transformed into a base64 string:
{
"data" : "VGhpcyBpcyBhIHRlc3QuCg==",
"fileName": "",
"lineCount": 0
}
Notice that, in the example above, the "data" property isn't presented as an array, but as a single base64 string.
IMPORTANT: the manipulation of files inside a pipeline is made in a protected way. All the files are accessed through one temporary directory only, that is created at each pipeline execution.