ValidateCsv
Description
Validates the contents of FlowFiles against a user-specified CSV schema. Take a look at the additional documentation of this processor for some schema examples.
Tags
csv, schema, validation
Properties
In the list below required Properties are shown with an asterisk (*). Other properties are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
| Display Name | API Name | Default Value | Allowable Values | Description |
|---|---|---|---|---|
| Schema * | validate-csv-schema | The schema to be used for validation. Is expected a comma-delimited string representing the cell processors to apply. The following cell processors are allowed in the schema definition: [ParseBigDecimal, ParseBool, ParseChar, ParseDate, ParseDouble, ParseInt, ParseLong, Optional, DMinMax, Equals, ForbidSubStr, LMinMax, NotNull, Null, RequireHashCode, RequireSubStr, Strlen, StrMinMax, StrNotNullOrEmpty, StrRegEx, Unique, UniqueHashCode, IsIncludedIn]. Note: cell processors cannot be nested except with Optional. Supports Expression Language, using FlowFile attributes and Environment variables. | ||
| Header * | validate-csv-header | true |
| True if the incoming flow file contains a header to ignore, false otherwise. |
| Delimiter character * | validate-csv-delimiter | , | Character used as 'delimiter' in the incoming data. Example: , Supports Expression Language, using FlowFile attributes and Environment variables. | |
| Quote character * | validate-csv-quote | " | Character used as 'quote' in the incoming data. Example: " Supports Expression Language, using FlowFile attributes and Environment variables. | |
| End of line symbols * | validate-csv-eol | \n | Symbols used as 'end of line' in the incoming data. Example: \n Supports Expression Language, using FlowFile attributes and Environment variables. | |
| Validation strategy * | validate-csv-strategy | FlowFile validation |
| Strategy to apply when routing input files to output relationships. |
| Include all violations * | validate-csv-violations | false |
| If true, the validation.error.message attribute would include the list of all the violations for the first invalid line. Note that setting this property to true would slightly decrease the performances as all columns would be validated. If false, a line is invalid as soon as a column is found violating the specified constraint and only this violation for the first invalid line will be included in the validation.error.message attribute. |
Dynamic Properties
This component does not support dynamic properties.
Relationships
| Name | Description |
|---|---|
| invalid | FlowFiles that are not valid according to the specified schema are routed to this relationship |
| valid | FlowFiles that are successfully validated against the schema are routed to this relationship |
Reads Attributes
This processor does not read attributes.
Writes Attributes
| Name | Description |
|---|---|
| count.invalid.lines | If line by line validation, number of invalid lines extracted from the source data |
| count.total.lines | If line by line validation, total number of lines in the source data |
| count.valid.lines | If line by line validation, number of valid lines extracted from the source data |
| validation.error.message | For flow files routed to invalid, message of the first validation error |
State Management
This component does not store state.
Restricted
This component is not restricted.
Input Requirement
This component requires an incoming relationship.
System Resource Considerations
This component does not specify system resource considerations.