[jira] [Commented] (NIFI-6670) Create a RecordReader that reads lines of text into single-field records
[
https://issues.apache.org/jira/browse/NIFI-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308923#comment-17308923
]
Matt Burgess commented on NIFI-6670:
Closing my PR for now due to inactivity and the fact that you can use a
GrokReader with Schema Access Strategy set to "Use String Fields from Grok
Expression" (the default) and a Grok Expression of
{{%{GREEDYDATA:message}}}
This creates a record with a field "message" containing the line. It will also
include additional fields for stack trace (on error) and the line as a "raw"
value in the "_raw" field (via NIFI-4081)
> Create a RecordReader that reads lines of text into single-field records
>
>
> Key: NIFI-6670
> URL: https://issues.apache.org/jira/browse/NIFI-6670
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> It would be nice to have a reader that can take any textual input and treat
> each "line" as a single-field record. This is like CSVReader but there
> wouldn't be a field delimiter; rather, a property to specify the name of the
> field, and each line becomes a value for that field in the record.
> Additional capabilities could be added as well, such as skipping header
> lines, grouping lines together as a single field value, ignoring empty lines,
> etc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-6670) Create a RecordReader that reads lines of text into single-field records
[ https://issues.apache.org/jira/browse/NIFI-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929438#comment-16929438 ] Matt Burgess commented on NIFI-6670: This might also be possible with GrokReader, but you have to supply the nominal schema rather than just specifying the field name. TextLineReader would be much more straightforward. One of the use cases is to take a file full of SQL statements and execute them as a transaction using PutDatabaseRecord. That avoids the need for splitting and using PutSQL, and results in a more efficient flow. > Create a RecordReader that reads lines of text into single-field records > > > Key: NIFI-6670 > URL: https://issues.apache.org/jira/browse/NIFI-6670 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Matt Burgess >Priority: Major > > It would be nice to have a reader that can take any textual input and treat > each "line" as a single-field record. This is like CSVReader but there > wouldn't be a field delimiter; rather, a property to specify the name of the > field, and each line becomes a value for that field in the record. > Additional capabilities could be added as well, such as skipping header > lines, grouping lines together as a single field value, ignoring empty lines, > etc. -- This message was sent by Atlassian Jira (v8.3.2#803003)
