Filed https://issues.apache.org/jira/browse/NIFI-11077
On Thu, Jan 19, 2023 at 4:02 PM Joe Witt <joe.w...@gmail.com> wrote: > Matt > > We're probably thinking of similar concepts but wanted to be sure - my > thought here is the RecordReader interface keeps track of how many records > have been read. If a call to next reader bails for any given impl then one > could reliably check how many records have been read and thus know the > index of the failed one. I've not looked closely at the API so there is > probably a cleaner model. But this feels like a RecordReader level > capability rather than specific to any particular implementation such as > CSVRecordReader. > > Thanks > > On Thu, Jan 19, 2023 at 1:59 PM Matt Burgess <mattyb...@apache.org> wrote: > > > I was thinking the same, CSVRecordReader could keep track of the > > number of records read and if an exception is thrown during iteration > > over reading the records, we can output the number of records read > > successfully. > > > > On Thu, Jan 19, 2023 at 3:47 PM Joe Witt <joe.w...@gmail.com> wrote: > > > > > > Dan, > > > > > > Seems like our record reader mechanism should offer the concept of > > tracking > > > which record it is on such that this could be logged. It looks from a > > > quick check like we track record count on writing so something similar > on > > > the interface of the reader could be quite helpful. > > > > > > Perhaps best to file a JIRA. Someone else might have a better idea of > > what > > > you can do now. > > > > > > Thanks > > > > > > On Thu, Jan 19, 2023 at 1:39 PM Dan S <dsti...@gmail.com> wrote: > > > > > > > For both QueryRecord and ValidateRecord when I use a CSVReader on a > > file > > > > which has different delimiters than the rest of the file, the error > > message > > > > logged does not include the line number where the parsing failed. > When > > > > looking at the code, I did not see any hooks for getting that > > information. > > > > Is there a way to get the line number so it would be easy to identify > > > > which lines would need to be fixed? > > > > > > >
