[jira] [Commented] (NIFI-4496) Improve performance of CSVReader

[ASF GitHub Bot (JIRA)]

    [ 
https://issues.apache.org/jira/browse/NIFI-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16242089#comment-16242089
 ] 

ASF GitHub Bot commented on NIFI-4496:
--------------------------------------

Github user jdye64 commented on the issue:

    https://github.com/apache/nifi/pull/2245
  
    @mattyb149 I'm seeing invalid output when I run run an existing flow with 
this PR. I had an existing flow that used ConvertRecord and Apache Commons CSV. 
That was working fine and giving me the output I expected. However when I 
switched to using the Jackson implementation all of the output was empty. I 
have attached a screenshot from my debugger session in hopes that will help 
shed some light into what is going on.
    
    <img width="1348" alt="screen shot 2017-11-06 at 4 52 08 pm" 
src="https://user-images.githubusercontent.com/2127235/32498256-32f8ffc6-c39d-11e7-86dd-cde8f7d3a758.png";>



> Improve performance of CSVReader
> --------------------------------
>
>                 Key: NIFI-4496
>                 URL: https://issues.apache.org/jira/browse/NIFI-4496
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>            Reporter: Matt Burgess
>            Assignee: Matt Burgess
>
> During some throughput testing, it was noted that the CSVReader was not as 
> fast as desired, processing less than 50k records per second. A look at [this 
> benchmark|https://github.com/uniVocity/csv-parsers-comparison] implies that 
> the Apache Commons CSV parser (used by CSVReader) is quite slow compared to 
> others.
> From that benchmark it appears that CSVReader could be enhanced by using a 
> different CSV parser under the hood. Perhaps Jackson is the best choice, as 
> it is fast when values are quoted, and is a mature and maintained codebase.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)