pvillard31 commented on PR #7893: URL: https://github.com/apache/nifi/pull/7893#issuecomment-1793568862
Test with 15,598,440 records (295MB): - Normal ConvertRecord (Parquet/Parquet) - 2 min 13 sec - Calculate Parquet Offsets (1M records) -> ConvertRecord 8 concurrent tasks - 16FFs - 38 seconds It seems to be very similar to the previous test since the time is really related to how many records should be in every split (1M in this case). It's a 4x improvement when optimizing the number of splits versus the number of concurrent tasks so it's definitely a nice improvement. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
