JingGe edited a comment on pull request #17520: URL: https://github.com/apache/flink/pull/17520#issuecomment-961072361
@tsreaper many thanks for your effort and for sharing the benchmark data. The option of using BulkFormat + ArrayList is almost the same as using StreamFormat+StreamFormatAdapter, except the memory size control. Have you tried to control the number of records each batchRead() will fetch instead of fetch all records (code reference see [here](https://github.com/tsreaper/flink/commit/3b86337cea499cd4245a34550a6b597239be3066#diff-07c21eca7aca500ba4675ecd3ace539cf31d69797fd254b7b914d22198a789baR145-R148) line 145-148)of the current block in one shot? For the option of Stream Format based on Stephan's draft, may I know how you controlled the `StreamFormat.FETCH_IO_SIZE`? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
