JingGe edited a comment on pull request #17520: URL: https://github.com/apache/flink/pull/17520#issuecomment-961072361
@tsreaper many thanks for your effort and for sharing the benchmark data. The option of using BulkFormat + ArrayList is almost the same as using StreamFormat+StreamFormatAdapter, except the memory size control. Have you tried to control the number of records each batchRead() will fetch instead of fetch all records of the current block in one shot? Code reference please see [here](https://github.com/tsreaper/flink/commit/3b86337cea499cd4245a34550a6b597239be3066#diff-07c21eca7aca500ba4675ecd3ace539cf31d69797fd254b7b914d22198a789baR145-R148) line 145-148. For the option of Stream Format based on Stephan's draft, may I know how you controlled the `StreamFormat.FETCH_IO_SIZE`? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
