ahmedabu98 opened a new pull request, #34020: URL: https://github.com/apache/beam/pull/34020
Experimenting with GroupIntoBatches' key parallelism when writing to Iceberg. Configure the following options to run under different scenarios: ``` java // ======== experiment with these numbers =========== int numShards = 1; long payloadSize = 1 << 10; // 1KB int numIcebergPartitions = 0; // ================================================== ``` Try to figure out what the best throughput-per-shard is that leads to large files while maintaining good performance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
