ManvithPanyam commented on issue #25892: URL: https://github.com/apache/beam/issues/25892#issuecomment-5998218371
Hi @lostluck, I'd like to work on this issue as a Go SDK contribution. I traced the current Go TextIO splitting path on master and confirmed that SplitRestriction uses a fixed 64 MB chunk size, while the Java/Python implementations use the adaptive sqrt-based strategy described here. I'd like to investigate implementing the reusable chunk-sizing logic in the offsetrange restriction layer and update TextIO to use it, along with regression/unit tests. Please let me know if this approach still makes sense for the current Beam 3.0 direction. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
