christogav commented on PR #26042: URL: https://github.com/apache/beam/pull/26042#issuecomment-1513121984
> Style comments aside, is there any reason we shouldn't just make the standard Read and Query calls use an SDF with partitioned queries under the hood? Adding splitting at that level improves existing pipelines when they update Beam, and avoids unnecessary thinking about "should I ReadBatch, or Read?". > > In particular when without an SDF the benefits of ReadBatch here aren't clear. @lostluck Not all queries can be partitioned and the value of forcing partitioned reading on an implementor in lower volume scenarios may(?) decrease performance, not increase it. I'm not sure that implementors can avoid consideration of whether batched reads are appropriate. I believe the java maintainers must have faced a similar dilemma as they have Naive/Batch also. Thoughts? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
