pabloem commented on pull request #15246: URL: https://github.com/apache/beam/pull/15246#issuecomment-892889266
hmmm I think we usually try to avoid giving this kind of knob to users, because it's easy for these parameters to become obsolete, and hardcoded in legacy pipelines.. I'm wondering - do you think there's another way of accomplishing this? In Dataflow we have GroupIntoBatches.withAutoSharding, which provides a sharding parameter that scales automatically with the number of workers. Perhaps this would help? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
