Eliaaazzz commented on issue #40644: URL: https://github.com/apache/beam/issues/40644#issuecomment-6229962391
Thanks for flagging this, and sorry for the trouble. No, this was never finished; I left that gap in #37428. Since 2.72, max_batch_weight has been passed to BatchElements, which does not accept it. My tests only checked the kwargs dict, so they missed the pipeline construction failure. #37532 added support in SortAndBatchElements, which RunInference never uses. I reproduced both other issues on master. A first element heavier than the current target triggers an empty batch in both non-stateful DoFns. The keyed wrappers pass the full tuple to element_size_fn, so len returns 2 for every keyed example. I lean toward fixing it, since BatchElements has no separate count and weight limits today, and your #40603 already covers all three issues. If that works for you, I can open a PR on top of #40603 with the validation you mentioned and a test that builds and runs a pipeline. If you plan to land #40603 yourself, I am glad to review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
