tvalentyn commented on issue #40644: URL: https://github.com/apache/beam/issues/40644#issuecomment-6226908874
Couple other issues identified by an LLM (I haven't looked in detail but seems convincing): BatchElements emits empty batches Symptom: when a single element's size exceeds the current target batch size, BatchElements first emits an empty batch [], then the element. Cause: both non-stateful DoFns flush when the running size plus the next element's size exceeds the target, without checking whether the batch is empty. Impact: RunInference calls run_inference([]) on the model handler. Handlers that index batch[0], such as MaybeKeyedModelHandler, or stack tensors can crash. This gets more likely once weight limits work (bug 1), because one long input is enough to trigger it. Keyed handlers pass (key, example) pairs to element_size_fn and batch_length_fn Symptom: with KeyedModelHandler or MaybeKeyedModelHandler, the wrapped handler's functions receive a (key, example) tuple instead of the example. For example, element_size_fn=len returns 2 for every element. Cause: both wrappers forward the inner handler's batch_elements_kwargs() unchanged, but BatchElements runs on the keyed elements. Impact: size limits and length bucketing are silently wrong for keyed inference, a common pattern for joining predictions back to inputs. Two 60-character examples were batched together despite a weight cap of 100, and every element lands in length bucket 0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
