Eliaaazzz commented on code in PR #40718:
URL: https://github.com/apache/beam/pull/40718#discussion_r4355912492
##########
sdks/python/apache_beam/ml/inference/base.py:
##########
@@ -179,6 +179,25 @@ class KeyModelPathMapping(Generic[KeyT]):
model_id: str = ''
+def _validate_batching_args(
+ max_batch_weight=None,
+ element_size_fn=None,
+ batch_length_fn=None,
+ batch_bucket_boundaries=None,
+):
+ if max_batch_weight is not None:
+ if element_size_fn is None:
+ raise ValueError('max_batch_weight requires element_size_fn to be set.')
+ if batch_length_fn is not None or batch_bucket_boundaries is not None:
Review Comment:
Could we also mention this restriction in the public parameter docs for
`max_batch_weight` and `batch_length_fn`, including the remote handlers? They
currently describe the options independently, so a user combining a token
budget with length bucketing only discovers the incompatibility when
constructing the handler. A short sentence saying they're mutually exclusive
would make the supported combinations clearer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
