schizophrenicmaniac opened a new pull request, #41407: URL: https://github.com/apache/beam/pull/41407
Fixes #41351 RunInference caches the loaded model with `shared.Shared`, which keeps a weak reference to it. If a `ModelHandler.load_model()` returns something like a dict (a common way to bundle a tokenizer and a model), setup fails with `TypeError: cannot create weak reference to 'dict' object`. `_RunInferenceDoFn._load_model` now checks whether the loaded model can be weakly referenced and, if not, caches it inside a small holder object. It unwraps the model before handing it to `run_inference`, so handlers still get the dict they returned. The DoFn keeps a strong reference to the cached holder so it isn't collected while in use. The multi-process and ModelManager paths don't go through `shared.Shared` and are unchanged. This also fixes the `with_exception_handling()` docstring. It said the dead-letter outputs contain `(failed batch, exception)`, but what's actually emitted is `(failed batch, (exception type, repr(exception), traceback.format_exception() lines))`. The docstring now says that. Added tests for running a pipeline with a dict model, for reusing a cached dict model across new DoFn instances, and for the shape of the error info in `failed_inferences`. Also added a CHANGES.md entry. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
