schizophrenicmaniac opened a new pull request, #41407:
URL: https://github.com/apache/beam/pull/41407

   Fixes #41351
   
   RunInference caches the loaded model with `shared.Shared`, which keeps a 
weak reference to it. If a `ModelHandler.load_model()` returns something like a 
dict (a common way to bundle a tokenizer and a model), setup fails with 
`TypeError: cannot create weak reference to 'dict' object`. 
`_RunInferenceDoFn._load_model` now checks whether the loaded model can be 
weakly referenced and, if not, caches it inside a small holder object. It 
unwraps the model before handing it to `run_inference`, so handlers still get 
the dict they returned. The DoFn keeps a strong reference to the cached holder 
so it isn't collected while in use. The multi-process and ModelManager paths 
don't go through `shared.Shared` and are unchanged.
   
   This also fixes the `with_exception_handling()` docstring. It said the 
dead-letter outputs contain `(failed batch, exception)`, but what's actually 
emitted is `(failed batch, (exception type, repr(exception), 
traceback.format_exception() lines))`. The docstring now says that.
   
   Added tests for running a pipeline with a dict model, for reusing a cached 
dict model across new DoFn instances, and for the shape of the error info in 
`failed_inferences`. Also added a CHANGES.md entry.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to