tannnmayy opened a new pull request, #41343:
URL: https://github.com/apache/beam/pull/41343

   `large_model=True` on an embedding transform, for example 
`SentenceTransformerEmbeddings`, is documented as sharing the model across 
processes, but it has no effect. MLTransform wraps the model handler in 
`_EmbeddingHandler`, which doesn't forward `share_model_across_processes()`, so 
RunInference always sees `False`.
   
   This PR forwards it to the wrapped handler.
   
   Testing:
   - New unit test in `base_test.py`. It fails without the change.
   - Multi-process pipelines with `large_model=True`: 
`SentenceTransformerEmbeddings` (`all-MiniLM-L6-v2`, 4 workers) loads the model 
4 times on master and once with this change. `TensorflowHubTextEmbeddings` (3 
workers) goes from 3 loads to 1, with the same output.
   
   #41051 added `_DelegatingModelHandler` for the wrappers in 
`ml/inference/base.py`. `_EmbeddingHandler` isn't one of them and still doesn't 
forward other hooks, such as `model_copies()`. I kept this PR to the one hook 
with a user-facing option. I can move `_EmbeddingHandler` onto 
`_DelegatingModelHandler` in a follow-up if you'd prefer that.
   
   R: @damccorm
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to