damccorm commented on code in PR #37186:
URL: https://github.com/apache/beam/pull/37186#discussion_r3292824966
##########
sdks/python/apache_beam/examples/inference/pytorch_imagenet_rightfit.py:
##########
@@ -495,7 +475,9 @@ def run(
predictions = (
to_infer
- | 'RunInference' >> RunInference(KeyedModelHandler(model_handler)))
+ | 'RunInference' >> RunInference(
+ KeyedModelHandler(model_handler)).with_resource_hints(
+
accelerator="type:nvidia-tesla-t4;count:1;install-nvidia-driver"))
Review Comment:
Thanks for adding the resource hint. However, because of how resource hints
work, this will currently add a GPU for the whole pipeline. The way resource
hints work is they ask for the requested resource for the whole stage
containing this step. Because this pipeline doesn't contain any shuffles, it
will all end up being in the same stage, so it will all get the T4.
To fix this, could you please add a Reshuffle step directly before
RunInference?
I'm not sure if this will make the pipeline run more efficiently or not;
because it is primarily bound on IO, not compute, the benefits might not
outweigh the shuffle costs. That is good for us to understand regardless,
though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
