nitinlkoin1984 commented on issue #23932: URL: https://github.com/apache/beam/issues/23932#issuecomment-1340179401
thanks for reply. One of our requirement is to perform large scale parallel/distributed ML inference. Our ML models need GPUs to do quick inference. I was wondering how the data is transferred between spark worker node and SDK service. Seems like converting data and send it back and forth between spark worker and sdk service may involve lot of overhead. Will a native Spark job be faster then a Beam Job? Is there a performance hit when we write jobs in Beam instead of Native Spark? I am evaluating Beam because it provides with RunInference API to do large scale distributed ML inference. I cannot find such functionality in Native Spark. I will have to write my own custom component if I have to go with Spark. Where is the inference code executed. Is it executed in the SDK harness service. If so can that service use the underlying GPUs. Also can I run any pytorch and HuggingFace Transformed model using RunInference. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
