tgravescs commented on issue #26078: [SPARK-29151][CORE] Support fractional resources for task resource scheduling URL: https://github.com/apache/spark/pull/26078#issuecomment-545435935 So this feature is specifically to allow multiple tasks to use the same GPU. Many times one task can't keep the gpu busy, so running multiple tasks gets better utilization and through put. For instance one task can copy the data to gpu and start running and while its running another task can start to copy the data so that when the first task finishes the data for second task is there and ready to run. Also many times one tasks data/kernel isn't enough to fully utilize the GPU so you can run 2 tasks in parallel. Obviously it requires the framework/code that runs on the GPU to be able to handle this.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
