aIbrahiim commented on issue #30644: URL: https://github.com/apache/beam/issues/30644#issuecomment-4335982756
> > Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver > > This is confusing - this means we actually did spin up the job, but drivers weren't accessible; do you have an example of a job like this? > > > I also observed quota related signals in the same run window (T4 GPU quota pressure in us-central1) > > Do you have an example of other quota signals? Do you mean we're experiencing stockouts? One option would be to create a reservation - http://console.cloud.google.com/compute/reservations?referrer=search&project=apache-beam-testing&tab=reservations yes right maybe I got confused but the job started, but workers failed at CUDA init with Found no NVIDIA driver (torch._C._cuda_init()), i.e. GPU runtime mismatch for this benchmark path. Example failing job: 2026-04-11_17_06_13-14337956449089835368 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
