raulcd opened a new issue, #48783: URL: https://github.com/apache/arrow/issues/48783
### Describe the enhancement requested The runs-on GPU instances we use are built [with CUDA 12](https://github.com/runs-on/runner-images-for-aws?tab=readme-ov-file#gpu). See log from `nvidia-smi`: ``` | NVIDIA-SMI 575.57.08 Driver Version: 575.57.08 CUDA Version: 12.9 | ``` I tried installing `cuda-compat-13-0` as suggested [here](https://github.com/apache/arrow/pull/48583#issuecomment-3719253654): ``` +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 575.57.08 Driver Version: 575.57.08 CUDA Version: 13.0 | |-----------------------------------------+------------------------+----------------------+ ``` But the job still failed with: ``` cuda.core._utils.cuda_utils.CUDAError: CUDA_ERROR_INVALID_IMAGE: This indicates that the device kernel image is invalid. This can also indicate an invalid CUDA module. ``` We might need to, build our own instances with packer installing newer versions of CUDA as suggested [here](https://runs-on.com/runners/gpu/#workflow-job-definition-1), wait until runs-on provides images with CUDA 13 or try reinstalling the drivers on the instance. This is a follow up issue to: - https://github.com/apache/arrow/issues/48582 ### Component(s) Continuous Integration, GPU -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
