senjacob commented on issue #651:
URL: https://github.com/apache/tvm-ffi/issues/651#issuecomment-4882690311
Details from ChatGPT..
## Root cause
The backend detection appears to rely on `_find_rocm_home()`.
Current implementation:
```python
def _find_rocm_home():
...
hipcc_path = shutil.which("hipcc")
if hipcc_path is not None:
...
else:
rocm_home = "/opt/rocm"
if not Path(rocm_home).exists():
raise RuntimeError(...)
```
The backend detection uses `_find_rocm_home()` to determine whether HIP is
available.
If `/opt/rocm` exists, ROCm is considered available, even if:
* `hipcc` does not exist
* PyTorch is CUDA-only
* the active GPU is NVIDIA
This causes CUDA systems with an installed ROCm directory to be
misclassified as ROCm systems.
---
## Suggested fixes
### Option 1 (preferred)
Use PyTorch to determine the active backend:
```python
import torch
if torch.version.cuda is not None:
return "cuda"
if torch.version.hip is not None:
return "hip"
```
### Option 2
Require `hipcc` to exist before enabling the HIP backend.
Instead of:
```python
rocm_home = "/opt/rocm"
if Path(rocm_home).exists():
...
```
check for:
```python
Path("/opt/rocm/bin/hipcc").exists()
```
before selecting ROCm.
---
## Impact
This prevents SGLang AWQ Marlin JIT compilation on CUDA systems that happen
to have a ROCm installation directory present, even though the system is using
NVIDIA GPUs exclusively.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
