tqchen commented on PR #17927:
URL: https://github.com/apache/tvm/pull/17927#issuecomment-2859152015
## Benchmark
Env CPU: AMD Ryzen 9 7950X
```bash
> python ffi/scripts/benchmark_dlpack.py
-----------------------------
Benchmark f(x, y, z) overhead
-----------------------------
numpy.add 1.921653747558594e-07 sec/call
torch.add[cpu] 6.330013275146484e-07 sec/call
torch.add[cuda] 2.330756187438965e-06 sec/call
tvm.ffi.nop 3.983736038208008e-07 sec/call
tvm.ffi.nop+from_dlpack(torch) 4.368019104003906e-06 sec/call
tvm.ffi.nop+from_dlpack(numpy) 1.1694192886352538e-06 sec/call
tvm.ffi.nop+from_dlpack(tvm) 1.4580249786376954e-06 sec/call
tvm.ffi.nop+from_dlpack(torch.utils) 3.2754182815551756e-06 sec/call
tvm.ffi.nop.autodlpack(torch[cpu]) 3.567361831665039e-06 sec/call
tvm.ffi.nop.autodlpack(torch[cuda]) 3.5606861114501952e-06 sec/call
tvm.ffi.nop.autodlpack(numpy) 1.6696929931640624e-06 sec/call
-------------------------------
Benchmark x.__dlpack__ overhead
-------------------------------
torch.utils.dlpack.to_dlpack 4.5762062072753906e-07 sec/call
torch.__dlpack__ 9.840965270996094e-07 sec/call
numpy.__dlpack__ 5.011558532714844e-08 sec/call
tvm.__dlpack__ 1.5852451324462892e-07 sec/call
---------------------------------------------------
Benchmark x.__dlpack__(max_version=(1,1)) overhead
---------------------------------------------------
torch.__dlpack__(max_version=(1,1)) Tensor.__dlpack__() got an
unexpected keyword argument 'max_version'
numpy.__dlpack__(max_version=(1,1)) 6.172657012939454e-08 sec/call
tvm.__dlpack__(max_version=(1,1)) 1.720428466796875e-07 sec/call
## Discussions
- First, we can see that the overall FFI overhead of python c++ is roughly
at 0.2us -3us level
- Notably, each torch.add eager call in cuda is around 2.4 us
- AutoDLPack as of now can get to about 3.6us for a call of f(x, y, z) that
needs three import calls
- One can observe that `torch.__dlpack__` overhead is larger than
`numpy.__dlpack__`
- AutoDLPack from numpy arguments have about 1.6us overhead
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
