Lunderberg commented on PR #16183: URL: https://github.com/apache/tvm/pull/16183#issuecomment-2273515528
Thank you, and I'm looking into where the performance overhead comes from. Unfortunately, I'm not sure if the de-coupling of the boxed types from the automatic conversions is possible. 1. Since the FFI is used during runtime, the default FFI conversions must not use IR types. Therefore, a function that accepts `ObjectRef`, and is passed a python integer, has the argument converted to `runtime::Int` instead of the previous `IntImm`. 2. A python list must be converted to an `tvm::Array` to be passed through the FFI. Since `Array` holds an `ObjectRef`, this requires converting any primitive values into `ObjectRef` types. With (1), this would produce an array of `runtime::Int` rather than the previous array of `IntImm`. 3. For a function that accepts `Array<IntImm>`, existing callers in python could pass the array `[1,2,3]`, and this usage should not be broken. With the new default conversion for `[1,2,3]`, this would raise an error. Breaking any one of these three steps would avoid the error, but each comes with downsides: * Break step (3): Automatic conversion from `Array<runtime::Int>` to `Array<IntImm>`. This is what was implemented in the PR, but results in a two-step conversion. * Break step (2): Change `Array` to hold a `TVMRetValue` instead of `ObjectRef`. This would avoid the repeated nested conversion, since the C++ API could ide, but would significantly change the C++ side of the API. * Break step (1): An explicit change to the Python code to convert to `IntImm` before calling the FFI. This would work in some cases where the desired C++ type is known (e.g. the shape of a TIR buffer), but wouldn't work in others (e.g. the target attributes, which are stored as `ObjectRef`, and whose desired type isn't known until they are used). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
