js8544 commented on issue #39024: URL: https://github.com/apache/arrow/issues/39024#issuecomment-1835688067
The problem occurs because the kernel you registered is for Int16, hence the `checked_cast<arrow::internal::PrimitiveScalarBase&>`. There are no generic support for `ExtensionType`s in arrow compute. Reusing the storage types' kernels may work (like your `add(arr1, arr2)`) but there's no guarantee. So for now you'll have to write your own kernel to make sure it works. However, I think it's worthwhile to provide general support for ExtensionTypes, and it's not hard to do so: 1. First check if the function has a matching kernel for the extention types. If so, execute it and return. 2. Replace the extension types with their storage types and dispatch again. If there is a match, replace the input `ExtensionArray` or `ExtensionScalar` with their storage values and execute the kernel. This way users can simply define their own extention types and reuse the existing compute functions. No need to write extra kernels and registrations etc. Does this sound like a viable approach? Also cc @pitrou. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
