[jira] [Commented] (ARROW-7017) [C++] Refactor AddKernel to support other operations and types
[ https://issues.apache.org/jira/browse/ARROW-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116221#comment-17116221 ] Wes McKinney commented on ARROW-7017: - I think the path forward here is to refactor to utilize common implementations of for both the LLVM IR and pre-compiled kernels. In other words, what is currently in the gandiva/precompiled directory would be moved to some place where we can arrange so that these implementations are translated to LLVM IR for use in Gandiva, while available as inline C/C++ functions for use in creating pre-compiled vectorized kernels. Having multiple implementations of the scalar "unit of work" does not seem desirable > [C++] Refactor AddKernel to support other operations and types > -- > > Key: ARROW-7017 > URL: https://issues.apache.org/jira/browse/ARROW-7017 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Francois Saint-Jacques >Assignee: Wes McKinney >Priority: Major > Labels: analytics > > * Should avoid using builders (and/or NULLs) since the output shape is known > a compute time. > * Should be refatored to support other operations, e.g. Substraction, > Multiplication. > * Should have a overflow, underflow detection mode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7017) [C++] Refactor AddKernel to support other operations and types
[ https://issues.apache.org/jira/browse/ARROW-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17092916#comment-17092916 ] Wes McKinney commented on ARROW-7017: - I'm working on this > [C++] Refactor AddKernel to support other operations and types > -- > > Key: ARROW-7017 > URL: https://issues.apache.org/jira/browse/ARROW-7017 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Francois Saint-Jacques >Assignee: Wes McKinney >Priority: Major > Labels: analytics > > * Should avoid using builders (and/or NULLs) since the output shape is known > a compute time. > * Should be refatored to support other operations, e.g. Substraction, > Multiplication. > * Should have a overflow, underflow detection mode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7017) [C++] Refactor AddKernel to support other operations and types
[ https://issues.apache.org/jira/browse/ARROW-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969394#comment-16969394 ] Wes McKinney commented on ARROW-7017: - Thanks, yes let's discuss more there. It seems like some investigation is indeed required. I think having LLVM as a build-time dependency is more palatable than as a runtime dependency in some applications. > [C++] Refactor AddKernel to support other operations and types > -- > > Key: ARROW-7017 > URL: https://issues.apache.org/jira/browse/ARROW-7017 > Project: Apache Arrow > Issue Type: Improvement > Components: C++, C++ - Compute >Reporter: Francois Saint-Jacques >Priority: Major > Labels: analytics > > * Should avoid using builders (and/or NULLs) since the output shape is known > a compute time. > * Should be refatored to support other operations, e.g. Substraction, > Multiplication. > * Should have a overflow, underflow detection mode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7017) [C++] Refactor AddKernel to support other operations and types
[ https://issues.apache.org/jira/browse/ARROW-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968993#comment-16968993 ] Micah Kornfield commented on ARROW-7017: I created [https://jira.apache.org/jira/browse/ARROW-7083] to track this instead of having the conversation on this JIRA. > [C++] Refactor AddKernel to support other operations and types > -- > > Key: ARROW-7017 > URL: https://issues.apache.org/jira/browse/ARROW-7017 > Project: Apache Arrow > Issue Type: Improvement > Components: C++, C++ - Compute >Reporter: Francois Saint-Jacques >Priority: Major > Labels: analytics > > * Should avoid using builders (and/or NULLs) since the output shape is known > a compute time. > * Should be refatored to support other operations, e.g. Substraction, > Multiplication. > * Should have a overflow, underflow detection mode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7017) [C++] Refactor AddKernel to support other operations and types
[ https://issues.apache.org/jira/browse/ARROW-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968992#comment-16968992 ] Micah Kornfield commented on ARROW-7017: This seems plausible, but it still sounds like LLVM becomes a build-time dependency? Is that ok for different platforms/toolchain (e.g. windows)? > [C++] Refactor AddKernel to support other operations and types > -- > > Key: ARROW-7017 > URL: https://issues.apache.org/jira/browse/ARROW-7017 > Project: Apache Arrow > Issue Type: Improvement > Components: C++, C++ - Compute >Reporter: Francois Saint-Jacques >Priority: Major > Labels: analytics > > * Should avoid using builders (and/or NULLs) since the output shape is known > a compute time. > * Should be refatored to support other operations, e.g. Substraction, > Multiplication. > * Should have a overflow, underflow detection mode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7017) [C++] Refactor AddKernel to support other operations and types
[ https://issues.apache.org/jira/browse/ARROW-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964416#comment-16964416 ] Wes McKinney commented on ARROW-7017: - We should see what is possible as far as generating a shared library from a bundle of LLVM IR. If Gandiva developed the functionality to * pre-compile IR for all of its built-in functions (on a single-function basis) * generate a shared library from this * have a way to construct a callable kernel given input types using the generated shared library then that seems like a pretty good way forward. I see having multiple implementations of the same thing long term as being undesirable. In fairness I think in several places we may have extra logic in casting that is not present in Gandiva, but it would be worth investing the effort to try to have single implementations of things with suitable options to opt in to optional behaviors. > [C++] Refactor AddKernel to support other operations and types > -- > > Key: ARROW-7017 > URL: https://issues.apache.org/jira/browse/ARROW-7017 > Project: Apache Arrow > Issue Type: Improvement > Components: C++, C++ - Compute >Reporter: Francois Saint-Jacques >Priority: Major > Labels: analytics > > * Should avoid using builders (and/or NULLs) since the output shape is known > a compute time. > * Should be refatored to support other operations, e.g. Substraction, > Multiplication. > * Should have a overflow, underflow detection mode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7017) [C++] Refactor AddKernel to support other operations and types
[ https://issues.apache.org/jira/browse/ARROW-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962715#comment-16962715 ] Micah Kornfield commented on ARROW-7017: I'd in particular like to be careful that if we do introduce a requirement on LLVM is isolated to compute code and doesn't leak into other modules. For instance the Cast kernel is depended on by other modules. > [C++] Refactor AddKernel to support other operations and types > -- > > Key: ARROW-7017 > URL: https://issues.apache.org/jira/browse/ARROW-7017 > Project: Apache Arrow > Issue Type: Improvement > Components: C++, C++ - Compute >Reporter: Francois Saint-Jacques >Priority: Major > Labels: analytics > > * Should avoid using builders (and/or NULLs) since the output shape is known > a compute time. > * Should be refatored to support other operations, e.g. Substraction, > Multiplication. > * Should have a overflow, underflow detection mode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7017) [C++] Refactor AddKernel to support other operations and types
[ https://issues.apache.org/jira/browse/ARROW-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961597#comment-16961597 ] Wes McKinney commented on ARROW-7017: - I think the jury is out (for example, I'm not totally convinced) on having LLVM as a hard requirement for running simple expressions. I'm not sure what will end up being most desirable long term > [C++] Refactor AddKernel to support other operations and types > -- > > Key: ARROW-7017 > URL: https://issues.apache.org/jira/browse/ARROW-7017 > Project: Apache Arrow > Issue Type: Improvement > Components: C++, C++ - Compute >Reporter: Francois Saint-Jacques >Priority: Major > Labels: analytics > > * Should avoid using builders (and/or NULLs) since the output shape is known > a compute time. > * Should be refatored to support other operations, e.g. Substraction, > Multiplication. > * Should have a overflow, underflow detection mode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7017) [C++] Refactor AddKernel to support other operations and types
[ https://issues.apache.org/jira/browse/ARROW-7017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961565#comment-16961565 ] Jacques Nadeau commented on ARROW-7017: --- What's the thinking of building these a second time here as opposed to just adding utility methods over Gandiva for specific patterns? My experience is that it is very rare that people only need to do a single expression. > [C++] Refactor AddKernel to support other operations and types > -- > > Key: ARROW-7017 > URL: https://issues.apache.org/jira/browse/ARROW-7017 > Project: Apache Arrow > Issue Type: Improvement > Components: C++, C++ - Compute >Reporter: Francois Saint-Jacques >Priority: Major > Labels: analytics > > * Should avoid using builders (and/or NULLs) since the output shape is known > a compute time. > * Should be refatored to support other operations, e.g. Substraction, > Multiplication. > * Should have a overflow, underflow detection mode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
