Re: [DISCUSS] Contribution of Gandiva to Apache Arrow

2018-08-06 Thread Wes McKinney
hi Ravindra, Thanks for the update. I will start a vote to approve the donation of the codebase soon if there are no objections. The merge will be subject to completing the standard ASF IP Clearance and having a Software Grant filed by Dremio. I noticed some things in the Gandiva codebase vs. the

Re: [DISCUSS] Contribution of Gandiva to Apache Arrow

2018-08-05 Thread Pindikura Ravindra
Sorry for the delay, Wes. We plan to finish off some feature development work next week (eg. Filter support), and then, start work on merging gandiva into arrow (third week of august). Since we have been using only the public arrow APIs in the code and in tests, we do not foresee any major hiccups

Re: [DISCUSS] Contribution of Gandiva to Apache Arrow

2018-08-04 Thread Wes McKinney
It's been about 2 weeks since the discussion started. What are the next steps? It would be great to hear from one or more of the Gandiva developers directly here. best Wes On Wed, Jul 25, 2018 at 5:11 PM, Li Jin wrote: > Although I am not familiar with LLVM details but I think at the high level

Re: [DISCUSS] Contribution of Gandiva to Apache Arrow

2018-07-25 Thread Li Jin
Although I am not familiar with LLVM details but I think at the high level such component fits in Arrow project. Hopefully I can learn more and help as well. +1 On Wed, Jul 25, 2018 at 8:45 PM, Philipp Moritz wrote: > +1 on merging it and also agreed with Uwe that we will need to deal with > LL

Re: [DISCUSS] Contribution of Gandiva to Apache Arrow

2018-07-25 Thread Philipp Moritz
+1 on merging it and also agreed with Uwe that we will need to deal with LLVM version conflicts. In addition it would be good to come up with a plan on how it can be useful for other DataFrame open source projects. Having end-to-end applications that let people profit from this code will help adopt

Re: [DISCUSS] Contribution of Gandiva to Apache Arrow

2018-07-25 Thread Uwe L. Korn
Having it in Arrow also will enable us to better promote it on the Python side with the pyarrow package. It will be a great addition but we will need to figure out then the problem of dealing with in-memory LLVM versions so that we don't conflict with packages like Numba. This is a problem on th

Re: [DISCUSS] Contribution of Gandiva to Apache Arrow

2018-07-24 Thread Phillip Cloud
While I'm not a level 83 LLVM wizard like Antoine :) I have a small amount of experience with it and would also be happy to review/merge patches. Having Gandiva in arrow will simplify packaging and building the library, which IME has always been annoying with large cross-platform and cross-languag

Re: [DISCUSS] Contribution of Gandiva to Apache Arrow

2018-07-24 Thread Uwe L. Korn
Small update: > * CentOS 6 and later (we use the RedHat devtoolset-2 compiler toolchain) We still build the Python wheels on CentOS 5. This should be no problem as llvmlite can also be build on that OS. It looks like there will be a new Linux Python wheel standard soon where we will upgrade to

Re: [DISCUSS] Contribution of Gandiva to Apache Arrow

2018-07-24 Thread Antoine Pitrou
Le 23/07/2018 à 21:22, Wes McKinney a écrit : >> With regards to (2), there are two main questions. >> a) How do we make sure the Gandiva team (who are not Arrow committers) get >> ample support from other committers to make good continued progress. Since >> they will no longer be able to commit

Re: [DISCUSS] Contribution of Gandiva to Apache Arrow

2018-07-23 Thread Wes McKinney
hi Jacques, I can say that I for one am excited to see this work happen in Apache Arrow, and I would like to contribute to the initiative. Some responses to queries inline: On Mon, Jul 23, 2018 at 2:50 PM, Jacques Nadeau wrote: > Hey All, > > As I mentioned on some other threads, since the start

[DISCUSS] Contribution of Gandiva to Apache Arrow

2018-07-23 Thread Jacques Nadeau
Hey All, As I mentioned on some other threads, since the start, the goal of the Dremio team was to contribute the Gandiva back into the Arrow project once we made some progress in turning it into something real. I think we're getting close to this point and wanted to have a bit of a discussion abo