I think someone should reach out to Matei and Shoumik, and see if they would like to collaborate. Wes, would you like to do that?
Also, reach out to the Spark community. Are they aware of Arrow? Are they planning to use it, or are they developing an alternative? Julian > On Dec 13, 2016, at 10:23 AM, Wes McKinney <wesmck...@gmail.com> wrote: > > Looks like there will be a talk about Weld at Strata next Spring: > > http://conferences.oreilly.com/strata/strata-ca/public/schedule/detail/57646 > > It would be nice to see if there's any work that we can be doing in > Arrow to try to collaborate on such closely-related efforts. > > - Wes > > On Sat, Nov 26, 2016 at 3:53 PM, Wes McKinney <wesmck...@gmail.com> wrote: >> I've always contended that building a sort of "runtime" for in-memory >> column expressions was a natural next step after hardening Arrow as a >> data structure and memory exchange mechanism. I'm hopeful that we'll >> see some cross-pollination between projects like Weld and Arrow to >> produce open source solutions that help drive more collaboration and >> consolidation around shared low-level needs for building analytical >> systems. We still have significant work to do to solidify Arrow as a >> standard -- anyone all reading can do to help work against >> fragmentation in data structures / memory representations would be >> very much appreciated! >> >> - Wes >> >> On Wed, Nov 23, 2016 at 11:35 AM, Donald Foss <donald.f...@gmail.com> wrote: >>> I had that in my queue, and your watching and thoughts about it mean I need >>> to watch it today ;) >>> >>> Thanks for the link to where you've played with abstraction before. I'm >>> going to fork a copy and do some playing myself. I'm very interested in how >>> universal it can be, and a library to go from expression to universal >>> intermediate, them output the code in the language of one's choice. The >>> optimizations could be performed in the intermediate layer, the output >>> later, or both if there are specific code optimizations available after >>> translation. >>> >>> -Donald >>> >>> On Nov 22, 2016 2:31 PM, Julien Le Dem >>> wrote:
