jnturton commented on issue #2421: URL: https://github.com/apache/drill/issues/2421#issuecomment-1004748679
Ted Dunning wrote: As a little bit of perspective from somebody who *was* at MapR at the time, here are my recollections. Arrow is pretty much the value vectors from Drill with some lessons learned and all dependencies removed so that Arrow can be consumed separately from Drill. The spinout of the Dremio team didn't happen because of the lack of integration with Arrow ... it was more the other way around ... because a significant chunk of the Drill team left to form Dremio, the driving force that could have pushed for integration just wasn't around any more because they were off doing Dremio and weren't working on Drill any more very much. The motive for the spinout had mostly to do with the fact that Tomer and Jacques recognized the opportunity to build a largely in-memory analytical engine based on zero serialization techniques and also recognized that this could never be a priority for MapR because it was outside the center of mass there. Once the Dremio team was out, though, they had a huge need for interoperability with systems like Spark and Cassandra, and they needed to not impose all of Drill as a dependency if they wanted these other systems to take on Arrow. This history doesn't really impact the merits or methods of integrating present-day Drill with Arrow, but it is nice to get the story the right way around. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
