Hey Andy! In the last couple of days I was digging arrow and iron-arrow (https://link.getmailspring.com/link/local-f599f61d-1722-v1.1.4-22d9f20d@kszucs-mbp.local/0?redirect=https%3A%2F%2Fgithub.com%2Fjihoonson%2Firon-arrow&recipient=dev%40arrow.apache.org) in order to create a rust impl for arrow. My background is mostly p[c]ythonic, so I'd gladly team up if You are interested.
Krisztian On Mar 7 2018, at 4:25 pm, Andy Grove <andygrov...@gmail.com> wrote: > > Hi, > I just wanted to introduce myself to the group before I start asking lots > of questions. I'm a software engineer mostly working with > Scala/Spark/Kudu/Parquet in my day job and in my spare time I have been > working on a POC of a distributed data platform implemented in Rust. The > project is called DataFusion (https://www.datafusion.rs/). > > The project is very early and the implementation is currently very simple > row-based processing but the performance is already quite exciting to me > (current test case is 4x faster than Apache Spark). > > I have decided that I should now concentrate on making Apache Arrow the > native memory format so that I can implement more efficient data processing > and make it easier in the future to be able to integrate with things like > Kudu and Parquet. It's also just a great way for me to learn about > columnar-processing. > > I'm just in the process of getting Arrow compiling and reading the docs. > I'll be back soon with questions I'm sure. > > Thanks, > Andy.
