In the last couple of days I was digging arrow and iron-arrow
in order to create a rust impl for arrow.
My background is mostly p[c]ythonic, so I'd gladly team up if You are
On Mar 7 2018, at 4:25 pm, Andy Grove <andygrov...@gmail.com> wrote:
> I just wanted to introduce myself to the group before I start asking lots
> of questions. I'm a software engineer mostly working with
> Scala/Spark/Kudu/Parquet in my day job and in my spare time I have been
> working on a POC of a distributed data platform implemented in Rust. The
> project is called DataFusion (https://www.datafusion.rs/).
> The project is very early and the implementation is currently very simple
> row-based processing but the performance is already quite exciting to me
> (current test case is 4x faster than Apache Spark).
> I have decided that I should now concentrate on making Apache Arrow the
> native memory format so that I can implement more efficient data processing
> and make it easier in the future to be able to integrate with things like
> Kudu and Parquet. It's also just a great way for me to learn about
> I'm just in the process of getting Arrow compiling and reading the docs.
> I'll be back soon with questions I'm sure.