Re: Python Parquet package

2016-09-21 Thread Wes McKinney
Sure, I'm happy to do that. Do you want me to take care of refactoring to account for the arrow::io API changes I just made? Then we can go ahead and remove arrow/parquet from the Arrow project. On Wed, Sep 21, 2016 at 3:47 PM, Uwe Korn wrote: > Sounds reasonable for me. I will then to continue t

Re: Python Parquet package

2016-09-21 Thread Uwe Korn
Sounds reasonable for me. I will then to continue to implement the missing interfaces for Parquet in pyarrow.parquet. @wesm Can you take care that we easily depend on a pinned version of parquet-cpp in pyarrow’s travis builds? Uwe > Am 21.09.2016 um 20:07 schrieb Wes McKinney : > > I don't a

Re: Python Parquet package

2016-09-21 Thread Wes McKinney
I don't agree with this approach right now. Here are my reasons: 1. The Parquet Python integration will need to depend both on PyArrow and the Arrow C++ libraries, so these libraries would generally need to be developed together 2. PyArrow would need to define and maintain a C++ or Cython API so

Python Parquet package

2016-09-21 Thread Uwe Korn
Hello, as we have moved the Arrow<->Parquet C++ integration into parquet-cpp, we still have to decide on how we are going to proceed with the Arrow<->Parquet Python integration. For the moment, it seems that the best way to go ahead is to pull the pyarrow.parquet module out into a separate Py