RE: Parquet to arrow java converter

[Wenbo Zhao]

Thanks Julien and Wes. There is an ongoing PR 
https://github.com/apache/parquet-mr/pull/443 (update Arrow version to 0.8.0) 
which I may be depending on. Should I wait for this? 

Wenbo 

-Original Message-
From: Julien Le Dem [mailto:julien.le...@gmail.com] 
Sent: Tuesday, March 6, 2018 5:27 PM
To: dev@arrow.apache.org
Subject: Re: Parquet to arrow java converter

I would put in the parquet-mr codebase. I have contributed the schéma 
conversion code there. I’m happy to provide feedback on PRs in this area. 

Julien

> On Mar 6, 2018, at 12:18, Wes McKinney <wesmck...@gmail.com> wrote:
> 
> When it had been discussed in the past, the thinking had been to 
> implement it in the Parquet Java codebase. I'd be interested in 
> others' opinions about this (since I'm not an expert on Java matters)
> 
> - Wes
> 
>> On Tue, Mar 6, 2018 at 2:27 PM, Wenbo Zhao <wenbo.z...@twosigma.com> wrote:
>> Hi,
>> 
>> Sorry that if someone may have asked the same question before. We are 
>> interested in providing a java convertor from Parquet to Arrow. Should I 
>> implement this converter in Parquet-mr/Parquet-arrow or under the Arrow 
>> project? I have the feeling that putting the implementation in 
>> Parquet-mr/Parquet-arrow would be preferable 
>> https://www.mail-archive.com/dev@arrow.apache.org/msg02606.html?
>> 
>> Thanks,
>> 
>> Wenbo

Re: Parquet to arrow java converter

[Julien Le Dem]

I would put in the parquet-mr codebase. I have contributed the schéma 
conversion code there. I’m happy to provide feedback on PRs in this area. 

Julien

> On Mar 6, 2018, at 12:18, Wes McKinney  wrote:
> 
> When it had been discussed in the past, the thinking had been to
> implement it in the Parquet Java codebase. I'd be interested in
> others' opinions about this (since I'm not an expert on Java matters)
> 
> - Wes
> 
>> On Tue, Mar 6, 2018 at 2:27 PM, Wenbo Zhao  wrote:
>> Hi,
>> 
>> Sorry that if someone may have asked the same question before. We are 
>> interested in providing a java convertor from Parquet to Arrow. Should I 
>> implement this converter in Parquet-mr/Parquet-arrow or under the Arrow 
>> project? I have the feeling that putting the implementation in 
>> Parquet-mr/Parquet-arrow would be preferable 
>> https://www.mail-archive.com/dev@arrow.apache.org/msg02606.html?
>> 
>> Thanks,
>> 
>> Wenbo

Re: Parquet to arrow java converter

[Wes McKinney]

When it had been discussed in the past, the thinking had been to
implement it in the Parquet Java codebase. I'd be interested in
others' opinions about this (since I'm not an expert on Java matters)

- Wes

On Tue, Mar 6, 2018 at 2:27 PM, Wenbo Zhao  wrote:
> Hi,
>
> Sorry that if someone may have asked the same question before. We are 
> interested in providing a java convertor from Parquet to Arrow. Should I 
> implement this converter in Parquet-mr/Parquet-arrow or under the Arrow 
> project? I have the feeling that putting the implementation in 
> Parquet-mr/Parquet-arrow would be preferable 
> https://www.mail-archive.com/dev@arrow.apache.org/msg02606.html?
>
> Thanks,
>
> Wenbo

Re: Parquet to arrow java converter

[Li Jin]

This definitely sounds like a useful tool. It seems like Julien started
some of work in Parquet-arrow a while back.

Julien, I am wondering what's your thoughts on whether such code should
live in parquet-mr or arrow codebase?

On Tue, Mar 6, 2018 at 2:27 PM, Wenbo Zhao  wrote:

> Hi,
>
> Sorry that if someone may have asked the same question before. We are
> interested in providing a java convertor from Parquet to Arrow. Should I
> implement this converter in Parquet-mr/Parquet-arrow or under the Arrow
> project? I have the feeling that putting the implementation in
> Parquet-mr/Parquet-arrow would be preferable https://www.mail-archive.com/
> dev@arrow.apache.org/msg02606.html?
>
> Thanks,
>
> Wenbo
>