the goal is to make sure that the parquet libraries compiled against hadoop 1 will also work with hadoop 2
On Fri, Apr 3, 2015 at 4:12 AM, Cheng Lian <lian.cs....@gmail.com> wrote: > Thanks Julien! This is very helpful. If I understand this correctly, the > hadoop-2 profile is only used together with Travis CI for testing purposes? > > Cheng > > > On 4/3/15 7:14 AM, Julien Le Dem wrote: > >> 1. Parquet works with hadoop 1 and 2, we work around the Hadoop >> incompatibilities. >> See: >> https://github.com/apache/incubator-parquet-mr/blob/ >> master/parquet-hadoop/src/main/java/parquet/hadoop/util/ContextUtil.java >> Both profiles are run in travis-ci: >> https://github.com/apache/incubator-parquet-mr/blob/ >> master/.travis.yml#L26 >> >> 2. You do not need to recompile parquet to the version of hadoop you use. >> For the same reason described above. >> >> >> >> >> On Sun, Mar 29, 2015 at 4:46 AM, Cheng Lian <lian.cs....@gmail.com> >> wrote: >> >> Hi developers, >>> >>> I have two questions about the Hadoop version Parquet uses. In the Maven >>> POM of >>> Parquet, the default Hadoop version is 1.1.0, while with `hadoop-2` >>> profile >>> enabled, the version is 2.3.0. So my questions are: >>> >>> 1. What's the Hadoop version the official parquet-hadoop Maven artifact >>> (take >>> 1.6.0rc7 as an example) depends on? >>> >>> Namely, when compiled and published to Maven repo, is `hadoop-2` >>> profile >>> enabled? (I must confess that I'm not quite familiar with Maven >>> release >>> management process, apologize in advance if this is a dumb question.) >>> >>> 2. When building an application which depends on a specific Hadoop >>> version, do I >>> have to compile my own `parquet-hadoop` library if the Hadoop >>> version is >>> inconsistent with the `parquet-hadoop` Maven artifact? >>> >>> Thanks! >>> Cheng >>> >>> >
