Hello all!
I'm assembling some environment to make it easier to develop hive patches/test things out/etc - today I've bumped into some issues with partitioned tables; and so far I've
came to a conclusion that the root cause is that hadoop 2/3 contains some incompatible changes.
Tez 0.9.x releases seem to be built against hadoop 2.7.x - and there are no
more recent releases available.
Hive 3 and up is built against hadoop-3.x
I'm not sure if this is intentional or not...but executing a hive operator plan using tez - it expects hadoop-3 "like" apis; meanwhile tez brings a whole bunch of
hadoop-2.7.x jars
it goes well up to a certain point; but I bumped into issues when Hive expected that "FileStatus" has a compareTo method with an argument of Filestatus; while in 2.7 the
argument type was Object...
I see that the current master is built against 3.0.3 - so 0.10 and up might be the way to go...but right now I don't think I know "how" to run more complicated queries with
vanilla releases.
I was checking one more thing while I was writing this latter: since the incompatible change in question introduced in hadoop 2.8 by bumping hadoop version to 2.8.4 made it
possible to build tez against that; and it "works" with it (no exception anymore)...so - updating branch-0.9 to that version might be possible without much effort...
cheers,
Zoltan
