On 8/24/14, 6:56 PM, Jianfeng (Jeff) Zhang wrote:
not sure whether this could help you ( it's for HDP2, maybe you can tweak
it for your environment )
https://github.com/t3rmin4t0r/tez-autobuild
That is missing a single line change to work at the moment.
You need to edit that and say
HIVE_VERSION=0.14.0-SNAPSHOT
-HIVE_BRANCH=trunk
+HIVE_BRANCH=tez
For it to work.
All changes from hive-tez-0.5.patch (and a few more features) are in
that branch at the moment.
That script was written to work on Apache Hadoop (just edit the
HADOOP_VERSION in that) - it should work on HDP 2.1, CDH5 or any other
distro which ships YARN & 2.2+.
Cheers,
Gopal
On Mon, Aug 25, 2014 at 9:29 AM, Grandl Robert <[email protected]> wrote:
Hi Jeff,
Actually I am a bit confused.
Before I was running 0.5.0 incubating with hive 0.13 and worked fine.
Now, when trying to rebuilt and make them to work is not possible anymore.
I was trying apache tez official branch.
Now I switched back to incubating branch. Also I observed that hive 0.13
has tez.version = 0.4.1-incubating as depedency. If I try to change it to
0.5.0-incubating it throws a lot of missing things when compiling it.
I managed to run tez 0.4.1 + hive-0.13, but tez 0.4.1 does not have some
classes I need so I really need 0.5(incubating or not).
Can you point me to a version of tez(0.5.+) and hive 0.13 and github
branches, to compile from source and work together ? I spent 2 days trying
to do this setup but with no success yet.
Thanks,
Robert
On Sunday, August 24, 2014 6:10 PM, Jianfeng (Jeff) Zhang <
[email protected]> wrote:
Hi Robert,
Tez 0.5 is not released yet. I check MRHelpers which do not have this
method now but have it before some commit. I guess you may have 2 versions
of tez jar in your classpath, please check again.
Best Regards,
Jeff Zhang
On Mon, Aug 25, 2014 at 7:22 AM, Grandl Robert <[email protected]> wrote:
Hi guys,
I came back to run hive atop tez, but unfortunately I am again into issues
running it.
I have tez-0.5.0 and hive-0.13.1 compiled from sources.
I am also running hadoop-2.4.
However, running tez jobs works well. Running hive with mapreduce works
well.
But trying to run hive atop tez, it gives me this error:
hadoop@nectar-11:~/rgrandl$ hive
Logging initialized using configuration in
jar:file:/home/hadoop/rgrandl/dag_project/hive-0.13.1/lib/hive-common-0.13.1.jar!/hive-log4j.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/home/hadoop/rgrandl/dag_project/tez-0.5.0/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/hadoop/hadoop-2.4.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.tez.mapreduce.hadoop.MRHelpers.updateEnvironmentForMRAM(Lorg/apache/hadoop/conf/Configuration;Ljava/util/Map;)V
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:182)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:123)
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:355)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
It looks like some incompatibility, missing classpath ?
I have both hive_home and tez jars into hadoop classpath.
Can someone gives me some directions of what I am doing wrong ?
Thanks,
Robert
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity
to which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
