Hi Gopal, please see my correspondence about Tez in tez user group. I forwarded to hive user group.
thanks Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* http://talebzadehmich.wordpress.com On 30 May 2016 at 12:30, Gopal Vijayaraghavan <gop...@apache.org> wrote: > > I do not use any vendor's product., All my own set up, build and > >configure. > > My autobuild scripts should serve as readable documentation for this, > since nearly everything's in a single Makefile with an install: target. > > Or take the easy route with > > $ make dist install > > In case you use the llap branch, just do "set > hive.llap.execution.mode=none;" to use Tez. > > > java version "1.8.0_77" > > Hadoop 2.6.0 > ... > > https://tez.apache.org/install.html > > Looks good so far. > > > Ok I just need to make it work as I have hive on spark engine as well. > > You're missing 3 things approximately - if you read through the Makefile > in github. > > First, a good tez-site.xml in the classpath (remember, tez.lib.uris needs > to be an HDFS path - for the rest, see the base file from autobuild). > > I usually update Tez to ${fs.default.name}/user/gopal/tez/tez.tar.gz and I > do not use the minimal tarball, but the full dist tarball. > > The fixed tarball means it hits all the good localization characteristics > of YARN, which can add up to minutes on a >250+ node cluster. > > Second, put that in the classpath for Hive (append to > $INSTALL_ROOT/hive/bin/hive-config.sh) > > > export > >HADOOP_CLASSPATH="$INSTALL_ROOT/tez/*:$INSTALL_ROOT/tez/lib/*:$INSTALL_ROO > >T/tez/conf/:$HADOOP_CLASSPATH" > > > export HADOOP_USER_CLASSPATH_FIRST=true > > > Replace $INSTALL_ROOT with wherever Tez is located. > > Third, disable the hive-1.x jars coming from SparkSQL (append/create in > $INSTALL_ROOT/hive/conf/hive-env.sh) > > > export HIVE_SKIP_SPARK_ASSEMBLY=true > > > After that, you can do > > > hive --hiveconf hive.execution.engine=tez > > to get Tez working (add --hiveconf tez.queue.name=<queue> to use queues). > > Cheers, > Gopal > > >
