Re: Hive LLAP service is not starting
Hello Rajesh, Here's what we use on Hive 2.3 and Hadoop 2.7.4 export PATH=$PATH:/usr/lib/slider/bin export HADOOP_CONF_DIR=/usr/lib/hadoop/conf export JAVA_HOME=/usr/java/default export HADOOP_HOME=/usr/lib/hadoop /usr/lib/hive/bin/hive --service llap --name llaptest --instances 2 --size 6144m --cache 512m --slider-am-container-mb 2048 --queue default --executors 4 --xmx 5120m --loglevel INFO --auxjars --args " -XX:+AlwaysPreTouch -XX:+UseG1GC -XX:TLABSize=8m -XX:+ResizeTLAB -XX:+UseNUMA -XX:+AggressiveOpts -XX:InitiatingHeapOccupancyPercent=40 -XX:G1ReservePercent=20 -XX:MaxGCPauseMillis=200 -XX:MetaspaceSize=512m" --slider-appconfig-global site.global.app_user=hadoop this will create a run.sh file in the folder that you are running the command from. The run the run.sh, which will submit to YARN. ./llap-slider-24Aug2017/run.sh you can check the status either thru the YARN UI and/or slider command Regards, Dano On Tue, Sep 12, 2017 at 3:15 AM Rajesh Narayanan < rajesh.r.naraya...@oracle.com> wrote: > Hi Premal > > Thanks for the reply. I am following below document. Can you please > provide any document or steps to start LLAP through slide > > > https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_command-line-installation/content/intro_new_book.html > > Regards > > Rajesh > > *From:* Premal Shah [mailto:premal.j.s...@gmail.com] > *Sent:* Monday, September 11, 2017 11:29 PM > *To:* user@hive.apache.org > *Subject:* Re: Hive LLAP service is not starting > > > > Hi Rajesh, > > I thought the recommended way to start/stop LLAP was to use Slider > > this is what we do to start > > > > slider create llap --resources $BASEDIR/resources.json --template > $BASEDIR/appConfig.json > > > > and to stop before you can start again > > > > slider stop llap > > slider destroy llap --force || slider destroy llap > > > > > > On Mon, Sep 11, 2017 at 5:36 AM, Rajesh Narayanan < > rajesh.r.naraya...@oracle.com> wrote: > > Hi All, > > I am using Hadoop 2.7.3 and Hive 2.2.0 and TEZ 0.9.0 in linux. and able to > execute hive query in yarn as TEZ type. > > When I start hive LLAP service -./hive --service llap --name @llap > --instances 1 --cache 1024m --xmx 2048m --size 3225m --loglevel DEBUG > --args " -XX:+UseG1GC -XX:+ResizeTLAB -XX:+UseNUMA -XX:-ResizePLAB" > > Getting below exception. Could you please suggest what missing here. > > > > Failed: java.io.FileNotFoundException: > /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory) > java.util.concurrent.ExecutionException: java.io.FileNotFoundException: > /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory) > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at > org.apache.hadoop.hive.llap.cli.LlapServiceDriver.run(LlapServiceDriver.java:605) > at > org.apache.hadoop.hive.llap.cli.LlapServiceDriver.main(LlapServiceDriver.java:113) > Caused by: java.io.FileNotFoundException: > /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory) > at java.io.FileInputStream.open0(Native Method) > at java.io.FileInputStream.open(FileInputStream.java:195) > at java.io.FileInputStream.(FileInputStream.java:138) > at > org.apache.hadoop.hive.common.CompressionUtils.unTar(CompressionUtils.java:152) > at > org.apache.hadoop.hive.llap.cli.LlapServiceDriver$1.call(LlapServiceDriver.java:361) > at > org.apache.hadoop.hive.llap.cli.LlapServiceDriver$1.call(LlapServiceDriver.java:348) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > INFO cli.LlapServiceDriver: LLAP service driver finished > > > > Thanks > > Rajesh > > > > > > -- > > Regards, > Premal Shah. >
RE: Hive LLAP service is not starting
Hi Premal Thanks for the reply. I am following below document. Can you please provide any document or steps to start LLAP through slide https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_command-line-installation/content/intro_new_book.html Regards Rajesh From: Premal Shah [mailto:premal.j.s...@gmail.com] Sent: Monday, September 11, 2017 11:29 PM To: user@hive.apache.org Subject: Re: Hive LLAP service is not starting Hi Rajesh, I thought the recommended way to start/stop LLAP was to use Slider this is what we do to start slider create llap --resources $BASEDIR/resources.json --template $BASEDIR/appConfig.json and to stop before you can start again slider stop llap slider destroy llap --force || slider destroy llap On Mon, Sep 11, 2017 at 5:36 AM, Rajesh Narayanan mailto:rajesh.r.naraya...@oracle.com"; \nrajesh.r.naraya...@oracle.com> wrote: Hi All, I am using Hadoop 2.7.3 and Hive 2.2.0 and TEZ 0.9.0 in linux. and able to execute hive query in yarn as TEZ type. When I start hive LLAP service -./hive --service llap --name @llap --instances 1 --cache 1024m --xmx 2048m --size 3225m --loglevel DEBUG --args " -XX:+UseG1GC -XX:+ResizeTLAB -XX:+UseNUMA -XX:-ResizePLAB" Getting below exception. Could you please suggest what missing here. Failed: java.io.FileNotFoundException: /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory) java.util.concurrent.ExecutionException: java.io.FileNotFoundException: /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory) at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.hadoop.hive.llap.cli.LlapServiceDriver.run(LlapServiceDriver.java:605) at org.apache.hadoop.hive.llap.cli.LlapServiceDriver.main(LlapServiceDriver.java:113) Caused by: java.io.FileNotFoundException: /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory) at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open(FileInputStream.java:195) at java.io.FileInputStream.(FileInputStream.java:138) at org.apache.hadoop.hive.common.CompressionUtils.unTar(CompressionUtils.java:152) at org.apache.hadoop.hive.llap.cli.LlapServiceDriver$1.call(LlapServiceDriver.java:361) at org.apache.hadoop.hive.llap.cli.LlapServiceDriver$1.call(LlapServiceDriver.java:348) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) INFO cli.LlapServiceDriver: LLAP service driver finished Thanks Rajesh -- Regards, Premal Shah.
Re: Hive LLAP service is not starting
> java.util.concurrent.ExecutionException: java.io.FileNotFoundException: > /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory) LLAP expects to find a tarball where tez.lib.uris is - looks like you've got a directory? Cheers, Gopal
Re: Hive LLAP service is not starting
Hi Rajesh, I thought the recommended way to start/stop LLAP was to use Slider this is what we do to start slider create llap --resources $BASEDIR/resources.json --template $BASEDIR/appConfig.json and to stop before you can start again slider stop llap slider destroy llap --force || slider destroy llap On Mon, Sep 11, 2017 at 5:36 AM, Rajesh Narayanan < rajesh.r.naraya...@oracle.com> wrote: > Hi All, > > I am using Hadoop 2.7.3 and Hive 2.2.0 and TEZ 0.9.0 in linux. and able to > execute hive query in yarn as TEZ type. > > When I start hive LLAP service -./hive --service llap --name @llap > --instances 1 --cache 1024m --xmx 2048m --size 3225m --loglevel DEBUG > --args " -XX:+UseG1GC -XX:+ResizeTLAB -XX:+UseNUMA -XX:-ResizePLAB" > > Getting below exception. Could you please suggest what missing here. > > > > Failed: java.io.FileNotFoundException: > /tmp/staging-slider-HHIwk3/lib/tez.tar.gz > (Is a directory) > java.util.concurrent.ExecutionException: java.io.FileNotFoundException: > /tmp/staging-slider-HHIwk3/lib/tez.tar.gz (Is a directory) > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at org.apache.hadoop.hive.llap.cli.LlapServiceDriver.run( > LlapServiceDriver.java:605) > at org.apache.hadoop.hive.llap.cli.LlapServiceDriver.main( > LlapServiceDriver.java:113) > Caused by: java.io.FileNotFoundException: > /tmp/staging-slider-HHIwk3/lib/tez.tar.gz > (Is a directory) > at java.io.FileInputStream.open0(Native Method) > at java.io.FileInputStream.open(FileInputStream.java:195) > at java.io.FileInputStream.(FileInputStream.java:138) > at org.apache.hadoop.hive.common.CompressionUtils.unTar( > CompressionUtils.java:152) > at org.apache.hadoop.hive.llap.cli.LlapServiceDriver$1.call( > LlapServiceDriver.java:361) > at org.apache.hadoop.hive.llap.cli.LlapServiceDriver$1.call( > LlapServiceDriver.java:348) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > INFO cli.LlapServiceDriver: LLAP service driver finished > > > > Thanks > > Rajesh > -- Regards, Premal Shah.