Re: tez + union stmt

2016-12-25 Thread Elliot West
I believe that tez will generate subfolders for unioned data. As far as I know, this is the expected behaviour and there is no alternative. Presumably this is to prevent multiple tasks from attempting to write the same file? We've experienced issues when switching from mr to tez; downstream jobs

Re: tez + union stmt

2016-12-25 Thread Stephen Sprague
Thanks Elliot. Nice christmas present. Those settings in that stackoverflow link look to me to be exactly what i need to set for MR jobs to pick that data up that Tez created. Cheers, Stephen. On Sun, Dec 25, 2016 at 2:45 AM, Elliot West wrote: > I believe that tez will

Re: hive2.1.0 one partition has two locations

2016-12-25 Thread 徐 鹏
Hi Elliot： I raised a new issue :https://issues.apache.org/jira/browse/HIVE-15512 在 2016年12月22日，下午5:05，Elliot West > 写道： rais

Re: u...@tez.apache.org

2016-12-25 Thread 김동원
Thanks a lot! I’m also aware of tez-autobuild from you. It works well in my cluster but this time I want to install hive/slider/tez by myself without tez-autobuild. Setting up and running LLAP are really tricky and I wrote a separate post in my blog to introduce my experience of setting up

Re: u...@tez.apache.org

2016-12-25 Thread Rajesh Balamohan
Very nice post. Thanks for sharing. btw, tez-autobuild is created and maintained by Gopal. :). I contribute minor patches to it. ~Rajesh.B On Mon, Dec 26, 2016 at 11:17 AM, 김동원 wrote: > Thanks a lot! > > I’m also aware of tez-autobuild from you. > It works well in my

u...@tez.apache.org

2016-12-25 Thread 김동원
I want my IntelliJ to attach to the running LLAP damons by adding the following JVM option (for a debugging purpose): -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000 Where is the right place to append the line? My previous attempt was as follows : - Before executing “hive

Re: u...@tez.apache.org

2016-12-25 Thread Rajesh Balamohan
Much easier option is to make use of https://github.com/t3rmin4t0r/tez-autobuild (edit/set args in slider-gen.sh). ~Rajesh.B On Mon, Dec 26, 2016 at 11:02 AM, Rajesh Balamohan wrote: > Here is an example: > > hive --service llap --instances 1 --args "-XX:+UseG1GC >

Re: u...@tez.apache.org

2016-12-25 Thread Rajesh Balamohan
Here is an example: hive --service llap --instances 1 --args "-XX:+UseG1GC -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000" --cache 48000m --executors 8 --iothreads 8 --size 18m --xmx 128000m --loglevel INFO --javaHome /usr/jdk64/jdk1.8.0_77/ This would generate a "run.sh"