Re: tez and beeline and hs2

Awesome. Thank you Gopal. I re-copied yarn-site.xml and tez-site.xml to all nodes in the cluster and i restarted yarn - but dang it all i forgot to restart HS2. Once i did that this morning after reviewing my steps we now have lift-off! Naturally it was fumbling on my part and the hadoop

Re: tez and beeline and hs2

ld be useful to > produce a wiki page about this. > > > > Thanks > > Bikas > > > > *From:* Stephen Sprague [mailto:sprag...@gmail.com] > *Sent:* Monday, February 22, 2016 6:59 AM > *To:* user@tez.apache.org > *Subject:* Re: tez and beeline and hs2 > > > > just an u

Re: tez and beeline and hs2

t it is shutting down means that > there is something else wrong ( which should have happened before the > shutdown sequence kicked in ). Feel free to file a bug with the logs > attached if you cannot attach them over the mailing list here. > > thanks > — Hitesh > > &g

Re: jansi dependendency?

ng the jar file containing the older > version of Ansi. Afterwards you have to organize the class path in a way > that the more up-to-date version is found first. > > I hope this helps. > Cheers > Jan > > > Am 15.02.2016 um 20:27 schrieb Stephen Sprague: > > hey guys,

tez and beeline

Hey guys, sorry to bother you again but i was trying to get tez working with beeline now (it does work great with the local hive client i might add) so i'm sure i'm missing something simple. if i use the 'local' beeline client as such 'beeline -u jdbc:hive2://' tez works fine but that comes

Re: tez and beeline

ehalf Of > Gopal > Vijayaraghavan > Sent: Wednesday, February 17, 2016 1:13 PM > To: user@tez.apache.org > Cc: Stephen Sprague <sprag...@gmail.com> > Subject: Re: tez and beeline > > >set mapreduce.framework.name > > That can cause queries to fail enti

tez and beeline and hs2

Hey guys, I back on the tez user group just because i think my problem is more focused on Tez -- but lemme know if you think otherwise. Thanks to Gopal again for getting me to think about the hadoop classpath when starting HS2. I again changed classpath in the same way i had to change it for the

Re: Wrong column is picked in HIVE 2.0.0 + TEZ 0.8.2 left join

very interesting. so this did work correctly on your previous distribution of these two products? May i ask what they were? On Mon, Feb 29, 2016 at 8:24 PM, GAO Chi wrote: > Hi all, > > > > We encountered a strange behavior after upgrading to HIVE 2.0.0 + TEZ > 0.8.2. >

Re: hive 2.0.0 + tez

-2.0.0.jar org.apache.hive.service.server.HiveServer2 so my first question is where is the "TezTask PermGen space" defined? Thanks, Stephen. On Wed, Mar 9, 2016 at 7:48 AM, Stephen Sprague <sprag...@gmail.com> wrote: > ok, thanks. very strange then that i can run hive 1.2.1 and hive 2.0.

hive 2.0.0 + tez

hey guys, ​obvious question i suppose but i gotta ask. :) the tez jars i built for hive 1.2.1 will not work with hive 2.0.0 i presume? the reason ask is: ​ i can switch back forth between hive clients 1.2.1 and 2.0.0 and run m/r jobs no problem. however i can not do this with tez. I'm

Re: hive 2.0.0 + tez

, Mar 9, 2016 at 2:51 AM, Andre Kelpe <ake...@concurrentinc.com> wrote: > tez has no dependency on hive, it is the other way around. I don't > think, that's the problem you are facing. > > - André > > On Wed, Mar 9, 2016 at 6:20 AM, Stephen Sprague <sprag...@gmail

Re: hive 2.0.0 + tez

thanks as always Gopal. lemme use jdk8 for starters and eliminate that as a possibility. On Wed, Mar 9, 2016 at 10:30 AM, Gopal Vijayaraghavan wrote: > > > ok, thanks. very strange then that i can run hive 1.2.1 and hive 2.0.0 > >m/r jobs against the same hadoop cluster w/o

Re: Wrong column is picked in HIVE 2.0.0 + TEZ 0.8.2 left join

r we > upgraded to new version at 24th. > > > > > > Previously we’re using HIVE 1.2.0 + TEZ 0.7.0 > > > > > > Thanks! > > > > Chi > > > > *From:* Stephen Sprague [mailto:sprag...@gmail.com] > *Sent:* Tuesday, March 1, 2016 12:31 PM

pretty quiet over here!

are things really that nice and smooth? no news == paranoia? lol!

Re: config issues

Hi Gopal, hmmm. i thought it would just be a passthrough type operation but given everything else i think i'll follow that lead. Thanks for catching that. here's how that line starts out above which i chopped off: 2016-09-17 15:56:37,510 INFO com.pepperdata.supervisor.launcher.impl.j: Command

config issues

here's the info i have where i'm obviously missing something. a) i've read and re-read the tez install instructions. still no dice. b) we're running yarn. c) we're also running pepperdata. d) trying to get tez 0.8.4 working. e) i think the client side is pretty easy. get the tez jars in the

Re: config issues

pushed to hdfs. so user error as usual! :) Cheers, Stephen On Sat, Sep 17, 2016 at 7:25 PM, Stephen Sprague <sprag...@gmail.com> wrote: > Hi Gopal, > hmmm. i thought it would just be a passthrough type operation but given > everything else i think i'll follow that lead. Than

Re: Tez UI

-- ./yarn-daemon.sh --config /etc/hadoop/conf start timelineserver hard to believe that jackson version from 1.8.8 to 1.9.13 made that difference. great call Hitesh. many thanks. I've been wrestling with this for quite some time. Cheers, Stephen. On Tue, Oct 18, 2016 at 8:09 PM, Stephen Sprague

Re: Tez UI

hadoop/lib/jackson-mapper-asl-1.8.8.jar so jersey jars look to be in sync but the jackson ones look a step behind. what do you think? should i force the 1.9's into the ATS CLASSPATH? can't hurt would be my guess. lemme try. Cheers, Stephen. On Mon, Oct 17, 2016 at 2:44 PM, Stephen Spra

Re: Tez UI

;- check the hadoop classpath for jackson/jersey jars and compare the > versions in it to the versions in the tez tarball. > > thanks > — Hitesh > > > On Oct 16, 2016, at 9:24 PM, Stephen Sprague <sprag...@gmail.com> wrote: > > > > thanks Allan. so i enabled DEBUG,c

Re: Tez UI

is supposed to display a list of DAGs. Are you able to view > them? > > Thanks, > Sreenath > > From: Stephen Sprague <sprag...@gmail.com> > Reply-To: "user@tez.apache.org" <user@tez.apache.org> > Date: Sunday, October 16, 2016 at 7:16 AM > To: "use

Tez UI

hey guys, i'm having hard time getting the Tez UI to work. I'm sure i'm doing something wrong but i can't seem to figure out. Here's my scenario. 1. i'm using nginx as the webserver. port 9766. using that port without params correctly displays index.html. (i followed the instructions on

Re: Tez UI

:46 PM, Allan Wilson <wilsoncr...@gmail.com> wrote: > Just saw Gopals response...that def needs updating too. > > Sent from my iPhone > > On Oct 15, 2016, at 9:31 PM, Stephen Sprague <sprag...@gmail.com> wrote: > > thanks guys. lemme answer. > > Sreenath- &

Re: Tez UI

gs to be > more specific: syslog_dag* files) to see if there are any > warnings/exceptions being logged related to history event handling. > > thanks > — Hitesh > > > On Oct 15, 2016, at 9:58 PM, Stephen Sprague <sprag...@gmail.com> wrote: > > > > hmm

Re: Tez UI

thing like > "DEBUG;org.apache.hadoop.ipc=INFO;org.apache.hadoop.security=INFO;” > where everything except the ipc and security package classes log at DEBUG. > You can use the same format to set the log level to > “INFO;timeline.client.package=DEBUG;” > > — Hitesh > >

Re: Tez GC issues perhaps? not sure.

aha. i sense we're getting closer. here are my settings for yarn.timeline-service.* yarn.timeline-service.address=${yarn.timeline-service.hostname}:10200 yarn.timeline-service.client.max-retries=30 yarn.timeline-service.client.retry-interval-ms=1000 yarn.timeline-service.enabled=true

Re: Tez GC issues perhaps? not sure.

n Tue, Dec 13, 2016 at 6:24 PM, Stephen Sprague <sprag...@gmail.com> wrote: > interesting thank you. pretty sure they are being submitted through > the HS2 service. > > On Tue, Dec 13, 2016 at 5:21 PM, Harish JP <h...@hortonworks.com> wrote: > >> Hi St

Re: Tez GC issues perhaps? not sure.

Thanks Gopal. I'll set the ttl flag to false and see what gives. Cheers, Stephen On Tue, Dec 13, 2016 at 10:48 PM, Gopal Vijayaraghavan wrote: > > yarn.timeline-service.ttl-enable=true > > Let us validate that this is due to the TTL GC kicking in and disable the > TTL flag

Re: Tez GC issues perhaps? not sure.

ted 52952K, reserved 1095680K class spaceused 5696K, capacity 5868K, committed 5888K, reserved 1048576K So definitely looks GC-ish related, yeah? okay, onward looking for that RollingLevelDb class next... Cheers, STephen. On Wed, Dec 14, 2016 at 10:03 AM, Stephen Sprague <sprag...@

Re: Tez GC issues perhaps? not sure.

ah. 2016-12-14 14:05:07,855 [WARN] [AMShutdownThread] |ats.ATSHistoryLoggingService|: ATSService being stopped, eventQueueBacklog=14820, maxTimeLeftToFlush=-1, waitForever=true 2016-12-14 14:05:37,877 [ERROR] [AMShutdownThread] |impl.TimelineClientImpl|: Failed to get the response from the

tez + union stmt

all, i'm running tez with the sql pattern: * create table foo as select * from (select... UNION select... UNION select...) in the logs the final step is this: * Moving data to directory hdfs:// dwrnn1.sv2.trulia.com:8020/user/hive/warehouse/staging.db/tmp_pv_v4c__loc_4 from hdfs://

Re: tez "sessions"

Tez session exits just fine. On the other hand we have some complex Tez queries and its those that stick around. kinda strange why that is. i guess i bounce the time-line server and see what happens next. if you see other insights from that stack trace please let me know. Cheers! Stephen. On Tue, Mar

tez "sessions"

hi guys, this seems to be a familiar subject but i still don't have a handle on it, alas. I'm totally misunderstanding something here. in our case we use tez and submit many different jobs to our queue called "batch_sql". this is works great until the Tez job finishes (100% complete) and instead

Re: tez "sessions"

Thanks Gopal. lemme see what i can do with your insights and report back with my findings. Cheers, Stephen. On Tue, Mar 14, 2017 at 6:19 AM, Gopal Vijayaraghavan wrote: > > Looking at the doc i thought this config setting would influence those > Tez jobs from hanging