I did not intentionally do anything strange with the CDH set-up, just install CDH5.4.
A little googling tells me that the Hadoop-0.20-mapred package is MR1. It is optional, but valid to have both MR1 and MR2 installed on a system at the same time. So I'm not sure if this is a mismatch between hive and trafodion classpaths. --Steve > -----Original Message----- > From: Dave Birdsall [mailto:[email protected]] > Sent: Thursday, April 7, 2016 8:32 AM > To: [email protected] > Subject: RE: Change in sqenvcom.sh - Response required > > Hi Selva, > > I'm no expert in this, but your short-term change to sqenvcom.sh sounds > good. > > I wonder if Steve could comment on the Jenkins angle? > > Dave > > -----Original Message----- > From: Selva Govindarajan [mailto:[email protected]] > Sent: Wednesday, April 6, 2016 3:56 PM > To: [email protected] > Subject: Change in sqenvcom.sh - Response required > > Change in sqenvcom.sh - Response required > > Changed the subject line to avoid filtering…. > > _____________________________________________ > *From:* Selva Govindarajan [mailto:[email protected] > <[email protected]>] > *Sent:* Wednesday, April 6, 2016 3:25 PM > *To:* '[email protected]' < > [email protected]> > *Subject:* RE: Trafodion master Daily Test Result - 167 - Still Failing - > Change in sqenvcom.sh > > It looks like color didn’t show up in the message correctly. Line 351 was > added. > > _____________________________________________ > *From:* Selva Govindarajan [*mailto:[email protected]* > <[email protected]>] > *Sent:* Wednesday, April 6, 2016 3:21 PM > *To:* '[email protected]' < > *[email protected]* > <[email protected]>> > *Subject:* RE: Trafodion master Daily Test Result - 167 - Still Failing - > Change in sqenvcom.sh > > I added the line in red to sqenvcom.sh and removed ms.env and regenerated > ms.env. > > 347 # directories with jar files and list of jar files > > 348 # (could try to reduce the number of jars in the classpath) > > 349 export HADOOP_JAR_DIRS="/usr/lib/hadoop > > 350 /usr/lib/hadoop/lib > > 351 /usr/lib/hadoop-0.20-mapreduce" > > > This makes hive to work from Trafodion. But, I am not sure if the Jenkins > installation is done differently from other CDH installation. Also, I am > not > sure if it is ok to change sqenvcom.sh this way. If so, we can make this > change as a short term solution. So, please provide your feedback on this. > > However, as a long term solution I feel sqenvcom.sh is becoming quite > complex. There is a need to simplify it. One of the suggestion would be to > generate sqenvcom.sh based on the distro rather than making sqenvcom.sh to > cater to different distros. Even if we don’t generate the file fully, we > should consider generating CLASSPATH for different distro and source in > to > sqenvcom.sh only the generated installed distro CLASSPATH. > > To generate the CLASSPATH, Narendra said we can do the following > > hadoop classpath > > hbase classpath > > And expand all the * in the resulting class path to include all the jar > files in the path (We don’t expand *, the jar files are not seen from > JNI). > This would make it better to cater to installation specific classpath > changes too. > > Selva > > -----Original Message----- > From: Steve Varnau [*mailto:[email protected]* > <[email protected]> ] > Sent: Wednesday, April 6, 2016 9:55 AM > To: *[email protected]* > <[email protected]> > Subject: RE: Trafodion master Daily Test Result - 167 - Still Failing > > Yes. Before we hung trying to load the tpc-ds data into hive. That script > seems to be running fine now, and the test suite actually runs, though all > the hive related tests fail. So something else is not right. > > Test automation saves the hbase logs, but not hive logs, so I think we > will > need to spin up a vm to debug it. Who would like a crack at this one? > > --Steve > > > -----Original Message----- > > > From: Selva Govindarajan [mailto:[email protected] > <[email protected]>] > > > Sent: Wednesday, April 6, 2016 9:19 AM > > > To: [email protected] > > > Subject: RE: Trafodion master Daily Test Result - 167 - Still Failing > > > > > > I think it looks like a class path issue for hive tests in CDH. Need > > > to be analyzed in CDH vm. > > > > > > Selva > > > > > > -----Original Message----- > > > From: Sandhya Sundaresan [mailto:[email protected] > <[email protected]>] > > > Sent: Wednesday, April 6, 2016 7:48 AM > > > To: [email protected] > > > Subject: RE: Trafodion master Daily Test Result - 167 - Still Failing > > > > > > The executor suite passed after weeks but the problematic test > > > (TEST106) has been commented out until the new hang for "drop table" > > > is fixed just so the rest of the tests in the suite get run. The old > > > hang seems to be fixed now. > > > > > > Steve, looks like the Hive metadata is corrupt or not setup properly > > > on the VMs where the core and hive suites are run. > > > > > > Arvind, I suppose you're following up on the phoenix failures ? > > > > > > Sandhya > > > > > > -----Original Message----- > > > From: [email protected] [mailto:[email protected] > <[email protected]>] > > > Sent: Wednesday, April 6, 2016 4:02 AM > > > To: [email protected] > > > Subject: Trafodion master Daily Test Result - 167 - Still Failing > > > > > > Daily Automated Testing master > > > > > > Jenkins Job: https://jenkins.esgyn.com/job/Check-Daily-master/167/ > > > Archived Logs: http://traf-testlogs.esgyn.com/Daily-master/167 > > > Bld Downloads: http://traf-builds.esgyn.com > > > > > > Changes since previous daily build: > > > [Steve Varnau] [TRAFODION-1816] License info for binary distribution > > > of REST bundle > > > > > > [selvaganesang] [TRAFODION-1918] Remove auto flush concept in > > > Trafodion > > > > > > [selvaganesang] [TRAFODION-1918] Remove disabling auto flush concept > > > in Trafodion > > > > > > > > > > > > Test Job Results: > > > > > > FAILURE core-regress-core-cdh (1 hr 34 min) FAILURE > > > core-regress-hive-cdh > > > (22 min) FAILURE core-regress-privs1-cdh (41 min) FAILURE > > > phoenix_part1_T2-cdh (11 min) FAILURE phoenix_part1_T2-hdp (39 min) > > > FAILURE phoenix_part2_T2-cdh (11 min) FAILURE phoenix_part2_T2-hdp (38 > > > min) SUCCESS build-master-debug (25 min) SUCCESS build-master-release > > > (26 min) SUCCESS core-regress-charsets-cdh (27 min) SUCCESS > > > core-regress-charsets-hdp (1 hr > > > 29 min) SUCCESS core-regress-compGeneral-cdh (1 hr 25 min) SUCCESS > > > core-regress-compGeneral-hdp (47 min) SUCCESS core-regress-core-hdp (1 > > > hr > > > 1 > > > min) SUCCESS core-regress-executor-cdh (49 min) SUCCESS > > > core-regress-executor-hdp (1 hr 3 min) SUCCESS > > > core-regress-fullstack2-cdh > > > (1 hr 2 min) SUCCESS core-regress-fullstack2-hdp (14 min) SUCCESS > > > core-regress-hive-hdp (41 min) SUCCESS core-regress-privs1-hdp (1 hr 1 > > > min) > > > SUCCESS core-regress-privs2-cdh (35 min) SUCCESS > > > core-regress-privs2-hdp > > > (43 > > > min) SUCCESS core-regress-qat-cdh (14 min) SUCCESS > > > core-regress-qat-hdp (1 hr 13 min) SUCCESS core-regress-seabase-cdh (1 > > > hr 45 min) SUCCESS core-regress-seabase-hdp (2 hr 7 min) SUCCESS > > > core-regress-udr-cdh (18 > > > min) > > > SUCCESS core-regress-udr-hdp (40 min) SUCCESS jdbc_test-cdh (50 min) > > > SUCCESS jdbc_test-hdp (57 min) SUCCESS phoenix_part1_T4-cdh (42 min) > > > SUCCESS phoenix_part1_T4-hdp (1 hr 26 min) SUCCESS > > > phoenix_part2_T4-cdh (44 min) SUCCESS phoenix_part2_T4-hdp (1 hr 4 > > > min) SUCCESS pyodbc_test-cdh (10 min) SUCCESS pyodbc_test-hdp (38 min)
