I did the change to JDK8 on Ptest now. Please let me tknow if there are issues with Java8.
- Sergio On Fri, May 27, 2016 at 2:45 PM, Sergio Pena <sergio.p...@cloudera.com> wrote: > Thanks Mohit. > > I will plan to do the JDK8 change on Jenkins today EOD and monitor any > issues through the weekend. > > - Sergio > > > On Fri, May 27, 2016 at 2:58 AM, Mohit Sabharwal <mo...@cloudera.com> > wrote: > >> Update on moving Hive2 tests to JDK8: I've addressed almost all test >> failures in HIVE-13547 on java8 branch. There is one remaining >> open item (HIVE-13834) which is currently assigned. Given current >> state of flaky test runs, there might be few more. >> >> I will work with Sergio to merge the test fixes to master and switch >> the Hive2 pre-commit job to use JDK8, hopefully sometime tomorrow. >> >> After Hive2 tests switch, if your patch sees ordering related test >> failures in pre-commit run, it's likely JDK version related and you'll >> need to build & re-run the test using JDK8. Number of such tests >> should be relatively small. >> >> On Tue, Apr 19, 2016 at 10:43 AM, Mohit Sabharwal <mo...@cloudera.com> >> wrote: >> >> > Created HIVE-13547 to track switching 2x tests to JDK8. >> > >> > On Wed, Apr 13, 2016 at 10:02 AM, Sergio Pena <sergio.p...@cloudera.com >> > >> > wrote: >> > >> >> I agree with such change as JDK7 is not longer supported. >> >> >> >> Changes on Jenkins and Hive PTest shouldn't be hard. We just need to >> >> replace the path from java7 to java8. But I think we should fix all >> JDK8 >> >> issues or most of them before doing the change or we will end up >> having a >> >> lot of failures on all JIRAs running pre-commit tests. >> >> >> >> +1 with the change. >> >> >> >> On Mon, Apr 11, 2016 at 9:34 PM, Siddharth Seth <ss...@apache.org> >> wrote: >> >> >> >> > Option 3 sounds good. I'd ideally like to make JDK8 the minimum >> >> requirement >> >> > soon as well. >> >> > >> >> > On Mon, Apr 11, 2016 at 4:59 PM, Szehon Ho <sze...@cloudera.com> >> wrote: >> >> > >> >> > > Sounds like a good plan, +1 >> >> > > >> >> > > On Mon, Apr 11, 2016 at 4:31 PM, Mohit Sabharwal < >> mo...@cloudera.com> >> >> > > wrote: >> >> > > >> >> > > > Thanks, Ashutosh. Makes sense to keep the source and target as >> 1.7 >> >> > since >> >> > > > we're not using any JDK8 specific features yet. So, option (3) >> >> > > essentially >> >> > > > just means using JDK8 exclusively to build & test Hive2. >> >> > > > >> >> > > > On Sat, Apr 9, 2016 at 12:23 PM, Ashutosh Chauhan < >> >> > hashut...@apache.org> >> >> > > > wrote: >> >> > > > >> >> > > > > Hi Mohit, >> >> > > > > >> >> > > > > I also think option 3 makes sense. We should strive to keep >> test >> >> > matrix >> >> > > > > small so that we can do fast QA runs for dev patches. >> >> > > > > We can just use jdk7 to build & test Hive1 and jdk8 to build & >> >> test >> >> > > > Hive2. >> >> > > > > However, I am not sure of explicitly dropping support >> altogether >> >> for >> >> > > jdk7 >> >> > > > > on Hive2. We should make sure that in pom.xml java source & >> target >> >> > > > > compatibility is still 1.7 (which already is the case >> currently) >> >> so >> >> > > that >> >> > > > > Hive2 is still compatible with jdk7. Unit tests as I said we >> can >> >> run >> >> > on >> >> > > > > jdk8. >> >> > > > > >> >> > > > > Thanks, >> >> > > > > Ashutosh >> >> > > > > >> >> > > > > On Fri, Apr 8, 2016 at 4:34 PM, Mohit Sabharwal < >> >> mo...@cloudera.com> >> >> > > > > wrote: >> >> > > > > >> >> > > > > > Hi folks, >> >> > > > > > >> >> > > > > > Oracle EOL'ed (ended public updates) for Java 7 in April >> 2015. >> >> > > > > > In order to support Java 8, we fixed quite a bunch of tests >> in >> >> > > > > > HIVE-8607 (*) early last year. However, since our pre-commit >> >> tests >> >> > > run >> >> > > > > > on JDK7 only, any JDK8 test failures are getting ignored. As >> a >> >> > > result, >> >> > > > > > the count has crept up >> >> > > > > > < >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > >> >> > >> >> >> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-TRUNK-JAVA8/ >> >> > > > > > > >> >> > > > > > back >> >> > > > > > from zero to 125. >> >> > > > > > >> >> > > > > > Some options to address this: >> >> > > > > > >> >> > > > > > (1) Run pre-commit tests on both JDK7 and JDK8, in both 1.x >> >> > > > > > and 2.x: This will further slow down the pre-commit run. >> >> > > > > > >> >> > > > > > (2) Alternate pre-commit test runs between JDK7 and JDK8 (in >> >> > > > > > both 1.x and 2.x): It's a cheap hack. More messy, though >> >> failures >> >> > > > > > won't get ignored. >> >> > > > > > >> >> > > > > > (3) Drop support for JDK7 in Hive 2.x, i.e. build and test >> Hive >> >> 2.x >> >> > > on >> >> > > > > > JDK8 only. For Hive 1.x, continue running JDK7 pre-commit >> tests. >> >> > > > > > The pros are: >> >> > > > > > - Simple test matrix. >> >> > > > > > - No need to slow down test run or maintain version specific >> >> golden >> >> > > > files >> >> > > > > > (**). >> >> > > > > > - 2.x looks like the logical place to move to JDK8. >> >> > > > > > - Users transitioning to JDK8 for all other services do not >> >> have to >> >> > > > > > maintain >> >> > > > > > multiple java versions on the cluster. >> >> > > > > > >> >> > > > > > Option (3) looks most attractive to me. >> >> > > > > > >> >> > > > > > Moving to JDK8 also lines us up better for Java 9, which is >> on >> >> the >> >> > > > > > horizon (Oracle will end public updates for Java 8 in Sep >> 2017) >> >> > > > > > >> >> > > > > > Around 100 of the latest crop of failures are due to one >> cause >> >> > > > > > (HIVE-13409). >> >> > > > > > I can take a pass at triaging the rest if there is consensus >> >> around >> >> > > > what >> >> > > > > > to do overall. >> >> > > > > > >> >> > > > > > Thanks, >> >> > > > > > Mohit >> >> > > > > > >> >> > > > > > (*) Most test failures are due to JDK8 using a different hash >> >> > > function >> >> > > > > for >> >> > > > > > HashMap compared to JDK7. This results in (mostly benign, >> but >> >> hard >> >> > > > > > to debug) ordering differences in q-file output related to >> >> > > > serialization >> >> > > > > > order of map entries, numbering of stages in query plan, etc. >> >> > > > > > >> >> > > > > > (**) In some cases, hash function related ordering >> differences >> >> > > > originate >> >> > > > > > in external libraries like Avro, antlr, json ,Thrift's map, >> etc. >> >> > for >> >> > > > > which >> >> > > > > > code >> >> > > > > > changes are either more involved or led to more test >> failures. >> >> To >> >> > > > > address >> >> > > > > > this, >> >> > > > > > we added support for version specific golden files >> (HIVE-9109). >> >> > > > > Currently, >> >> > > > > > there >> >> > > > > > are ~40 golden files with ".java1.8.out" extension. >> >> > > > > > >> >> > > > > >> >> > > > >> >> > > >> >> > >> >> >> > >> > >> > >
