aturoczy commented on PR #4290: URL: https://github.com/apache/hive/pull/4290#issuecomment-1536561628
> Well I have an almost green build, the only test failing is a regular offender, there is already a ticket for that. > > Summary of changes: > > * Addition of **junit-jupiter-engine** as a dependency in one module: It is coming transitively & now it doesn't come so had to add it. > * **BouncyCastle Upgrade**: Hadoop did that and seems this dependency ain't backward compatible, so we have to upgrade else test fails with NoClassDef Ex. > * The message change in exception: is expired to has expired: **HDFS-16563** changed the exception message, So had to adapt to that. > * Now the interesting part: J**ersey upgrade from 9.4.40.v20210413 to 9.4.45.v20220203** > > Why needed: Hadoop upgraded Jersey and If we don't we get NoClassDef Exceptions > > ``` > java.lang.NoClassDefFoundError: org/eclipse/jetty/server/SymlinkAllowedResourceAliasChecker > at org.apache.hadoop.http.HttpServer2.addDefaultApps(HttpServer2.java:875) > ``` > > Ref: http://ci.hive.apache.org/job/hive-precommit/job/PR-4290/1/testReport/junit/org.apache.hadoop.hive.ql/TestDDLWithRemoteMetastoreSecondNamenode/Testing___split_01___PostProcess___testCreateDatabaseWithTableNonDefaultNameNode/ > > Why not moving to 9.4.48.v20220622 which is same as Hadoop version: > > * **Hbase Cries** > > ``` > Caused by: java.lang.IllegalArgumentException: headerCacheSize 65536 exceeds maximum value of 65534 > at org.eclipse.jetty.server.HttpConfiguration.setHeaderCacheSize(HttpConfiguration.java:440) > at org.apache.hadoop.hbase.http.HttpServer$Builder.build(HttpServer.java:384) > at org.apache.hadoop.hbase.http.InfoServer.<init>(InfoServer.java:87) > at org.apache.hadoop.hbase.regionserver.HRegionServer.putUpWebUI(HRegionServer.java:2075) > ``` > > So, **the chosen version of Jetty is higher than the current Hive version but lower than the current hadoop 3.3.5 version.** > > One solution was to move to Hbase 2.4.0 and leverage [HBASE-23834](https://issues.apache.org/jira/browse/HBASE-23834), but that upgrade doesn't help either, The Jetty exception goes away but the Hbase cluster doesn't start itself. So, playing with Hbase ain't very cool here, We can chase that separately in a different ticket, If it works? Good, If it doesn't work? Try some profiles or something for Hbase, if it still doesn't work. Will start a thread to drop Hbase support from Hive because I have a valid reason "it will block our future Hadoop upgrades". Count my +1(Binding) for that if we reach to that state :) > > The remaining part: Need to try in an actual cluster, to see how things behave in an actual env It is not possible to use the 3.0.0-alpha to resolve the issue? I know it is not the best, but as we move ahead with 4.0 or 4.1 in upstream maybe acceptable. I know it is not the best, just an idea. This HBase issue seems pretty problematic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
