abstractdog commented on PR #3279:
URL: https://github.com/apache/hive/pull/3279#issuecomment-1141946273
> I've deployed
>
> * hadoop-3.3.3
> * tez 0.10.1
> * hive from the PR
> running a simple insert failed with:
>
> ```
> Caused by: java.lang.NoSuchMethodError:
org.eclipse.jetty.server.session.SessionHandler.getSessionManager()Lorg/eclipse/jetty/server/SessionManager;
> at
org.apache.hadoop.http.HttpServer2.initializeWebServer(HttpServer2.java:569)
> at org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:550)
> at org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:117)
> at
org.apache.hadoop.http.HttpServer2$Builder.build(HttpServer2.java:425)
> at org.apache.hadoop.yarn.webapp.WebApps$Builder.build(WebApps.java:341)
> at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:432)
> at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:428)
> at
org.apache.tez.dag.app.web.WebUIService.serviceStart(WebUIService.java:94)
> at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
> at
org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency.start(DAGAppMaster.java:1800)
> at
org.apache.tez.dag.app.DAGAppMaster$ServiceThread.run(DAGAppMaster.java:1821)
> 2022-05-31 09:17:19,422 [INFO] [shutdown-hook-0] |app.DAGAppMaster|:
DAGAppMasterShutdownHook invoked
> ```
>
> maybe I've missed something - but it seems like the tez dagappmaster has
issues running with the jetty because of hadoop 3.3.3
>
> I've these settings:
>
> ```
> tez/tez-site/tez.lib.uris ${fs.defaultFS}/apps/tez/tez.tar.gz
> tez/tez-site/tez.use.cluster.hadoop-libs true
> ```
>
> @abstractdog is hadoop-3.3.3 supported with tez-0.10.1?
thanks @kgyrtkirk for trying this out, I've just created TEZ-4420, as tez is
on hadoop 3.3.1, and I'm not sure about compatibility
checked tez.tar.gz contents before and after bump and I got:
```
hadoop 3.3.1
tar tf tez-dist/target/tez-0.10.2-SNAPSHOT.tar.gz | grep jetty
lib/jetty-server-9.4.40.v20210413.jar
lib/jetty-http-9.4.40.v20210413.jar
lib/jetty-io-9.4.40.v20210413.jar
lib/jetty-util-9.4.40.v20210413.jar
lib/jetty-servlet-9.4.40.v20210413.jar
lib/jetty-security-9.4.40.v20210413.jar
lib/jetty-util-ajax-9.4.40.v20210413.jar
lib/jetty-webapp-9.4.40.v20210413.jar
lib/jetty-xml-9.4.40.v20210413.jar
lib/jetty-client-9.4.40.v20210413.jar
hadoop 3.3.3
tar tf tez-dist/target/tez-0.10.2-SNAPSHOT.tar.gz | grep jetty
lib/jetty-server-9.4.43.v20210629.jar
lib/jetty-http-9.4.43.v20210629.jar
lib/jetty-io-9.4.43.v20210629.jar
lib/jetty-util-9.4.43.v20210629.jar
lib/jetty-servlet-9.4.43.v20210629.jar
lib/jetty-security-9.4.43.v20210629.jar
lib/jetty-util-ajax-9.4.43.v20210629.jar
lib/jetty-webapp-9.4.43.v20210629.jar
lib/jetty-xml-9.4.43.v20210629.jar
lib/jetty-client-9.4.43.v20210629.jar
```
tez packs jetty from hadoop, so hadoop upgrade means jetty upgrade too, so
there is a chance that an old tez.tar.gz can clash with new hadoop
I've just uplodaded the new tez.tar.gz, is there a chance you can give it a
try?
https://drive.google.com/file/d/18RMfh40s6kKdFt77E7HpS-j4EJhd2DKi/view?usp=sharing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
