[
https://issues.apache.org/jira/browse/YARN-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183131#comment-16183131
]
Varun Saxena commented on YARN-7190:
------------------------------------
Before updating a patch, thought of taking an opinion from others. For backward
compatibility sake, on branch-2 I am planning to move only hbase specific jars
and their dependencies to a separate folder (share/hadoop/yarn/serverlib). We
can do the same for trunk/branch-3.0 as well even though backward compatibility
is not required. Thoughts?
*Before changes, set of jars in lib folder is as under:*
{noformat}
varun@e7450:~/Projects/hadoop/hadoop-dist/target/hadoop-3.1.0-SNAPSHOT/share/hadoop/yarn$
ll lib
total 21088
-rw-rw-r-- 1 varun varun 4467 Sep 28 00:36 aopalliance-1.0.jar
-rw-rw-r-- 1 varun varun 34827 Sep 28 00:36 commons-csv-1.0.jar
-rw-rw-r-- 1 varun varun 112341 Sep 28 00:36 commons-el-1.0.jar
-rw-rw-r-- 1 varun varun 305001 Sep 28 00:36 commons-httpclient-3.1.jar
-rw-rw-r-- 1 varun varun 988514 Sep 28 00:36 commons-math-2.2.jar
-rw-rw-r-- 1 varun varun 79576 Sep 28 00:36 disruptor-3.3.0.jar
-rw-rw-r-- 1 varun varun 1726527 Sep 28 00:36 ehcache-3.3.1.jar
-rw-rw-r-- 1 varun varun 15322 Sep 28 00:36 findbugs-annotations-1.3.9-1.jar
-rw-rw-r-- 1 varun varun 387689 Sep 28 00:36 fst-2.50.jar
-rw-rw-r-- 1 varun varun 55236 Sep 28 00:36
geronimo-jcache_1.0_spec-1.0-alpha-1.jar
-rw-rw-r-- 1 varun varun 668235 Sep 28 00:36 guice-4.0.jar
-rw-rw-r-- 1 varun varun 76983 Sep 28 00:36 guice-servlet-4.0.jar
-rw-rw-r-- 1 varun varun 20865 Sep 28 00:36 hbase-annotations-1.2.6.jar
-rw-rw-r-- 1 varun varun 1306445 Sep 28 00:36 hbase-client-1.2.6.jar
-rw-rw-r-- 1 varun varun 582558 Sep 28 00:36 hbase-common-1.2.6.jar
-rw-rw-r-- 1 varun varun 100739 Sep 28 00:36 hbase-hadoop2-compat-1.2.6.jar
-rw-rw-r-- 1 varun varun 36991 Sep 28 00:36 hbase-hadoop-compat-1.2.6.jar
-rw-rw-r-- 1 varun varun 102056 Sep 28 00:36 hbase-prefix-tree-1.2.6.jar
-rw-rw-r-- 1 varun varun 123671 Sep 28 00:36 hbase-procedure-1.2.6.jar
-rw-rw-r-- 1 varun varun 4378437 Sep 28 00:36 hbase-protocol-1.2.6.jar
-rw-rw-r-- 1 varun varun 4184325 Sep 28 00:36 hbase-server-1.2.6.jar
-rw-rw-r-- 1 varun varun 134308 Sep 28 00:36 HikariCP-java7-2.4.12.jar
-rw-rw-r-- 1 varun varun 1475955 Sep 28 00:36 htrace-core-3.1.0-incubating.jar
-rw-rw-r-- 1 varun varun 29947 Sep 28 00:36 jackson-jaxrs-base-2.7.8.jar
-rw-rw-r-- 1 varun varun 16776 Sep 28 00:36
jackson-jaxrs-json-provider-2.7.8.jar
-rw-rw-r-- 1 varun varun 34578 Sep 28 00:36
jackson-module-jaxb-annotations-2.7.8.jar
-rw-rw-r-- 1 varun varun 24543 Sep 28 00:36 jamon-runtime-2.4.1.jar
-rw-rw-r-- 1 varun varun 408133 Sep 28 00:36 jasper-compiler-5.5.23.jar
-rw-rw-r-- 1 varun varun 76844 Sep 28 00:36 jasper-runtime-5.5.23.jar
-rw-rw-r-- 1 varun varun 58487 Sep 28 00:36 java-util-1.9.0.jar
-rw-rw-r-- 1 varun varun 2497 Sep 28 00:36 javax.inject-1.jar
-rw-rw-r-- 1 varun varun 1291164 Sep 28 00:36 jcodings-1.0.8.jar
-rw-rw-r-- 1 varun varun 134021 Sep 28 00:36 jersey-client-1.19.jar
-rw-rw-r-- 1 varun varun 16151 Sep 28 00:36 jersey-guice-1.19.jar
-rw-rw-r-- 1 varun varun 187292 Sep 28 00:36 joni-2.1.2.jar
-rw-rw-r-- 1 varun varun 75232 Sep 28 00:36 json-io-2.5.1.jar
-rw-rw-r-- 1 varun varun 1024680 Sep 28 00:36 jsp-2.1-6.1.14.jar
-rw-rw-r-- 1 varun varun 134910 Sep 28 00:36 jsp-api-2.1-6.1.14.jar
-rw-rw-r-- 1 varun varun 82123 Sep 28 00:36 metrics-core-2.2.0.jar
-rw-rw-r-- 1 varun varun 85449 Sep 28 00:36 metrics-core-3.0.1.jar
-rw-rw-r-- 1 varun varun 792442 Sep 28 00:36 mssql-jdbc-6.2.1.jre7.jar
-rw-rw-r-- 1 varun varun 132368 Sep 28 00:36 servlet-api-2.5-6.1.14.jar
{noformat}
*After change, jars under lib and serverlib folders would be as under
(excluding hbase-server jar and its dependencies):*
{noformat}
varun@e7450:~/Projects/hadoop/hadoop-dist/target/hadoop-3.1.0-SNAPSHOT/share/hadoop/yarn$
ll *lib
lib:
total 4268
-rw-rw-r-- 1 varun varun 4467 Sep 28 00:29 aopalliance-1.0.jar
-rw-rw-r-- 1 varun varun 34827 Sep 28 00:29 commons-csv-1.0.jar
-rw-rw-r-- 1 varun varun 1726527 Sep 28 00:29 ehcache-3.3.1.jar
-rw-rw-r-- 1 varun varun 387689 Sep 28 00:29 fst-2.50.jar
-rw-rw-r-- 1 varun varun 55236 Sep 28 00:29
geronimo-jcache_1.0_spec-1.0-alpha-1.jar
-rw-rw-r-- 1 varun varun 668235 Sep 28 00:29 guice-4.0.jar
-rw-rw-r-- 1 varun varun 76983 Sep 28 00:29 guice-servlet-4.0.jar
-rw-rw-r-- 1 varun varun 134308 Sep 28 00:29 HikariCP-java7-2.4.12.jar
-rw-rw-r-- 1 varun varun 29947 Sep 28 00:29 jackson-jaxrs-base-2.7.8.jar
-rw-rw-r-- 1 varun varun 16776 Sep 28 00:29
jackson-jaxrs-json-provider-2.7.8.jar
-rw-rw-r-- 1 varun varun 34578 Sep 28 00:29
jackson-module-jaxb-annotations-2.7.8.jar
-rw-rw-r-- 1 varun varun 58487 Sep 28 00:29 java-util-1.9.0.jar
-rw-rw-r-- 1 varun varun 2497 Sep 28 00:29 javax.inject-1.jar
-rw-rw-r-- 1 varun varun 134021 Sep 28 00:29 jersey-client-1.19.jar
-rw-rw-r-- 1 varun varun 16151 Sep 28 00:29 jersey-guice-1.19.jar
-rw-rw-r-- 1 varun varun 75232 Sep 28 00:29 json-io-2.5.1.jar
-rw-rw-r-- 1 varun varun 85449 Sep 28 00:29 metrics-core-3.0.1.jar
-rw-rw-r-- 1 varun varun 792442 Sep 28 00:29 mssql-jdbc-6.2.1.jre7.jar
serverlib:
total 9140
-rw-rw-r-- 1 varun varun 15322 Sep 28 00:29 findbugs-annotations-1.3.9-1.jar
-rw-rw-r-- 1 varun varun 20865 Sep 28 00:29 hbase-annotations-1.2.6.jar
-rw-rw-r-- 1 varun varun 1306445 Sep 28 00:29 hbase-client-1.2.6.jar
-rw-rw-r-- 1 varun varun 582558 Sep 28 00:29 hbase-common-1.2.6.jar
-rw-rw-r-- 1 varun varun 4378437 Sep 28 00:29 hbase-protocol-1.2.6.jar
-rw-rw-r-- 1 varun varun 1475955 Sep 28 00:29 htrace-core-3.1.0-incubating.jar
-rw-rw-r-- 1 varun varun 1291164 Sep 28 00:29 jcodings-1.0.8.jar
-rw-rw-r-- 1 varun varun 187292 Sep 28 00:29 joni-2.1.2.jar
-rw-rw-r-- 1 varun varun 82123 Sep 28 00:29 metrics-core-2.2.0.jar
{noformat}
> Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user
> classpath
> ------------------------------------------------------------------------------------
>
> Key: YARN-7190
> URL: https://issues.apache.org/jira/browse/YARN-7190
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineclient, timelinereader, timelineserver
> Reporter: Vrushali C
> Assignee: Varun Saxena
>
> [~jlowe] had a good observation about the user classpath getting extra jars
> in hadoop 2.x brought in with TSv2. If users start picking up Hadoop 2,x's
> version of HBase jars instead of the ones they shipped with their job, it
> could be a problem.
> So when TSv2 is to be used in 2,x, the hbase related jars should come into
> only the NM classpath not the user classpath.
> Here is a list of some jars
> {code}
> commons-csv-1.0.jar
> commons-el-1.0.jar
> commons-httpclient-3.1.jar
> disruptor-3.3.0.jar
> findbugs-annotations-1.3.9-1.jar
> hbase-annotations-1.2.6.jar
> hbase-client-1.2.6.jar
> hbase-common-1.2.6.jar
> hbase-hadoop2-compat-1.2.6.jar
> hbase-hadoop-compat-1.2.6.jar
> hbase-prefix-tree-1.2.6.jar
> hbase-procedure-1.2.6.jar
> hbase-protocol-1.2.6.jar
> hbase-server-1.2.6.jar
> htrace-core-3.1.0-incubating.jar
> jamon-runtime-2.4.1.jar
> jasper-compiler-5.5.23.jar
> jasper-runtime-5.5.23.jar
> jcodings-1.0.8.jar
> joni-2.1.2.jar
> jsp-2.1-6.1.14.jar
> jsp-api-2.1-6.1.14.jar
> jsr311-api-1.1.1.jar
> metrics-core-2.2.0.jar
> servlet-api-2.5-6.1.14.jar
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
