[jira] [Commented] (HADOOP-17424) Replace HTrace with No-Op tracer
[
https://issues.apache.org/jira/browse/HADOOP-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799029#comment-17799029
]
Ayush Saxena commented on HADOOP-17424:
---
What is the fix to this incompatibility?
It creates issues with Tez which is on 3.3.1 & Hive which is on 3.3.6, both
hive & tez jars are required during runtime,
{noformat}
2023-12-20T19:36:01,147 WARN [main] server.HiveServer2: Error starting
HiveServer2 on attempt 3, will retry in 6ms
java.lang.NoSuchMethodError:
org.apache.hadoop.fs.FsTracer.get(Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/htrace/core/Tracer;
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:323)
~[hadoop-hdfs-client-3.3.1.jar:?]
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:308)
~[hadoop-hdfs-client-3.3.1.jar:?]
at
org.apache.hadoop.hdfs.DistributedFileSystem.initDFSClient(DistributedFileSystem.java:201)
~[hadoop-hdfs-client-3.3.1.jar:?]
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:186)
~[hadoop-hdfs-client-3.3.1.jar:?]
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3611)
~[hadoop-common-3.3.6.jar:?]
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:174)
~[hadoop-common-3.3.6.jar:?]
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3712)
~[hadoop-common-3.3.6.jar:?]
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3663)
~[hadoop-common-3.3.6.jar:?]
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:557)
~[hadoop-common-3.3.6.jar:?]
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
~[hadoop-common-3.3.6.jar:?]
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:705)
~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:669)
~[hive-exec-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:134)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at org.apache.hive.service.cli.CLIService.init(CLIService.java:119)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hive.service.CompositeService.init(CompositeService.java:59)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:243)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1114)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hive.service.server.HiveServer2.access$1500(HiveServer2.java:146)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1476)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at
org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1283)
~[hive-service-4.0.0-beta-2-SNAPSHOT.jar:4.0.0-beta-2-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_342]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_342]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_342]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_342]
at org.apache.hadoop.util.RunJar.run(RunJar.java:328)
~[hadoop-common-3.3.6.jar:?]
at org.apache.hadoop.util.RunJar.main(RunJar.java:241)
~[hadoop-common-3.3.6.jar:?]{noformat}
Similar stuff here as well:
[https://github.com/exasol/cloud-storage-extension/issues/219]
We could have explore keeping the package name same right?
> Replace HTrace with No-Op tracer
>
>
> Key: HADOOP-17424
> URL: https://issues.apache.org/jira/browse/HADOOP-17424
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Siyao Meng
>Assignee: Siyao Meng
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
> Time Spent: 10.5h
> Remaining Estimate: 0h
>
> Remove HTrace dependency as it is depending on old jackson jars. Use a no-op
> tracer for now to eliminate potential security issues.
> The plan is to move part of the code in
> [PR#1846|https://github.com/apache/hadoop/pull/1846] out here for faster
> review.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail:
[jira] [Commented] (HADOOP-17424) Replace HTrace with No-Op tracer
[ https://issues.apache.org/jira/browse/HADOOP-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17692097#comment-17692097 ] Steve Loughran commented on HADOOP-17424: - [~bpatel] -[~smeng] is correct. There is a version of the 3.3.0 release with this feature, it is called "3.3.2". please upgrade. > Replace HTrace with No-Op tracer > > > Key: HADOOP-17424 > URL: https://issues.apache.org/jira/browse/HADOOP-17424 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 10.5h > Remaining Estimate: 0h > > Remove HTrace dependency as it is depending on old jackson jars. Use a no-op > tracer for now to eliminate potential security issues. > The plan is to move part of the code in > [PR#1846|https://github.com/apache/hadoop/pull/1846] out here for faster > review. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17424) Replace HTrace with No-Op tracer
[ https://issues.apache.org/jira/browse/HADOOP-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691638#comment-17691638 ] Siyao Meng commented on HADOOP-17424: - Hi [~bpatel], it should be easy to cherry-pick this onto the 3.3.0 branch as it is in 3.3.2. However, this won't be done upstream since 3.3.0 branch is a released branch and is [frozen|https://github.com/apache/hadoop/tree/branch-3.3.0]. If you would like to, you can cherry-pick the [commit](1a205cc3adffa568c814a5241e041b08e2fcd3eb) and try to apply it to your fork's branch-3.3.0 branch and compile it. > Replace HTrace with No-Op tracer > > > Key: HADOOP-17424 > URL: https://issues.apache.org/jira/browse/HADOOP-17424 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 10.5h > Remaining Estimate: 0h > > Remove HTrace dependency as it is depending on old jackson jars. Use a no-op > tracer for now to eliminate potential security issues. > The plan is to move part of the code in > [PR#1846|https://github.com/apache/hadoop/pull/1846] out here for faster > review. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17424) Replace HTrace with No-Op tracer
[ https://issues.apache.org/jira/browse/HADOOP-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645947#comment-17645947 ] Bhavik Patel commented on HADOOP-17424: --- Is it possible to backport to 3.3.0? > Replace HTrace with No-Op tracer > > > Key: HADOOP-17424 > URL: https://issues.apache.org/jira/browse/HADOOP-17424 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 10.5h > Remaining Estimate: 0h > > Remove HTrace dependency as it is depending on old jackson jars. Use a no-op > tracer for now to eliminate potential security issues. > The plan is to move part of the code in > [PR#1846|https://github.com/apache/hadoop/pull/1846] out here for faster > review. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17424) Replace HTrace with No-Op tracer
[ https://issues.apache.org/jira/browse/HADOOP-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17500308#comment-17500308 ] Rajesh Krishnamurthy commented on HADOOP-17424: --- Hi there, Is there any plans/update on completely removing/getting rid off Htrace dependency from the Hadoop? Since the Htace is a no longer being actively developed for close to 4 years, but the Htrace is having hard dependency on jackson-databind old version, hadoop versions have been flagged for various vulnerabilities due to the lingering older version of databind. > Replace HTrace with No-Op tracer > > > Key: HADOOP-17424 > URL: https://issues.apache.org/jira/browse/HADOOP-17424 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.2 > > Time Spent: 10.5h > Remaining Estimate: 0h > > Remove HTrace dependency as it is depending on old jackson jars. Use a no-op > tracer for now to eliminate potential security issues. > The plan is to move part of the code in > [PR#1846|https://github.com/apache/hadoop/pull/1846] out here for faster > review. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17424) Replace HTrace with No-Op tracer
[ https://issues.apache.org/jira/browse/HADOOP-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17424396#comment-17424396 ] Masatake Iwasaki commented on HADOOP-17424: --- I submitted [#3520|https://github.com/apache/hadoop/pull/3520] for branch-3.3. > Replace HTrace with No-Op tracer > > > Key: HADOOP-17424 > URL: https://issues.apache.org/jira/browse/HADOOP-17424 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 9h 10m > Remaining Estimate: 0h > > Remove HTrace dependency as it is depending on old jackson jars. Use a no-op > tracer for now to eliminate potential security issues. > The plan is to move part of the code in > [PR#1846|https://github.com/apache/hadoop/pull/1846] out here for faster > review. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17424) Replace HTrace with No-Op tracer
[ https://issues.apache.org/jira/browse/HADOOP-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17411325#comment-17411325 ] Steve Loughran commented on HADOOP-17424: - [~smeng] have you tried to backport this to branch-3.3? I'd like to get it in there > Replace HTrace with No-Op tracer > > > Key: HADOOP-17424 > URL: https://issues.apache.org/jira/browse/HADOOP-17424 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 9h > Remaining Estimate: 0h > > Remove HTrace dependency as it is depending on old jackson jars. Use a no-op > tracer for now to eliminate potential security issues. > The plan is to move part of the code in > [PR#1846|https://github.com/apache/hadoop/pull/1846] out here for faster > review. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17424) Replace HTrace with No-Op tracer
[ https://issues.apache.org/jira/browse/HADOOP-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356482#comment-17356482 ] Steve Loughran commented on HADOOP-17424: - bq. Is there any plan to backport this change to current stable version 3.2.x? unlikely, it's why it has the term "stable". Expect changes in branch-3.3.; we could target 3.3.2 with this. > Replace HTrace with No-Op tracer > > > Key: HADOOP-17424 > URL: https://issues.apache.org/jira/browse/HADOOP-17424 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 9h > Remaining Estimate: 0h > > Remove HTrace dependency as it is depending on old jackson jars. Use a no-op > tracer for now to eliminate potential security issues. > The plan is to move part of the code in > [PR#1846|https://github.com/apache/hadoop/pull/1846] out here for faster > review. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17424) Replace HTrace with No-Op tracer
[ https://issues.apache.org/jira/browse/HADOOP-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1735#comment-1735 ] Jason Wen commented on HADOOP-17424: Is there any plan to backport this change to current stable version 3.2.x? > Replace HTrace with No-Op tracer > > > Key: HADOOP-17424 > URL: https://issues.apache.org/jira/browse/HADOOP-17424 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 9h > Remaining Estimate: 0h > > Remove HTrace dependency as it is depending on old jackson jars. Use a no-op > tracer for now to eliminate potential security issues. > The plan is to move part of the code in > [PR#1846|https://github.com/apache/hadoop/pull/1846] out here for faster > review. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17424) Replace HTrace with No-Op tracer
[
https://issues.apache.org/jira/browse/HADOOP-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17276195#comment-17276195
]
Siyao Meng commented on HADOOP-17424:
-
Thanks [~iwasakims]. Indeed the removal of {{TraceAdminProtocol}} (among other
interfaces used by HTrace) might cause compatibility issues and thus this
should only go in 3.4.0.
> Replace HTrace with No-Op tracer
>
>
> Key: HADOOP-17424
> URL: https://issues.apache.org/jira/browse/HADOOP-17424
> Project: Hadoop Common
> Issue Type: Sub-task
>Reporter: Siyao Meng
>Assignee: Siyao Meng
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 9h
> Remaining Estimate: 0h
>
> Remove HTrace dependency as it is depending on old jackson jars. Use a no-op
> tracer for now to eliminate potential security issues.
> The plan is to move part of the code in
> [PR#1846|https://github.com/apache/hadoop/pull/1846] out here for faster
> review.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17424) Replace HTrace with No-Op tracer
[ https://issues.apache.org/jira/browse/HADOOP-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17276053#comment-17276053 ] Masatake Iwasaki commented on HADOOP-17424: --- [~smeng] I marked this as incompatible change and added a release note. Feel free to update the words. Thanks. > Replace HTrace with No-Op tracer > > > Key: HADOOP-17424 > URL: https://issues.apache.org/jira/browse/HADOOP-17424 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 8h 40m > Remaining Estimate: 0h > > Remove HTrace dependency as it is depending on old jackson jars. Use a no-op > tracer for now to eliminate potential security issues. > The plan is to move part of the code in > [PR#1846|https://github.com/apache/hadoop/pull/1846] out here for faster > review. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17424) Replace HTrace with No-Op tracer
[ https://issues.apache.org/jira/browse/HADOOP-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247437#comment-17247437 ] Siyao Meng commented on HADOOP-17424: - [~iwasakims] Ah my bad. Forgot to put that one under the umbrella jira. Thought I forgot to create it. Will do. > Replace HTrace with No-Op tracer > > > Key: HADOOP-17424 > URL: https://issues.apache.org/jira/browse/HADOOP-17424 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > > Remove HTrace dependency as it is depending on old jackson jars. Use a no-op > tracer for now to eliminate potential security issues. > The plan is to move part of the code in > [PR#1846|https://github.com/apache/hadoop/pull/1846] out here for faster > review. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17424) Replace HTrace with No-Op tracer
[ https://issues.apache.org/jira/browse/HADOOP-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17247172#comment-17247172 ] Masatake Iwasaki commented on HADOOP-17424: --- [~smeng] Should we close HADOOP-17387 as duplicate? > Replace HTrace with No-Op tracer > > > Key: HADOOP-17424 > URL: https://issues.apache.org/jira/browse/HADOOP-17424 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > > Remove HTrace dependency as it is depending on old jackson jars. Use a no-op > tracer for now to eliminate potential security issues. > The plan is to move part of the code in > [PR#1846|https://github.com/apache/hadoop/pull/1846] out here for faster > review. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
