Istvan Toth created HDFS-17759:
----------------------------------
Summary: Explicitly depend on jackson-core in hadoop-hdfs
Key: HDFS-17759
URL: https://issues.apache.org/jira/browse/HDFS-17759
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs
Affects Versions: 3.4.1, 3.5.0
Reporter: Istvan Toth
Assignee: Istvan Toth
Attachments: deptree.current, deptree.patched
HDFS does not depend on jacks-core, even though it directly uses it.
Apart from being a bad practice, this breaks consumers of Hadoop.
Hadoop builds are not affected, because the and old incompatible jackson-core
is brought in via hadoop-common and Avro, which is then dependencyManaged to
the correct version.
Howerver, this dependencyManagement does not apply to consumers of Hadoop,
which get the original old incompatible jackson-core.
Unless those projects happen to already correctly manage the jackson versions,
they will fail with something like this:
{noformat}
java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/util/JacksonFeature
3456 at
com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:656)
3457 at
com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:558)
3458 at
org.apache.hadoop.hdfs.server.blockmanagement.SlowPeerTracker.<clinit>(SlowPeerTracker.java:78)
3459 at
org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.initSlowPeerTracker(DatanodeManager.java:373)
3460 at
org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.<init>(DatanodeManager.java:263)
3461 at
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.<init>(BlockManager.java:502)
3462 at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:926)
3463 at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:851)
3464 at
org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1396)
3465 at
org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:495)
3466 at
org.apache.hadoop.hdfs.DFSTestUtil.formatNameNode(DFSTestUtil.java:256)
3467 at
org.apache.hadoop.hdfs.MiniDFSCluster.configureNameService(MiniDFSCluster.java:1158)
3468 at
org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:1042)
3469 at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:974)
3470 at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:888)
3471 at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:689)
3472 at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:669)
3473 at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1141)
3474 at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1106)
3475 at
org.apache.omid.timestamp.storage.TestHBaseTimestampStorage.setUpClass(TestHBaseTimestampStorage.java:65)
3476 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
3477 at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
3478 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
3479 at java.lang.reflect.Method.invoke(Method.java:498)
3480 at
org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:136)
3481 at
org.testng.internal.invokers.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:65)
3482 at
org.testng.internal.invokers.ConfigInvoker.invokeConfigurationMethod(ConfigInvoker.java:381)
3483 at
org.testng.internal.invokers.ConfigInvoker.invokeConfigurations(ConfigInvoker.java:319)
3484 at
org.testng.internal.invokers.TestMethodWorker.invokeBeforeClassMethods(TestMethodWorker.java:178)
3485 at
org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:122)
3486 at java.util.ArrayList.forEach(ArrayList.java:1259)
3487 at org.testng.TestRunner.privateRun(TestRunner.java:808)
3488 at org.testng.TestRunner.run(TestRunner.java:603)
3489 at org.testng.SuiteRunner.runTest(SuiteRunner.java:429)
3490 at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:423)
3491 at org.testng.SuiteRunner.privateRun(SuiteRunner.java:383)
3492 at org.testng.SuiteRunner.run(SuiteRunner.java:326)
3493 at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
3494 at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
3495 at org.testng.TestNG.runSuitesSequentially(TestNG.java:1249)
3496 at org.testng.TestNG.runSuitesLocally(TestNG.java:1169)
3497 at org.testng.TestNG.runSuites(TestNG.java:1092)
3498 at org.testng.TestNG.run(TestNG.java:1060)
3499 at
org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:155)
3500 at
org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:102)
3501 at
org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:91)
3502 at
org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:137)
3503 at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385)
3504 at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
3505 at
org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507)
3506 at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495)
3507Caused by: java.lang.ClassNotFoundException:
com.fasterxml.jackson.core.util.JacksonFeature
3508 at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
3509 at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
3510 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
3511 at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
3512 ... 51 more{noformat}
For reference, I'm attaching the dependency tree output from a project trying
to use HDFS Minicluster with and without the patch.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
