advancedxy commented on PR #464: URL: https://github.com/apache/incubator-uniffle/pull/464#issuecomment-1377281477
> > > > Does this mean we are getting rid of `log4j:1.2.17`? > > > > When I was working on spark code, I noticed spark still depends on `log4j:1.2.17`. > > > > > > > > > Will it be a problem? > > > > > > if we cannot get rid of `log4j:1.2.17`, the CVE issues doesn't go away? Thus maybe this PR is not that urgent? > > P.S: I have no objection for merging this PR. > > We can't control the Spark. We only need to guarantee that rss service don't have the danger. And Uniffle can be used for multiple frameworks. The problem is that if we are depending on the big data ecosystem, there might be no good way to avoid this. A quick ` dependency:tree` shows that hadoop-2.8.5 also relies on log4j:1.2.17 ``` [INFO] -------------------< org.apache.uniffle:coordinator >------------------- [INFO] Building Apache Uniffle Coordinator 0.7.0-snapshot [4/13] [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- maven-dependency-plugin:2.10:tree (default-cli) @ coordinator --- [INFO] org.apache.uniffle:coordinator:jar:0.7.0-snapshot [INFO] +- org.apache.uniffle:rss-common:jar:0.7.0-snapshot:compile [INFO] | +- org.apache.uniffle:rss-proto:jar:0.7.0-snapshot:compile [INFO] | | \- javax.annotation:javax.annotation-api:jar:1.3.2:compile [INFO] | +- info.picocli:picocli:jar:4.5.2:compile [INFO] | +- io.prometheus:simpleclient:jar:0.9.0:compile [INFO] | +- io.prometheus:simpleclient_hotspot:jar:0.9.0:compile [INFO] | +- io.prometheus:simpleclient_httpserver:jar:0.9.0:compile [INFO] | | \- io.prometheus:simpleclient_common:jar:0.9.0:compile [INFO] | +- io.prometheus:simpleclient_jetty:jar:0.9.0:compile [INFO] | | +- org.eclipse.jetty:jetty-server:jar:9.0.2.v20130417:compile [INFO] | | | +- org.eclipse.jetty.orbit:javax.servlet:jar:3.0.0.v201112011016:compile [INFO] | | | +- org.eclipse.jetty:jetty-http:jar:9.0.2.v20130417:compile [INFO] | | | | \- org.eclipse.jetty:jetty-util:jar:9.0.2.v20130417:compile [INFO] | | | \- org.eclipse.jetty:jetty-io:jar:9.0.2.v20130417:compile [INFO] | | \- org.eclipse.jetty:jetty-servlet:jar:9.0.2.v20130417:compile [INFO] | | \- org.eclipse.jetty:jetty-security:jar:9.0.2.v20130417:compile [INFO] | +- io.prometheus:simpleclient_servlet:jar:0.9.0:compile [INFO] | +- io.prometheus:simpleclient_pushgateway:jar:0.9.0:compile [INFO] | | \- javax.xml.bind:jaxb-api:jar:2.3.0:compile [INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.10.0:compile [INFO] | | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.10.0:compile [INFO] | | \- com.fasterxml.jackson.core:jackson-core:jar:2.10.0:compile [INFO] | +- org.roaringbitmap:RoaringBitmap:jar:0.9.15:compile [INFO] | | \- org.roaringbitmap:shims:jar:0.9.15:runtime [INFO] | \- net.jpountz.lz4:lz4:jar:1.3.0:compile [INFO] +- com.google.protobuf:protobuf-java-util:jar:3.19.2:compile [INFO] | +- com.google.protobuf:protobuf-java:jar:3.19.2:compile [INFO] | +- com.google.guava:guava:jar:31.0.1-jre:compile [INFO] | | +- com.google.guava:failureaccess:jar:1.0.1:compile [INFO] | | +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile [INFO] | | \- org.checkerframework:checker-qual:jar:3.12.0:compile [INFO] | +- com.google.j2objc:j2objc-annotations:jar:1.3:compile [INFO] | +- com.google.code.findbugs:jsr305:jar:3.0.2:compile [INFO] | \- com.google.code.gson:gson:jar:2.9.0:compile [INFO] +- io.grpc:grpc-netty-shaded:jar:1.47.0:runtime [INFO] | +- io.perfmark:perfmark-api:jar:0.25.0:runtime [INFO] | \- io.grpc:grpc-core:jar:1.47.0:runtime (version selected from constraint [1.47.0,1.47.0]) [INFO] | +- com.google.android:annotations:jar:4.1.1.4:runtime [INFO] | \- org.codehaus.mojo:animal-sniffer-annotations:jar:1.19:runtime [INFO] +- io.grpc:grpc-protobuf:jar:1.47.0:compile [INFO] | +- io.grpc:grpc-api:jar:1.47.0:compile [INFO] | +- com.google.api.grpc:proto-google-common-protos:jar:2.0.1:compile [INFO] | \- io.grpc:grpc-protobuf-lite:jar:1.47.0:compile [INFO] +- io.grpc:grpc-stub:jar:1.47.0:compile [INFO] +- io.grpc:grpc-testing:jar:1.47.0:test [INFO] | \- io.grpc:grpc-context:jar:1.47.0:compile [INFO] +- org.apache.commons:commons-lang3:jar:3.10:compile [INFO] +- org.apache.uniffle:rss-common:test-jar:tests:0.7.0-snapshot:test [INFO] +- org.apache.hadoop:hadoop-common:jar:2.8.5:provided [INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.8.5:provided [INFO] | | \- jdk.tools:jdk.tools:jar:1.8:system [INFO] | +- commons-cli:commons-cli:jar:1.2:provided [INFO] | +- org.apache.commons:commons-math3:jar:3.1.1:provided [INFO] | +- xmlenc:xmlenc:jar:0.52:provided [INFO] | +- org.apache.httpcomponents:httpclient:jar:4.5.2:provided [INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.4.4:provided [INFO] | +- commons-codec:commons-codec:jar:1.9:provided [INFO] | +- commons-io:commons-io:jar:2.4:provided [INFO] | +- commons-net:commons-net:jar:3.1:provided [INFO] | +- commons-collections:commons-collections:jar:3.2.2:provided [INFO] | +- javax.servlet:servlet-api:jar:2.5:provided [INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:provided [INFO] | +- org.mortbay.jetty:jetty-util:jar:6.1.26:provided [INFO] | +- org.mortbay.jetty:jetty-sslengine:jar:6.1.26:provided [INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:provided [INFO] | +- com.sun.jersey:jersey-core:jar:1.9:provided [INFO] | +- com.sun.jersey:jersey-json:jar:1.9:provided [INFO] | | +- org.codehaus.jettison:jettison:jar:1.1:provided [INFO] | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:provided [INFO] | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:provided [INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.9.13:provided [INFO] | +- com.sun.jersey:jersey-server:jar:1.9:provided [INFO] | | \- asm:asm:jar:3.1:provided [INFO] | +- commons-logging:commons-logging:jar:1.2:provided [INFO] | +- log4j:log4j:jar:1.2.17:compile [INFO] | +- net.java.dev.jets3t:jets3t:jar:0.9.0:provided [INFO] | | \- com.jamesmurty.utils:java-xmlbuilder:jar:0.4:provided [INFO] | +- commons-lang:commons-lang:jar:2.6:provided [INFO] | +- commons-configuration:commons-configuration:jar:1.6:provided [INFO] | | +- commons-digester:commons-digester:jar:1.8:provided [INFO] | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:provided [INFO] | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:provided [INFO] | +- org.slf4j:slf4j-api:jar:1.7.25:compile [INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:provided [INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:provided [INFO] | +- org.apache.avro:avro:jar:1.7.4:provided [INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:provided [INFO] | | \- org.xerial.snappy:snappy-java:jar:1.1.8.4:provided [INFO] | +- org.apache.hadoop:hadoop-auth:jar:2.8.5:provided [INFO] | | +- com.nimbusds:nimbus-jose-jwt:jar:4.41.1:provided [INFO] | | | \- com.github.stephenc.jcip:jcip-annotations:jar:1.0-1:provided [INFO] | | +- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:provided [INFO] | | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:provided [INFO] | | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:provided [INFO] | | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:provided [INFO] | | \- org.apache.curator:curator-framework:jar:2.7.1:provided [INFO] | +- com.jcraft:jsch:jar:0.1.54:provided [INFO] | +- org.apache.curator:curator-client:jar:2.7.1:provided [INFO] | +- org.apache.curator:curator-recipes:jar:2.7.1:provided [INFO] | +- org.apache.htrace:htrace-core4:jar:4.0.1-incubating:provided [INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.6:provided [INFO] | | \- io.netty:netty:jar:3.7.0.Final:provided [INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:provided [INFO] | \- org.tukaani:xz:jar:1.0:provided [INFO] +- org.apache.hadoop:hadoop-minicluster:jar:2.8.5:test [INFO] | +- org.apache.hadoop:hadoop-common:test-jar:tests:2.8.5:test [INFO] | +- org.apache.hadoop:hadoop-hdfs:test-jar:tests:2.8.5:test [INFO] | | +- org.apache.hadoop:hadoop-hdfs-client:jar:2.8.5:test [INFO] | | | \- com.squareup.okhttp:okhttp:jar:2.4.0:test [INFO] | | | \- com.squareup.okio:okio:jar:1.4.0:test [INFO] | | +- commons-daemon:commons-daemon:jar:1.0.13:test [INFO] | | +- io.netty:netty-all:jar:4.1.68.Final:test [INFO] | | +- xerces:xercesImpl:jar:2.9.1:test [INFO] | | | \- xml-apis:xml-apis:jar:1.3.04:test [INFO] | | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:test [INFO] | +- org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:2.8.5:test [INFO] | | +- org.apache.hadoop:hadoop-yarn-server-common:jar:2.8.5:test [INFO] | | +- org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:2.8.5:test [INFO] | | | +- com.sun.jersey:jersey-client:jar:1.9:test [INFO] | | | +- com.google.inject:guice:jar:3.0:test [INFO] | | | | +- javax.inject:javax.inject:jar:1:test [INFO] | | | | \- aopalliance:aopalliance:jar:1.0:test [INFO] | | | \- com.sun.jersey.contribs:jersey-guice:jar:1.9:test [INFO] | | +- org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.8.5:test [INFO] | | | +- org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:2.8.5:test [INFO] | | | +- org.apache.curator:curator-test:jar:2.7.1:test [INFO] | | | | +- org.javassist:javassist:jar:3.18.1-GA:test [INFO] | | | | \- org.apache.commons:commons-math:jar:2.2:test [INFO] | | | \- org.apache.zookeeper:zookeeper:test-jar:tests:3.4.6:test [INFO] | | \- org.apache.hadoop:hadoop-yarn-common:jar:2.8.5:test [INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:test-jar:tests:2.8.5:test [INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.8.5:test [INFO] | | | \- org.apache.hadoop:hadoop-yarn-client:jar:2.8.5:test [INFO] | | +- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.8.5:test [INFO] | | \- com.google.inject.extensions:guice-servlet:jar:3.0:test [INFO] | +- org.apache.hadoop:hadoop-hdfs:jar:2.8.5:test [INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.8.5:test [INFO] | | \- org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.8.5:test [INFO] | +- org.apache.hadoop:hadoop-yarn-api:jar:2.8.5:test [INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.8.5:test [INFO] | +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.8.5:test [INFO] | \- org.apache.hadoop:hadoop-mapreduce-client-hs:jar:2.8.5:test [INFO] +- org.mockito:mockito-inline:jar:3.12.4:test [INFO] | \- org.mockito:mockito-core:jar:3.12.4:test [INFO] | +- net.bytebuddy:byte-buddy:jar:1.11.13:test [INFO] | +- net.bytebuddy:byte-buddy-agent:jar:1.11.13:test [INFO] | \- org.objenesis:objenesis:jar:3.2:test [INFO] +- org.slf4j:slf4j-log4j12:jar:1.7.25:compile [INFO] +- com.google.errorprone:error_prone_annotations:jar:2.10.0:compile [INFO] +- org.awaitility:awaitility:jar:4.2.0:test [INFO] | \- org.hamcrest:hamcrest:jar:2.1:test [INFO] +- org.junit.jupiter:junit-jupiter:jar:5.8.2:test [INFO] | +- org.junit.jupiter:junit-jupiter-api:jar:5.8.2:test [INFO] | | +- org.opentest4j:opentest4j:jar:1.2.0:test [INFO] | | \- org.junit.platform:junit-platform-commons:jar:1.8.2:test [INFO] | +- org.junit.jupiter:junit-jupiter-params:jar:5.8.2:test [INFO] | \- org.junit.jupiter:junit-jupiter-engine:jar:5.8.2:test [INFO] +- org.junit.platform:junit-platform-launcher:jar:1.8.2:test [INFO] | +- org.junit.platform:junit-platform-engine:jar:1.8.2:test [INFO] | \- org.apiguardian:apiguardian-api:jar:1.1.2:test [INFO] \- uk.org.webcompere:system-stubs-jupiter:jar:2.0.1:test [INFO] \- uk.org.webcompere:system-stubs-core:jar:2.0.1:test ``` https://issues.apache.org/jira/browse/HADOOP-16206 https://issues.apache.org/jira/browse/HADOOP-12956 Seems that there's no good way for hadoop to get rid of log4j 1x -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
