LuciferYang commented on code in PR #38384:
URL: https://github.com/apache/spark/pull/38384#discussion_r1163566789
##########
connector/protobuf/pom.xml:
##########
@@ -93,6 +93,7 @@
<artifactId>maven-shade-plugin</artifactId>
<configuration>
<shadedArtifactAttached>false</shadedArtifactAttached>
+ <shadeTestJar>true</shadeTestJar>
Review Comment:
@rangadi May I ask why `shadeTestJar` is configured to be true here? From
the compilation logs, it can be seen that Maven shade does not have any tests
jars being shaded, but sometimes, it will try to download non-existent jar
packages, which may affect compilation speed
```
[INFO] --- shade:3.4.1:shade (default) @ spark-protobuf_2.12 ---
[INFO] Including com.google.protobuf:protobuf-java:jar:3.22.0 in the shaded
jar.
Downloading from jitpack.io:
https://jitpack.io/com/google/protobuf/protobuf-java/3.22.0/protobuf-java-3.22.0-tests.jar
Downloading from ossrh:
https://oss.sonatype.org/content/repositories/snapshots/com/google/protobuf/protobuf-java/3.22.0/protobuf-java-3.22.0-tests.jar
[WARNING] Could not get tests for
com.google.protobuf:protobuf-java:jar:3.22.0:compile
[INFO] Excluding org.apache.orc:orc-core:jar:shaded-protobuf:1.8.3 from the
shaded jar.
[INFO] Excluding org.apache.orc:orc-shims:jar:1.8.3 from the shaded jar.
[INFO] Excluding io.airlift:aircompressor:jar:0.21 from the shaded jar.
[INFO] Excluding org.jetbrains:annotations:jar:17.0.0 from the shaded jar.
[INFO] Excluding org.threeten:threeten-extra:jar:1.7.1 from the shaded jar.
[INFO] Excluding org.apache.orc:orc-mapreduce:jar:shaded-protobuf:1.8.3 from
the shaded jar.
[INFO] Excluding org.apache.hive:hive-storage-api:jar:2.8.1 from the shaded
jar.
[INFO] Excluding org.apache.parquet:parquet-column:jar:1.12.3 from the
shaded jar.
[INFO] Excluding org.apache.parquet:parquet-common:jar:1.12.3 from the
shaded jar.
[INFO] Excluding org.apache.parquet:parquet-encoding:jar:1.12.3 from the
shaded jar.
[INFO] Excluding org.apache.yetus:audience-annotations:jar:0.13.0 from the
shaded jar.
[INFO] Excluding org.apache.parquet:parquet-hadoop:jar:1.12.3 from the
shaded jar.
[INFO] Excluding org.apache.parquet:parquet-format-structures:jar:1.12.3
from the shaded jar.
[INFO] Excluding org.apache.parquet:parquet-jackson:jar:1.12.3 from the
shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.14.2 from
the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.14.2
from the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.14.2 from the
shaded jar.
[INFO] Excluding org.apache.avro:avro:jar:1.11.1 from the shaded jar.
[INFO] Excluding org.apache.avro:avro-mapred:jar:1.11.1 from the shaded jar.
[INFO] Excluding org.apache.avro:avro-ipc:jar:1.11.1 from the shaded jar.
[INFO] Excluding org.tukaani:xz:jar:1.9 from the shaded jar.
[INFO] Excluding org.apache.hadoop:hadoop-client-api:jar:3.3.5 from the
shaded jar.
[INFO] Excluding org.apache.hadoop:hadoop-client-runtime:jar:3.3.5 from the
shaded jar.
[INFO] Excluding commons-logging:commons-logging:jar:1.1.3 from the shaded
jar.
[INFO] Excluding javax.activation:activation:jar:1.1.1 from the shaded jar.
[INFO] Excluding org.apache.curator:curator-recipes:jar:2.13.0 from the
shaded jar.
[INFO] Excluding org.apache.curator:curator-framework:jar:2.13.0 from the
shaded jar.
[INFO] Excluding org.apache.curator:curator-client:jar:2.13.0 from the
shaded jar.
[INFO] Excluding org.apache.zookeeper:zookeeper:jar:3.6.3 from the shaded
jar.
[INFO] Excluding org.apache.zookeeper:zookeeper-jute:jar:3.6.3 from the
shaded jar.
[INFO] Excluding org.apache.commons:commons-compress:jar:1.22 from the
shaded jar.
[INFO] Excluding org.apache.commons:commons-lang3:jar:3.12.0 from the shaded
jar.
[INFO] Excluding com.google.code.findbugs:jsr305:jar:3.0.0 from the shaded
jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:2.0.7 from the shaded jar.
[INFO] Excluding org.apache.logging.log4j:log4j-slf4j2-impl:jar:2.20.0 from
the shaded jar.
[INFO] Excluding org.apache.logging.log4j:log4j-api:jar:2.20.0 from the
shaded jar.
[INFO] Excluding org.apache.logging.log4j:log4j-core:jar:2.20.0 from the
shaded jar.
[INFO] Excluding org.apache.logging.log4j:log4j-1.2-api:jar:2.20.0 from the
shaded jar.
[INFO] Excluding org.xerial.snappy:snappy-java:jar:1.1.9.1 from the shaded
jar.
[INFO] Excluding com.github.luben:zstd-jni:jar:1.5.4-2 from the shaded jar.
[INFO] Excluding org.scala-lang:scala-library:jar:2.12.17 from the shaded
jar.
[INFO] Excluding org.apache.spark:spark-tags_2.12:jar:3.5.0-SNAPSHOT from
the shaded jar.
[INFO] Excluding org.spark-project.spark:unused:jar:1.0.0 from the shaded
jar.
Downloading from aliyun-nexus:
https://maven.aliyun.com/nexus/content/groups/public/org/apache/hadoop/hadoop-client-api/3.3.5/hadoop-client-api-3.3.5-tests.jar
Downloading from aliyun-central:
https://maven.aliyun.com/repository/central/org/apache/hadoop/hadoop-client-api/3.3.5/hadoop-client-api-3.3.5-tests.jar
Downloading from gcs-maven-central-mirror:
https://maven-central.storage-download.googleapis.com/maven2/org/apache/hadoop/hadoop-client-api/3.3.5/hadoop-client-api-3.3.5-tests.jar
Downloading from apache.snapshots.https:
https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-client-api/3.3.5/hadoop-client-api-3.3.5-tests.jar
Downloading from repository.jboss.org:
https://repository.jboss.org/nexus/content/groups/public/org/apache/hadoop/hadoop-client-api/3.3.5/hadoop-client-api-3.3.5-tests.jar
Downloading from sonatype-releases:
https://oss.sonatype.org/content/repositories/releases/org/apache/hadoop/hadoop-client-api/3.3.5/hadoop-client-api-3.3.5-tests.jar
Downloading from jitpack.io:
https://jitpack.io/org/apache/hadoop/hadoop-client-api/3.3.5/hadoop-client-api-3.3.5-tests.jar
Downloading from ossrh:
https://oss.sonatype.org/content/repositories/snapshots/org/apache/hadoop/hadoop-client-api/3.3.5/hadoop-client-api-3.3.5-tests.jar
[WARNING] Could not get tests for
org.apache.hadoop:hadoop-client-api:jar:3.3.5:compile
Downloading from aliyun-nexus:
https://maven.aliyun.com/nexus/content/groups/public/org/tukaani/xz/1.9/xz-1.9-tests.jar
Downloading from aliyun-central:
https://maven.aliyun.com/repository/central/org/tukaani/xz/1.9/xz-1.9-tests.jar
Downloading from gcs-maven-central-mirror:
https://maven-central.storage-download.googleapis.com/maven2/org/tukaani/xz/1.9/xz-1.9-tests.jar
Downloading from apache.snapshots.https:
https://repository.apache.org/content/repositories/snapshots/org/tukaani/xz/1.9/xz-1.9-tests.jar
Downloading from repository.jboss.org:
https://repository.jboss.org/nexus/content/groups/public/org/tukaani/xz/1.9/xz-1.9-tests.jar
Downloading from sonatype-releases:
https://oss.sonatype.org/content/repositories/releases/org/tukaani/xz/1.9/xz-1.9-tests.jar
Downloading from jitpack.io:
https://jitpack.io/org/tukaani/xz/1.9/xz-1.9-tests.jar
Downloading from ossrh:
https://oss.sonatype.org/content/repositories/snapshots/org/tukaani/xz/1.9/xz-1.9-tests.jar
[WARNING] Could not get tests for org.tukaani:xz:jar:1.9:compile
[WARNING] Could not get tests for
com.google.protobuf:protobuf-java:jar:3.22.0:compile
Downloading from aliyun-nexus:
https://maven.aliyun.com/nexus/content/groups/public/org/apache/curator/curator-framework/2.13.0/curator-framework-2.13.0-tests.jar
Downloading from aliyun-central:
https://maven.aliyun.com/repository/central/org/apache/curator/curator-framework/2.13.0/curator-framework-2.13.0-tests.jar
Downloading from gcs-maven-central-mirror:
https://maven-central.storage-download.googleapis.com/maven2/org/apache/curator/curator-framework/2.13.0/curator-framework-2.13.0-tests.jar
Downloading from apache.snapshots.https:
https://repository.apache.org/content/repositories/snapshots/org/apache/curator/curator-framework/2.13.0/curator-framework-2.13.0-tests.jar
Downloading from repository.jboss.org:
https://repository.jboss.org/nexus/content/groups/public/org/apache/curator/curator-framework/2.13.0/curator-framework-2.13.0-tests.jar
Downloading from sonatype-releases:
https://oss.sonatype.org/content/repositories/releases/org/apache/curator/curator-framework/2.13.0/curator-framework-2.13.0-tests.jar
Downloading from jitpack.io:
https://jitpack.io/org/apache/curator/curator-framework/2.13.0/curator-framework-2.13.0-tests.jar
Downloading from ossrh:
https://oss.sonatype.org/content/repositories/snapshots/org/apache/curator/curator-framework/2.13.0/curator-framework-2.13.0-tests.jar
[WARNING] Could not get tests for
org.apache.curator:curator-framework:jar:2.13.0:compile
Downloading from aliyun-nexus:
https://maven.aliyun.com/nexus/content/groups/public/org/apache/logging/log4j/log4j-slf4j2-impl/2.20.0/log4j-slf4j2-impl-2.20.0-tests.jar
Downloading from aliyun-central:
https://maven.aliyun.com/repository/central/org/apache/logging/log4j/log4j-slf4j2-impl/2.20.0/log4j-slf4j2-impl-2.20.0-tests.jar
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
