kevinjqliu opened a new pull request, #18899: URL: https://github.com/apache/iceberg/pull/18899
Instead of pulling in all three shaded cloud bundle fat JARs (`:iceberg-aws-bundle`, `:iceberg-azure-bundle`, `:iceberg-gcp-bundle`), `:iceberg-bigquery`, and JUnit runtime JARs into `:iceberg-open-api:stageRESTFixtureRuntime`, this PR replaces them with only the direct runtime SDKs needed by the 3 cloud `FileIO` implementations (`S3FileIO`, `ADLSFileIO`, `GCSFileIO`) and moves `libs.junit.jupiter` to `testFixturesCompileOnly`. For context, Docker publishing was [paused over `LICENSE`/`NOTICE` issues](https://github.com/apache/iceberg/pull/16163#issuecomment-4348194370) (see #18421 and #18877). This is the second step (following #16214) of the [plan to fix `iceberg-rest-fixture` packaging and licensing](https://github.com/apache/iceberg/pull/16163#issuecomment-4356143385). Follow-up PRs will reconcile `open-api/LICENSE` and `open-api/NOTICE` against the staged runtime JARs. #### What goes into the Docker Image `:iceberg-open-api:stageRESTFixtureRuntime` output is reduced from **273 JARs (348 MB)** down to **219 JARs (133 MB)**: - 9 Iceberg JARs: `iceberg-api`, `iceberg-common`, `iceberg-core`, `iceberg-core-tests`, `iceberg-aws`, `iceberg-azure`, `iceberg-gcp`, `iceberg-bundled-guava`, `iceberg-open-api-test-fixtures` (drops `iceberg-aws-bundle`, `iceberg-azure-bundle`, `iceberg-gcp-bundle`, and `iceberg-bigquery`) - 210 third-party runtime & transitive JARs: AWS SDK v2 (`s3`, `kms`, `sts`, `apache5-client`, `url-connection-client`), Azure SDK (`azure-storage-file-datalake`, `azure-identity`) + Reactor/Netty, Google Cloud SDK (`google-cloud-storage`) + gRPC/Protobuf/OpenTelemetry, Hadoop 3.4.3, Jetty 12.1.14, Jackson 2.22.4, SQLite JDBC, SLF4J, Commons, Avro, Caffeine, RoaringBitmap, etc. <details> <summary>Full list of 219 staged JARs</summary> ```text RoaringBitmap-1.6.29.jar aircompressor-2.0.3.jar animal-sniffer-annotations-1.27.jar annotations-2.57.8.jar annotations-4.1.1.4.jar apache5-client-2.57.8.jar api-common-2.71.0.jar arns-2.57.8.jar auth-2.57.8.jar auto-value-annotations-1.11.0.jar avro-1.12.2.jar aws-core-2.57.8.jar aws-json-protocol-2.57.8.jar aws-query-protocol-2.57.8.jar aws-xml-protocol-2.57.8.jar azure-core-1.60.0.jar azure-core-http-netty-1.16.8.jar azure-identity-1.18.7.jar azure-json-1.5.1.jar azure-storage-blob-12.36.0.jar azure-storage-common-12.35.0.jar azure-storage-file-datalake-12.29.0.jar azure-storage-internal-avro-12.21.0.jar azure-xml-1.2.1.jar bcprov-jdk18on-1.86.jar caffeine-3.3.1.jar checker-qual-3.49.0.jar checksums-2.57.8.jar checksums-spi-2.57.8.jar common-0.13.0.jar commons-codec-1.19.0.jar commons-collections4-4.4.jar commons-compress-1.28.0.jar commons-configuration2-2.10.1.jar commons-io-2.20.0.jar commons-lang3-3.18.0.jar commons-logging-1.3.0.jar commons-math3-3.6.1.jar commons-net-3.9.0.jar commons-text-1.14.0.jar conscrypt-openjdk-uber-2.6.2.jar crt-core-2.57.8.jar detector-resources-support-0.31.0.jar endpoints-spi-2.57.8.jar error_prone_annotations-2.50.0.jar eventstream-2.57.8.jar exporter-metrics-0.33.0.jar failsafe-3.3.2.jar failureaccess-1.0.3.jar gapic-google-cloud-storage-v2-2.76.0.jar gax-2.88.0.jar gax-grpc-2.88.0.jar gax-httpjson-2.88.0.jar google-api-client-2.9.2.jar google-api-services-storage-v1-rev20260524-2.0.0.jar google-auth-library-credentials-1.55.0.jar google-auth-library-oauth2-http-1.55.0.jar google-cloud-core-2.78.0.jar google-cloud-core-grpc-2.78.0.jar google-cloud-core-http-2.78.0.jar google-cloud-monitoring-3.101.0.jar google-cloud-storage-2.76.0.jar google-http-client-2.2.0.jar google-http-client-apache-v2-2.2.0.jar google-http-client-appengine-2.2.0.jar google-http-client-gson-2.2.0.jar google-http-client-jackson2-2.2.0.jar google-oauth-client-1.39.0.jar grpc-alts-1.83.0.jar grpc-api-1.83.0.jar grpc-auth-1.83.0.jar grpc-context-1.83.0.jar grpc-core-1.83.0.jar grpc-google-cloud-storage-v2-2.76.0.jar grpc-googleapis-1.83.0.jar grpc-grpclb-1.83.0.jar grpc-inprocess-1.83.0.jar grpc-netty-shaded-1.83.0.jar grpc-opentelemetry-1.83.0.jar grpc-protobuf-1.83.0.jar grpc-protobuf-lite-1.83.0.jar grpc-rls-1.83.0.jar grpc-services-1.83.0.jar grpc-stub-1.83.0.jar grpc-util-1.83.0.jar grpc-xds-1.83.0.jar gson-2.14.0.jar guava-33.7.2-jre.jar hadoop-annotations-3.4.3.jar hadoop-auth-3.4.3.jar hadoop-common-3.4.3.jar hadoop-shaded-guava-1.5.0.jar hadoop-shaded-protobuf_3_25-1.5.0.jar http-auth-2.57.8.jar http-auth-aws-2.57.8.jar http-auth-aws-eventstream-2.57.8.jar http-auth-spi-2.57.8.jar http-client-spi-2.57.8.jar httpclient-4.5.14.jar httpclient5-5.6.5.jar httpcore-4.4.16.jar httpcore5-5.4.4.jar httpcore5-h2-5.4.4.jar iceberg-api-1.13.0-SNAPSHOT.jar iceberg-aws-1.13.0-SNAPSHOT.jar iceberg-azure-1.13.0-SNAPSHOT.jar iceberg-bundled-guava-1.13.0-SNAPSHOT.jar iceberg-common-1.13.0-SNAPSHOT.jar iceberg-core-1.13.0-SNAPSHOT-tests.jar iceberg-core-1.13.0-SNAPSHOT.jar iceberg-gcp-1.13.0-SNAPSHOT.jar iceberg-open-api-1.13.0-SNAPSHOT-test-fixtures.jar identity-spi-2.57.8.jar j2objc-annotations-3.1.jar jackson-annotations-2.22.jar jackson-core-2.22.4.jar jackson-databind-2.22.4.jar jackson-dataformat-xml-2.22.4.jar jackson-datatype-jsr310-2.22.4.jar jakarta.activation-api-1.2.1.jar jakarta.servlet-api-6.0.0.jar javax.servlet-api-3.1.0.jar jettison-1.5.4.jar jetty-compression-common-12.1.14.jar jetty-compression-gzip-12.1.14.jar jetty-compression-server-12.1.14.jar jetty-ee10-servlet-12.1.14.jar jetty-http-12.1.14.jar jetty-io-12.1.14.jar jetty-security-12.1.14.jar jetty-server-12.1.14.jar jetty-session-12.1.14.jar jetty-util-12.1.14.jar jna-5.17.0.jar jna-platform-5.17.0.jar json-utils-2.57.8.jar jsp-api-2.1.jar jspecify-1.0.1.jar jsr305-3.0.2.jar kms-2.57.8.jar listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar metrics-spi-2.57.8.jar msal4j-1.26.0.jar msal4j-persistence-extension-1.3.0.jar netty-buffer-4.2.19.Final.jar netty-codec-4.2.19.Final.jar netty-codec-base-4.2.19.Final.jar netty-codec-compression-4.2.19.Final.jar netty-codec-dns-4.2.19.Final.jar netty-codec-http-4.2.19.Final.jar netty-codec-http2-4.2.19.Final.jar netty-codec-marshalling-4.2.19.Final.jar netty-codec-protobuf-4.2.19.Final.jar netty-codec-socks-4.2.19.Final.jar netty-common-4.2.19.Final.jar netty-handler-4.2.19.Final.jar netty-handler-proxy-4.2.19.Final.jar netty-nio-client-2.57.8.jar netty-resolver-4.2.19.Final.jar netty-resolver-dns-4.2.19.Final.jar netty-resolver-dns-classes-macos-4.2.19.Final.jar netty-resolver-dns-native-macos-4.2.19.Final-osx-x86_64.jar netty-tcnative-boringssl-static-2.0.84.Final.jar netty-tcnative-classes-2.0.84.Final.jar netty-transport-4.2.19.Final.jar netty-transport-classes-epoll-4.2.19.Final.jar netty-transport-classes-kqueue-4.2.19.Final.jar netty-transport-native-epoll-4.2.19.Final-linux-x86_64.jar netty-transport-native-epoll-4.2.19.Final.jar netty-transport-native-kqueue-4.2.19.Final-osx-x86_64.jar netty-transport-native-unix-common-4.2.19.Final.jar nimbus-jose-jwt-10.4.jar opencensus-api-0.31.1.jar opencensus-contrib-http-util-0.31.1.jar opentelemetry-api-1.62.0.jar opentelemetry-common-1.62.0.jar opentelemetry-context-1.62.0.jar opentelemetry-gcp-resources-1.37.0-alpha.jar opentelemetry-sdk-1.62.0.jar opentelemetry-sdk-common-1.62.0.jar opentelemetry-sdk-extension-autoconfigure-spi-1.62.0.jar opentelemetry-sdk-logs-1.62.0.jar opentelemetry-sdk-metrics-1.62.0.jar opentelemetry-sdk-trace-1.62.0.jar opentelemetry-semconv-1.29.0-alpha.jar perfmark-api-0.27.0.jar profiles-2.57.8.jar proto-google-cloud-monitoring-v3-3.101.0.jar proto-google-cloud-storage-v2-2.76.0.jar proto-google-common-protos-2.79.0.jar proto-google-iam-v1-1.74.0.jar protobuf-0.13.0.jar protobuf-java-4.33.6.jar protobuf-java-util-4.33.6.jar protocol-core-2.57.8.jar re2j-1.8.jar reactive-streams-1.0.4.jar reactor-core-3.7.19.jar reactor-netty-core-1.2.18.jar reactor-netty-http-1.2.18.jar regions-2.57.8.jar retries-2.57.8.jar retries-spi-2.57.8.jar runtime-0.13.0.jar s3-2.57.8.jar sdk-core-2.57.8.jar shared-resourcemapping-0.33.0.jar slf4j-api-2.0.18.jar slf4j-simple-2.0.18.jar sqlite-jdbc-3.54.1.0.jar stax2-api-4.3.0.jar sts-2.57.8.jar third-party-jackson-core-2.57.8.jar threeten-extra-1.8.0.jar threetenbp-1.7.0.jar url-connection-client-2.57.8.jar utils-2.57.8.jar utils-lite-2.57.8.jar woodstox-core-7.2.0.jar ``` </details> --- **AI Disclosure** - Model: Gemini 4 Argon (`gemini-4-argon`) - Platform/Tool: GitHub Copilot - Human Oversight: [unknown - human to fill in] - Prompt Summary: Prune `:iceberg-open-api` REST fixture runtime dependencies to the 3 cloud `FileIO` SDKs for Step 2 of the REST fixture Docker packaging plan -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
