slachiewicz commented on PR #897: URL: https://github.com/apache/incubator-xtable/pull/897#issuecomment-5381652374
Tested this against live S3, live ADLS Gen2 and live GCS. No regression found. Differential: baseline jar at ef88e02 (the commit under this PR) versus this PR's jar, identical configs and identical cloud paths. EC2 `m6i.xlarge` in eu-north-1, Corretto 11, AWS credentials from an instance profile so no static key was involved. Sizes confirm the table in the description exactly: baseline 1,097,347,589 bytes, this PR 423,222,450. **ServiceLoader diff** — the #736 failure shape. 131 of 137 service files are byte-identical. Exactly six differ, and all six are intended drops: ``` io.grpc.xds.XdsCredentialsProvider javax.script.ScriptEngineFactory org.codehaus.groovy.plugins.Runners org.codehaus.groovy.runtime.ExtensionModule org.codehaus.groovy.source.Extensions org.codehaus.groovy.transform.ASTTransformation ``` No AWS SDK registration was lost. `javax.script.ScriptEngineFactory` is the Groovy JSR-223 engine leaving with `groovy-all`. **AWS, real bucket:** | run | this PR | baseline | |---|---|---| | RunSync DELTA→ICEBERG over `s3a://`, shipped default creds | pass | pass | | RunSync over `s3a://`, assume-role config | pass | not run | | write+read, `DefaultCredentialsProvider` | pass | — | | write+read, AWS SDK provider named by class in config | pass | — | | write+read, `AssumedRoleCredentialProvider` | pass | pass | | 9 MB upload at 5 MB multipart threshold | pass | — | `-verbose:class` shows the swapped artifacts are genuinely reached, not merely present: `sts` 195 classes including `DefaultStsClient`, `AssumeRoleRequest` and `s3a.auth.STSClientFactory`; `s3-transfer-manager` 73; `netty-nio-client` 194. `software.amazon.awssdk.crt` loads 0 classes, matching your note that CRT was never shipped. **Azure**, real ADLS Gen2 with hierarchical namespace, through the shipped `OAuth`/`ClientCredsTokenProvider` default with a scoped service principal: ABFS list/mkdir/write/read pass, and RunSync DELTA→ICEBERG over `abfss://` passes on both jars. **GCS**, real bucket, service-account JSON: RunSync DELTA→ICEBERG over `gs://` passes on both jars. Worth noting for anyone reasoning about blast radius — a GCS operation loads 0 `software.amazon.awssdk` classes, so the AWS artifact swap cannot reach that path. ### Re-tested rebased onto #906 The first pass needed a workaround: the bundled jar cannot read Delta sources at all, because shade had no `ServicesResourceTransformer` and Delta's `DataSourceRegister` entry was being overwritten. That is pre-existing and unrelated to this PR — the service file is byte-identical in both jars — and is now #905, fixed by #906. I rebased this branch onto #906 and re-ran everything with no workaround. Clean rebase, no conflicts; the two PRs touch different sections of `xtable-utilities/pom.xml`. Combined jar is 423,224,546 bytes, so the size win survives at a cost of about 2 KB of merged service files. RunSync over `s3a://` and with assume-role both pass, the credential matrix passes against a config that no longer needs `fs.file.impl`, and 0 of 428 ServiceLoader registrations dangle. One interaction worth flagging for reviewers. With #906 merging service files, this PR's `grpc-xds` removal becomes visible where it previously was not: `io.grpc.LoadBalancerProvider` and `io.grpc.NameResolverProvider` lose their 11 `io.grpc.xds.*` entries. Before #906 those entries were being clobbered anyway, so nothing changed observably. Classes and registrations go together here — 4,805 `io/grpc/xds` classes to 0, 1,398 groovy to 0 — so nothing is left dangling, and the DNS, grpclb, pick-first, round-robin, RLS and GoogleCloudToProd resolvers all remain. Only `xds://` targets are affected, which is the stated intent. ### What this does not settle The gap in the description is narrowed, not closed. Three credential providers named by class resolved, including an AWS SDK class named directly in configuration. What no amount of testing here can enumerate is an arbitrary provider or signer some deployment names that falls outside the whitelist. Also untested: Hudi in either direction, `RunCatalogSync`, and whether the RocksDB natives this PR trims are reachable from `RunSync` at all — nothing I ran loaded RocksDB. *This comment was created with AI assistance.* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
