loustler commented on PR #11648:
URL: https://github.com/apache/seatunnel/pull/11648#issuecomment-5191461062
Both blockers are fixed in `128b94d85`. I also pulled the raw logs for all
eight red jobs on `fdff4b65c` and attributed each one, which turned up two more
instances of the same bug family that the audit hadn't reached.
## Issue 1 — `connector-iceberg` on `commons-collections4`
`IcebergCatalog.java:42` and `ExpressionUtils.java:22` now import
`org.apache.commons.collections4.CollectionUtils`, and
`connector-iceberg/pom.xml` declares `org.apache.commons:commons-collections4`
explicitly with a comment in the same style as `imap-storage-file`. Both call
sites pass a `List` to `isNotEmpty`, so the semantics are unchanged.
One correction to the mechanism, since it changes what the fix has to be.
The 3.x class did not reach the compile classpath through the uber jar — it
comes from `hive-exec:2.3.9:core`, which this module declares `provided`:
```
connector-iceberg
\- org.apache.hive:hive-exec:jar:core:2.3.9:provided
\- commons-collections:commons-collections:jar:3.2.2
```
`provided` means it is never shaded into the connector jar, and it is still
there after the bump. So the compile classpath looked identical before and
after — the build could not have caught this. What disappeared is the *runtime*
provider: `org/apache/commons/collections/**` used to be bundled inside the
uber jar and is not at 3.4.3. Declaring `commons-collections4` at compile scope
puts the class inside the connector jar rather than leaving it to whatever
another artifact happens to bundle.
## The four other `commons-collections` 3.x usages
You asked for an explicit statement rather than an assumption, so I resolved
each module's classpath and checked which jar actually supplies the class:
| file | 3.x supplier | reachable from SeaTunnel's shaded classpath? |
|---|---|---|
| `seatunnel-flink-starter-common/.../TransformExecuteProcessor.java` |
`commons-collections:3.2.2:provided`, via `flink-java` → `flink-core` | **No**
— Flink's own dependency, and at runtime `flink-dist` supplies it |
| `seatunnel-spark-starter-common/.../TransformExecuteProcessor.java` |
`commons-collections:3.2.2:provided`, via `spark-core_2.12:3.3.0` | **No** —
same, from the Spark installation |
| `connector-paimon-e2e/PaimonIT.java` | `connector-paimon.jar` itself
carries an unrelocated `org/apache/commons/collections/CollectionUtils.class` |
**No** — resolves from the connector jar, not from Hadoop |
| `connector-paimon-e2e/PaimonWithS3IT.java` | same | **No** |
Neither starter depends on the uber jar for it, so this PR's classpath
change does not reach them. That is measured, not inferred —
`commons-collections-3.2.2.jar` is on both starters' resolved classpaths at
`provided` scope regardless of the Hadoop version.
## Issue 2 — `hadoop-minikdc` paired
Your Kerby reasoning was right, and the poms confirm it: `hadoop-minikdc`
3.1.4 pins `kerb-simplekdc` **1.0.1**, while 3.4.3 pins `${kerby.version}` =
**2.0.3**. A Kerby major-version gap between the KDC issuing the ticket and the
`UserGroupInformation` reading it is exactly the `Asn1Exception: Identifier
doesn't match expected value`.
`hadoop-minikdc.version` now lives in the root pom at 3.4.3, next to
`hadoop-aws.version` and `hadoop-aliyun.version`, with the coupling written
down; `connector-file-base` inherits it. `HadoopLoginFactoryTest` locally:
**Tests run: 4, Failures: 0, Errors: 0** (CI had 4 run / 2 errors).
## Two more of the same family, from the remaining red jobs
`paimon-connector-it` and `all-connectors-it-5` both fail with the break
this PR exists to fix:
```
java.lang.NoSuchMethodError:
org.apache.hadoop.util.SemaphoredDelegatingExecutor
.<init>(Lcom/google/common/util/concurrent/ListeningExecutorService;IZ)V
at org.apache.hadoop.fs.s3a.S3AFileSystem.create(S3AFileSystem.java:769)
at org.apache.paimon.s3.HadoopCompliantFileIO.newOutputStream(...)
```
`PaimonWithS3IT` (4/4 failed) and `HudiSeatunnelS3MultiTableIT` (1/1 failed)
each `wget` `hadoop-aws` 3.1.4 and the AWS SDK v1 bundle from Maven Central
into the container's `lib/`, so an S3A compiled against the removed guava-typed
constructor ran against `hadoop-common` 3.4.3. This is the same defect I fixed
in `connector-file-s3-e2e`; I had not swept the other e2e modules for it. Both
now copy the locally built `seatunnel-hadoop-aws.jar` into `lib/`, the same
mechanism `SeaTunnelContainer` already uses for the uber jar, which also
removes four network downloads.
Two modules with the same pattern I deliberately did **not** change, rather
than change blind:
- `connector-iceberg-s3-e2e/IcebergSourceIT` — injects `hadoop-aws` 3.1.4
too, but it is `@DisabledOnContainer` for the `SEATUNNEL` and `FLINK` engines,
so it only runs on Spark, where Hadoop comes from the Spark distribution rather
than the uber jar. Not proven broken, and I cannot verify a change there.
- `connector-hive-e2e` (`HiveIT`, `HiveKerberosIT`, `HiveOverwriteIT`) —
wgets `hadoop-aws` 3.1.4, `hadoop-aliyun` 3.1.4, `aliyun-sdk-oss` 3.4.1 and
`hadoop-cos` into `plugins/Hive/lib`. Only the four
`testFakeSinkHiveOn{HDFS,S3,OSS,Cos}` tests use that directory and all four are
`@Disabled`, so nothing there is verifiable.
## The remaining three red jobs
- **`all-connectors-it-3`** — confirmed as you described:
`PostgresCDCIT.testPostgresCdcSnapshotOnlyAndCommittedOffsetStartupModes`,
`ConditionTimeoutException`, no Hadoop/AWS/S3A in the failing path. Unrelated
to this diff.
- **`kafka-connector-it`** — `NoClassDefFoundError:
org/apache/kafka/common/network/Selector$CloseMode`, thrown from
`Utils.closeQuietly` during admin-client shutdown inside the Flink 1.13.6
container. This job is **also red on `dev`**: run `30981832352` (`befd31c45`)
fails `kafka-connector-it`. Pre-existing.
- **`engine-k8s-it`** — `KubernetesIT` fails in 0.005 s and 0.134 s with
`NoClassDefFoundError: Could not initialize class
okhttp3.internal.concurrent.TaskRunner` and `NoSuchFieldError: Companion`, i.e.
a Kotlin/okio ABI mismatch in the test JVM before any container work. **I could
not attribute this to the diff and am not going to claim it is unrelated.**
What I checked: the only change to that module is the uber jar path rename; the
3.4.3 uber jar contains **0** entries under `okhttp3/`, `kotlin/`, `okio/` and
`com/squareup/`; `okhttp3:4.12.0` with `kotlin-stdlib-jdk8:1.8.21` is a
pre-existing direct dependency of `seatunnel-engine-k8s-e2e`, unchanged here;
and the `com.squareup.okhttp:okhttp` 2.7.5 I added to `seatunnel-engine-server`
does not propagate to that module, so its `okio` 1.6.0 never meets `okio` 3.6.0
there. It did not fail on either `dev` run I checked, so I would like a rerun
to see whether it reproduces.
## Attribution of all eight jobs
| job | failure | cause | status |
|---|---|---|---|
| `all-connectors-it-8` | `IcebergSourceIT`, `JdbcToIcebergTimestampIT`, 32×
`NoClassDefFoundError` | Issue 1 | fixed |
| `unit-test (8)` | `HadoopLoginFactoryTest`, 2 errors | Issue 2 | fixed,
4/4 locally |
| `edge-agent-it (11)` | `HadoopLoginFactoryTest`, 2 errors | Issue 2 |
fixed |
| `paimon-connector-it` | `PaimonWithS3IT` 4/4, `NoSuchMethodError` |
injected `hadoop-aws` 3.1.4 | fixed |
| `all-connectors-it-5` | `HudiSeatunnelS3MultiTableIT`, same | injected
`hadoop-aws` 3.1.4 | fixed |
| `all-connectors-it-3` | `PostgresCDCIT`, timeout | unrelated | rerun |
| `kafka-connector-it` | `Selector$CloseMode` | also red on `dev` |
pre-existing |
| `engine-k8s-it` | `KubernetesIT`, okhttp3/Kotlin | unattributed | rerun |
On `@Disabled`/coverage, per your note in 2.2: I did not touch
`IcebergSourceIT`, `JdbcToIcebergTimestampIT` or `HadoopLoginFactoryTest`, so
nothing those tests assert has been narrowed — the only test files in this
commit are the two S3 harnesses, and there only the jar-provisioning override
changed.
## Gates on `128b94d85`
Full reactor `install -DskipTests` **BUILD SUCCESS, 284/284** (13:28, JDK
8); `ExpressionUtilsTest` + `IcebergCatalogTest` + `PreviewActionTest`
**16/16**; `HadoopLoginFactoryTest` **4/4**;
`tools/dependencies/checkLicense.sh` **exit 0, empty diff**
(`commons-collections4-4.4.jar` was already in `known-dependencies.txt` and
`hadoop-minikdc` is `test` scope, so neither adds an entry); root
`spotless:check` **exit 0**, with `spotless:apply` touching no file outside the
seven in this commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
