loustler commented on issue #11655:
URL: https://github.com/apache/seatunnel/issues/11655#issuecomment-5229058293
@SEZ9 thanks — everything you asked for here already exists, in #11673,
which you reviewed on 2026-08-08 and confirmed resolves both of your findings.
Pointers so this issue isn't a dead end for anyone reading it later:
**The shade plugin config for `seatunnel-hadoop-aws`** — the "before" state
was that `seatunnel-shade/seatunnel-hadoop-aws/pom.xml` had *no* Jackson
relocation at all, while `seatunnel-hadoop3-3.1.4-uber` relocates
`com.fasterxml.jackson` into
`${seatunnel.shade.package}.hadoop.com.fasterxml.jackson`. That mismatch is the
whole bug. #11673 adds the matching relocation to `seatunnel-hadoop-aws`, using
the uber jar's shaded pattern rather than a new one, so the two agree by
construction.
**The unrelocated Jackson class name in the trace** — the PR body carries
the `javap` before/after on `JsonSerialization`, which is more precise than the
stack trace: the failing call site resolves
`org/apache/seatunnel/shade/hadoop/com/fasterxml/jackson/databind/ObjectMapper`
in the uber jar but `com/fasterxml/jackson/databind/ObjectMapper` in
`hadoop-aws`, so the JVM cannot match the method by name + descriptor and
throws `NoSuchMethodError` rather than `NoClassDefFoundError`. On hadoop-aws
3.1.4 that is 3 of the 194 classes under `org/apache/hadoop/` — `RoleModel`,
`RoleModel$Policy`, `RoleModel$Statement` — and the same 194 classes come back
clean once the relocation is applied.
**On the two options you listed** — "include hadoop-aws in the shading" is
the one #11673 takes. "Stop relocating Jackson in the uber jar" was considered
and rejected: the uber jar's relocation is load-bearing for consumers that
already depend on it, and unwinding it is a much wider blast radius than making
one module consistent with it.
**On your regression-test point** — agreed, and it is in the PR rather than
left as a follow-up. `tools/dependencies/check_shaded_jackson_refs.py` scans
the built jar's `org/apache/hadoop/` classes for unrelocated
`com/fasterxml/jackson` references and is wired into the `dependency-license`
CI job. Two things about it worth knowing, both found the hard way: it has to
run *before* `checkLicense.sh` (which starts with `mvnw clean` and would wipe
the jar), and it fails loudly when it scans zero classes, so a build that
silently produces nothing can't pass vacuously.
Also worth flagging for scope, since it isn't in the original report:
`RoleModel` is the loud symptom, but on hadoop-aws **3.4.3** the same check
flags 6 of 466 classes, not 3. The extra three are S3A committer classes whose
references are `@JsonProperty` *annotations* rather than method calls — Jackson
looks for its own relocated annotations, doesn't find them, and so does not
throw at all; it silently falls back to field names. That failure mode produces
no stack trace to search for, which is why the guard is a static jar check
rather than a runtime test. Detail is in the PR body.
Your scope note is right for the throwing case, incidentally: only S3A
configurations exercising `RoleModel` (`fs.s3a.assumed.role.*`) hit the
`NoSuchMethodError`; access-key and instance-profile credentials do not. The
silent 3.4.3 case is not bounded by that condition.
Happy to answer anything else here, but I think this issue is fully covered
by #11673 at this point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
