CryoThrust opened a new pull request, #13264:
URL: https://github.com/apache/seatunnel/pull/13264
### Purpose of this pull request
This contribution to Apache SeaTunnel fixes #12766. Loading Hadoop XML
currently does not log in from a keytab, so the Hudi catalog and writers can
access a Kerberos-protected HDFS without an authenticated identity.
Add the Hadoop-family options `kerberos_principal`, `kerberos_keytab_path`
and `krb5_path` to both sink and catalog factories. Both credential options
must be nonblank; the supplied Hadoop configuration must enable Kerberos. With
neither credential option configured, the existing execution path is retained.
Each catalog and direct/cached write-client provider owns a runtime UGI
returned by `loginUserFromKeytabAndReturnUGI`. Initialization, catalog I/O,
writes/deletes, automatic commits and cleanup use that identity. Check ticket
renewal before operations; still attempt cleanup if renewal fails. Close the
identity-owned filesystem cache and log out after client shutdown. Neither UGI
nor its executor is serialized with the provider.
Hudi 0.15's Java engine and `HoodieListData` use parallel streams, so
wrapping only the submitting thread in `doAs` is insufficient when the common
pool already exists. Authenticated operations run in a private ForkJoinPool
whose worker lifetime is scoped to the returned UGI. Hudi's queue executor
creates ordinary child threads within that scope. The existing configuration
disables asynchronous cleaning and the embedded timeline server.
The unused global-login helper remains uncalled. Hadoop security-mode
initialization and the optional krb5 system property follow the synchronized
HadoopLoginFactory precedent; those settings are still JVM-wide. The EN/ZH
documentation describes this shared-worker constraint and a real-KDC
verification recipe.
### Does this PR introduce any user-facing change?
Yes. Hudi can explicitly authenticate with the three canonical options
above. Partial/blank credentials and credentials combined with a non-Kerberos
Hadoop configuration fail before filesystem/client initialization. The dotted
JDBC options are not introduced as Hudi options. The authentication helper
omits raw credential-bearing causes, and HudiSinkConfig.toString excludes the
credential fields.
No dependencies or default write/commit settings change. Open PR #12440
changes the Hudi commit model and is not included here; whichever merges second
will need to cover its aggregated-committer path too.
### How was this patch tested?
- All 34 connector unit tests pass on both JDK 11 and 17. They cover option
declaration/config text, conditional authentication, login-error sanitization,
identity isolation, renewal failure and cleanup, an already initialized common
pool, actual Hudi Java-engine/ListData parallel work, and Hudi child threads.
- The direct and cached providers are exercised through HudiRecordWriter for
client creation, upsert, delete/startCommit and close with identity assertions.
Catalog tests exercise local filesystem initialization, database/table
creation, listing, lookup and deletion with a returned UGI.
- A real local Hudi client writes, automatically commits and deletes under a
returned test UGI. The KDC login itself is stubbed; this is not a Kerberos/HDFS
deployment test.
- The existing standalone local-write fixture disables its embedded timeline
server, avoiding dependence on a routable advertised host address.
- Spotless and the connector's eight-module dependency reactor pass
skip-tests verification, including Checkstyle/license checks.
- The new negative case in the existing HudiIT passes on a real Zeta
container: credentials with a non-Kerberos Hadoop configuration reject the job
with the expected configuration diagnostic (1 test, no failures/errors/skips).
The selected Zeta case uses direct test goals after building prerequisites,
avoiding unrelated Spark 3.5 SDK staging.
Portable connector commands (JDK 11):
```bash
./mvnw -pl seatunnel-connectors-v2/connector-hudi spotless:apply
./mvnw -pl seatunnel-connectors-v2/connector-hudi test
./mvnw -pl seatunnel-connectors-v2/connector-hudi -am -DskipTests verify
RUN_ENGINES=zeta ./mvnw -pl
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-hudi-e2e \
resources:testResources compiler:testCompile failsafe:integration-test
failsafe:verify \
-DskipIT=false
-Dit.test=HudiIT#testRejectKerberosCredentialsWithoutSecurityConfiguration
```
No real KDC login, Kerberized HDFS, live ticket-expiry/renewal or full
multi-engine suite was run. These remain deployment validation, not inferred
from the credential-free tests.
### Check list
- [x] EN/ZH connector documentation updated.
- [x] Unit tests and an existing E2E suite updated.
- [x] No new dependency or binary package.
- [x] No new connector registration or distribution entry required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]