vinothchandar opened a new pull request, #19193:
URL: https://github.com/apache/hudi/pull/19193
### Describe the issue this Pull Request addresses
`hudi-common` has accumulated classes in packages that no longer reflect
their role:
engine-facing
domain machinery sitting in the root `org.apache.hudi` package, shared
substrate under a
top-level
`internal.*` namespace, write-client config classes outside the common
config namespace, and
lock/validator/file-IO contracts under
`org.apache.hudi.client.*`/`org.apache.hudi.io.*` where
they
collide conceptually with other modules' namespaces.
This PR establishes a deliberate two-tier package taxonomy inside
`hudi-common`, without
changing
the module structure:
- **`org.apache.hudi.core.*`** — domain-aware code implementing core
read/write behavior
(depends on both `hudi-io` primitives and Hudi domain models).
- **`org.apache.hudi.common.*`** — shared substrate used across all
modules and engines
(type system, expressions, configs, utilities), with no domain awareness.
This pre-stages a future `hudi-common`/`hudi-core` module split so that
the eventual module
move is
FQN-stable, and removes one piece of dead code found along the way.
### Summary and Changelog
No behavior changes; this is a package reorganization (423 files,
overwhelmingly import-line
churn). One commit per relocation:
- `refactor(common)`: `InProcessLockProvider`, `NoopLockProvider` →
`o.a.h.core.transaction.lock`.
Because the old `InProcessLockProvider` FQN is a documented
`hoodie.write.lock.provider`
config
value resolved via reflection, a deprecated stub is kept at the old FQN,
marked with a new
`@CompatAlias(of, since)` annotation (added next to `@PublicAPIClass` in
`hudi-io`). The
three
sites that string-compare the configured provider class (clock-skew
inference, MDT
multi-writer
guard, single-writer auto-adjust) now accept both old and new FQNs via
one predicate.
- `refactor(common)!`: `BasePreCommitValidator`, `ValidationContext` →
`o.a.h.core.validator`.
- `refactor(common)!`: 7 metrics config classes →
`o.a.h.common.config.metrics`, joining
`HoodieCommonConfig`/`HoodieMetadataConfig`; avoids a split package with
`hudi-io`'s
`o.a.h.common.metrics`.
- `refactor(common)!`: 19 file I/O contract classes
(`HoodieFileReader/Writer` + factories,
`HoodieIOFactory`, HFile/bootstrap readers,
`HoodieParquetConfigInjector`) →
`o.a.h.core.io[.storage]`.
Engine implementations (`o.a.h.io.storage.hadoop.*`, Spark/Flink
packages) do not move.
- `refactor(common)!`: removed `ByteBufferBackedInputFile` and its
`o.a.h.parquet.io` package —
zero references repo-wide (orphaned by an earlier parquet log-block
refactor).
- `refactor(common)!`: `ParquetAdapter` → `o.a.h.stats`, next to its only
consumers.
- `refactor(common)!`: `BaseHoodieTableFileIndex` → `o.a.h.core.read`.
- `refactor(common)!`: `expression` package → `o.a.h.common.expression`.
- `refactor(common)!`: `internal.schema` (schema evolution type system, 21
classes) →
`o.a.h.common.schema.internal`, nested under the existing
`o.a.h.common.schema`.
- `fix(trino)`: `hudi-trino-plugin` was excluded from the sweep — it
compiles against released
Hudi 1.0.2, not the in-repo modules; its imports update when
`dep.hudi.version` is bumped.
The moves surfaced several hidden same-package couplings, now made
explicit: spark-common scala
classes importing the file-index base explicitly instead of bare
same-package references;
`HoodieBaseRelation` using the `metadataConfig` getter instead of
cross-package protected field
access; two read-only file-index introspection methods made public; a
validator test accessing
protected state through an accessor on its mock. No code was copied.
### Impact
No config keys, defaults, or semantics change. No storage-format impact:
none of the moved FQNs
are persisted (`hoodie.properties`, commit metadata, and serialized
schemas carry no Java class
names), verified per move.
Breaking (source/binary) for externally compiled code only, always failing
loudly
(`NoClassDefFoundError`/compile error), never silently:
- Custom pre-commit validators extending `BasePreCommitValidator`
(EVOLVING API, shipped only
in
1.2.0; the `hoodie.precommit.validators` config names the user's own
class, which is
unchanged).
- Programmatic construction of metrics configs (`withMetricsConfig(..)`
builders, `.key()`
constants). Property-based metrics configuration is unaffected; reporter
selection is
enum-based.
- Custom `HoodieIOFactory` subclasses / `HoodieParquetConfigInjector`
implementations (both
EVOLVING-era plug points; the config values name the user's or shipped
implementation class,
unchanged — e.g. default
`org.apache.hudi.io.storage.hadoop.HoodieHadoopIOFactory`).
- Direct imports of `BaseHoodieTableFileIndex` (engine subclass FQNs
`HiveHoodieTableFileIndex`/
`SparkHoodieTableFileIndex` are unchanged and bundles are
self-consistent, so existing
Presto/Trino deployments are unaffected at runtime), Hudi `Expression`
construction, and
`InternalSchema`/`SerDeHelper` usage in custom integrations.
Users configuring Hudi via `hoodie.*` properties see zero impact; existing
configs carrying the
old `InProcessLockProvider` FQN keep working via the compatibility stub.
No performance impact.
### Risk Level
low — mechanical relocations with compatibility handling where user
configs are involved.
Verification: full 52-module reactor build green (`-Dspark4.1 -Dflink2.1`,
checkstyle/scalastyle
enforced); ~330 targeted unit tests across all moved areas (lock provider
incl. reflective load
of
the legacy FQN, validators across client-common/utilities/flink, metrics
configs/reporters,
HFile
reader/writer factories, file index, expressions, full schema-evolution
suites); functional
verification via `TestHoodieClientMultiWriter` (47 tests, concurrent
writers through
LockManager)
and transaction/lock-manager/time-generator suites covering every
reflection path touched.
### Documentation Update
none — no config keys or defaults change. The config reference regenerates
class-location
mentions
automatically (doc strings referencing moved FQNs were updated in-repo).
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
