fhan688 opened a new pull request, #20147:
URL: https://github.com/apache/hudi/pull/20147
### Describe the issue this Pull Request addresses
`EmbeddedTimelineService` tracks active table paths using a `Set<String>`.
When multiple clients in the same JVM independently acquire the same reusable
service for the same table, their acquisitions collapse into a single entry.
Closing one client removes that entry and can shut down the service while
another client still depends on it. If other tables keep the service alive, the
unconditional `unregisterBasePath()` call still prematurely clears the affected
table's file-system view.
The shutdown path also checks and closes the server outside the lifecycle
lock, allowing competing releases to attempt duplicate closure. Removing a
registry entry by identifier alone can evict a different service instance with
the same identifier.
### Summary and Changelog
Keep a shared embedded timeline service available until all owning clients
have released their references.
- Replace the base-path set with per-base-path reference counts.
- Clear a table's file-system view only when its last reference is
released.
- Serialize reference updates and view cleanup with service acquisition,
preventing an earlier release from clearing a newly acquired table view.
- Close the service only after all table references have been released.
- Detach the server under `SERVICE_LOCK` and close it outside the lock,
assigning closure responsibility to a single caller.
- Remove registry entries only when they still point to the instance being
released.
- Apply the same detach-and-close lifecycle to
`shutdownAllTimelineServers()`, preventing later client releases from closing
the server or decrementing its metric again.
- Continue releasing resources when view cleanup or server closure throws.
- Add regression tests for shared-table references, multiple tables,
reacquisition, concurrent release, replacement services, forced shutdown, and
cleanup failures.
### Impact
- Prevents premature service shutdown and file-system view cleanup for
clients sharing the same table.
- Preserves the existing distinction between owned and borrowed services:
each successful factory acquisition requires one release; borrowed services do
not acquire an additional ownership reference.
- Does not change public method signatures, configuration defaults,
dependencies, or storage formats.
- Keeps full server shutdown outside the global lifecycle lock. Table-view
cleanup now runs under that lock to serialize it with reacquisition, which may
increase acquisition/release latency when view cleanup is slow.
- Does not change the set of services managed by
`shutdownAllTimelineServers()`.
### Risk Level
medium
This change affects shared-service lifecycle and synchronization,
including concurrent acquisition, release, and forced shutdown.
Validation was performed with JDK 11 and the Spark 3.5 build profile:
```bash
mvn -Pspark3.5 \
-pl hudi-client/hudi-client-common -am \
-DskipITs \
-Dtest=TestEmbeddedTimelineService,TestTimelineService \
-DfailIfNoTests=false \
-Dsurefire.failIfNoSpecifiedTests=false \
test-compile surefire:test
```
Results:
- `TestEmbeddedTimelineService`: 16 tests passed, including 9 new test
invocations.
- `TestTimelineService`: 2 tests passed.
- No failures, errors, or skipped tests.
- Compilation, Checkstyle, and license checks passed.
- `git diff --check` passed.
Concurrency tests use latches and bounded waits to exercise lifecycle
transitions. The full project test suite was not run.
### Documentation Update
none
This fixes existing lifecycle behavior without introducing new features or
configuration options. Method documentation now clarifies acquisition and
release responsibilities.
### Contributor's checklist
- [ ] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
