1fanwang opened a new pull request, #19787: URL: https://github.com/apache/hudi/pull/19787
### Describe the issue this Pull Request addresses When one Spark driver writes several Hudi tables with metrics enabled, clearing one table's local metrics can abort another table's commit. `LocalRegistry.clear()` can run between `containsKey()` and `get()`, causing a concurrent metric update to fail with: ```text java.lang.NullPointerException: Cannot invoke "org.apache.hudi.common.metrics.Counter.add(long)" because the return value of "org.apache.hudi.common.metrics.LocalRegistry.getCounter(String)" is null ``` Closes https://github.com/apache/hudi/issues/19570. ### Summary and Changelog Counter lookup now uses `ConcurrentHashMap.computeIfAbsent()`, so a concurrent clear cannot make it return null. The atomic lookup matches the fix proposed in https://github.com/apache/hudi/pull/19584. This PR adds a deterministic interleaving test that reproduces the failure. ### Impact There is no public API change. Concurrent metric updates no longer fail when the local registry is cleared. ### Risk Level Low. The change replaces a compound access to an existing `ConcurrentHashMap` with its atomic lookup operation. ### Testing <details> <summary>Raw failing and passing regression</summary> ```console $ export JAVA_HOME=$(/usr/libexec/java_home -v 17) $ git restore --source=upstream/master --worktree hudi-io/src/main/java/org/apache/hudi/common/metrics/LocalRegistry.java $ mvn -pl hudi-io -am -Dtest=org.apache.hudi.common.metrics.TestLocalRegistry -Dsurefire.failIfNoSpecifiedTests=false test [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0 org.opentest4j.AssertionFailedError: Unexpected exception thrown: java.util.concurrent.ExecutionException: java.lang.NullPointerException: Cannot invoke "org.apache.hudi.common.metrics.Counter.add(long)" because the return value of "org.apache.hudi.common.metrics.LocalRegistry.getCounter(String)" is null [INFO] BUILD FAILURE $ git restore --source=HEAD --worktree hudi-io/src/main/java/org/apache/hudi/common/metrics/LocalRegistry.java $ mvn -pl hudi-io -am -Dtest=org.apache.hudi.common.metrics.TestLocalRegistry -Dsurefire.failIfNoSpecifiedTests=false test [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 [INFO] BUILD SUCCESS ``` </details> <details> <summary>Full hudi-io suite</summary> ```console $ mvn -pl hudi-io -am test [INFO] Tests run: 126, Failures: 0, Errors: 0, Skipped: 0 [INFO] hudi-io ............................................ SUCCESS [INFO] BUILD SUCCESS ``` </details> ### Documentation Update None. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute). Reviewed the contributor guide and repository PR template. - [x] Enough context is provided in the sections above. The issue and prior implementation are linked. - [x] Adequate tests were added if applicable. The deterministic regression fails on `upstream/master`, passes on this branch, and the full `hudi-io` suite passes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
