ravjot28 opened a new pull request, #20591:
URL: https://github.com/apache/hudi/pull/20591

   ### Describe the issue this Pull Request addresses
   
   Closes #20581
   
   `RocksDBDAO` forwards RocksDB's info log to a Java `org.rocksdb.Logger`, 
which RocksDB calls from its background threads,
   attaching each to the JVM. When the JVM exits while such a thread logs for a 
store that is still open, `exit()` runs
   RocksDB's static destructors, which wait for that thread, while the thread 
waits to attach to the exiting JVM: the
   process never exits.
   
   ### Summary and Changelog
   
   No RocksDB thread calls into the JVM any more: RocksDB writes its info log 
natively.
   
   - `RocksDBDAO#init` sets no Java logger. RocksDB creates its own info log 
`LOG` in the store's folder, rolled at 16 MiB
     with two files kept; the folder, and the log with it, is removed by 
`close()` as before.
   - Why the logger is removed rather than guarded: alternatives that keep the 
Java logger do not close the window. A
     higher info log level still attaches the thread for every message at or 
above it, and closing stores from a
     shutdown hook would close stores that other threads may still be using. 
#19998 kept the Java logger on purpose and
     made its handles releasable on `close()`; that does not help a store that 
is still open when the JVM exits, which is
     the case here. Its level-mapping test goes with the logger.
   - `RocksDBLogger` and the `logger` field are removed; `close()` still 
releases the `DBOptions` and the `Statistics`.
     `statistics` gets a package-private getter for the tests.
   - `TestRocksDBDAO`: new `testInfoLogIsANativeFileInTheStoreFolder` and 
`testCloseReleasesNativeOptionsAndStatistics`;
     the init-failure and repeated-cycle tests check the `Statistics` instead 
of the logger; the logger-specific tests are
     removed with the logger. `TestRocksDbBasedFileSystemView` checks the 
options and statistics instead of the logger.
   
   ### Impact
   
   RocksDB's messages no longer appear in the process log under `RocksDBDAO`; 
they are in the `LOG` file of each store
   while it is open. Errors still reach callers as `RocksDBException`. No 
configuration change. Disk: the native info log
   uses a 16 MiB rollover threshold and retains two files per open store; this 
is not a strict byte cap.
   
   Affects every `RocksDBDAO` user: the `EMBEDDED_KV_STORE` file system view, 
`RocksDbDiskMap`, `RocksDBBasedMap`, the
   Flink RocksDB index backend and lookup cache.
   
   **Backport:** please include this fix in the next 1.2.x patch release; 1.2.1 
sets the same Java logger. The commit does
   not cherry-pick cleanly onto `release-1.2.1`, which lacks the handle release 
of #19998 that this change builds on
   (`RocksDBDAO`, `TestRocksDBDAO` and `TestRocksDbBasedFileSystemView` 
conflict). I can open an adapted change against
   `branch-1.2.x`.
   
   ### Risk Level
   
   low. The change removes a callback; a native logger is RocksDB's default. 
The exit hang itself is timing dependent: on
   `release-0.14.1`, `TestHoodieJavaClientOnMergeOnReadStorage` run alone hung 
in 3 of 10 runs, and in 0 of 10 runs with
   the same change applied to that tag (the Java logger removed, the same info 
log limits). The tests below show the
   mechanism (no Java logger, a native info log), not the hang.
   
   Tests, run on master with JDK 11 and passing checkstyle checks for main and 
test sources (scalastyle did not execute
   in this Java-only reactor), with `mvn test -pl hudi-hadoop-common -am 
-Dtest=...` and the classes listed below:
   - Without the main code change (master `RocksDBDAO` with its Java logger, 
plus the package-private `statistics` getter
     and the two log constants the tests read): 
`TestRocksDBDAO#testInfoLogIsANativeFileInTheStoreFolder` fails (no `LOG`
     file in the store folder while the Java logger is set); the other 15 tests 
of the class pass.
   - With the change: `TestRocksDBDAO` 16/16, `TestRocksDbBasedFileSystemView` 
52/52, `TestRocksDBBasedIncrementalFSViewSync` 5/5,
     `TestRocksDbBasedMap` 1/1, `TestRocksDbDiskMap` 6/6 pass.
   
   ### Documentation Update
   
   none
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to