Prashant Wason created HUDI-6525:
------------------------------------
Summary: Syncing the view in Embedded TimelineServer can cause
NullPointerException if MDT is enabled
Key: HUDI-6525
URL: https://issues.apache.org/jira/browse/HUDI-6525
Project: Apache Hudi
Issue Type: Bug
Reporter: Prashant Wason
Assignee: Prashant Wason
Fix For: 0.14.0
Assume Embedded TimelineServer is being used and there are many executors
sending requests.
Executor 1: Sends request which causes TimelineServer to sync its view
syncIfLocalViewBehind() -> view.sync();
If the view is HoodieMetadataFileSystemView, its sync is written as follows:
public void sync() {
super.sync(); // -> REFRESHES TIMELINE ON TImelineServer
tableMetadata.reset(); // CLOSES MDT readers and OPENS them again
}
The issue is that once super.sync() completes (almost immediately), the
requests from other executors wont detect the local view to be behind and hence
would not call syncIfLocalViewBehind(). They would directly start calling
functions on the view (e.g. view.getLatestFileSlices()). But until MDT is read
(log merging may take time when there are large number of log blocks),
view.getLatestFileSlices() will lead to NPE.
The fix is to reverse the order of reset and sync in
HoodieMetadataFileSystemView.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
