Prashant Wason created HUDI-1794: ------------------------------------ Summary: Generating a new instant time in HoodieActiveTimeline is not thread safe Key: HUDI-1794 URL: https://issues.apache.org/jira/browse/HUDI-1794 Project: Apache Hudi Issue Type: Bug Reporter: Prashant Wason Assignee: Prashant Wason
When generating a new instant time in HoodieActiveTimeline, a static instance of SimpleDateFormat is used. This class is not thread safe. We have a production usecase where multiple HUDI datasets are processed in parallel in different threads of a ThreadPool. Each of these threads creates its own SparkRDDBackedWriteClient and calls startCommit() which generates a new commit time. Because SimpleDateFormat is not thread safe, we get corrupted instant times in several threads. The solution is to use a thread-specific instance of the SimpleDateFormat for generating new instant times. -- This message was sent by Atlassian Jira (v8.3.4#803005)