kasured commented on issue #5843: URL: https://github.com/apache/hudi/issues/5843#issuecomment-1154342855
@nsivabalan I have been able to reproduce the behaviour locally with hoodie 0.9.0 Please check the repository here https://github.com/kasured/hudi-timeline-5843. When you run the code as instructed in branch `main` you will be able to see some commits that are created in the future. I was able to get commits 1 hour in the future. But I would say that might depend on multiple factors and the difference might be as displayed in the description of the issues. ``` ################################################################################################## Number of generated(attempted) instants : 107 Number of commits in the future : 9 Number of ArrayIndexOutOfBoundsException exceptions: 0 ################################################################################################## ``` I have not been able to reproduce another which I believe relating to the same root cause issue `ArrayIndexOutOfBoundsException` that we from time to time get on the EMR cluster ``` message: Micro batch id=1297 threw following exception: java.lang.ArrayIndexOutOfBoundsException: -1 at java.util.Calendar.getDisplayName(Calendar.java:2114) at java.text.SimpleDateFormat.subFormat(SimpleDateFormat.java:1125) at java.text.SimpleDateFormat.format(SimpleDateFormat.java:966) at java.text.SimpleDateFormat.format(SimpleDateFormat.java:936) at java.text.DateFormat.format(DateFormat.java:345) at org.apache.hudi.common.table.timeline.HoodieActiveTimeline.lambda$createNewInstantTime$0(HoodieActiveTimeline.java:92) ``` In that repository that I created If you switch to branch `hoodie/0.10.0` though and run the code as described you will see significant difference in the number of generated instants and also I was not able to reproduce that future commit issue. So I wonder if https://github.com/apache/hudi/pull/4073 can be back-ported to 0.9.0. Without that code this version is susceptible to subtle issue and data corruptions ``` ################################################################################################## Number of generated(attempted) instants : 28722 Number of commits in the future : 0 Number of ArrayIndexOutOfBoundsException exceptions: 0 ################################################################################################## ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
