xushiyan commented on a change in pull request #2671:
URL: https://github.com/apache/hudi/pull/2671#discussion_r593402899
##########
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/DatePartitionPathSelector.java
##########
@@ -173,7 +171,8 @@ public DatePartitionPathSelector(TypedProperties props,
Configuration hadoopConf
* Prunes date level partitions to last few days configured by
'NUM_PREV_DAYS_TO_LIST' from
* 'CURRENT_DATE'. Parallelizes listing by leveraging
HoodieSparkEngineContext's methods.
*/
- public List<String> pruneDatePartitionPaths(HoodieSparkEngineContext
context, FileSystem fs, String rootPath) {
+ public List<String> pruneDatePartitionPaths(HoodieSparkEngineContext
context, FileSystem fs, String rootPath, LocalDate currentDate) {
+ LocalDate fromDate = currentDate.minusDays(numPrevDaysToList);
Review comment:
can we move this to L199 so it's not computed until it's needed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
