[
https://issues.apache.org/jira/browse/GOBBLIN-1809?focusedWorklogId=856269&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-856269
]
ASF GitHub Bot logged work on GOBBLIN-1809:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 11/Apr/23 21:46
Start Date: 11/Apr/23 21:46
Worklog Time Spent: 10m
Work Description: homatthew commented on code in PR #3670:
URL: https://github.com/apache/gobblin/pull/3670#discussion_r1163363722
##########
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/version/finder/LookbackDateTimeDatasetVersionFinder.java:
##########
@@ -67,25 +66,20 @@ public LookbackDateTimeDatasetVersionFinder(FileSystem fs,
Config config, Instan
Preconditions.checkArgument(config.hasPath(VERSION_LOOKBACK_PERIOD) ,
"Missing required property " + VERSION_LOOKBACK_PERIOD);
PeriodFormatter periodFormatter =
new
PeriodFormatterBuilder().appendYears().appendSuffix("y").appendMonths().appendSuffix("M").appendDays()
-
.appendSuffix("d").appendHours().appendSuffix("h").appendMinutes().appendSuffix("m").toFormatter();
- this.stepDuration = Duration.standardMinutes(1);
+ .appendSuffix("d").appendHours().appendSuffix("h").toFormatter();
+ this.stepDuration = Duration.standardHours(1);
this.pathPrefix = ConfigUtils.getString(config, VERSION_PATH_PREFIX, "");
this.lookbackPeriod =
periodFormatter.parsePeriod(config.getString(VERSION_LOOKBACK_PERIOD));
this.endTime = endTime;
}
- @Override
- public Class<? extends FileSystemDatasetVersion> versionClass() {
- return TimestampedDatasetVersion.class;
- }
-
@Override
public Collection<TimestampedDatasetVersion> findDatasetVersions(Dataset
dataset) throws IOException {
FileSystemDataset fsDataset = (FileSystemDataset) dataset;
Set<TimestampedDatasetVersion> versions = new HashSet<>();
Instant startTime = endTime.minus(lookbackPeriod.toStandardDuration());
- while (startTime.isBefore(endTime)) {
+ while (!startTime.isAfter(endTime)) {
Review Comment:
Does making this a for loop make it easier to read? E.g.
```
for (Instant time = startTime; !time.isAfter(endTime);
time.plus(stepDuration))
```
Issue Time Tracking
-------------------
Worklog Id: (was: 856269)
Time Spent: 2h (was: 1h 50m)
> Add new lookback version finder for use with iceberg retention
> --------------------------------------------------------------
>
> Key: GOBBLIN-1809
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1809
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Jack Moseley
> Priority: Major
> Time Spent: 2h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
