Blazer-007 commented on code in PR #4044:
URL: https://github.com/apache/gobblin/pull/4044#discussion_r1982758855
##########
gobblin-core/src/main/java/org/apache/gobblin/source/PartitionedFileSourceBase.java:
##########
@@ -367,7 +384,25 @@ private long getLowWaterMark(Iterable<WorkUnitState>
previousStates, String lowW
return lowWaterMarkValue + getRetriever().getWatermarkIncrementMs();
}
+ /** Returns the low watermark value based on lookback which is equal to
current time minus lookback time. */
+ private long getLowWaterMarkFromLookbackTime(String lookBackTime) {
+ try {
+ Duration lookBackDuration =
PartitionAwareFileRetrieverUtils.getLookbackTimeDuration(lookBackTime);
+ return new DateTime().minus(lookBackDuration).getMillis();
+ } catch (IOException e) {
+ Throwables.propagate(e);
Review Comment:
replaced with `log.error("Failed to parse lookback time: {} , Returning 0 as
low watermark value, {}", lookBackTime, e.getMessage());`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
