[
https://issues.apache.org/jira/browse/HUDI-2055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17375918#comment-17375918
]
ASF GitHub Bot commented on HUDI-2055:
--------------------------------------
vinothchandar commented on a change in pull request #3129:
URL: https://github.com/apache/hudi/pull/3129#discussion_r664772534
##########
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java
##########
@@ -289,6 +289,8 @@ public void refreshTimeline() throws IOException {
srcRecordsWithCkpt.getRight().getLeft(), metrics,
overallTimerContext);
}
+ metrics.updateDeltaStreamerSyncMetrics(System.currentTimeMillis());
Review comment:
Yes. This patch is okay as long as you can set your alerts to catch - no
sync metrics being emitted for a period of time.
Another way to monitor would be to emit a 1 for success and 0 for failure.
And it lets you set alerts on a rolling window aggregate.
I have done both personally. Cant say one is better over other by a lot
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
> Add metric for time of lastSync
> -------------------------------
>
> Key: HUDI-2055
> URL: https://issues.apache.org/jira/browse/HUDI-2055
> Project: Apache Hudi
> Issue Type: New Feature
> Components: DeltaStreamer, metrics
> Reporter: Sebastian Bernauer
> Priority: Minor
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> We monitor our running instances of DeltaStreamer. We currently have the
> problem that we can't see from our monitoring if the DeltaStreamer is not
> doing any commits because it has problems or because there are simply no
> events to consume.
> This Ticket adds a metric {{lastSync}} that is similar to {{commitTime}} but
> increments after every successful sync and not only a successful commit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
