leesf commented on a change in pull request #1647:
URL: https://github.com/apache/incubator-hudi/pull/1647#discussion_r429542752
##########
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java
##########
@@ -416,10 +425,12 @@ public DeltaSync getDeltaSync() {
jssc.setLocalProperty("spark.scheduler.pool",
SchedulerConfGenerator.DELTASYNC_POOL_NAME);
}
try {
+ int iteration = 1;
while (!isShutdownRequested()) {
try {
long start = System.currentTimeMillis();
- Option<String> scheduledCompactionInstant = deltaSync.syncOnce();
+ HoodieMetrics.setTableName(cfg.metricsTableName + "_" +
iteration);
Review comment:
@pratyakshsharma just a raw idea, how about adding a static variable
(iterator) in HoodieMetrics, and ++iterator after `updateCommitMetrics`. And
back to the PR, if we only fix the HoodieDeltaStreamer, I think we would simply
pass the iteration_time to `syncOnce` method to create a new metric name
instead of adding a static tableName to HoodieMetrics, I think it is a little
bit weird.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
