codope commented on code in PR #7367:
URL: https://github.com/apache/hudi/pull/7367#discussion_r1041813184
##########
hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/util/SyncUtilHelpers.java:
##########
@@ -104,4 +107,22 @@ static HoodieSyncTool instantiateMetaSyncTool(String
syncToolClassName,
+ ": no valid constructor found.");
}
}
+
+ public static HoodieException
getExceptionFromList(Collection<HoodieException> exceptions) {
+ if (exceptions.size() == 1) {
+ return exceptions.stream().findFirst().get();
+ }
+ StringBuilder sb = new StringBuilder();
+ sb.append("Multiple exceptions during meta sync:\n********\n");
+ exceptions.forEach(e -> {
+ StringWriter sw = new StringWriter();
+ PrintWriter pw = new PrintWriter(sw);
+ e.printStackTrace(pw);
Review Comment:
Let's avoid printStacktrace and just append exception.getMessage() to string
builder.
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java:
##########
@@ -744,11 +744,20 @@ public void runMetaSync() {
props.getInteger(HoodieIndexConfig.BUCKET_INDEX_NUM_BUCKETS.key())));
}
+ //Collect exceptions in list because we want all sync to run. Then we
can throw
+ List<HoodieException> metaSyncExceptions = new ArrayList<>();
for (String impl : syncClientToolClasses) {
- Timer.Context syncContext = metrics.getMetaSyncTimerContext();
- SyncUtilHelpers.runHoodieMetaSync(impl.trim(), metaProps, conf, fs,
cfg.targetBasePath, cfg.baseFileFormat);
- long metaSyncTimeMs = syncContext != null ? syncContext.stop() : 0;
-
metrics.updateDeltaStreamerMetaSyncMetrics(getSyncClassShortName(impl),
metaSyncTimeMs);
+ try {
+ Timer.Context syncContext = metrics.getMetaSyncTimerContext();
+ SyncUtilHelpers.runHoodieMetaSync(impl.trim(), metaProps, conf, fs,
cfg.targetBasePath, cfg.baseFileFormat);
+ long metaSyncTimeMs = syncContext != null ? syncContext.stop() : 0;
+
metrics.updateDeltaStreamerMetaSyncMetrics(getSyncClassShortName(impl),
metaSyncTimeMs);
+ } catch (HoodieException e) {
+ metaSyncExceptions.add(e);
Review Comment:
Let's log the error and full stack trace here itself instead of doing
printStackTrace.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
