vamshikrishnakyatham commented on code in PR #14115:
URL: https://github.com/apache/hudi/pull/14115#discussion_r2446126995
##########
hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/hudi/DataSourceUtils.java:
##########
@@ -323,7 +323,27 @@ public boolean validate(long totalRecords, long
totalErroredRecords, Option<Hood
if (totalErroredRecords > 0) {
hasErrored.set(true);
ValidationUtils.checkArgument(writeStatusesOpt.isPresent(), "RDD
<WriteStatus> expected to be present when there are errors");
- LOG.error("{} failed with errors", writeOperationType);
+ List<String> errorKeys =
HoodieJavaRDD.getJavaRDD(writeStatusesOpt.get())
+ .filter(WriteStatus::hasErrors)
+ .flatMap(ws -> ws.getErrors().keySet().stream().iterator())
+ .take(10)
+ .stream()
+ .map(Object::toString)
+ .collect(Collectors.toList());
+
+ String errorSummary = String.format(
+ "%s operation failed with %d error(s).\n\nFailed records (first %d
of %d):\n%s\n\n"
+ + "Check for error stacktraces in the driver logs which could
give more information on the failure.",
+ writeOperationType,
+ totalErroredRecords,
+ Math.min(10, errorKeys.size()),
+ errorKeys.size(),
+ errorKeys.stream()
+ .map(k -> " - Record Key: " + k)
+ .collect(Collectors.joining("\n")));
+
+ LOG.error(errorSummary);
Review Comment:
right, updated it with just count
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
