xiarixiaoyao commented on a change in pull request #3330:
URL: https://github.com/apache/hudi/pull/3330#discussion_r684725178
##########
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/SparkRDDWriteClient.java
##########
@@ -460,6 +487,17 @@ protected void preCommit(String instantTime,
HoodieCommitMetadata metadata) {
Option.of(metadata), config,
txnManager.getLastCompletedTransactionOwner());
}
+ @Override
+ protected void saveStatisticsInfo(List<String> touchFiles, String cols,
String indexPath, String saveMode) {
+ if (touchFiles.isEmpty() || cols.isEmpty() || indexPath.isEmpty()) {
+ LOG.warn("save nothing to index table");
+ return;
+ }
+ HoodieSparkEngineContext sparkEngineContext =
(HoodieSparkEngineContext)context;
+ Zoptimize$.MODULE$.saveStatisticsInfo(sparkEngineContext
Review comment:
not for log file. since when the zorder/hilbert optimize has been done,
old files(include log file if the table is mor table) will replaced by new
parquet file, so i donot consider log files. but i agree with you that it
is better to build better abstractions to make it easily extend to future
usecases
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
