vinothchandar commented on a change in pull request #4060:
URL: https://github.com/apache/hudi/pull/4060#discussion_r756962217
##########
File path: style/checkstyle.xml
##########
@@ -275,7 +275,9 @@
<module name="EmptyStatement" />
<!-- Checks for Java Docs. -->
- <module name="JavadocStyle"/>
+ <module name="JavadocStyle">
Review comment:
I prefer these in separate PRs
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java
##########
@@ -245,14 +246,17 @@ private synchronized FileSystemViewManager
getViewManager() {
public abstract HoodieWriteMetadata<O>
insertOverwriteTable(HoodieEngineContext context, String instantTime, I
records);
/**
- * update statistics info for current table.
- * to do adaptation, once RFC-27 is finished.
+ * Updates Metadata Indexes (like Z-Index)
+ * TODO rebase onto metadata table (post RFC-27)
*
- * @param context HoodieEngineContext
- * @param instantTime Instant time for the replace action
- * @param isOptimizeOperation whether current operation is OPTIMIZE type
- */
- public abstract void updateStatistics(HoodieEngineContext context,
List<HoodieWriteStat> stats, String instantTime, Boolean isOptimizeOperation);
+ * @param context instance of {@link HoodieEngineContext}
+ * @param instantTime instant of the carried operation triggering the update
+ */
+ public abstract void updateMetadataIndexes(
Review comment:
rename: updateColumnStats? for now, this is not connected to the
metadata table at all and even then would only update one partition. So kind
of misleading
##########
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/SparkRDDWriteClient.java
##########
@@ -389,15 +389,16 @@ private void
completeClustering(HoodieReplaceCommitMetadata metadata, JavaRDD<Wr
writeTableMetadata(table, metadata, new
HoodieInstant(HoodieInstant.State.INFLIGHT,
HoodieTimeline.REPLACE_COMMIT_ACTION, clusteringCommitTime));
finalizeWrite(table, clusteringCommitTime, writeStats);
try {
- // try to save statistics info to hudi
- if (config.isDataSkippingEnabled() &&
config.isLayoutOptimizationEnabled() &&
!config.getClusteringSortColumns().isEmpty()) {
- table.updateStatistics(context, writeStats, clusteringCommitTime,
true);
+ // Update outstanding metadata indexes
+ if (config.isLayoutOptimizationEnabled()
Review comment:
I had a similar comment before. this. is a good change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
