yihua commented on code in PR #13290:
URL: https://github.com/apache/hudi/pull/13290#discussion_r2112926147
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -2909,6 +2918,26 @@ public int getSecondaryIndexParallelism() {
return metadataConfig.getSecondaryIndexParallelism();
}
+ /**
+ * Whether to enable streaming writes to metadata table or not.
+ * We have support for streaming writes only in SPARK engine (due to spark
task retries intricacies) and for table version > 8 due to the
Review Comment:
```suggestion
* We have support for streaming writes only in SPARK engine (due to spark
task retries intricacies) and for table version >= 8 due to the
```
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -841,6 +841,15 @@ public class HoodieWriteConfig extends HoodieConfig {
.sinceVersion("1.0.0")
.withDocumentation("Whether to enable incremental table service. So far
Clustering and Compaction support incremental processing.");
+ public static final ConfigProperty<Boolean>
STREAMING_WRITES_TO_METADATA_TABLE = ConfigProperty
+ .key("hoodie.write.streaming.writes.to.metadata")
+ .defaultValue(false)
+ .markAdvanced()
+ .sinceVersion("1.1.0")
+ .withDocumentation("Whether to enable streaming writes to metadata table
or not. With streaming writes, we execute writes to both data table and
metadata table "
+ + "using one RDD stage boundary. If not, writes to data table and
metadata table happens across stage boundaries. By default "
Review Comment:
nit: generalize the docs on "one RDD stage boundary"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
