nsivabalan commented on code in PR #7709:
URL: https://github.com/apache/hudi/pull/7709#discussion_r1081777492
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -235,10 +235,7 @@ private void enablePartition(final MetadataPartitionType
partitionType, final Ho
* @param writeConfig {@code HoodieWriteConfig} of the main dataset writer
*/
private HoodieWriteConfig createMetadataWriteConfig(HoodieWriteConfig
writeConfig) {
- int parallelism = writeConfig.getMetadataInsertParallelism();
-
- int minCommitsToKeep = Math.max(writeConfig.getMetadataMinCommitsToKeep(),
writeConfig.getMinCommitsToKeep());
- int maxCommitsToKeep = Math.max(writeConfig.getMetadataMaxCommitsToKeep(),
writeConfig.getMaxCommitsToKeep());
+ int parallelism =
HoodieMetadataConfig.INSERT_PARALLELISM_VALUE.defaultValue();
Review Comment:
we can let this be overriden by user.
##########
hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java:
##########
@@ -60,13 +60,15 @@ public final class HoodieMetadataConfig extends
HoodieConfig {
.withDocumentation("Enable publishing of metrics around metadata
table.");
// Parallelism for inserts
+ // NOTE: This is an internal config and setting this will not overwrite the
value actually used
public static final ConfigProperty<Integer> INSERT_PARALLELISM_VALUE =
ConfigProperty
.key(METADATA_PREFIX + ".insert.parallelism")
.defaultValue(1)
.sinceVersion("0.7.0")
.withDocumentation("Parallelism to use when inserting to the metadata
table");
Review Comment:
I meant to fix the user documentation. i.e. L 68
```
.withDocumentation("Parallelism to use when inserting to the metadata
table");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
