kbuci commented on code in PR #18295:
URL: https://github.com/apache/hudi/pull/18295#discussion_r2992837104
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataWriteUtils.java:
##########
@@ -147,14 +167,37 @@ public static HoodieWriteConfig createMetadataWriteConfig(
HoodieTableVersion datatableVersion) {
String tableName = writeConfig.getTableName() + METADATA_TABLE_NAME_SUFFIX;
boolean isStreamingWritesToMetadataEnabled =
writeConfig.isMetadataStreamingWritesEnabled(datatableVersion);
- WriteConcurrencyMode concurrencyMode = isStreamingWritesToMetadataEnabled
- ? WriteConcurrencyMode.NON_BLOCKING_CONCURRENCY_CONTROL :
WriteConcurrencyMode.SINGLE_WRITER;
- HoodieLockConfig lockConfig = isStreamingWritesToMetadataEnabled
- ?
HoodieLockConfig.newBuilder().withLockProvider(InProcessLockProvider.class).build()
: HoodieLockConfig.newBuilder().build();
- // HUDI-9407 tracks adding support for separate lock configuration for
MDT. Until then, all writes to MDT will happen within data table lock.
-
- if (isStreamingWritesToMetadataEnabled) {
+ WriteConcurrencyMode metadataWriteConcurrencyMode =
+
WriteConcurrencyMode.valueOf(writeConfig.getMetadataConfig().getWriteConcurrencyMode());
+
+ WriteConcurrencyMode concurrencyMode;
+ HoodieLockConfig lockConfig;
+ if (metadataWriteConcurrencyMode.supportsMultiWriter()) {
+ // Configuring Multi-writer directly on metadata table is intended for
executing table service plans, not for writes.
+ checkState(!isStreamingWritesToMetadataEnabled,
+ "Streaming writes to metadata table must be disabled when using
multi-writer concurrency mode "
Review Comment:
Oh hmm I might be misunderstanding. My understanding was that NBCC is a
"type" of multiwriter concurrency mode (like OCC), that can be set regardless
of wether `streamingWritesToMetadataEnabled` is set. But if
`streamingWritesToMetadataEnabled` is set, then concurrency mode must be NBCC
(in data table and MDT). So I decided to allow both OCC/NBCC (allow all
multiwriter concurrency modes) but explicitly not allow
`streamingWritesToMetadataEnabled` to be enabled in data table config
https://github.com/apache/hudi/pull/18295/changes/BASE..cc60d66f90ea53f922b194a249fd76fd2f923533#r2914296574
. Since I figured it might be a bit confusing/misleading, since
`METADATA_PREFIX + ".write.concurrency.mode` is not supposed to be meant for
clients writing to the data table
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
