voonhous commented on code in PR #18065:
URL: https://github.com/apache/hudi/pull/18065#discussion_r3354424099
##########
hudi-hadoop-common/src/main/java/org/apache/hudi/io/storage/hadoop/HoodieAvroFileWriterFactory.java:
##########
@@ -123,9 +124,37 @@ protected HoodieFileWriter newOrcFileWriter(
private HoodieAvroWriteSupport getHoodieAvroWriteSupport(HoodieSchema schema,
HoodieConfig
config, boolean enableBloomFilter) {
Option<BloomFilter> filter = enableBloomFilter ?
Option.of(createBloomFilter(config)) : Option.empty();
+ HoodieSchema effectiveSchema =
HoodieAvroWriteSupport.generateEffectiveSchema(schema, config);
+ Properties props = config.getProps();
+ // Auto-detect variant shredding provider from classpath if not explicitly
configured
+ if (!props.containsKey(PARQUET_VARIANT_SHREDDING_PROVIDER_CLASS.key())) {
+ String detected = detectShreddingProvider();
+ if (detected != null) {
+ props.setProperty(PARQUET_VARIANT_SHREDDING_PROVIDER_CLASS.key(),
detected);
Review Comment:
Good catch, fixed. `HoodieConfig.getProps()` is a Lombok getter returning
the internal `props` reference, so `setProperty` was mutating the shared
config. Now working on a copy via `TypedProperties.copy(config.getProps())` and
passing that copy to the write support (the previous code re-fetched
`config.getProps()`, which only worked because of the in-place mutation). I
used `TypedProperties.copy` rather than `new Properties(config.getProps())`
because the latter installs the source as a defaults fallback, which would make
the `containsKey` check behave incorrectly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
