prashantwason commented on a change in pull request #3889:
URL: https://github.com/apache/hudi/pull/3889#discussion_r748694504
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/storage/HoodieHFileWriter.java
##########
@@ -85,14 +86,18 @@ public HoodieHFileWriter(String instantTime, Path file,
HoodieHFileConfig hfileC
this.taskContextSupplier = taskContextSupplier;
HFileContext context = new
HFileContextBuilder().withBlockSize(hfileConfig.getBlockSize())
- .withCompression(hfileConfig.getCompressionAlgorithm())
- .build();
+ .withCompression(hfileConfig.getCompressionAlgorithm())
+ .build();
conf.set(CacheConfig.PREFETCH_BLOCKS_ON_OPEN_KEY,
String.valueOf(hfileConfig.shouldPrefetchBlocksOnOpen()));
conf.set(HColumnDescriptor.CACHE_DATA_IN_L1,
String.valueOf(hfileConfig.shouldCacheDataInL1()));
conf.set(DROP_BEHIND_CACHE_COMPACTION_KEY,
String.valueOf(hfileConfig.shouldDropBehindCacheCompaction()));
CacheConfig cacheConfig = new CacheConfig(conf);
- this.writer = HFile.getWriterFactory(conf, cacheConfig).withPath(this.fs,
this.file).withFileContext(context).create();
+ this.writer = HFile.getWriterFactory(conf, cacheConfig)
+ .withPath(this.fs, this.file)
+ .withFileContext(context)
+ .withComparator(new HoodieMetadataKVComparator())
Review comment:
Why not read this class from the hfileConfig instead of hardcoding it
here?
By calling this HoodieMetadataKVComparator, this implies that somehow the
HoodieHFileWriter is ONLY for metadata table. As currently implemented, we can
have the dataset itself use hfile file format.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
