XuQianJin-Stars commented on code in PR #6284:
URL: https://github.com/apache/hudi/pull/6284#discussion_r938382783
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieMergedLogRecordScanner.java:
##########
@@ -92,11 +92,12 @@ protected HoodieMergedLogRecordScanner(FileSystem fs,
String basePath, List<Stri
forceFullScan, partitionName, internalSchema);
try {
// Store merged records for all versions for this log file, set the
in-memory footprint to maxInMemoryMapSize
- this.records = new ExternalSpillableMap<>(maxMemorySizeInBytes,
spillableMapBasePath, new DefaultSizeEstimator(),
+ this.records = new ExternalSpillableMap<>(maxMemorySizeInBytes, basePath
+ spillableMapBasePath, new DefaultSizeEstimator(),
new HoodieRecordSizeEstimator(readerSchema), diskMapType,
isBitCaskDiskMapCompressionEnabled);
+
Review Comment:
> not sure if we can do this. spillableMapbase path is configurable. If one
does not want "/tmp/" which is the default, they can always override using the
configs.
It is more troublesome for users to use, and this option needs to be
specified additionally, and considering that the basepath will definitely be in
a large data disk, the temporary directory can be put into the same disk as the
basepath.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
