lwllvyb commented on code in PR #2216:
URL:
https://github.com/apache/incubator-uniffle/pull/2216#discussion_r1810401133
##########
common/src/main/java/org/apache/uniffle/common/config/RssBaseConf.java:
##########
@@ -294,17 +294,29 @@ public class RssBaseConf extends RssConf {
+ " first combining the username and the password with a
colon (uniffle:uniffle123)"
+ ", and then by encoding the resulting string in base64
(dW5pZmZsZTp1bmlmZmxlMTIz).");
- public static final ConfigOption<String> RSS_STORAGE_WRITE_DATA_BUFFER_SIZE =
- ConfigOptions.key("rss.storage.write.dataBufferSize")
+ public static final ConfigOption<String>
RSS_STORAGE_LOCALFILE_WRITE_DATA_BUFFER_SIZE =
+ ConfigOptions.key("rss.storage.localfile.write.dataBufferSize")
.stringType()
.defaultValue("8k")
- .withDescription("The buffer size to cache the write data content.");
+ .withDescription("The buffer size to cache the write data content
for LOCALFILE.");
- public static final ConfigOption<String> RSS_STORAGE_WRITE_INDEX_BUFFER_SIZE
=
- ConfigOptions.key("rss.storage.write.indexBufferSize")
+ public static final ConfigOption<String>
RSS_STORAGE_LOCALFILE_WRITE_INDEX_BUFFER_SIZE =
+ ConfigOptions.key("rss.storage.localfile.write.indexBufferSize")
.stringType()
.defaultValue("8k")
- .withDescription("The buffer size to cache the write index
content.");
+ .withDescription("The buffer size to cache the write index content
for LOCALFILE.");
+
+ public static final ConfigOption<String>
RSS_STORAGE_HDFS_WRITE_DATA_BUFFER_SIZE =
+ ConfigOptions.key("rss.storage.hdfs.write.dataBufferSize")
+ .stringType()
+ .defaultValue("8k")
Review Comment:
<img width="986" alt="image"
src="https://github.com/user-attachments/assets/05f43826-c21f-4366-9ecb-7aec7177bd6f";>
<img width="761" alt="image"
src="https://github.com/user-attachments/assets/9fd916e1-1a0a-4cc0-a63f-fa33eca9d7c6";>
I have checked the source code of hadoop. The `io.file.buffer.size` default
value is 4096.
So i think `8k` as the default value is ok for HDFS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
