Hisoka-X commented on code in PR #9391:
URL: https://github.com/apache/seatunnel/pull/9391#discussion_r2125336656
##########
seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/config/FileBaseSourceOptions.java:
##########
@@ -88,4 +88,20 @@ public class FileBaseSourceOptions extends FileBaseOptions {
.noDefaultValue()
.withDescription(
"File pattern. The connector will filter some
files base on the pattern.");
+
+ public static final Option<Integer> BINARY_CHUNK_SIZE =
+ Options.key("binary_chunk_size")
+ .intType()
+ .defaultValue(1024)
+ .withDescription(
+ "The chunk size (in bytes) for reading binary
files. Default is 1024 bytes. "
+ + "Larger values may improve performance
for large files but use more memory.");
+
+ public static final Option<Boolean> BINARY_COMPLETE_FILE_MODE =
+ Options.key("binary_complete_file_mode")
+ .booleanType()
+ .defaultValue(false)
+ .withDescription(
+ "Whether to read the complete file as a single
chunk instead of splitting into chunks. "
+ + "When enabled, the entire file content
will be read into memory at once.");
Review Comment:
```suggestion
+ "When enabled, the entire file content
will be read into memory at once. Only valid when file_format_type is binary");
```
##########
seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/config/FileBaseSourceOptions.java:
##########
@@ -88,4 +88,20 @@ public class FileBaseSourceOptions extends FileBaseOptions {
.noDefaultValue()
.withDescription(
"File pattern. The connector will filter some
files base on the pattern.");
+
+ public static final Option<Integer> BINARY_CHUNK_SIZE =
+ Options.key("binary_chunk_size")
+ .intType()
+ .defaultValue(1024)
+ .withDescription(
+ "The chunk size (in bytes) for reading binary
files. Default is 1024 bytes. "
+ + "Larger values may improve performance
for large files but use more memory.");
Review Comment:
```suggestion
+ "Larger values may improve performance
for large files but use more memory. Only valid when file_format_type is
binary");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
