Hisoka-X commented on code in PR #9526:
URL: https://github.com/apache/seatunnel/pull/9526#discussion_r2234805095
##########
docs/en/connector-v2/source/CosFile.md:
##########
@@ -75,6 +75,8 @@ To use this connector you need put
hadoop-cos-{hadoop.version}-{version}.jar and
| binary_chunk_size | int | no | 1024 |
| binary_complete_file_mode | boolean | no | false |
| common-options | | no | - |
+| file_filter_modified_start | string | no | - |
File modification time filter. The connector will filter some files base on the
last modification start time (include start time). the default data format is
`yyyy-MM-dd HH:mm:ss`.
|
Review Comment:
```suggestion
| file_filter_modified_start | string | no | - |
File modification time filter. The connector will filter some files base on the
last modification start time (include start time). The default data format is
`yyyy-MM-dd HH:mm:ss`.
|
```
##########
seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/reader/AbstractReadStrategy.java:
##########
@@ -121,7 +127,10 @@ public List<String> getFileNamesByPath(String path) throws
IOException {
if (fileStatus.isFile() && filterFileByPattern(fileStatus) &&
fileStatus.getLen() > 0) {
// filter '_SUCCESS' file
if (!fileStatus.getPath().getName().equals("_SUCCESS")
- && !fileStatus.getPath().getName().startsWith(".")) {
+ && !fileStatus.getPath().getName().startsWith(".")
+ // &&
filterFileByModificationDate(fileStatus)
Review Comment:
why?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
