RexLyc commented on issue #10350:
URL: https://github.com/apache/seatunnel/issues/10350#issuecomment-3804580451
I remembered at least LocalFile/S3File has this problem. Further more, as I
said above, this problem seems can also be appeared under others situations.
Such as user give a column that doesn't exist, and this column required
nullable check. In this case, we also get null value without any error thrown.
The configuration and example data below has been tested, and you should get
same error under seatunnel 2.3.12 version.
```conf
env {
job.mode = "BATCH"
job.name = "seatunnel_job"
parallelism = 1
}
source {
LocalFile {
path = "/home/kpad/connector/data/localfs"
file_format_type = "csv"
file_filter_pattern = "alice-local-table.csv"
csv_use_header_line = true
field_delimiter = ","
schema {
columns = [
{
name = id
type = bigint
nullable = false
},
{
name = x
type = double
nullable = true
}
]
}
}
}
sink {
LocalFile {
path = "/home/kpad/var/storage/data"
file_format_type = "csv"
single_file_mode = true
custom_filename = true
is_enable_transaction = false
enable_header_write = true
create_empty_file_when_no_data = true
file_name_expression = "alice-local-import"
filename_extension = ".csv"
}
}
```
This is my input csv file, which placed in
/home/kpad/connector/data/localfs/alice-local-table.csv
```csv
id,x
,9.9
```
And I got the result csv placed in
/home/kpad/var/storage/data/alice-local-import.csv
```csv
id,x
,9.9
```
In this case, nullable check doesn't work.
And even if I set my input csv like this.
```csv
x
9.9
10.0
```
In this case, I totally removed column id. And I still got the result.
```
id,x
,9.9
,10.0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
