LuciferYang opened a new issue, #10929:
URL: https://github.com/apache/paimon/issues/10929

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/paimon/issues) 
and found nothing similar.
   
   ### Paimon version
   
   master (30d91278af)
   
   ### Compute Engine
   
   Spark
   
   ### Minimal reproduce step
   
   A directory with two CSV files whose names contain a space: `good data.csv` 
with `1,Alice` and `2,Bob`, and `bad data.csv` with `abc,Charlie`.
   
   ```sql
   CREATE TABLE t (id INT, name STRING);
   
   COPY INTO t FROM '/path/to/dir' FILE_FORMAT = (TYPE = CSV) ON_ERROR = 
SKIP_FILE;
   ```
   
   ### What doesn't meet your expectations?
   
   Both files are reported as `LOADED` with 0 rows loaded, 0 rows parsed and 0 
errors. The rows of `good data.csv` are in the table and `bad data.csv` is 
skipped, as expected, but `bad data.csv` is also recorded in the load history, 
so running the same statement again reports it as `SKIPPED` instead of trying 
it again. With `ON_ERROR = CONTINUE`, a file whose rows all fail is reported as 
`LOADED` the same way, and with the default `ON_ERROR = ABORT_STATEMENT` the 
rows loaded of a file with such a name are always 0.
   
   The per-file row counts and errors are keyed by the file name taken from 
`input_file_name()`, which Spark URL-encodes (`good%20data.csv`), while the 
result and the load history look them up with the decoded name (`good 
data.csv`). Names without characters that need encoding are not affected.
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to