LuciferYang opened a new pull request, #10930: URL: https://github.com/apache/paimon/pull/10930
### Purpose close #10929 COPY INTO counts rows and errors per file under the file name from `input_file_name()`, which Spark URL-encodes, and looked them up with the decoded name. For a file name with a space, `#` or `%` the lookup always missed: rows loaded was reported as 0, and with `ON_ERROR = SKIP_FILE` or `CONTINUE` a file whose rows all failed was reported as `LOADED` and recorded in the load history, so a rerun skipped it. The lookup now uses the encoded base name of the file (`Path.toUri`), the same form as `input_file_name()`. The result still shows the decoded name, and the load history still records the full decoded path, so files whose names need no encoding behave as before. Files that were already recorded as loaded by mistake stay in the history and need `FORCE = TRUE` to be loaded again. ### Tests - `CopyIntoTestBase` "rows_loaded count of a file name with a space": the default `ABORT_STATEMENT` mode reports 3 rows loaded for `my data.csv`. On master it reports 0. - `CopyIntoOnErrorTest` "SKIP_FILE with file names that contain spaces": `bad data.csv` is `LOAD_FAILED` and is tried again on a rerun, `good data.csv` is `LOADED` with 2 rows and `SKIPPED` on the rerun. On master both are `LOADED` with 0 rows. - `CopyIntoOnErrorTest` "CONTINUE with a failed file name that contains a space": a file whose rows all fail is `LOAD_FAILED` with 2 errors, also on a rerun. On master it is `LOADED`. ### API and Format no ### Documentation no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
