thswlsqls opened a new pull request, #10377:
URL: https://github.com/apache/paimon/pull/10377
### Purpose
fix #10375
- `lineSep` is Spark's JSON line separator option. `CsvOptions` and
`TextOptions` already accept it as a fallback key, but `json.line-delimiter`
does not.
- With the default `format-table.implementation=paimon`, a JSON format table
created with `lineSep` is read and written with `\n`, while the `engine`
implementation honours it.
- Add `.withFallbackKeys("lineSep")` to `JsonOptions.LINE_DELIMITER`,
following #9170 and #9565.
- Mapping `json.line-delimiter` to `lineSep` on the Spark engine side is out
of scope.
### Tests
- Added `JsonFileFormatTest#testJsonLineDelimiterFallbackKey`: with only
`lineSep=|` set, checks the resolved option, that written bytes use `|` instead
of `\n`, that a `|`-delimited file reads as two rows, and a round trip.
- Without the fix, it fails at `options.get(JsonOptions.LINE_DELIMITER)`
(expected `"|"`, got `"\n"`).
- `mvn -pl paimon-format clean install`: BUILD SUCCESS, 768 tests, 0
failures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
