Baymine opened a new pull request, #66040:
URL: https://github.com/apache/doris/pull/66040
### What problem does this PR solve?
Issue Number: close #66039
Problem Summary:
When reading Hive text (and CSV) data,
`DataTypeArraySerDe::deserialize_one_cell_from_hive_text`
and `DataTypeMapSerDe::deserialize_one_cell_from_hive_text` returned
`Status::InvalidArgument("slice is empty!")` for an empty field. This is
wrong for a Hive-text
empty field, which should map to an empty collection (`[]` / `{}`) to match
Spark/Hive semantics.
For a non-nullable array/map column this surfaced as a load/read error. For
a nullable column the
`InvalidArgument` was silently swallowed by `DataTypeNullableSerDe` (which
fills NULL on any nested
error), so an empty field became NULL instead of an empty collection.
The fix makes both serdes append an empty collection (advance offsets by 0)
and return
`Status::OK()` when the input slice is empty.
Observable behavior change: an empty array/map field in Hive text is now
read as an empty
collection (`[]` / `{}`) instead of NULL (nullable columns) or a read error
(non-nullable columns).
An explicit NULL field (`\N`, the default `null_format`) is unchanged and
still deserializes to NULL.
Struct serde is intentionally left unchanged (empty-struct semantics are
ambiguous, tracked
separately).
### Release note
An empty array/map field in Hive text format is now read as an empty
collection instead of NULL
(or a load error for non-nullable columns). An explicit NULL (`\N`) is
unchanged.
### Check List (For Author)
- Test: Unit Test
- Added
`be/test/core/data_type_serde/empty_complex_type_hive_text_test.cpp` with 6
cases
(empty array/map, non-empty array, nullable empty array/map, explicit
`\N` -> NULL).
All 6 pass under ASAN: `[ PASSED ] 6 tests.`
- clang-format (v16) clean on all 3 changed files.
- Behavior changed: Yes (empty nullable array/map field now reads as empty
collection instead of
NULL; `\N` still means NULL).
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
