Doris-Breakwater commented on issue #68112: URL: https://github.com/apache/doris/issues/68112#issuecomment-5708717830
Thanks for the unusually complete reproducer. This is actionable as a BE/VARIANT serialization bug; it is not a type-inference issue, and no additional logs or profile are needed to start the fix. ### Verified from the 4.1.3 source - `ColumnVariant::Subcolumn::serialize_text_json()` delegates the value to its type SerDe ([`column_variant.cpp`](https://github.com/apache/doris/blob/7126cf65d96ebc43fce0906f51e92c1a2ccf24a6/be/src/core/column/column_variant.cpp#L1399-L1425)). - For a `BOOLEAN` subcolumn this reaches `DataTypeNumberSerDe<TYPE_BOOLEAN>::serialize_one_cell_to_json()`. That implementation groups booleans with integers and calls `write_number(data)`, which emits `1`/`0` ([`data_type_number_serde.cpp`](https://github.com/apache/doris/blob/7126cf65d96ebc43fce0906f51e92c1a2ccf24a6/be/src/core/data_type_serde/data_type_number_serde.cpp#L203-L220)). This is the immediate cause of the dense scalar result. - `DataTypeArraySerDe` delegates each element to the nested SerDe ([`data_type_array_serde.cpp`](https://github.com/apache/doris/blob/7126cf65d96ebc43fce0906f51e92c1a2ccf24a6/be/src/core/data_type_serde/data_type_array_serde.cpp#L46-L68)), so `array<boolean>` reaches the same numeric emission path. - Sparse entries are decoded into a temporary typed `Subcolumn` and then sent through `serialize_text_json()` again ([`column_variant.cpp`](https://github.com/apache/doris/blob/7126cf65d96ebc43fce0906f51e92c1a2ccf24a6/be/src/core/column/column_variant.cpp#L2011-L2024)). That explains why the sparse boundary changes a JSONB value's rendering even though the binary sparse value retains its primitive type. - A JSONB subcolumn instead uses `JsonbToJson::jsonb_to_json_string()` ([`data_type_jsonb_serde.cpp`](https://github.com/apache/doris/blob/7126cf65d96ebc43fce0906f51e92c1a2ccf24a6/be/src/core/data_type_serde/data_type_jsonb_serde.cpp#L64-L74)), explaining the correct `true`/`false` control cases. Therefore the report's diagnosis is correct: the stored `BOOLEAN` type is preserved, but VARIANT JSON reconstruction uses a numeric boolean formatter. This is separate from #68016, which changes boolean/numeric least-supertype selection; a boolean-only path never enters that mixed-type merge. There is also an existing `FormatOptions::is_bool_value_num` switch whose documented false behavior is `[true]` rather than `[1]` ([`data_type_serde.h`](https://github.com/apache/doris/blob/7126cf65d96ebc43fce0906f51e92c1a2ccf24a6/be/src/core/data_type_serde/data_type_serde.h#L187-L194)), but the 4.1.3 boolean JSON serializer does not consult it. This looks like the narrowest implementation hook, subject to tests. ### Suggested next step Have VARIANT JSON serialization request JSON boolean literals and make the boolean number SerDe honor that request. Keep the default numeric behavior unchanged for non-VARIANT MySQL/CSV/string output to avoid an unrelated compatibility change. This should cover dense booleans, scalar/root values, nested values, `array<boolean>`, and sparse values through the shared SerDe path. Please add: 1. A `variant_p0` regression for `true` and `false` covering the whole document, path extraction, nested objects, and `array<boolean>`. 2. A sparse-column case using a deliberately small `variant_max_subcolumns_count`, rather than generating 2049 keys, and a JSONB control case on both sides of the boundary. 3. A SerDe unit test proving both settings of `is_bool_value_num`, plus a non-VARIANT compatibility assertion so ordinary BOOLEAN output does not change unintentionally. 4. Validation on current master and branch-4.1 to establish the affected-version/backport matrix. I verified the source path at tag `4.1.3`; I did not independently run a Doris cluster here. Issue metadata currently has no labels or assignee; a BE/VARIANT bug label and owner would be appropriate under the repository's label conventions. Breakwater-GitHub-Analysis-Slot: slot_7b3d8b295425 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
