matanper opened a new issue, #69865: URL: https://github.com/apache/doris/issues/69865
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 4.1.4 (VARIANT V2 storage). master is the same: - write: `be/src/storage/segment/variant/v2/variant_shredder.cpp:181-186` canonicalizes a child path into the dotted on-disk namespace (`child = PathInData(child.get_path())`), so each `.` in a key becomes a path part, with no bound on the number of parts; - read: `be/src/storage/segment/variant/v2/variant_assembler.cpp:362` and `:651` assert `DORIS_CHECK_LE(path.get_parts().size(), VARIANT_MAX_NESTING_DEPTH)` (128). ### Description Object nesting depth is bounded when a value is encoded, but a single flat key with many dots is depth 1 there. The shredder then splits it into one path part per dot and writes the path. A key such as `"a.a.a....a"` with more than 128 dots therefore loads successfully, but every later read of that VARIANT column in the segment hits the `DORIS_CHECK_LE` in the assembler: the query fails (release build) or the BE aborts (debug build). The data is accepted and then unreadable, including by compaction. ### Solution - In the shredder, refuse a path of more than `VARIANT_MAX_NESTING_DEPTH` parts with `INVALID_ARGUMENT` at load time, naming the row and (a prefix of) the key. - In the assembler, report a stored path deeper than the limit as `Status::Corruption` instead of a `DORIS_CHECK`, so already-written segments fail cleanly. ### How to Reproduce? We reproduced it with a BE unit test that shreds such a key and reads it back. A SQL sketch of the same input (not run as written): ```sql CREATE TABLE t (k INT, v VARIANT) ... ; -- VARIANT V2 storage -- key with 200 dot-separated parts INSERT INTO t SELECT 1, CAST(concat('{"', repeat('a.', 199), 'a": 1}') AS VARIANT); SELECT v FROM t; -- expected to fail the DORIS_CHECK_LE on the path part count ``` ### Anything Else? Keys like this are unusual, but they are valid JSON and can come from untrusted log input. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
