morningman commented on PR #68280: URL: https://github.com/apache/doris/pull/68280#issuecomment-5750433728
**The root cause in plain words, and where parquet fits in** A nullable BOOLEAN column in the BE is two arrays: the value bytes (one per row, by contract only 0 or 1) and the null map (one per row, "this row is NULL"). Three things line up: 1. `JSON_EXTRACT_BOOL(j, '$.flag')` returns NULL when the key is missing. The cast that implements it allocates the value array, then for NULL rows it only sets the null-map bit and never writes the value byte. The allocation is recycled memory, so that byte keeps whatever was there before, typically characters from an earlier string column (`'A'` = 65). It is like reusing an answer sheet without erasing the old marks. 2. `... OR (a > b)` is evaluated branch-free: `value = left | right`, and the result is flagged not-NULL as soon as the right side is TRUE. For those rows the value becomes `old_mark | 1`, e.g. 65, i.e. a boolean that is neither NULL nor 0/1. The `3, 5, 7, 65, ...` values in the probe query are exactly these bytes. 3. `CASE WHEN c1 THEN ... WHEN c2 THEN ...` also avoids branches and computes the branch to take as `branch = cond_byte * branch_number`. With `cond_byte = 65` it asks for branch 65 of a CASE that has two or three, reads a `ColumnPtr` past the end of the branch vector and dereferences garbage. Depending on what happens to be in that memory the BE either crashes (`SIGSEGV` in `update_result_normal` / `insert_from_with_type`) or silently returns the wrong rows. The fix writes a 0 payload for NULL rows in the jsonb cast (root cause) and makes `VCaseExpr` treat any non-zero condition byte as 1 (so a stray byte from anywhere else can never become an out-of-range branch index again). **Why parquet shows up in the production stack** Parquet is only *where* this expression happened to be evaluated. The CASE lived in the view's `WHERE`, the planner pushed it down as a conjunct of the Hive parquet scan, and since 4.1.4 that scan is FileScannerV2, which runs the conjunct inside the reader right after decoding the columns. That is why the production stack starts at `parquet_scan.cpp: execute_batch_filters` and ends in `VCaseExpr`. The same expression code runs for an internal table inside `SegmentIterator`, with the same bottom frames. The scanner only changes which stale bytes are sitting in the recycled buffers, which is why the crash looked "new in 4.1.4" and went away after flipping an unrelated session variable: 4.1.3 and master carry the same code. **Why the reproduction uses an internal table** It needs no Hive or parquet environment and one query is enough; the expression frames of the stack are identical. The parquet path was verified separately on this branch: the original view over a parquet file read through `FileScannerV2` (`local()` TVF) crashes on the first run without the fix, with the full production stack (`FileScannerV2::_get_block_impl` → `ParquetReader::get_block` → `execute_batch_filters` → `VCaseExpr::update_result_normal` → `insert_from_with_type`), and returns stable, correct results with it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
