Doris-Breakwater commented on issue #66365: URL: https://github.com/apache/doris/issues/66365#issuecomment-5161175734
Breakwater-GitHub-Analysis-Slot: slot_130eccec93a1 This content is generated by AI for reference only. Initial assessment: **the RowId materialization path is convincingly isolated, but the exact root cause is not yet proven. The strongest code-backed lead is an already-fixed branch-4.1 compatibility issue for tables with `light_schema_change=false`.** The issue currently has no labels. ### Verified facts - The attachment gives a useful path-level reproducer: only projections that defer non-key columns hang; removing `LIMIT` or setting `experimental_topn_lazy_materialization_threshold=0` avoids `VMaterializeNode`. With the default threshold, limit 10 is eligible for `LazyMaterializeTopN`, which replaces deferred columns with `__DORIS_GLOBAL_ROWID_COL__` below TopN and fetches those columns afterward. - The reported build is exactly commit `aec169d20256a788de448f46737b5fb53ee3b4e3`. At that revision, FE does not reject TopN lazy materialization for an OLAP table with `light_schema_change=false`. - On that revision, BE rebuilds the scan `TabletSchema` from FE's `columns_desc` only when the first column has `col_unique_id >= 0`. A non-light-schema-change table uses `col_unique_id=-1`, so the rebuilt schema (including `GLOBAL_ROWID_COL`) is skipped. This is the failure mechanism documented and fixed by [#64441](https://github.com/apache/doris/pull/64441), backported to branch-4.1 by [#64634](https://github.com/apache/doris/pull/64634) after the reported RC commit. The fix falls back to the normal scan path for such tables. - The RC commit also lacks [#62513](https://github.com/apache/doris/pull/62513), backported by [#64628](https://github.com/apache/doris/pull/64628), which propagates a non-zero RowId-fetch RPC response status from `MaterializationOperator`. Without it, the useful BE error can be masked instead of being returned cleanly to the client. - Apache Doris 4.1.3 contains both branch-4.1 fixes above. ### Current judgment If `t_entities_host` has `light_schema_change=false`, this is a high-confidence match for the known RC gap, with the missing RowId-fetch status propagation explaining why the client may not receive the underlying BE failure clearly. However, the attached DDL omits the table `PROPERTIES` section, so that condition is unverified. The evidence does **not** yet establish a new deadlock or a new RowId lookup defect. If the table has `light_schema_change=true`, or the same minimal case still hangs on 4.1.3, the known fix does not explain the report and the blocked runtime location must be captured before assigning a different root cause. ### Missing information Please provide: 1. Exact `SHOW CREATE TABLE t_entities_host` output, including all `PROPERTIES`, especially `light_schema_change`; also state whether this is an existing upgraded table or a table newly created from the posted DDL. 2. The query ID, how long it was allowed to run, the effective query/execution timeout, and whether cancellation succeeds. 3. FE and all three BE log excerpts for that query ID and time window. The most relevant messages contain `field name is invalid`, `multiget_data_v2`, `Row id fetch failed`, `file_mapping not found`, `id_file_map not found`, `rowset_id not found`, or `Materialization Sink node id`. 4. A live runtime profile while blocked, including the `VMaterializeNode`/Materialization operator, `MaxRpcTime`, `MergeResponseTime`, and any `RowIDFetcher` child profiles, plus the complete `EXPLAIN VERBOSE` plan. 5. If it still reproduces on 4.1.3, a thread dump from the coordinator BE and any BE handling `multiget_data_v2` at the same timestamp. ### Recommended next steps 1. First confirm the table property and retry the minimal SQL on 4.1.3 (or another build known to include #64634 and #64628) without disabling the optimization. 2. If 4.1.3 succeeds and the table has `light_schema_change=false`, treat the report as covered by the existing fallback and error-propagation fixes; the current threshold setting remains a valid temporary workaround for the RC build. 3. If it persists with `light_schema_change=true` on 4.1.3, keep the issue open as a distinct bug and use the profile/log/thread evidence above to determine whether it is waiting in the RowId RPC scheduler, inside `read_batch_doris_format_row`, or after RPC completion during response merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
