Doris-Breakwater commented on issue #67625:
URL: https://github.com/apache/doris/issues/67625#issuecomment-5571578387
Initial triage: this looks like a valid replay-related bug, but the
root-cause explanation in the issue does not match the current master code. The
likely scope is broader than external-catalog-backed views.
### Verified from current master
- On commit `82646c38c0069eee4b1704eafef416f969bd4b93`,
`CreatePolicyCommand.validate()` does reject every `UnboundSlot` for which
`tableIf.getColumn(slot.getName())` returns null.
- `Table.nameToColumn` is a derived, non-serialized field. The no-argument
replay constructor initializes it empty, while `fullSchema` is deserialized.
- `Table.gsonPostProcess()` rebuilds `nameToColumn` from `fullSchema` using
`Column.getName()`.
- `View` overrides `gsonPostProcess()` but does **not** call
`super.gsonPostProcess()`. The Gson post-process adapter invokes the overriding
method, so the base-table map rebuild is skipped for a replayed `View`.
- The stated `defineName` explanation is not evidenced by this code:
`Column.defineName` is not serialized, and `BaseViewInfo.createFinalCols()`
creates the persisted view columns with `new Column(output.getName(), ...)`. I
found no view-creation assignment that would persist a qualified source
reference as `defineName`.
- The observed asymmetry is consistent with the implementation:
`LogicalView.computeOutput()` uses `view.getFullSchema()`, whereas CREATE ROW
POLICY uses `getColumn()`. Existing view row-policy regression coverage creates
the policy without an FE restart, and `CreateViewTest` only checks
`getColumn()` on live (not serialized/replayed) views.
### Current assessment
The strongest code-backed hypothesis is that after Gson metadata replay the
view still has its `fullSchema`, but its `nameToColumn` map remains empty
because `View.gsonPostProcess()` skips the superclass reconstruction. If so,
`getColumn("col")` fails for **any** replayed persisted view, including a view
over an internal table; the external catalog is incidental rather than causal.
The proposed `getFullSchema()` case-insensitive scan would unblock this
specific DDL validation, but by itself it would leave the documented `Table`
invariant (`fullSchema` and `nameToColumn` contain the columns) broken for
replayed views and could hide failures in other `getColumn()` callers. The
underlying replay reconstruction should be fixed first (most directly, make
`View.gsonPostProcess()` perform the superclass post-processing), then decide
whether CREATE POLICY should additionally resolve against the visible schema
contract.
This is a fail-closed administration bug: the new policy creation reports an
error. The source path suggests existing stored predicates are later merged
against analyzed plan output rather than this map, but runtime enforcement
after restart still needs an end-to-end restricted-user test before that claim
is treated as verified.
### Missing evidence / requested checks
1. Please provide the exact Doris build/commit SHA; `master` is moving.
2. Run the same restart sequence with `CREATE VIEW ... AS SELECT col FROM
internal_db.internal_table`. If it also fails, that confirms the broader
replay-map hypothesis.
3. For both controls, capture sanitized pre/post-replay values for
`view.getFullSchema()` column names, `view.getColumns()`, and
`view.getColumn("col")`. If a non-null `defineName` is observed, also show
where it is populated or persisted.
4. Distinguish image load from edit-log-only replay if they behave
differently.
5. Create a policy before restart and, after restart, verify `SHOW ROW
POLICY` plus the filtered query as the restricted user. This confirms
persistence and enforcement separately from CREATE-time validation.
### Recommended next steps
1. Add a focused FE unit test that round-trips a `View` through the
production Gson/Table serialization path and asserts that `getFullSchema()` and
case-insensitive `getColumn()` agree after deserialization.
2. Add an FE-restart regression covering CREATE ROW POLICY on a view over an
internal table; add the external-catalog variant only if it demonstrates a
distinct path.
3. Fix the replay invariant, then retain/add the CREATE POLICY negative test
for a truly missing column.
4. No labels are currently set. Suggested labels are `kind/bug-fix`,
`area/auth`, and `area/nereids`; catalog/metadata ownership may also be
appropriate because the likely defect is in view replay.
I could not execute FE tests in the configured local checkout because the
repository-required `thirdparty/installed/bin/protoc` is absent, so the replay
mechanism above is verified by static tracing but still needs the focused
serialization/restart test.
Breakwater-GitHub-Analysis-Slot: slot_3c8221f52b77
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
