Zoltán Borók-Nagy created IMPALA-15171:
------------------------------------------
Summary: NULL out INPUT__FILE__NAME in IcebergDeleteNode if not
needed downstream
Key: IMPALA-15171
URL: https://issues.apache.org/jira/browse/IMPALA-15171
Project: IMPALA
Issue Type: Improvement
Components: Backend, Frontend
Reporter: Zoltán Borók-Nagy
pala's IcebergScanPlanner.addDataVirtualPositionSlots() materializes the
INPUT__FILE__NAME (file path) virtual column on the data-scan tuple so the
IcebergDeleteJoinNode can use it as a position-delete join key.
When the user's query doesn't otherwise need it, the slot is still carried in
the join's output tuple and propagates through every downstream EXCHANGE, where
the (long, per-data-file) file-path string is deep-copied and serialized on
every surviving row — pure overhead.
*Proposed fix*
Have the planner detect whether it created the INPUT__FILE__NAME slot itself
(vs. the user/DML already referencing it) and, if so, tell the backend to NULL
it out on the delete join's output rows.
{*}Frontend{*}:
- In addDataVirtualPositionSlots(), probe
analyzer.getSlotDescriptor(resolvedPath) == null before adding the slot. All
user/DELETE/UPDATE/MERGE references to INPUT__FILE__NAME are registered during
analysis (before planning), so "absent" reliably means "no operator above the
join needs it."
- If planner-created, pass the slot id to IcebergDeleteJoinNode via a new
TIcebergDeleteNode field.
{*}Backend{*}:
- In iceberg-delete-node.cc ProcessProbeBatch(), SetNull() the flagged slot
on output rows. This is safe (node never re-reads emitted rows) and NULLing a
var-len string slot skips both the deep copy and the wire serialization
(Tuple::DeepCopyVarlenData / VarlenByteSize guard on the null bit).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
