[
https://issues.apache.org/jira/browse/IMPALA-15171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltán Borók-Nagy reassigned IMPALA-15171:
------------------------------------------
Assignee: Zoltán Borók-Nagy
> NULL out INPUT__FILE__NAME in IcebergDeleteNode if not needed downstream
> ------------------------------------------------------------------------
>
> Key: IMPALA-15171
> URL: https://issues.apache.org/jira/browse/IMPALA-15171
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend, Frontend
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: impala-iceberg, impala-iceberg-active-backlog
>
> pala's IcebergScanPlanner.addDataVirtualPositionSlots() materializes the
> INPUT__FILE__NAME (file path) virtual column on the data-scan tuple so the
> IcebergDeleteJoinNode can use it as a position-delete join key.
> When the user's query doesn't otherwise need it, the slot is still carried in
> the join's output tuple and propagates through every downstream EXCHANGE,
> where the (long, per-data-file) file-path string is deep-copied and
> serialized on every surviving row — pure overhead.
> *Proposed fix*
> Have the planner detect whether it created the INPUT__FILE__NAME slot itself
> (vs. the user/DML already referencing it) and, if so, tell the backend to
> NULL it out on the delete join's output rows.
> {*}Frontend{*}:
> - In addDataVirtualPositionSlots(), probe
> analyzer.getSlotDescriptor(resolvedPath) == null before adding the slot. All
> user/DELETE/UPDATE/MERGE references to INPUT__FILE__NAME are registered
> during analysis (before planning), so "absent" reliably means "no operator
> above the join needs it."
> - If planner-created, pass the slot id to IcebergDeleteJoinNode via a new
> TIcebergDeleteNode field.
> {*}Backend{*}:
> - In iceberg-delete-node.cc ProcessProbeBatch(), SetNull() the flagged slot
> on output rows. This is safe (node never re-reads emitted rows) and NULLing a
> var-len string slot skips both the deep copy and the wire serialization
> (Tuple::DeepCopyVarlenData / VarlenByteSize guard on the null bit).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
