sunchao commented on code in PR #5889:
URL: https://github.com/apache/datafusion-comet/pull/5889#discussion_r4433048299
##########
native/core/src/execution/planner.rs:
##########
@@ -965,7 +965,12 @@ impl PhysicalPlanner {
ExprStruct::Subquery(expr) => {
let id = expr.id;
let data_type =
to_arrow_datatype(expr.datatype.as_ref().unwrap());
- Ok(Arc::new(Subquery::new(self.exec_context_id, id,
data_type)))
+ if matches!(data_type, DataType::Struct(_)) {
+ let value = Subquery::resolve_struct(self.exec_context_id,
id, &data_type)?;
Review Comment:
Each occurrence of the same struct subquery calls `resolve_struct`, which
serializes the full JVM row and decodes another Arrow struct. Projecting
multiple fields from one reused scalar subquery can therefore keep multiple
complete copies alive in the same task. Could the planner share the resolved
scalar by subquery ID and type within an execution context? Two field
projections from one subquery would make a useful serialization-count test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
