kosiew commented on code in PR #22451:
URL: https://github.com/apache/datafusion/pull/22451#discussion_r3297424807
##########
datafusion/physical-plan/src/joins/hash_join/partitioned_hash_eval.rs:
##########
@@ -337,7 +335,31 @@ impl PhysicalExpr for HashTableLookupExpr {
}
}
}
-
+ #[cfg(feature = "proto")]
+ fn try_to_proto(
+ &self,
+ _ctx:
&datafusion_physical_expr_common::physical_expr::proto_encode::PhysicalExprEncodeCtx<'_>,
+ ) -> Result<Option<datafusion_proto_models::protobuf::PhysicalExprNode>> {
+ use datafusion_proto_models::protobuf;
+ use datafusion_proto_models::protobuf::physical_expr_node::ExprType;
+
+ // HashTableLookupExpr holds a runtime Arc<Map> (the build-side hash
+ // table) that cannot be serialized. We replace it with lit(true).
+ //
+ // This is safe because dynamic filtering is a performance optimisation
+ // only — lit(true) passes all rows so correctness is preserved.
+ // When a serialized plan is re-executed, HashJoinExec reconstructs
+ // fresh dynamic filters at runtime anyway.
Review Comment:
This comment is helpful and explains why `lit(true)` is correctness-safe.
One small optional tweak would be to mirror Jay's pre-execution and
post-execution wording a bit more directly, and maybe mention `reset_state`
since that is where the runtime reconstruction behavior is tied together.
Not blocking from my side, just a readability suggestion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
