AlanConfluent commented on code in PR #28061:
URL: https://github.com/apache/flink/pull/28061#discussion_r3228705859
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/optimize/StreamNonDeterministicUpdatePlanVisitor.java:
##########
@@ -958,6 +966,71 @@ private StreamPhysicalRel visitJoinChild(
return transmitDeterminismRequirement(rel, inputRequireDeterminism);
}
+ private StreamPhysicalRel visitPtf(
+ final StreamPhysicalProcessTableFunction ptf,
+ final ImmutableBitSet requireDeterminism) {
+ final RexCall call = ptf.getCall();
+
+ // Concern 1: PTF function itself is non-deterministic.
+ // requireDeterminism is the set of PTF output columns downstream
requires to be
+ // deterministic (for correct retract matching). All PTF output
columns are assumed to
+ // come from PTF's computation — there are pass-through input columns
potentially, but
+ // they are not considered specially for now. If the PTF is
non-deterministic and any of
+ // those columns are required, we cannot satisfy the requirement.
+ if (!requireDeterminism.isEmpty()) {
+ final Optional<String> ndCall =
FlinkRexUtil.getNonDeterministicCallName(call);
Review Comment:
I agree, that would be ideal. Really, this is true of a normal table
function as well and arguably could also be true for a Scalar returning a Row
-- in fact you could have nested structures where only some nested bit is
non-deterministic. Currently `FunctionDefinition` gives us this crude
mechanism that just says about the whole returned value.
It would be ideal if we could expose some more granular BitSet for the
columns of the returned row. That seems like a more complex thing that would
probably require a FLIP since it would be extending the public API. I can
create a JIRA to track that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
