Tim Armstrong has posted comments on this change. Change subject: IMPALA-4624: Implement Parquet dictionary filtering ......................................................................
Patch Set 14: (1 comment) http://gerrit.cloudera.org:8080/#/c/5904/14/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java File fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java: Line 243: public boolean isDeterministic() { > Switched this over to isBuiltinRandom/isBuiltinRandomFn. I agree that it is We ran into a similar problem with non-deterministic UDFs when we added expression rewrites. We never provided any guarantees about how non-deterministic UDFs were evaluated, and it was already inconsistent. E.g. partitions have been statically pruned based on UDF output since many releases ago, so we don't guarantee that predicates with a non-deterministic UDF are evaluated against every row anyway. The solution we went before was to change the behaviour (since it wasn't contractual) but provide the enable_expr_rewrites safety valve in case someone was relying on the behaviour. I believe we have a JIRA somewhere to add metadata about whether a UDF is non-determistic. -- To view, visit http://gerrit.cloudera.org:8080/5904 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3a7cc3bd0523fbf3c79bd924219e909ef671cfd7 Gerrit-PatchSet: 14 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Joe McDonnell <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Lars Volker <[email protected]> Gerrit-Reviewer: Marcel Kornacker <[email protected]> Gerrit-Reviewer: Matthew Mulder <[email protected]> Gerrit-Reviewer: Mostafa Mokhtar <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-HasComments: Yes
