mihaibudiu commented on code in PR #4848: URL: https://github.com/apache/calcite/pull/4848#discussion_r3025075110
########## arrow/src/main/java/org/apache/calcite/adapter/arrow/ConditionToken.java: ########## @@ -0,0 +1,100 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to you under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.calcite.adapter.arrow; + +import com.google.common.collect.ImmutableList; + +import org.checkerframework.checker.nullness.qual.Nullable; + +import java.util.List; + +import static java.util.Objects.requireNonNull; + +/** + * A structured representation of a single Gandiva predicate condition. + * + * <p>A condition is either unary (e.g. {@code IS NULL}) or binary + * (e.g. {@code =}, {@code <}). Unary conditions have a field name + * and operator; binary conditions additionally have a literal value + * and its type. + * + * <p>This class replaces the raw {@code List<String>} representation Review Comment: In general JavaDoc does not need to discuss things that do no longer exists. I think you can just remove this comment. ########## arrow/src/test/java/org/apache/calcite/adapter/arrow/ArrowAdapterTest.java: ########## @@ -962,6 +1028,34 @@ static void initializeArrowState(@TempDir Path sharedTempDir) .explainContains(plan); } + /** When a filter condition exceeds the CNF node limit, the Arrow adapter + * falls back to the Enumerable convention (EnumerableCalc) instead of + * using ArrowFilter. The query should still return correct results. */ + @Test void testCnfExceedsLimitFallsBackToEnumerable() { Review Comment: I am assuming you have validated these Arrow plans somehow. ########## arrow/src/main/java/org/apache/calcite/adapter/arrow/ArrowTranslator.java: ########## @@ -61,13 +61,30 @@ public static ArrowTranslator create(RexBuilder rexBuilder, return new ArrowTranslator(rexBuilder, rowType); } - List<String> translateMatch(RexNode condition) { - List<RexNode> disjunctions = RelOptUtil.disjunctions(condition); - if (disjunctions.size() == 1) { - return translateAnd(disjunctions.get(0)); - } else { - throw new UnsupportedOperationException("Unsupported disjunctive condition " + condition); + /** The maximum number of nodes allowed during CNF conversion. + * + * <p>If exceeded, {@link RexUtil#toCnf(RexBuilder, int, RexNode)} returns + * the original expression unchanged, which may cause the subsequent + * translation to Gandiva predicates to fail with an + * {@link UnsupportedOperationException}. That exception is caught by + * {@link ArrowRules.ArrowFilterRule#onMatch}, which silently skips the Review Comment: You are making here some assumptions about who is calling this code which may not hold; in general, when you write a library, you don't know how it will be used. You can be more precise by saying that this is a possible path: "when invoked by the module to convert to Arrow..." ########## arrow/src/main/java/org/apache/calcite/adapter/arrow/ArrowTable.java: ########## @@ -184,6 +180,26 @@ private static RelDataType deduceRowType(Schema schema, return builder.build(); } + /** Parses a single {@link ConditionToken} into a Gandiva {@link TreeNode}. */ + private TreeNode parseSingleCondition(ConditionToken token) { Review Comment: "parse" is probably no longer appropriate, perhaps "convertConditionToGandiva?" ########## arrow/src/main/java/org/apache/calcite/adapter/arrow/ArrowTranslator.java: ########## @@ -61,13 +61,30 @@ public static ArrowTranslator create(RexBuilder rexBuilder, return new ArrowTranslator(rexBuilder, rowType); } - List<String> translateMatch(RexNode condition) { - List<RexNode> disjunctions = RelOptUtil.disjunctions(condition); - if (disjunctions.size() == 1) { - return translateAnd(disjunctions.get(0)); - } else { - throw new UnsupportedOperationException("Unsupported disjunctive condition " + condition); + /** The maximum number of nodes allowed during CNF conversion. + * + * <p>If exceeded, {@link RexUtil#toCnf(RexBuilder, int, RexNode)} returns + * the original expression unchanged, which may cause the subsequent + * translation to Gandiva predicates to fail with an + * {@link UnsupportedOperationException}. That exception is caught by + * {@link ArrowRules.ArrowFilterRule#onMatch}, which silently skips the + * Arrow convention and falls back to an Enumerable plan. */ + private static final int MAX_CNF_NODE_COUNT = 256; Review Comment: Ideally this would not be a constant, but a parameter of translateMatch, similar to toCNF, but I don't know if that is possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
