Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/21109 )
Change subject: IMPALA-12872: Use Calcite for optimization - part 1: simple queries ...................................................................... Patch Set 20: (10 comments) http://gerrit.cloudera.org:8080/#/c/21109/20//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21109/20//COMMIT_MSG@44 PS20, Line 44: Can you add a paragraph about type support? E.g. - all Impala types are supported with the exception of: - STRING is represented as VARCHAR(INT_MAX) - complex types are not supported - implemented in ImpalaTypeSystemImpl and ImpalaTypeConverter http://gerrit.cloudera.org:8080/#/c/21109/20/java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeSystemImpl.java File java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeSystemImpl.java: http://gerrit.cloudera.org:8080/#/c/21109/20/java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeSystemImpl.java@40 PS20, Line 40: 15 Can you add more comments about what precision means in different cases? For example I am pretty confused about float/double. http://gerrit.cloudera.org:8080/#/c/21109/20/java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeSystemImpl.java@48 PS20, Line 48: private static final int DEFAULT_FLOAT_PRECISION = 7; RelDataTypeSystemImpl uses 15 for float too. What is the difference behind overriding this? https://github.com/apache/calcite/blob/cc1d46a4c4f88962c059e4ad0689ddfbb784ea96/core/src/main/java/org/apache/calcite/rel/type/RelDataTypeSystemImpl.java#L101 http://gerrit.cloudera.org:8080/#/c/21109/20/java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeSystemImpl.java@86 PS20, Line 86: getMaxPrecision It looks scary that for some types getDefaultPrecision calls getMaxPrecision, while for other it is the other way around. I think that the clearest would be to only call getMaxPrecision from getDefaultPrecision http://gerrit.cloudera.org:8080/#/c/21109/20/java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeSystemImpl.java@87 PS20, Line 87: case CHAR: Shouldn't this return -1, as the user is expected to provide the precision? https://github.com/apache/calcite/blob/cc1d46a4c4f88962c059e4ad0689ddfbb784ea96/core/src/main/java/org/apache/calcite/rel/type/RelDataTypeSystem.java#L43 http://gerrit.cloudera.org:8080/#/c/21109/20/java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeSystemImpl.java@89 PS20, Line 89: case VARCHAR: same as for CHAR http://gerrit.cloudera.org:8080/#/c/21109/20/java/calcite-planner/src/main/java/org/apache/impala/calcite/validate/ImpalaConformance.java File java/calcite-planner/src/main/java/org/apache/impala/calcite/validate/ImpalaConformance.java: http://gerrit.cloudera.org:8080/#/c/21109/20/java/calcite-planner/src/main/java/org/apache/impala/calcite/validate/ImpalaConformance.java@121 PS20, Line 121: @Override public boolean allowGeometry() { : return true; : } This sounds great, but I think that it is not true yet. Even HMS doesn't have geometry columns AFAIK. Btw does Hive have a similar Conformance class? I couldn't find it. http://gerrit.cloudera.org:8080/#/c/21109/20/testdata/workloads/functional-query/queries/QueryTest/calcite.test File testdata/workloads/functional-query/queries/QueryTest/calcite.test: http://gerrit.cloudera.org:8080/#/c/21109/20/testdata/workloads/functional-query/queries/QueryTest/calcite.test@3 PS20, Line 3: alltypes a smaller table like alltypestiny could be queried so that the results could be checked http://gerrit.cloudera.org:8080/#/c/21109/20/testdata/workloads/functional-query/queries/QueryTest/calcite.test@15 PS20, Line 15: drop table if exists calcite_alltypes; This shouldn't be needed with unique_database http://gerrit.cloudera.org:8080/#/c/21109/20/testdata/workloads/functional-query/queries/QueryTest/calcite.test@47 PS20, Line 47: decimal_tbl Can you also check the other missing types? For binary: binary_tbl For date: date_tbl For char, varchar see https://github.com/apache/impala/blob/61ceb16d880a7be07241f682138bfb286ec2a80e/testdata/workloads/functional-query/queries/QueryTest/chars.test#L19 -- To view, visit http://gerrit.cloudera.org:8080/21109 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I453fd75b7b705f4d7de1ed73c3e24cafad0b8c98 Gerrit-Change-Number: 21109 Gerrit-PatchSet: 20 Gerrit-Owner: Steve Carlin <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Steve Carlin <[email protected]> Gerrit-Comment-Date: Tue, 16 Apr 2024 14:48:02 +0000 Gerrit-HasComments: Yes
