Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/24590 )
Change subject: IMPALA-9821: Change DataSketches functions to return BINARY ...................................................................... Patch Set 10: (3 comments) http://gerrit.cloudera.org:8080/#/c/24590/6/testdata/workloads/functional-query/queries/QueryTest/datasketches-hll-hive-orc.test File testdata/workloads/functional-query/queries/QueryTest/datasketches-hll-hive-orc.test: http://gerrit.cloudera.org:8080/#/c/24590/6/testdata/workloads/functional-query/queries/QueryTest/datasketches-hll-hive-orc.test@16 PS6, Line 16: > Yeah, Step 1 is basically a test for IMPALA-9482, and this changed fixed th I added an update to the Jira description about the state after BINARY support. It is a bit unlucky that the jira description touched 2 different things: the difference in functions signatures + the immediate issue with ORC that we can't read BINARY columns even if we alter the column type to STRING (which is solved a long time ago). The replicate the exact original scenario, after the Hive CTAS Hive or Impala should do an alter table that changes "sketch" to STRING from BINARY. Due to the signature change in ds_hll_estimate you will also need to add a cast to step 2: cast(sketch as binary). This will exercise the original "write as BINARY in Hive, read as STRING in Impala" path. http://gerrit.cloudera.org:8080/#/c/24590/10/testdata/workloads/functional-query/queries/QueryTest/datasketches-hll-hive-orc.test File testdata/workloads/functional-query/queries/QueryTest/datasketches-hll-hive-orc.test: http://gerrit.cloudera.org:8080/#/c/24590/10/testdata/workloads/functional-query/queries/QueryTest/datasketches-hll-hive-orc.test@14 PS10, Line 14: e ORC fix (IMPALA-9482). I wouldn't refer to IMPALA-9482 as ORC fix, as it is the epic for the whole BINARY support. I would phrase this like: "Before BINARY support (IMPALA-9482) reading an ORC BINARY column as STRING resulted in <error msg>." In the commit message I wouldn't mention the error msg, just that datasketches-hll-hive-orc.test adds a regression test for the original issue in the Jira description, which was solved by IMPALA-9482 but no test was added for this scenario. http://gerrit.cloudera.org:8080/#/c/24590/10/testdata/workloads/functional-query/queries/QueryTest/datasketches-hll-hive-orc.test@29 PS10, Line 29: ds_hll_estimate(sketch) >= 1 Why not check the exact type and do the same with Hive? This would show that Impala works the same with ds_hll_estimate as Hive. -- To view, visit http://gerrit.cloudera.org:8080/24590 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id4a6b54089dd356e37257bc24adeb1eb98e82c25 Gerrit-Change-Number: 24590 Gerrit-PatchSet: 10 Gerrit-Owner: Mihaly Szjatinya <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Mihaly Szjatinya <[email protected]> Gerrit-Reviewer: Peter Rozsa <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Wed, 23 Sep 2026 15:17:40 +0000 Gerrit-HasComments: Yes
