Steve Carlin has uploaded a new patch set (#4). (
http://gerrit.cloudera.org:8080/24257 )
Change subject: IMPALA-14940: Calcite planner: handle broadcast and shuffle
hints.
......................................................................
IMPALA-14940: Calcite planner: handle broadcast and shuffle hints.
These hints have been added to the parser, rules, and physical node
converter in a similar way that the straight join was processed.
Other added changes to handle all hints include:
- The straight join was only being handled at the beginning of the query,
e.g. "select straight_join ...". The broadcast and shuffle hints are
placed near the JOIN keyword. Calcite had this mechanism in place, but
it needed to be added and placed in the correct position for the validator
to handle it.
- The Calcite hints are handled with "[]" or "/*+ */", and this code was
also added.
- Some joins were not parsing correctly that are now fixed. Impala allows
joins that do not have the ON clause or even a comma separator. For instance,
this SQL statement now passes in Impala that did not pass in Calcite:
"select min(t1.`1` + t2.`1`) from t t1 join t t2;"
Some logic was added to the parser to handle this. The "joinType" is treated
as a "COMMA" join type since any join condition will exist in the WHERE clause
and not attached to the SqlJoin directly.
Testing: While no direct tests are added, this is one step among many that
will ensure we can do a Jenkins run without the original planner as the
fallback planner. Any test that tests shuffle and broadcast will succeed now.
Also, the above query without the ON clause is in the test framework and this
query, among others, will also no longer fallback to the original planner.
Change-Id: Ie5a932cc09d3d370aa627228d0e226a92fa03168
---
M java/calcite-planner/src/main/codegen/templates/Parser.jj
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaJoinRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/ImpalaCoreRules.java
M
testdata/workloads/functional-query/queries/QueryTest/overlap_min_max_filters_on_sorted_columns.test
M testdata/workloads/functional-query/queries/QueryTest/spilling.test
M tests/query_test/test_hbo.py
M tests/query_test/test_spilling.py
7 files changed, 81 insertions(+), 28 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/24257/4
--
To view, visit http://gerrit.cloudera.org:8080/24257
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie5a932cc09d3d370aa627228d0e226a92fa03168
Gerrit-Change-Number: 24257
Gerrit-PatchSet: 4
Gerrit-Owner: Steve Carlin <[email protected]>
