[
https://issues.apache.org/jira/browse/DRILL-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15243656#comment-15243656
]
ASF GitHub Bot commented on DRILL-4539:
---------------------------------------
Github user amansinha100 commented on a diff in the pull request:
https://github.com/apache/drill/pull/462#discussion_r59941224
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTableConfig.java
---
@@ -22,25 +22,36 @@
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonTypeName;
+import org.apache.drill.exec.physical.impl.join.JoinUtils.JoinComparator;
@JsonTypeName("hashtable-config")
public class HashTableConfig {
- static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(HashTableConfig.class);
-
private final int initialCapacity;
private final float loadFactor;
private final NamedExpression[] keyExprsBuild;
private final NamedExpression[] keyExprsProbe;
+ private final JoinComparator[] comparators;
@JsonCreator
public HashTableConfig(@JsonProperty("initialCapacity") int
initialCapacity, @JsonProperty("loadFactor") float loadFactor,
@JsonProperty("keyExprsBuild") NamedExpression[]
keyExprsBuild,
- @JsonProperty("keyExprsProbe") NamedExpression[]
keyExprsProbe) {
+ @JsonProperty("keyExprsProbe") NamedExpression[]
keyExprsProbe,
+ @JsonProperty("comparators") JoinComparator[]
comparators) {
this.initialCapacity = initialCapacity;
this.loadFactor = loadFactor;
this.keyExprsBuild = keyExprsBuild;
this.keyExprsProbe = keyExprsProbe;
+ if (comparators != null) {
+ this.comparators = comparators;
+ } else {
+ // default is nulls are equal
--- End diff --
It might seem confusing at first (it was to me) to consider the
nulls-are-equal as the default case since that's not true for joins; however it
is true for a group-by operation. Could you have the caller pass in explicit
comparator such that the HashTableConfig does not have to decide ? The
HashAggregate could instantiate the hash table with an IS_NOT_DISTINCT_FROM
comparator. What do you think ? (one minor point is that the 'JoinComparator'
is now being used for both Joins and Aggregates...we could call it Comparator
or something generic).
> Add support for Null Equality Joins
> -----------------------------------
>
> Key: DRILL-4539
> URL: https://issues.apache.org/jira/browse/DRILL-4539
> Project: Apache Drill
> Issue Type: Improvement
> Reporter: Jacques Nadeau
> Assignee: Venki Korukanti
>
> Tableau frequently generates queries similar to this:
> {code}
> SELECT `t0`.`city` AS `city`,
> `t2`.`X_measure__B` AS `max_Calculation_DFIDBHHAIIECCJFDAG_ok`,
> `t0`.`state` AS `state`,
> `t0`.`sum_stars_ok` AS `sum_stars_ok`
> FROM (
> SELECT `business`.`city` AS `city`,
> `business`.`state` AS `state`,
> SUM(`business`.`stars`) AS `sum_stars_ok`
> FROM `mongo.academic`.`business` `business`
> GROUP BY `business`.`city`,
> `business`.`state`
> ) `t0`
> INNER JOIN (
> SELECT MAX(`t1`.`X_measure__A`) AS `X_measure__B`,
> `t1`.`city` AS `city`,
> `t1`.`state` AS `state`
> FROM (
> SELECT `business`.`city` AS `city`,
> `business`.`state` AS `state`,
> `business`.`business_id` AS `business_id`,
> SUM(`business`.`stars`) AS `X_measure__A`
> FROM `mongo.academic`.`business` `business`
> GROUP BY `business`.`city`,
> `business`.`state`,
> `business`.`business_id`
> ) `t1`
> GROUP BY `t1`.`city`,
> `t1`.`state`
> ) `t2` ON (((`t0`.`city` = `t2`.`city`) OR ((`t0`.`city` IS NULL) AND
> (`t2`.`city` IS NULL))) AND ((`t0`.`state` = `t2`.`state`) OR ((`t0`.`state`
> IS NULL) AND (`t2`.`state` IS NULL))))
> {code}
> If you look at the join condition, you'll note that the join condition is an
> equality condition which also allows null=null. We should add a planning
> rewrite rule and execution join option to allow null equality so that we
> don't treat this as a cartesian join.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
