[GitHub] [arrow-datafusion] liukun4515 commented on a diff in pull request #2721: Add additional data types are supported in hash join

[GitBox]

liukun4515 commented on code in PR #2721:
URL: https://github.com/apache/arrow-datafusion/pull/2721#discussion_r902236962


##########
datafusion/core/src/physical_plan/hash_join.rs:
##########
@@ -1054,6 +1110,116 @@ fn equal_rows(
             DataType::LargeUtf8 => {
                 equal_rows_elem!(LargeStringArray, l, r, left, right, 
null_equals_null)
             }
+            DataType::Decimal(_, lscale) => match r.data_type() {

Review Comment:
   I create two table in the spark，
   ```
   spark-sql> desc t1;
   c1                      decimal(10,3)
   
   spark-sql> desc t2;
   c1                      decimal(10,4)
   
   and join two table with the eq condition: t1.c1 = t2.c1, get the plan
   
   spark-sql> explain select * from t1,t2 where t1.c1 = t2.c1;
   == Physical Plan ==
   AdaptiveSparkPlan isFinalPlan=false
   +- SortMergeJoin [cast(c1#111 as decimal(11,4))], [cast(c1#112 as 
decimal(11,4))], Inner
      :- Sort [cast(c1#111 as decimal(11,4)) ASC NULLS FIRST], false, 0
      :  +- Exchange hashpartitioning(cast(c1#111 as decimal(11,4)), 200), 
ENSURE_REQUIREMENTS, [id=#293]
      :     +- Filter isnotnull(c1#111)
      :        +- Scan hive default.t1 [c1#111], HiveTableRelation 
[`default`.`t1`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: 
[c1#111], Partition Cols: []]
      +- Sort [cast(c1#112 as decimal(11,4)) ASC NULLS FIRST], false, 0
         +- Exchange hashpartitioning(cast(c1#112 as decimal(11,4)), 200), 
ENSURE_REQUIREMENTS, [id=#294]
            +- Filter isnotnull(c1#112)
               +- Scan hive default.t2 [c1#112], HiveTableRelation 
[`default`.`t2`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: 
[c1#112], Partition Cols: []]
   ```
   From the from, we can get the the t1.c1 and t2.c1 will be casted to coerced 
type(decimal(11,4));



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org