ravirahman opened a new issue, #37729:
URL: https://github.com/apache/arrow/issues/37729

   ### Describe the bug, including details regarding any error messages, 
version, and platform.
   
   I am running into an issue where inner joins between two tables are 
incorrectly computed. To reproduce this issue, 
[here](https://github.com/apache/arrow/files/12614288/example_tables.zip) is a 
zip file containing two example tables (`table_1.parquet` and 
`table_2.parquet`). I have encountered this problem on pyarrow 10.0.0 - 13.0.0:
   
   The schema for `table_1` is:
   
   ```
   col_1: timestamp[us, tz=UTC]  # All values are the same timestamp 
("2023-09-14 21:43:18.678917")
   col_2: large_string  
   col_3: large_string  # All values are the same ("foo")
   ```
   
   And for `table_2`:
   ```
   col_1: timestamp[us, tz=UTC]  # All values are the same timestamp 
("2023-09-14 21:43:18.678917")
   col_2: large_string
   col_3: large_string  # All values are the same ("foo")
   col_4: large_string  # All values are the same ("bar")
   ```
   
   Below is an example to reproduce the issue I am running in to with inner 
joins:
   
   ```python
   import pyarrow.parquet as pq
   
   table_1 = pq.read_table("table_1.parquet")
   table_2 = pq.read_table("table_2.parquet")
   
   inner_joined = table_1.join(table_2, ["col_1", "col_2", "col_3"], 
join_type="inner",)
   print(len(inner_joined))  # prints 3596 (incorrect)
   
   inner_joined = table_2.join(table_1, ["col_1", "col_2", "col_3"], 
join_type="inner", use_threads=False)
   print(len(inner_joined))  # prints 3601 (incorrect). "A inner join B" should 
be the same as "B inner join A"?
   
   outer_joined = table_1.join(table_2, ["col_1", "col_2", "col_3"], 
join_type="left outer")
   print(len(outer_joined))  # prints 6289 (correct)
   
   # Table 2 is a superset of table 1. Thus, the null count for `col_4` when 
doing a left join should be zero
   print(outer_joined.column("col_4").null_count)  # prints 0 (correct)
   ```
   
   I believe this is a bug but please let me know if I am missing anything.
   
   ### Component(s)
   
   Python


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to