ravirahman opened a new issue, #37729: URL: https://github.com/apache/arrow/issues/37729
### Describe the bug, including details regarding any error messages, version, and platform. I am running into an issue where inner joins between two tables are incorrectly computed. To reproduce this issue, [here](https://github.com/apache/arrow/files/12614288/example_tables.zip) is a zip file containing two example tables (`table_1.parquet` and `table_2.parquet`). I have encountered this problem on pyarrow 10.0.0 - 13.0.0: The schema for `table_1` is: ``` col_1: timestamp[us, tz=UTC] # All values are the same timestamp ("2023-09-14 21:43:18.678917") col_2: large_string col_3: large_string # All values are the same ("foo") ``` And for `table_2`: ``` col_1: timestamp[us, tz=UTC] # All values are the same timestamp ("2023-09-14 21:43:18.678917") col_2: large_string col_3: large_string # All values are the same ("foo") col_4: large_string # All values are the same ("bar") ``` Below is an example to reproduce the issue I am running in to with inner joins: ```python import pyarrow.parquet as pq table_1 = pq.read_table("table_1.parquet") table_2 = pq.read_table("table_2.parquet") inner_joined = table_1.join(table_2, ["col_1", "col_2", "col_3"], join_type="inner",) print(len(inner_joined)) # prints 3596 (incorrect) inner_joined = table_2.join(table_1, ["col_1", "col_2", "col_3"], join_type="inner", use_threads=False) print(len(inner_joined)) # prints 3601 (incorrect). "A inner join B" should be the same as "B inner join A"? outer_joined = table_1.join(table_2, ["col_1", "col_2", "col_3"], join_type="left outer") print(len(outer_joined)) # prints 6289 (correct) # Table 2 is a superset of table 1. Thus, the null count for `col_4` when doing a left join should be zero print(outer_joined.column("col_4").null_count) # prints 0 (correct) ``` I believe this is a bug but please let me know if I am missing anything. ### Component(s) Python -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
