waterWang opened a new pull request, #19200:
URL: https://github.com/apache/pinot/pull/19200
## Description
Fixes a bug in `LookupJoinOperator` where a lookup join returns 0 rows when
a dimension table primary-key component is supplied as a literal in the join
condition.
### Root Cause
When a join condition like `dim.currency = 'gbp' AND dim.rate_start_date =
fact.rate_start_date` is used, Calcite's `analyzeCondition()` classifies the
`dim.currency = 'gbp'` part as a non-equi condition (right-column-to-literal
equality, not a left-right column equality). The `LookupJoinOperator` then
builds the lookup key using only the left-key columns (`rate_start_date`),
resulting in an incomplete key that doesn't match the dimension table's
composite primary key (`[currency, rate_start_date]`). The lookup returns
`null`, and the join produces 0 rows.
### Fix
In `LookupJoinOperator`'s constructor, analyze the non-equi conditions to
find `=` equality expressions where one operand is an `InputRef` (pointing to a
right-side column) and the other is a `Literal`. When the right-side column is
part of the dimension table's primary key, use the literal value as a key
component. This ensures the lookup key is complete and matches the dimension
table's primary key.
### Key changes
- **`LookupJoinOperator.java`**: Add `_keyColumnCount`, `_keyColumnLeftIds`,
and `_keyColumnLiteralValues` fields. In the constructor, extract literal-based
key components from non-equi conditions. Update `getKey()` and `fillKey()` to
build the complete primary key including literals. Update
`buildJoinedDataBlockSemi` and `buildJoinedDataBlockAnti` to use the full key
size.
- **`LookupJoin.json`**: Add a test case `lookup_join_literal_key` that
reproduces the bug.
### Test
```
SELECT /*+ joinOptions(join_strategy='lookup') */ {dim_tbl}.currency,
{dim_tbl}.rate
FROM {fact_tbl} JOIN {dim_tbl}
ON {dim_tbl}.currency = 'gbp' AND {dim_tbl}.rate_start_date =
{fact_tbl}.rate_start_date
```
Expected: `[["gbp", 125]]` (1 row)
Before fix: `[]` (0 rows)
Closes #19188
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
