wangyum opened a new pull request, #6864:
URL: https://github.com/apache/hive/pull/6864
### What changes were proposed in this pull request?
`PartitionProjectionEvaluator.buildQueryForSingleValuedFields` decided
whether to add the `SERDES` join using `foundSD || find(SERDE_PATTERN)`. Since
`foundSD` becomes `true` whenever *any* `sd.*` field is requested, this
unconditionally joined `SERDES` even when no serde field was projected at all.
The fix gates the `SERDES` join on `find(SERDE_PATTERN)` alone, matching the
intent of the comment above it ("if serde fields are selected add join clause
on serdes").
Example: requesting only `sd.location` previously generated
```sql
select ... from "PARTITIONS"
left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID"
left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID"
where ...
```
and now generates
```sql
select ... from "PARTITIONS"
left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID"
where ...
```
This only affects the `DirectSQL` partition-projection code path
(`MetaStoreDirectSql` → `PartitionProjectionEvaluator`); the JDO fallback path
(`TableStoreImpl#getJdoResult`) builds its result via DataNucleus'
`query.setResult(...)` and has no equivalent join logic.
### Why are the changes needed?
The extra `SERDES` join is pure overhead: the SELECT column list and the
`serdeIdNode` inclusion in `roots` were already independently gated by the same
`find(SERDE_PATTERN)` check, so the join never contributed to the returned data
— it only added unnecessary work to every partition-projection query that
touches any storage-descriptor field (a very common case, e.g. requesting just
`sd.location`).
### Does this PR introduce any user-facing change?
No. This is a pure SQL-generation optimization; the returned partition data
is unchanged.
### How was this patch tested?
Added two unit tests to `TestPartitionProjectionEvaluator` that exercise the
real `getPartitionsUsingProjectionList()` entry point with a mocked
`PersistenceManager`/`Query`, capturing the SQL text passed to
`pm.newQuery(...)` and asserting on the join clauses:
- `testSerdesNotJoinedWhenOnlySdFieldsRequested`: projecting only
`sd.location` joins `SDS` but not `SERDES`.
- `testSerdesJoinedWhenSerdeFieldRequested`: projecting
`sd.serdeInfo.serializationLib` joins both `SDS` and `SERDES`.
Verified RED → GREEN: temporarily reverted the fix and confirmed
`testSerdesNotJoinedWhenOnlySdFieldsRequested` fails for the expected reason,
then restored the fix and confirmed all 7 tests in the class pass.
Also ran the full `TestGetPartitionsUsingProjectionAndFilterSpecs`
integration suite (23 cases, including several using `sd.location` projections)
against the embedded Derby metastore to confirm no behavioral regression in
returned partition data.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
