devinrsmith opened a new pull request, #18929:
URL: https://github.com/apache/iceberg/pull/18929

   `SortOrder#satisfies` compares the two orders field by field with 
`SortField#satisfies`. That allows a finer transform to stand in for a coarser 
one, such as `ts` for `day(ts)`. This is only valid for the last field of the 
order being satisfied.
   
   When another field follows, rows that tie on the coarse value must still be 
ordered by the next field. Sorting by `[ts, symbol]` orders the rows within a 
day by `ts` first. `symbol` is not ordered within the day, so the data is not 
sorted by `[day(ts), symbol]`. Before this change, `[ts, 
symbol].satisfies([day(ts), symbol])` returned `true`.
   
   This change requires every field of the other order except the last to match 
exactly (`equals`). Only the last field may be satisfied by a finer transform:
   
   | this | other | before | after |
   |---|---|---|---|
   | `[ts]` | `[day(ts)]` | true | true |
   | `[symbol, ts]` | `[symbol, day(ts)]` | true | true |
   | `[ts, symbol]` | `[day(ts), symbol]` | true | **false** |
   
   Impact: `SortOrder#satisfies` is public API and nothing in this repo calls 
it from main code. An engine that uses it to decide whether existing data is 
already sorted could skip a needed sort and write wrongly ordered output. 
Callers affected by this change will now do the sort they previously skipped. 
`SortOrderUtil` calls `Transform#satisfiesOrderOf` directly, stops after the 
first non-matching field, and is unaffected.
   
   Tests: added `TestSortOrder#testSatisfiesTransformFollowedByAnotherField`.
   
   AI disclosure: AI assistance was used in this PR. I reviewed the logic and 
ran `TestSortOrder`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to