EdwardShen125 opened a new pull request, #1640:
URL: https://github.com/apache/arrow-go/pull/1640

   ### Rationale for this change
   
   `SeekToRow` on a repeated column returns wrong rows without an error when
   the file has V1 data pages and no page index (#1522). Thanks to @hcrosse
   for the root cause and repro: `serializedPageReader.Next` advances
   `rowsSeen` by a V1 page's `num_values`, which counts leaf values, not
   records. On his 300-row `list<int32>` repro, `SeekToRow(150)` returned 243
   rows starting at `[195, 196, 197]`.
   
   ### What changes are included in this PR?
   
   A V1 page's records are now counted from its level-0 repetition levels
   when the column repeats; flat columns keep the `num_values` fast path.
   `rowsSeen` drives both the seek loop and the chunk-termination check.
   
   Records can span page boundaries, so the page a seek lands on can start
   with the tail of a record a previous page already counted. `skipRows`
   discards that tail now. The repetition level buffer is reset per page (V2
   already did this), so a seek can't carry one page's levels into the next.
   
   Encrypted page readers set `maxRepLevel`/`maxDefLevel` like the
   unencrypted one, so they get the same count.
   
   The cost: repetition levels on V1 repeated pages are decoded twice, once
   for the count and once for the values.
   
   ### Are these changes tested?
   
   `TestRecordReaderSeekToRowRepeatedV1WithoutPageIndex` writes `list<int32>`
   with tiny pages and no page index, seeks to rows 0/1/150/299 for both page
   versions, and reads to the end. The V1 legs fail without the change and
   pass with it.
   
   ```sh
   go test -count=1 ./parquet/file/... ./parquet/pqarrow/...
   go test -race -count=1 -run 'SeekToRow' ./parquet/file/... 
./parquet/pqarrow/...
   ```
   
   ### Are there any user-facing changes?
   
   `SeekToRow` returns the requested rows for repeated V1 columns without a
   page index, instead of silently wrong ones. No API change.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to