hcrosse opened a new pull request, #1647:
URL: https://github.com/apache/arrow-go/pull/1647

   ### Rationale for this change
   
   Follows up on #1588. Without an offset index, seeking a chunk whose 
dictionary page sits at `data_page_offset` reads and decompresses the 
dictionary twice. When `dictionary_page_offset` exceeds `data_page_offset`, 
`GetDictionaryPage` reports a generic page header error instead of naming the 
offsets.
   
   ### What changes are included in this PR?
   
   `serializedPageReader` now caches the decoded dictionary page metadata 
alongside `dictPageBuffer` so `GetDictionaryPage` reuses the already decoded 
buffer instead of reading it again from the stream, clearing the cache on 
`Reset` and before resizing `dictPageBuffer`.
   
   `GetDictionaryPage` now checks `span < 0` explicitly and returns an error 
naming `dictionary_page_offset` and `data_page_offset`.
   
   ### Are these changes tested?
   
   Yes. I added tests verifying that seeking reads the dictionary once, that a 
failed subsequent dictionary decode invalidates the cache, and that invalid 
offset ordering reports both offsets.
   
   ### Are there any user-facing changes?
   
   Yes. Invalid dictionary offset ordering now produces an error naming both 
offsets. There is no API change.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to