zeroshade commented on code in PR #37112:
URL: https://github.com/apache/arrow/pull/37112#discussion_r1293809736
##########
go/parquet/internal/utils/bit_writer.go:
##########
@@ -74,18 +79,19 @@ type BitWriter struct {
// NewBitWriter initializes a new bit writer to write to the passed in
interface
// using WriteAt to write the appropriate offsets and values.
-func NewBitWriter(w io.WriterAt) *BitWriter {
+func NewBitWriter(w WriterAtWithLen) *BitWriter {
return &BitWriter{wr: w}
}
-// ReserveBytes reserves the next aligned nbytes, skipping them and returning
+// SkipBytes reserves the next aligned nbytes, skipping them and returning
// the offset to use with WriteAt to write to those reserved bytes. Used for
// RLE encoding to fill in the indicators after encoding.
-func (b *BitWriter) ReserveBytes(nbytes int) int {
+func (b *BitWriter) SkipBytes(nbytes int) (int, error) {
b.Flush(true)
ret := b.byteoffset
b.byteoffset += nbytes
- return ret
+ b.wr.Reserve(b.byteoffset)
Review Comment:
Because the `BitWriter` only ever calls `WriteAt`, the buffer's `pos`
variable never updates. (Only `Write` updates the internal `pos`, i.e.
position). When we call `Reserve` it verifies that `pos + nbytes < capacity`.
So the `BitWriter` needs to always call `Reserve` with the full `b.byteoffset`
in order to ensure that the bytes get reserved correctly. So it won't end up
allocating more than we want to (aside from the fact that we always round up to
the next power of two when allocating with reserve so that we don't have too
many allocations)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
