zeroshade opened a new pull request, #1630: URL: https://github.com/apache/arrow-go/pull/1630
### Rationale for this change Fixes #1629. Since #1331, zstd decoders come from a process-wide `sync.Pool`. A `zstd.Decoder` owns a channel that every `DecodeAll` receives from, and a channel made inside a `testing/synctest` bubble must not be used outside that bubble. When a reader inside a bubble created a decoder, the pool could hand it to a reader outside the bubble or in another bubble, and the runtime aborted with `fatal error: receive on synctest channel from outside bubble`. A decoder made outside a bubble is safe to use inside one, so the crash only happens after a decoder was created in a bubble. ### What changes are included in this PR? - `Reader` and `FileReader` each keep their own idle zstd decoders, and the process-wide decoder pool is removed. Go's HTTP/2 server made the same change for the same problem (golang/go#75674). - `FileReader.RecordBatchAt` may be called concurrently, so a `FileReader` keeps as many decoders as were in use at once. - Encoders stay pooled process-wide. They are only used through the streaming API, which keeps no channel between streams, so an encoder used inside a bubble is safe to reuse outside it. A comment on `zstdEncoderPools` records why `EncodeAll` must not be used there. Cost, from `BenchmarkReaderCompressed`, which creates a new reader on every iteration: one decoder per reader, about 1.5 KiB and 11 allocations, with no measurable change in time. ### Are these changes tested? Yes. ### Are there any user-facing changes? Reading zstd-compressed IPC data inside `testing/synctest` bubbles no longer crashes the process. There is no API change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
