elsteveogrande opened a new pull request, #1434: URL: https://github.com/apache/arrow-go/pull/1434
### Rationale for this change #1250 removed the per-child goroutines from `structReader.SeekToRow` and `LoadBatch` for serial reads. `FileReader.getReader` still builds a struct's child readers through an `errgroup` with `SetLimit(1)` when `Parallel=false`, so each child reader costs a goroutine and a semaphore handoff. We hit this compacting Parquet files with tens of thousands of struct columns, where reader construction runs once per column per row group. ### What changes are included in this PR? - `getReader` builds struct child readers in a plain loop when `Parallel=false`. The `errgroup` path, without `SetLimit`, stays for parallel reads. - On error, the serial loop returns the first error immediately. Before, the canceled `errgroup` context still started the remaining children, one at a time, before returning the same first error. - `BenchmarkGetFieldReaderNestedStructSerial` times `GetFieldReader` on the nested struct table from `BenchmarkReadNestedStructSerial`. ### Are these changes tested? Yes, by the existing reader tests: - `PARQUET_TEST_DATA=<parquet-testing-data> go test ./parquet/...` - `go test -race ./parquet/pqarrow -run '^(TestStructReaderSerialOperationsVisitEveryChild|TestRecordReaderSerial|TestRecordReaderParallel|TestRecordReaderSeekToRow|TestPartialStructColumnRead)$' -count=1` - `go vet ./parquet/pqarrow`, `git diff --check` Benchmark on an Apple M4 Max, `main` at `56815b7`, 500ms per sample, 5 samples: | children | main | this PR | change | allocs/op | | ---: | ---: | ---: | ---: | ---: | | 1 | 5.39 us | 3.42 us | 36.6% faster | 61 -> 55 | | 8 | 36.8 us | 22.7 us | 38.3% faster | 348 -> 328 | | 32 | 136.6 us | 77.9 us | 43.0% faster | 1366 -> 1298 | | 128 | 466.9 us | 268.9 us | 42.4% faster | 5398 -> 5138 | ```text go test ./parquet/pqarrow -run '^$' -bench '^BenchmarkGetFieldReaderNestedStructSerial$' -benchmem -benchtime=500ms -count=5 ``` ### Are there any user-facing changes? No. Serial reads build the same child readers and return the same first error; only construction is faster. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
