fallintoplace opened a new pull request, #1190: URL: https://github.com/apache/arrow-go/pull/1190
### Rationale for this change `Writer.Write` currently converts columns to strings, then copies every string header into a row-major `[][]string` before calling `WriteAll`. This allocates one `[]string` for every row. Large record batches spend a lot of memory on this temporary matrix. ### What changes are included in this PR? - Keep the converted strings in their existing column slices. - Reuse one row-sized `[]string` while writing each CSV record. - Preserve the existing flush and error behavior. For the existing 1,000-row, 16-column benchmark, medians from 6 runs on an Apple M1 Pro were: | | main | this PR | change | |---|---:|---:|---:| | time/op | 4.78 ms | 4.61 ms | -3.5% | | B/op | 2,292,279 | 2,012,183 | -12.2% | | allocs/op | 41,514 | 40,515 | -2.4% | This removes 999 allocations per call for 1,000-row batches. ### Are these changes tested? - `go test ./arrow/... -count=1` - `go test ./arrow/csv -run ^'$' -bench ^'BenchmarkWrite$' -benchmem -count=6`\n\n### Are there any user-facing changes?\n\nNo. The generated CSV output is unchanged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
