candiduslynx commented on issue #36095:
URL: https://github.com/apache/arrow/issues/36095#issuecomment-1593262301
> That makes sense to me, you'd want more than 10k rows per row-group.
Personally I don't want to change the default out from under people given the
semantic difference that would happen by changing this default.
@zeroshade I tried with 100 records (each with a single row), the difference
still persists:
```sh
go test \
-test.run=BenchmarkWrite \
-test.bench=BenchmarkWrite \
-test.count 10 -test.benchmem -test.benchtime 100x
```
<details><summary>Write</summary>
```
goos: darwin
goarch: arm64
pkg: github.com/cloudquery/filetypes/v3/parquet
BenchmarkWrite-10 100 3579741 ns/op 5923408
B/op 45296 allocs/op
BenchmarkWrite-10 100 3579718 ns/op 5923500
B/op 45296 allocs/op
BenchmarkWrite-10 100 3542423 ns/op 5923485
B/op 45296 allocs/op
BenchmarkWrite-10 100 3539510 ns/op 5923228
B/op 45294 allocs/op
BenchmarkWrite-10 100 3582708 ns/op 5923466
B/op 45295 allocs/op
BenchmarkWrite-10 100 3631385 ns/op 5923999
B/op 45297 allocs/op
BenchmarkWrite-10 100 3615708 ns/op 5923475
B/op 45296 allocs/op
BenchmarkWrite-10 100 3573869 ns/op 5923664
B/op 45297 allocs/op
BenchmarkWrite-10 100 3625930 ns/op 5923538
B/op 45296 allocs/op
BenchmarkWrite-10 100 3643602 ns/op 5923328
B/op 45294 allocs/op
PASS
ok github.com/cloudquery/filetypes/v3/parquet 4.667s
```
</details>
<details><summary>WriteBuffered</summary>
```
goos: darwin
goarch: arm64
pkg: github.com/cloudquery/filetypes/v3/parquet
BenchmarkWrite-10 100 1063981 ns/op 1288338
B/op 17082 allocs/op
BenchmarkWrite-10 100 1067547 ns/op 1288197
B/op 17083 allocs/op
BenchmarkWrite-10 100 1034933 ns/op 1288312
B/op 17082 allocs/op
BenchmarkWrite-10 100 1044867 ns/op 1288250
B/op 17081 allocs/op
BenchmarkWrite-10 100 1020637 ns/op 1288325
B/op 17081 allocs/op
BenchmarkWrite-10 100 1013818 ns/op 1288170
B/op 17083 allocs/op
BenchmarkWrite-10 100 1028132 ns/op 1288359
B/op 17083 allocs/op
BenchmarkWrite-10 100 1012284 ns/op 1288004
B/op 17082 allocs/op
BenchmarkWrite-10 100 1009228 ns/op 1288231
B/op 17082 allocs/op
BenchmarkWrite-10 100 1038570 ns/op 1288419
B/op 17082 allocs/op
PASS
ok github.com/cloudquery/filetypes/v3/parquet 2.183s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
