tschaub commented on PR #37785:
URL: https://github.com/apache/arrow/pull/37785#issuecomment-1744224804
@zeroshade - I only tested a couple different cases, but I don't see any
negative effect with the latest changes.
Here are some benchmarks transforming a `small.parquet` file (2400 rows, 20
physical columns, uncompressed):
```
Benchmark 1: ./gpq-noasm convert small.parquet small-noasm.parquet
Time (mean ± σ): 72.2 ms ± 2.2 ms [User: 103.9 ms, System: 25.2
ms]
Range (min … max): 68.0 ms … 77.4 ms 41 runs
Benchmark 2: ./gpq-before convert small.parquet small-before.parquet
Time (mean ± σ): 68.6 ms ± 2.2 ms [User: 99.9 ms, System: 25.1
ms]
Range (min … max): 63.1 ms … 74.9 ms 42 runs
Benchmark 3: ./gpq-after convert small.parquet small-after.parquet
Time (mean ± σ): 68.5 ms ± 2.6 ms [User: 99.0 ms, System: 24.5
ms]
Range (min … max): 63.5 ms … 73.5 ms 40 runs
Summary
./gpq-after convert small.parquet small-after.parquet ran
1.00 ± 0.05 times faster than ./gpq-before convert small.parquet
small-before.parquet
1.05 ± 0.05 times faster than ./gpq-noasm convert small.parquet
small-noasm.parquet
```
And here are some numbers from a `medium.parquet` file (162709 rows, 20
physical columns, uncompressed):
```
Benchmark 1: ./gpq-noasm convert medium.parquet medium-noasm.parquet
Time (mean ± σ): 986.3 ms ± 12.3 ms [User: 1456.6 ms, System:
281.5 ms]
Range (min … max): 965.2 ms … 1007.7 ms 10 runs
Benchmark 2: ./gpq-before convert medium.parquet medium-before.parquet
Time (mean ± σ): 878.9 ms ± 7.2 ms [User: 1343.1 ms, System:
269.5 ms]
Range (min … max): 865.4 ms … 888.1 ms 10 runs
Benchmark 3: ./gpq-after convert medium.parquet medium-after.parquet
Time (mean ± σ): 879.1 ms ± 9.7 ms [User: 1341.6 ms, System:
268.6 ms]
Range (min … max): 858.7 ms … 891.8 ms 10 runs
Summary
./gpq-before convert medium.parquet medium-before.parquet ran
1.00 ± 0.01 times faster than ./gpq-after convert medium.parquet
medium-after.parquet
1.12 ± 0.02 times faster than ./gpq-noasm convert medium.parquet
medium-noasm.parquet
```
So no significant difference in those between the before (c9693c5f9b33) and
after (9d4c29b390a2) cases. The other case is a build with `-tags noasm`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
