andygrove commented on PR #7405:
URL:
https://github.com/apache/datafusion-comet/pull/7405#issuecomment-6585458139
`CometPlanOutputBenchmark` results on Linux x86 at 566fc872c (AMD Ryzen 9
7950X3D, Ubuntu 22.04, OpenJDK 17.0.20, `local[1]`, `make release` build,
pinned with `taskset -c 0-7,16-23` to the V-Cache CCD). Wall time is the best
of each case's runs. CPU is process CPU time per batch.
| Output columns | Off | On | Off (repeat) | CPU per batch, off | CPU per
batch, on |
| --- | ---: | ---: | ---: | ---: | ---: |
| 1 × `bigint` | 53 ms | 51 ms | 52 ms | 68.3 µs | 65.0 µs |
| 10 × `bigint` | 52 ms | 52 ms | 52 ms | 78.0 µs | 84.6 µs |
| 100 × `bigint` | 163 ms | 120 ms | 162 ms | 227.6 µs | 188.6 µs |
| 1 × `string` | 95 ms | 95 ms | 96 ms | 104.1 µs | 104.1 µs |
| 10 × `string` | 97 ms | 95 ms | 94 ms | 117.1 µs | 110.6 µs |
| 100 × `string` | 191 ms | 151 ms | 190 ms | 289.4 µs | 253.7 µs |
| 1 × `struct<bigint,string>` | 139 ms | 139 ms | 140 ms | 149.6 µs | 149.6
µs |
| 10 × `struct<bigint,string>` | 142 ms | 143 ms | 143 ms | 201.6 µs | 188.6
µs |
| 100 × `struct<bigint,string>` | 548 ms | 422 ms | 549 ms | 708.9 µs |
578.9 µs |
| 100 distinct `bigint`, 100 columns read | 690 ms | 680 ms | 682 ms |
5710.6 µs | 5658.9 µs |
This matches the M3 Max results in the description. With 100 projected
columns, wall time is 1.36x faster for `bigint`, 1.26x for `string` and 1.30x
for `struct<bigint,string>`, and CPU per batch is 12 to 18% lower. The gains
are a little larger than on macOS. With 1 or 10 output columns the settings are
within noise (the 10 × `bigint` CPU sample went the other way by 6.5 µs), and
the 100-distinct-column case differs by under 1%. The schema-count checks
passed in every case: 1025 schemas with the cache off and 1 with it on, or 129
and 3 for the 3-task case.
<details>
<summary>Full results file</summary>
```text
1 bigint output columns, 1 column read, cacheOutputSchema=false: 1025
batches in 1 tasks, the schema sent with 1025 of them
1 bigint output columns, 1 column read, cacheOutputSchema=true: 1025 batches
in 1 tasks, the schema sent with 1 of them
OpenJDK 64-Bit Server VM 17.0.20.1+1-1-22.04-Ubuntu on Linux
6.8.0-138-generic
AMD Ryzen 9 7950X3D 16-Core Processor
1 bigint output columns, 1 column read: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cacheOutputSchema OFF 53 55
1 158.1 6.3 1.0X
cacheOutputSchema ON 51 53
1 163.8 6.1 1.0X
cacheOutputSchema OFF (repeat) 52 54
2 161.4 6.2 1.0X
1 bigint output columns, 1 column read: process CPU per batch 68.3 us with
the schema in every batch, 65.0 us with it cached, 3.3 us (4.8%) less
10 bigint output columns, 1 column read, cacheOutputSchema=false: 1025
batches in 1 tasks, the schema sent with 1025 of them
10 bigint output columns, 1 column read, cacheOutputSchema=true: 1025
batches in 1 tasks, the schema sent with 1 of them
OpenJDK 64-Bit Server VM 17.0.20.1+1-1-22.04-Ubuntu on Linux
6.8.0-138-generic
AMD Ryzen 9 7950X3D 16-Core Processor
10 bigint output columns, 1 column read: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cacheOutputSchema OFF 52 53
1 160.6 6.2 1.0X
cacheOutputSchema ON 52 55
2 161.0 6.2 1.0X
cacheOutputSchema OFF (repeat) 52 54
1 162.3 6.2 1.0X
10 bigint output columns, 1 column read: process CPU per batch 78.0 us with
the schema in every batch, 84.6 us with it cached, -6.5 us (-8.3%) less
100 bigint output columns, 1 column read, cacheOutputSchema=false: 1025
batches in 1 tasks, the schema sent with 1025 of them
100 bigint output columns, 1 column read, cacheOutputSchema=true: 1025
batches in 1 tasks, the schema sent with 1 of them
OpenJDK 64-Bit Server VM 17.0.20.1+1-1-22.04-Ubuntu on Linux
6.8.0-138-generic
AMD Ryzen 9 7950X3D 16-Core Processor
100 bigint output columns, 1 column read: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cacheOutputSchema OFF 163 165
2 51.6 19.4 1.0X
cacheOutputSchema ON 120 123
1 69.9 14.3 1.4X
cacheOutputSchema OFF (repeat) 162 165
2 51.7 19.3 1.0X
100 bigint output columns, 1 column read: process CPU per batch 227.6 us
with the schema in every batch, 188.6 us with it cached, 39.0 us (17.1%) less
1 string output columns, 1 column read, cacheOutputSchema=false: 1025
batches in 1 tasks, the schema sent with 1025 of them
1 string output columns, 1 column read, cacheOutputSchema=true: 1025 batches
in 1 tasks, the schema sent with 1 of them
OpenJDK 64-Bit Server VM 17.0.20.1+1-1-22.04-Ubuntu on Linux
6.8.0-138-generic
AMD Ryzen 9 7950X3D 16-Core Processor
1 string output columns, 1 column read: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cacheOutputSchema OFF 95 96
2 88.6 11.3 1.0X
cacheOutputSchema ON 95 98
1 88.1 11.4 1.0X
cacheOutputSchema OFF (repeat) 96 98
1 87.5 11.4 1.0X
1 string output columns, 1 column read: process CPU per batch 104.1 us with
the schema in every batch, 104.1 us with it cached, 0.0 us (0.0%) less
10 string output columns, 1 column read, cacheOutputSchema=false: 1025
batches in 1 tasks, the schema sent with 1025 of them
10 string output columns, 1 column read, cacheOutputSchema=true: 1025
batches in 1 tasks, the schema sent with 1 of them
OpenJDK 64-Bit Server VM 17.0.20.1+1-1-22.04-Ubuntu on Linux
6.8.0-138-generic
AMD Ryzen 9 7950X3D 16-Core Processor
10 string output columns, 1 column read: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cacheOutputSchema OFF 97 99
1 86.1 11.6 1.0X
cacheOutputSchema ON 95 99
1 88.1 11.4 1.0X
cacheOutputSchema OFF (repeat) 94 96
1 89.4 11.2 1.0X
10 string output columns, 1 column read: process CPU per batch 117.1 us with
the schema in every batch, 110.6 us with it cached, 6.5 us (5.6%) less
100 string output columns, 1 column read, cacheOutputSchema=false: 1025
batches in 1 tasks, the schema sent with 1025 of them
100 string output columns, 1 column read, cacheOutputSchema=true: 1025
batches in 1 tasks, the schema sent with 1 of them
OpenJDK 64-Bit Server VM 17.0.20.1+1-1-22.04-Ubuntu on Linux
6.8.0-138-generic
AMD Ryzen 9 7950X3D 16-Core Processor
100 string output columns, 1 column read: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cacheOutputSchema OFF 191 194
3 43.9 22.8 1.0X
cacheOutputSchema ON 151 153
2 55.4 18.1 1.3X
cacheOutputSchema OFF (repeat) 190 193
4 44.2 22.6 1.0X
100 string output columns, 1 column read: process CPU per batch 289.4 us
with the schema in every batch, 253.7 us with it cached, 35.8 us (12.4%) less
1 struct<bigint,string> output columns, 1 column read,
cacheOutputSchema=false: 1025 batches in 1 tasks, the schema sent with 1025 of
them
1 struct<bigint,string> output columns, 1 column read,
cacheOutputSchema=true: 1025 batches in 1 tasks, the schema sent with 1 of them
OpenJDK 64-Bit Server VM 17.0.20.1+1-1-22.04-Ubuntu on Linux
6.8.0-138-generic
AMD Ryzen 9 7950X3D 16-Core Processor
1 struct<bigint,string> output columns, 1 column read: Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------
cacheOutputSchema OFF 139
142 1 60.2 16.6 1.0X
cacheOutputSchema ON 139
143 3 60.3 16.6 1.0X
cacheOutputSchema OFF (repeat) 140
142 1 59.8 16.7 1.0X
1 struct<bigint,string> output columns, 1 column read: process CPU per batch
149.6 us with the schema in every batch, 149.6 us with it cached, 0.0 us (0.0%)
less
10 struct<bigint,string> output columns, 1 column read,
cacheOutputSchema=false: 1025 batches in 1 tasks, the schema sent with 1025 of
them
10 struct<bigint,string> output columns, 1 column read,
cacheOutputSchema=true: 1025 batches in 1 tasks, the schema sent with 1 of them
OpenJDK 64-Bit Server VM 17.0.20.1+1-1-22.04-Ubuntu on Linux
6.8.0-138-generic
AMD Ryzen 9 7950X3D 16-Core Processor
10 struct<bigint,string> output columns, 1 column read: Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------
cacheOutputSchema OFF 142
145 1 59.0 17.0 1.0X
cacheOutputSchema ON 143
145 2 58.6 17.1 1.0X
cacheOutputSchema OFF (repeat) 143
146 2 58.6 17.1 1.0X
10 struct<bigint,string> output columns, 1 column read: process CPU per
batch 201.6 us with the schema in every batch, 188.6 us with it cached, 13.0 us
(6.5%) less
100 struct<bigint,string> output columns, 1 column read,
cacheOutputSchema=false: 1025 batches in 1 tasks, the schema sent with 1025 of
them
100 struct<bigint,string> output columns, 1 column read,
cacheOutputSchema=true: 1025 batches in 1 tasks, the schema sent with 1 of them
OpenJDK 64-Bit Server VM 17.0.20.1+1-1-22.04-Ubuntu on Linux
6.8.0-138-generic
AMD Ryzen 9 7950X3D 16-Core Processor
100 struct<bigint,string> output columns, 1 column read: Best Time(ms)
Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------------------
cacheOutputSchema OFF 548
551 2 15.3 65.3 1.0X
cacheOutputSchema ON 422
425 1 19.9 50.3 1.3X
cacheOutputSchema OFF (repeat) 549
553 3 15.3 65.4 1.0X
100 struct<bigint,string> output columns, 1 column read: process CPU per
batch 708.9 us with the schema in every batch, 578.9 us with it cached, 130.1
us (18.3%) less
100 bigint output columns, 100 columns read, cacheOutputSchema=false: 129
batches in 3 tasks, the schema sent with 129 of them
100 bigint output columns, 100 columns read, cacheOutputSchema=true: 129
batches in 3 tasks, the schema sent with 3 of them
OpenJDK 64-Bit Server VM 17.0.20.1+1-1-22.04-Ubuntu on Linux
6.8.0-138-generic
AMD Ryzen 9 7950X3D 16-Core Processor
100 bigint output columns, 100 columns read: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
cacheOutputSchema OFF 690 692
3 1.5 657.9 1.0X
cacheOutputSchema ON 680 684
7 1.5 648.3 1.0X
cacheOutputSchema OFF (repeat) 682 689
8 1.5 650.6 1.0X
100 bigint output columns, 100 columns read: process CPU per batch 5710.6 us
with the schema in every batch, 5658.9 us with it cached, 51.7 us (0.9%) less
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
