[
https://issues.apache.org/jira/browse/ARROW-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weston Pace updated ARROW-12395:
--------------------------------
Description:
It should be pretty comparable to the other task schedulers. I've drafted
something already and it seems to be somewhere between the spawn/serial task
group benchmarks (which are pretty optimized) and the submit benchmark (which
is pretty slow)
Example output
{code:java}
-----------------------------------------------------------------------------------------------------------------
Benchmark Time
CPU Iterations UserCounters...
-----------------------------------------------------------------------------------------------------------------
ReferenceWorkloadCost/task_cost:1000/real_time 693 ns
692 ns 940807 items_per_second=1.4437M/s
ReferenceWorkloadCost/task_cost:10000/real_time 6930 ns
6929 ns 95882 items_per_second=144.297k/s
ReferenceWorkloadCost/task_cost:100000/real_time 69740 ns
69724 ns 9719 items_per_second=14.3391k/s
SerialTaskGroup/task_cost:1000/real_time 6969856 ns
6968315 ns 105 items_per_second=1.43489M/s
SerialTaskGroup/task_cost:10000/real_time 6510709 ns
6509131 ns 96 items_per_second=153.747k/s
SerialTaskGroup/task_cost:100000/real_time 6599298 ns
6597857 ns 95 items_per_second=15.3047k/s
RunInSerialExecutor/task_cost:1000/real_time 1221 ns
1221 ns 563428 items_per_second=818.804k/s
RunInSerialExecutor/task_cost:10000/real_time 7768 ns
7766 ns 88250 items_per_second=128.735k/s
RunInSerialExecutor/task_cost:100000/real_time 72916 ns
72898 ns 9395 items_per_second=13.7145k/s
ThreadPoolSpawn/threads:1/task_cost:1000/real_time 199102152 ns
78682656 ns 4 items_per_second=1004.51k/s
ThreadPoolSpawn/threads:1/task_cost:10000/real_time 148326220 ns
5915479 ns 5 items_per_second=134.845k/s
ThreadPoolSpawn/threads:1/task_cost:100000/real_time 144870312 ns
805224 ns 5 items_per_second=13.8124k/s
ThreadedTaskGroup/threads:1/task_cost:1000/real_time 9177560 ns
18874 ns 73 items_per_second=1089.72k/s
ThreadedTaskGroup/threads:1/task_cost:10000/real_time 6899864 ns
21636 ns 90 items_per_second=145.075k/s
ThreadedTaskGroup/threads:1/task_cost:100000/real_time 7307121 ns
24868 ns 95 items_per_second=13.8221k/s
ThreadPoolSubmit/threads:1/task_cost:1000/real_time 33442213 ns
21155761 ns 20 items_per_second=299.053k/s
ThreadPoolSubmit/threads:1/task_cost:10000/real_time 11966838 ns
1763623 ns 56 items_per_second=83.6478k/s
ThreadPoolSubmit/threads:1/task_cost:100000/real_time 10652720 ns
244028 ns 59 items_per_second=9.48115k/s
{code}
was:It should be pretty comparable to the other task schedulers. I've
drafted something already and it seems to be somewhere between the spawn/serial
task group benchmarks (which are pretty optimized) and the submit benchmark
(which is pretty slow)
> [C++]: Create RunInSerialExecutor benchmark
> -------------------------------------------
>
> Key: ARROW-12395
> URL: https://issues.apache.org/jira/browse/ARROW-12395
> Project: Apache Arrow
> Issue Type: Improvement
> Reporter: Weston Pace
> Priority: Major
>
> It should be pretty comparable to the other task schedulers. I've drafted
> something already and it seems to be somewhere between the spawn/serial task
> group benchmarks (which are pretty optimized) and the submit benchmark (which
> is pretty slow)
>
> Example output
> {code:java}
> -----------------------------------------------------------------------------------------------------------------
> Benchmark Time
> CPU Iterations UserCounters...
> -----------------------------------------------------------------------------------------------------------------
> ReferenceWorkloadCost/task_cost:1000/real_time 693 ns
> 692 ns 940807 items_per_second=1.4437M/s
> ReferenceWorkloadCost/task_cost:10000/real_time 6930 ns
> 6929 ns 95882 items_per_second=144.297k/s
> ReferenceWorkloadCost/task_cost:100000/real_time 69740 ns
> 69724 ns 9719 items_per_second=14.3391k/s
> SerialTaskGroup/task_cost:1000/real_time 6969856 ns
> 6968315 ns 105 items_per_second=1.43489M/s
> SerialTaskGroup/task_cost:10000/real_time 6510709 ns
> 6509131 ns 96 items_per_second=153.747k/s
> SerialTaskGroup/task_cost:100000/real_time 6599298 ns
> 6597857 ns 95 items_per_second=15.3047k/s
> RunInSerialExecutor/task_cost:1000/real_time 1221 ns
> 1221 ns 563428 items_per_second=818.804k/s
> RunInSerialExecutor/task_cost:10000/real_time 7768 ns
> 7766 ns 88250 items_per_second=128.735k/s
> RunInSerialExecutor/task_cost:100000/real_time 72916 ns
> 72898 ns 9395 items_per_second=13.7145k/s
> ThreadPoolSpawn/threads:1/task_cost:1000/real_time 199102152 ns
> 78682656 ns 4 items_per_second=1004.51k/s
> ThreadPoolSpawn/threads:1/task_cost:10000/real_time 148326220 ns
> 5915479 ns 5 items_per_second=134.845k/s
> ThreadPoolSpawn/threads:1/task_cost:100000/real_time 144870312 ns
> 805224 ns 5 items_per_second=13.8124k/s
> ThreadedTaskGroup/threads:1/task_cost:1000/real_time 9177560 ns
> 18874 ns 73 items_per_second=1089.72k/s
> ThreadedTaskGroup/threads:1/task_cost:10000/real_time 6899864 ns
> 21636 ns 90 items_per_second=145.075k/s
> ThreadedTaskGroup/threads:1/task_cost:100000/real_time 7307121 ns
> 24868 ns 95 items_per_second=13.8221k/s
> ThreadPoolSubmit/threads:1/task_cost:1000/real_time 33442213 ns
> 21155761 ns 20 items_per_second=299.053k/s
> ThreadPoolSubmit/threads:1/task_cost:10000/real_time 11966838 ns
> 1763623 ns 56 items_per_second=83.6478k/s
> ThreadPoolSubmit/threads:1/task_cost:100000/real_time 10652720 ns
> 244028 ns 59 items_per_second=9.48115k/s
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
