[
https://issues.apache.org/jira/browse/IGNITE-9432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Dmitriev updated IGNITE-9432:
-----------------------------------
Description:
When we make ScanQuery via Binary Client Protocol it works very slowly in case
we need to retrieve a big amount of objects (all objects in our case). The most
slower part is a waiting for response from Apache Ignite. This also mentioned
on devlist
[http://apache-ignite-developers.2346864.n4.nabble.com/How-to-reduce-Scan-Query-execution-time-td34212.html].
To reproduce the problem we've prepared a small example:
[slow-scan-query-reproducer|https://github.com/dmitrievanthony/slow-scan-query-reproducer].
In this example we creates a cache with 500 objects 1Mb each (on localhost),
we make ScanQuery with different page sizes and calculate time between the
moment when the request has been sent and the moment when the response is ready
to be receive. The measurements are here:
Page size 5 Mb, waiting time 119.85 ± 6.72 ms
Page size 10 Mb, waiting time 157.70 ± 15.35 ms
Page size 20 Mb, waiting time 204.50 ± 19.18 ms
Page size 50 Mb, waiting time 264.70 ± 22.30 ms
Page size 100 Mb, waiting time 463.35 ± 17.12 ms
Page size 150 Mb, waiting time 672.50 ± 21.98 ms
As result we spend ~4ms per every megabyte on _something_. It means that we
will be able to achieve 250Mb/s throughput in best case. It's too slow, isn't
it?
was:
When we make ScanQuery via Binary Client Protocol it works very slowly in case
we need to retrieve a big amount of objects (all objects in our case). The most
slower part is a waiting for response from Apache Ignite. This also mentioned
on devlist
[http://apache-ignite-developers.2346864.n4.nabble.com/How-to-reduce-Scan-Query-execution-time-td34212.html].
To reproduce the problem we've prepared a small example:
[slow-scan-query-reproducer|https://github.com/dmitrievanthony/slow-scan-query-reproducer].
In this example we creates a cache with 500 objects 1Mb each (on localhost),
we make ScanQuery with different page sizes and calculate time between the
moment when the request has been sent and the moment when the response is ready
to be receive. The measurements are here:
{{Page size 5 Mb, waiting time 119.85 ± 6.72 ms}}
Page size 10 Mb, waiting time 157.70 ± 15.35 ms
Page size 20 Mb, waiting time 204.50 ± 19.18 ms
Page size 50 Mb, waiting time 264.70 ± 22.30 ms
Page size 100 Mb, waiting time 463.35 ± 17.12 ms
Page size 150 Mb, waiting time 672.50 ± 21.98 ms
As result we spend ~4ms per every megabyte on _something_. It means that we
will be able to achieve 250Mb/s throughput in best case. It's too slow, isn't
it?
> Investigate ability to make ScanQuery faster
> --------------------------------------------
>
> Key: IGNITE-9432
> URL: https://issues.apache.org/jira/browse/IGNITE-9432
> Project: Ignite
> Issue Type: Improvement
> Components: ml
> Affects Versions: 2.7
> Reporter: Anton Dmitriev
> Priority: Major
> Fix For: 2.7
>
>
> When we make ScanQuery via Binary Client Protocol it works very slowly in
> case we need to retrieve a big amount of objects (all objects in our case).
> The most slower part is a waiting for response from Apache Ignite. This also
> mentioned on devlist
> [http://apache-ignite-developers.2346864.n4.nabble.com/How-to-reduce-Scan-Query-execution-time-td34212.html].
> To reproduce the problem we've prepared a small example:
> [slow-scan-query-reproducer|https://github.com/dmitrievanthony/slow-scan-query-reproducer].
> In this example we creates a cache with 500 objects 1Mb each (on localhost),
> we make ScanQuery with different page sizes and calculate time between the
> moment when the request has been sent and the moment when the response is
> ready to be receive. The measurements are here:
> Page size 5 Mb, waiting time 119.85 ± 6.72 ms
> Page size 10 Mb, waiting time 157.70 ± 15.35 ms
> Page size 20 Mb, waiting time 204.50 ± 19.18 ms
> Page size 50 Mb, waiting time 264.70 ± 22.30 ms
> Page size 100 Mb, waiting time 463.35 ± 17.12 ms
> Page size 150 Mb, waiting time 672.50 ± 21.98 ms
> As result we spend ~4ms per every megabyte on _something_. It means that we
> will be able to achieve 250Mb/s throughput in best case. It's too slow, isn't
> it?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
