Hi Naveen, Ignite implements page memory concept since version 2.0 has been released. This means, Ignite reads page from disk if it is not loaded to memory and evict pages to disk to free memory. Page load and eviction work automatically. So, I can't understand what "feature like eviction" and "hint to go to backing store" do you mean.
Join performance issue usually caused by lack of H2 optimizer which sometimes make full table scan (or index scan with low selectivity) and then join other tables to first one. You can always try to change table join order to reduce dataset that will be scanned. EnforceJoinOrder flag is designed to helps you with this by disabling H2 join order optimization. If join order changes is not helpful and you always face slow scan with low CPU usage you can try to increase query parallelism [1]. QueryParallelism value is equals to number of parts cache indices will be splitted to. This works similar way as adding new nodesand allow you to utilize more CPU on every node by running query in several threads (equals to the option value) in parallel. Each thread will work with lesser dataset that should reduce query execution time. Of course there is a limitation, you can't join tables from (Partitioned) caches with different parallelizm level. Replicated cache parallelizm level can be >1 which is default. Another issue related to this option is simple queries latency can be increased as heavy query speed trade off. [1] https://apacheignite.readme.io/v2.0/docs/sql-performance-and-debugging#query-parallelism On Fri, Mar 30, 2018 at 8:14 AM, Naveen <[email protected]> wrote: > Lazy is something which we can use when we are expecting huge result set to > minimize the memory consumption at the cost of little performance hit. > However my requirement to run a query by joining couple of tables with > complex where clause to debug some data related issues, this query may not > return huge result set, but scan the the complete table to get the result > set, during this time it should not hit the performance. > > So my question here is, can we try applying the same feature like eviction, > in case of eviction, if the record is not there in RAM, it will go to > backing store and gets the data and also load it to memory. Similar way, > can > we hint the cluster to run the query on backing store instead of RAM, so > that that the impact on RAM would be less if you are running the query on > backing store. > > In our case, we have 2 TB servers with 128 core machine, very high end > machines, RAM is utilized upto 60 to 70%, however CPU resources are not > utilized. > > Hope I was able to convey my adhoc requirements to run various SQL queries > to debug data related issues and consequences of these queries on the > performance on the cluster > > Thanks > Naveen > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > -- Best regards, Andrey V. Mashenkov
