Hello! I have been working on Ignite and comparing its performance to traditional databases. However, the performance was not significantly better as expected. Any advice on how to improve the performance would be greatly appreciated!
The dataset I have is around 10M records, stored on a cluster with 4 machines. I'm mainly concerned with cache read performance as the service is expected to handle read requests frequently. I have been using Ignite SqlQuery which gives the best performance. The way I implement it is to generate a sql string from request parameters, create a SqlQuery instance with that string, use cache.query() to get a QueryCursor instance, iterate through the QueryCursor and obtain results. Everything is done following the Ignite API. The most commonly used sql string here is a simple string of conditions like "fieldA = a AND fieldB = b ...". I have also added individual indexes for each of the fields that are commonly queried on. The execution plan I observed in H2 debug console showed that the indexes were being used. I have pretty much tried out everything in this link: https://apacheignite.readme.io/docs/performance-tips <https://apacheignite.readme.io/docs/performance-tips> , but none of them seem to help significantly. I also tried to fire multiple queries on different partitions of the cache in parallel to increase IO throughput. However, I think this could potentially make other requests to the same cache starve. In addition, using ScanQuery instead of SqlQuery made execution several times slower... I would really hope to seek help on if I made any mistakes or if there are any suggestions on what I could try out and see if it results in better performance? Thank you very much! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
