Hi Andres, Yes, it appears that current ScanQuery implementation is less than optimal in some places, what causes some slowdown. Probably Denis could provide more details about the ticket IGNITE-2921. I do not think it makes sense to extend Iterator API as ScanQuery already covers most use cases. Instead, it is better to fix ScanQuery issues.
Vladimir. On Thu, Mar 31, 2016 at 3:09 AM, abarbaro <[email protected]> wrote: > Vladimir and Ignite Community > > I'm attaching my source code and results for tests comparing the > performance > of ScanQuery versus Entry Iterator. The results show Entry Iterator is > significantly faster in all tests performed. > > Here's a summary of the tests and the results: > > *1) Perform computation on Client node (not really useful in real-life > cases, only included for completeness)* > *Cache of 1,000,000 Entries* > /Iterator is 1.05-1.09X faster than ScanQuery/ > *Cache of 10,000,000 Entries* > /Iterator is 1.1X faster than ScanQuery/ > > *2) Perform computation on Server node, scanning all partitions in one > task, > 1-Server Cluster* > *Cache of 1,000,000 Entries* > /Iterator is 3.2X faster than ScanQuery/ > *Cache of 10,000,000 Entries* > /Iterator is 3.4X faster than ScanQuery/ > > *3) Perform computation on Server node, scanning by partitions with > multiple > tasks per node, 2-Server Cluster* > *Cache of 10,000,000 Entries* > /Iterator is 2.7X faster than ScanQuery/ > Note that iterating over entries seems to be > blocking, so I can't run more than one task per node when using iterator. > > Source code for test: ignite_test_scanquery_performance.jar > < > http://apache-ignite-users.70518.x6.nabble.com/file/n3801/ignite_test_scanquery_performance.jar > > > > Detail results: Ignite_ScanQuery_vs_Iterator_Performance.xlsx > < > http://apache-ignite-users.70518.x6.nabble.com/file/n3801/Ignite_ScanQuery_vs_Iterator_Performance.xlsx > > > > > I also noticed an issue was created today on the Dev mailing list and is > related to this subject: IGNITE-2921: ScanQueries over local partitions > are > not optimal <https://issues.apache.org/jira/browse/IGNITE-2921> . > > > Let me know what you find out, and then we can continue the conversation > about a potential enhancement to be able to iterate/scan over a set of > partitions in one operation (very applicable for map-reduce tasks). > > Thanks, > > > Andres > > > > > > -- > View this message in context: > http://apache-ignite-users.70518.x6.nabble.com/Ignite-Cache-and-Compute-Performance-tp3722p3801.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. >
