[jira] [Commented] (IGNITE-5571) Make sure that cache-less execution works as good as cache-based
[ https://issues.apache.org/jira/browse/IGNITE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16336012#comment-16336012 ] ASF GitHub Bot commented on IGNITE-5571: GitHub user alexpaschenko opened a pull request: https://github.com/apache/ignite/pull/3424 IGNITE-5571 with 4191 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-5571-with-4191 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/3424.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3424 commit ac475bc18dee0c6f19aa5fb9f1cfeae164deea94 Author: sboikovDate: 2017-09-28T10:56:41Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit 7f4defd0926325dfb9d6840156dfa64309ae7956 Author: sboikov Date: 2017-09-29T11:20:00Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit 761e43d3039cf8c58c9c7b0ec2dde68238d71647 Author: sboikov Date: 2017-09-29T11:29:03Z ignite-3479 Coordinators reassign on failure commit fb3ee2478eccad3a9ab12c42ef99d0364af5194f Author: sboikov Date: 2017-09-29T11:51:28Z ignite-3478 commit 8cf2aad733aeda8ae4575e093315c052b682b455 Author: sboikov Date: 2017-09-29T12:38:37Z ignite-3478 Support for streamer commit e8a5a082026c47d40c1b6e482f5a9d83115b927a Author: sboikov Date: 2017-10-02T07:48:05Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit 685c688f7a1ec2ea1f5904c5c7df7baff41c15af Author: sboikov Date: 2017-10-02T09:39:29Z ignite-3478 commit 27b2be4b98e12a32037b1fe6146b85d939c95952 Author: sboikov Date: 2017-10-03T07:41:18Z ignite-3478 commit 07a56028a2150208c79e64c280dc989624c47c47 Author: sboikov Date: 2017-10-03T14:34:20Z ignite-3478 commit fdfe779f063d015dd0fcb7f2880a1b227a3e190f Author: sboikov Date: 2017-10-03T14:34:30Z ignite-3478 commit fc7b7e21c87689866eee83c6e73771333ab0492e Author: sboikov Date: 2017-10-04T08:35:00Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/GridKernalContext.java # modules/core/src/main/java/org/apache/ignite/internal/GridKernalContextImpl.java # modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/reader/StandaloneGridKernalContext.java commit a1d9ddd10dfa935b81703c4ba1e2670fba282389 Author: sboikov Date: 2017-10-04T08:35:53Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 # Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/GridKernalContext.java # modules/core/src/main/java/org/apache/ignite/internal/GridKernalContextImpl.java # modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java # modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/reader/StandaloneGridKernalContext.java commit 410c84fcf0d6f7bbe8373f996b6633c4d5b15b61 Author: sboikov Date: 2017-10-05T08:40:04Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit 921404a6f5b256cf6404822bc7e439f2cad44c5d Author: sboikov Date: 2017-10-09T10:41:58Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit fd53c1a8f4e905a7aba469eb5decf38c50b7708e Author: sboikov Date: 2017-10-09T11:42:43Z ignite-5937 Added BPlusTree.iterate for more optimal mvcc search commit 61b46c47f01c702a17b683c69a80a7c85c389b84 Author: sboikov Date: 2017-10-10T11:39:21Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit 69fd367dc2eb9e76d89f344063d9788171cb359f Author: sboikov Date: 2017-10-11T08:18:10Z Merge remote-tracking branch 'remotes/origin/master' into ignite-3478 commit 970cf47a51dc6e754677b00e85e60effc48083ba Author: sboikov Date: 2017-10-11T08:24:32Z ignite-3478 Support for removes commit a8a46084c711ce029d391915e3bfa493220664a0 Author: Alexander Paschenko Date: 2017-10-11T16:19:47Z Merge branch 'master' into ignite-5571 commit 35cb06848097a6db56322358d1d48cacf9f1f35a Author: Alexander Paschenko Date: 2017-10-11T16:21:14Z Post-merge fx 1 commit 23742962f8d539aac33a7ac953f09a1407b330e9 Author: sboikov Date: 2017-10-12T10:36:52Z Merge remote-tracking
[jira] [Commented] (IGNITE-5571) Make sure that cache-less execution works as good as cache-based
[ https://issues.apache.org/jira/browse/IGNITE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334775#comment-16334775 ] ASF GitHub Bot commented on IGNITE-5571: Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/2283 > Make sure that cache-less execution works as good as cache-based > > > Key: IGNITE-5571 > URL: https://issues.apache.org/jira/browse/IGNITE-5571 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Vladimir Ozerov >Assignee: Alexander Paschenko >Priority: Critical > Fix For: 2.4 > > > Compare the following two methods: > 1) {{GridQueryProcessor.querySqlFields}} - old good entry point for query > execution; > 2) {{GridQueryProcessor.querySqlFieldsNoCache}} - new method for "cache-less" > execution. > Note how cache context is used in the first method: > 1) First, it helps determine whether query can be converted to "local" > 2) Second, it gets query parallelism of current cache, and if it differs from > {{1}}, then it turns on {{distributedJoins}}. > Neither of this happens in the second implementation. Moreover, I had to > throw an exception for local queries, as I didn't know how to handle them > properly. > We need to investigate and fix these two deficiencies somehow. Probably some > inputs from [~sergi.vladykin] would be required, to understand what is going > on. > Our ultimate goal is to make "cache-less" execution as good as the old one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-5571) Make sure that cache-less execution works as good as cache-based
[ https://issues.apache.org/jira/browse/IGNITE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333607#comment-16333607 ] Vladimir Ozerov commented on IGNITE-5571: - [~al.psc], looks like final round of conflict resolution is needed (conflicts with recent error handling improvements). > Make sure that cache-less execution works as good as cache-based > > > Key: IGNITE-5571 > URL: https://issues.apache.org/jira/browse/IGNITE-5571 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Vladimir Ozerov >Assignee: Alexander Paschenko >Priority: Critical > Fix For: 2.4 > > > Compare the following two methods: > 1) {{GridQueryProcessor.querySqlFields}} - old good entry point for query > execution; > 2) {{GridQueryProcessor.querySqlFieldsNoCache}} - new method for "cache-less" > execution. > Note how cache context is used in the first method: > 1) First, it helps determine whether query can be converted to "local" > 2) Second, it gets query parallelism of current cache, and if it differs from > {{1}}, then it turns on {{distributedJoins}}. > Neither of this happens in the second implementation. Moreover, I had to > throw an exception for local queries, as I didn't know how to handle them > properly. > We need to investigate and fix these two deficiencies somehow. Probably some > inputs from [~sergi.vladykin] would be required, to understand what is going > on. > Our ultimate goal is to make "cache-less" execution as good as the old one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-5571) Make sure that cache-less execution works as good as cache-based
[ https://issues.apache.org/jira/browse/IGNITE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16304737#comment-16304737 ] Alexander Paschenko commented on IGNITE-5571: - [~vozerov] All done. > Make sure that cache-less execution works as good as cache-based > > > Key: IGNITE-5571 > URL: https://issues.apache.org/jira/browse/IGNITE-5571 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Vladimir Ozerov >Assignee: Alexander Paschenko >Priority: Critical > Fix For: 2.4 > > > Compare the following two methods: > 1) {{GridQueryProcessor.querySqlFields}} - old good entry point for query > execution; > 2) {{GridQueryProcessor.querySqlFieldsNoCache}} - new method for "cache-less" > execution. > Note how cache context is used in the first method: > 1) First, it helps determine whether query can be converted to "local" > 2) Second, it gets query parallelism of current cache, and if it differs from > {{1}}, then it turns on {{distributedJoins}}. > Neither of this happens in the second implementation. Moreover, I had to > throw an exception for local queries, as I didn't know how to handle them > properly. > We need to investigate and fix these two deficiencies somehow. Probably some > inputs from [~sergi.vladykin] would be required, to understand what is going > on. > Our ultimate goal is to make "cache-less" execution as good as the old one. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (IGNITE-5571) Make sure that cache-less execution works as good as cache-based
[ https://issues.apache.org/jira/browse/IGNITE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292157#comment-16292157 ] Vladimir Ozerov commented on IGNITE-5571: - [~al.psc], I think I understood why we set distributed joins flag for local query with parallelism - see {{DistributedJoinMode.LOCAL_ONLY}}. > Make sure that cache-less execution works as good as cache-based > > > Key: IGNITE-5571 > URL: https://issues.apache.org/jira/browse/IGNITE-5571 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Vladimir Ozerov >Assignee: Alexander Paschenko >Priority: Critical > Fix For: 2.4 > > > Compare the following two methods: > 1) {{GridQueryProcessor.querySqlFields}} - old good entry point for query > execution; > 2) {{GridQueryProcessor.querySqlFieldsNoCache}} - new method for "cache-less" > execution. > Note how cache context is used in the first method: > 1) First, it helps determine whether query can be converted to "local" > 2) Second, it gets query parallelism of current cache, and if it differs from > {{1}}, then it turns on {{distributedJoins}}. > Neither of this happens in the second implementation. Moreover, I had to > throw an exception for local queries, as I didn't know how to handle them > properly. > We need to investigate and fix these two deficiencies somehow. Probably some > inputs from [~sergi.vladykin] would be required, to understand what is going > on. > Our ultimate goal is to make "cache-less" execution as good as the old one. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (IGNITE-5571) Make sure that cache-less execution works as good as cache-based
[ https://issues.apache.org/jira/browse/IGNITE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16259088#comment-16259088 ] Alexander Paschenko commented on IGNITE-5571: - [~vozerov], 1; 4; 5 - fixed. 2; 3 - stuff like GROUP BY didn't participate in analysis before; thus, this flag was calculated solely relying on characteristics of the cache the query was issued on and query flags - like this {{boolean loc = (qry.isReplicatedOnly() && cctx.isReplicatedAffinityNode()) || cctx.isLocal() || qry.isLocal();}} Therefore, I believe that more intelligent and in-depth analysis of the queries should be moved to a separate task as of now as long as suggested patch does not in fact change existing behavior. > Make sure that cache-less execution works as good as cache-based > > > Key: IGNITE-5571 > URL: https://issues.apache.org/jira/browse/IGNITE-5571 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Vladimir Ozerov >Assignee: Alexander Paschenko >Priority: Critical > Fix For: 2.4 > > > Compare the following two methods: > 1) {{GridQueryProcessor.querySqlFields}} - old good entry point for query > execution; > 2) {{GridQueryProcessor.querySqlFieldsNoCache}} - new method for "cache-less" > execution. > Note how cache context is used in the first method: > 1) First, it helps determine whether query can be converted to "local" > 2) Second, it gets query parallelism of current cache, and if it differs from > {{1}}, then it turns on {{distributedJoins}}. > Neither of this happens in the second implementation. Moreover, I had to > throw an exception for local queries, as I didn't know how to handle them > properly. > We need to investigate and fix these two deficiencies somehow. Probably some > inputs from [~sergi.vladykin] would be required, to understand what is going > on. > Our ultimate goal is to make "cache-less" execution as good as the old one. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (IGNITE-5571) Make sure that cache-less execution works as good as cache-based
[ https://issues.apache.org/jira/browse/IGNITE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210981#comment-16210981 ] Pavel Tupitsyn commented on IGNITE-5571: [~al.psc], looks like if a local SQL query was executed, same non-local query can return incomplete results after that. > Make sure that cache-less execution works as good as cache-based > > > Key: IGNITE-5571 > URL: https://issues.apache.org/jira/browse/IGNITE-5571 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Vladimir Ozerov >Assignee: Alexander Paschenko >Priority: Critical > Fix For: 2.4 > > > Compare the following two methods: > 1) {{GridQueryProcessor.querySqlFields}} - old good entry point for query > execution; > 2) {{GridQueryProcessor.querySqlFieldsNoCache}} - new method for "cache-less" > execution. > Note how cache context is used in the first method: > 1) First, it helps determine whether query can be converted to "local" > 2) Second, it gets query parallelism of current cache, and if it differs from > {{1}}, then it turns on {{distributedJoins}}. > Neither of this happens in the second implementation. Moreover, I had to > throw an exception for local queries, as I didn't know how to handle them > properly. > We need to investigate and fix these two deficiencies somehow. Probably some > inputs from [~sergi.vladykin] would be required, to understand what is going > on. > Our ultimate goal is to make "cache-less" execution as good as the old one. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (IGNITE-5571) Make sure that cache-less execution works as good as cache-based
[ https://issues.apache.org/jira/browse/IGNITE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210852#comment-16210852 ] Alexander Paschenko commented on IGNITE-5571: - [~ptupitsyn], hello, could you please have a look at .NET test failures? https://ci.ignite.apache.org/viewLog.html?buildId=898285=buildResultsDiv=Ignite20Tests_IgnitePlatformNet > Make sure that cache-less execution works as good as cache-based > > > Key: IGNITE-5571 > URL: https://issues.apache.org/jira/browse/IGNITE-5571 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Vladimir Ozerov >Assignee: Alexander Paschenko >Priority: Critical > Fix For: 2.4 > > > Compare the following two methods: > 1) {{GridQueryProcessor.querySqlFields}} - old good entry point for query > execution; > 2) {{GridQueryProcessor.querySqlFieldsNoCache}} - new method for "cache-less" > execution. > Note how cache context is used in the first method: > 1) First, it helps determine whether query can be converted to "local" > 2) Second, it gets query parallelism of current cache, and if it differs from > {{1}}, then it turns on {{distributedJoins}}. > Neither of this happens in the second implementation. Moreover, I had to > throw an exception for local queries, as I didn't know how to handle them > properly. > We need to investigate and fix these two deficiencies somehow. Probably some > inputs from [~sergi.vladykin] would be required, to understand what is going > on. > Our ultimate goal is to make "cache-less" execution as good as the old one. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (IGNITE-5571) Make sure that cache-less execution works as good as cache-based
[ https://issues.apache.org/jira/browse/IGNITE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085618#comment-16085618 ] Vladimir Ozerov commented on IGNITE-5571: - [~al.psc], my comments: 1) {{AffinityTopologyVersion}} is never passed from DML processor. Looks like a bug to me. IMO, we should not pass it to {{querySqlFields}} at all. Instead, we should ask {{GridQueryProcessor}} for backup filter when we understand it is needed. Let's remove {{AffinityTopologyVersion}} argument. 2) Please no {{T*}} classes. This is antipattern. Normal value class should be created instead. 3) {{IgniteH2Indexing:1376}} - incorrect {{distributedJoins}} is set here. I do not know whether it has any adverse affects, but let's make this code cleaner. We do not need {{GridH2QueryContext}} at this point. Let's set only after we calculated final value for {{distributedJoins}} flag. > Make sure that cache-less execution works as good as cache-based > > > Key: IGNITE-5571 > URL: https://issues.apache.org/jira/browse/IGNITE-5571 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Vladimir Ozerov >Assignee: Alexander Paschenko >Priority: Critical > Fix For: 2.2 > > > Compare the following two methods: > 1) {{GridQueryProcessor.querySqlFields}} - old good entry point for query > execution; > 2) {{GridQueryProcessor.querySqlFieldsNoCache}} - new method for "cache-less" > execution. > Note how cache context is used in the first method: > 1) First, it helps determine whether query can be converted to "local" > 2) Second, it gets query parallelism of current cache, and if it differs from > {{1}}, then it turns on {{distributedJoins}}. > Neither of this happens in the second implementation. Moreover, I had to > throw an exception for local queries, as I didn't know how to handle them > properly. > We need to investigate and fix these two deficiencies somehow. Probably some > inputs from [~sergi.vladykin] would be required, to understand what is going > on. > Our ultimate goal is to make "cache-less" execution as good as the old one. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (IGNITE-5571) Make sure that cache-less execution works as good as cache-based
[ https://issues.apache.org/jira/browse/IGNITE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082465#comment-16082465 ] ASF GitHub Bot commented on IGNITE-5571: GitHub user alexpaschenko opened a pull request: https://github.com/apache/ignite/pull/2283 IGNITE-5571 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-5571 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/2283.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2283 commit 5aa613a4caa13250f4db19beb62f5c30e944be3b Author: Alexander PaschenkoDate: 2017-07-07T17:08:23Z IGNITE-5571 Started refactoring. commit df84a95b703146e0414246243e08b43a1ac9ea7f Author: Alexander Paschenko Date: 2017-07-09T22:32:15Z IGNITE-5571 Refactoring continued. commit c5966dd33e12d9e1855c165854b965a8633b1869 Author: Andrey Novikov Date: 2017-07-11T10:02:56Z IGNITE-5726 Web Console: Fixed duplication of dependencies in maven project. commit bbf55dde32d42162206a616308dd8509af0b8479 Author: Alexander Paschenko Date: 2017-07-11T10:13:57Z IGNITE-5571 Refactoring continued. commit a9ae95342dd566b210fe835221214ecc67e6ed16 Author: Pavel Tupitsyn Date: 2017-07-11T11:13:14Z IGNITE-5446 Always use late affinity assignment mode - fix .NET tests commit 1942db3af6538e6d1f0731b52eda94071f533ccb Author: Pavel Tupitsyn Date: 2017-07-11T11:23:56Z .NET: Fix code analysis warnings for deprecated IsLateAffinityAssignment commit 1b2f5052ed34776d14d7614812b78e0aa66c69b1 Author: Ivan Rakov Date: 2017-07-11T11:19:14Z Visor fails to connect to secure grid commit e9f194a328b1704a7881f1bf1a148a3426cd1b53 Author: Alexey Kuznetsov Date: 2017-07-11T12:16:06Z ignite-2.1 Improved web-console primitives. commit 914fdc7b83ea5e73d2ee665fbb32bfdedc5e2dc7 Author: Alexey Kuznetsov Date: 2017-07-11T12:16:58Z ignite-2.1 Web Console: improved demo. commit 66ccf8594cc6de74c3d87620bc3c85cd86b571fa Author: Alexey Kuznetsov Date: 2017-07-11T12:17:48Z ignite-2.1 Minor update of generated classnames. commit 2c737f0c6f42e983606e70faab5e9e6336b0a241 Author: Alexander Paschenko Date: 2017-07-11T12:23:26Z IGNITE-5711: Allowed to run queries on caches without query entities. This closes #2264. commit a0b56442365bad1100b078da33f00beb6a844cf0 Author: Sergey Chugunov Date: 2017-07-11T12:59:16Z Minor fix - stopping nodes after test commit 28ba989d7d864a8181cd9634a7d632a6cf38bb7e Author: Alexander Paschenko Date: 2017-07-11T16:25:05Z IGNITE-5571 Refactoring (almost) finished commit 54f3b92872faa6447049190e119c0179b6439f30 Author: Alexander Paschenko Date: 2017-07-11T16:25:56Z Merge remote-tracking branch 'origin/ignite-2.1.2' into ignite-5571 > Make sure that cache-less execution works as good as cache-based > > > Key: IGNITE-5571 > URL: https://issues.apache.org/jira/browse/IGNITE-5571 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Vladimir Ozerov >Assignee: Alexander Paschenko >Priority: Critical > Fix For: 2.2 > > > Compare the following two methods: > 1) {{GridQueryProcessor.querySqlFields}} - old good entry point for query > execution; > 2) {{GridQueryProcessor.querySqlFieldsNoCache}} - new method for "cache-less" > execution. > Note how cache context is used in the first method: > 1) First, it helps determine whether query can be converted to "local" > 2) Second, it gets query parallelism of current cache, and if it differs from > {{1}}, then it turns on {{distributedJoins}}. > Neither of this happens in the second implementation. Moreover, I had to > throw an exception for local queries, as I didn't know how to handle them > properly. > We need to investigate and fix these two deficiencies somehow. Probably some > inputs from [~sergi.vladykin] would be required, to understand what is going > on. > Our ultimate goal is to make "cache-less" execution as good as the old one. -- This message was sent by Atlassian JIRA (v6.4.14#64029)