[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-04-26 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984801#comment-15984801
 ] 

ASF GitHub Bot commented on IGNITE-4523:


Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1858


> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
>  Labels: important
> Fix For: 2.0
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-04-21 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15978960#comment-15978960
 ] 

ASF GitHub Bot commented on IGNITE-4523:


GitHub user ascherbakoff opened a pull request:

https://github.com/apache/ignite/pull/1858

IGNITE-4523

Allow distributed SQL query execution over explicit set of partitions.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ascherbakoff/ignite ignite-4523

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1858.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1858


commit 915d1d834afed68956772ec5ed09a137ec3e5c28
Author: ascherbakoff 
Date:   2017-04-01T19:08:09Z

IGNITE-4523 Merging with master.

commit 2a7c84383c7ab742e4d00b359469f8ab2ded6a82
Author: ascherbakoff 
Date:   2017-04-01T19:36:11Z

IGNITE-4523 Added test to suite.

commit 972719137c5cabb80ef363e6c1596bd7c532aa64
Author: ascherbakoff 
Date:   2017-04-02T11:49:54Z

IGNITE-4523 Removed code duplication.

commit 0b611e2b21f41ed1b3fe4c1caee0a062cb11673b
Author: ascherbakoff 
Date:   2017-04-02T12:49:52Z

Merge branch 'master' of https://github.com/apache/ignite into ignite-4523

commit 6bcb3f1e20fd628ee28c2d57c04e17f37c05d608
Author: ascherbakoff 
Date:   2017-04-02T13:13:04Z

IGNITE-4523 Throw an exception if no nodes mapping found.

commit 469677645d92a3ec484d02b82729ac88d87f8963
Author: ascherbakoff 
Date:   2017-04-02T13:23:30Z

IGNITE-4523 Throw an exception if no nodes mapping found.

commit 3d092e5bc89c4a62f0ffeff885356d3678162b02
Author: ascherbakoff 
Date:   2017-04-02T13:31:08Z

IGNITE-4523 Undo wrong commit.

commit 7fe979b3505cbe6f1c2758ab7c0b552829f14cb9
Author: Aleksei Scherbakov 
Date:   2017-04-12T14:12:09Z

IGNITE-4523 Improved partitions validation.

commit 8bd00a681ff0c315de1bf57a9eabf3da7e21281f
Author: Aleksei Scherbakov 
Date:   2017-04-12T16:04:28Z

IGNITE-4523 Fixing codestyle issues.

commit a56d7232e60e76cd75696c7fae9ccb5cb9a821cb
Author: Aleksei Scherbakov 
Date:   2017-04-14T15:15:58Z

IGNITE-4523 Separate field for query partitions.

commit 6df5feded90d1d5ce5e1673683eb8c65a834fff4
Author: Aleksei Scherbakov 
Date:   2017-04-14T17:12:18Z

IGNITE-4523 Unstable topology test.

commit 19b2d12187535ce6a538f88ca28fd3ab1ffb0329
Author: Aleksei Scherbakov 
Date:   2017-04-14T17:17:47Z

IGNITE-4523 Unstable topology test.

commit 81be5aac9ca9ec735bd89331d0de3a94c029612b
Author: Aleksei Scherbakov 
Date:   2017-04-14T17:19:46Z

IGNITE-4523 Unstable topology test.

commit ed2bed870bd49819ef8841d7db1991958e18ae28
Author: Aleksei Scherbakov 
Date:   2017-04-17T17:17:17Z

IGNITE-4523 Unstable topology test.

commit 1ac48adb88d086d4924f6dc2d4f95e0a7e42e1b2
Author: Aleksei Scherbakov 
Date:   2017-04-18T10:33:52Z

IGNITE-4523 Fixed issues for unstable topology.

commit 66a7eb3c6bd77d4af1a04f2224feac5c1a7e9fcb
Author: Sergi Vladykin 
Date:   2017-04-18T14:21:20Z

master - test adjusted

commit 5d8e31806ac8c013169c9892c0f88b79d6833358
Author: Anton Vinogradov 
Date:   2017-04-18T12:32:29Z

Special profiles to set versions and check compilation across all modules
(cherry picked from commit 3565e4e)

commit 0abf6601fc1ff1e6e659381666ae706e31b51de0
Author: Yury Babak 
Date:   2017-04-18T13:47:15Z

IGNITE-5000 Rename Ignite Math module to Ignite ML module
  added missed licenses
  renamed packages
  fixed wrong ml profile activation
(cherry picked from commit d78e071)

commit f66bd778a408340457937d59faf7c0b7b5638a89
Author: Aleksei Scherbakov 
Date:   2017-04-18T15:32:23Z

IGNITE-4523 Fixed tests.

commit 364c38661a1864e0b823be737998ed66927d11ca
Author: Alexey Goncharuk 
Date:   2017-04-18T15:40:27Z

master - Mute test with a correct link

commit 0da51d4110ab13f42f0ee5a2fc3181fc50656cd4
Author: Alexey Goncharuk 
Date:   2017-04-18T15:40:55Z

master - Fix mvcc tests after CLOCK mode removal

commit 27fd74f1ebd0cb89a99be69c5c74829307e3
Author: Ignite Teamcity 
Date:   2017-04-18T16:43:28Z

2.1.0-SNAPSHOT

commit

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-04-19 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15974176#comment-15974176
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

Merged 2.0 changes.
Added new test on unstable topology, fixed issues with unmapped partitions.
TC result looks acceptable [1]
Waiting for review [2]

[1] http://ci.ignite.apache.org/viewLog.html?buildId=564989;
[2] reviews.ignite.apache.org/ignite/review/IGNT-CR-155

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
>  Labels: important
> Fix For: 2.0
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-04-18 Thread Vladimir Ozerov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973362#comment-15973362
 ] 

Vladimir Ozerov commented on IGNITE-4523:
-

[~ascherbakov], [~sergi.vladykin], I propose to move the ticket to {{2.1}}. Too 
late for major changes in {{2.0}} scope.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
>  Labels: important
> Fix For: 2.0
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-04-11 Thread Sergi Vladykin (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964344#comment-15964344
 ] 

Sergi Vladykin commented on IGNITE-4523:


I reviewed the patch, my comments are in Upsource.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
>  Labels: important
> Fix For: 2.0
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-04-02 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15952727#comment-15952727
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

Made final fixes, waiting for TC.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
>  Labels: important
> Fix For: 2.0
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-04-02 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15952647#comment-15952647
 ] 

ASF GitHub Bot commented on IGNITE-4523:


Github user skalashnikov closed the pull request at:

https://github.com/apache/ignite/pull/1508


> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
>  Labels: important
> Fix For: 2.0
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-03-25 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941703#comment-15941703
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

Chances are very high because SQL related code is finished.

Next week I try finding time to revert changes related to non-sql queries and 
create separate ticket for it.

After what the ticket'll be ready for review.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
>  Labels: important
> Fix For: 2.0
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-03-22 Thread Denis Magda (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937600#comment-15937600
 ] 

Denis Magda commented on IGNITE-4523:
-

[~ascherbakov], the feature is planned for 2.0 release. What are the chances 
you complete it in time?

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
>  Labels: important
> Fix For: 2.0
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-02-08 Thread Sergey Kalashnikov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858197#comment-15858197
 ] 

Sergey Kalashnikov commented on IGNITE-4523:


Added the check to forbid queries that have both partitions and distributed 
joins options set.
Fixed some tests.
New PR https://github.com/apache/ignite/pull/1508

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-02-08 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858189#comment-15858189
 ] 

ASF GitHub Bot commented on IGNITE-4523:


GitHub user skalashnikov opened a pull request:

https://github.com/apache/ignite/pull/1508

IGNITE-4523 Allow distributed SQL query execution over explicit set of 
partitions



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4523

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1508.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1508


commit 153d4a374fe6cf6c80eb895920ec279f483ab1d7
Author: ascherbakoff 
Date:   2017-01-06T15:56:07Z

IGNITE-4523 WIP.

commit 31c5e42e67fc958e77f3014c364a8e14b236bd8d
Author: ascherbakoff 
Date:   2017-01-07T13:11:58Z

IGNITE-4523 WIP.

commit dea221c15b8e2136f35ab6ab0abacb3b658aa626
Author: Aleksei Scherbakov 
Date:   2017-01-10T17:30:10Z

IGNITE-4523 WIP.

commit 3238365c3c04fcd7db309650012f43aa26bd6a8d
Author: Aleksei Scherbakov 
Date:   2017-01-10T17:30:34Z

IGNITE-4523 WIP.

commit 570cfd7457feb6d1bb95ef4dbae77d0c8d0c5e6c
Author: Aleksei Scherbakov 
Date:   2017-01-10T17:32:31Z

Merge branch 'master' of https://github.com/apache/ignite into ignite-4523

commit 1fe53872446c31da15ac7ab2fe101bb3988bd4e8
Author: ascherbakoff 
Date:   2017-01-10T20:50:14Z

IGNITE-4523 Moved PartitionSet field to base class.

commit b2eda3058463c4eceb0054052b66fbd13783df18
Author: Aleksei Scherbakov 
Date:   2017-01-11T16:22:25Z

IGNITE-4523 Style cleanup.

commit d48aa541733f0725a73bd09d6f136b81137c729a
Author: Aleksei Scherbakov 
Date:   2017-01-11T16:30:36Z

IGNITE-4523 Style cleanup.

commit d8ee9edda9e74cc88ac5f756b7f59639ef4b9f3e
Author: Aleksei Scherbakov 
Date:   2017-01-11T16:33:14Z

Merge branch 'ignite-4523' of https://github.com/ascherbakoff/ignite into 
ignite-4523

commit f4992a0daed06f47dee7e21c5fc9aee5f3a79d6c
Author: Aleksei Scherbakov 
Date:   2017-01-11T16:42:22Z

IGNITE-4523 Style cleanup.

commit 4f2843e3c4a212d605d1627c66de61ae8304d386
Author: Aleksei Scherbakov 
Date:   2017-01-11T16:49:57Z

IGNITE-4523 Style cleanup.

commit 901c0e69a3cf28807dbdc182fc76f0ba51c36855
Author: Aleksei Scherbakov 
Date:   2017-01-11T17:03:54Z

IGNITE-4523 Style cleanup.

commit 4dcfaad50aac20370a96acbc6e6d0768b8f90b7c
Author: Aleksei Scherbakov 
Date:   2017-01-11T17:17:31Z

IGNITE-4523 Style cleanup.

commit 1945e63b684f4779a99ef32ca59df7650e03f358
Author: Aleksei Scherbakov 
Date:   2017-01-16T15:49:32Z

IGNITE-4523 Public API is updated.

commit 432c44f517bf25c14825328b233123f540fc4e03
Author: ascherbakoff 
Date:   2017-01-16T21:49:27Z

IGNITE-4523 GridIntSet.

commit af5cfddb483ef7f13c54eaf66fb1ca54c9063de9
Author: Aleksei Scherbakov 
Date:   2017-01-17T16:36:14Z

IGNITE-4523 Working on integer set.

commit 825adc6451dd8b3fe2bfa2be2a749eef7e27fa6b
Author: ascherbakoff 
Date:   2017-01-17T20:54:49Z

IGNITE-4523 Refactored to int[].

commit 24944785d46ff52dbd50672d6ee50854ae7b8bf9
Author: ascherbakoff 
Date:   2017-01-17T20:58:33Z

IGNITE-4523 Refactored to int[].

commit 3299916e00f2d707c64844b506832f3aea98
Author: ascherbakoff 
Date:   2017-01-17T21:06:13Z

IGNITE-4523 Refactored to int[].

commit defa40c5e456b8d67bec8379ee020a0df029ac1f
Author: Aleksei Scherbakov 
Date:   2017-01-18T17:17:05Z

IGNITE-4523 WIP

commit e8874398a2b882af769c5777f19544b9db843dc4
Author: Aleksei Scherbakov 
Date:   2017-01-18T17:17:33Z

IGNITE-4523 WIP

commit 0491acb5c4ae50ea0c55a6af186a17937b1868ec
Author: Aleksei Scherbakov 
Date:   2017-01-18T17:22:42Z

IGNITE-4523 Add test to suite.

commit 0f95f6902fa63d02b756eb970961a227a9c77f48
Author: ascherbakoff 
Date:   2017-01-18T21:01:10Z

IGNITE-4523 Better API validation.

commit f27e40c515336c037d5e0da6f9042c9a7839f2f8
Author: ascherbakoff 
Date:   2017-01-18T21:01:26Z

IGNITE-4523 Better API validation.

commit

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-02-02 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15850419#comment-15850419
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

Same for remote mode.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-02-01 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848906#comment-15848906
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

Finished adding support for multiple partitions in ScanQueries local mode.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-25 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838446#comment-15838446
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

Currently I'm working on extending ScanQuery to support multiple partitions for 
query execution.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-21 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832911#comment-15832911
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

After talking with the community, I decided to implement support for other 
query types.

Still need SQL queries part review.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-20 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831966#comment-15831966
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

I wanted to do so from the beginning, but Vladimir Ozerov has asked me to move 
it to upper level, and throw an exception for unsupported cases.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-20 Thread Sergi Vladykin (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831946#comment-15831946
 ] 

Sergi Vladykin commented on IGNITE-4523:


Then move setPartitions method out of Query and add it only to supported 
subclasses.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-20 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831943#comment-15831943
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

This ticket is about SQL queries. 

I have no plans for implementing partitions support  for other query types 
right now.

I suggest to do it in another ticket, and throw an exception from corresponding 
queries meanwhile.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-20 Thread Sergi Vladykin (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831873#comment-15831873
 ] 

Sergi Vladykin commented on IGNITE-4523:


If you have added method setPartitions on Query, did you make sure that all the 
Query descendants (like SpiQuery, ScanQuery, etc...) support it correctly? I 
think we have to have respective tests for all of them.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-20 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831859#comment-15831859
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

Added support for local queries.

As was requested on dev list, I'm adding short description of API changes:

Query class is extended with methods:

{noformat}
/**
 * Gets partitions for query, in ascending order.
 */
@Nullable public int[] getPartitions() {
return parts;
}

/**
 * Sets partitions for a query.
 * The query will be executed only on nodes which are primary for specified 
partitions.
 *
 * @param parts Partitions.
 * @return {@code this} for chaining.
 */
public Query setPartitions(@Nullable int... parts) {
this.parts = parts;

if (this.parts != null) {
A.notEmpty(parts, "Partitions");

// Validate partitions.
for (int i = 0; i < parts.length; i++) {
if (i < parts.length - 1)
A.ensure(parts[i] != parts[i + 1], "Partition duplicates 
are not allowed");

A.ensure(0 <= parts[i] && parts[i] < 
CacheConfiguration.MAX_PARTITIONS_COUNT, "Illegal partition");
}

Arrays.sort(this.parts);
}

return this;
}
{noformat}

ScanQuery class methods are marked for deprecation, because we already have 
get/setPartitions(int... parts):

{noformat}
/**
 * Sets partition number over which this query should iterate. If {@code 
null}, query will iterate over
 * all partitions in the cache. Must be in the range [0, N) where N is 
partition number in the cache.
 *
 * @param part Partition number over which this query should iterate.
 * @return {@code this} for chaining.
 *
 * @deprecated Use {@link #setPartitions(int... parts)} instead.}
 */
@Deprecated
public ScanQuery setPartition(@Nullable Integer part) {
if (part == null)
setPartitions(null);
else
setPartitions(part);

return this;
}

/**
 * Gets partition number over which this query should iterate. Will return 
{@code null} if partition was not
 * set. In this case query will iterate over all partitions in the cache.
 *
 * @return Partition number or {@code null}.
 *
 * @deprecated Use {@link #getPartitions()} instead.}
 */
@Deprecated
@Nullable public Integer getPartition() {
return getPartitions() == null ? null : getPartitions()[0];
}
{noformat}



> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-18 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828777#comment-15828777
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

I've found out what current implementation doens't work with distributed 
queries.

But I think this feature should be compatible with it.

Could anyone take a look at current PR state while I'm looking deeper into 
dsitributed queries code?

TC looks OK: http://ci.ignite.apache.org/viewQueued.html?itemId=424438

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-17 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826828#comment-15826828
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

Created the ticket https://issues.apache.org/jira/browse/IGNITE-4554

Finalized API changes. Waiting for TC results.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-13 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15821539#comment-15821539
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

PartitionSet hides internal implementation of int array. 

This allows as to efficiently represent contiguous range of partitions and 
defines clear API for ordered iteration over partitions and containment check.

Even better to go with compressed bitmap, as I mentioned in ticket comment. 
This will allow us to minimize heap footprint for this object.

Moreover, it will be useful to create reusable compressed bitmap implementation 
in Ignite and use it in other cases, on example, for replacing H2's IntArray 
and Set.

Should I create a ticket for this ?

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> 3Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-12 Thread Sergi Vladykin (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15821078#comment-15821078
 ] 

Sergi Vladykin commented on IGNITE-4523:


I looked at the code. The  {{PartitionSet}} concept looks overengineered to me, 
why wouldn't we just go with int[]?

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-12 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15820870#comment-15820870
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

Dmitriy, I briefly'll describe API changes as you asked on dev list.

I've introduced PartitionSet [1] class as a new field in Query class.

It represents random or contiguous ordered set of partitions.

Currently this field is only handled by SQL engine, other query types ignore it.

[1] 
https://github.com/ascherbakoff/ignite/blob/4dcfaad50aac20370a96acbc6e6d0768b8f90b7c/modules/core/src/main/java/org/apache/ignite/cache/query/PartitionSet.java

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-12 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15820857#comment-15820857
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

TC status is acceptable.

http://ci.ignite.apache.org/viewLog.html?buildId=415424=buildResultsDiv=IgniteTests_RunAll

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-10 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816183#comment-15816183
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

BTW, I think the best data structure for represeting partition set is a 
compressed bitmap. Why don't we have the implementation in Ignite? It may be 
useful in other curcumstances.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-10 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816147#comment-15816147
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

Almost ready for TC.

Have to add a test with distributed join.

PR: https://github.com/apache/ignite/pull/1418

Please review API changes please while I'm finishing the test.

I'm followed [~vozerov]'s API proposal, just added support for efficient 
partition range assignment.

This is handy in case we split partitions in ranges and assign independent data 
set to each range, which is described in details in IGNITE-4378.



> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 1.9
>
>
> Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-10 Thread Vladimir Ozerov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814257#comment-15814257
 ] 

Vladimir Ozerov commented on IGNITE-4523:
-

IGNITE-4510 is a bit different beast - I meant that in some cases our SQL 
engine can determine partitions automatically, e.g. when there is a simple 
{{WHERE}} condition on affinity column. Probably you meant IGNITE-4517 where I 
propose to have an API to set target partitions by hand.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 2.0
>
>
> Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-09 Thread Dmitriy Setrakyan (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813090#comment-15813090
 ] 

Dmitriy Setrakyan commented on IGNITE-4523:
---

I think this is a very questionable feature. We already plan support for 
sending a query to a right partition in IGNITE-4510, which should be enough. 
Don't think we need anything beyond that.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 2.0
>
>
> Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-09 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812657#comment-15812657
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

Added test for quering over partition subset per primary node.

Fixing it to not fail.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 2.0
>
>
> Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-07 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15807556#comment-15807556
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

I implemented query execution on nodes assigned as primary for partitions from 
a set passed to a query.

Implemented a test for the scenario mentioned in previous post: partitions are 
divided to subsets and each subset is bound to a some region, on example, 
geographic.

So we have data in different regions and are able to do distributed queries 
over a region without broadcasting to whole cluster and skip index definition 
on region.

Need to add more tests before making PR.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 2.0
>
>
> Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-05 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801815#comment-15801815
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

Vladimir, I suppose you are right.

Let's mark this as a duplicate for IGNITE-4517

As for partition definition, I mentioned an ability to define partition range 
in addition to random partition assignments.

I have real-world scenario where it'll be very useful. It's related to linked 
ticket IGNITE-4378.




> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 2.0
>
>
> Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-04 Thread Vladimir Ozerov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15800616#comment-15800616
 ] 

Vladimir Ozerov commented on IGNITE-4523:
-

Alex,

IGNITE-4517 is about distributed queries of course :-) This is why I though 
that this ticket is duplicate - they both aim to restrict distributed query 
execution to certain partitions. As per int[]. may be array is not the best 
choice. However, I also do not like partition range - it appears to be too 
specific. Usually partitions are distributed in "random" fashion from user 
perspective.

> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 2.0
>
>
> Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-4523) Allow distributed SQL query execution over explicit set of partitions

2017-01-04 Thread Alexei Scherbakov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15798722#comment-15798722
 ] 

Alexei Scherbakov commented on IGNITE-4523:
---

[~vozerov], I think IGNITE-4517 is more about local queries.

The main purpose of the issue is to restrict network traffic for distributed 
SQL queries. 

I need this functionality very urgently for one of my customers.

Also I don't like the approach with int[] arrays as arguments to Query.

In case if contiguos partitions it is much better to have something like 
{{PartitionSet}} with ability to define range for performance reasons.

I think it's better to fix this special case first, and when extend to 
IGNITE-4517.

What do you think?



> Allow distributed SQL query execution over explicit set of partitions
> -
>
> Key: IGNITE-4523
> URL: https://issues.apache.org/jira/browse/IGNITE-4523
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache, SQL
>Affects Versions: 1.8
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
> Fix For: 2.0
>
>
> Currently distributed SQL query is executed on all nodes containing primary 
> partitions for a cache, sending map query requests on all nodes in grid.
> Sometimes we know in advance which partitions hold a data for query, on 
> example, in case of custom affinity function. 
> Therefore it's possible to reduce number of nodes receiving map query request 
> by providing explicit set of partitions, which will give significant 
> performance advantage and traffic reduction in case of very large clusters.
> Internally we already have such functionality, so the only necessary thing is 
> to provide public API for what.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)