Analyzing clustered data with Tableau BI

[Denis Magda]

Thanks to Igor Sapego we finally prepared and polished technical documentation 
for the integration in between Apache Ignite and well know BI tool - Tableau. 
Enjoy ;)

https://apacheignite-mix.readme.io/docs/tableau 


Prachi, please do the final review. 

Is there anyone who would like to blog about this?

—
Denis

Re: Sort nodes in the ring in order to minimize the number of reconnections

[Denis Magda]

Alexander,

This is something different and looks unrelated to the discussion we have over 
here.

A transaction will not be rolled back the way you’re describing. It will be 
either committed once or rolled back once. There can be and will be inter nodes 
communication when something fails at the commit phase but this depends on how 
the affinity function distributes the keys and partitions and not how the nodes 
are connected at the discovery SPI layer.
 
Here you can learn more about failures handling by 2 phase commit protocol
http://gridgain.blogspot.com/2014/09/two-phase-commit-for-in-memory-caches.html 


—
Denis

> On Dec 23, 2016, at 12:24 PM, Александр Меньшиков  
> wrote:
> 
> I in fact worried about the following situation:
> 
> Like i said we have ring A->F->B->E->C->D->A, and connection between A,B,C
> and D,E,F was been broken. But nodes will detect the fact of the
> unavailability of nodes not at the same time. And meanwhile the client will
> perform transactional operations. Transactions may rollback many times in
> the following sequence of events:
> 
> 0. Everything is fine: A->F->B->E->C->D->A.
> 1. Connection between A,B,C and D,E,F is broken.
> 2. "A" sees "F" falls out of topology and reconnect to "B", all
> transactions using the "F" are rolled back and begin with backup node ("B",
> for example).
> 3. After that "B" sees "E" falls out of topology and reconnect to "C", all
> transaction using "E" are rolled back and begin with backup node ("C", for
> example).
> 4. After that "C" sees "D" falls out of topology and reconnect to "A", all
> transaction using "D" are rolled back and begin with backup node ("A", for
> example).
> 
> And we get 3 different set of rollbacks, instead one set of rollbacks.
> 
> 2016-12-23 22:43 GMT+03:00 Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
> 
>> Hi Vyacheslav,
>> 
>> Discovery logic is incapsulated in TcpDiscoverySpi.
>> TcpDiscoveryMulticastIpFinder in one of many implementations of IP finder.
>> The only purpose of the IP finder is to provide list of addresses where a
>> node can send initial join request, and the fact that it sends this initial
>> request to node A doesn't actually mean that it will be connected to A
>> within a ring. Having said that, I doubt that IP finder will be somehow
>> affected in case the discussed change is implemented.
>> 
>> Discovery protocol already maintains consistent information about the ring,
>> so any node in topology already knows everything about other nodes,
>> including ordering in the ring. So on discovery level it should not be very
>> difficult to customize where a joining node is placed on the ring.
>> 
>> However, here is the concern I have. Currently when a new node joins,
>> coordinator assigns order number to this node (e.g. if we already have
>> nodes 1,2 and 3, new node will have order 4). This node will then be the
>> last one on the ring, i.e. nodes are always ordered in the ring by this
>> order number (1->2->3->4->1). If we change this, we will basically allow a
>> node to be placed anywhere else (smth like 1->2->4->3->1). I'm not 100%
>> sure if this is going to cause issues, but sounds dangerous.
>> 
>> Yakov, can you please chime in and share your thoughts on this?
>> 
>> -Val
>> 
>> On Fri, Dec 23, 2016 at 2:46 AM, Vyacheslav Daradur 
>> wrote:
>> 
>>> Thanks for reply.
>>> 
>>> I have some questions:
>>> 
>>> 1. Where the logic of Ignite cluster building is realized? DiscoverySpi
>> and
>>> TcpDiscoveryMulticastIpFinder?
>>> 
>>> 2. Which standart Ignite metrics you can recommend to use for
>>> node-ordering?
>>> 
>>> 2016-12-22 19:08 GMT+03:00 Dmitriy Setrakyan :
>>> 
 I think having some user-defined ordering can be beneficial. However,
>> we
 are only talking about node discovery protocol here to maintain the
 cluster. All other communication between nodes happens directly (does
>> not
 go through the ring).
 
 D.
 
 On Thu, Dec 22, 2016 at 6:32 AM, Vyacheslav Daradur <
>> daradu...@gmail.com
 
 wrote:
 
> Hello, Alex!
> 
> I think it is a great idea.
> 
> I suggest to build communications between nodes on weight (or
>>> priority).
> 
> For example, ordering on latency:
> - nodes on one host = 1
> - nodes in one rack-blade = 2
> - nodes in one server-rack = 3
> - nodes in one physical cluster = 4
> - nodes in one subnet = 5
> - etc.
> 
> Maybe it'll be better to use some metrics from ClusterMetrics
>>> interface.
> 
> The algorithm of ordering can be implemented in a class such as
 Comparator
> and use it when we build a cluster or we select a place for a new
>> node.
> 
> --
> With best regards,
> Vyacheslav Daradur
> 
> 2016-12-22 13:59 GMT+03:00 Александр Меньшиков 

Re: Sort nodes in the ring in order to minimize the number of reconnections

[Александр Меньшиков]

I in fact worried about the following situation:

Like i said we have ring A->F->B->E->C->D->A, and connection between A,B,C
and D,E,F was been broken. But nodes will detect the fact of the
unavailability of nodes not at the same time. And meanwhile the client will
perform transactional operations. Transactions may rollback many times in
the following sequence of events:

0. Everything is fine: A->F->B->E->C->D->A.
1. Connection between A,B,C and D,E,F is broken.
2. "A" sees "F" falls out of topology and reconnect to "B", all
transactions using the "F" are rolled back and begin with backup node ("B",
for example).
3. After that "B" sees "E" falls out of topology and reconnect to "C", all
transaction using "E" are rolled back and begin with backup node ("C", for
example).
4. After that "C" sees "D" falls out of topology and reconnect to "A", all
transaction using "D" are rolled back and begin with backup node ("A", for
example).

And we get 3 different set of rollbacks, instead one set of rollbacks.

2016-12-23 22:43 GMT+03:00 Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Hi Vyacheslav,
>
> Discovery logic is incapsulated in TcpDiscoverySpi.
> TcpDiscoveryMulticastIpFinder in one of many implementations of IP finder.
> The only purpose of the IP finder is to provide list of addresses where a
> node can send initial join request, and the fact that it sends this initial
> request to node A doesn't actually mean that it will be connected to A
> within a ring. Having said that, I doubt that IP finder will be somehow
> affected in case the discussed change is implemented.
>
> Discovery protocol already maintains consistent information about the ring,
> so any node in topology already knows everything about other nodes,
> including ordering in the ring. So on discovery level it should not be very
> difficult to customize where a joining node is placed on the ring.
>
> However, here is the concern I have. Currently when a new node joins,
> coordinator assigns order number to this node (e.g. if we already have
> nodes 1,2 and 3, new node will have order 4). This node will then be the
> last one on the ring, i.e. nodes are always ordered in the ring by this
> order number (1->2->3->4->1). If we change this, we will basically allow a
> node to be placed anywhere else (smth like 1->2->4->3->1). I'm not 100%
> sure if this is going to cause issues, but sounds dangerous.
>
> Yakov, can you please chime in and share your thoughts on this?
>
> -Val
>
> On Fri, Dec 23, 2016 at 2:46 AM, Vyacheslav Daradur 
> wrote:
>
> > Thanks for reply.
> >
> > I have some questions:
> >
> > 1. Where the logic of Ignite cluster building is realized? DiscoverySpi
> and
> > TcpDiscoveryMulticastIpFinder?
> >
> > 2. Which standart Ignite metrics you can recommend to use for
> > node-ordering?
> >
> > 2016-12-22 19:08 GMT+03:00 Dmitriy Setrakyan :
> >
> > > I think having some user-defined ordering can be beneficial. However,
> we
> > > are only talking about node discovery protocol here to maintain the
> > > cluster. All other communication between nodes happens directly (does
> not
> > > go through the ring).
> > >
> > > D.
> > >
> > > On Thu, Dec 22, 2016 at 6:32 AM, Vyacheslav Daradur <
> daradu...@gmail.com
> > >
> > > wrote:
> > >
> > > > Hello, Alex!
> > > >
> > > > I think it is a great idea.
> > > >
> > > > I suggest to build communications between nodes on weight (or
> > priority).
> > > >
> > > > For example, ordering on latency:
> > > > - nodes on one host = 1
> > > > - nodes in one rack-blade = 2
> > > > - nodes in one server-rack = 3
> > > > - nodes in one physical cluster = 4
> > > > - nodes in one subnet = 5
> > > > - etc.
> > > >
> > > > Maybe it'll be better to use some metrics from ClusterMetrics
> > interface.
> > > >
> > > > The algorithm of ordering can be implemented in a class such as
> > > Comparator
> > > > and use it when we build a cluster or we select a place for a new
> node.
> > > >
> > > > --
> > > > With best regards,
> > > > Vyacheslav Daradur
> > > >
> > > > 2016-12-22 13:59 GMT+03:00 Александр Меньшиков  >:
> > > >
> > > > > Hello everyone,
> > > > >
> > > > > As far as I know nodes are connected in a ring. For example if i
> > have 6
> > > > > nodes, with names A, B, C, D, E, and F they can connect in ring any
> > > > > possible way: A-B-C-D-E-F-A, or A-F-B-E-C-D-A, etc. And if some
> node
> > > > falls
> > > > > out of topology neighboring nodes must reconnect. If nodes A,B and
> C
> > > > > located in the same physical location, and D, E and F in another,
> and
> > > in
> > > > > some time one physical location is not available in another, we can
> > get
> > > > > different number of reconnections. Best case scenario if we have
> ring
> > > > like
> > > > > A-B-CxD-E-FxA ('x' mean disconnect) -- then we get only one
> reconnect
> > > (C
> > > > > reconnect to A or F reconnect to D -- depending on what part of the
> > > > 

Re: Sort nodes in the ring in order to minimize the number of reconnections

[Valentin Kulichenko]

Hi Vyacheslav,

Discovery logic is incapsulated in TcpDiscoverySpi.
TcpDiscoveryMulticastIpFinder in one of many implementations of IP finder.
The only purpose of the IP finder is to provide list of addresses where a
node can send initial join request, and the fact that it sends this initial
request to node A doesn't actually mean that it will be connected to A
within a ring. Having said that, I doubt that IP finder will be somehow
affected in case the discussed change is implemented.

Discovery protocol already maintains consistent information about the ring,
so any node in topology already knows everything about other nodes,
including ordering in the ring. So on discovery level it should not be very
difficult to customize where a joining node is placed on the ring.

However, here is the concern I have. Currently when a new node joins,
coordinator assigns order number to this node (e.g. if we already have
nodes 1,2 and 3, new node will have order 4). This node will then be the
last one on the ring, i.e. nodes are always ordered in the ring by this
order number (1->2->3->4->1). If we change this, we will basically allow a
node to be placed anywhere else (smth like 1->2->4->3->1). I'm not 100%
sure if this is going to cause issues, but sounds dangerous.

Yakov, can you please chime in and share your thoughts on this?

-Val

On Fri, Dec 23, 2016 at 2:46 AM, Vyacheslav Daradur 
wrote:

> Thanks for reply.
>
> I have some questions:
>
> 1. Where the logic of Ignite cluster building is realized? DiscoverySpi and
> TcpDiscoveryMulticastIpFinder?
>
> 2. Which standart Ignite metrics you can recommend to use for
> node-ordering?
>
> 2016-12-22 19:08 GMT+03:00 Dmitriy Setrakyan :
>
> > I think having some user-defined ordering can be beneficial. However, we
> > are only talking about node discovery protocol here to maintain the
> > cluster. All other communication between nodes happens directly (does not
> > go through the ring).
> >
> > D.
> >
> > On Thu, Dec 22, 2016 at 6:32 AM, Vyacheslav Daradur  >
> > wrote:
> >
> > > Hello, Alex!
> > >
> > > I think it is a great idea.
> > >
> > > I suggest to build communications between nodes on weight (or
> priority).
> > >
> > > For example, ordering on latency:
> > > - nodes on one host = 1
> > > - nodes in one rack-blade = 2
> > > - nodes in one server-rack = 3
> > > - nodes in one physical cluster = 4
> > > - nodes in one subnet = 5
> > > - etc.
> > >
> > > Maybe it'll be better to use some metrics from ClusterMetrics
> interface.
> > >
> > > The algorithm of ordering can be implemented in a class such as
> > Comparator
> > > and use it when we build a cluster or we select a place for a new node.
> > >
> > > --
> > > With best regards,
> > > Vyacheslav Daradur
> > >
> > > 2016-12-22 13:59 GMT+03:00 Александр Меньшиков :
> > >
> > > > Hello everyone,
> > > >
> > > > As far as I know nodes are connected in a ring. For example if i
> have 6
> > > > nodes, with names A, B, C, D, E, and F they can connect in ring any
> > > > possible way: A-B-C-D-E-F-A, or A-F-B-E-C-D-A, etc. And if some node
> > > falls
> > > > out of topology neighboring nodes must reconnect. If nodes A,B and C
> > > > located in the same physical location, and D, E and F in another, and
> > in
> > > > some time one physical location is not available in another, we can
> get
> > > > different number of reconnections. Best case scenario if we have ring
> > > like
> > > > A-B-CxD-E-FxA ('x' mean disconnect) -- then we get only one reconnect
> > (C
> > > > reconnect to A or F reconnect to D -- depending on what part of the
> > > cluster
> > > > we leave alive). But now possible that case AxFxBxExCxDxA -- then we
> > get
> > > a
> > > > lot of reconnections (A to B, B to C, C to A -- in general n/2
> > > > reconnections, where n -- number of nodes). And i think to add
> > something
> > > to
> > > > ensure that we always have good sorting of nodes connections
> > > > (A-B-C-...-Z-A).
> > > >
> > > > Of course in real world we can have multiple levels of physical
> > > closeness.
> > > >
> > > > In my opinion enough to add one parameter of 'int' to configuration
> > (with
> > > > name like 'ExtraNodeOrder') and to change the method of comparison
> > nodes
> > > so
> > > > that it first compared the 'ExtraNodeOrder', and then according to
> the
> > > old
> > > > criterion (as far as I know Ignite use topology version). So if some
> > > users
> > > > have multiple levels of physical closeness, he can use different
> bits.
> > > For
> > > > example use 16 high bits for DC number, and low 16 bits for racks.
> > > >
> > > > Alternatively, we can add array of ‘int’ to configuration and compare
> > > nodes
> > > > in sequence from the zero element to the last.
> > > >
> > >
> >
>

Re: [jira] [Created] (IGNITE-4487) NPE on query execution

[Александр Меньшиков]

I found that: in inner class
'GridCacheQueryAdapter.ScanQueryFallbackClosableIterator' in constructor is
called with method 'init()', but method 'init()' cannot be called with an
empty field 'nodes'. In source code it looks like:


private ScanQueryFallbackClosableIterator(int part, GridCacheQueryAdapter
qry,
GridCacheQueryManager qryMgr, GridCacheContext cctx) {
this.qry = qry;
this.qryMgr = qryMgr;
this.cctx = cctx;
this.part = part;

nodes = fallbacks(cctx.discovery().topologyVersionEx());
// !!! Here nodes.isEmpty()==true, and init() will fail in the
future. !!!
init();
}


I can fix it by adding some check in code, but i must know what behavior
are best in this case? As I understand it, the list of nodes is empty if
there are no nodes with the current partition, which means data loss, and
either need to return a meaningful exception, or ignore this situation. But
maybe I missed something.


2016-12-23 14:47 GMT+03:00 Dmitry Karachentsev (JIRA) :

> Dmitry Karachentsev created IGNITE-4487:
> ---
>
>  Summary: NPE on query execution
>  Key: IGNITE-4487
>  URL: https://issues.apache.org/jira/browse/IGNITE-4487
>  Project: Ignite
>   Issue Type: Bug
>   Components: cache
> Affects Versions: 1.8
> Reporter: Dmitry Karachentsev
>  Fix For: 2.0
>
>
> NPE may be thrown when called destroyCache() and started querying.
>
> Attached example reproduces this case when 
> GridDiscoveryManager#removeCacheFilter
> called but cache state haven't been changed to STOPPED
> org.apache.ignite.internal.processors.cache.GridCacheGateway#onStopped.
> {code:none}
> javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException:
> null
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheProxy.query(IgniteCacheProxy.java:740)
> at com.intellica.evam.engine.event.future.FutureEventWorker.
> processFutureEvents(FutureEventWorker.java:117)
> at com.intellica.evam.engine.event.future.FutureEventWorker.run(
> FutureEventWorker.java:66)
> Caused by: class org.apache.ignite.IgniteCheckedException: null
> at org.apache.ignite.internal.processors.query.GridQueryProcessor.
> executeQuery(GridQueryProcessor.java:1693)
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheProxy.query(IgniteCacheProxy.java:494)
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheProxy.query(IgniteCacheProxy.java:732)
> ... 2 more
> Caused by: java.lang.NullPointerException
> at org.apache.ignite.internal.processors.cache.query.
> GridCacheQueryAdapter$ScanQueryFallbackClosableIterator.init(
> GridCacheQueryAdapter.java:712)
> at org.apache.ignite.internal.processors.cache.query.
> GridCacheQueryAdapter$ScanQueryFallbackClosableIterator.(
> GridCacheQueryAdapter.java:677)
> at org.apache.ignite.internal.processors.cache.query.
> GridCacheQueryAdapter$ScanQueryFallbackClosableIterator.(
> GridCacheQueryAdapter.java:628)
> at org.apache.ignite.internal.processors.cache.query.
> GridCacheQueryAdapter.executeScanQuery(GridCacheQueryAdapter.java:548)
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheProxy$2.applyx(IgniteCacheProxy.java:497)
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheProxy$2.applyx(IgniteCacheProxy.java:495)
> at org.apache.ignite.internal.util.lang.IgniteOutClosureX.
> apply(IgniteOutClosureX.java:36)
> at org.apache.ignite.internal.processors.query.GridQueryProcessor.
> executeQuery(GridQueryProcessor.java:1670)
> ... 4 more
> {code}
>
>
>
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>

Re: DML Documentation Readiness

[Denis Magda]

The URL will be the same as the name of the page which is “Performance and 
Debugging”. I wouldn’t not rename the page because it’s already located under 
“SQL Grid” section. However, even if we rename the page the URL will be left 
the same. This is how readme.io works.

—
Denis

> On Dec 23, 2016, at 10:33 AM, Dmitriy Setrakyan  wrote:
> 
> Shouldn't the URL be "sql-performance-and-debugging"?
> 
> On Fri, Dec 23, 2016 at 10:30 AM, Denis Magda  wrote:
> 
>> Alex, great! Thanks for sharing more details about DML feature.
>> 
>> I’ll put this new content on this page after the review is finished
>> https://apacheignite.readme.io/docs/performance-and-debugging <
>> https://apacheignite.readme.io/docs/performance-and-debugging>
>> 
>> —
>> Denis
>> 
>>> On Dec 23, 2016, at 6:13 AM, Alexander Paschenko <
>> alexander.a.pasche...@gmail.com> wrote:
>>> 
>>> Denis,
>>> 
>>> I have also added a doc titled 'Fast Updates' - it's a separate
>>> document and it's hidden for now, please find the link to it in dash
>>> at the bottom of 'SQL Grid' section on the left. It describes an
>>> optimization technique that allows to run particular queries with
>>> performance of bare cache operations reducing SQL related overhead to
>>> nearly none at all. Hence I thought it was important to describe this
>>> feature. Please review it and include into main doc as your heart
>>> desires.
>>> 
>>> - Alex
>>> 
>>> 2016-12-23 14:42 GMT+03:00 Alexander Paschenko
>>> :
 Denis, Prachi,
 
 All links from ToC to subsections are broken on this page:
 http://apacheignite.gridgain.org/docs/dml
 
 - Alex
 
 2016-12-23 3:54 GMT+03:00 Prachi Garg :
> Sure, will look into it.
> 
> -P
> 
> On Thu, Dec 22, 2016 at 4:00 PM, Denis Magda 
>> wrote:
> 
>> I’ve finished the review and released all the parts of DML
>> documentation
>> [1].
>> 
>> Prachi, could you review whole SQL Grid documentation [2] including
>> [1]?
>> Also it will be great if you check this newly added section [3].
>> 
>> [1] http://apacheignite.gridgain.org/docs/dml
>> [2] http://apacheignite.gridgain.org/docs/sql-grid
>> [3] http://apacheignite.gridgain.org/docs/binary-
>> marshaller#handling-hash-code-generation-and-equals-execution
>> 
>> —
>> Denis
>> 
>> On Dec 9, 2016, at 12:34 AM, Alexander Paschenko <
>> alexander.a.pasche...@gmail.com> wrote:
>> 
>> Denis,
>> 
>> About your question: 'putIfAbsent' is used only when we have single
>> key-value pair to process as long as we don't have method like 'put
>> all if absent'. When there's a bunch of pairs, an entry processor is
>> created for each pair to make its processing atomic (effectively CAS
>> style processing), and all those processors are then executed via
>> 'invokeAll'.
>> 
>> - Alex
>> 
>> 2016-12-09 8:17 GMT+03:00 Denis Magda :
>> 
>> Alexander, thanks for all the material you’ve prepared so far.
>> 
>> Actually, I’ve reviewed, restructured and released only a part of it
>> [1].
>> 
>> The rest, that is missing, requires more time from my side and I’ll be
>> releasing additional blocks throughout the next week. The full
>> documentation
>> with the unreviewed parts is invisible and located here [2].
>> 
>> Please don’t modify neither of the docs ([1] and [2]) until I’ve
>> finished
>> with the overall review. Send all your suggestions over the email.
>> 
>> At the moment I have the following question. What cache method(s) do
>> we
>> convert INSERT statements to [3]? Is it ‘putIfAbsent’?
>> 
>> [1] https://apacheignite.readme.io/docs/dml
>> [2] https://apacheignite.readme.io/docs/distributed-dml-to-be-reviewe
>> [3] https://apacheignite.readme.io/docs/dml#section-insert
>> 
>> —
>> Denis
>> 
>> On Dec 8, 2016, at 1:35 AM, Alexander Paschenko
>>  wrote:
>> 
>> Denis,
>> 
>> it's all ready, JDBC driver page is also updated (I've created a
>> hidden clone page, please find it on the left) as well as one on
>> binary marshaller. Also I've renamed 'distributed DML' page to just
>> 'DML' as long as there's much less to distribution of DML operations
>> themselves than to queries and there's clearly no need to make another
>> doc page to distinguish. Removed batches mention from limitations.
>> 
>> - Alex
>> 
>> 2016-12-08 5:26 GMT+03:00 Denis Magda :
>> 
>> Alexander,
>> 
>> How close are you to the finalization of the DML doc?
>> https://apacheignite.readme.io/docs/distributed-dml
>> 
>> Since we’re approaching 1.8 release I would like to do a final review
>> of it
>> polishing whatever 

Re: DML Documentation Readiness

[Dmitriy Setrakyan]

Shouldn't the URL be "sql-performance-and-debugging"?

On Fri, Dec 23, 2016 at 10:30 AM, Denis Magda  wrote:

> Alex, great! Thanks for sharing more details about DML feature.
>
> I’ll put this new content on this page after the review is finished
> https://apacheignite.readme.io/docs/performance-and-debugging <
> https://apacheignite.readme.io/docs/performance-and-debugging>
>
> —
> Denis
>
> > On Dec 23, 2016, at 6:13 AM, Alexander Paschenko <
> alexander.a.pasche...@gmail.com> wrote:
> >
> > Denis,
> >
> > I have also added a doc titled 'Fast Updates' - it's a separate
> > document and it's hidden for now, please find the link to it in dash
> > at the bottom of 'SQL Grid' section on the left. It describes an
> > optimization technique that allows to run particular queries with
> > performance of bare cache operations reducing SQL related overhead to
> > nearly none at all. Hence I thought it was important to describe this
> > feature. Please review it and include into main doc as your heart
> > desires.
> >
> > - Alex
> >
> > 2016-12-23 14:42 GMT+03:00 Alexander Paschenko
> > :
> >> Denis, Prachi,
> >>
> >> All links from ToC to subsections are broken on this page:
> >> http://apacheignite.gridgain.org/docs/dml
> >>
> >> - Alex
> >>
> >> 2016-12-23 3:54 GMT+03:00 Prachi Garg :
> >>> Sure, will look into it.
> >>>
> >>> -P
> >>>
> >>> On Thu, Dec 22, 2016 at 4:00 PM, Denis Magda 
> wrote:
> >>>
>  I’ve finished the review and released all the parts of DML
> documentation
>  [1].
> 
>  Prachi, could you review whole SQL Grid documentation [2] including
> [1]?
>  Also it will be great if you check this newly added section [3].
> 
>  [1] http://apacheignite.gridgain.org/docs/dml
>  [2] http://apacheignite.gridgain.org/docs/sql-grid
>  [3] http://apacheignite.gridgain.org/docs/binary-
>  marshaller#handling-hash-code-generation-and-equals-execution
> 
>  —
>  Denis
> 
>  On Dec 9, 2016, at 12:34 AM, Alexander Paschenko <
>  alexander.a.pasche...@gmail.com> wrote:
> 
>  Denis,
> 
>  About your question: 'putIfAbsent' is used only when we have single
>  key-value pair to process as long as we don't have method like 'put
>  all if absent'. When there's a bunch of pairs, an entry processor is
>  created for each pair to make its processing atomic (effectively CAS
>  style processing), and all those processors are then executed via
>  'invokeAll'.
> 
>  - Alex
> 
>  2016-12-09 8:17 GMT+03:00 Denis Magda :
> 
>  Alexander, thanks for all the material you’ve prepared so far.
> 
>  Actually, I’ve reviewed, restructured and released only a part of it
> [1].
> 
>  The rest, that is missing, requires more time from my side and I’ll be
>  releasing additional blocks throughout the next week. The full
>  documentation
>  with the unreviewed parts is invisible and located here [2].
> 
>  Please don’t modify neither of the docs ([1] and [2]) until I’ve
> finished
>  with the overall review. Send all your suggestions over the email.
> 
>  At the moment I have the following question. What cache method(s) do
> we
>  convert INSERT statements to [3]? Is it ‘putIfAbsent’?
> 
>  [1] https://apacheignite.readme.io/docs/dml
>  [2] https://apacheignite.readme.io/docs/distributed-dml-to-be-reviewe
>  [3] https://apacheignite.readme.io/docs/dml#section-insert
> 
>  —
>  Denis
> 
>  On Dec 8, 2016, at 1:35 AM, Alexander Paschenko
>   wrote:
> 
>  Denis,
> 
>  it's all ready, JDBC driver page is also updated (I've created a
>  hidden clone page, please find it on the left) as well as one on
>  binary marshaller. Also I've renamed 'distributed DML' page to just
>  'DML' as long as there's much less to distribution of DML operations
>  themselves than to queries and there's clearly no need to make another
>  doc page to distinguish. Removed batches mention from limitations.
> 
>  - Alex
> 
>  2016-12-08 5:26 GMT+03:00 Denis Magda :
> 
>  Alexander,
> 
>  How close are you to the finalization of the DML doc?
>  https://apacheignite.readme.io/docs/distributed-dml
> 
>  Since we’re approaching 1.8 release I would like to do a final review
> of it
>  polishing whatever is needed tomorrow.
> 
>  As I see that only the limitations section is left. The first note is
> that
>  I
>  wouldn’t mention “JDBC Batching mode” for now because this is
> something
>  that
>  doesn’t affect the usability and SQL API's scope of support.
> 
>  —
>  Denis
> 
> 
> 
>
>

Re: Update ignite version for Camel, Zeppelin, Vert.x and MyBatis

[Denis Magda]

Thanks, guys!

—
Denis

> On Dec 23, 2016, at 6:23 AM, Andrey Gura  wrote:
> 
> Vert.x - updated.
> Zeppelin - PR created and should be merged soon.
> 
> On Mon, Dec 19, 2016 at 3:50 PM, Andrey Gura  wrote:
>> Denis,
>> 
>> I'll update Vert.x and Zeppelin dependency versions during this week.
>> 
>> On Tue, Dec 13, 2016 at 11:42 PM, Denis Magda  wrote:
>>> Raul K., Andrey G., Roman S.,
>>> 
>>> Would you mind updating pom files of the external projects [1] Ignite is
>>> integrated with by setting its version to 1.8.0?
>>> 
>>> https://cwiki.apache.org/confluence/display/IGNITE/External+Integrations
>>> 
>>> —
>>> Denis

Re: DML Documentation Readiness

[Denis Magda]

Alex, great! Thanks for sharing more details about DML feature.

I’ll put this new content on this page after the review is finished
https://apacheignite.readme.io/docs/performance-and-debugging 


—
Denis

> On Dec 23, 2016, at 6:13 AM, Alexander Paschenko 
>  wrote:
> 
> Denis,
> 
> I have also added a doc titled 'Fast Updates' - it's a separate
> document and it's hidden for now, please find the link to it in dash
> at the bottom of 'SQL Grid' section on the left. It describes an
> optimization technique that allows to run particular queries with
> performance of bare cache operations reducing SQL related overhead to
> nearly none at all. Hence I thought it was important to describe this
> feature. Please review it and include into main doc as your heart
> desires.
> 
> - Alex
> 
> 2016-12-23 14:42 GMT+03:00 Alexander Paschenko
> :
>> Denis, Prachi,
>> 
>> All links from ToC to subsections are broken on this page:
>> http://apacheignite.gridgain.org/docs/dml
>> 
>> - Alex
>> 
>> 2016-12-23 3:54 GMT+03:00 Prachi Garg :
>>> Sure, will look into it.
>>> 
>>> -P
>>> 
>>> On Thu, Dec 22, 2016 at 4:00 PM, Denis Magda  wrote:
>>> 
 I’ve finished the review and released all the parts of DML documentation
 [1].
 
 Prachi, could you review whole SQL Grid documentation [2] including [1]?
 Also it will be great if you check this newly added section [3].
 
 [1] http://apacheignite.gridgain.org/docs/dml
 [2] http://apacheignite.gridgain.org/docs/sql-grid
 [3] http://apacheignite.gridgain.org/docs/binary-
 marshaller#handling-hash-code-generation-and-equals-execution
 
 —
 Denis
 
 On Dec 9, 2016, at 12:34 AM, Alexander Paschenko <
 alexander.a.pasche...@gmail.com> wrote:
 
 Denis,
 
 About your question: 'putIfAbsent' is used only when we have single
 key-value pair to process as long as we don't have method like 'put
 all if absent'. When there's a bunch of pairs, an entry processor is
 created for each pair to make its processing atomic (effectively CAS
 style processing), and all those processors are then executed via
 'invokeAll'.
 
 - Alex
 
 2016-12-09 8:17 GMT+03:00 Denis Magda :
 
 Alexander, thanks for all the material you’ve prepared so far.
 
 Actually, I’ve reviewed, restructured and released only a part of it [1].
 
 The rest, that is missing, requires more time from my side and I’ll be
 releasing additional blocks throughout the next week. The full
 documentation
 with the unreviewed parts is invisible and located here [2].
 
 Please don’t modify neither of the docs ([1] and [2]) until I’ve finished
 with the overall review. Send all your suggestions over the email.
 
 At the moment I have the following question. What cache method(s) do we
 convert INSERT statements to [3]? Is it ‘putIfAbsent’?
 
 [1] https://apacheignite.readme.io/docs/dml
 [2] https://apacheignite.readme.io/docs/distributed-dml-to-be-reviewe
 [3] https://apacheignite.readme.io/docs/dml#section-insert
 
 —
 Denis
 
 On Dec 8, 2016, at 1:35 AM, Alexander Paschenko
  wrote:
 
 Denis,
 
 it's all ready, JDBC driver page is also updated (I've created a
 hidden clone page, please find it on the left) as well as one on
 binary marshaller. Also I've renamed 'distributed DML' page to just
 'DML' as long as there's much less to distribution of DML operations
 themselves than to queries and there's clearly no need to make another
 doc page to distinguish. Removed batches mention from limitations.
 
 - Alex
 
 2016-12-08 5:26 GMT+03:00 Denis Magda :
 
 Alexander,
 
 How close are you to the finalization of the DML doc?
 https://apacheignite.readme.io/docs/distributed-dml
 
 Since we’re approaching 1.8 release I would like to do a final review of it
 polishing whatever is needed tomorrow.
 
 As I see that only the limitations section is left. The first note is that
 I
 wouldn’t mention “JDBC Batching mode” for now because this is something
 that
 doesn’t affect the usability and SQL API's scope of support.
 
 —
 Denis
 
 
 

Re: SELECT subqueries in DML statements

[Denis Magda]

Sergi,

Thanks for the feedback. I’ve updated the doc. Please have a look at it one 
more time and do additional corrections if it help to make things clearer
https://apacheignite.readme.io/v1.8/docs/dml#known-limitations 


If refer to the ticket are you working on [1], could you list the cases when 
the subqueries will be distributed further at the main query is executed on the 
node? In the ticket you briefly mention group by, aggregation and unions. Would 
be great to document this info providing examples.

[1] https://issues.apache.org/jira/browse/IGNITE-3860 


—
Denis

> On Dec 22, 2016, at 10:31 PM, Sergi Vladykin  wrote:
> 
> 1. Actually this doc looks misleading and wrong to me. What does it mean "it 
> will not be distributed"? It only means that we will not execute any 
> distributed joins inside of this subquery. In all other means this operation 
> will be correctly distributed.
> 
> We will generate query like this : 
> 
> SELECT _key, _val FROM Person WHERE _key IN (subquery)
> 
> And execute it on all the nodes and if the subquery is correctly colocated 
> with Person everything will work fine.
> 
> 2. This is not something we are going to change because subquery will be 
> executed for each row in Person. If it will have a distributed join in it, 
> this operation will never return and most probably will kill the cluster by 
> message flooding.
> 
> Sergi
> 
> 
> 
> 2016-12-23 3:05 GMT+03:00 Denis Magda  >:
> Sergi,
> 
> Presently in the DML statement doc we mention the following limitation 
> related to SELECT subqueries
> http://apacheignite.gridgain.org/docs/dml#section-subqueries-in-where-clause 
> 
> 
> Does it mean that this limitation will be no longer true for DML statements 
> as well once the improvement below gets released?
> https://issues.apache.org/jira/browse/IGNITE-3860 
> 
> 
> —
> Denis
> 

Re: EXPLAIN for DML statements

[Denis Magda]

I got annoyed with readme.io that periodically switches to the legacy 
apacheignite.gridgain.org domain. Hope that with the latest upgrade to 
readme.io 2.0 this is no longer an issue.

https://apacheignite.readme.io/docs/dml#section-explain-support-for-dml-statements
 


—
Denis

> On Dec 23, 2016, at 9:44 AM, Dmitriy Setrakyan  wrote:
> 
> Denis, the link does not work again.
> 
> On Thu, Dec 22, 2016 at 4:09 PM, Denis Magda  wrote:
> 
>> Alex P., Sergi,
>> 
>> All we know that EXPLAIN is not supported for DML statements at the moment.
>> http://apacheignite.gridgain.org/docs/dml#section-explain-
>> support-for-dml-statements > org/docs/dml#section-explain-support-for-dml-statements>
>> 
>> However, I can’t get how I can get over this limitation for UPDATE and
>> DELETE statements following this recommendation
>> 
>> "One possible approach is to execute EXPLAIN for the SELECT automatically
>> generated (UPDATE, DELETE) or used (INSERT, MERGE) by DML statements. This
>> will give an insight on the indexes that are used while a DML operation is
>> executed.”
>> 
>> How can I get SELECT queries that are generated by these DML operations
>> automatically?
>> 
>> —
>> Denis

Re: EXPLAIN for DML statements

[Dmitriy Setrakyan]

Denis, the link does not work again.

On Thu, Dec 22, 2016 at 4:09 PM, Denis Magda  wrote:

> Alex P., Sergi,
>
> All we know that EXPLAIN is not supported for DML statements at the moment.
> http://apacheignite.gridgain.org/docs/dml#section-explain-
> support-for-dml-statements  org/docs/dml#section-explain-support-for-dml-statements>
>
> However, I can’t get how I can get over this limitation for UPDATE and
> DELETE statements following this recommendation
>
> "One possible approach is to execute EXPLAIN for the SELECT automatically
> generated (UPDATE, DELETE) or used (INSERT, MERGE) by DML statements. This
> will give an insight on the indexes that are used while a DML operation is
> executed.”
>
> How can I get SELECT queries that are generated by these DML operations
> automatically?
>
> —
> Denis

Re: DML Documentation Readiness

[Dmitriy Setrakyan]

Denis, I cannot access the links. Are you sure they are correct?

On Thu, Dec 22, 2016 at 4:00 PM, Denis Magda  wrote:

> I’ve finished the review and released all the parts of DML documentation
> [1].
>
> Prachi, could you review whole SQL Grid documentation [2] including [1]?
> Also it will be great if you check this newly added section [3].
>
> [1] http://apacheignite.gridgain.org/docs/dml <
> http://apacheignite.gridgain.org/docs/dml>
> [2] http://apacheignite.gridgain.org/docs/sql-grid <
> http://apacheignite.gridgain.org/docs/sql-grid>
> [3] http://apacheignite.gridgain.org/docs/binary-marshaller#
> handling-hash-code-generation-and-equals-execution <
> http://apacheignite.gridgain.org/docs/binary-marshaller#
> handling-hash-code-generation-and-equals-execution>
>
> —
> Denis
>
> > On Dec 9, 2016, at 12:34 AM, Alexander Paschenko <
> alexander.a.pasche...@gmail.com> wrote:
> >
> > Denis,
> >
> > About your question: 'putIfAbsent' is used only when we have single
> > key-value pair to process as long as we don't have method like 'put
> > all if absent'. When there's a bunch of pairs, an entry processor is
> > created for each pair to make its processing atomic (effectively CAS
> > style processing), and all those processors are then executed via
> > 'invokeAll'.
> >
> > - Alex
> >
> > 2016-12-09 8:17 GMT+03:00 Denis Magda >:
> >> Alexander, thanks for all the material you’ve prepared so far.
> >>
> >> Actually, I’ve reviewed, restructured and released only a part of it
> [1].
> >>
> >> The rest, that is missing, requires more time from my side and I’ll be
> >> releasing additional blocks throughout the next week. The full
> documentation
> >> with the unreviewed parts is invisible and located here [2].
> >>
> >> Please don’t modify neither of the docs ([1] and [2]) until I’ve
> finished
> >> with the overall review. Send all your suggestions over the email.
> >>
> >> At the moment I have the following question. What cache method(s) do we
> >> convert INSERT statements to [3]? Is it ‘putIfAbsent’?
> >>
> >> [1] https://apacheignite.readme.io/docs/dml
> >> [2] https://apacheignite.readme.io/docs/distributed-dml-to-be-reviewe <
> https://apacheignite.readme.io/docs/distributed-dml-to-be-reviewe>
> >> [3] https://apacheignite.readme.io/docs/dml#section-insert <
> https://apacheignite.readme.io/docs/dml#section-insert>
> >>
> >> —
> >> Denis
> >>
> >> On Dec 8, 2016, at 1:35 AM, Alexander Paschenko
> >>  wrote:
> >>
> >> Denis,
> >>
> >> it's all ready, JDBC driver page is also updated (I've created a
> >> hidden clone page, please find it on the left) as well as one on
> >> binary marshaller. Also I've renamed 'distributed DML' page to just
> >> 'DML' as long as there's much less to distribution of DML operations
> >> themselves than to queries and there's clearly no need to make another
> >> doc page to distinguish. Removed batches mention from limitations.
> >>
> >> - Alex
> >>
> >> 2016-12-08 5:26 GMT+03:00 Denis Magda :
> >>
> >> Alexander,
> >>
> >> How close are you to the finalization of the DML doc?
> >> https://apacheignite.readme.io/docs/distributed-dml
> >>
> >> Since we’re approaching 1.8 release I would like to do a final review
> of it
> >> polishing whatever is needed tomorrow.
> >>
> >> As I see that only the limitations section is left. The first note is
> that I
> >> wouldn’t mention “JDBC Batching mode” for now because this is something
> that
> >> doesn’t affect the usability and SQL API's scope of support.
> >>
> >> —
> >> Denis
>
>

[jira] [Created] (IGNITE-4490) Optimize DML for fast INSERT and MERGE

[Alexander Paschenko (JIRA)]

Alexander Paschenko created IGNITE-4490:
---

 Summary: Optimize DML for fast INSERT and MERGE
 Key: IGNITE-4490
 URL: https://issues.apache.org/jira/browse/IGNITE-4490
 Project: Ignite
  Issue Type: Improvement
  Components: SQL
Affects Versions: 1.8
Reporter: Alexander Paschenko
Assignee: Alexander Paschenko
 Fix For: 1.8


It's possible to avoid any SQL querying and map some INSERT and MERGE 
statements to cache operations in a way similar to that of UPDATE and DELETE - 
i.e. don't make queries when there are no expressions to evaluate in the query 
and enhance update plans to perform direct cache operations when INSERT and 
MERGE affect columns _key and _value only.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4489) Maintain correct MERGE semantic in DML

[Alexander Paschenko (JIRA)]

Alexander Paschenko created IGNITE-4489:
---

 Summary: Maintain correct MERGE semantic in DML
 Key: IGNITE-4489
 URL: https://issues.apache.org/jira/browse/IGNITE-4489
 Project: Ignite
  Issue Type: Bug
  Components: SQL
Affects Versions: 1.8
Reporter: Alexander Paschenko
Assignee: Alexander Paschenko
 Fix For: 1.8


Currently it's impossible to MERGE object in UPDATE style - i.e. when key is 
present in cache, unaffected field values should be retained, and instead of 
building new object we should base it on previous one for given key.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-4488) ODBC: First query execution takes much more time than any subsequent

[Igor Sapego (JIRA)]

Igor Sapego created IGNITE-4488:
---

 Summary: ODBC: First query execution takes much more time than any 
subsequent
 Key: IGNITE-4488
 URL: https://issues.apache.org/jira/browse/IGNITE-4488
 Project: Ignite
  Issue Type: Bug
  Components: odbc, platforms
Affects Versions: 1.8
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.0


When we perform SQL query on small dataset just to measure latency, the very 
first call takes much more time to execute than the same subsequent requests 
(~70ms vs ~10ms).

The same behavior can be observed with C++ client.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Update ignite version for Camel, Zeppelin, Vert.x and MyBatis

[Andrey Gura]

Vert.x - updated.
Zeppelin - PR created and should be merged soon.

On Mon, Dec 19, 2016 at 3:50 PM, Andrey Gura  wrote:
> Denis,
>
> I'll update Vert.x and Zeppelin dependency versions during this week.
>
> On Tue, Dec 13, 2016 at 11:42 PM, Denis Magda  wrote:
>> Raul K., Andrey G., Roman S.,
>>
>> Would you mind updating pom files of the external projects [1] Ignite is
>> integrated with by setting its version to 1.8.0?
>>
>> https://cwiki.apache.org/confluence/display/IGNITE/External+Integrations
>>
>> —
>> Denis

Re: DML Documentation Readiness

[Alexander Paschenko]

Denis,

I have also added a doc titled 'Fast Updates' - it's a separate
document and it's hidden for now, please find the link to it in dash
at the bottom of 'SQL Grid' section on the left. It describes an
optimization technique that allows to run particular queries with
performance of bare cache operations reducing SQL related overhead to
nearly none at all. Hence I thought it was important to describe this
feature. Please review it and include into main doc as your heart
desires.

- Alex

2016-12-23 14:42 GMT+03:00 Alexander Paschenko
:
> Denis, Prachi,
>
> All links from ToC to subsections are broken on this page:
> http://apacheignite.gridgain.org/docs/dml
>
> - Alex
>
> 2016-12-23 3:54 GMT+03:00 Prachi Garg :
>> Sure, will look into it.
>>
>> -P
>>
>> On Thu, Dec 22, 2016 at 4:00 PM, Denis Magda  wrote:
>>
>>> I’ve finished the review and released all the parts of DML documentation
>>> [1].
>>>
>>> Prachi, could you review whole SQL Grid documentation [2] including [1]?
>>> Also it will be great if you check this newly added section [3].
>>>
>>> [1] http://apacheignite.gridgain.org/docs/dml
>>> [2] http://apacheignite.gridgain.org/docs/sql-grid
>>> [3] http://apacheignite.gridgain.org/docs/binary-
>>> marshaller#handling-hash-code-generation-and-equals-execution
>>>
>>> —
>>> Denis
>>>
>>> On Dec 9, 2016, at 12:34 AM, Alexander Paschenko <
>>> alexander.a.pasche...@gmail.com> wrote:
>>>
>>> Denis,
>>>
>>> About your question: 'putIfAbsent' is used only when we have single
>>> key-value pair to process as long as we don't have method like 'put
>>> all if absent'. When there's a bunch of pairs, an entry processor is
>>> created for each pair to make its processing atomic (effectively CAS
>>> style processing), and all those processors are then executed via
>>> 'invokeAll'.
>>>
>>> - Alex
>>>
>>> 2016-12-09 8:17 GMT+03:00 Denis Magda :
>>>
>>> Alexander, thanks for all the material you’ve prepared so far.
>>>
>>> Actually, I’ve reviewed, restructured and released only a part of it [1].
>>>
>>> The rest, that is missing, requires more time from my side and I’ll be
>>> releasing additional blocks throughout the next week. The full
>>> documentation
>>> with the unreviewed parts is invisible and located here [2].
>>>
>>> Please don’t modify neither of the docs ([1] and [2]) until I’ve finished
>>> with the overall review. Send all your suggestions over the email.
>>>
>>> At the moment I have the following question. What cache method(s) do we
>>> convert INSERT statements to [3]? Is it ‘putIfAbsent’?
>>>
>>> [1] https://apacheignite.readme.io/docs/dml
>>> [2] https://apacheignite.readme.io/docs/distributed-dml-to-be-reviewe
>>> [3] https://apacheignite.readme.io/docs/dml#section-insert
>>>
>>> —
>>> Denis
>>>
>>> On Dec 8, 2016, at 1:35 AM, Alexander Paschenko
>>>  wrote:
>>>
>>> Denis,
>>>
>>> it's all ready, JDBC driver page is also updated (I've created a
>>> hidden clone page, please find it on the left) as well as one on
>>> binary marshaller. Also I've renamed 'distributed DML' page to just
>>> 'DML' as long as there's much less to distribution of DML operations
>>> themselves than to queries and there's clearly no need to make another
>>> doc page to distinguish. Removed batches mention from limitations.
>>>
>>> - Alex
>>>
>>> 2016-12-08 5:26 GMT+03:00 Denis Magda :
>>>
>>> Alexander,
>>>
>>> How close are you to the finalization of the DML doc?
>>> https://apacheignite.readme.io/docs/distributed-dml
>>>
>>> Since we’re approaching 1.8 release I would like to do a final review of it
>>> polishing whatever is needed tomorrow.
>>>
>>> As I see that only the limitations section is left. The first note is that
>>> I
>>> wouldn’t mention “JDBC Batching mode” for now because this is something
>>> that
>>> doesn’t affect the usability and SQL API's scope of support.
>>>
>>> —
>>> Denis
>>>
>>>
>>>

[GitHub] ignite pull request #1381: Merge pull request #1 from apache/master

[zmyer]

GitHub user zmyer opened a pull request:

https://github.com/apache/ignite/pull/1381

Merge pull request #1 from apache/master

test

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zmyer/ignite master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1381.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1381


commit e5311a8c6aa05c46679b42a90d0470d72f220fc6
Author: zmyer 
Date:   2016-12-17T15:30:48Z

Merge pull request #1 from apache/master

test




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Document issue

[李玉珏]

I was translating chinese documents, but I didn't use AWS or Google 
compute, I just thought the document was a little weird.



在 2016/12/23 17:59, Nikolai Tikhonov 写道:

Denis,
Yes, you are right. The pages contains the latest information about cloud
images.

If you have problem with deployment, could you point on step which wrong?



On Thu, Dec 22, 2016 at 8:39 PM, Denis Magda  wrote:


Hi,

Do be honest I don’t see any issue there. Apache Ignite images for AWS and
GCE run inside of a docker container. So the latest docker related commands
look valid for me. Nick, please confirm.

—
Denis


On Dec 22, 2016, at 8:10 AM, 李玉珏@163 <18624049...@163.com> wrote:

Hi，

The following two pages of documents, more errors, I hope the community

can be a comprehensive review, and then fix it.

https://apacheignite.readme.io/docs/aws-deployment

https://apacheignite.readme.io/docs/google-compute-deployment


For example, under the two page, are all about docker information.

[jira] [Created] (IGNITE-4487) NPE on query execution

[Dmitry Karachentsev (JIRA)]

Dmitry Karachentsev created IGNITE-4487:
---

 Summary: NPE on query execution
 Key: IGNITE-4487
 URL: https://issues.apache.org/jira/browse/IGNITE-4487
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 1.8
Reporter: Dmitry Karachentsev
 Fix For: 2.0


NPE may be thrown when called destroyCache() and started querying.

Attached example reproduces this case when 
GridDiscoveryManager#removeCacheFilter called but cache state haven't been 
changed to STOPPED 
org.apache.ignite.internal.processors.cache.GridCacheGateway#onStopped.
{code:none}
javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException: null
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:740)
at 
com.intellica.evam.engine.event.future.FutureEventWorker.processFutureEvents(FutureEventWorker.java:117)
at 
com.intellica.evam.engine.event.future.FutureEventWorker.run(FutureEventWorker.java:66)
Caused by: class org.apache.ignite.IgniteCheckedException: null
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:1693)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:494)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:732)
... 2 more
Caused by: java.lang.NullPointerException
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter$ScanQueryFallbackClosableIterator.init(GridCacheQueryAdapter.java:712)
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter$ScanQueryFallbackClosableIterator.(GridCacheQueryAdapter.java:677)
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter$ScanQueryFallbackClosableIterator.(GridCacheQueryAdapter.java:628)
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter.executeScanQuery(GridCacheQueryAdapter.java:548)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxy$2.applyx(IgniteCacheProxy.java:497)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxy$2.applyx(IgniteCacheProxy.java:495)
at 
org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:1670)
... 4 more
{code}






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] ignite pull request #1377: IGNITE-4471: Fixed ODBC string-reading utility fu...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1377


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Sort nodes in the ring in order to minimize the number of reconnections

[Vyacheslav Daradur]

Thanks for reply.

I have some questions:

1. Where the logic of Ignite cluster building is realized? DiscoverySpi and
TcpDiscoveryMulticastIpFinder?

2. Which standart Ignite metrics you can recommend to use for node-ordering?

2016-12-22 19:08 GMT+03:00 Dmitriy Setrakyan :

> I think having some user-defined ordering can be beneficial. However, we
> are only talking about node discovery protocol here to maintain the
> cluster. All other communication between nodes happens directly (does not
> go through the ring).
>
> D.
>
> On Thu, Dec 22, 2016 at 6:32 AM, Vyacheslav Daradur 
> wrote:
>
> > Hello, Alex!
> >
> > I think it is a great idea.
> >
> > I suggest to build communications between nodes on weight (or priority).
> >
> > For example, ordering on latency:
> > - nodes on one host = 1
> > - nodes in one rack-blade = 2
> > - nodes in one server-rack = 3
> > - nodes in one physical cluster = 4
> > - nodes in one subnet = 5
> > - etc.
> >
> > Maybe it'll be better to use some metrics from ClusterMetrics interface.
> >
> > The algorithm of ordering can be implemented in a class such as
> Comparator
> > and use it when we build a cluster or we select a place for a new node.
> >
> > --
> > With best regards,
> > Vyacheslav Daradur
> >
> > 2016-12-22 13:59 GMT+03:00 Александр Меньшиков :
> >
> > > Hello everyone,
> > >
> > > As far as I know nodes are connected in a ring. For example if i have 6
> > > nodes, with names A, B, C, D, E, and F they can connect in ring any
> > > possible way: A-B-C-D-E-F-A, or A-F-B-E-C-D-A, etc. And if some node
> > falls
> > > out of topology neighboring nodes must reconnect. If nodes A,B and C
> > > located in the same physical location, and D, E and F in another, and
> in
> > > some time one physical location is not available in another, we can get
> > > different number of reconnections. Best case scenario if we have ring
> > like
> > > A-B-CxD-E-FxA ('x' mean disconnect) -- then we get only one reconnect
> (C
> > > reconnect to A or F reconnect to D -- depending on what part of the
> > cluster
> > > we leave alive). But now possible that case AxFxBxExCxDxA -- then we
> get
> > a
> > > lot of reconnections (A to B, B to C, C to A -- in general n/2
> > > reconnections, where n -- number of nodes). And i think to add
> something
> > to
> > > ensure that we always have good sorting of nodes connections
> > > (A-B-C-...-Z-A).
> > >
> > > Of course in real world we can have multiple levels of physical
> > closeness.
> > >
> > > In my opinion enough to add one parameter of 'int' to configuration
> (with
> > > name like 'ExtraNodeOrder') and to change the method of comparison
> nodes
> > so
> > > that it first compared the 'ExtraNodeOrder', and then according to the
> > old
> > > criterion (as far as I know Ignite use topology version). So if some
> > users
> > > have multiple levels of physical closeness, he can use different bits.
> > For
> > > example use 16 high bits for DC number, and low 16 bits for racks.
> > >
> > > Alternatively, we can add array of ‘int’ to configuration and compare
> > nodes
> > > in sequence from the zero element to the last.
> > >
> >
>

Re: Document issue

[Nikolai Tikhonov]

Denis,
Yes, you are right. The pages contains the latest information about cloud
images.

If you have problem with deployment, could you point on step which wrong?



On Thu, Dec 22, 2016 at 8:39 PM, Denis Magda  wrote:

> Hi,
>
> Do be honest I don’t see any issue there. Apache Ignite images for AWS and
> GCE run inside of a docker container. So the latest docker related commands
> look valid for me. Nick, please confirm.
>
> —
> Denis
>
> > On Dec 22, 2016, at 8:10 AM, 李玉珏@163 <18624049...@163.com> wrote:
> >
> > Hi，
> >
> > The following two pages of documents, more errors, I hope the community
> can be a comprehensive review, and then fix it.
> >
> > https://apacheignite.readme.io/docs/aws-deployment
> >
> > https://apacheignite.readme.io/docs/google-compute-deployment
> >
> >
> > For example, under the two page, are all about docker information.
> >
> >
>
>

[GitHub] ignite pull request #1380: IGNITE-4486 Added serialVersionUID to AttributeNo...

[alexpaschenko]

GitHub user alexpaschenko opened a pull request:

https://github.com/apache/ignite/pull/1380

IGNITE-4486 Added serialVersionUID to AttributeNodeFilter



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4486

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1380.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1380


commit d0b5845b9246301c4ab2cc9e83323b0144d59a7d
Author: Alexander Paschenko 
Date:   2016-12-23T09:29:11Z

IGNITE-4486 Added serialVersionUID to AttributeNodeFilter




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-4486) Add serialVersionUID to AttributeNodeFilter

[Alexander Paschenko (JIRA)]

Alexander Paschenko created IGNITE-4486:
---

 Summary: Add serialVersionUID to AttributeNodeFilter
 Key: IGNITE-4486
 URL: https://issues.apache.org/jira/browse/IGNITE-4486
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 1.8
Reporter: Alexander Paschenko
Assignee: Alexander Paschenko
 Fix For: 1.8


Subj. - in particular, without it, TC build fails all at once



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)