IgniteCacheTestSuite3 on TeamCity

[Semyon Boikov]

Hi guys,
I need run IgniteCacheTestSuite3 on TeamCity, but can not find related
configuration. 'Cache 3' configuration runs only
IgniteBinaryObjectsCacheTestSuite3, could you please take a look?

Thank you

Re: DiscoverySpi based on Apache ZooKeeper

[Semyon Boikov]

Andrey, Vladimir,

Zookeeper does provide required ordering guarantees, but Zookeeper watcher
API really does not provide functionality required for DiscoverySpi out of
the box. To detect nodes join/failure we need to watch for
creation/deletion of znodes, but in case if some client disconnects from
Zookeeper and reconnects after some time, then it can miss some change
events and after reconnect it will get only current state. For example:
there are 2 znodes (related to Ignite cluster nodes): A, B. One of
zookeeper clients disconnects from Zookeeper and tries to reconnect, at
this moment new cluster node joins and creates znode C, then it immediately
fails and znode C is removed. When disconnected client restores connection
it still sees znodes A and B and is not aware about node C. This means that
different clients can see different Zookeeper events, to overcome this
issue ZookeeperDiscoverySpi has single coordinator node which listens for
Zookeeper notifications and transforms then into DiscoverySpi events
(scenario when znode C is created and removed while coordinator is
disconnected is still possible, but it is not an issue, this means cluster
node C failed before it finished join). With such approach with single
coordinator I don't see scenario when different nodes can see different
events or some event can be missed.

Semyon

On Wed, Jan 10, 2018 at 10:38 AM, Vladimir Ozerov <voze...@gridgain.com>
wrote:

> Hi Andrey,
>
> Could you please share detail of this API mismatch? AFAIK, the main
> guarantee we need for disco SPI is total message ordering. Zookeeper
> provides this guarantee. Moreover, Zookeeper is proven to be correct and
> reliable coordinator service by many users and Jepsen tests, as opposed to
> various in-house implementations (e.g. Zen of Elasticsearch).
>
> вт, 9 янв. 2018 г. в 21:53, Andrey Kornev <andrewkor...@hotmail.com>:
>
>> Semyon,
>>
>> Not to discourage you or anything, just a interesting fact from recent
>> history.
>>
>> I vaguely remember already trying to implement DiscoverySpi on top of
>> Zookeeper back in 2012. After a few failed attempts and a lot of help from
>> Zookeeper's original developers (Flavio Junqueira and Ben Reed) we (Dmitriy
>> S. and I) concluded that its not possible to  implement DiscoverySpi on top
>> of Zookeeper due to strict(er) semantics of DiscoverySpi. Unfortunately I
>> do not remember details, but essentially, in some cases it was not possible
>> to establish total ordering of watcher events and under certain
>> circumstances loss of such events was possible.
>>
>> It's not to say that Zookeeper can't be used to implement the cluster
>> membership tracking in general. The problem is rather with DiscoverySpi
>> semantics that require a different set of APIs than what Zookeeper provides.
>>
>> Regards
>> Andrey
>>
>> 
>> From: Semyon Boikov <sboi...@apache.org>
>> Sent: Tuesday, January 9, 2018 3:39 AM
>> To: dev@ignite.apache.org
>> Subject: DiscoverySpi based on Apache ZooKeeper
>>
>> Hi all,
>>
>> Currently I'm working on implementation of DiscoverySpi based on Apache
>> ZooKeeper (ZookeeperDiscoverySpi) and want to share results of this work.
>>
>> In very large clusters (>1000 nodes) current default implementation of
>> DiscoverySpi - TcpDiscoverySpi - has some significant drawbacks:
>> - TcpDiscoverySpi organizes nodes in ring, and all messages are passed
>> sequentially via ring. More nodes there are in ring, more time it takes to
>> pass message. In very large clusters such architecture can cause slowdown
>> of important operations (node join, node stop, cache start, etc).
>> - in TcpDiscoverySpi's protocol each node in ring is able to fail next one
>> in case of network issues, and it is possible that two nodes can 'kill'
>> each other (it is possible in complex scenarios when network is broken and
>> then restored back after some time, such problems were observed in real
>> environments), and with current TcpDiscoverySpi protocol there is no easy
>> way to completely fix such problems.
>> - when some node in ring fails, then previous node tries to restore ring
>> and sequentially tries to connect to next nodes. If large part of ring
>> fails then it takes long time to sequentially detect failure of all nodes.
>> - with TcpDiscoverySpi split brain is possible (one ring can split into
>> two
>> independent parts), separate mechanism is needed to protect from split
>> brain when TcpDiscoverySpi is used
>>
>> Even though most probably some of these problems can be somehow fixed in
>> TcpDiscoverySpi, it seems more robust and fast Disc

Re: DiscoverySpi based on Apache ZooKeeper

[Semyon Boikov]

Dmitriy, no problem, I'll rename it.

Semyon

On Tue, Jan 9, 2018 at 11:20 PM, Dmitriy Setrakyan <dsetrak...@apache.org>
wrote:

> Thanks Semyon!
>
> I have a naming nitpick. Can we rename Problem to Failure, e.g.
> CommunicationProblemResolver to CommunicationFailureResolver?
>
> D.
>
> On Tue, Jan 9, 2018 at 3:39 AM, Semyon Boikov <sboi...@apache.org> wrote:
>
> > Hi all,
> >
> > Currently I'm working on implementation of DiscoverySpi based on Apache
> > ZooKeeper (ZookeeperDiscoverySpi) and want to share results of this work.
> >
> > In very large clusters (>1000 nodes) current default implementation of
> > DiscoverySpi - TcpDiscoverySpi - has some significant drawbacks:
> > - TcpDiscoverySpi organizes nodes in ring, and all messages are passed
> > sequentially via ring. More nodes there are in ring, more time it takes
> to
> > pass message. In very large clusters such architecture can cause slowdown
> > of important operations (node join, node stop, cache start, etc).
> > - in TcpDiscoverySpi's protocol each node in ring is able to fail next
> one
> > in case of network issues, and it is possible that two nodes can 'kill'
> > each other (it is possible in complex scenarios when network is broken
> and
> > then restored back after some time, such problems were observed in real
> > environments), and with current TcpDiscoverySpi protocol there is no easy
> > way to completely fix such problems.
> > - when some node in ring fails, then previous node tries to restore ring
> > and sequentially tries to connect to next nodes. If large part of ring
> > fails then it takes long time to sequentially detect failure of all
> nodes.
> > - with TcpDiscoverySpi split brain is possible (one ring can split into
> two
> > independent parts), separate mechanism is needed to protect from split
> > brain when TcpDiscoverySpi is used
> >
> > Even though most probably some of these problems can be somehow fixed in
> > TcpDiscoverySpi, it seems more robust and fast DiscoverySpi can be
> > implemented on top of some existing coordination service.
> >
> > Apache ZooKeeper is known reliable service and it provides all mechanisms
> > and consistency guarantees required for Ignite's DiscoverySpi. Some
> > technical details of ZookeeperDiscoverySpi implementation can be found in
> > description prepared by Sergey Puchnin:
> > https://cwiki.apache.org/confluence/display/IGNITE/
> > Discovery+SPI+by+ZooKeeper
> > .
> >
> > In our preliminary tests we were able to successfully start 4000+ nodes
> > with ZookeeperDiscoverySpi. New implementation works faster than
> > TcpDiscoverySpi and does not have mentioned TcpDiscoverySpi's drawbacks:
> > - nodes alive status is controlled by ZooKeeper, so nodes never can kill
> > each other
> > - ZooKeeper has protection from split brain
> > - with ZooKeeper it is possible to detect nodes join/failures in batches
> so
> > time to detect join/failure of 1 vs 100+ nodes is almost the same
> >
> > I'm going to finalize implementation of ZookeeperDiscoverySpi in next few
> > days. But in Ignite there is one more issue caused by the fact that
> > DiscoverySpi and CommunicationSpi are two independent component: it is
> > possible that DiscoverySpi considers some node as alive, but at the same
> > time CommunicationSpi is not able to send message to this node (this
> issue
> > exists not only for ZookeeperDiscoverySpi for but for TcpDiscoverySpi
> too).
> > Such case is very critical since all internal Ignite's code assumes that
> if
> > node is alive then CommunicationSpi is able to send/receive messages
> > to/from this node. If it is not the case, then any Ignite operation can
> > hang (with ZooKeeper such situation is possible when due to network
> > failures nodes have connection with ZooKeeper, but can not establish TCP
> > connection to each other).
> >
> > If such case arises, then Ignite should have some mechanism to forcibly
> > kill faulty nodes to keep cluster operational. But note that in case of
> > 'split brain' scenarios each independent part of cluster will consider
> > others as 'failed' and there should be some way to choose which part
> should
> > be killed. It would be good to provide generic solution for this problem
> as
> > part of work on new DiscoverySpi.
> >
> > We discussed this with Yakov Zhdanov and suggest following: in case when
> > communication fails to send message to some node and this node is
> > considered as alive, then Ignite should trigger global 'communication
> error
>

DiscoverySpi based on Apache ZooKeeper

[Semyon Boikov]

Hi all,

Currently I'm working on implementation of DiscoverySpi based on Apache
ZooKeeper (ZookeeperDiscoverySpi) and want to share results of this work.

In very large clusters (>1000 nodes) current default implementation of
DiscoverySpi - TcpDiscoverySpi - has some significant drawbacks:
- TcpDiscoverySpi organizes nodes in ring, and all messages are passed
sequentially via ring. More nodes there are in ring, more time it takes to
pass message. In very large clusters such architecture can cause slowdown
of important operations (node join, node stop, cache start, etc).
- in TcpDiscoverySpi's protocol each node in ring is able to fail next one
in case of network issues, and it is possible that two nodes can 'kill'
each other (it is possible in complex scenarios when network is broken and
then restored back after some time, such problems were observed in real
environments), and with current TcpDiscoverySpi protocol there is no easy
way to completely fix such problems.
- when some node in ring fails, then previous node tries to restore ring
and sequentially tries to connect to next nodes. If large part of ring
fails then it takes long time to sequentially detect failure of all nodes.
- with TcpDiscoverySpi split brain is possible (one ring can split into two
independent parts), separate mechanism is needed to protect from split
brain when TcpDiscoverySpi is used

Even though most probably some of these problems can be somehow fixed in
TcpDiscoverySpi, it seems more robust and fast DiscoverySpi can be
implemented on top of some existing coordination service.

Apache ZooKeeper is known reliable service and it provides all mechanisms
and consistency guarantees required for Ignite's DiscoverySpi. Some
technical details of ZookeeperDiscoverySpi implementation can be found in
description prepared by Sergey Puchnin:
https://cwiki.apache.org/confluence/display/IGNITE/Discovery+SPI+by+ZooKeeper
.

In our preliminary tests we were able to successfully start 4000+ nodes
with ZookeeperDiscoverySpi. New implementation works faster than
TcpDiscoverySpi and does not have mentioned TcpDiscoverySpi's drawbacks:
- nodes alive status is controlled by ZooKeeper, so nodes never can kill
each other
- ZooKeeper has protection from split brain
- with ZooKeeper it is possible to detect nodes join/failures in batches so
time to detect join/failure of 1 vs 100+ nodes is almost the same

I'm going to finalize implementation of ZookeeperDiscoverySpi in next few
days. But in Ignite there is one more issue caused by the fact that
DiscoverySpi and CommunicationSpi are two independent component: it is
possible that DiscoverySpi considers some node as alive, but at the same
time CommunicationSpi is not able to send message to this node (this issue
exists not only for ZookeeperDiscoverySpi for but for TcpDiscoverySpi too).
Such case is very critical since all internal Ignite's code assumes that if
node is alive then CommunicationSpi is able to send/receive messages
to/from this node. If it is not the case, then any Ignite operation can
hang (with ZooKeeper such situation is possible when due to network
failures nodes have connection with ZooKeeper, but can not establish TCP
connection to each other).

If such case arises, then Ignite should have some mechanism to forcibly
kill faulty nodes to keep cluster operational. But note that in case of
'split brain' scenarios each independent part of cluster will consider
others as 'failed' and there should be some way to choose which part should
be killed. It would be good to provide generic solution for this problem as
part of work on new DiscoverySpi.

We discussed this with Yakov Zhdanov and suggest following: in case when
communication fails to send message to some node and this node is
considered as alive, then Ignite should trigger global 'communication error
resolve' process (obviously, this process should use for messaging internal
discovery mechanisms). As part of this process CommunicationSpi on each
node should try to establish connection to all others alive nodes
(TcpCommunicationSpi can do this efficiently using async NIO) and send
results of this connection test to some coordinator node (e.g. oldest
cluster node). When coordinator receives results of connection test from
all nodes it calls user-defined CommunicationProblemResolver to choose
which nodes should be killed (CommunicationProblemResolver should be set in
IgniteConfiguration):

public interface CommunicationProblemResolver {
public void resolve(CommunicationProblemContext ctx);
}

CommunicationProblemResolver  receives CommunicationProblemContext which
provides results of CommunicationSpi connection test. Also it can be useful
to have information about started caches and current cache data
distribution to decide which part of cluster should be killed:

public interface CommunicationProblemContext {
/**
 * @return Current topology snapshot.
 */
public List topologySnapshot();

/**
 * @param node1 

Re: Logical Cache Documented

[Semyon Boikov]

Hi,

Regarding question about  default cache group: by default cache groups are
not enabled, each cache is started in separate group. Cache group is
enabled only if groupName is set in CacheConfiguration.

Thanks

On Sat, Sep 30, 2017 at 11:55 PM,  wrote:

> Why not? Obviously compression would have to be enabled per group, not per
> cache.
>
> ⁣D.​
>
> On Sep 29, 2017, 10:50 PM, at 10:50 PM, Vladimir Ozerov <
> voze...@gridgain.com> wrote:
> >And it will continue hitting us in future. For example, when data
> >compression is implemented, for logical caches compression rate will be
> >poor, as it would be impossbile to build efficient dictionaries in
> >mixed
> >data pages.
> >
> >On Sat, Sep 30, 2017 at 8:48 AM, Vladimir Ozerov 
> >wrote:
> >
> >> Folks,
> >>
> >> Honesly, to me logical caches appears to be a dirty shortcut to
> >mitigate
> >> some inefficient internal implementation. Why can't we merge
> >partition maps
> >> in runtime? This should not be a problem for context-independent
> >affinity
> >> functions (e.g. RendezvousAffinityFunction). From user perspective
> >logic
> >> caches feature is:
> >> 1) Bad API. One cannot define group configuration. All you can do is
> >to
> >> define group name on cache lavel and hope that nobody started another
> >cache
> >> in the same group with different configuration before.
> >> 2) Performance impact for scans, as you have to iterate over mixed
> >data.
> >>
> >> Couldn't we fix partition map problem without cache groups?
> >>
> >> On Sat, Sep 30, 2017 at 2:35 AM, Denis Magda 
> >wrote:
> >>
> >>> Guys,
> >>>
> >>> Another question. Does this capability enabled by default? If yes,
> >how do
> >>> we decide which group a cache goes to?
> >>>
> >>> —
> >>> Denis
> >>>
> >>> > On Sep 29, 2017, at 3:58 PM, Denis Magda 
> >wrote:
> >>> >
> >>> > Igniters,
> >>> >
> >>> > I’ve put on paper the feature from the subj:
> >>> > https://apacheignite.readme.io/docs/logical-caches <
> >>> https://apacheignite.readme.io/docs/logical-caches>
> >>> >
> >>> > Sam, will appreciate if you read through it and confirm I
> >explained the
> >>> topic 100% technically correct.
> >>> >
> >>> > However, are there any negative impacts of having logical caches?
> >This
> >>> page has “Possible Impacts” section unfilled:
> >>> > https://cwiki.apache.org/confluence/display/IGNITE/Logical+Caches
> ><
> >>> https://cwiki.apache.org/confluence/display/IGNITE/Logical+Caches>
> >>> >
> >>> > —
> >>> > Denis
> >>>
> >>>
> >>
>

Re: MVCC configuration

[Semyon Boikov]

Yes, we'll add this validation.

On Wed, Sep 20, 2017 at 10:09 AM, Dmitriy Setrakyan <dsetrak...@apache.org>
wrote:

> On Tue, Sep 19, 2017 at 11:31 PM, Semyon Boikov <sboi...@gridgain.com>
> wrote:
>
> > > Can caches within the same group have different MVCC configuration?
> >
> > Do you think we really need have in the same group caches with different
> > mvcc configuration? for simplicity I would do not allow this.
> >
>
> I agree, let's not allow it. In that case, are you going to have extra
> validation on startup that caches in the same group must have identical
> MVCC config?
>

Re: MVCC configuration

[Semyon Boikov]

> Can caches within the same group have different MVCC configuration?

I think it is possible to implement, but there are some issues:
- for mvcc we need store mvcc version in hash index item (for now it is 16
bytes), since index items have fixed size then if we store in this index
data for caches with disabled mvcc, then it will have unnecessary 16 bytes
overhead
- for mvcc caches we need create correct hash index in advance, so if group
was created with mvcc disabled, then later it is not possible to add in
this group mvcc enabled cache

Do you think we really need have in the same group caches with different
mvcc configuration? for simplicity I would do not allow this.

Thanks

On Wed, Sep 20, 2017 at 7:30 AM, Dmitriy Setrakyan <dsetrak...@apache.org>
wrote:

> Can caches within the same group have different MVCC configuration?
>
> On Tue, Sep 19, 2017 at 2:34 AM, Vladimir Ozerov <voze...@gridgain.com>
> wrote:
>
> > What I mean is that it might be not applicable for DML by design. E.g.
> may
> > be we will have to fallback to per-memory-policy approach, or to
> > per-cache-group. As we do not know it at the moment and there is no clear
> > demand from users, I would simply put it aside to avoid in mess in public
> > API in future.
> >
> > Moreover, per-cache flag raises additional questions which can be put out
> > of scope otherwise. E.g. is it legal to mix MVCC and non-MVCC caches in a
> > single transaction? If yes, what is the contract? Without fine-grained
> > per-cache approach in the first iteration we can avoid answering it.
> >
> > On Tue, Sep 19, 2017 at 12:25 PM, Semyon Boikov <sboi...@gridgain.com>
> > wrote:
> >
> > > If it is not valid for DML then we can easily detect this situation and
> > > throw exception, but if I do not use DML why non make it configurable
> > > per-cache?
> > >
> > > On Tue, Sep 19, 2017 at 12:22 PM, Vladimir Ozerov <
> voze...@gridgain.com>
> > > wrote:
> > >
> > > > I would say that per-cache configuration should be out of scope as
> well
> > > for
> > > > the first iteration. Because we do not know whether it will be valid
> > for
> > > > DML.
> > > >
> > > > On Tue, Sep 19, 2017 at 12:15 PM, Semyon Boikov <
> sboi...@gridgain.com>
> > > > wrote:
> > > >
> > > > > Folks, thank you for feedback, I want to summarize some decisions:
> > > > >
> > > > > 1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
> > > > > per-cache flag - CacheConfiguration.isMvccEnabled, default value
> for
> > > all
> > > > > caches - IgniteConfiguration.isMvccEnabled.
> > > > > 2. For initial implementation mvcc for ATOMIC cache is out of
> scope,
> > it
> > > > can
> > > > > be enabled only for TRANSACTIONAL caches.
> > > > > 3. Mvcc coordinator can be any server node (oldest server node is
> > > > selected
> > > > > automatically). Also I believe we need possibility to have
> > *dedicated*
> > > > mvcc
> > > > > coordinator nodes which will process only internal mvcc messages.
> > Node
> > > > can
> > > > > be marked as dedicated coordinator via new flag
> > > > > IgniteConfiguration.isMvccCoordinator or we can add separate
> > > > > MvccConfiguration bean. But let's skip this decision for now before
> > we
> > > > have
> > > > > benchmarks numbers.
> > > > > 4. Need add some metrics to monitor mvcc coordinator performance.
> > > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > > On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov <
> > > voze...@gridgain.com>
> > > > > wrote:
> > > > >
> > > > > > This could be something like "preferredMvccCoordinator".
> > > > > >
> > > > > > On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> > > > > > alexey.goncha...@gmail.com> wrote:
> > > > > >
> > > > > > > >
> > > > > > > > I agree that we need coordinator nodes, but I do not
> understand
> > > why
> > > > > > can't
> > > > > > > > we reuse some cache nodes for it? Why do we need to ask user
> to
> > > > start
> > > > > > up
> > > > > > > > yet another type of node?
> > > > > > > >
> > > > > > >
> > > > > > > Dmitriy,
> > > > > > >
> > > > > > > My understanding is that Semyon does not deny a cache node to
> be
> > > used
> > > > > as
> > > > > > a
> > > > > > > coordinator. This property will allow to optionally have a
> > > > *dedicated*
> > > > > > node
> > > > > > > serving as a coordinator to improve cluster throughput under
> > heavy
> > > > > load.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: MVCC configuration

[Semyon Boikov]

If it is not valid for DML then we can easily detect this situation and
throw exception, but if I do not use DML why non make it configurable
per-cache?

On Tue, Sep 19, 2017 at 12:22 PM, Vladimir Ozerov <voze...@gridgain.com>
wrote:

> I would say that per-cache configuration should be out of scope as well for
> the first iteration. Because we do not know whether it will be valid for
> DML.
>
> On Tue, Sep 19, 2017 at 12:15 PM, Semyon Boikov <sboi...@gridgain.com>
> wrote:
>
> > Folks, thank you for feedback, I want to summarize some decisions:
> >
> > 1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
> > per-cache flag - CacheConfiguration.isMvccEnabled, default value for all
> > caches - IgniteConfiguration.isMvccEnabled.
> > 2. For initial implementation mvcc for ATOMIC cache is out of scope, it
> can
> > be enabled only for TRANSACTIONAL caches.
> > 3. Mvcc coordinator can be any server node (oldest server node is
> selected
> > automatically). Also I believe we need possibility to have *dedicated*
> mvcc
> > coordinator nodes which will process only internal mvcc messages. Node
> can
> > be marked as dedicated coordinator via new flag
> > IgniteConfiguration.isMvccCoordinator or we can add separate
> > MvccConfiguration bean. But let's skip this decision for now before we
> have
> > benchmarks numbers.
> > 4. Need add some metrics to monitor mvcc coordinator performance.
> >
> >
> > Thanks
> >
> > On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov <voze...@gridgain.com>
> > wrote:
> >
> > > This could be something like "preferredMvccCoordinator".
> > >
> > > On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> > > alexey.goncha...@gmail.com> wrote:
> > >
> > > > >
> > > > > I agree that we need coordinator nodes, but I do not understand why
> > > can't
> > > > > we reuse some cache nodes for it? Why do we need to ask user to
> start
> > > up
> > > > > yet another type of node?
> > > > >
> > > >
> > > > Dmitriy,
> > > >
> > > > My understanding is that Semyon does not deny a cache node to be used
> > as
> > > a
> > > > coordinator. This property will allow to optionally have a
> *dedicated*
> > > node
> > > > serving as a coordinator to improve cluster throughput under heavy
> > load.
> > > >
> > >
> >
>

Re: MVCC configuration

[Semyon Boikov]

Folks, thank you for feedback, I want to summarize some decisions:

1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
per-cache flag - CacheConfiguration.isMvccEnabled, default value for all
caches - IgniteConfiguration.isMvccEnabled.
2. For initial implementation mvcc for ATOMIC cache is out of scope, it can
be enabled only for TRANSACTIONAL caches.
3. Mvcc coordinator can be any server node (oldest server node is selected
automatically). Also I believe we need possibility to have *dedicated* mvcc
coordinator nodes which will process only internal mvcc messages. Node can
be marked as dedicated coordinator via new flag
IgniteConfiguration.isMvccCoordinator or we can add separate
MvccConfiguration bean. But let's skip this decision for now before we have
benchmarks numbers.
4. Need add some metrics to monitor mvcc coordinator performance.


Thanks

On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov 
wrote:

> This could be something like "preferredMvccCoordinator".
>
> On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> alexey.goncha...@gmail.com> wrote:
>
> > >
> > > I agree that we need coordinator nodes, but I do not understand why
> can't
> > > we reuse some cache nodes for it? Why do we need to ask user to start
> up
> > > yet another type of node?
> > >
> >
> > Dmitriy,
> >
> > My understanding is that Semyon does not deny a cache node to be used as
> a
> > coordinator. This property will allow to optionally have a *dedicated*
> node
> > serving as a coordinator to improve cluster throughput under heavy load.
> >
>

Re: MVCC configuration

[Semyon Boikov]

Vladimir, thanks for comments

> 2) I would also avoid this flag until we clearly understand it is needed.
> All numbers will be assigned from a single thread. For this reason even
> peak load on coordinator should not consume too much resources. I think we
> can assign coordinators automatically in first iteration.

For me need of dedicated coordinator nodes is clear: each mvcc
transaction/query will wait for mvcc coordinator response, if coordinator
will also process cache operations/compute jobs then any user code executed
on coordinator and consuming lot of CPU/heap will slowdown ALL mvcc
transactions/queries. As a user I want to make sure that coordinator node
will process only internal requests related to mvcc.

Also why do you think that all numbers should be assigned from single
thread?

Thanks

On Mon, Sep 18, 2017 at 2:59 PM, Semyon Boikov <sboi...@gridgain.com> wrote:

> Nikolay, thanks for comments
>
>
> > How will Ignite handle "mvcc coordinator" fail?
> > What will happen with if coordinator fails in the middle of a
> transaction?
> > Could tx be committed or rollbacked?
>
> I think coordinator failure will be handled in the same way as failure of
> one of transaction's 'primary' node: if coordinator fails during 'prepare'
> phase then tx is rolledback.
>
> >> Will we have some user notification if coordinator becomes slower?
>
> Now in Ignite we do not have common notion of 'user notification's, but we
> can add some metrics for coordinator performance on public API.
>
> Thanks
>
>
> On Mon, Sep 18, 2017 at 1:01 PM, Николай Ижиков <nizhikov@gmail.com>
> wrote:
>
>> Hello, Semyon!
>>
>> > It seems we need introduce special 'dedicated mvcc coordinator' node
>> role
>>
>> How will Ignite handle "mvcc coordinator" fail?
>>
>> What will happen with if coordinator fails in the middle of a transaction?
>> Could tx be committed or rollbacked?
>>
>> Will we have some user notification if coordinator becomes slower?
>>
>> > IgniteConfiguration.isMvccCoordinator
>>
>> flag name seems OK.
>>
>>
>> 2017-09-18 12:39 GMT+03:00 Semyon Boikov <sboi...@apache.org>:
>>
>> > Hi all,
>> >
>> > Currently I'm working on MVCC feature (IGNITE-3478) and need your
>> opinion
>> > on related configuration options.
>> >
>> > 1. MVCC will definitely bring some performance overhead, so I think it
>> > should not be enabled by default, I'm going to add special flag on cache
>> > configuration: CacheConfiguration.isMvccEnabled.
>> >
>> > 2. In current mvcc architecture there should be some node in cluster
>> > assigning versions for tx updates and queries (mvcc coordinator). Mvcc
>> > coordinator is crucial component and it should perform as fast as
>> possible.
>> > It seems we need introduce special 'dedicated mvcc coordinator' node
>> role:
>> > it should not be possible to start cache on such node and it should not
>> > process user's compute jobs. At the same time it should be possible that
>> > any regular server node can become mvcc coordinator: this can be useful
>> > during development (no extra setup for mvcc will be needed), or support
>> > scenario when all dedicated coordinator nodes fail. So we need a way to
>> > make node a 'dedicated mvcc coordinator', we can add special flag on
>> ignite
>> > configuration: IgniteConfiguration.isMvccCoordinator.
>> >
>> > What do you think?
>> >
>> > Thanks
>> >
>>
>>
>>
>> --
>> Nikolay Izhikov
>> nizhikov@gmail.com
>>
>
>

Re: MVCC configuration

[Semyon Boikov]

Nikolay, thanks for comments


> How will Ignite handle "mvcc coordinator" fail?
> What will happen with if coordinator fails in the middle of a transaction?
> Could tx be committed or rollbacked?

I think coordinator failure will be handled in the same way as failure of
one of transaction's 'primary' node: if coordinator fails during 'prepare'
phase then tx is rolledback.

>> Will we have some user notification if coordinator becomes slower?

Now in Ignite we do not have common notion of 'user notification's, but we
can add some metrics for coordinator performance on public API.

Thanks


On Mon, Sep 18, 2017 at 1:01 PM, Николай Ижиков <nizhikov@gmail.com>
wrote:

> Hello, Semyon!
>
> > It seems we need introduce special 'dedicated mvcc coordinator' node role
>
> How will Ignite handle "mvcc coordinator" fail?
>
> What will happen with if coordinator fails in the middle of a transaction?
> Could tx be committed or rollbacked?
>
> Will we have some user notification if coordinator becomes slower?
>
> > IgniteConfiguration.isMvccCoordinator
>
> flag name seems OK.
>
>
> 2017-09-18 12:39 GMT+03:00 Semyon Boikov <sboi...@apache.org>:
>
> > Hi all,
> >
> > Currently I'm working on MVCC feature (IGNITE-3478) and need your opinion
> > on related configuration options.
> >
> > 1. MVCC will definitely bring some performance overhead, so I think it
> > should not be enabled by default, I'm going to add special flag on cache
> > configuration: CacheConfiguration.isMvccEnabled.
> >
> > 2. In current mvcc architecture there should be some node in cluster
> > assigning versions for tx updates and queries (mvcc coordinator). Mvcc
> > coordinator is crucial component and it should perform as fast as
> possible.
> > It seems we need introduce special 'dedicated mvcc coordinator' node
> role:
> > it should not be possible to start cache on such node and it should not
> > process user's compute jobs. At the same time it should be possible that
> > any regular server node can become mvcc coordinator: this can be useful
> > during development (no extra setup for mvcc will be needed), or support
> > scenario when all dedicated coordinator nodes fail. So we need a way to
> > make node a 'dedicated mvcc coordinator', we can add special flag on
> ignite
> > configuration: IgniteConfiguration.isMvccCoordinator.
> >
> > What do you think?
> >
> > Thanks
> >
>
>
>
> --
> Nikolay Izhikov
> nizhikov@gmail.com
>

Re: MVCC configuration

[Semyon Boikov]

Guys,

I do not really understand mvcc for atomic cache, could you please provide
some real use case.

Thank you

On Mon, Sep 18, 2017 at 1:37 PM, Yakov Zhdanov  wrote:

> Ouch... of course it makes sense for atomic caches. Seems I am not fully
> switched on after weekend =)
>
> Agree on other points.
>
> --Yakov
>

MVCC configuration

[Semyon Boikov]

Hi all,

Currently I'm working on MVCC feature (IGNITE-3478) and need your opinion
on related configuration options.

1. MVCC will definitely bring some performance overhead, so I think it
should not be enabled by default, I'm going to add special flag on cache
configuration: CacheConfiguration.isMvccEnabled.

2. In current mvcc architecture there should be some node in cluster
assigning versions for tx updates and queries (mvcc coordinator). Mvcc
coordinator is crucial component and it should perform as fast as possible.
It seems we need introduce special 'dedicated mvcc coordinator' node role:
it should not be possible to start cache on such node and it should not
process user's compute jobs. At the same time it should be possible that
any regular server node can become mvcc coordinator: this can be useful
during development (no extra setup for mvcc will be needed), or support
scenario when all dedicated coordinator nodes fail. So we need a way to
make node a 'dedicated mvcc coordinator', we can add special flag on ignite
configuration: IgniteConfiguration.isMvccCoordinator.

What do you think?

Thanks

Re: ContinuousQueryWithTransformer implementation questions - 2

[Semyon Boikov]

Hi,

I had an impression that current behavior is required by jcache, but now I
can not find anything about this neither in spec nor in jcache tck tests.
So I think we can change current behavior.

Thanks

On Tue, Aug 29, 2017 at 9:48 PM, Nikolay Izhikov 
wrote:

> Yakov.
>
> I found following description in jcache javadoc [1] -
>
> ===
> Returns:
>true if the evaluation passes, otherwise false.
>*The effect of returning true is that listener will be invoked*
> ===
>
> JSR doesn't specify how filter exception has to be handled.
> As far as I can understand *only* way to pass filter is return true from
> `evaluate`.
>
> I think we has to change current behavior.
>
> Should I file a ticket?
>
> [1] https://static.javadoc.io/javax.cache/cache-api/1.0.0/javax/
> cache/event/CacheEntryEventFilter.html#evaluate(javax.cache.
> event.CacheEntryEvent)
>
>
> 29.08.2017 17:09, Yakov Zhdanov пишет:
>
> Igniters,
>>
>> Does anyone else see potential issues on user side with current approach?
>>
>> Sam, is this JCache requirement?
>>
>> --Yakov
>>
>> 2017-08-29 15:16 GMT+03:00 Nikolay Izhikov :
>>
>> Yakov.
>>>
>>> I think exception equals `true` is intended behavior.
>>>
>>> Filter evaluation implementation from master - [1]
>>> Test from master to check filter exception(without explicit asserts
>>> checking listeners call) - [2]
>>>
>>> Here is my quick test with asserts on listener call after filter
>>> exception:
>>>
>>> ```
>>> package org.apache.ignite.internal.processors.cache.query.continuous;
>>>
>>> //... imports
>>>
>>> public class GridCacheContinuousQueryFilterExceptionTest extends
>>> GridCacheContinuousQueryAbstractSelfTest implements Serializable {
>>>  /**
>>>   * @throws Exception If failed.
>>>   */
>>>  public void testListenerAfterFilterException() throws Exception {
>>>  IgniteCache cache =
>>> grid(0).cache(DEFAULT_CACHE_NAME);
>>>
>>>  ContinuousQuery qry = new ContinuousQuery<>();
>>>
>>>  final CountDownLatch latch = new CountDownLatch(100);
>>>
>>>  qry.setLocalListener(new CacheEntryUpdatedListener>> Integer>() {
>>>  @Override public void onUpdated(Iterable> extends Integer, ? extends Integer>> evts) {
>>>  for (CacheEntryEvent>> Integer>
>>> evt : evts)
>>>  latch.countDown();
>>>  }
>>>  });
>>>
>>>  qry.setRemoteFilter(new CacheEntryEventSerializableFil
>>> ter>> Integer>() {
>>>  @Override public boolean evaluate(CacheEntryEvent>> Integer, ? extends Integer> evt) {
>>>  throw new RuntimeException("Test error.");
>>>  }
>>>  });
>>>
>>>  try (QueryCursor> ignored =
>>> cache.query(qry)) {
>>>  for (int i = 0; i < 100; i++)
>>>  cache.put(i, i);
>>>
>>>  assertTrue(latch.await(10, SECONDS));
>>>  }
>>>  }
>>>
>>>  @Override protected CacheMode cacheMode() {
>>>  return CacheMode.REPLICATED;
>>>  }
>>>
>>>  @Override protected int gridCount() {
>>>  return 1;
>>>  }
>>> }
>>> ```
>>>
>>> [1] https://github.com/apache/ignite/blob/master/modules/core/
>>> src/main/java/org/apache/ignite/internal/processors/cache/
>>> query/continuous/CacheContinuousQueryHandler.java#L791
>>>
>>> [2] https://github.com/apache/ignite/blob/master/modules/core/
>>> src/test/java/org/apache/ignite/internal/processors/cache/
>>> query/continuous/GridCacheContinuousQueryAbstractSelfTest.java#L359
>>>
>>>
>>> 29.08.2017 14:46, Yakov Zhdanov пишет:
>>>
>>> If filter throws exception entry would be passed to listener.
>>>

>
 this is strange. Imagine a filter that very rarely throws some runtime
 exception due to external or environmental reasons, but in case of
 normal
 execution filter evaluates to false. In case of error entry is passed
 to a
 local listener which can lead to some serious consequences and
 inconsistencies in business logic. We probably need to send entry with a
 notion that there was an error on server.

 --Yakov



>>

Re: .Net client call tx,close in other thread

[Semyon Boikov]

Hi,

Currently tx is AutoCloseable and 'close' should be always called by
interface contract.

Currently Transaction is single-threaded and should not be used by multiple
concurrent threads. But if you call commitAsync and then call 'close' from
commitAsync listener this is correct usage and we should not prohibit
calling 'close' from another thread (such assert if it was added should be
removed).

Thanks
Semyon

On Wed, Jul 26, 2017 at 12:48 PM, Anton Vinogradov  wrote:

> close() for Transaction is a method inherited from AutoCloseable.
>
> It used to rollback tx in case it was not rollbacked or commited inside try
> section
>
> here's the code of close:
>
> @Override public void close() throws IgniteCheckedException {
> TransactionState state = state();
>
>if (state != ROLLING_BACK && state != ROLLED_BACK && state !=
> COMMITTING && state != COMMITTED)
> rollback();
> ...
>
> I see no reason to call close manually or at another thread.
>
> On Wed, Jul 26, 2017 at 12:41 PM, Pavel Tupitsyn 
> wrote:
>
> > We still need to confirm the correct behavior:
> > 1) Should it be possible to call Transaction.close() from a different
> > thread?
> > 2) Do we need to call close() after commit()? What about commitAsync()?
> > What if exception happens?
> >
> > Can someone clarify this?
> >
> > On Tue, Jul 25, 2017 at 11:08 PM, Nikolay Izhikov <
> nizhikov@gmail.com>
> > wrote:
> >
> > > Pavel.
> > >
> > > You will fix .Net client so I don't need to change threadId checks in
> my
> > > pull request.
> > > Is that correct?
> > >
> > > 25.07.2017 20:19, Pavel Tupitsyn пишет:
> > >
> > > Further investigation shows that platform code performs unnecessary
> > close()
> > >> calls for committed & rolled back transactions (sync and async).
> > >> Ticket filed: https://issues.apache.org/jira/browse/IGNITE-5834
> > >>
> > >> I'll fix this tomorrow and it should resolve your issue.
> > >>
> > >> Thanks,
> > >> Pavel
> > >>
> > >> On Tue, Jul 25, 2017 at 7:56 PM, Pavel Tupitsyn  >
> > >> wrote:
> > >>
> > >> Anyway, disallowing close() from another thread would be a breaking
> > >>> change, we can't do that.
> > >>>
> > >>> On Tue, Jul 25, 2017 at 7:36 PM, Pavel Tupitsyn <
> ptupit...@apache.org>
> > >>> wrote:
> > >>>
> > >>> I've reproduced the issue, it happens because in .NET we auto-close
> the
> >  transaction
> >  on commit/rollback. This involves two things:
> >  * release .NET reference to Java object
> >  * call Transaction.close()
> > 
> >  With sync methods this is trivial, but with
> commitAsync/rollbackAsync
> > we
> >  have to call close()
> >  in IgniteFuture listener, which is called in a different thread.
> > 
> >  I think we need to be able to close() a transaction from any thread.
> >  Otherwise I don't see a non-blocking way to deal with
> >  commitAsync/rollbackAsync.
> > 
> >  Thanks,
> >  Pavel
> > 
> >  On Tue, Jul 25, 2017 at 6:10 PM, Николай Ижиков <
> > nizhikov@gmail.com
> >  >
> >  wrote:
> > 
> >  Hi, Pavel
> > >
> > > You can check my pull request.
> > > https://github.com/apache/ignite/pull/2334
> > >
> > > For now I return checks so it has to fail on .Net test suite
> > >
> > >
> > > 2017-07-25 17:16 GMT+03:00 Pavel Tupitsyn :
> > >
> > > Hi Николай,
> > >>
> > >> .NET test in question (TestTxStateAndExceptions) does not do any
> > >> multithreading,
> > >> everything is called from a single thread.
> > >>
> > >> I see that the latest .NET run of your pull request on TeamCity
> > >>
> > > finished
> > >
> > >> successfully:
> > >> http://ci.ignite.apache.org/viewLog.html?buildId=738277
> > >>
> > >> Do you still have a problem with this? If yes, how can I reproduce
> > it?
> > >> Can you prepare a branch where the test fails, so I can run it
> > >> locally?
> > >>
> > >> Thanks,
> > >> Pavel
> > >>
> > >> On Tue, Jul 25, 2017 at 4:47 PM, Николай Ижиков <
> > >>
> > > nizhikov@gmail.com>
> > >
> > >> wrote:
> > >>
> > >> Hello, Igniters.
> > >>>
> > >>> I working on issue https://issues.apache.org/
> > jira/browse/IGNITE-5712
> > >>> I found that .Net client perform transaction
> > operation(`tx.close()`)
> > >>>
> > >> in
> > >
> > >> thread that not owns transaction.
> > >>>
> > >>> So I can't include checks for threadId in my pull request.
> > >>>
> > >>> Is it a bug or a desirable behabiour?
> > >>>
> > >>> With my new check I got following stack trace:
> > >>>
> > >>> Check:
> > >>>
> > >>> `assert (threadId() == Thread.currentThread().getId());`
> > >>>
> > >>> Exception:
> > >>>
> > >>> Test(s) failed. System.AggregateException : One or more errors
> 

Re: Resurrect FairAffinityFunction

[Semyon Boikov]

Valentin,

As far as I know in 2.0 some changes were made in rendezvous function so
now it can provide better result. Do you have some numbers for 2.0 so that
we can compare rendezvous and fair affinity functions?

Thanks

On Tue, Jul 25, 2017 at 5:13 AM,  wrote:

> Agree with Val, we should bring it back.
>
> ⁣D.​
>
> On Jul 24, 2017, 8:14 PM, at 8:14 PM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
> >Guys,
> >
> >Some time ago we removed FairAffinityFunction from the project.
> >However, my
> >communication with users clearly shows that is was a rush decision.
> >Distribution showed by Fair AF is much better than default and for some
> >users it's extremely important. Basically, there are cases when
> >rendezvous
> >function is no-go.
> >
> >The reason for removal was that it was possible to get inconsistent
> >results
> >in case multiple caches were created on different topologies. However,
> >I
> >think this is fixable. As far as I understand, the only thing we need
> >to do
> >is to maintain a single AffinityFunctionContext for all the caches with
> >same affinity function. Currently for each cache we have separate
> >context
> >which holds the state used by Fair AF. If the state is different, we
> >have
> >an issue.
> >
> >The only question is how to check whether two functions are the same or
> >not. In case both cache node filter and backup filter are not
> >configured,
> >this is easy - if number of partitions and excludeNeighbors flag are
> >equal
> >for two functions, these functions are also equal.
> >
> >With filters it's a bit more complicated as these are custom
> >implementations and in general case we don't know how to compare them.
> >Although, to solve this problem, we can enforce user to implement
> >equals()
> >method for these implementation if Fair AF is used.
> >
> >I propose to make changes described above and bring Fair AF back.
> >
> >Thoughts?
> >
> >-Val
>

Re: IGNITE-5123 Review

[Semyon Boikov]

Evgeniy and Dmitry, thanks for the fix! Merged into master.

Thanks!

On Sun, Jul 23, 2017 at 9:21 PM, Dmitry Pavlov 
wrote:

> Hi Evgeniy,
>
> From my point of view there are no problems with this fix. My testing
> didn't show any issues with fix.
>
> Igniters,
>
> Are there any additional comments on this issue? Can we proceed?
>
> Sincerely,
> Dmitriy Pavlov
>
> чт, 20 июл. 2017 г. в 20:04, Dmitry Pavlov :
>
> > Hi Evgeniy,
> >
> > Thank you for such a careful research of the issue.
> >
> > If don’t mind, I would like to do additional tests with this PR changes.
> >
> > I will come back with result in couple of days
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > чт, 20 июл. 2017 г. в 19:18, Evgeniy Ignatiev <
> yevgeniy.ignat...@gmail.com
> > >:
> >
> >> onIgniteStart was called in Ignite 1.X in
> >> GridPluginComponent#onKernalStart as one of the calls to the component
> >> callbacks, probably the order in which components were called, ensured
> >> that contract of PluginProvider#onIgniteStart was not violated. But in
> >> 2.0 the GridPluginComponent instances are explicitly skipped from this
> >> cycle (lines 1019-1020 in Ignite 2.0 release source) and PluginProviders
> >> are notified before the internal component callbacks.
> >>
> >> As far as I can see the change, that moved PluginProvider#onIgniteStart
> >> notification before component callbacks, was introduced by this commit -
> >>
> >> https://github.com/apache/ignite/commit/6b7bf97158c097b80bcf5c2150e67a
> 5210269e6d
> >> - but I have no clue what was the reason.
> >>
> >>
> >> On 7/20/2017 7:51 PM, Dmitry Pavlov wrote:
> >> > Hi Nick,
> >> >
> >> > Thank you for your comment. Was onIgniteStart called after
> >> onKernalStart in
> >> > 1.9? Or caches were available, but other of initialization was the
> same?
> >> >
> >> > Sincerely.
> >> > Dmitriy Pavlov
> >> >
> >> > ср, 19 июл. 2017 г. в 17:06, Nick Pordash :
> >> >
> >> >> Hi Dmitriy,
> >> >>
> >> >> The ticket was a regression from 1.9 to 2.0. I don't think anyone
> >> would be
> >> >> expecting the behavior in 2.0 as it doesn't align with the javadoc
> and
> >> has
> >> >> only been broken since the 2.0 release.
> >> >>
> >> >> -Nick
> >> >>
> >> >> On Wed, Jul 19, 2017, 6:55 AM Dmitry Pavlov 
> >> wrote:
> >> >>
> >> >>> Hi Evgeniy,
> >> >>>
> >> >>> Thank you. Ignite Basic is one suite from approximately 80 suites
> that
> >> >>> covers Ignite by automated tests. Which is why I suggested to use
> >> RunAll
> >> >>> chain in ignite 2.0 group. Yes, several tests may fail, especially
> if
> >> it
> >> >> is
> >> >>> flaky tests or failure is related to the specific JIRA issue.
> >> >>>
> >> >>> About change itself: This change seems to be very impact. There is
> >> >>> possiblity that many of existing plugins relies on existing order of
> >> >>> initialization. This change may break plugin initialization in
> >> unexpected
> >> >>> manner.
> >> >>>
> >> >>> Could we
> >> >>> - fix javadoc according to existing order in code
> >> >>> - find out new solution?
> >> >>>
> >> >>> Sincerely,
> >> >>> Dmitriy Pavlov
> >> >>>
> >> >>>
> >> >>> ср, 19 июл. 2017 г. в 16:40, Evgeniy Ignatiev <
> >> >> yevgeniy.ignat...@gmail.com
> >>  :
> >> 
> >> >>
> >> http://ci.ignite.apache.org/viewLog.html?buildId=720722;
> tab=buildResultsDiv=IgniteTests_IgniteBasic
> >>  - this one - there seem to be no new failed platform tests, other
> >> >> failed
> >>  tests seem to fail in several other reviews too and are unrelated
> to
> >> my
> >>  changes.
> >> 
> >> 
> >>  On 19.07.2017 17:35, Dmitry Pavlov wrote:
> >> > Hi Evgeniy,
> >> >
> >> > I was not able to find Teamcity run for this change.
> >> > Could you please run http://ci.ignite.apache.org test for example
> >> on
> >>  branch
> >> > pull/2285/head using 'Ignite 2.0 Tests' target 'Run All'.
> >> > Or could you please share link to previous run on this changes?
> >> >
> >> > Sincerely,
> >> > Dmitriy Pavlov
> >> >
> >> > ср, 19 июл. 2017 г. в 15:26, Anton Vinogradov :
> >> >
> >> >> Igniters,
> >> >>
> >> >> Could somebody review the fix today?
> >> >>
> >> >> On Wed, Jul 19, 2017 at 1:30 PM, Evgeniy Ignatiev <
> >> >> yevgeniy.ignat...@gmail.com> wrote:
> >> >>
> >> >>> Hello, Igniters.
> >> >>>
> >> >>> Could anyone review my request - https://issues.apache.org/jira
> >> >>> /browse/IGNITE-5123? - My previous pings seems to got lost.
> >> >>>
> >> >>> Best regards,
> >> >>>
> >> >>> Yevgeniy
> >> >>>
> >> >>>
> >> 
> >>
> >>
>

Re: [VOTE] Apache Ignite 2.1.0 RC3

[Semyon Boikov]

+1 binding

On Fri, Jul 21, 2017 at 1:09 PM, Yakov Zhdanov  wrote:

> +1 binding
>
> Checked maven build, imported and built examples project in IDEA and run
> several compute exampled and started 2 nodes with default config.
>
> --Yakov
>
> 2017-07-21 12:21 GMT+03:00 Anton Vinogradov :
>
> > +1 (binding)
> >
> > On Fri, Jul 21, 2017 at 12:08 PM, Pavel Tupitsyn 
> > wrote:
> >
> > > +1
> > >
> > > Checked .NET: nodes start and join from exe file, bat file, NuGet, user
> > > project.
> > >
> > > On Fri, Jul 21, 2017 at 11:49 AM, Vyacheslav Daradur <
> > daradu...@gmail.com>
> > > wrote:
> > >
> > > > +1
> > > >
> > > > Build success now. Node starts without issues.
> > > >
> > > > 2017-07-21 11:38 GMT+03:00 Kozlov Maxim :
> > > >
> > > > > +1
> > > > >
> > > > > > 21 июля 2017 г., в 7:48, Valentin Kulichenko <
> > > > > valentin.kuliche...@gmail.com> написал(а):
> > > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > On Thu, Jul 20, 2017 at 9:35 PM, Sasha Belyak <
> rtsfo...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > >> +1
> > > > > >>
> > > > > >> 2017-07-21 5:34 GMT+07:00 Denis Magda :
> > > > > >>
> > > > > >>> Igniters,
> > > > > >>>
> > > > > >>> Setting off the vote one more time. Hope I’ll be successful
> this
> > > > time,
> > > > > >>> keeping fingers crossed :)
> > > > > >>>
> > > > > >>> We have uploaded a 2.1.0 release candidate to
> > > > > >>> https://dist.apache.org/repos/dist/dev/ignite/2.1.0-rc3/
> > > > > >>>
> > > > > >>> Git tag name is
> > > > > >>> 2.1.0-rc3
> > > > > >>>
> > > > > >>> This release includes the following changes:
> > > > > >>>
> > > > > >>> Ignite:
> > > > > >>> * Persistent cache store
> > > > > >>> * Added IgniteFuture.listenAsync() and
> IgniteFuture.chainAsync()
> > > > > mehtods
> > > > > >>> * Deprecated IgniteConfiguration.marshaller
> > > > > >>> * Updated Lucene dependency to version 5.5.2
> > > > > >>> * Machine learning: implemented K-means clusterization
> algorithm
> > > > > >> optimized
> > > > > >>> for distributed storages
> > > > > >>> * SQL: CREATE TABLE and DROP TABLE commands support
> > > > > >>> * SQL: New thin JDBC driver
> > > > > >>> * SQL: Improved performance of certain queries, when affinity
> > node
> > > > can
> > > > > be
> > > > > >>> calculated in advance
> > > > > >>> * SQL: Fixed return type of AVG() function
> > > > > >>> * SQL: BLOB type support added to thick JDBC driver
> > > > > >>> * SQL: Improved LocalDate, LocalTime and LocalDateTime support
> > for
> > > > > Java 8
> > > > > >>> * SQL: Added FieldsQueryCursor interface to get fields metadata
> > for
> > > > > >>> SqlFieldsQuery
> > > > > >>> * ODBC: Implemented DML statement batching
> > > > > >>> * Massive performance and stability improvements
> > > > > >>>
> > > > > >>> Ignite.NET:
> > > > > >>> * Automatic remote assembly loading
> > > > > >>> * NuGet-based standalone node deployment
> > > > > >>> * Added conditional data removeal via LINQ DeleteAll
> > > > > >>> * Added TimestampAttribute to control DateTime serialization
> mode
> > > > > >>> * Added local collections joins support to LINQ.
> > > > > >>>
> > > > > >>> Ignite CPP:
> > > > > >>> * Added Compute::Call and Compute::Broadcast methods
> > > > > >>>
> > > > > >>> Web Console:
> > > > > >>> * Implemented support for UNIQUE indexes for key fields on
> import
> > > > model
> > > > > >>> from RDBMS
> > > > > >>> * Added option to show full stack trace on Queries screen
> > > > > >>> * Added PK alias generation on Models screen.
> > > > > >>>
> > > > > >>> Complete list of closed issues:
> > > > > >>> https://issues.apache.org/jira/issues/?jql=project%20%
> > > > > 3D%20IGNITE%20AND%
> > > > > >>> 20fixVersion%20%3D%202.1%20AND%20(status%20%3D%
> > > > > >>> 20closed%20or%20status%20%3D%
> > > > > >>> 20resolved)
> > > > > >>>
> > > > > >>> DEVNOTES
> > > > > >>> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_
> > > > > >>> plain;f=DEVNOTES.txt;hb=refs/tags/2.1.0-rc3
> > > > > >>>
> > > > > >>> RELEASE NOTES
> > > > > >>> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_
> > > > > >>> plain;f=RELEASE_NOTES.txt;hb=refs/tags/2.1.0-rc3
> > > > > >>>
> > > > > >>> Please start voting.
> > > > > >>>
> > > > > >>> +1 - to accept Apache Ignite 2.1.0-rc3
> > > > > >>> 0 - don't care either way
> > > > > >>> -1 - DO NOT accept Apache Ignite 2.1.0-rc3 (explain why)
> > > > > >>>
> > > > > >>> This vote will go for 72 hours.
> > > > > >>>
> > > > > >>> —
> > > > > >>> Denis
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > > > --
> > > > > Best Regards,
> > > > > Max K.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best Regards, Vyacheslav D.
> > > >
> > >
> >
>

Re: Local-specific vs cluster-wide configuration setting for cache groups

[Semyon Boikov]

Let's treat consistency of rebalanceMode as critical (this really can cause
problems if one node has rebalanceMode NONE and other SYNC or ASYNC).

Thanks

On Thu, Jul 20, 2017 at 12:46 PM, Sergey Chugunov <sergey.chugu...@gmail.com
> wrote:

> Semen,
>
> What about attribute *rebalanceMode*? From
> *ClusterCachesInfo::validateCacheGroupConfiguration* I see that
> consistency
> of the attribute is treated as critical and node even fails to start if
> different caches in the same group have different rebalanceMode setting.
>
> I've already implemented approach that node simply overrides local value on
> rebalanceMode with the one received from grid.
>
> Do you think this approach is reasonable? Wouldn't it be better to fail
> joining node if such inconsistency is detected?
>
> Thanks,
> Sergey.
>
>
> On Wed, Jul 19, 2017 at 4:27 PM, Semyon Boikov <sboi...@gridgain.com>
> wrote:
>
> > Hi Sergey,
> >
> > I don't think that this really will be needed, but let's allow for local
> > config to override all 'rebalanceXXX' properties.
> > Also please make sure we print warning if any of these properties differs
> > from config received from grid.
> >
> > Thanks!
> >
> > On Fri, Jul 14, 2017 at 2:04 PM, Sergey Chugunov <
> > sergey.chugu...@gmail.com>
> > wrote:
> >
> > > Hello folks,
> > >
> > > Investigating failing test [1] I found that cache group setting
> > > *rebalanceDelay
> > > *was rewritten on joining node by configuration received from the grid
> > when
> > > it supposed to be local-specific.
> > >
> > > I fixed this and test resumed passing, but I have a broader question:
> > which
> > > configuration settings of cache group are cluster-wide and which are
> > > local-specific?
> > >
> > > In [1] I fixed only *rebalanceDelay *setting, what about
> > *rebalanceOrder*?
> > >
> > > Lets compose a list of all local-specific settings in this thread.
> > >
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-5542
> > >
> >
>

Re: Local-specific vs cluster-wide configuration setting for cache groups

[Semyon Boikov]

Hi Sergey,

I don't think that this really will be needed, but let's allow for local
config to override all 'rebalanceXXX' properties.
Also please make sure we print warning if any of these properties differs
from config received from grid.

Thanks!

On Fri, Jul 14, 2017 at 2:04 PM, Sergey Chugunov 
wrote:

> Hello folks,
>
> Investigating failing test [1] I found that cache group setting
> *rebalanceDelay
> *was rewritten on joining node by configuration received from the grid when
> it supposed to be local-specific.
>
> I fixed this and test resumed passing, but I have a broader question: which
> configuration settings of cache group are cluster-wide and which are
> local-specific?
>
> In [1] I fixed only *rebalanceDelay *setting, what about *rebalanceOrder*?
>
> Lets compose a list of all local-specific settings in this thread.
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-5542
>

Re: [VOTE] Apache Ignite 2.1.0 RC2

[Semyon Boikov]

+1 (binding)

On Tue, Jul 18, 2017 at 9:45 PM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> +1 (binding)
>
> On Tue, Jul 18, 2017 at 11:40 AM, Denis Magda  wrote:
>
> > +1 (binding)
> >
> > —
> > Denis
> >
> > > On Jul 17, 2017, at 9:42 PM, Vladimir Ozerov 
> > wrote:
> > >
> > > Igniters!
> > >
> > > We have uploaded a 2.1.0 release candidate to
> > > https://dist.apache.org/repos/dist/dev/ignite/2.1.0-rc2/
> > >
> > > Git tag name is
> > > 2.1.0-rc2
> > >
> > > This release includes the following changes:
> > >
> > > Ignite:
> > > * Persistent cache store
> > > * Added IgniteFuture.listenAsync() and IgniteFuture.chainAsync()
> mehtods
> > > * Deprecated IgniteConfiguration.marshaller
> > > * Updated Lucene dependency to version 5.5.2
> > > * Machine learning: implemented K-means clusterization algorithm
> > optimized
> > > for distributed storages
> > > * SQL: CREATE TABLE and DROP TABLE commands support
> > > * SQL: New thin JDBC driver
> > > * SQL: Improved performance of certain queries, when affinity node can
> be
> > > calculated in advance
> > > * SQL: Fixed return type of AVG() function
> > > * SQL: BLOB type support added to thick JDBC driver
> > > * SQL: Improved LocalDate, LocalTime and LocalDateTime support for
> Java 8
> > > * SQL: Added FieldsQueryCursor interface to get fields metadata for
> > > SqlFieldsQuery
> > > * ODBC: Implemented DML statement batching
> > > * Massive performance and stability improvements
> > >
> > > Ignite.NET:
> > > * Automatic remote assembly loading
> > > * NuGet-based standalone node deployment
> > > * Added conditional data removeal via LINQ DeleteAll
> > > * Added TimestampAttribute to control DateTime serialization mode
> > > * Added local collections joins support to LINQ.
> > >
> > > Ignite CPP:
> > > * Added Compute::Call and Compute::Broadcast methods
> > >
> > > Web Console:
> > > * Implemented support for UNIQUE indexes for key fields on import model
> > > from RDBMS
> > > * Added option to show full stack trace on Queries screen
> > > * Added PK alias generation on Models screen.
> > >
> > > Complete list of closed issues:
> > > https://issues.apache.org/jira/issues/?jql=project%20%
> 3D%20IGNITE%20AND%
> > > 20fixVersion%20%3D%202.1%20AND%20(status%20%3D%
> > 20closed%20or%20status%20%3D%
> > > 20resolved)
> > >
> > > DEVNOTES
> > > https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_
> > plain;f=DEVNOTES.txt;hb=refs/tags/2.1.0-rc2
> > >
> > > RELEASE NOTES
> > > https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=blob_
> > plain;f=RELEASE_NOTES.txt;hb=refs/tags/2.1.0-rc2
> > >
> > > Please start voting.
> > >
> > > +1 - to accept Apache Ignite 2.1.0-rc2
> > > 0 - don't care either way
> > > -1 - DO NOT accept Apache Ignite 2.1.0-rc2 (explain why)
> > >
> > > This vote will go for 72 hours.
> >
> >
>

Re: Testing of backward compatibility in Ignite

[Semyon Boikov]

It is possible to implement a test plugin which will
override DiscoveryNodeValidationProcessor.

Also IMO it is better to start different Ignite versions in different
processes to avoid potential problems with custom classloaders (there is
already some helper classes for multi-jvm tests).

Thanks

On Wed, Jul 12, 2017 at 12:44 PM, Anton Vinogradov <a...@apache.org> wrote:

> Semen,
>
> As far as I know, it is possible to allow nodes with different versions to
> join topology.
> Some products based on Ignite provides this feature.
>
> We'd like to check that changes at Ignite will not break compatibility
> feature of these products.
>
> Could you provide some tips how to connect different versions of Ignite in
> proper way?
>
>
> On Wed, Jul 12, 2017 at 12:05 PM, Semyon Boikov <sboi...@gridgain.com>
> wrote:
>
> > Hi Vyacheslav,
> >
> > Currently Ignite nodes with different versions can not join the same
> > cluster, look at OsDiscoveryNodeValidationProcessor (this check is
> called
> > from discovery ServerImpl IgniteSpiContext.validateNode).
> >
> > Thanks
> >
> > On Wed, Jul 12, 2017 at 11:09 AM, Vyacheslav Daradur <
> daradu...@gmail.com>
> > wrote:
> >
> > > Hi Igniters.
> > >
> > > There is a need to test backward compatibility with previous release
> > within
> > > a task[1].
> > >
> > > As far as I understand: it needs to provide methods in the
> > > Ignite-test-framework to start instance of specific version of Ignite.
> > >
> > > I popose to add such api in the project, for example: "Ignite
> > > startGrid(String ver)"
> > >
> > > My main idea of implementation is:
> > > 1) download "jar" from the Maven repo
> > > 2) Wrap it in a custom classloader
> > > 3) Start instance via the classloader
> > >
> > > I've created a ticket[2] for this task.
> > >
> > > Guys, please, share your thoughts about described problem and ways of
> > > implementation?
> > >
> > > [1] BinaryMarshaller should write ints in "varint" encoding where it
> > makes
> > > sense <https://issues.apache.org/jira/browse/IGNITE-5097>
> > > [2] Provide API to test compatibility with old releases
> > > <https://issues.apache.org/jira/browse/IGNITE-5732>
> > >
> > > --
> > > Best Regards, Vyacheslav D.
> > >
> >
>

Re: Testing of backward compatibility in Ignite

[Semyon Boikov]

Hi Vyacheslav,

Currently Ignite nodes with different versions can not join the same
cluster, look at OsDiscoveryNodeValidationProcessor (this check is called
from discovery ServerImpl IgniteSpiContext.validateNode).

Thanks

On Wed, Jul 12, 2017 at 11:09 AM, Vyacheslav Daradur 
wrote:

> Hi Igniters.
>
> There is a need to test backward compatibility with previous release within
> a task[1].
>
> As far as I understand: it needs to provide methods in the
> Ignite-test-framework to start instance of specific version of Ignite.
>
> I popose to add such api in the project, for example: "Ignite
> startGrid(String ver)"
>
> My main idea of implementation is:
> 1) download "jar" from the Maven repo
> 2) Wrap it in a custom classloader
> 3) Start instance via the classloader
>
> I've created a ticket[2] for this task.
>
> Guys, please, share your thoughts about described problem and ways of
> implementation?
>
> [1] BinaryMarshaller should write ints in "varint" encoding where it makes
> sense 
> [2] Provide API to test compatibility with old releases
> 
>
> --
> Best Regards, Vyacheslav D.
>

Re: golang client for Ignite

[Semyon Boikov]

Hi Aleksandr,

Regarding transactions: now implementation of transactions assumes that
transaction always belongs to some Ignite node, and it seems it is not
simple task to support transactions for 'thin' clients.

Thanks

On Fri, Jun 30, 2017 at 9:55 AM, Aleksandr Sokolovskii 
wrote:

> Dear Vladimir,
>
> Thanks for your answer.
>
> Now I understand the difference between ‘client’ and ‘thin client’.
>
> Just to be clear I develop ‘thin client’ for DataGrid now:
> https://github.com/amsokol/go-ignite-client
> and all my questions are in context of ‘thin client’.
>
> So the questions are:
> 1) Does EdgeNode provide binary endpoint that supports cache management,
> put-get, sql exec queries, fetch, transactions and I can use it by ‘thin’
> golang client?
> 2) If the answer to the question (1) is ‘NO’ – is it possible to add
> transactions to ODBC endpoint?
> 3) If the answer to the question (2) is ‘NO’ – is it possible to add
> transactions to REST API?
>
> Thanks,
> Aleksandr
>
> From: Vladimir Ozerov
> Sent: 30 июня 2017 г. 8:18
> To: dev@ignite.apache.org
> Cc: Roman Shtykh
> Subject: Re: golang client for Ignite
>
> Aleksandr,
>
> Currently Ignite.NET and Ignite C++ work as follows:
> >>> Client[platform->*jni->java*]->network->Server[java]
>
> Or as follows in some cases (e.g. calling remote .NET job from local .NET
> node):
> >>> Client[platform->*jni->java*]->network->Server[java->*jni->platform*]
>
> Possible thin client will work as follows (ODBC driver already works this
> way):
> >>> Client[platform]->*network*->EdgeNode[java]->network->Server[java]
>
> Note additional network hop from client to edge node because thin client
> lacks routing logic.
>
>
> On Fri, Jun 30, 2017 at 12:38 AM, Aleksandr Sokolovskii  >
> wrote:
>
> > Dear Vladimir,
> >
> > Thanks for your answer.
> >
> > I’m a little bit confused.
> >
> > You write:
> > We have a plan to implement platform-independent thin client protocol
> which
> > could be re-used from many languages. But you should understand, that in
> > most cases it will be *slower* than current implementation, because Java
> > core contains essential logic for efficient request routing.
> >
> > Current C++ realization in Ignite now:
> > Client[cpp]->network->Server[cpp->java]
> >
> > If you recommend use “Java core contains essential logic for efficient
> > request routing” why you don’t implement the following for C++?:
> > Client[cpp->java]->network->Server[java]
> >
> >
> > Why platform-independent protocol:
> > Client[cpp->]->network->Server[java]
> >
> > Will be slow than current?:
> > Client[cpp]->network->Server[cpp->java]
> >
> > Thanks,
> > Aleksandr
> >
> > From: Vladimir Ozerov
> > Sent: 30 июня 2017 г. 0:25
> > To: dev@ignite.apache.org
> > Cc: Roman Shtykh
> > Subject: Re: golang client for Ignite
> >
> > Aleksandr,
> >
> > Ignite is distributed system. Typical call to get/put/sql is of micro-
> and
> > millisecond magnitude. Java->CPP call takes several dozen nanoseconds.
> > CPP->Java (so-called "reverse JNI") takes several hundred nanoseconds.
> > Typical marshalling overhead is of nano- and micro-second scale as well.
> > That is, total overhead of platform->Java->platform interaction is
> several
> > orders of magnitude *smaller* than any call to remote node. So if Golang
> > users are afraid of JNI, than any distributed system would scare them to
> > death.
> >
> > We have a plan to implement platform-independent thin client protocol
> which
> > could be re-used from many languages. But you should understand, that in
> > most cases it will be *slower* than current implementation, because Java
> > core contains essential logic for efficient request routing. Native thin
> > client could be faster only if you port many and many thousands of
> relevant
> > non-trivial Java code to native platform, which can be estimated not to
> > man-months, but to man-years to complete.
> >
> > Having said that, I would recommend you to look closer to current
> JNI-based
> > implementation :-)
> >
> >
> > On Thu, Jun 29, 2017 at 9:11 AM, Aleksandr Sokolovskii <
> amso...@gmail.com>
> > wrote:
> >
> > > Guys,
> > >
> > > Thanks for your answers.
> > >
> > > So If users want to use Ignite DataGrid from “unsupported” language
> they
> > > have two choice for now:
> > >
> > > 1) REST API
> > > It supports: cache management, put-get, sql exec queries, fetch
> > > It does not support: sql transactions
> > > REST API is easy to use.
> > > REST API has a lot of overhead (HTTP, json marshalling/unmarshalling,..
> > > you know)
> > >
> > > 2) develop pure SQL driver for Ignite ODBC endpoint
> > > It supports: sql exec queries, fetch
> > > It does not support: cache management, put-get, sql transactions
> > > Roman is right – using cgo is not very good idea.
> > > Honestly It’s not trivial task to develop pure SQL driver for Ignite
> ODBC
> > > endpoint.
> > > I spent some hours to remember how STL serializes std::string to
> > > 

Re: Request for contributor permissions

[Semyon Boikov]

Done.

On Thu, Jun 22, 2017 at 12:21 PM, Вячеслав Коптилин <
slava.kopti...@gmail.com> wrote:

> Hello Igniters,
>
> My name is Slava and I want to contribute to the project.
> In accordance with the following guide
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
> I need contributor permissions. So can I request such permissions?
>
> My jira account: slava.koptilin
>
> Best Regards,
> Slava.
>

Re: More JIRA rights

[Semyon Boikov]

Done.

On Wed, Jun 21, 2017 at 11:44 AM, Jokser  wrote:

> Hi guys,
>
> Could you please give me JIRA task workflow rights (assign, change status),
> edit descriptions and so on?
> My JIRA account is Jokser.
>
>
>
> --
> View this message in context: http://apache-ignite-
> developers.2346864.n4.nabble.com/More-JIRA-rights-tp19006.html
> Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
>

Re: TeamCity triggers change proposal

[Semyon Boikov]

Hi Dmitry,

Is it possible instead to set priority for manually started builds higher
than for builds triggered by git?

Thanks

On Fri, Jun 16, 2017 at 2:37 PM, Dmitry Pavlov 
wrote:

> Hi Igniters,
>
>
>
> Most of us at least once have faced with situation that we have important
> fix to be tested, but TeamCity is busy and queue is too long. Ignite 2.0
> test introduction helped to reduce time to wait, but still there is
> potential to get results for your PR tests faster.
>
>
>
> Contribution itself is tested under PR and TC ‘Run All’ started manually.
> Second test will be performed later: all test suites will be started for
> after merge into master (ignite 2.0).
>
>
>
> I offer to change VSC trigger to schedule trigger for master and 2.0
> branches. Schedule trigger will be launched daily at period of low activity
> (TC queue is empty), for example at 2am GMT.
>
>
>
> This solution help to get results faster for each PR. Solution still
> protects master from merge problems. If master tests are required due to
> high risk, committer will be still able run start RunAll For master.
>
>
>
> Any thoughts or comments? I would like apply this change by the end of next
> week.
>
>
>
> Sincerely,
>
> Dmitriy Pavlov
>

Persistence enabled for all caches

[Semyon Boikov]

Hi guys,

As I understand, currently by design, if I set
IgniteConfiguration.persistentStoreConfiguration then persistence is
automatically enabled for ALL caches. Don't you think it can be useful to
have possibility to configure persistence per cache?

Thanks!

Re: Deprecate IgniteConfiguration.marshaller and make it no-op!

[Semyon Boikov]

As far as I know before 2.0 SQL worked without BinaryMarshaller, is this
support was removed intentionally? Also I think it is possible to implement
DML without BinaryMashaller as well?

I'm not against Vladimir's suggestion, but I think it make sense to ask on
user list to know if somebody uses custom marshallers.

Thanks

On Fri, Jun 9, 2017 at 3:15 AM, Dmitriy Setrakyan 
wrote:

> On Thu, Jun 8, 2017 at 2:30 PM, Vladimir Ozerov 
> wrote:
>
> > Denis,
> >
> > I think we should not provide distinction between "data grid", "compute
> > grid" and other use case. We should have consistent product which just
> > works.
> >
> > Anyway,I am OK to just deprecate these property for 2.0 and remove 3.0 if
> > community thinks that custom marshallers are still needed.
> >
>
> Vladimir, I tend to agree with you here. +1 on deprecating it.
>

Re: Only lateAffinityAssignment for 2.1

[Semyon Boikov]

Created https://issues.apache.org/jira/browse/IGNITE-5446.

Thanks

On Wed, Jun 7, 2017 at 5:25 AM, Dmitriy Setrakyan <dsetrak...@apache.org>
wrote:

> On Tue, Jun 6, 2017 at 4:23 AM, Semyon Boikov <sboi...@gridgain.com>
> wrote:
>
> > Now affinity version is switched only when migration for new primaries
> > finishes for all partitions and for all caches.
> >
>
> Sounds great. Semyon, is there documentation that needs to be updated as
> well?
>

Only lateAffinityAssignment for 2.1

[Semyon Boikov]

Hi all,

Currently in Igntie we have two modes for cache affinity changing
(IgniteConfiguration.isLateAffinityAssignment=true/false). Late affinity
assignment mode is enabled by default and I do not see when it can be
useful to disable it. Does it make sense for 2.1 to deprecate property
IgniteConfiguration.isLateAffinityAssignment and internally always use late
affinity assignment mode? This will allow to simplify cache code (e.g. get
rid of ForceKeyRequests/Responses).

What do you think?

Thanks!

Persistent Store and IgniteServices

[Semyon Boikov]

Hi guys,

If persistent store feature is enabled, will IgniteServices survive cluster
restrart?

Thanks!

Re: Persistent Store Stabilization for release

[Semyon Boikov]

Also in persistence code I see some TODOs (//TODO GG-), all TODOs should
reference some ingite JIRA issue,

Thanks!

On Fri, Jun 2, 2017 at 11:29 AM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> Semyon, thanks for spotting this. I've removed the fails as tests should be
> passing now.
>
> 2017-06-02 10:20 GMT+03:00 Semyon Boikov <sboi...@gridgain.com>:
>
> > Guys,
> >
> > I see some tests in PDS suite fail with message
> > "junit.framework.AssertionFailedError: only for one run, must be removed
> > soon". We never add suche messages without reference to JIRA issue,
> please
> > fix.
> >
> > Thanks!
> >
> > On Thu, Jun 1, 2017 at 7:22 PM, Alexey Goncharuk <
> > alexey.goncha...@gmail.com
> > > wrote:
> >
> > > Guys,
> > >
> > > I merged recent changes from the master branch (SQL-related changes) to
> > the
> > > integration branch, we need another TC run to see if the merge broke
> > > something.
> > >
> > > 2017-06-01 19:14 GMT+03:00 Dmitriy Setrakyan <dsetrak...@apache.org>:
> > >
> > > > BTW, can anyone explain to me why do we keep the new persistence code
> > in
> > > a
> > > > separate module instead of merging it to core?
> > > >
> > > > On Thu, Jun 1, 2017 at 4:32 AM, Sergey Chugunov <
> > > sergey.chugu...@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hello Denis,
> > > > >
> > > > > There are three suites [1] [2] [3] configured on TeamCity with
> bunch
> > of
> > > > > tests for PDS functionality.
> > > > >
> > > > > As you can see most of tests are passing now (only 6 tests are
> > > > > failing, for *partitionLoss
> > > > > *tests problem was almost sorted out), it looks like that this
> > > > > functionality is pretty stable.
> > > > >
> > > > >
> > > > > [1] Ignite PDS 1
> > > > > <http://ci.ignite.apache.org/viewType.html?buildTypeId=
> > > > > Ignite20Tests_IgnitePds1_Ignite20Tests=ignite-
> > > > > 5267=buildTypeStatusDiv>
> > > > > [2] Ignite PDS 2
> > > > > <http://ci.ignite.apache.org/viewType.html?buildTypeId=
> > > > > Ignite20Tests_IgnitePds2_Ignite20Tests=ignite-
> > > > > 5267=buildTypeStatusDiv>
> > > > > [3] Ignite OutOfMemory
> > > > > <http://ci.ignite.apache.org/viewType.html?buildTypeId=
> > Ignite20Tests_
> > > > > IgnitePdsOutOfMemory_Ignite20Tests=ignite-5267=
> > > > > buildTypeStatusDiv>
> > > > >
> > > > > On Thu, Jun 1, 2017 at 1:24 AM, Dmitriy Setrakyan <
> > > dsetrak...@apache.org
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Thanks Denis. I also would like to encourage to post all the
> > > > > stabilization
> > > > > > updates for the donated branch here. This will help the community
> > to
> > > > get
> > > > > > familiar with the new code and functionality.
> > > > > >
> > > > > > In addition, everyone in the community should feel free to peruse
> > > > through
> > > > > > the code in this branch and ask any questions that may come up:
> > > > > >
> > > > > > https://github.com/apache/ignite/tree/ignite-5267/modules/pds
> > > > > >
> > > > > > D.
> > > > > >
> > > > > >
> > > > > > On Wed, May 31, 2017 at 2:37 PM, Denis Magda <dma...@apache.org>
> > > > wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > The Persistent Store donation was accepted by the community and
> > now
> > > > we
> > > > > > > should think of its readiness for a public release.
> > > > > > >
> > > > > > > I know that many of you are involved it the donation
> > stabilization.
> > > > > Could
> > > > > > > you share the current status of the feature? How far is it from
> > the
> > > > > > > releasable state?
> > > > > > >
> > > > > > > —
> > > > > > > Denis
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Persistent Store Stabilization for release

[Semyon Boikov]

Guys,

I see some tests in PDS suite fail with message
"junit.framework.AssertionFailedError: only for one run, must be removed
soon". We never add suche messages without reference to JIRA issue, please
fix.

Thanks!

On Thu, Jun 1, 2017 at 7:22 PM, Alexey Goncharuk  wrote:

> Guys,
>
> I merged recent changes from the master branch (SQL-related changes) to the
> integration branch, we need another TC run to see if the merge broke
> something.
>
> 2017-06-01 19:14 GMT+03:00 Dmitriy Setrakyan :
>
> > BTW, can anyone explain to me why do we keep the new persistence code in
> a
> > separate module instead of merging it to core?
> >
> > On Thu, Jun 1, 2017 at 4:32 AM, Sergey Chugunov <
> sergey.chugu...@gmail.com
> > >
> > wrote:
> >
> > > Hello Denis,
> > >
> > > There are three suites [1] [2] [3] configured on TeamCity with bunch of
> > > tests for PDS functionality.
> > >
> > > As you can see most of tests are passing now (only 6 tests are
> > > failing, for *partitionLoss
> > > *tests problem was almost sorted out), it looks like that this
> > > functionality is pretty stable.
> > >
> > >
> > > [1] Ignite PDS 1
> > >  > > Ignite20Tests_IgnitePds1_Ignite20Tests=ignite-
> > > 5267=buildTypeStatusDiv>
> > > [2] Ignite PDS 2
> > >  > > Ignite20Tests_IgnitePds2_Ignite20Tests=ignite-
> > > 5267=buildTypeStatusDiv>
> > > [3] Ignite OutOfMemory
> > >  > > IgnitePdsOutOfMemory_Ignite20Tests=ignite-5267=
> > > buildTypeStatusDiv>
> > >
> > > On Thu, Jun 1, 2017 at 1:24 AM, Dmitriy Setrakyan <
> dsetrak...@apache.org
> > >
> > > wrote:
> > >
> > > > Thanks Denis. I also would like to encourage to post all the
> > > stabilization
> > > > updates for the donated branch here. This will help the community to
> > get
> > > > familiar with the new code and functionality.
> > > >
> > > > In addition, everyone in the community should feel free to peruse
> > through
> > > > the code in this branch and ask any questions that may come up:
> > > >
> > > > https://github.com/apache/ignite/tree/ignite-5267/modules/pds
> > > >
> > > > D.
> > > >
> > > >
> > > > On Wed, May 31, 2017 at 2:37 PM, Denis Magda 
> > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > The Persistent Store donation was accepted by the community and now
> > we
> > > > > should think of its readiness for a public release.
> > > > >
> > > > > I know that many of you are involved it the donation stabilization.
> > > Could
> > > > > you share the current status of the feature? How far is it from the
> > > > > releasable state?
> > > > >
> > > > > —
> > > > > Denis
> > > >
> > >
> >
>

Re: contention on DataStructure creation/removing

[Semyon Boikov]

Hi Mikhail,

As far as I remember for some reason we wanted to guarantee that all data
structures have unique names. But now I don't see why this can be needed
and it seems we do not need this data structures map at all, if nobody have
objection I think you can implement suggested change.

Thanks!

On Wed, May 31, 2017 at 3:04 PM, Mikhail Cherkasov 
wrote:

> Hi all,
>
> All DataStructures are stored in one Map which itself is stored in
> utilityCache, this makes high contention on DS creation or removing, it
> requires to acquire Map's lock and manipulation with the Map under the
> lock. So all threads in cluster should wait for this lock to create or
> remove DS.
>
> I don't see any reason to store all DS in one map,  we already have
> utilityCache and can save DSs directly in utilityCache, to distinguish DS
> with other objects in utilityCache I use composite key, the first part of
> which is DATA_STRUCTURES_KEY, second one is DS's name, also DS type can be
> added, this will allow us to create different DS with the same name.
>
> There is draft implementations, all DSs are stored with unique key in
> utilityCache:
> https://github.com/apache/ignite/pull/2046/files
>
> May be there's some reason to store all DS in one Map that I missed?
> Any thoughts?
>
>
> --
> Thanks,
> Mikhail.
>

Re: Ignite 2.0 TeamCity Run configurations

[Semyon Boikov]

What about 'Ignite H2 Indexing' suite', it does not execute any test?

On Fri, May 26, 2017 at 2:18 PM, Dmitry Pavlov <dpavlov@gmail.com>
wrote:

> Hi Semen,
>
>
>
> Thank you for this fix. It is correct to replace to IgniteCacheTestSuite4
> in new 2.0 run configurations as we don’t have Binary* analogue.
>
>
>
> IgniteBinaryBasicTestSuite was not deleted to avoid failures in old run
> configurations. I assumed missing test suite will cause failure on
> teamcity. But the first issue shows us it is not a problem. I’ll prepare PR
> for removing suite.
>
>
>
> IgniteBinaryObjectsCacheTestSuite3 is kept together with
> IgniteCacheTestSuite3 for backward compatibility. In Ignite 2.0 tests first
> is used, and in Ignite tests - second. And if someone runs old run configs
> then most test will be executed anyway. Practice shows not everyone in
> community uses new Run Configs for master-based PRs. In future we will be
> able to remove not necessary suite.
>
>
> Sincerely,
>
> Dmitriy Pavlov
>
>
>
> пт, 26 мая 2017 г. в 14:00, Semyon Boikov <sboi...@gridgain.com>:
>
> > I just found that 'Ignite Cache 4' was not executed on TeamCity since it
> > has from suite name in configuration, I fixed it to use
> > IgniteCacheTestSuite4, is it correct?
> >
> > Aslo why we need both IgniteBinaryObjectsCacheTestSuite3 and
> > IgniteCacheTestSuite3? And why IgniteBinaryBasicTestSuite is not deleted?
> >
> > Thanks
> >
> > On Mon, May 22, 2017 at 6:39 PM, Dmitry Pavlov <dpavlov@gmail.com>
> > wrote:
> >
> > > Igniters,
> > >
> > > I've created new PR https://github.com/apache/ignite/pull/1986 without
> > > merge conflicts against master. Please review these changes instead of
> > > PR1956.
> > >
> > > Thank you
> > >
> > > Best Regards,
> > > Dmitry Pavlov
> > >
> > > пн, 22 мая 2017 г. в 13:21, Alexey Goncharuk <
> alexey.goncha...@gmail.com
> > >:
> > >
> > > > Thanks, Dmitriy!
> > > >
> > > > If there are no other objections, I will merge the changes shortly.
> > > >
> > > > 2017-05-22 13:10 GMT+03:00 Dmitry Pavlov <dpavlov@gmail.com>:
> > > >
> > > > > Hi, Igniters!
> > > > >
> > > > > Could you please review and merge changes from
> > > > > https://github.com/apache/ignite/pull/1956 into master.
> > > > >
> > > > > These changes makes running tests with similar results in TC Run
> > > configs
> > > > > "Ignite 2.0 Tests" as it is in "Ignite Tests".
> > > > >
> > > > > Best regards,
> > > > > Dmitry Pavlov
> > > > >
> > > > > ср, 17 мая 2017 г. в 20:52, Alexey Goncharuk <
> > > alexey.goncha...@gmail.com
> > > > >:
> > > > >
> > > > > > Great news! Looking forward to getting rid of the unnecessary TC
> > > > > > configurations.
> > > > > >
> > > > > > Once this is done, I think we should also work out if we can use
> > > build
> > > > > > once, run tests approach for RunAll configuration. Does anybody
> > have
> > > a
> > > > > clue
> > > > > > if this is possible?
> > > > > >
> > > > > > 2017-05-17 20:46 GMT+03:00 Dmitry Pavlov <dpavlov@gmail.com
> >:
> > > > > >
> > > > > > > Hi, Igniters!
> > > > > > >
> > > > > > > Alex (al.psc) has created new Teamcity run configurations
> > optimized
> > > > for
> > > > > > > running 2.0 version and later. This project and its run
> > > > configurations
> > > > > > > avoid duplicate tests running for binary and default
> marshallers
> > > > > because
> > > > > > it
> > > > > > > is now the same for 2.0+.
> > > > > > >
> > > > > > > Usage of this run configuration saves us at least 12
> > machine-hours
> > > > for
> > > > > > each
> > > > > > > test run and in future will allow us to get test results
> faster.
> > > > > > >
> > > > > > > Do you have concerns or remarks about using new run
> > configurations
> > > > for
> > > > > > all
> > > > > > > 2.0+ pull requests?
> > > > > > >
> > > > > > > We are now doing final checks of compatibility of these run
> > > > > > configurations
> > > > > > > with code. Any ideas about what is to be migrated are
> > appreciated.
> > > > > > >
> > > > > > > Best Regards,
> > > > > > > Dmitry Pavlov
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Ignite 2.0 TeamCity Run configurations

[Semyon Boikov]

I just found that 'Ignite Cache 4' was not executed on TeamCity since it
has from suite name in configuration, I fixed it to use
IgniteCacheTestSuite4, is it correct?

Aslo why we need both IgniteBinaryObjectsCacheTestSuite3 and
IgniteCacheTestSuite3? And why IgniteBinaryBasicTestSuite is not deleted?

Thanks

On Mon, May 22, 2017 at 6:39 PM, Dmitry Pavlov 
wrote:

> Igniters,
>
> I've created new PR https://github.com/apache/ignite/pull/1986 without
> merge conflicts against master. Please review these changes instead of
> PR1956.
>
> Thank you
>
> Best Regards,
> Dmitry Pavlov
>
> пн, 22 мая 2017 г. в 13:21, Alexey Goncharuk :
>
> > Thanks, Dmitriy!
> >
> > If there are no other objections, I will merge the changes shortly.
> >
> > 2017-05-22 13:10 GMT+03:00 Dmitry Pavlov :
> >
> > > Hi, Igniters!
> > >
> > > Could you please review and merge changes from
> > > https://github.com/apache/ignite/pull/1956 into master.
> > >
> > > These changes makes running tests with similar results in TC Run
> configs
> > > "Ignite 2.0 Tests" as it is in "Ignite Tests".
> > >
> > > Best regards,
> > > Dmitry Pavlov
> > >
> > > ср, 17 мая 2017 г. в 20:52, Alexey Goncharuk <
> alexey.goncha...@gmail.com
> > >:
> > >
> > > > Great news! Looking forward to getting rid of the unnecessary TC
> > > > configurations.
> > > >
> > > > Once this is done, I think we should also work out if we can use
> build
> > > > once, run tests approach for RunAll configuration. Does anybody have
> a
> > > clue
> > > > if this is possible?
> > > >
> > > > 2017-05-17 20:46 GMT+03:00 Dmitry Pavlov :
> > > >
> > > > > Hi, Igniters!
> > > > >
> > > > > Alex (al.psc) has created new Teamcity run configurations optimized
> > for
> > > > > running 2.0 version and later. This project and its run
> > configurations
> > > > > avoid duplicate tests running for binary and default marshallers
> > > because
> > > > it
> > > > > is now the same for 2.0+.
> > > > >
> > > > > Usage of this run configuration saves us at least 12 machine-hours
> > for
> > > > each
> > > > > test run and in future will allow us to get test results faster.
> > > > >
> > > > > Do you have concerns or remarks about using new run configurations
> > for
> > > > all
> > > > > 2.0+ pull requests?
> > > > >
> > > > > We are now doing final checks of compatibility of these run
> > > > configurations
> > > > > with code. Any ideas about what is to be migrated are appreciated.
> > > > >
> > > > > Best Regards,
> > > > > Dmitry Pavlov
> > > > >
> > > >
> > >
> >
>

Re: [VOTE] Accept Contribution of Ignite Persistent Store

[Semyon Boikov]

+1

On Tue, May 23, 2017 at 12:55 AM, Denis Magda  wrote:

> Igniters,
>
> This branch (https://github.com/apache/ignite/tree/ignite-5267) adds a
> distributed and transactional Persistent Store to Apache Ignite project.
> The store seamlessly integrates with Apache Ignite 2.0 page memory
> architecture. One of the main advantages of the store is that Apache Ignite
> becomes fully operational from disk (SSD or Flash) without any need to
> preload the data in memory. Plus, with full SQL support already available
> in Apache Ignite, this feature will allow Apache Ignite serve as a
> distributed SQL database, both in memory or on disk, while continuing to
> support all the existing functionality on the current API.
> More information here:
> - Persistent Store Overview: https://cwiki.apache.org/
> confluence/display/IGNITE/Persistent+Store+Overview
> - Persistent Store Internal Design: https://cwiki.apache.org/
> confluence/display/IGNITE/Persistent+Store+Internal+Design
> The Persistent Store was developed by GridGain outside of Apache community
> because it was requested by one of GridGain’s customers. Presently,
> GridGain looks forward to donating the Persistent Store to ASF and given
> the size of the contribution, it is prudent to follow Apache's IP clearance
> process.
> The SGA has been submitted and acknowledged by ASF Secretary. The IP
> clearance form can be found here: http://incubator.apache.org/
> ip-clearance/persistent-distributed-store-ignite.html
> This vote is to discover if the Apache Ignite PMC and community are in
> favour of accepting this contribution.
> This vote will be open for at least 72 hours:
> [ ] +1, accept contribution of the Persistent Store into the project
> [ ] 0, no opinion
> [ ] -1, reject contribution because...
>
> Regards,
> Denis
>
>

Re: IGNITE-1084 is ready for review

[Semyon Boikov]

Hi Vadim,

I see that testNaturalIdCache still fails for
HibernateL2CacheTransactionalSelfTest, need investigate and find root cause
of failure.

Thanks

On Wed, May 10, 2017 at 5:03 PM, Вадим Опольский <vaopols...@gmail.com>
wrote:

> Hello guys!
>
> Semen, Sergey is issue https://issues.apache.org/jira/browse/IGNITE-1084 
> resolved
> ?
>
> Vadim Opolski
>
> 2017-04-19 14:06 GMT+03:00 Semyon Boikov <sboi...@gridgain.com>:
>
>> Vadim,
>>
>> I found that real cause of IGNITE-1084 is IGNITE-4760, so suggested test
>> fix in pull/1828 is not correct. I added new comments in IGNITE-4760,
>> please take a look.
>>
>> Thanks,
>> Semyon
>>
>> On Wed, Apr 19, 2017 at 12:17 PM, Вадим Опольский <vaopols...@gmail.com>
>> wrote:
>>
>>> Hello guys!
>>>
>>> The issue IGNITE-1084 https://issues.apache.org/jira/browse/IGNITE-1794 
>>> fixed
>>> originally by Milap Wadhwa.
>>>
>>> I reassigned it on myself because she didn't make improvements from
>>> Anton Vinogradov's comments. Also I just finished another issue with
>>> hibernate's test.
>>>
>>> Review it please - https://github.com/apache/ignite/pull/1828
>>>
>>> Vadim Opolski
>>>
>>
>>
>

Re: IGNITE-4447 ready for review

[Semyon Boikov]

Hi Vadim,

Did you check test pass on TeamCity in this suite, could you please provide
link for related TeamCity run?

Thanks

On Tue, May 16, 2017 at 1:35 PM, Вадим Опольский 
wrote:

> Hi guys!
>
> IgniteCache150ClientsTest.java was removed from
> IgniteClientTestSuite.java to IgniteCacheTestSuite.java. Single test suite
> was deleted.
>
> Review please this fix - https://github.com/apache/ignite/pull/1952
>
> Vadim Opolskii
>
>
>
> 2017-05-15 11:38 GMT+03:00 Вадим Опольский :
>
> > Hi guys!
> >
> > Vladimir, is issue https://issues.apache.org/jira/browse/IGNITE-4447
> > actual now?
> >
> > Which tests do u mean? Detail please.
> >
> > Vadim
> >
>

Re: IGNITE-4437 Make sure data structures do not use outTx call

[Semyon Boikov]

Hi,

Yes, this was done as part of IGNITE-2893, I closed ticket.

Semyon

On Tue, May 2, 2017 at 4:33 PM, Дмитрий Рябов  wrote:

> Hello, igniters. Seems like someone already cut all outTx calls, because
> 2.0rc2 doesn't have any of them. So we can just close this ticket.
>

Re: joining exchange

[Semyon Boikov]

To send message to coordinator we need establish new connection, it is
preferably to use existing connections. Actually we tested this approach,
and on large clusters (> 200 nodes) it did not give start time improvments.

Thanks

On Wed, Apr 26, 2017 at 1:06 PM, ALEKSEY KUZNETSOV  wrote:

> Hi, Igntrs!
> When new node joins topology, it sends join TcpDiscoveryJoinRequestMessage
> to arbitrary node N1.
> If N1 is not coordinator, then it will resend to all nodes.
>
> Why doesn't N1 send the message directly to coordinator ?
> --
>
> *Best Regards,*
>
> *Kuznetsov Aleksey*
>

Re: Proper collocation of computations and data.

[Semyon Boikov]

Alexei,

I think AffinityKeyMapped supposed to work for ComputeJobs and any closures
executed by IgniteCompute, but it seems there are no tests for this and
this functionalty is broken now, and I think this should be fixed.

Thanks,
Semyon

On Wed, Apr 19, 2017 at 9:59 PM, Alexei Scherbakov <
alexey.scherbak...@gmail.com> wrote:

> Dmitriy,
>
> I know I could, but it requires too much work and messing with various APIs
> for correct routing/failover.
>
> What about automatic partition reservation for consitency? How could I
> achieve what using only affinity API ?
>
> I think this should be available out of the box, as for affinityRun/Call.
>
> Moreover, I could say the same for affinityRun/Call. One can use affinity
> API and determine node to send a closure. Why bother with additional
> methods ?
>
> And javadoc in AffinityKeyMapped definitely should be fixed, because it
> refers to unexistent things.
>
> 2017-04-19 21:38 GMT+03:00 Dmitriy Setrakyan :
>
> > Alexey,
> >
> > Have you taken a look at the Affinity API in Ignite? It seems that it has
> > all the functionality you may need to map partitions to nodes. You can
> take
> > that info, and use it to route your computations.
> >
> > https://ignite.apache.org/releases/latest/javadoc/org/
> > apache/ignite/cache/affinity/Affinity.html
> >
> > D.
> >
> > On Wed, Apr 19, 2017 at 10:09 AM, Alexei Scherbakov <
> > alexey.scherbak...@gmail.com> wrote:
> >
> > > Guys,
> > >
> > > Currently I'm looking into the problem how to properly deliver
> > computation
> > > to data in most efficient way.
> > >
> > > Basically I need to iterate over all cache partitions on all grid
> nodes,
> > > compute some function on each key-value pair and return aggregated
> result
> > > to a caller.
> > >
> > > This is a job for map-reduce API.
> > >
> > > But it seems where is no possibility to easily manage automatic routing
> > and
> > > failover of compute jobs to data nodes containing specific partitions.
> > >
> > > I found interesting paragraph in javadoc for @AffinityKeyMapped
> > annotation.
> > >
> > > Collocating Computations And Data
> > > It is also possible to route computations to the nodes where the data
> is
> > > cached. This concept is otherwise known as Collocation Of Computations
> > And
> > > Data 
> > >
> > > which makes strong sense for me.
> > >
> > > But in fact this is not working. Instead we only have automatic
> > routing(and
> > > failover) for special cases: affinityCall and affinityRun with explicit
> > > partition. And it seems I can't longer use task sessions for them with
> > > recent changes in Compute API (removed withAsync support)
> > >
> > > I think this is not OK and we should allow jobs to be automatically
> > routed
> > > to data if they have some annotation attached to them specifying
> > partition
> > > and cache names, same as for affinityCall/Run. Probably we should
> > introduce
> > > special task type for such workflows, something like
> AffinityComputeTask,
> > > without explicit mapping phase, for convenient usage.
> > >
> > > I'm willing to make a JIRA ticket for this.
> > >
> > > Thoughs ?
> > >
> > > --
> > >
> > > Best regards,
> > > Alexei Scherbakov
> > >
> >
>
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>

Re: IGNITE-1084 is ready for review

[Semyon Boikov]

Vadim,

I found that real cause of IGNITE-1084 is IGNITE-4760, so suggested test
fix in pull/1828 is not correct. I added new comments in IGNITE-4760,
please take a look.

Thanks,
Semyon

On Wed, Apr 19, 2017 at 12:17 PM, Вадим Опольский 
wrote:

> Hello guys!
>
> The issue IGNITE-1084 https://issues.apache.org/jira/browse/IGNITE-1794 fixed
> originally by Milap Wadhwa.
>
> I reassigned it on myself because she didn't make improvements from Anton
> Vinogradov's comments. Also I just finished another issue with hibernate's
> test.
>
> Review it please - https://github.com/apache/ignite/pull/1828
>
> Vadim Opolski
>

Re: IGNITE-1084 is ready for review

[Semyon Boikov]

Thanks Vadim, I'll do review today.

Semyon

On Wed, Apr 19, 2017 at 12:17 PM, Вадим Опольский 
wrote:

> Hello guys!
>
> The issue IGNITE-1084 https://issues.apache.org/jira/browse/IGNITE-1794 fixed
> originally by Milap Wadhwa.
>
> I reassigned it on myself because she didn't make improvements from Anton
> Vinogradov's comments. Also I just finished another issue with hibernate's
> test.
>
> Review it please - https://github.com/apache/ignite/pull/1828
>
> Vadim Opolski
>

Re: Page Memory behavior with default memory policy

[Semyon Boikov]

I think Ignite can behave like JVM: we can have -Xms -Xmx settings with
defaults depending on available memory.

Thanks

On Tue, Apr 18, 2017 at 4:56 AM, Dmitriy Setrakyan 
wrote:

> Denis,
>
> If what you are suggesting is true, then we can always allocate about 80%
> of available memory by default. By the way, it must also work on Windows,
> so we should definitely test it.
>
> Alexey G, can you comment?
>
> D.
>
> On Mon, Apr 17, 2017 at 6:17 PM, Denis Magda  wrote:
>
> > Dmitriy,
> >
> > > Denis, it sounds like with this approach, in case of the
> over-allocation,
> > > the system will just get slower and slower and users will end up
> blaming
> > > Ignite for it. Am I understanding your suggestion correctly?
> >
> >
> > This will not happen (at least in Unix) unless all the nodes really used
> > all the allocated memory by putting data there or touching all the memory
> > range somehow else.
> >
> > > How was this handled in Ignite 1.9?
> >
> >
> > If you are talking about the legacy off-heap impl then we requested small
> > chunks of data from an operating system rather than a continuous memory
> > region as in the page memory. But I would think of the page memory as of
> > Java heap which also can request 8 GB continuous memory region on a 8 GB
> > machine following heap settings of an app but an operating system will
> not
> > return the whole range immediately unless Java app occupies the whole
> heap
> > or use special parameters.
> >
> > At all, I think it’s safe to use approach suggested by me unless I miss
> > something.
> >
> > —
> > Denis
> >
> > > On Apr 17, 2017, at 6:05 PM, Dmitriy Setrakyan 
> > wrote:
> > >
> > > On Mon, Apr 17, 2017 at 6:00 PM, Denis Magda 
> wrote:
> > >
> > >> Dmitriy,
> > >>
> > >> All the nodes will request its own continuous memory region that takes
> > >> 70-80% of all RAM from an underlying operation system. However, the
> > >> operating system will not outfit the nodes with physical pages mapped
> to
> > >> RAM immediately allowing every node's process to start successfully.
> The
> > >> nodes will communicate to RAM via a virtual memory which in its turn
> > will
> > >> give an access to physical pages whenever is needed applying low level
> > >> eviction and swapping techniques.
> > >>
> > >
> > > Denis, it sounds like with this approach, in case of the
> over-allocation,
> > > the system will just get slower and slower and users will end up
> blaming
> > > Ignite for it. Am I understanding your suggestion correctly?
> > >
> > > How was this handled in Ignite 1.9?
> > >
> > > D.
> >
> >
>

Re: IGNITE - 4760 ready for review

[Semyon Boikov]

Hi Vadim,

I added comments in JIRA. I'll apply the same fix for hibernate5 when fix
is finalized.

Thanks!

On Thu, Apr 13, 2017 at 12:10 PM, Вадим Опольский <vaopols...@gmail.com>
wrote:

> Hi, guys!
>
> Semyon, have you had a time to review IGNITE-4760
> https://github.com/apache/ignite/pull/1768 ?
>
> Is this fix actual for hibernate5 module?
>
> Vadim Opolski
>
> 2017-04-11 13:40 GMT+03:00 Semyon Boikov <sboi...@gridgain.com>:
>
>> Thanks Vadim, I'll try to do review today.
>>
>> Semyon
>>
>> On Mon, Apr 10, 2017 at 8:15 PM, Вадим Опольский <vaopols...@gmail.com>
>> wrote:
>>
>>> Hello guys!
>>>
>>> Semyon, review please again. Test check corresponding IgniteCaches
>>> contain expected number of entries. Test fails for
>>> HibernateNonStrictAccessStrategy.
>>> And per-cache thread local in method threadLocalForCache fix this issue.
>>>
>>> https://github.com/apache/ignite/pull/1768/files
>>>
>>> Vadim Opolski
>>>
>>>
>>> 2017-04-07 14:15 GMT+03:00 Semyon Boikov <sboi...@gridgain.com>:
>>>
>>>> Hi Vadim,
>>>>
>>>> Test does not look correct to me. I think test need check that
>>>> corresponding IgniteCaches contain expected number of entries like
>>>> 'testCacheUsage' does.
>>>>
>>>> Thanks
>>>>
>>>> On Wed, Apr 5, 2017 at 3:26 PM, Вадим Опольский <vaopols...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello everybody!
>>>>>
>>>>> Added test. Test fails after session.update(e2forUpdate). This update
>>>>> must put into ENTITY2_NAME region, but it puts into ENTITY1_NAME and
>>>>> ENTITY2_NAME regions.
>>>>>
>>>>> https://github.com/vadopolski/ignite/pull/1
>>>>>
>>>>> Is it true?
>>>>>
>>>>> I have no idea how to change the method threadLocalForCache to
>>>>> support NONSTRICT_READ_WRITE strategy. I tried to change it in accordance
>>>>> with Cameroon Braid report.
>>>>>
>>>>> Vadim Opolski
>>>>>
>>>>>
>>>>> -- Forwarded message --
>>>>> From: Вадим Опольский <vaopols...@gmail.com>
>>>>> Date: 2017-04-03 17:39 GMT+03:00
>>>>> Subject: Re: IGNITE - 4760 : working in hibernate module
>>>>> To: dev@ignite.apache.org
>>>>> Cc: Valentin Kulichenko <valentin.kuliche...@gmail.com>, Semyon
>>>>> Boikov <sboi...@gridgain.com>
>>>>>
>>>>>
>>>>> Hello everyone!
>>>>>
>>>>> I added some change to method threadLocalForCache  and added test
>>>>> testEntityCacheNonStrictFails.
>>>>>
>>>>> How to reproduce situation when updates can be recorded to another
>>>>> region?
>>>>>
>>>>> https://github.com/vadopolski/ignite/blob/5aa25f3830fef14ac5
>>>>> 07ed73872d62b2969a7411/modules/hibernate/src/main/java/org/a
>>>>> pache/ignite/cache/hibernate/HibernateRegionFactory.java
>>>>>
>>>>> https://github.com/vadopolski/ignite/blob/5aa25f3830fef14ac5
>>>>> 07ed73872d62b2969a7411/modules/hibernate/src/test/java/org/a
>>>>> pache/ignite/cache/hibernate/HibernateL2CacheConfigurationSe
>>>>> lfTest.java
>>>>>
>>>>> PullRequest
>>>>> https://github.com/vadopolski/ignite/pull/4/files
>>>>>
>>>>> Vadim
>>>>>
>>>>>
>>>>>
>>>>> 2017-03-27 18:20 GMT+03:00 Denis Magda <dma...@apache.org>:
>>>>>
>>>>>> Vadim,
>>>>>>
>>>>>> What IDE do you use? My recommendation would be to set up everything
>>>>>> let’s say under IntellijIDEA or Eclipse and after that trying to compile
>>>>>> from a terminal.
>>>>>>
>>>>>> This is how you can easily prepare the dev env in IntellijIDEA:
>>>>>> https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup <
>>>>>> https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup>
>>>>>>
>>>>>> —
>>>>>> Denis
>>>>>>
>>>>>> > On Mar 27, 2017, at 7:14 AM, Вадим Опольский <vaopols...@gmai

Re: IGNITE-1794 is ready for review

[Semyon Boikov]

Hi Vadim,

I already fixed all issues with hibernate5 tests and merged in
ignite-3477-master, it will be merged in master soon.

Thanks

On Tue, Apr 11, 2017 at 7:51 PM, Вадим Опольский 
wrote:

> Hello guys!
>
> I added folder hibernate5 as module to project settings and discovered some
> errors. Fixed errors in pull request -
> https://github.com/vopolski/ignite/pull/1/files
> But some tests from hibernate5 is failed. I'll fix them.
>
> How much time do I have?
>
> Vadim Opolski
>
> 2017-04-06 13:04 GMT+03:00 Вадим Опольский :
>
> > Dear sirs!
> >
> > Sorry for incorrect subject.
> >
> > https://github.com/apache/ignite/pull/1643
> >
> > -- Forwarded message --
> > From: Вадим Опольский 
> > Date: 2017-03-24 15:48 GMT+03:00
> > Subject: ready for review IGNITE-933
> > To: dev@ignite.apache.org, Denis Magda , Valentin
> > Kulichenko 
> >
> >
> > Hello everyone!
> >
> > Denis, Valentin, what should I do to close issue
> > https://issues.apache.org/jira/browse/IGNITE-1794 ?
> >
> > The tests in Team City executed successfully.
> >
> > Valentin, I renamed IgniteHibernateTestSuite ->
> IgniteHibernate5TestSuite.
> > Can you create TeamCity Configuration ?
> >
> >
> >
> > Vadim Opolski
> >
> > 2017-03-21 11:30 GMT+03:00 Вадим Опольский :
> >
> >> Hello everybody.
> >>
> >> The issue  https://issues.apache.org/jira/browse/IGNITE-1794 fixed
> >> originally by Mykola Pereyma.
> >> I re-assigned issue https://issues.apache.org/jira/browse/IGNITE-1794
> on
> >> myself, because we haven’t got a note from him for a while.
> >>
> >> Denis, I merged pull request https://github.com/apache/ignite/pull/1146
> with
> >> my fork.
> >> Then I changed hibernate version to 5.2.7 and deleted ByteArrayType.java
> >> as per CI comments.
> >> The tests from list below executed succesfully.
> >>
> >> Prepared new pull request - https://github.com/apache/ignite/pull/1643
> >>
> >> What's the next step ?
> >>
> >> Tests:
> >> HibernateL2CacheConfigurationSelfTest.java
> >> HibernateL2CacheSelfTest.java
> >> HibernateL2CacheTransactionalSelfTest.java
> >> HibernateL2CacheTransactionalUseSyncSelfTest.java
> >> CacheHibernateBlobStoreNodeRestartTest.java
> >> CacheHibernateBlobStoreSelfTest.java
> >> CacheHibernateStoreFactorySelfTest.java
> >> CacheHibernateStoreSessionListenerSelfTest.java
> >>
> >>
> >> Vadim Opolski
> >>
> >>
> >> 2017-02-25 2:38 GMT+03:00 Denis Magda :
> >>
> >>> > Yes, I have some experience with Hibernate. The issue № IGNITE-1974
> is
> >>> interesting for me. How I can assignee it to me? It is assigning with
> >>> Mykola Pereyma now.
> >>> >
> >>> > https://issues.apache.org/jira/browse/IGNITE-1794 <
> >>> https://issues.apache.org/jira/browse/IGNITE-1794>
> >>>
> >>> Just re-assign it on yourself ;) Hope that Mykola is fine with this
> >>> because we haven’t got a note from him for a while.
> >>>
> >>> —
> >>> Denis
> >>>
> >>> > On Feb 24, 2017, at 12:24 PM, Вадим Опольский 
> >>> wrote:
> >>> >
> >>> > Hi Denis,
> >>> >
> >>> > OK, I spotted the problem in the code and I will be try to resolve
> it.
> >>> >
> >>> > https://issues.apache.org/jira/browse/IGNITE-933 <
> >>> https://issues.apache.org/jira/browse/IGNITE-933>
> >>> >
> >>> > Yes, I have some experience with Hibernate. The issue № IGNITE-1974
> is
> >>> interesting for me. How I can assignee it to me? It is assigning with
> >>> Mykola Pereyma now.
> >>> >
> >>> > https://issues.apache.org/jira/browse/IGNITE-1794 <
> >>> https://issues.apache.org/jira/browse/IGNITE-1794>
> >>> >
> >>> > Vadim Opolski
> >>> >
> >>> > 2017-02-24 5:55 GMT+03:00 Denis Magda >> dma...@apache.org>>:
> >>> > Hi Vadim,
> >>> >
> >>> > Yes, this issue might be still relevant. I can’t guide you through
> >>> but, basically, you need to reproduce the issue, spot it in the code
> and
> >>> propose a fix.
> >>> >
> >>> > BTW, do you have any experience with Hibernate? If so, I would be
> >>> amazing if you pick up this ticket reassigning on yourself:
> >>> > https://issues.apache.org/jira/browse/IGNITE-1794 <
> >>> https://issues.apache.org/jira/browse/IGNITE-1794> <
> >>> https://issues.apache.org/jira/browse/IGNITE-1794 <
> >>> https://issues.apache.org/jira/browse/IGNITE-1794>>
> >>> >
> >>> > —
> >>> > Denis
> >>> >
> >>> > > On Feb 23, 2017, at 2:40 AM, Вадим Опольский  >>> > wrote:
> >>> > >
> >>> > > Dear sirs !
> >>> > >
> >>> > > I want to resolve issue IGNITE-933
> >>> > >
> >>> > > https://issues.apache.org/jira/browse/IGNITE-933 <
> >>> https://issues.apache.org/jira/browse/IGNITE-933>
> >>> > >
> >>> > > Is it actual ?
> >>> > >
> >>> > > In which class and method you want me to make changes ?
> >>> > >
> >>> > > Vadim Opolski
> >>> >
> >>> >
> >>>
> >>>
> >>
> >
> >
>

Re: IGNITE - 4760 : added test

[Semyon Boikov]

Hi Vadim,

Test does not look correct to me. I think test need check that
corresponding IgniteCaches contain expected number of entries like
'testCacheUsage' does.

Thanks

On Wed, Apr 5, 2017 at 3:26 PM, Вадим Опольский <vaopols...@gmail.com>
wrote:

> Hello everybody!
>
> Added test. Test fails after session.update(e2forUpdate). This update must
> put into ENTITY2_NAME region, but it puts into ENTITY1_NAME and
> ENTITY2_NAME regions.
>
> https://github.com/vadopolski/ignite/pull/1
>
> Is it true?
>
> I have no idea how to change the method threadLocalForCache to support
> NONSTRICT_READ_WRITE strategy. I tried to change it in accordance with
> Cameroon Braid report.
>
> Vadim Opolski
>
>
> -- Forwarded message --
> From: Вадим Опольский <vaopols...@gmail.com>
> Date: 2017-04-03 17:39 GMT+03:00
> Subject: Re: IGNITE - 4760 : working in hibernate module
> To: dev@ignite.apache.org
> Cc: Valentin Kulichenko <valentin.kuliche...@gmail.com>, Semyon Boikov <
> sboi...@gridgain.com>
>
>
> Hello everyone!
>
> I added some change to method threadLocalForCache  and added test
> testEntityCacheNonStrictFails.
>
> How to reproduce situation when updates can be recorded to another region?
>
> https://github.com/vadopolski/ignite/blob/5aa25f3830fef14ac5
> 07ed73872d62b2969a7411/modules/hibernate/src/main/
> java/org/apache/ignite/cache/hibernate/HibernateRegionFactory.java
>
> https://github.com/vadopolski/ignite/blob/5aa25f3830fef14ac5
> 07ed73872d62b2969a7411/modules/hibernate/src/test/
> java/org/apache/ignite/cache/hibernate/HibernateL2CacheConf
> igurationSelfTest.java
>
> PullRequest
> https://github.com/vadopolski/ignite/pull/4/files
>
> Vadim
>
>
>
> 2017-03-27 18:20 GMT+03:00 Denis Magda <dma...@apache.org>:
>
>> Vadim,
>>
>> What IDE do you use? My recommendation would be to set up everything
>> let’s say under IntellijIDEA or Eclipse and after that trying to compile
>> from a terminal.
>>
>> This is how you can easily prepare the dev env in IntellijIDEA:
>> https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup <
>> https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup>
>>
>> —
>> Denis
>>
>> > On Mar 27, 2017, at 7:14 AM, Вадим Опольский <vaopols...@gmail.com>
>> wrote:
>> >
>> > Valentin, OK.
>> >
>> > To enabled it in my environment I done next:
>> > - built project with command - mvn clean package -DskipTests
>> -Prelease,lgpl
>> > - added folder hibernate to modules in project structure
>> > - added library to dependencies (without it import doesn't working)
>> >
>> > After that I have a lot of error, for instance:
>> > - Class 'AccessStrategy' must either be declared abstract or implement
>> abstract method 'remove(SharedSessionContractImplementor, Object) in
>> 'RegionAccessStrategy'
>> >
>> > generateCacheKey
>> > getCacheKeyId
>> > getRegion
>> > insert
>> > afterInsert
>> > update
>> > afterUpdate
>> > insert
>> > afterInsert
>> > update
>> > get
>> > putFromLoad
>> > lockItem
>> > unlockItem
>> > remove
>> >
>> > Do anybody know the easier way to resolve this issue?
>> >
>> > Also tried to reimport all maven projects and cleansed repository in
>> .m2.
>> > Vadim Opolski
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > 2017-03-25 2:42 GMT+03:00 Valentin Kulichenko <
>> valentin.kuliche...@gmail.com <mailto:valentin.kuliche...@gmail.com>>:
>> > Vadim,
>> >
>> > ignite-hibernate module is a part of 'lgpl' profile. Apparently it's not
>> > enabled in your environment.
>> >
>> > -Val
>> >
>> > On Fri, Mar 24, 2017 at 4:38 PM, Вадим Опольский <vaopols...@gmail.com
>> <mailto:vaopols...@gmail.com>>
>> > wrote:
>> >
>> > > Hello everybody,
>> > >
>> > > I want to resolve issue №4760
>> > > https://issues.apache.org/jira/browse/IGNITE-4760 <
>> https://issues.apache.org/jira/browse/IGNITE-4760>
>> > >
>> > > To find solution I'm going to change method threadLocalForCache and
>> to add
>> > > Junit test.
>> > >
>> > > Why folder hibernate is not a module ? Can I added it ?
>> > >
>> > > Vadim Opolski
>> > >
>> >
>>
>>
>
>

Re: IgniteConfiguration.gridName is very confusing

[Semyon Boikov]

Thank you Alexander, I merged these changes.

I recommend everybody get latest chages from ignite-2.0.

Thanks!

On Mon, Mar 13, 2017 at 1:23 PM, Alexander Fedotov <
alexander.fedot...@gmail.com> wrote:

> PR updated
>
> On Mon, Mar 13, 2017 at 1:05 PM, Alexander Fedotov <
> alexander.fedot...@gmail.com> wrote:
>
> > Okay. Will do it shortly.
> >
> > On Mon, Mar 13, 2017 at 1:03 PM, Semyon Boikov <sboi...@gridgain.com>
> > wrote:
> >
> >> Alexander,
> >>
> >> I see there are conflicts again, could you plase resolve them, I'm going
> >> to
> >> review and merge these changes today.
> >>
> >> Thanks!
> >>
> >> On Fri, Mar 10, 2017 at 5:50 PM, Yakov Zhdanov <yzhda...@apache.org>
> >> wrote:
> >>
> >> > Thanks, Alex!
> >> >
> >> > Sam, can you please also take a look to make sure we catch all
> possible
> >> > issues on review? Let's merge this on Monday since this is very
> >> > conflict-prone change.
> >> >
> >> > --Yakov
> >> >
> >> > 2017-03-10 12:57 GMT+03:00 Alexander Fedotov <
> >> alexander.fedot...@gmail.com
> >> > >:
> >> >
> >> > > Hi,
> >> > > PR updated. Currently no conflicts at
> >> > > https://github.com/apache/ignite/pull/1435.
> >> > >
> >> > > On Thu, Mar 9, 2017 at 6:50 PM, Alexander Fedotov <
> >> > > alexander.fedot...@gmail.com> wrote:
> >> > >
> >> > > > Sure. Will take a look.
> >> > > >
> >> > > > On Thu, Mar 9, 2017 at 6:05 PM, Yakov Zhdanov <
> yzhda...@apache.org>
> >> > > wrote:
> >> > > >
> >> > > >> Alexander,
> >> > > >>
> >> > > >> Page https://github.com/apache/ignite/pull/1435 reports several
> >> > > >> conflicts.
> >> > > >> Can you please check and resolve if necessary. Then resubmit for
> >> > review
> >> > > >> again.
> >> > > >>
> >> > > >> --Yakov
> >> > > >>
> >> > > >> 2017-03-03 13:24 GMT+03:00 Alexander Fedotov <
> >> > > >> alexander.fedot...@gmail.com>:
> >> > > >>
> >> > > >> > Hi, it's ready for review
> >> > > >> > http://reviews.ignite.apache.org/ignite/review/IGNT-CR-81
> >> > > >> >
> >> > > >> > On Fri, Mar 3, 2017 at 11:39 AM, Yakov Zhdanov <
> >> yzhda...@apache.org
> >> > >
> >> > > >> > wrote:
> >> > > >> >
> >> > > >> > > Guys, I want to bring this up. What is the status of this
> >> ticket
> >> > and
> >> > > >> > > further steps?
> >> > > >> > >
> >> > > >> > > --Yakov
> >> > > >> > >
> >> > > >> > > 2017-01-30 16:37 GMT+03:00 Alexander Fedotov <
> >> > > >> > alexander.fedot...@gmail.com
> >> > > >> > > >:
> >> > > >> > >
> >> > > >> > > > Done. But it looks like something went wrong since Upsource
> >> > > reports:
> >> > > >> > > > "Review has too many files (1244), aborting".
> >> > > >> > > >
> >> > > >> > > > Also guys, I believe we need to merge this change in short
> >> time
> >> > > >> because
> >> > > >> > > > it's targeted for 2.0 and chances for a conflict are high.
> >> > > >> > > >
> >> > > >> > > >
> >> > > >> > > >
> >> > > >> > > > On Mon, Jan 30, 2017 at 4:16 PM, Pavel Tupitsyn <
> >> > > >> ptupit...@apache.org>
> >> > > >> > > > wrote:
> >> > > >> > > >
> >> > > >> > > > > Alexander,
> >> > > >> > > > >
> >> > > >> > > > > Please name the review appropriately and link it in the
> >> ticket
> >> > > as
> >> > > >> > > > > described:
> >> > > >

Re: IgniteConfiguration.gridName is very confusing

[Semyon Boikov]

Alexander,

I see there are conflicts again, could you plase resolve them, I'm going to
review and merge these changes today.

Thanks!

On Fri, Mar 10, 2017 at 5:50 PM, Yakov Zhdanov  wrote:

> Thanks, Alex!
>
> Sam, can you please also take a look to make sure we catch all possible
> issues on review? Let's merge this on Monday since this is very
> conflict-prone change.
>
> --Yakov
>
> 2017-03-10 12:57 GMT+03:00 Alexander Fedotov  >:
>
> > Hi,
> > PR updated. Currently no conflicts at
> > https://github.com/apache/ignite/pull/1435.
> >
> > On Thu, Mar 9, 2017 at 6:50 PM, Alexander Fedotov <
> > alexander.fedot...@gmail.com> wrote:
> >
> > > Sure. Will take a look.
> > >
> > > On Thu, Mar 9, 2017 at 6:05 PM, Yakov Zhdanov 
> > wrote:
> > >
> > >> Alexander,
> > >>
> > >> Page https://github.com/apache/ignite/pull/1435 reports several
> > >> conflicts.
> > >> Can you please check and resolve if necessary. Then resubmit for
> review
> > >> again.
> > >>
> > >> --Yakov
> > >>
> > >> 2017-03-03 13:24 GMT+03:00 Alexander Fedotov <
> > >> alexander.fedot...@gmail.com>:
> > >>
> > >> > Hi, it's ready for review
> > >> > http://reviews.ignite.apache.org/ignite/review/IGNT-CR-81
> > >> >
> > >> > On Fri, Mar 3, 2017 at 11:39 AM, Yakov Zhdanov  >
> > >> > wrote:
> > >> >
> > >> > > Guys, I want to bring this up. What is the status of this ticket
> and
> > >> > > further steps?
> > >> > >
> > >> > > --Yakov
> > >> > >
> > >> > > 2017-01-30 16:37 GMT+03:00 Alexander Fedotov <
> > >> > alexander.fedot...@gmail.com
> > >> > > >:
> > >> > >
> > >> > > > Done. But it looks like something went wrong since Upsource
> > reports:
> > >> > > > "Review has too many files (1244), aborting".
> > >> > > >
> > >> > > > Also guys, I believe we need to merge this change in short time
> > >> because
> > >> > > > it's targeted for 2.0 and chances for a conflict are high.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Mon, Jan 30, 2017 at 4:16 PM, Pavel Tupitsyn <
> > >> ptupit...@apache.org>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Alexander,
> > >> > > > >
> > >> > > > > Please name the review appropriately and link it in the ticket
> > as
> > >> > > > > described:
> > >> > > > > https://cwiki.apache.org/confluence/display/IGNITE/How+
> > >> > > > > to+Contribute#HowtoContribute-ReviewWithUpsource
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > > Pavel
> > >> > > > >
> > >> > > > > On Mon, Jan 30, 2017 at 4:00 PM, Alexander Fedotov <
> > >> > > > > alexander.fedot...@gmail.com> wrote:
> > >> > > > >
> > >> > > > > > Hi,
> > >> > > > > >
> > >> > > > > > Created Upsource review for the subject:
> > >> > > > > > http://reviews.ignite.apache.org/ignite/review/IGNT-CR-81
> > >> > > > > >
> > >> > > > > > On Thu, Jan 19, 2017 at 7:59 PM, Alexander Fedotov <
> > >> > > > > > alexander.fedot...@gmail.com> wrote:
> > >> > > > > >
> > >> > > > > > > Hi,
> > >> > > > > > >
> > >> > > > > > > I've completed working on IGNITE-3207
> > >> > > > > > > https://issues.apache.org/jira/browse/IGNITE-3207
> > >> > > > > > >
> > >> > > > > > > Looks like TC test results don't have problems related to
> my
> > >> > > changes
> > >> > > > > > > http://ci.ignite.apache.org/viewLog.html?buildId=423955;
> > >> > > > > > > tab=buildResultsDiv=IgniteTests_RunAll
> > >> > > > > > >
> > >> > > > > > > Kindly take a look at PR https://github.com/apache/
> > >> > > ignite/pull/1435/
> > >> > > > > > >
> > >> > > > > > > On Thu, Jan 12, 2017 at 1:16 AM, Denis Magda <
> > >> dma...@apache.org>
> > >> > > > > wrote:
> > >> > > > > > >
> > >> > > > > > >> Support Pavel’s point of view.
> > >> > > > > > >>
> > >> > > > > > >> Also Alexander please make sure that your changes are
> > merged
> > >> > into
> > >> > > > > > >> ignite-2.0 branch rather than to the master. I think this
> > >> > > > > functionality
> > >> > > > > > >> has to be available in 2.0 first. Finally, please update
> > 2.0
> > >> > > > Migration
> > >> > > > > > >> Guide once you’ve finished with this task:
> > >> > > > > > >> https://cwiki.apache.org/confluence/display/IGNITE/
> Apache+
> > >> > > > > > >> Ignite+2.0+Migration+Guide <
> https://cwiki.apache.org/conf
> > >> > > > > > >> luence/display/IGNITE/Apache+Ignite+2.0+Migration+Guide>
> > >> > > > > > >>
> > >> > > > > > >> —
> > >> > > > > > >> Denis
> > >> > > > > > >>
> > >> > > > > > >> > On Jan 10, 2017, at 1:58 AM, Pavel Tupitsyn <
> > >> > > ptupit...@apache.org
> > >> > > > >
> > >> > > > > > >> wrote:
> > >> > > > > > >> >
> > >> > > > > > >> > I think we should fix log output as well and replace
> all
> > >> > "grid"
> > >> > > > > > >> occurences
> > >> > > > > > >> > with "instance".
> > >> > > > > > >> >
> > >> > > > > > >> > On Tue, Jan 10, 2017 at 12:55 PM, Alexander Fedotov <
> > >> > > > > > >> > alexander.fedot...@gmail.com> wrote:
> > >> > > > > > >> >
> > >> > > > > > >> >> Hi,
> > >> > > > > > >> >>
> > >> > > 

Re: issue with Hibernate 2L cache region factory ignite-1.8

[Semyon Boikov]

Hi,

Thank you for reporting this bug, but it seems fix you suggested will not
work for HibernateReadWriteAccessStrategy since single thread local is
needed to use single cross-cache transaction.

I created JIRA issue: https://issues.apache.org/jira/browse/IGNITE-4760.

Thanks!

On Sun, Feb 26, 2017 at 12:25 PM, Cameron Braid 
wrote:

> I have just discovered that the ignite hibernate layer 2 cache region
> factory does something weird with tracking changes - the wrong
> regions/caches were being updated  -  I would end up with cache entries for
> other regions in one region.
>
> for example if I say loaded updated a product entity then updated a
> customer entity the product cache could contain the update to the customer.
>
> It looks like someone didn't finish the implementation as there was a note
> in source : /** Map needed to provide the same transaction context for
> different regions. */
>
> So I made the following two lines of change in
> modules/hibernate/src/main/java/org/apache/ignite/cache/hibernate/
> HibernateRegionFactory
>
> Line 102 in 1.8 :
>
> -private final ThreadLocal threadLoc = new ThreadLocal();
> +private ConcurrentMap threadLocMap = new
> ConcurrentHashMap<>();
>
> Line 222 in 1.8 :
>
>  ThreadLocal threadLocalForCache(String cacheName) {
> -return threadLoc;
> +return threadLocMap.computeIfAbsent(cacheName, (k)->new
> ThreadLocal());;
>  }
>
>
> Cameron
>

Re: IGNITE-3727： why did not merge into the 1.8 version?

[Semyon Boikov]

OK, I'll try to review/merge it today.

Thanks!

On Tue, Feb 14, 2017 at 3:20 PM, 李玉珏@163 <18624049...@163.com> wrote:

> Semen,
>
>
> Can this issue be merged into version 1.9?
>
>
> 在 2016/12/13 16:18, Semyon Boikov 写道:
>
>> Hi,
>>
>> We had a lot of more priority issues so I somehow missed IGNITE-3732. As a
>> workaround it is possible to create special thread pool and when message
>> is
>> sent to local node execute message listener there instead of
>> IgniteMessaging usage.
>>
>> Thanks
>>
>> On Sun, Dec 11, 2016 at 5:56 AM, Denis Magda <dma...@apache.org> wrote:
>>
>> Alright, looks like I confused it with some other task initially.
>>>
>>> The ticket you’re talking about is still under review and looks abandoned
>>> https://issues.apache.org/jira/browse/IGNITE-3727 <
>>> https://issues.apache.org/jira/browse/IGNITE-3727>
>>>
>>> Dmitriy G., Sam, is the ticket ready for the merge?
>>>
>>> —
>>> Denis
>>>
>>> On Dec 10, 2016, at 6:29 PM, 李玉珏@163 <18624049...@163.com> wrote:
>>>>
>>>> Denis，
>>>>
>>>> I found  the branch 3727:
>>>>
>>>> https://github.com/gridgain/apache-ignite/tree/ignite-3727-2
>>>>
>>>> But it appear that the code is not merged into the master?
>>>>
>>>> for example, IgniteMessagingImpl.java,the related code is still an old
>>>>
>>> version,8 months ago.
>>>
>>>>
>>>> 在 2016/12/10 22:56, Denis Magda 写道:
>>>>
>>>>> Hi,
>>>>>
>>>>> It was released in 1.8. I’ve just checked git logs.
>>>>>
>>>>> How did you come up with such a conclusion? Something doesn't work on
>>>>>
>>>> your side?
>>>
>>>> —
>>>>> Denis
>>>>>
>>>>> On Dec 10, 2016, at 5:39 AM, 李玉珏@163 <18624049...@163.com> wrote:
>>>>>>
>>>>>> Hi：
>>>>>>
>>>>>>
>>>>>> IGNITE-3727,this issue has been fixed and tested in september，why did
>>>>>>
>>>>> not merge into the 1.8 version?
>>>
>>>>
>>>>>>
>>>
>
>

Re: IGNITE-4492

[Semyon Boikov]

It is closed because I just merged fix (rev
8e12513efb24cc6df1da0968560ac932544ee68d).

Thanks!

On Tue, Feb 14, 2017 at 3:16 PM, ALEKSEY KUZNETSOV <alkuznetsov...@gmail.com
> wrote:

> asfgit <https://github.com/asfgit> has closed my PR for somehow
>
> вт, 14 февр. 2017 г. в 14:46, Semyon Boikov <sboi...@gridgain.com>:
>
> > Thanks Aleksey, I'll review it today.
> >
> > On Tue, Feb 14, 2017 at 2:30 PM, ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> > > wrote:
> >
> > > again, plz review my PR :
> > > https://github.com/apache/ignite/pull/1491
> > >
> > > https://issues.apache.org/jira/browse/IGNITE-4492
> > > --
> > >
> > > *Best Regards,*
> > >
> > > *Kuznetsov Aleksey*
> > >
> >
> --
>
> *Best Regards,*
>
> *Kuznetsov Aleksey*
>

Re: IGNITE-4492

[Semyon Boikov]

Thanks Aleksey, I'll review it today.

On Tue, Feb 14, 2017 at 2:30 PM, ALEKSEY KUZNETSOV  wrote:

> again, plz review my PR :
> https://github.com/apache/ignite/pull/1491
>
> https://issues.apache.org/jira/browse/IGNITE-4492
> --
>
> *Best Regards,*
>
> *Kuznetsov Aleksey*
>

Re: IGNITE-817: [Test] Disabled tests of GridCacheOffHeapTest

[Semyon Boikov]

Hi,

GridCacheOffHeapTest is some very old class and I think it can be removed.
Currently all actual benchmarks are in 'benchmarks' and 'yardstick' modules.

Thanks

On Sat, Feb 11, 2017 at 3:18 AM, Denis Magda  wrote:

> Yakov, Sam?
>
> Please join and share your thoughts.
>
> —
> Denis
>
> > On Feb 10, 2017, at 7:08 AM, Александр Меньшиков 
> wrote:
> >
> > Denis, thank you for information. I see JMH in pom file in benchmarks
> > module. So this must be removed or it's okay for micro-benchmarks?
> >
> > 2017-02-10 0:04 GMT+03:00 Denis Magda :
> >
> >> Alexander,
> >>
> >> We use Yardstick benchmarks for performance measurements of Ignite:
> >> https://github.com/apacheignite/yardstick-ignite  >> apacheignite/yardstick-ignite>
> >>
> >> In 1.9 it will be much easier to execute the benchmarks directly from
> >> Apache Ignite bundle:
> >> http://apache-ignite-developers.2346864.n4.nabble.
> com/IGNITE-4212-Ignite-
> >> Benchmarking-Simplification-and-Automation-td13079.html <
> >> http://apache-ignite-developers.2346864.n4.nabble.
> com/IGNITE-4212-Ignite-
> >> Benchmarking-Simplification-and-Automation-td13079.html>
> >>
> >> As for GridCacheOffHeapTest it looks like a candidate for removal.
> *Yakov,
> >> Sam*, do we still need this test?
> >>
> >>> On Feb 9, 2017, at 6:25 AM, Александр Меньшиков 
> >> wrote:
> >>>
> >>> Hello, everyone.
> >>>
> >>> GridCacheOffHeapTest  is about the Off-heap benchmarks.
> >>>
> >>> As I understand it right now we haven't clear vision how better make
> >>> benchmarks in Ignite.
> >>>
> >>> If it's not true, please make me know.
> >>>
> >>> I think it's a good idea to make benchmarks with JMH framework. It can
> be
> >>> load with maven, right now it's part of OpenJDK project. But it will be
> >>> included in Java 9 like standart framework for benchmarks.
> >>>
> >>> So what do you think, Igniters?
> >>
> >>
>
>

Re: Make async API great again

[Semyon Boikov]

Hi Taras,

Why 'async' methods return ComputeTaskFuture, not just IgniteFuture? It
seems that ComputeTaskFuture is needed only for tasks?




On Fri, Jan 20, 2017 at 5:18 PM, Taras Ledkov  wrote:

> Gents
>
> I've done changes of the IgniteCompute as a subtask of the whole async API
> refactoring (https://issues.apache.org/jira/browse/IGNITE-4580).
> Please check the new version of the public API (
> https://github.com/gridgain/apache-ignite/blob/b81621bf2e8a
> 35b20989f95ff52c0f6d91dd75d6/modules/core/src/main/java/
> org/apache/ignite/IgniteCompute.java)
>
> Please look through the new API and let me know any comments.
>
> --
> Taras Ledkov
> Mail-To: tled...@gridgain.com
>
>

Re: Explicit lock hang

[Semyon Boikov]

I'll take a look.

Thanks

On Sat, Dec 17, 2016 at 2:01 AM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Folks,
>
> Can anyone take a look at this ticket?
> https://issues.apache.org/jira/browse/IGNITE-4450
>
> Looks like pretty serious issue. Most likely it can happen with
> transactions as well, not only explicit locks.
>
> Issue is reported by one of the users:
> http://apache-ignite-users.70518.x6.nabble.com/Cluster-
> hung-after-a-node-killed-td8965.html
>
> -Val
>

Re: IGNITE-3727： why did not merge into the 1.8 version?

[Semyon Boikov]

Hi,

We had a lot of more priority issues so I somehow missed IGNITE-3732. As a
workaround it is possible to create special thread pool and when message is
sent to local node execute message listener there instead of
IgniteMessaging usage.

Thanks

On Sun, Dec 11, 2016 at 5:56 AM, Denis Magda  wrote:

> Alright, looks like I confused it with some other task initially.
>
> The ticket you’re talking about is still under review and looks abandoned
> https://issues.apache.org/jira/browse/IGNITE-3727 <
> https://issues.apache.org/jira/browse/IGNITE-3727>
>
> Dmitriy G., Sam, is the ticket ready for the merge?
>
> —
> Denis
>
> > On Dec 10, 2016, at 6:29 PM, 李玉珏@163 <18624049...@163.com> wrote:
> >
> > Denis，
> >
> > I found  the branch 3727:
> >
> > https://github.com/gridgain/apache-ignite/tree/ignite-3727-2
> >
> > But it appear that the code is not merged into the master?
> >
> > for example, IgniteMessagingImpl.java,the related code is still an old
> version,8 months ago.
> >
> >
> > 在 2016/12/10 22:56, Denis Magda 写道:
> >> Hi,
> >>
> >> It was released in 1.8. I’ve just checked git logs.
> >>
> >> How did you come up with such a conclusion? Something doesn't work on
> your side?
> >>
> >> —
> >> Denis
> >>
> >>> On Dec 10, 2016, at 5:39 AM, 李玉珏@163 <18624049...@163.com> wrote:
> >>>
> >>> Hi：
> >>>
> >>>
> >>> IGNITE-3727,this issue has been fixed and tested in september，why did
> not merge into the 1.8 version?
> >>>
> >>>
> >
>
>

Re: IGNITE-4157 design proposal

[Semyon Boikov]

Hi Sergey,

I have few comments:

- when new node joins cluster all existing nodes provide current data from
'collectDiscoveryData' callback and then this data is sent in
TcpDiscoveryNodeAddedMessage as 'nodeId -> nodeData' map. In case of
'typeId->typeName' mapping it is supposed that when 'collectDiscoveryData'
is called all nodes will have the same mapping, and we should avoid send
duplicated data in TcpDiscoveryNodeAddedMessage (otherwise it can grow too
large in case there are many nodes in topology). Probably for this we will
need to change DiscoverySpiDataExchange interface (actually now we have the
same issue with duplicated caches configurations sent in
TcpDiscoveryNodeAddedMessage, but this can be fixed separately).

- when new mapping is added there can be typeId collision error, make sure
new implementation will also handle such errors

- please take into account that after all server nodes processed custom
discovery event there is no guarantee that all client nodes also processed
this event (discovery events are sent to clients asynchronously)

Thanks,
Semyon

On Thu, Nov 3, 2016 at 5:43 PM, Sergey Chugunov 
wrote:

> Hello everyone,
>
> I've started working on marshaller-related part of IGNITE-4157
>  and think that
> solution
> may look like this:
>
> *Storing mapping locally instead of using cache*
> Instead using marshaller cache for typeId->typeName mapping each node has
> it's own local copy of mapping in, for instance, ConcurrentHashMap.
>
> Each time when a node wants to use new class and therefore to create new
> mapping it notifies other nodes using DiscoveryCustomMessage; every other
> node updates its local mapping upon receiving this notification.
> As DCMs are guaranteed to be delivered to all nodes in the cluster it is
> possible to keep these local mappings in sync across the cluster.
>
> When a node joins the cluster it can obtain current state of mapping using
> existing collectDiscoveryData/onDiscoveryDataReceived methods in
> GridCacheProcessor class.
>
> *Extended mapping API*
> Right now marshaller mapping API is very simple and allows only to register
> or get a single class name under/by a single typeId. This restriction has
> implications on .NET platform and described in IGNITE-2703
> .
> To get rid of this restriction I'm going to add additional byte parameter
> to registerClass and getClass methods indicating for which platform class
> is registered or requested. This should allow platforms other than Java to
> use typeId<->typeName mapping.
>
> If you have any suggestions or concerns about proposed solution please
> reply to this email or comment under corresponding JIRA (IGNITE-4157
> ).
>
> Thanks,
> Sergey.
>

Re: IGNITE-3722 Cached in a file text must be written with UTF-8 charset, not default

[Semyon Boikov]

Hi,

I reviewed and merged your fix. Thanks for contribution!

Thanks!

On Tue, Oct 4, 2016 at 10:06 PM, Saikat Maitra  wrote:

> Hi,
>
> I have raised PR[1] for the following jira ticket[2].
>
> Please review and let me know if any changes required.
>
> Regards,
>
> Saikat
>
> [1] https://github.com/apache/ignite/pull/1144
>
> [2] https://issues.apache.org/jira/browse/IGNITE-3722
>

Re: Deadlock during Client Continuous Query deserialization

[Semyon Boikov]

Here is JIRA issue: https://issues.apache.org/jira/browse/IGNITE-3606

On Fri, Jul 29, 2016 at 5:48 PM, Semyon Boikov <sboi...@gridgain.com> wrote:

> Hi,
>
> I reproduced this issue, thank you for test! After quick debugging It
> seems that this is some problem with Ignite backpressure mechanism in
> communication SPI, I'll create JIRA issue with more details.
>
> As a workaround could you try to disable backpressure:
> tcpCommunicationSpi.setMessageQueueLimit(0);
>
> Thanks!
>
> On Fri, Jul 29, 2016 at 1:15 PM, ross.anderson <ross.ander...@caplin.com>
> wrote:
>
>> Hi, has anyone had a chance to take a look at this?
>> I'd imagine it's a critical issue if a client can cause a cluster to
>> freeze
>> indefinitely based solely on a smallish number of queries running?
>> If there's anything else I can provide to assist please do let me know, as
>> this is a blocker for us using Ignite.
>> Thanks, Ross
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-ignite-users.70518.x6.nabble.com/Deadlock-during-Client-Continuous-Query-deserialization-tp6565p6616.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>

Re: Apache Ignite - New Release

[Semyon Boikov]

Dmitry, all Ignite features are always thoroughly tested =)

I created branch ignite-1.7.0.

On Wed, Jul 27, 2016 at 9:32 AM, Dmitriy Setrakyan <dsetrak...@apache.org>
wrote:

> On Wed, Jul 27, 2016 at 2:29 AM, Semyon Boikov <sboi...@gridgain.com>
> wrote:
>
> > Sure Dmitry, we will run tests with large data set.
> >
> > Regarding 1.7. release: If there are no objections I'm going to cut off
> 1.7
> > branch and start prepare it for release.
> >
>
> In my opinion this is a great addition to Ignite and is definitely worth a
> release, again, assuming that we thoroughly test it.
>
>
> >
> >
> > On Wed, Jul 27, 2016 at 9:15 AM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> > wrote:
> >
> > > On Wed, Jul 27, 2016 at 2:09 AM, Semyon Boikov <sboi...@gridgain.com>
> > > wrote:
> > >
> > > > Regarding distributed join testsing: we added tests verifying correct
> > > join
> > > > behavior and correct execution plan generation for various SQL
> queries,
> > > > tests for joins for various cache types (different number of backups,
> > > > partitioned/replicated), there are tests verifying correct
> distributed
> > > > joins results on changing topology with nodes restarts. Also we added
> > > > benchmarks which will be used to verify that there are no performance
> > > > degradation in this functionality between releases. These benchmarks
> > were
> > > > executed on real clusters, and in next few days we are going to run
> > more
> > > > load tests.
> > > >
> > >
> > > Thanks Semyon, sounds great! I would also test it on larger data sets
> to
> > > see how a join query will take, say, on 10GB of data. Is it possible?
> > >
> > >
> > > >
> > > > On Fri, Jul 22, 2016 at 5:39 PM, Dmitriy Setrakyan <
> > > dsetrak...@apache.org>
> > > > wrote:
> > > >
> > > > > On Fri, Jul 22, 2016 at 7:10 AM, Semyon Boikov <
> sboi...@gridgain.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Today I merged into master 'distributed join' implementation -
> > > > > > https://issues.apache.org/jira/browse/IGNITE-1232 (thanks to
> > Sergi,
> > > he
> > > > > > implemented this feature). I think this together with recent
> > bugfixes
> > > > > worth
> > > > > > 1.7 release. Do you think we can cut off 1.7 release branch from
> > > > master?
> > > > > >
> > > > >
> > > > > Great news. Can you describe that amount of testing we did for this
> > > > > feature?
> > > > >
> > > > >
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > > >
> > > > > > On Thu, Jul 21, 2016 at 7:47 PM, Alexandre Boudnik <
> > > > > > alexander.boud...@gmail.com> wrote:
> > > > > >
> > > > > > > Sorry, I missed the typo:
> > > > > > > To support the point, I would add that PosgreSQL has
> demonstrated
> > > the
> > > > > > > similar behavior (inspired by unix .dot files and ls):
> > > > > > > - they have "hidden" column: xmin and xmax in any table
> > > > > > > - they do not appear in select * list unless they are specified
> > > > > > explicitly
> > > > > > > Take care,
> > > > > > > Alexandre "Sasha" Boudnik
> > > > > > >
> > > > > > > call me via Google Voice:
> > > > > > > 1(405) BUDNIKA
> > > > > > > 1(405) 283-6452
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Jul 21, 2016 at 12:46 PM, Alexandre Boudnik
> > > > > > > <alexander.boud...@gmail.com> wrote:
> > > > > > > > To support the point, I would add that PosgreSQL has
> > demonstrated
> > > > the
> > > > > > > > similar behavior (inspired by unix .dot files and ls):
> > > > > > > > - they have "hidden" column: xmin and xmax in any table
> > > > > > > > - they appear in select * list unless they are specif

Re: Apache Ignite - New Release

[Semyon Boikov]

Regarding distributed join testsing: we added tests verifying correct join
behavior and correct execution plan generation for various SQL queries,
tests for joins for various cache types (different number of backups,
partitioned/replicated), there are tests verifying correct distributed
joins results on changing topology with nodes restarts. Also we added
benchmarks which will be used to verify that there are no performance
degradation in this functionality between releases. These benchmarks were
executed on real clusters, and in next few days we are going to run more
load tests.


On Fri, Jul 22, 2016 at 5:39 PM, Dmitriy Setrakyan <dsetrak...@apache.org>
wrote:

> On Fri, Jul 22, 2016 at 7:10 AM, Semyon Boikov <sboi...@gridgain.com>
> wrote:
>
> > Hi,
> >
> > Today I merged into master 'distributed join' implementation -
> > https://issues.apache.org/jira/browse/IGNITE-1232 (thanks to Sergi, he
> > implemented this feature). I think this together with recent bugfixes
> worth
> > 1.7 release. Do you think we can cut off 1.7 release branch from master?
> >
>
> Great news. Can you describe that amount of testing we did for this
> feature?
>
>
> >
> > Thanks
> >
> >
> > On Thu, Jul 21, 2016 at 7:47 PM, Alexandre Boudnik <
> > alexander.boud...@gmail.com> wrote:
> >
> > > Sorry, I missed the typo:
> > > To support the point, I would add that PosgreSQL has demonstrated the
> > > similar behavior (inspired by unix .dot files and ls):
> > > - they have "hidden" column: xmin and xmax in any table
> > > - they do not appear in select * list unless they are specified
> > explicitly
> > > Take care,
> > > Alexandre "Sasha" Boudnik
> > >
> > > call me via Google Voice:
> > > 1(405) BUDNIKA
> > > 1(405) 283-6452
> > >
> > >
> > >
> > > On Thu, Jul 21, 2016 at 12:46 PM, Alexandre Boudnik
> > > <alexander.boud...@gmail.com> wrote:
> > > > To support the point, I would add that PosgreSQL has demonstrated the
> > > > similar behavior (inspired by unix .dot files and ls):
> > > > - they have "hidden" column: xmin and xmax in any table
> > > > - they appear in select * list unless they are specified explicitly
> > > >
> > > > I'll add some notices to the ticket
> > > > Take care,
> > > > Alexandre "Sasha" Boudnik
> > > >
> > > > call me via Google Voice:
> > > > 1(405) BUDNIKA
> > > > 1(405) 283-6452
> > > >
> > > >
> > > >
> > > > On Fri, Jul 15, 2016 at 6:37 PM, Valentin Kulichenko
> > > > <valentin.kuliche...@gmail.com> wrote:
> > > >> Agree.
> > > >>
> > > >> On Fri, Jul 15, 2016 at 12:59 PM, Alexey Goncharuk <
> > > >> alexey.goncha...@gmail.com> wrote:
> > > >>
> > > >>> Looks like the ticket for removing _key and _value from selct * is
> a
> > > good
> > > >>> candidate for 2.0.
> > > >>>
> > > >>> 2016-07-15 5:12 GMT-07:00 Sergi Vladykin <sergi.vlady...@gmail.com
> >:
> > > >>>
> > > >>> > We will not be able to just change this, because it will brake
> > > >>> > compatibility. Still I believe that an option to define our SQL
> > > tables
> > > >>> > without _key and _value fields. But this is another story you can
> > > file a
> > > >>> > ticket for it, can we fix somehow our JDBC for now? Like
> returning
> > > >>> > BinaryObject instance or something?
> > > >>> >
> > > >>> > Sergi
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > On Fri, Jul 15, 2016 at 2:48 AM, Valentin Kulichenko <
> > > >>> > valentin.kuliche...@gmail.com> wrote:
> > > >>> >
> > > >>> > > IGNITE-3466 is not a JDBC-only issue. This happens because
> > 'select
> > > *'
> > > >>> > query
> > > >>> > > returns all the fields including _kay and _val which are
> created
> > by
> > > >>> > Ignite,
> > > >>> > > not by user. This is actually a usability issue that pops up
> > every
> > > now
> > > >>> > and
> > > >>> > > then. This is very counterint

Re: "ArrayIndexOutOfBoundsException" happened when doing qurey in version 1.6.0

[Semyon Boikov]

Yes, I'm working on it, updated ticket.

On Tue, Jul 26, 2016 at 5:06 PM, Dmitriy Setrakyan <dsetrak...@apache.org>
wrote:

> On Tue, Jul 26, 2016 at 1:43 AM, Semyon Boikov <sboi...@gridgain.com>
> wrote:
>
> > We already have similar issue reproduced in our benchmarks -
> > https://issues.apache.org/jira/browse/IGNITE-3300. I think this is
> > related to optimziation done in 1.6 to store key partition in the cache
> > key. I believe fix will be included in 1.7 release.
> >
>
> This issue has no updates in Jira. Semyon, it is assigned to you. Are you
> working on it?
>
>
> >
> > On Tue, Jul 26, 2016 at 2:13 AM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> >> Crossposting to dev@
> >>
> >> Folks,
> >>
> >> I noticed the exception below in couple of user threads already. This
> >> actually means that partition number for the key is -1, which doesn't
> make
> >> much sense to me. Does anyone has any ideas?
> >>
> >> Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
> >>  at java.util.ArrayList.elementData(ArrayList.java:400)
> >>  at java.util.ArrayList.get(ArrayList.java:413)
> >>  at
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.get(GridAffinityAssignment.java:152)
> >>  at
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.nodes(GridAffinityAssignmentCache.java:387)
> >>  at
> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.nodes(GridCacheAffinityManager.java:251)
> >>  at
> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primary(GridCacheAffinityManager.java:287)
> >>  at
> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primary(GridCacheAffinityManager.java:278)
> >>  at
> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primary(GridCacheAffinityManager.java:302)
> >>  at
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$7$3.apply(IgniteH2Indexing.java:1730)
> >>  at
> org.apache.ignite.internal.processors.query.h2.opt.GridH2IndexBase$FilteringIterator.accept(GridH2IndexBase.java:251)
> >>  at
> org.apache.ignite.internal.processors.query.h2.opt.GridH2IndexBase$FilteringIterator.accept(GridH2IndexBase.java:200)
> >>  at
> org.apache.ignite.internal.util.lang.GridFilteredIterator.hasNext(GridFilteredIterator.java:59)
> >>  at
> org.apache.ignite.internal.processors.query.h2.opt.GridH2Cursor.next(GridH2Cursor.java:59)
> >>  at org.h2.index.IndexCursor.next(IndexCursor.java:274)
> >>  at org.h2.table.TableFilter.next(TableFilter.java:359)
> >>  at org.h2.command.dml.Select.queryFlat(Select.java:527)
> >>  at org.h2.command.dml.Select.queryWithoutCache(Select.java:632)
> >>  at org.h2.command.dml.Query.query(Query.java:297)
> >>  at org.h2.command.dml.Query.query(Query.java:284)
> >>  at org.h2.command.dml.Query.query(Query.java:36)
> >>  at org.h2.command.CommandContainer.query(CommandContainer.java:91)
> >>  at org.h2.command.Command.executeQuery(Command.java:196)
> >>  ... 26 more
> >>
> >>
> >> -Val
> >>
> >>
> >> On Mon, Jul 25, 2016 at 5:07 AM, ght230 <ght...@163.com> wrote:
> >>
> >>> ignite-related.xml
> >>> <
> >>>
> http://apache-ignite-users.70518.x6.nabble.com/file/n6516/ignite-related.xml
> >>> >
> >>> Please refer to the attachment.
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> View this message in context:
> >>>
> http://apache-ignite-users.70518.x6.nabble.com/ArrayIndexOutOfBoundsException-happened-when-doing-qurey-in-version-1-6-0-tp6245p6516.html
> >>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
> >>>
> >>
> >>
> >
>

Re: "ArrayIndexOutOfBoundsException" happened when doing qurey in version 1.6.0

[Semyon Boikov]

We already have similar issue reproduced in our benchmarks -
https://issues.apache.org/jira/browse/IGNITE-3300. I think this is related
to optimziation done in 1.6 to store key partition in the cache key. I
believe fix will be included in 1.7 release.

On Tue, Jul 26, 2016 at 2:13 AM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Crossposting to dev@
>
> Folks,
>
> I noticed the exception below in couple of user threads already. This
> actually means that partition number for the key is -1, which doesn't make
> much sense to me. Does anyone has any ideas?
>
> Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
>   at java.util.ArrayList.elementData(ArrayList.java:400)
>   at java.util.ArrayList.get(ArrayList.java:413)
>   at 
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.get(GridAffinityAssignment.java:152)
>   at 
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.nodes(GridAffinityAssignmentCache.java:387)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.nodes(GridCacheAffinityManager.java:251)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primary(GridCacheAffinityManager.java:287)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primary(GridCacheAffinityManager.java:278)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primary(GridCacheAffinityManager.java:302)
>   at 
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$7$3.apply(IgniteH2Indexing.java:1730)
>   at 
> org.apache.ignite.internal.processors.query.h2.opt.GridH2IndexBase$FilteringIterator.accept(GridH2IndexBase.java:251)
>   at 
> org.apache.ignite.internal.processors.query.h2.opt.GridH2IndexBase$FilteringIterator.accept(GridH2IndexBase.java:200)
>   at 
> org.apache.ignite.internal.util.lang.GridFilteredIterator.hasNext(GridFilteredIterator.java:59)
>   at 
> org.apache.ignite.internal.processors.query.h2.opt.GridH2Cursor.next(GridH2Cursor.java:59)
>   at org.h2.index.IndexCursor.next(IndexCursor.java:274)
>   at org.h2.table.TableFilter.next(TableFilter.java:359)
>   at org.h2.command.dml.Select.queryFlat(Select.java:527)
>   at org.h2.command.dml.Select.queryWithoutCache(Select.java:632)
>   at org.h2.command.dml.Query.query(Query.java:297)
>   at org.h2.command.dml.Query.query(Query.java:284)
>   at org.h2.command.dml.Query.query(Query.java:36)
>   at org.h2.command.CommandContainer.query(CommandContainer.java:91)
>   at org.h2.command.Command.executeQuery(Command.java:196)
>   ... 26 more
>
>
> -Val
>
>
> On Mon, Jul 25, 2016 at 5:07 AM, ght230  wrote:
>
>> ignite-related.xml
>> <
>> http://apache-ignite-users.70518.x6.nabble.com/file/n6516/ignite-related.xml
>> >
>> Please refer to the attachment.
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-ignite-users.70518.x6.nabble.com/ArrayIndexOutOfBoundsException-happened-when-doing-qurey-in-version-1-6-0-tp6245p6516.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>

Distributed joins for JDBC

[Semyon Boikov]

Hi,

Last week distributed joins functionality was merged, but one thing was
overlooked. Distributed joins should be explicitly enabled using using
method 'setDistributedJoins' available in java API
(SqlQuery/SqlFieldsQuery). First, this parameter should be also added in
.Net/C++/REST API, this is straightforward. Also there should be
possibility to enable distributed joins for JDBC API. Does it make sense to
add Ignite-specific interface extending standard java.sql.Statement, so
'setDistributedJoins' method can be added there.
JDBC API already have 'unwrap' method to deal with vendor-specific
interfaces, code will look like this:
* IgniteStatement stmt =
connection.createStatement().unwrap(IgniteStatement.class);*
* stmt.setDistributedJoins(true);*
*stmt.executeQuery("...");*

What do you think?

Re: Apache Ignite - New Release

[Semyon Boikov]

Hi,

Today I merged into master 'distributed join' implementation -
https://issues.apache.org/jira/browse/IGNITE-1232 (thanks to Sergi, he
implemented this feature). I think this together with recent bugfixes worth
1.7 release. Do you think we can cut off 1.7 release branch from master?

Thanks


On Thu, Jul 21, 2016 at 7:47 PM, Alexandre Boudnik <
alexander.boud...@gmail.com> wrote:

> Sorry, I missed the typo:
> To support the point, I would add that PosgreSQL has demonstrated the
> similar behavior (inspired by unix .dot files and ls):
> - they have "hidden" column: xmin and xmax in any table
> - they do not appear in select * list unless they are specified explicitly
> Take care,
> Alexandre "Sasha" Boudnik
>
> call me via Google Voice:
> 1(405) BUDNIKA
> 1(405) 283-6452
>
>
>
> On Thu, Jul 21, 2016 at 12:46 PM, Alexandre Boudnik
>  wrote:
> > To support the point, I would add that PosgreSQL has demonstrated the
> > similar behavior (inspired by unix .dot files and ls):
> > - they have "hidden" column: xmin and xmax in any table
> > - they appear in select * list unless they are specified explicitly
> >
> > I'll add some notices to the ticket
> > Take care,
> > Alexandre "Sasha" Boudnik
> >
> > call me via Google Voice:
> > 1(405) BUDNIKA
> > 1(405) 283-6452
> >
> >
> >
> > On Fri, Jul 15, 2016 at 6:37 PM, Valentin Kulichenko
> >  wrote:
> >> Agree.
> >>
> >> On Fri, Jul 15, 2016 at 12:59 PM, Alexey Goncharuk <
> >> alexey.goncha...@gmail.com> wrote:
> >>
> >>> Looks like the ticket for removing _key and _value from selct * is a
> good
> >>> candidate for 2.0.
> >>>
> >>> 2016-07-15 5:12 GMT-07:00 Sergi Vladykin :
> >>>
> >>> > We will not be able to just change this, because it will brake
> >>> > compatibility. Still I believe that an option to define our SQL
> tables
> >>> > without _key and _value fields. But this is another story you can
> file a
> >>> > ticket for it, can we fix somehow our JDBC for now? Like returning
> >>> > BinaryObject instance or something?
> >>> >
> >>> > Sergi
> >>> >
> >>> >
> >>> >
> >>> > On Fri, Jul 15, 2016 at 2:48 AM, Valentin Kulichenko <
> >>> > valentin.kuliche...@gmail.com> wrote:
> >>> >
> >>> > > IGNITE-3466 is not a JDBC-only issue. This happens because 'select
> *'
> >>> > query
> >>> > > returns all the fields including _kay and _val which are created by
> >>> > Ignite,
> >>> > > not by user. This is actually a usability issue that pops up every
> now
> >>> > and
> >>> > > then. This is very counterintuitive that we return the fields that
> user
> >>> > > never defined (unless he explicitly asks for them, of course) and
> that
> >>> > > 'select *' requires class definitions on the client.
> >>> > >
> >>> > > Is it possible to fix this on SQL engine level instead of fixing
> only
> >>> for
> >>> > > JDBC? Sergi, what do you think?
> >>> > >
> >>> > > -Val
> >>> > >
> >>> > > On Thu, Jul 14, 2016 at 3:28 AM, Sergi Vladykin <
> >>> > sergi.vlady...@gmail.com>
> >>> > > wrote:
> >>> > >
> >>> > > > All these issues seem to be related to Jdbc driver rather than
> to SQL
> >>> > > > engine. I think Andrey Gura was the last who worked on it. IMO
> they
> >>> > must
> >>> > > be
> >>> > > > easy to fix.
> >>> > > >
> >>> > > > Sergi
> >>> > > >
> >>> > > > On Thu, Jul 14, 2016 at 9:06 AM, Denis Magda <
> dma...@gridgain.com>
> >>> > > wrote:
> >>> > > >
> >>> > > > > Yakov,
> >>> > > > >
> >>> > > > > I'm not the one who is eligible for review of IGNITE-3389.
> Assigned
> >>> > it
> >>> > > on
> >>> > > > > Andrey Gura. Andrey please find time for review.
> >>> > > > > Alexander B. when you need a review please send an email to
> the dev
> >>> > > list
> >>> > > > > and someone will assist you.
> >>> > > > >
> >>> > > > > As for IGNITE-3466, IGNITE-3467 and 3468 Sergi's opinion is
> needed.
> >>> > > Sergi
> >>> > > > > please have a look.
> >>> > > > >
> >>> > > > > --
> >>> > > > > Denis
> >>> > > > >
> >>> > > > >
> >>> > > > > On Wed, Jul 13, 2016 at 12:13 PM, Yakov Zhdanov <
> >>> yzhda...@apache.org
> >>> > >
> >>> > > > > wrote:
> >>> > > > >
> >>> > > > > > Sasha, ignite-3389 is in resolved state and I suppose is
> ready to
> >>> > be
> >>> > > > > > reviewed and merged. Denis, can you please do it? Make sure
> to
> >>> > check
> >>> > > TC
> >>> > > > > :)
> >>> > > > > >
> >>> > > > > > As far as Ignite-3466..3468, Igor, can you please provide
> >>> feedback
> >>> > to
> >>> > > > the
> >>> > > > > > issues and tell us if we can fit them as well.
> >>> > > > > >
> >>> > > > > > --Yakov
> >>> > > > > >
> >>> > > > > > 2016-07-12 23:33 GMT+03:00 Alexandre Boudnik <
> >>> > > > > alexander.boud...@gmail.com
> >>> > > > > > >:
> >>> > > > > >
> >>> > > > > > > Yakov,
> >>> > > > > > >
> >>> > > > > > > Is it possible to include several probably easy-to-fix
> bugs and
> >>> > > > > > > improvements; they are very annoying and they decrease the
> >>> value
> >>> > of
> >>> > > 

Re: New contributor: Support primitive type names in QueryEntity (IGNITE-3399)

[Semyon Boikov]

Hi,

I added you in the contributors list.

On Thu, Jul 21, 2016 at 1:54 PM, NoTrueScotsman 
wrote:

> Hi all,
>
> I'd like to attempt a fix for IGNITE-3399 (currently unclaimed in
> Jira) as my newbie contribution.
>
> Could you add me to the list of Ignite contributors?
> Jira username: thehoff
>
> Cheers
> Jens
>

Re: Rework "withAsync" in Apache 2.0

[Semyon Boikov]

Another issue which usually confuses users is Ignite 'implementation
details' of asynchronous execution: it operation is local it can be
executed from calling thread (for example, if 'async put' is executed in
atomic cache from primary node then cache store will be updated from
calling thread). Does it make sense to fix this as well?


On Thu, Jul 21, 2016 at 10:55 AM, Yakov Zhdanov  wrote:

> Agree with Alex. Vova, please go on with issues taking Alex's comments into
> consideration.
>
> Thanks!
>
> --Yakov
>
> 2016-07-21 10:43 GMT+03:00 Alexey Goncharuk :
>
> > Big +1 on this in general.
> >
> > I would also relax our guarantees on operations submitted from the same
> > thread. Currently we guarantee that sequential invocations of async
> > operations happen in the same order. I think that if a user wants such
> > guarantees, he must define these dependencies explicitly by calling
> chain()
> > on returning futures.
> >
> > This change will significantly improve cache operations performance in
> > async mode.
> >
> > 3) Sync operations normally* should not* be implemented through async.
> This
> > > is a long story - if we delegate to async, then we have to bother with
> > > additional threads, associated back-pressure control and all that crap.
> > > Sync call must be sync unless there is a very strong reason to go
> through
> > > async path.
> > >
> > Not sure about this, though. In most cases a cache operation implies
> > request/response over the network, so I think we should have explicit
> > synchronous counterparts only for methods that are guaranteed to be
> local.
> >
>

Re: ignite-2310

[Semyon Boikov]

I think we should detect such situation and throw exception. As I remember
for cross cache qieries we throw exception if caches have different
partitions distribution.

On Wed, Jul 6, 2016 at 3:14 PM, Yakov Zhdanov  wrote:

> Guys, this does not work in general case. If you provide more than one
> partition you can end up with a situation when they reside on more than one
> node.
>
> --Yakov
>
> 2016-07-06 13:50 GMT+03:00 Vladimir Ozerov :
>
> > If we add "partsToLock" to job execute request, then why we allow it only
> > for "affinity" methods? We can resort to "with" semantics instead:
> >
> > IgniteCompute.*withPartitionsToLock*(...).affinityRun();
> >
> > On Wed, Jul 6, 2016 at 12:23 PM, Taras Ledkov 
> > wrote:
> >
> > > Igniters,
> > >
> > > Lets discuss the changes of public API at the IgniteCompute.
> > > The new methods affinityRun & affinityCall is added by working on
> > > IGNITE-2310.
> > >
> > > https://issues.apache.org/jira/browse/IGNITE-2310
> > >
> > > Please take a look at the signature of the new methods:
> > >
> > >
> > >
> >
> https://github.com/gridgain/apache-ignite/commit/991fb60f563ee1630152ca0159d04b4969f883bf#diff-b276b8e6e14915f9e5f2f5daeeddec8a
> > >
> > > void affinityRun(@Nullable String cacheName, Object affKey,
> > IgniteRunnable
> > > job, Map partsToLock)
> > >
> > > The parameter Map partsToLock is added.
> > > Map contains the pairs of the cache name and array of partitions that
> > must
> > > be reserved on the target node before job execution.
> > >
> > > Dmitry, colleagues, please comment or approve.
> > >
> > > --
> > > Taras Ledkov
> > > Mail-To: tled...@gridgain.com
> > >
> > >
> >
>

Remaining distributed joins issues

[Semyon Boikov]

Hi,

We get back to 'distributed join' feature. I resurrected branch
'ignite-1232', did brief review of changes, added more tests and analysed
some failures. Here are issues identified so far, *Sergi*, could you please
comment:

- it seems current code does not properly handle case when custom
AffinityKeyMapper is set. As I understand when custom AffinityKeyMapper is
used, then it is not possible to reason about affinity key field?

- incorrect result when join partitioned and replicated caches. Schema is:
Account -> Person -> Organization. Person is REPLICATED cache.
Plan for this join query 'from Organization o, Person p, Account a where
p.orgId = o._key and p._key = a.personId':
SELECT
O.NAME AS __C0,
P._KEY AS __C1,
P.NAME AS __C2,
A.NAME AS __C3
FROM "org".ORGANIZATIONO
/* "org".ORGANIZATION.__SCAN_ */
INNER JOIN "person".PERSON P
/* *batched:broadcast* "person".PERSON.__SCAN_ */
ON 1=1
/* WHERE P.ORGID = O._KEY
*/
INNER JOIN "acc".ACCOUNT A
/* batched:broadcast "acc".ACCOUNT.__SCAN_ */
ON 1=1
WHERE (P.ORGID = O._KEY)
AND (P._KEY = A.PERSONID)

Look like here distributed join is erroneously used for
'ORGANIZATION/PERSON' join, regual local join can be used since PERSON in
replicated cache. As I understand DistributedLookupBatch should never be
used to get data from index on REPLICATED cache? I added minimal test to
reproduce this: IgniteCacheJoinPartitionedAndReplicatedTest.

- I noticed that a lot of data can be transferred when join condition does
not use index. For example there are Person[ogrName] and Organization[name]
caches (there are no indexes on orgName, name), try join 'Organization o,
Person p where o.name=p.name'.If on some node there are 5 Organizations
then DistributedLookupBatch.addSearchRows(SearchRow firstRow, SearchRow
lastRow) will be called 5 times with parameters firstRow=null, lastRow=null
(which means 'get all rows from index'). As result GridH2IndexRangeRequest
created by this DistributedLookupBatch will contain 5 identical
GridH2RowRangeBounds[first=null, last=null] and related
GridH2IndexRangeResponse will contain 5 identical GridH2RowRanges
containing all index rows. It seems this should be somehow optimized?
I added test with such model and query - IgniteCacheJoinNoIndexTest (test
pass, but can be used to debug query requests/response).

- another observation related to
DistributedLookupBatch.addSearchRows(SearchRow firstRow, SearchRow
lastRow): it is possible that this method will be called multiple times
with the same parameters (for example when 'join Person, Organization where
p.ordId=o.id', then 'addSearchRows' can be called for the same 'orgId'
multiple times). Does it make sense to try optimize this?

- our regular sql queries benchmarks show performance drop ~6%, I'll try to
investigate

Thanks!

Re: Maven conflicts within ignite-core from duplicated classes

[Semyon Boikov]

As far as I know the only reason sources of Nullable are in Ignite - we did
not want to have any dependecy for 'core' module.

On Mon, Jun 13, 2016 at 10:04 AM, Denis Magda  wrote:

> Hi Pete,
>
> CC-ing Ignite dev list to the thread as well.
>
> Igniters, up to today is there any reason why we need to include sources
> of Nullable JetBrain’s annotation in Ignite sources?
>
> As I see we can rather import that latest version from the maven repository
> https://maven-repository.com/artifact/org.jetbrains/annotations/15.0
>
> *Anton, *please take a look at Pete issue and suggest a workaround and
> generic solution overall.
>
> —
> Denis
>
> On Jun 9, 2016, at 1:32 PM, Pete Campton  wrote:
>
> Hello,
>
> I've come across an issue where there are a couple of classes duplicated
> from org.jetbrains.annotations within the ignite-core module which conflict
> with other part of our code base.
>
> As these files are duplicated rather than being referenced as a maven
> dependency I am unable to exclude them or reference a specific version of
> the org.jetbrains.annotations artifact.
>
> Specifically, the included version cannot be used to annotate types
> ignite-core-1.6.0.jar:org.jetbrains.annotations.Nullable ...
> @Target({ElementType.METHOD, ElementType.FIELD, ElementType.PARAMETER,
> ElementType.LOCAL_VARIABLE})
> annotations-15.0.jar:org.jetbrains.annotations.Nullable ...
> @Target({ElementType.METHOD, ElementType.FIELD, ElementType.PARAMETER,
> ElementType.LOCAL_VARIABLE, ElementType.TYPE_USE})
>
> As far as I know there isn't an easy way of excluding these files from
> within the ignite-core module. Is there are reason why these "external"
> files have been duplicated within the ignite code base rather than being
> referenced as a maven dependency? Or alternatively, has anyone come across
> a way of avoiding these conflicts?
>
> Thanks
> Pete
>
>
>

Re: [VOTE] Apache Ignite 1.6.0 RC1

[Semyon Boikov]

+1

On Wed, May 18, 2016 at 5:24 PM, Anton Vinogradov  wrote:

> Dear Sirs!
>
> We have uploaded release candidate to
> https://dist.apache.org/repos/dist/dev/ignite/1.6.0-rc1/
>
> This is very important release containing huge amount of fixes and
> improvements.
> Over 600 issues were closed since previous release.
> 8 new modules were added to the project.
>
> Tag name is
> 1.6.0-rc1
>
> 1.6.0 changes:
> Ignite .NET:
> * Added ability to use Java-based remote filters in continuous queries.
> * Added AtomicSequence and AtomicReference data structures
> * Added automatic Java detection: no need for JAVA_HOME
> * Added java-based continuous query filters
> * Added LINQ Provider for cache SQL queries
> * Added native configuration mechanism (C#, app.config, web.config -
> instead of Spring XML)
> * Added NuGet distribution
> * Binaries are now AnyCPU (instead of separate x64/x86)
> * Java-based services can be called the same way as .NET service are called
>
> Ignite C++:
> * Added Transactions API for C++ client.
> * Added Date and Timestamp types implementation for C++ client.
> * Simplified Autotools build process for C++ client.
>
> Ignite:
> * Added ability to get partition 'updateCntr' with continuous query public
> API.
> * Added asynchronous execution of ContinuousQuery's remote filter and local
> listener.
> * Added backup partitions storing to local store default behavior.
> * Added cache deadlock detection.
> * Added Cache Store implementation backed by Cassandra DB.
> * Added method to get versioned cache entry.
> * Added ODBC driver for Ignite.
> * Added support for join timeout while registering local addresses with IP
> finder in TcpDiscoverySpi.
> * Added support for JTA transactions via synchronization callback.
> * Added Web Console for Ignite.
> * Fixed a bug causing object deserialization when local store is configured
> for cache.
> * Fixed a problem with incorrect classloader picked in OSGI environment.
> * Fixed a race condition when evicted offheap data was not delivered to
> query engine leading to query timeout.
> * Fixed an issue in ContinuousQueries that caused missing of notifications.
> * Fixed background cache partition map exchange not to flood network.
> * Fixed BinaryContext to honor custom loader set through
> IgniteConfiguration.
> * Fixed BinaryObjectOffHeapImpl leak to public code.
> * Fixed cluster stability with 500+ clients.
> * Fixed continuous queries to send filter factory instead of filter.
> * Fixed continuous query deployment in case originating node has left.
> * Fixed issues with continuous query.
> * Fixed deadlock in services deployment when Spring is used.
> * Fixed discovery and node start procedure issues that may cause slowdowns
> during larger topologies start (>100 nodes).
> * Fixed eviction policy notification if swap or off heap is enabled and an
> entry is not loaded during preloading.
> * Fixed excessive thread stack usage in case of high contention on cache
> entries.
> * Fixed execution of Splunk MapReduce jobs on top of Ignite MR.
> * Fixed GridClosureProcessor internal closures to be deserialized by
> BinaryMarshaller.
> * Fixed issue with AWS dependencies.
> * Fixed java proxies deserialization with optimized marshaller.
> * Fixed local store behavior at cross cache transactions.
> * Fixed marshalling of Java collection and maps for BinaryMarshaller.
> * Fixed memory leak in IgniteH2Indexing.
> * Fixed NPE during rebalancing.
> * Fixed NPE in GridMergeIndex.
> * Fixed OOME when OFFHEAP_TIERED mode is used.
> * Fixed potential thread starvation during cache rebalancing.
> * Fixed race condition on load cache on changing topology.
> * Fixed race in marshalling logic that could arise when several nodes are
> running on the same host.
> * Fixed redeployment issues happened with ScanQueries filters.
> * Fixed service proxy not to make remote call for methods declared in
> java.lang.Object.
> * Fixed several ClassNotFoundException in OSGi environment.
> * Fixed SKIP_STORE flag behavior for transaction entry.
> * Fixed support for classes with equal simple name for binary marshaller.
> * Fixed system caches not to use user-defined TransactionConfiguration.
> * Fixed the issue when CacheStore was updated even if EntryProcessor didn't
> update an entry.
> * Fixed thread safety for TcpDiscoveryMulticastIpFinder.
> * Fixed unintentional deserialization of BinaryObjects in OFFHEAP mode with
> peer class loading enabled.
> * Fixed UTF-16 surrogate pairs marshalling.
> * Fixed value copying in entry processor with OptimizedMarshaller.
> * Fixed web session clustering with WebLogic.
> * Hadoop: fixed a bug causing exception during MR planning when input split
> file doesn't exist.
> * IGFS: Added configuration flag to disable default path modes under
> "/ignite" folder.
> * IGFS: Added pluggable factory interface for Hadoop FileSystem creation.
> * IGFS: Fixed file properties when running over secondary file system.
> * IGFS: Fixed issues 

Re: Ignite 1.6 release timelines

[Semyon Boikov]

In 1.6 we implemented 'late affinity assignment' mode (see
https://issues.apache.org/jira/browse/IGNITE-324 and
IgniteConfiguration.isLateAffinityAssignment)

On Mon, May 16, 2016 at 7:03 PM, Alexey Kuznetsov 
wrote:

> https://issues.apache.org/jira/browse/IGNITE-2832
> "CacheJdbcPojoStoreFactory.dataSource property should be replaced with
> Factory"
> Reviewed by Semen and merged into ignite-1.6
>
>
> On Mon, May 16, 2016 at 10:56 PM, Igor Sapego 
> wrote:
>
> > Hi, here is a list of major C++ features included in 1.6:
> >
> > * ODBC driver for Ignite (IGNITE-1786)
> > * Transactions API for CPP client (IGNITE-2805)
> > * Simplified Autotools build process (IGNITE-2823)
> > * Date and Timestamp types implementation (IGNITE-)
> >
> > Best Regards,
> > Igor
> >
> > On Fri, May 13, 2016 at 4:10 PM, Artem Shutak 
> > wrote:
> >
> > > I've finished with IGNITE-2959 Ignite JTA and WebSphere Application
> > Server
> > > 8.5.5  and has been
> > > merged to 1.6.
> > >
> > > Also, I'm going to finish the following issues before 1.6 relase (they
> > all
> > > almost done/under review/waiting for TC):
> > > [1] IGNITE-3056 Service implementation class is required even if it's
> not
> > > expected to be deployed on current node
> > > 
> > > [2] IGNITE-2921 ScanQueries over local partitions are not optimal
> > > 
> > > [3] IGNITE-2899 BinaryObject is deserialized before getting passed to
> > > CacheInterceptor 
> > >
> > > Thanks,
> > > -- Artem --
> > >
> > > On Fri, May 13, 2016 at 5:26 AM, Dmitriy Setrakyan <
> > dsetrak...@apache.org>
> > > wrote:
> > >
> > > > On Thu, May 12, 2016 at 9:08 AM, Alexey Kuznetsov <
> > > akuznet...@gridgain.com
> > > > >
> > > > wrote:
> > > >
> > > > > Also I already implemented "CacheJdbcPojoStoreFactory.dataSource
> > > property
> > > > > should be replaced with Factory"
> > > > > https://issues.apache.org/jira/browse/IGNITE-2832
> > > > >
> > > > > And I think it will be cool to merge into ignite-1.6
> > > > > But I need a review.
> > > > >
> > > > > Val, could you take a look? You may compare branch ignite-2832 with
> > > > master.
> > > > >
> > > >
> > > > To my knowledge, Val is traveling this weekend, so it would be nice
> if
> > > > someone else could pick it up.
> > > >
> > > >
> > > > >
> > > > > On Thu, May 12, 2016 at 10:26 PM, Alexey Kuznetsov <
> > > > > akuznet...@gridgain.com>
> > > > > wrote:
> > > > >
> > > > > > I'm going to merge Ignite Web Console into ignite-1.6
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/IGNITE-843
> > > > > >
> > > > > > On Thu, May 12, 2016 at 9:30 PM, Igor Sapego <
> isap...@gridgain.com
> > >
> > > > > wrote:
> > > > > >
> > > > > >> There is one more C++ ticket I'd like to be included in 1.6 -
> > > > > IGNITE-3113
> > > > > >> [1].
> > > > > >>
> > > > > >> [1] - https://issues.apache.org/jira/browse/IGNITE-3113
> > > > > >>
> > > > > >> Best Regards,
> > > > > >> Igor
> > > > > >>
> > > > > >> On Thu, May 12, 2016 at 3:44 PM, Pavel Tupitsyn <
> > > > ptupit...@gridgain.com
> > > > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >> > And one more .NET ticket: IGNITE-3118 .NET:
> > > > > >> > CacheConfiguration.EvictionPolicy
> > > > > >> >
> > > > > >> > On Thu, May 12, 2016 at 12:20 PM, Denis Magda <
> > > dma...@gridgain.com>
> > > > > >> wrote:
> > > > > >> >
> > > > > >> > > Yakov,
> > > > > >> > >
> > > > > >> > > Processed all the tickets mentioned in 1. The only one is
> left
> > > is
> > > > > the
> > > > > >> > > following
> > > > > >> > > https://issues.apache.org/jira/browse/IGNITE-2954 <
> > > > > >> > > https://issues.apache.org/jira/browse/IGNITE-2954>
> > > > > >> > >
> > > > > >> > > Alex G. I do remember you’ve fixed IGNITE-2954 before. Have
> > you
> > > > > >> merged it
> > > > > >> > > into the master? Is the fix in ignite-1.6 already?
> > > > > >> > >
> > > > > >> > > —
> > > > > >> > > Denis
> > > > > >> > >
> > > > > >> > > > On May 11, 2016, at 8:56 AM, Yakov Zhdanov <
> > > yzhda...@apache.org
> > > > >
> > > > > >> > wrote:
> > > > > >> > > >
> > > > > >> > > > Guys,
> > > > > >> > > >
> > > > > >> > > > I moved most of the tickets to 1.7 version.
> > > > > >> > > >
> > > > > >> > > > I had to leave the following tickets untouched (~80
> > tickets):
> > > > > >> > > > 1. labeled with - community important performance
> customer -
> > > > > Denis,
> > > > > >> > > > Vladimir and Val can you please review these tickets and
> > move
> > > > > >> > non-urgent
> > > > > >> > > > ones. You can use query below
> > > > > >> > > > 2. with status "in progress". Please everyone having such
> > > > tickets
> > > > > >> move
> > > > > >> > it
> > > > > >> > > > yourself or finish it in 1.6
> > > > > >> > > > 3. with status "patch 

Re: Cleaning internal thread locals

[Semyon Boikov]

As I understand in many palces thread locals can be used for performance
reasons, and it is not good idea to always clear it.

Initial problem in IGNITE-967 was about tomcat, since tomcat provides
solution for this problem do we really need fix something in Ignite?

Valentin, you created this ticket, what do you think?

On Wed, Apr 6, 2016 at 11:59 AM, Alexei Scherbakov <
alexey.scherbak...@gmail.com> wrote:

> It is a very good idea to try to fix internal thread local cleaning in all
> places.
> On example, it can be easily done in class GridSpinReadWriteLock
> Just replace code in line 172 with
> readLockEntryCnt.remove();
> and add line 388 with:
> if (update == 0) readLockEntryCnt.remove();
> ( needs more testing possibly )
>
> I've provided you with the list of offending places ( which may not be
> completed )
> But it is definitely a good place to start.
> Who will be responsible for fixiing it ?
>
>
> 2016-04-06 11:50 GMT+03:00 Denis Magda :
>
> > As far as I understand Alexei, the issue here is that Ignite kernal
> > doesn't have list of all the threads that accessed it during its
> lifetime.
> > So the kernal won't be able to iterate over all the threads (like threads
> > from Tomcat thread pool) and perform all the cleaning. This is the issue.
> >
> > --
> > Denis
> >
> >
> > On 4/6/2016 11:47 AM, Artem Shutak wrote:
> >
> >> Alexei,
> >>
> >> I would follow the Valentin's suggestion in the issue description and
> >> would
> >> add to Ignite tests a check on thread locals after stopping all grids.
> And
> >> then we will be able to fix thread local cleaning in all places.
> >>
> >> Thanks,
> >> -- Artem --
> >>
> >> On Wed, Apr 6, 2016 at 11:18 AM, Alexei Scherbakov <
> >> alexey.scherbak...@gmail.com> wrote:
> >>
> >> Valentin,
> >>>
> >>> I did test on demo web app where HttpSession is backed by Ignite cache
> >>> and
> >>> collect some logs.
> >>> There are many places in Ignite where thread locals are used.
> >>> Then cache methods are accessed in threads from servlet container pool,
> >>> we
> >>> get leaks.
> >>> I don't think it will be easy to add thread locals cleaning in all
> these
> >>> places without major code rewrite, but, for sure, it would be ideal
> >>> solution.
> >>>
> >>>
> >>>  05, 2016 4:21:52 PM
> >>> org.apache.catalina.loader.WebappClassLoaderBase
> >>> checkThreadLocalMapForLeaks
> >>> SEVERE: The web application [/app1] created a ThreadLocal with key of
> >>> type
> >>> [org.apache.ignite.internal.util.GridSpinReadWriteLock$1] (value
> >>> [org.apache.ignite.internal.util.GridSpinReadWriteLock$1@5fecc35f])
> and
> >>> a
> >>> value of type [java.lang.Integer] (value [0]) but failed to remove it
> >>> when
> >>> the web application was stopped. Threads are going to be renewed over
> >>> time
> >>> to try and avoid a probable memory leak.
> >>>  05, 2016 4:21:52 PM
> >>> org.apache.catalina.loader.WebappClassLoaderBase
> >>> checkThreadLocalMapForLeaks
> >>> SEVERE: The web application [/app1] created a ThreadLocal with key of
> >>> type
> >>> [org.apache.ignite.internal.util.tostring.GridToStringBuilder$1] (value
> >>>
> [org.apache.ignite.internal.util.tostring.GridToStringBuilder$1@4ecdfc04
> >>> ])
> >>> and a value of type [java.util.LinkedList] (value
> >>>
> >>>
> [[org.apache.ignite.internal.util.tostring.GridToStringThreadLocal@684a2791
> >>> ]])
> >>> but failed to remove it when the web application was stopped. Threads
> are
> >>> going to be renewed over time to try and avoid a probable memory leak.
> >>>  05, 2016 4:21:52 PM
> >>> org.apache.catalina.loader.WebappClassLoaderBase
> >>> checkThreadLocalMapForLeaks
> >>> SEVERE: The web application [/app1] created a ThreadLocal with key of
> >>> type
> >>> [org.apache.ignite.internal.binary.BinaryThreadLocalContext$1] (value
> >>> [org.apache.ignite.internal.binary.BinaryThreadLocalContext$1@4e9300e7
> ])
> >>> and a value of type
> >>> [org.apache.ignite.internal.binary.BinaryThreadLocalContext] (value
> >>> [org.apache.ignite.internal.binary.BinaryThreadLocalContext@7934a88])
> >>> but
> >>> failed to remove it when the web application was stopped. Threads are
> >>> going
> >>> to be renewed over time to try and avoid a probable memory leak.
> >>>  05, 2016 4:21:52 PM
> >>> org.apache.catalina.loader.WebappClassLoaderBase
> >>> checkThreadLocalMapForLeaks
> >>> SEVERE: The web application [/app1] created a ThreadLocal with key of
> >>> type
> >>> [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@56502b72]) and a
> >>> value of type
> >>> [org.apache.ignite.internal.binary.streams.BinaryMemoryAllocatorChunk]
> >>> (value
> >>>
> >>>
> >>>
> [org.apache.ignite.internal.binary.streams.BinaryMemoryAllocatorChunk@4eab9d2f
> >>> ])
> >>> but failed to remove it when the web application was stopped. Threads
> are
> >>> going to be renewed over time to try and avoid a probable memory leak.
> >>>  05, 2016 4:21:52 PM
> >>> org.apache.catalina.loader.WebappClassLoaderBase
> >>> 

Re: Deprecate GridFunc

[Semyon Boikov]

+1

Also need to check these classes with lots of static utility methods:
IgniteUtils and X.

On Fri, Apr 1, 2016 at 11:03 AM, Vladimir Ozerov 
wrote:

> Igniters,
>
> This is about our infamous *GridFunc *class. I suggest to deprecate it and
> strongly discourage any usage of any method from it.
>
> *Motivation:*
> Currently this class contains ~170 methods. Lots of this methods have poor
> performance characteristics or broken semantics.
> - Lots of collection manipulation methods are broken. You can easily loose
> O(1) or O(log N) and have O(N). You can easily loose Set semantics - merge
> two sets and this is not a set anymore. Etc, etc.. This is a nightmare. +
> you allocate garbage.
> - Some methods operate on varags while we always need no more than one
> element -> unnecessary allocations.
>
> But as these methods are convenient to use, developers tend to use them
> without understanding of implications. As our product is
> performance-sensitive, we certainly should avoid this.
>
> For example, here is a code snippet from cache IO manager:
>
> if (!F.exist(F.nodeIds(nodes), F0.not(F.contains(leftIds {
>
> ...
>
> }
>
>
> And how do you think, what is "nodes" variable? Here it is:
>
> F.view(F.viewReadOnly(ids, U.id2Node(ctx), p), F.notNull())
>
>
> WTF? We *MUST* stop that.
>
> *Proposal:*
> 1) Deprecate this class.
> 2) Move several useful and safe methods (like F.eq()) to separate
> specialized utility classes.
>
> Thoughts?
>
> Vladimir.
>

Re: API for asynchronous execution.

[Semyon Boikov]

Andrey,

I agree that current situation with threading in Ignite is very
inconvenient when user callbacks execute some non-trivial code. But
changing this to async dispatch is huge refactoring, even changing this
just for continuous queries callback is not so easy task.

We can start with https://issues.apache.org/jira/browse/IGNITE-2004, and if
more users complains arise we can think about changing others parts of
system.

For now we need decisions for these points:
- how to specify that callback should be run asynchronously (Nikolay
suggested marker interface IgniteAsyncCallback, or @IgniteAsyncCallback)
- where these callbacks are executed, AFAIK Nikolay added special pool
which is configured in IgniteConfiguration (something like
IgniteConfiguration.asyncCallbackThreadPoolSize)

Regards


On Tue, Mar 29, 2016 at 10:45 PM, Andrey Kornev 
wrote:

> Vladimir, Igniters
>
> Here are my 2 cents.
>
> The current situation with threading when it comes to executing user
> callbacks -- the CQ filters (either local or remote), the CQ listeners, the
> event listeners, the messaging listeners, the entry processors (did I miss
> anything?) -- is pretty sad. The callbacks may get executed on a system
> pool's thread, public pool's, utility pool's, discovery worker thread,
> application thread, to name a few. It causes a lot of grief and suffering,
> hard-to-fix races, dead locks and other bugs.
>
> I guess it's always possible to come up with a more or less reasonable
> explanation to such predicament (which usually boils down to "It is so
> because this is how it's implemented"), but I, as a user, could not care
> less. I want consistency. I want all my callbacks (including Entry
> Processors!) to be executed on the public pool's threads, to be precise.
> This is not the first time I complain about this, and I really think it's
> time to fix this mess.
>
> For a good example of how to implement ordered async dispatch of callbacks
> on large scale, one only needs to look at Akka (or Reactor
> https://github.com/reactor/reactor).  Coherence also managed to get it
> right (in my opinion, that is).
>
> Regards
> Andrey
>
>

Re: Transformers in SCAN queries

[Semyon Boikov]

I think scan query implementation can be more complex than just sending
closures to all nodes. e.g. it should handle topology changes. IMO it is
not good idea to use compute instead of queries.

On Thu, Feb 4, 2016 at 10:55 AM, Dmitriy Setrakyan 
wrote:

> On Wed, Feb 3, 2016 at 10:28 PM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Dmitry,
> >
> > The main difference in my view is that you lose pagination when sending
> > results from servers to client. What if one wants to iterate through all
> > entries in cache?
> >
>
> I see. Perhaps we should fix the pagination for compute instead of adding
> transformers for queries?
>
>
> >
> > On Wed, Feb 3, 2016 at 9:47 PM, Dmitriy Setrakyan  >
> > wrote:
> >
> > > Valentin,
> > >
> > > Wouldn’t the same effect be achieved by broadcasting a closure to the
> > > cluster and executing scan-query on every node locally?
> > >
> > > D.
> > >
> > > On Wed, Feb 3, 2016 at 9:17 PM, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com> wrote:
> > >
> > > > Igniters,
> > > >
> > > > I keep getting requests from our users to add optional transformers
> to
> > > SCAN
> > > > queries. This will allow to iterate through cache, but do not
> transfer
> > > > whole key-value pairs across networks (e.g., get only keys). The
> > feature
> > > > looks useful and I created a ticket [1].
> > > >
> > > > I am struggling with the design now. The problem is that I wanted to
> > > extend
> > > > existing ScanQuery object for this, but this seems to be impossible
> > > because
> > > > it already extends Query> and thus can iterate only
> > > > through entries.
> > > >
> > > > The only option I see now is to create a separate query type,
> > copy-paste
> > > > everything from ScanQuery and add *mandatory* transformer. Something
> > like
> > > > this:
> > > >
> > > > ScanTransformQuery extends Query {
> > > > IgniteBiPredicate filter;
> > > > IgniteClosure, R> transformer;
> > > > int part;
> > > > ...
> > > > }
> > > >
> > > > Thoughts? Does anyone has other ideas?
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-2546
> > > >
> > > > -Val
> > > >
> > >
> >
>

Re: hanging locks

[Semyon Boikov]

I fixed this issue with not release locks (
https://issues.apache.org/jira/browse/IGNITE-2008), fix will be available
in 1.5 release.


On Thu, Dec 3, 2015 at 9:28 AM, Dmitriy Setrakyan <dsetrak...@apache.org>
wrote:

> Thanks Semyon!
>
> On Wed, Dec 2, 2015 at 10:13 PM, Semyon Boikov <sboi...@gridgain.com>
> wrote:
>
> > Dmitry, Valentin,
> >
> > I'll look at this issue today.
> >
> > On Thu, Dec 3, 2015 at 6:26 AM, Dmitriy Setrakyan <dsetrak...@apache.org
> >
> > wrote:
> >
> > > Val,
> > >
> > > I think if lock futures are not complete, then the issue remains,
> because
> > > it can cause hanging clients. Did you get an idea of what it would take
> > to
> > > fix it?
> > >
> > > D.
> > >
> > > On Wed, Dec 2, 2015 at 7:25 PM, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com> wrote:
> > >
> > > > I played with this and it seems that locks are actually released when
> > > node
> > > > leaves, but lock futures that are waiting for this lock are never
> > > > completed. Therefore, this issue can be workarounded by replacing
> > lock()
> > > > method with tryLock() in a loop.
> > > >
> > > > Anyone knows why this can happen?
> > > >
> > > > -Val
> > > >
> > > > On Wed, Dec 2, 2015 at 2:30 PM, Dmitriy Setrakyan <
> > dsetrak...@apache.org
> > > >
> > > > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > Does anyone know the severity of this issue? In which cases is it
> > > > > reproduced?
> > > > > https://issues.apache.org/jira/browse/IGNITE-2008
> > > > >
> > > > > In my view, we already have support for it, so it should be just a
> > bug.
> > > > > Anyone knows how hard it is to fix?
> > > > >
> > > > > D.
> > > > >
> > > >
> > >
> >
>

Re: Branch deletion prohibited

[Semyon Boikov]

+1

On Thu, Nov 19, 2015 at 1:07 PM, Vladimir Ozerov 
wrote:

> I agree that there is absolutely no problems of have multiple ways to
> provide contributions.
>
> If you are contributor, you can:
> - Provide a patch;
> - Provide a PR using GitHub mirror.
>
> If you are committer, you can:
> - Provide a patch;
> - Provide a PR using GitHub mirror;
> - Use branch in ASF repo and remove it in the end.
>
> ASF branches removal is temporary restricted by INFRA. As soon as it is
> enabled again why not using it? It is the easiest way to provide
> contributions and review them.
>
> On Thu, Nov 19, 2015 at 12:49 PM, Raul Kripalani  wrote:
>
> > Lads,
> >
> > It is not clear to me whether branch deletion is prohibited ASF-wide
> > (Dmitriy: "we *cannot* delete branches") or by express project request.
> > I've understood both things from the thread. So let's wait for INFRA to
> > clarify: [1].
> >
> > Can someone please explain why we resort to Github in the first place?
> Was
> > it for CI integration purposes?
> >
> > Regards,
> >
> > [1]
> > https://issues.apache.org/jira/servicedesk/agent/INFRA/issue/INFRA-10798
> >
> > *Raúl Kripalani*
> > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
> > Messaging Engineer
> > http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> > http://blog.raulkr.net | twitter: @raulvk
> >
> > On Thu, Nov 19, 2015 at 9:34 AM, Pavel Tupitsyn 
> > wrote:
> >
> > > I'd like to add that there is virtually no difference between using a
> > > branch in original repo and a branch in your personal fork on GitHub.
> > > Merges and other operations work seamlessly between multiple remotes.
> > > Committers don't even have to create PRs (except to run a TC build).
> > >
> > > Thanks,
> > >
> > > On Thu, Nov 19, 2015 at 12:28 PM, Yakov Zhdanov 
> > > wrote:
> > >
> > > > But this leads to tons of garbage in repo and abandoned branches,
> etc.
> > > >
> > > > --Yakov
> > > >
> > > > 2015-11-19 12:19 GMT+03:00 Raul Kripalani :
> > > >
> > > > > As I said: "Pull requests make sense when outsiders want to make
> > > > > contributions."
> > > > >
> > > > > Committers with write access to ASF Git have no reason to develop
> in
> > > > > Github.
> > > > > On 19 Nov 2015 09:13, "Yakov Zhdanov"  wrote:
> > > > >
> > > > > > Disagree. This means none but committer can contribute.
> > > > > >
> > > > > > --Yakov
> > > > > >
> > > > > > 2015-11-19 12:08 GMT+03:00 Raul Kripalani :
> > > > > >
> > > > > > > I disagree.
> > > > > > >
> > > > > > > Code should be in the ASF infra.
> > > > > > >
> > > > > > > Pull requests make sense when outsiders want to make
> > contributions.
> > > > > > >
> > > > > > > The usage of ASF infra is not a mere formality.
> > > > > > >
> > > > > > > Raúl.
> > > > > > > On 19 Nov 2015 08:57, "Yakov Zhdanov" 
> > > wrote:
> > > > > > >
> > > > > > > > Guys, therefore, let's develop new functionality in personal
> > > forks,
> > > > > > test
> > > > > > > on
> > > > > > > > TC with pull requests and then merge to apache git as
> described
> > > > here
> > > > > -
> > > > > > > >
> > > > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
> > > > > > > >
> > > > > > > > We should create new branches only if this is really
> necessary.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > > --
> > > > > > > > Yakov Zhdanov, Director R
> > > > > > > > *GridGain Systems*
> > > > > > > > www.gridgain.com
> > > > > > > >
> > > > > > > > 2015-11-18 23:04 GMT+03:00 Dmitriy Setrakyan <
> > > > dsetrak...@apache.org
> > > > > >:
> > > > > > > >
> > > > > > > > > Raul,
> > > > > > > > >
> > > > > > > > > ASF is currently prohibiting deletion of GIT branches until
> > > > further
> > > > > > > > notice.
> > > > > > > > > Please add your branch to this Wiki page, so we don’t loose
> > > > track:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/Git+branches+to+delete
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > D.
> > > > > > > > >
> > > > > > > > > On Wed, Nov 18, 2015 at 10:16 AM, Raul Kripalani <
> > > > ra...@apache.org
> > > > > >
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Fellows,
> > > > > > > > > >
> > > > > > > > > > I'm trying to push a branch deletion and the ASF Git
> tells
> > me
> > > > > that
> > > > > > > > branch
> > > > > > > > > > deletion is prohibited.
> > > > > > > > > >
> > > > > > > > > > Has someone changed something?
> > > > > > > > > >
> > > > > > > > > > [raul@~/Workbench/Source/ignite$] git push -f origin
> > > > > :ignite-1790
> > > > > > > > > > remote: error: denying ref deletion for
> > > refs/heads/ignite-1790
> > > > > > > > > > To https://git-wip-us.apache.org/repos/asf/ignite
> > > > > > > > > > 

Re: Ignite-1.5 Release

[Semyon Boikov]

Just merged single get optimizations (improvements in the last benchmarks
run: ~10% for atomic-put-get, ~5% for tx-put-get).

Re: Ignite-1.5 Release

[Semyon Boikov]

Hi,

Yesterday I merged optimizations for tx update operations working single
key, got ~7% improvement in tx-put benchmark.

And today I finished to implement optimization for cache 'get' operation
with single key. Got another benchmark results improvements: ~10% with
atomic-put-get, ~5% with tx-put-get. Need to verify TC and hopefully will
merge these changes tomorrow.

​

Re: Ignite-1.5 Release

[Semyon Boikov]

I merged into 1.5 performance optimizations implemented by Yakov. With
these optimizations we got up to 20% throughput increase in transactional
cache benchmarks.

On Mon, Nov 2, 2015 at 4:35 PM, Yakov Zhdanov  wrote:

> Guys,
>
> I think we can start preparation to Ignite-1.5 release which will include
> many interesting features:
>
> 1. Portable object API
> https://issues.apache.org/jira/browse/IGNITE-1486
>
> 2. Ignite.NET and Ignite C++
> https://issues.apache.org/jira/browse/IGNITE-1282
>
> 3. Optimistic serializable transactions
> https://issues.apache.org/jira/browse/IGNITE-1607
>
> 4. Distributed SQL joins - we will be able to query non-collocated data as
> well
> https://issues.apache.org/jira/browse/IGNITE-1232
>
> 5. Enhanced Oracle and IBM JDK interoperability
> https://issues.apache.org/jira/browse/IGNITE-1526
>
> 6. MQTT streamer
> https://issues.apache.org/jira/browse/IGNITE-535
>
> 7. Continuous query failover
> https://issues.apache.org/jira/browse/IGNITE-426
>
> 8. Significant transactional cache performance optimizations - I will merge
> these changes from 'ignite-1.4-slow-server-debug' today or tomorrow.
>
> 9. Many stability and fault-tolerance fixes.
>
> 10. I would also like to include distributed Semaphore. Vladislav, any
> chance you can finish with it this week?
> https://issues.apache.org/jira/browse/IGNITE-
> 638
>
> Thanks to everyone involved! Guys, esp. assignees of mentioned issues,
> please respond to this email and let us know when can we expect your
> changes being merged to master and release branch?
>
> Can someone create ignite-1.5 release branch?
>
> --Yakov
>

Re: withKeepIgniteObject behavior

[Semyon Boikov]

Alexey,

We already implemented 'per-transaction entry' behavior  for 'skipStore'
flag and IgniteTxEntry already have 'flags' field where 'keepPortable' flag
can be added.

On Mon, Oct 19, 2015 at 5:58 PM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> Igniters,
>
> As a part of work on IGNITE-950 [1] ticket we wanted to implement a
> functionality which would allow users to plug their own implementation of
> IgniteObject, which in turn would allow to introspect objects for fields
> without deserialization and use of reflection.
>
> The design draft is available on wiki [2]. Comments are highly appreciated.
> One of the good use-cases of this feature will be an ability to index and
> query JSON objects.
>
> Now, as a part of preliminary work for this ticket I was looking at current
> implementation of Portable objects (not released, in private API) and found
> that cache modifier withKeepPortable() (will be renamed to
> withKeepIgniteObjects) is not working properly. Namely, this modifier is
> not honored in entry processors and interceptors for transactional cache.
>
> Now, consider the following code:
>
> cacheA = ignite.cache("A");
> cacheB = ignite.cache("B").withKeepIgniteObjects();
>
> try (Transaction tx = ignite.transactions.txStart()) {
> cacheA.invoke(key1, new EP());
> cacheB.invoke(key2, new EP2());
>
> tx.commit();
> }
>
> This code will require to store and transmit over the network a per-cache
> map of withKeepIgniteObjects flag. It is not hard to implement, but I looks
> like it is an additional overhead that we might not need. So to speak, same
> thing is not currently supported for withSkipStore flag.
>
> My question is - do we want/need to support this use-case?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-950
> [2]
> https://cwiki.apache.org/confluence/display/IGNITE/Pluggable+IgniteObjects
>

Optimistic/serializable transactions implementation

[Semyon Boikov]

Hello,

I'm working on new implementation for optimistic/serializable transaction
mode (current implementation is inefficient and have bugs). This mode is
supposed to be used when concurrent transactions do not update the same
keys, in this case transactions can be executed more efficiently, but if
concurrent transactions have read of write conflicts then
TransactionOptimisticException can be thrown and transaction should be
retried. Also with current transactions implementation user should order
updated keys, otherwise deadlock is possible, we want to remove this
requirement for optimistic/serializable mode.

Issue with read/write conflicts can be detected if when read is performed
entry version is stored and then compared with current version when
transaction lock is acquired.

Another issue is avoid deadlocks when transactions acquire keys in
different order.

I created one possible solution using 'try-lock' approach: for each cache
entry we already have queue with current lock owner and transactions trying
to acquire entry lock.
When optimistic/serializable transaction tries to acquire entry lock it
checks that entry is not locked and there are no others transactions
waiting for lock entry, otherwise transaction fails with
TransactionOptimisticException. But this approach does not work well when
two transaction have lot of intersecting keys: it is possible that these
transaction will constantly conflict and will both constantly fail with
TransactionOptimisticException.

It seems there is better approach to resolve these conflicts to do not fail
all conflicting transactions:
- we should order all transactions by some attribute (e.g. all transactions
already have unique version)
- transaction with greater order should always 'win' transaction with lower
order
- per-entry queue with waiting transactions should be sorted by this order
- when transaction tries to acquire entry lock it is added in waiting queue
if queue is empty or last waiting transaction have lower order, otherwise
transaction fails

With this approach transaction lock assignment is ordered and transactions
with lower order never wait for transactions with greater order, so this
algorithm should not cause deadlocks.

I also created unit test simulating this algorithm and it did not reveal
any issues. Also in this unit tests I measured percent of rollbacks when
concurrent updates have lots of conflicts: with 'try-lock' approach percent
of rollbacks is ~80%, with new algorithm is ~1% (but of course with
real-life test results can be different).

Does anyone see problems with this locking approach?

Re: Optimistic/serializable transactions implementation

[Semyon Boikov]

Jira issue is https://issues.apache.org/jira/browse/IGNITE-1607, branch
ignite-1607

On Thu, Oct 15, 2015 at 10:58 AM, Alexey Kuznetsov <akuznet...@gridgain.com>
wrote:

> Semyon, could you please give a link to JIRA issue (if any) and what branch
> contains your code?
>
> Also it is not clear for me, how transaction order is assigned /
> calculated?
> If I start transaction t1 on none n1 and t2 on node n2, how it will be
> calculated?
>
> Thanks.
>
> On Thu, Oct 15, 2015 at 2:00 PM, Semyon Boikov <sboi...@gridgain.com>
> wrote:
>
> > Hello,
> >
> > I'm working on new implementation for optimistic/serializable transaction
> > mode (current implementation is inefficient and have bugs). This mode is
> > supposed to be used when concurrent transactions do not update the same
> > keys, in this case transactions can be executed more efficiently, but if
> > concurrent transactions have read of write conflicts then
> > TransactionOptimisticException can be thrown and transaction should be
> > retried. Also with current transactions implementation user should order
> > updated keys, otherwise deadlock is possible, we want to remove this
> > requirement for optimistic/serializable mode.
> >
> > Issue with read/write conflicts can be detected if when read is performed
> > entry version is stored and then compared with current version when
> > transaction lock is acquired.
> >
> > Another issue is avoid deadlocks when transactions acquire keys in
> > different order.
> >
> > I created one possible solution using 'try-lock' approach: for each cache
> > entry we already have queue with current lock owner and transactions
> trying
> > to acquire entry lock.
> > When optimistic/serializable transaction tries to acquire entry lock it
> > checks that entry is not locked and there are no others transactions
> > waiting for lock entry, otherwise transaction fails with
> > TransactionOptimisticException. But this approach does not work well when
> > two transaction have lot of intersecting keys: it is possible that these
> > transaction will constantly conflict and will both constantly fail with
> > TransactionOptimisticException.
> >
> > It seems there is better approach to resolve these conflicts to do not
> fail
> > all conflicting transactions:
> > - we should order all transactions by some attribute (e.g. all
> transactions
> > already have unique version)
> > - transaction with greater order should always 'win' transaction with
> lower
> > order
> > - per-entry queue with waiting transactions should be sorted by this
> order
> > - when transaction tries to acquire entry lock it is added in waiting
> queue
> > if queue is empty or last waiting transaction have lower order, otherwise
> > transaction fails
> >
> > With this approach transaction lock assignment is ordered and
> transactions
> > with lower order never wait for transactions with greater order, so this
> > algorithm should not cause deadlocks.
> >
> > I also created unit test simulating this algorithm and it did not reveal
> > any issues. Also in this unit tests I measured percent of rollbacks when
> > concurrent updates have lots of conflicts: with 'try-lock' approach
> percent
> > of rollbacks is ~80%, with new algorithm is ~1% (but of course with
> > real-life test results can be different).
> >
> > Does anyone see problems with this locking approach?
> >
>
>
>
> --
> Alexey Kuznetsov
> GridGain Systems
> www.gridgain.com
>

Re: Optimistic/serializable transactions implementation

[Semyon Boikov]

>
> is starvation possible here?
> E.g. there are two nodes with GUIDs A and B. What will happen in the
> following case:
> 1) TX (A, 1) started and locked the key;
> 2) TX (B, 1) started and waiting for lock;
> 3) TX (A, 2) started and waiting for lock;
> 4) TX (A, 1) released the lock.
> 5) Who wins now? If this is (A, 2), then lock acquisition by the node B can
> be postponed indefinitely.


I assume transactions are ordered as TX(A, 1), TX(A, 2), TX(B, 1). In this
case when TX(A, 2) tries to get lock it sees that there is already
waiting TX(B,
1) with greater order and it fails. So TX(A, 1), TX(B, 1) will finish
succesfully, TX(A,2) should be retried.

On Thu, Oct 15, 2015 at 11:31 AM, Vladimir Ozerov 
wrote:

> is starvation possible here?
>
> E.g. there are two nodes with GUIDs A and B. What will happen in the
> following case:
> 1) TX (A, 1) started and locked the key;
> 2) TX (B, 1) started and waiting for lock;
> 3) TX (A, 2) started and waiting for lock;
> 4) TX (A, 1) released the lock.
> 5) Who wins now? If this is (A, 2), then lock acquisition by the node B can
> be postponed indefinitely.
>
> On Thu, Oct 15, 2015 at 11:18 AM, Alexey Kuznetsov <
> akuznet...@gridgain.com>
> wrote:
>
> > Just one more question:
> >
> > "- transaction with greater order should always 'win' transaction with
> > lower order"
> >
> > Greater order means "younger"?
> >
> > If it so, why should younger transactions win? Why not older?
> >
> > Or user will have possibility to configure this aspect of conflict
> > resolution?
> >
> > On Thu, Oct 15, 2015 at 3:07 PM, Alexey Goncharuk <
> > alexey.goncha...@gmail.com> wrote:
> >
> > > 2015-10-15 10:58 GMT+03:00 Alexey Kuznetsov :
> > > >
> > > > Also it is not clear for me, how transaction order is assigned /
> > > > calculated?
> > > > If I start transaction t1 on none n1 and t2 on node n2, how it will
> be
> > > > calculated?
> > > >
> > > I believe that we can utilize nearXidVersion for this ordering (or some
> > > sort of it's modification). Since cache version contains local order,
> > > topology version and node ID and also is comparable, it is guaranteed
> > that
> > > nearXidVersion is always unique and there is always an unambiguous
> order
> > > between any two Xid versions.
> > >
> >
> >
> >
> > --
> > Alexey Kuznetsov
> > GridGain Systems
> > www.gridgain.com
> >
>

Re: Optimistic/serializable transactions implementation

[Semyon Boikov]

We did not decided yet exactly by which attribute transactions should be
ordered, but logically it is better when wins older transaction or
transaction having more keys.

On Thu, Oct 15, 2015 at 11:18 AM, Alexey Kuznetsov 
wrote:

> Just one more question:
>
> "- transaction with greater order should always 'win' transaction with
> lower order"
>
> Greater order means "younger"?
>
> If it so, why should younger transactions win? Why not older?
>
> Or user will have possibility to configure this aspect of conflict
> resolution?
>
> On Thu, Oct 15, 2015 at 3:07 PM, Alexey Goncharuk <
> alexey.goncha...@gmail.com> wrote:
>
> > 2015-10-15 10:58 GMT+03:00 Alexey Kuznetsov :
> > >
> > > Also it is not clear for me, how transaction order is assigned /
> > > calculated?
> > > If I start transaction t1 on none n1 and t2 on node n2, how it will be
> > > calculated?
> > >
> > I believe that we can utilize nearXidVersion for this ordering (or some
> > sort of it's modification). Since cache version contains local order,
> > topology version and node ID and also is comparable, it is guaranteed
> that
> > nearXidVersion is always unique and there is always an unambiguous order
> > between any two Xid versions.
> >
>
>
>
> --
> Alexey Kuznetsov
> GridGain Systems
> www.gridgain.com
>

Re: vert.x integration

[Semyon Boikov]

I'm working on cache issues blocking vert.x integration, there is one more
issue reproducing in vert.x tests:
https://issues.apache.org/jira/browse/IGNITE-1534. Hopefully these issues
will be fixed within the next few days.

On Wed, Sep 30, 2015 at 9:06 PM, Dmitriy Setrakyan 
wrote:

> Igniters,
>
> Want to ask again about vert.x integration:
> https://issues.apache.org/jira/browse/IGNITE-1079
>
> To my knowledge most issues on vert.x integration were closed in 1.4.
> However, it looks like this issue is still blocked by these tickets:
>
> https://issues.apache.org/jira/browse/IGNITE-1221
> https://issues.apache.org/jira/browse/IGNITE-1135
>
> Is it still the case?
>
> D.
>

Re: QueryProcessor and CacheProcessor start order

[Semyon Boikov]

Yes, I tried this fix, TC results are good. I run queries suite 5 times and
don't see anymore failuers in queries restarts tests.

I agree that it does not fix issue with Cache.destroy, but IMO it is not
critical and I don't want to spend time on this now.

On Thu, Sep 3, 2015 at 10:47 PM, Sergi Vladykin <sergi.vlady...@gmail.com>
wrote:

> Did you try that fix, Semyon?
>
> I just think that switching start order may be not the safest solution,
> because it seems that the same error potentially can happen even
> for Cache.destroy.
>
> The correct solution should be to wait until all the queries currently
> running for this cache will end before actually stopping the cache.
>
> Thoughts?
>
> Sergi
>
>
> 2015-09-03 13:29 GMT+03:00 Sergi Vladykin <sergi.vlady...@gmail.com>:
>
> > I think it was because caches configured on start must be able to setup
> > SQL schema, so QueryProcessor must be ready.
> > If this not true anymore, then the order can be changed.
> >
> > Sergi
> >
> > 2015-09-03 13:20 GMT+03:00 Yakov Zhdanov <yzhda...@gridgain.com>:
> >
> >> Sam, I agree. Please try fix components start order and run tests.
> >>
> >> --
> >> Yakov Zhdanov, Director R
> >> *GridGain Systems*
> >> www.gridgain.com
> >>
> >> 2015-09-03 12:40 GMT+03:00 Semyon Boikov <sboi...@gridgain.com>:
> >>
> >> > Does anybody remember why QueryProcessor is started before
> >> CacheProcessor?
> >> >
> >> > With this order cache processor can be stopped at the moment when
> query
> >> > request is processed, and this causes sporadic errors like NPE (I see
> >> this
> >> > from time to time in IgniteCacheQueryNodeRestartSelfTest).
> >> >
> >> > Looks like cache processor does not depend on query processor so start
> >> > order should be changed.
> >> >
> >>
> >
> >
>

QueryProcessor and CacheProcessor start order

[Semyon Boikov]

Does anybody remember why QueryProcessor is started before CacheProcessor?

With this order cache processor can be stopped at the moment when query
request is processed, and this causes sporadic errors like NPE (I see this
from time to time in IgniteCacheQueryNodeRestartSelfTest).

Looks like cache processor does not depend on query processor so start
order should be changed.

Re: Deprecate IgniteConfiguration.getGridName

[Semyon Boikov]

We can add 'nodeName' on ClusterNode API, it is very easy to implement
since node name is already stored in node attributes.

On Wed, Aug 26, 2015 at 4:44 AM, Dmitriy Setrakyan dsetrak...@apache.org
wrote:

 Although I share the sentiment that gridName is confusing, I think renaming
 gridName to nodeName will be equally confusing. The nodeName property
 sounds like something that should be on ClusterNode API, which is not the
 case.

 If we rename if in one of the future versions, I think instanceName would
 be a better option, no?

 D.

 On Wed, Aug 19, 2015 at 1:36 PM, Yakov Zhdanov yzhda...@apache.org
 wrote:

  I tend to agree with Semyon. This is not a grid name, but the name of the
  local Ignite instance as an entry point to the grid. And this name is
 used
  to conveniently get local Ignite instance: Ignite ignite =
  Ignition.ignite(name);
 
  It's usage in internal marshalling should stop.
 
  Any other opinions?
 
  --Yakov
 
  2015-08-19 11:04 GMT+03:00 Alexey Kuznetsov akuznet...@gridgain.com:
 
   Semen, it is possible to distinguish nodes via attributes (put some
  marker
   to node attributes) ?
  
   On Wed, Aug 19, 2015 at 2:57 PM, Semyon Boikov sboi...@gridgain.com
   wrote:
  
In my opinion this propery just should be renamed to 'nodeName'.
Possibility to set node name is very useful in our tests. As I see
'setNodeId' is already deprecated in IgniteConfiguration, how we will
distinguish nodes in our tests?
   
On Tue, Aug 18, 2015 at 1:42 PM, Yakov Zhdanov yzhda...@apache.org
wrote:
   
 Guys,

 I am thinking of deprecating this property.

 1. nodes with different grid names can discover each other and
 build
  a
 single topology. I heard that some of the users expected that there
   would
 be separate topologies
 2. Locally started nodes can be accessed via their IDs through
  Ignition
 interface.
 3. Internally gridName is used in components serialization and
 deserializaiton, but this is very unreliable and can be broken if
different
 names are configured within the same topology -
 https://issues.apache.org/jira/browse/IGNITE-10

 Thoughts?

 --Yakov

   
  
  
  
   --
   Alexey Kuznetsov
   GridGain Systems
   www.gridgain.com