IGNITE-640: IgniteMultimap interface draft version

2016-08-11 Thread Amir Akhmedov
Hi guys,

I noticed that IgniteMultimap waits too long to be implemented and I
decided to pick it up :-)

For now I want to bring to community up a draft version for IgniteMultimap
interface. Will appreciate any feedback and once it's fine I will start its
implementation


package org.apache.ignite;

import org.apache.ignite.lang.IgniteCallable;
import org.apache.ignite.lang.IgniteRunnable;

import java.io.Closeable;
import java.util.*;

public interface IgniteMultimap extends Closeable {

/**
 * Returns the list of values to which the specified key is mapped,
 * or {@code null} if this map contains no mapping for the key.
 *
 * @param key the key whose associated values are to be returned
 * @return the list of values to which the specified key is mapped,
 * or {@code null} if this map contains no mapping for the key.
 * @throws ClassCastException if the key is of an inappropriate
type for this map
 */
public List get(K key);

/**
 * Returns the value at a certain index to which the specified key
is mapped,
 * or {@code null} if this map contains no mapping for the key.
 *
 * @param key the key whose associated values are to be returned
 * @param index index to lookup
 * @return the value at a certain index to which the specified key
is mapped,
 * or {@code null} if this map contains no mapping for the key.
 * @throws ClassCastException if the key is of an inappropriate
type for this map
 * @throws IndexOutOfBoundsException if the index is out of range
(index < 0 || index >= list.size())
 */
public V get(K key, int index);

/**
 * Returns the values for specified range of indexes, between min and max
 * to which the specified key is mapped, or {@code null} if this
map contains
 * no mapping for the key.
 *
 * @param key the key whose associated values are to be returned
 * @param min min index to lookup
 * @param max max index to lookup
 * @return the values for specified range of indexes, between min and max
 * to which the specified key is mapped, or {@code null} if this
map contains
 * no mapping for the key.
 * @throws ClassCastException if the key is of an inappropriate
type for this map
 * @throws IndexOutOfBoundsException if either of indexes min or
max is out of
 * range (index < 0 || index >= list.size())
 */
public List get(K key, int min, int max);

/**
 * Returns the list of values for specified indexes to which the
specified key
 * is mapped, or {@code null} if this map contains no mapping for the key.
 * @param key the key whose associated values are to be returned
 * @param indexes the indexes to lookup
 * @return the list of values for specified indexes to which the
specified key
 * is mapped, or {@code null} if this map contains no mapping for the key.
 * @throws ClassCastException if the key is of an inappropriate
type for this map
 * @throws IndexOutOfBoundsException if either of indexes is out of
 * range (index < 0 || index >= list.size())
 */
public List get(K key, Iterable indexes);

/**
 * Returns the list of values for a collection of keys to which
the keys are mapped.
 * {@code null} will be added to resulting list if this map
contains no mapping for the key.
 * @param keys collection of keys
 * @return the list of values for a collection of keys to which
the keys are mapped.
 * @throws ClassCastException if the key is of an inappropriate
type for this map
 * {@code null} will be added to resulting list if this map
contains no mapping for the key.
 */
public Map getAll(Collection keys);

/**
 * Returns a value at a certain index for a collection of keys to
which the keys are mapped.
 * {@code null} will be added to resulting list if this map
contains no mapping for the key.
 * @param keys collection of keys
 * @param index the index to lookup
 * @return a value at a certain index for a collection of keys to
which the keys are mapped.
 * {@code null} will be added to resulting list if this map
contains no mapping for the key.
 * @throws ClassCastException if the key is of an inappropriate
type for this map
 * @throws IndexOutOfBoundsException if the indexes is out of
range (index < 0 || index >= list.size())
 */
public Map getAll(Collection keys, int index);

/**
 * Returns the list of values for the range of indexes, between
min and max for a collection
 * of keys to which the keys are mapped. {@code null} will be
added to resulting list if this
 * map contains no mapping for the key.
 * @param keys collection of keys
 * @param min min index to lookup
 * @param max max index to lookup
 * @return the list of values for the range of indexes, between
min and max for a collection
 * of keys to which the keys are mapped. {@code null} will be
added to 

Re: Ignite 2.0 tasks/roadmap

2016-08-11 Thread Valentin Kulichenko
There is one more use case where several types per cache can be useful (I
know that it's utilized by some of our users). The use case is the
following: transactional updates with write-behind and foreign key
constraints on the database. In case data within transaction is collocated
and all types are in the same cache, it works, because there is only one
write-behind queue. Once we split different types into different caches,
there is no guarantee that objects will be written in the proper order and
that the constraints will not be violated.

However, I think this is not a very clean way to achieve the result.
Probably we should just provide better support for this use case in 2.0.
Basically, we somehow need to allow to share a single write-behind queue
between different caches.

Any thoughts?

-Val

On Thu, Aug 11, 2016 at 10:40 AM, Dmitriy Setrakyan 
wrote:

> On Thu, Aug 11, 2016 at 7:25 AM, Sergey Kozlov 
> wrote:
>
> > Alexey
> >
> > You're right. Of course I meant growth of caches number.
> >
> > I see a few points here:
> >
> > 1. If a grid used by various applications the cache names may overlap
> (like
> > "accounts") and the application needs to use prefixed names and etc.
> > 2. For clear or destory caches I need to know their names (or iterate
> over
> > caches but I'm not sure that it is supported by API). For destroy/clear
> > caches belonging to same group we will do it by single operation without
> > knowledge of cache names.
> > 3. Cache group can have cache attributes which will be inherited by a
> cache
> > created in that group (like eviction policy or cache mode).
> > 4. No reason to add specific feature like SqlShema if it applicable for
> > regular caches as well.
> >
>
> Sergey K, setting the same SQL schema for multiple caches, as proposed by
> Sergi, solves a different problem of having too many SQL schemas due to too
> many different caches. I think Sergi proposed a good solution for it.
>
>
> >
> > On Thu, Aug 11, 2016 at 6:58 PM, Alexey Kuznetsov <
> akuznet...@gridgain.com
> > >
> > wrote:
> >
> > > Sergey, I belive you mean "increase" instead of "reduce"?
> > >
> > > How grouping will help?
> > > To do some operation, for example, clear on group of cashes at once?
> > >
> > > 11 Авг 2016 г. 22:06 пользователь "Sergey Kozlov" <
> skoz...@gridgain.com>
> > > написал:
> > >
> > > > I mean not only SQL features for caches. Single type per cache
> > definitely
> > > > reduces number of caches for regular user and grouping caches will
> help
> > > to
> > > > manage caches in grid.
> > > >
> > > > On Thu, Aug 11, 2016 at 5:41 PM, Sergi Vladykin <
> > > sergi.vlady...@gmail.com>
> > > > wrote:
> > > >
> > > > > I'm aware of this issue. My plan was to allow setting the same
> schema
> > > > name
> > > > > to different caches using CacheConfiguration.setSqlSchema(...).
> This
> > > way
> > > > > we
> > > > > will have separate caches but from SQL point of view respective
> > tables
> > > > will
> > > > > reside in the same schema. No need to introduce new concepts.
> > > > >
> > > > > Sergi
> > > > >
> > > > >
> > > > > 2016-08-11 17:24 GMT+03:00 Sergey Kozlov :
> > > > >
> > > > > > HI
> > > > > >
> > > > > > Due to approach to disable to store more than one type per cache
> > the
> > > > > cache
> > > > > > use becomes similar the table use for databases.
> > > > > > So I suppose would be good to introduce a database/schema-similar
> > > > concept
> > > > > > for caches. It may be implemented as a non-default behavior for
> > > > backward
> > > > > > compatibility.
> > > > > >
> > > > > > On Sat, Aug 6, 2016 at 1:04 AM, Dmitriy Setrakyan <
> > > > dsetrak...@apache.org
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > On Fri, Aug 5, 2016 at 2:46 AM, Alexey Kuznetsov <
> > > > > > akuznet...@gridgain.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I remember couple more thing for 2.0
> > > > > > > >
> > > > > > > > How about to drop **ignite-scalar** module in Ignite 2.0?
> > > > > > > >
> > > > > > >
> > > > > > > Why?
> > > > > > >
> > > > > > >
> > > > > > > > And may be drop **ignite-spark-2.10** module support as of
> > > > **Spark**
> > > > > 2
> > > > > > is
> > > > > > > > on **scala 2.11**.
> > > > > > > >
> > > > > > >
> > > > > > > I would drop it only if it is difficult to support. Do we know
> > what
> > > > > kind
> > > > > > of
> > > > > > > impact will it have on the community? Anyone is still using
> 2.10?
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Aug 2, 2016 at 11:09 PM, Denis Magda <
> > > dma...@gridgain.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Let’s add this [1] issue to the list because it requires
> > > > > significant
> > > > > > > work
> > > > > > > > > on the side of metrics SPI.
> > > > > > > > >
> > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-3495 <
> > > > > > > > > 

[GitHub] ignite pull request #952: IGNITE-3678: Test for partition distribution

2016-08-11 Thread pperalta
GitHub user pperalta opened a pull request:

https://github.com/apache/ignite/pull/952

IGNITE-3678: Test for partition distribution

This submission includes a test that will assert the even and safe 
distribution of partitions when server nodes are forcibly killed.

Additionally, the following are included:

(1) A new `Assertion` interface. This looks similar on the surface to 
`IgnitePredicate` - however there are two subtle differences:
- it returns `void`, not `boolean`
- it throws `AssertionError`
Since this does not return `boolean`, the caller does not have to check for 
the result of the assertion; if the assertion fails an exception is thrown 
which contains the details of the failed assertion.

(2) `GridRollingRestartAbstractTest`: a new abstract test class that is 
extended for tests that require:
- multiple JVMs
- the first JVM to be a client (i.e. not own any partitions)
- the server JVMs to be killed forcibly (not gracefully) while the test 
executes

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pperalta/ignite ignite-3678

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/952.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #952


commit 9eab905d292c4e951b2f300dca57434d44d11aca
Author: Patrick Peralta 
Date:   2016-08-11T18:38:10Z

IGNITE-3678: Test for partition distribution

This submission includes a test that will assert the even and safe
distribution of partitions when server nodes are forcibly killed.

Additionally, the following are included:

1. A new Assertion interface. This looks similar on the surface
   to IgnitePredicate - however there are two subtle differences:
 (a) it returns void, not boolean
 (b) it throws AssertionError
   Since this does not return boolean, the caller does not have
   to check for the result of the assertion; if the assertion fails
   an exception is thrown which contains the details of the failed
   assertion.

2. GridRollingRestartAbstractTest: a new abstract test class that
   is extended for tests that require:
 (a) multiple JVMs
 (b) the first JVM to be a client (i.e. not own any partitions)
 (c) the server JVMs to be killed forcibly (not gracefully)
 while the test executes




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Created] (IGNITE-3678) Test for partition distribution functionality

2016-08-11 Thread Patrick Peralta (JIRA)
Patrick Peralta created IGNITE-3678:
---

 Summary: Test for partition distribution functionality
 Key: IGNITE-3678
 URL: https://issues.apache.org/jira/browse/IGNITE-3678
 Project: Ignite
  Issue Type: Test
  Components: cache
Affects Versions: 1.7
Reporter: Patrick Peralta
Assignee: Patrick Peralta
 Fix For: 1.8


Create a test that will assert the following conditions in a running cluster 
while server nodes are (forcibly) killed and restarted:

* even distribution of partitions 
* safe distribution of partitions (where data will not be lost if a single node 
is killed) - this means the cache is configured with a backup count >= 1 and 
partitions are backed up to a different node from the primary



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: Ignite 2.0 tasks/roadmap

2016-08-11 Thread Sergey Kozlov
Alexey

You're right. Of course I meant growth of caches number.

I see a few points here:

1. If a grid used by various applications the cache names may overlap (like
"accounts") and the application needs to use prefixed names and etc.
2. For clear or destory caches I need to know their names (or iterate over
caches but I'm not sure that it is supported by API). For destroy/clear
caches belonging to same group we will do it by single operation without
knowledge of cache names.
3. Cache group can have cache attributes which will be inherited by a cache
created in that group (like eviction policy or cache mode).
4. No reason to add specific feature like SqlShema if it applicable for
regular caches as well.


On Thu, Aug 11, 2016 at 6:58 PM, Alexey Kuznetsov 
wrote:

> Sergey, I belive you mean "increase" instead of "reduce"?
>
> How grouping will help?
> To do some operation, for example, clear on group of cashes at once?
>
> 11 Авг 2016 г. 22:06 пользователь "Sergey Kozlov" 
> написал:
>
> > I mean not only SQL features for caches. Single type per cache definitely
> > reduces number of caches for regular user and grouping caches will help
> to
> > manage caches in grid.
> >
> > On Thu, Aug 11, 2016 at 5:41 PM, Sergi Vladykin <
> sergi.vlady...@gmail.com>
> > wrote:
> >
> > > I'm aware of this issue. My plan was to allow setting the same schema
> > name
> > > to different caches using CacheConfiguration.setSqlSchema(...). This
> way
> > > we
> > > will have separate caches but from SQL point of view respective tables
> > will
> > > reside in the same schema. No need to introduce new concepts.
> > >
> > > Sergi
> > >
> > >
> > > 2016-08-11 17:24 GMT+03:00 Sergey Kozlov :
> > >
> > > > HI
> > > >
> > > > Due to approach to disable to store more than one type per cache the
> > > cache
> > > > use becomes similar the table use for databases.
> > > > So I suppose would be good to introduce a database/schema-similar
> > concept
> > > > for caches. It may be implemented as a non-default behavior for
> > backward
> > > > compatibility.
> > > >
> > > > On Sat, Aug 6, 2016 at 1:04 AM, Dmitriy Setrakyan <
> > dsetrak...@apache.org
> > > >
> > > > wrote:
> > > >
> > > > > On Fri, Aug 5, 2016 at 2:46 AM, Alexey Kuznetsov <
> > > > akuznet...@gridgain.com>
> > > > > wrote:
> > > > >
> > > > > > I remember couple more thing for 2.0
> > > > > >
> > > > > > How about to drop **ignite-scalar** module in Ignite 2.0?
> > > > > >
> > > > >
> > > > > Why?
> > > > >
> > > > >
> > > > > > And may be drop **ignite-spark-2.10** module support as of
> > **Spark**
> > > 2
> > > > is
> > > > > > on **scala 2.11**.
> > > > > >
> > > > >
> > > > > I would drop it only if it is difficult to support. Do we know what
> > > kind
> > > > of
> > > > > impact will it have on the community? Anyone is still using 2.10?
> > > > >
> > > > >
> > > > > >
> > > > > > On Tue, Aug 2, 2016 at 11:09 PM, Denis Magda <
> dma...@gridgain.com>
> > > > > wrote:
> > > > > >
> > > > > > > Let’s add this [1] issue to the list because it requires
> > > significant
> > > > > work
> > > > > > > on the side of metrics SPI.
> > > > > > >
> > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-3495 <
> > > > > > > https://issues.apache.org/jira/browse/IGNITE-3495>
> > > > > > >
> > > > > > > —
> > > > > > > Denis
> > > > > > >
> > > > > > > > On Aug 2, 2016, at 12:45 AM, Yakov Zhdanov <
> > yzhda...@apache.org>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > Not yet. The thing is that our recent experience showed that
> it
> > > was
> > > > > not
> > > > > > > > very good idea to go with caches. E.g. when you try to
> > > deserialize
> > > > > > inside
> > > > > > > > continuous query listener on client side it implicitly calls
> > > > > > cache.get()
> > > > > > > > which in turn may cause deadlock under some circumstances.
> > > > > > > >
> > > > > > > > --Yakov
> > > > > > > >
> > > > > > > > 2016-08-02 2:41 GMT+03:00 Dmitriy Setrakyan <
> > > dsetrak...@apache.org
> > > > >:
> > > > > > > >
> > > > > > > >> On Mon, Aug 1, 2016 at 3:46 AM, Yakov Zhdanov <
> > > > yzhda...@apache.org>
> > > > > > > wrote:
> > > > > > > >>
> > > > > > > >>> One more point.
> > > > > > > >>>
> > > > > > > >>> I insist on stop using marshaller and meta caches but
> switch
> > to
> > > > > > > spreading
> > > > > > > >>> this info via custom discovery events.
> > > > > > > >>>
> > > > > > > >>
> > > > > > > >> Do we have a ticket explaining why this needs to be done?
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>>
> > > > > > > >>> --Yakov
> > > > > > > >>>
> > > > > > > >>> 2016-07-27 19:57 GMT+03:00 Dmitriy Setrakyan <
> > > > > dsetrak...@apache.org
> > > > > > >:
> > > > > > > >>>
> > > > > > >  On Wed, Jul 27, 2016 at 11:36 AM, Yakov Zhdanov <
> > > > > > yzhda...@apache.org>
> > > > > > >  wrote:
> > > > > > > 
> > > > > > > > Guys, I think we can also split event notification for
> 

[GitHub] ignite pull request #951: Ignite 2559

2016-08-11 Thread ntikhonov
GitHub user ntikhonov opened a pull request:

https://github.com/apache/ignite/pull/951

Ignite 2559



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2559

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/951.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #951


commit 17ea9fc4de0f54f5337cfbc085c8a11e9a6cb881
Author: Tikhonov Nikolay 
Date:   2016-08-09T21:46:30Z

IGNITE-2256 WIP

commit 1d0cbb45cd61c5c8e6ec926d7e629eb94111b32f
Author: vd-pyatkov 
Date:   2016-08-11T05:43:50Z

IGNITE-3618: Client can not load data after server restarts. This closes 
#941.

commit 1139a9f76b5d37073261d729a15b1fbec674d48d
Author: vozerov-gridgain 
Date:   2016-08-11T05:47:48Z

Added missing license.

commit 039e1921d27ebf181d59fb2860f5c145c34fa900
Author: nikolay_tikhonov 
Date:   2016-08-11T12:34:46Z

IGNITE-2559 WIP

commit d4a6428b8d92e7cbdb61cf4d1dd36c020a5b19da
Author: nikolay_tikhonov 
Date:   2016-08-11T14:27:39Z

IGNITE-2559 WIP

commit a83663ad4e75a63a553873aa54495d4fd7b4ad4d
Author: nikolay_tikhonov 
Date:   2016-08-11T14:32:23Z

Merge branch 'ignite-1.6.5' into ignite-2559

commit de50fc504db6f1f0d22726904ed9f56912f57c05
Author: nikolay_tikhonov 
Date:   2016-08-11T16:33:17Z

IGNITE-2559 WIP




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] ignite pull request #950: IGNITE-3095 .NET: Reference NUnit via NuGet instea...

2016-08-11 Thread ptupitsyn
GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/950

IGNITE-3095 .NET: Reference NUnit via NuGet instead of predefined folder



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ptupitsyn/ignite ignite-3095

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/950.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #950


commit a24ea23636737a45d4ac3351f42568acd844f011
Author: Pavel Tupitsyn 
Date:   2016-08-11T15:40:05Z

IGNITE-3095 .NET: Reference NUnit via NuGet instead of predefined folder




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


Re: Ignite 2.0 tasks/roadmap

2016-08-11 Thread Sergey Kozlov
I mean not only SQL features for caches. Single type per cache definitely
reduces number of caches for regular user and grouping caches will help to
manage caches in grid.

On Thu, Aug 11, 2016 at 5:41 PM, Sergi Vladykin 
wrote:

> I'm aware of this issue. My plan was to allow setting the same schema name
> to different caches using CacheConfiguration.setSqlSchema(...). This way
> we
> will have separate caches but from SQL point of view respective tables will
> reside in the same schema. No need to introduce new concepts.
>
> Sergi
>
>
> 2016-08-11 17:24 GMT+03:00 Sergey Kozlov :
>
> > HI
> >
> > Due to approach to disable to store more than one type per cache the
> cache
> > use becomes similar the table use for databases.
> > So I suppose would be good to introduce a database/schema-similar concept
> > for caches. It may be implemented as a non-default behavior for backward
> > compatibility.
> >
> > On Sat, Aug 6, 2016 at 1:04 AM, Dmitriy Setrakyan  >
> > wrote:
> >
> > > On Fri, Aug 5, 2016 at 2:46 AM, Alexey Kuznetsov <
> > akuznet...@gridgain.com>
> > > wrote:
> > >
> > > > I remember couple more thing for 2.0
> > > >
> > > > How about to drop **ignite-scalar** module in Ignite 2.0?
> > > >
> > >
> > > Why?
> > >
> > >
> > > > And may be drop **ignite-spark-2.10** module support as of **Spark**
> 2
> > is
> > > > on **scala 2.11**.
> > > >
> > >
> > > I would drop it only if it is difficult to support. Do we know what
> kind
> > of
> > > impact will it have on the community? Anyone is still using 2.10?
> > >
> > >
> > > >
> > > > On Tue, Aug 2, 2016 at 11:09 PM, Denis Magda 
> > > wrote:
> > > >
> > > > > Let’s add this [1] issue to the list because it requires
> significant
> > > work
> > > > > on the side of metrics SPI.
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-3495 <
> > > > > https://issues.apache.org/jira/browse/IGNITE-3495>
> > > > >
> > > > > —
> > > > > Denis
> > > > >
> > > > > > On Aug 2, 2016, at 12:45 AM, Yakov Zhdanov 
> > > > wrote:
> > > > > >
> > > > > > Not yet. The thing is that our recent experience showed that it
> was
> > > not
> > > > > > very good idea to go with caches. E.g. when you try to
> deserialize
> > > > inside
> > > > > > continuous query listener on client side it implicitly calls
> > > > cache.get()
> > > > > > which in turn may cause deadlock under some circumstances.
> > > > > >
> > > > > > --Yakov
> > > > > >
> > > > > > 2016-08-02 2:41 GMT+03:00 Dmitriy Setrakyan <
> dsetrak...@apache.org
> > >:
> > > > > >
> > > > > >> On Mon, Aug 1, 2016 at 3:46 AM, Yakov Zhdanov <
> > yzhda...@apache.org>
> > > > > wrote:
> > > > > >>
> > > > > >>> One more point.
> > > > > >>>
> > > > > >>> I insist on stop using marshaller and meta caches but switch to
> > > > > spreading
> > > > > >>> this info via custom discovery events.
> > > > > >>>
> > > > > >>
> > > > > >> Do we have a ticket explaining why this needs to be done?
> > > > > >>
> > > > > >>
> > > > > >>>
> > > > > >>> --Yakov
> > > > > >>>
> > > > > >>> 2016-07-27 19:57 GMT+03:00 Dmitriy Setrakyan <
> > > dsetrak...@apache.org
> > > > >:
> > > > > >>>
> > > > >  On Wed, Jul 27, 2016 at 11:36 AM, Yakov Zhdanov <
> > > > yzhda...@apache.org>
> > > > >  wrote:
> > > > > 
> > > > > > Guys, I think we can also split event notification for user
> > > > listeners
> > > > > >>> and
> > > > > > internal system listeners. I have been seeing a lot of issues
> > > > caused
> > > > > >> by
> > > > > > some heavy or blocking operations in user-defined listeners.
> > This
> > > > may
> > > > >  block
> > > > > > internal component notification (e.g. on discovery event)
> > causing
> > > > >  topology
> > > > > > hangings.
> > > > > >
> > > > > 
> > > > >  Sure. There are a lot of features being added. Would be nice
> to
> > > > assign
> > > > > >> a
> > > > >  release manager for Ignite 2.0 and document all the discussed
> > > > features
> > > > > >> on
> > > > >  the Wiki.
> > > > > 
> > > > > 
> > > > > >
> > > > > > --Yakov
> > > > > >
> > > > > > 2016-06-25 2:42 GMT+03:00 Alexey Goncharuk <
> > > > > >> alexey.goncha...@gmail.com
> > > > >  :
> > > > > >
> > > > > >> Folks,
> > > > > >>
> > > > > >> Recently I have seen a couple of emails suggesting
> > > > > >> tasks/improvements
> > > > > > that
> > > > > >> we cannot do in 1.x releases due to API compatibility
> reasons,
> > > so
> > > > > >>> they
> > > > > > are
> > > > > >> postponed to 2.0. I would like to keep track of these tasks
> in
> > > > some
> > > > > >>> way
> > > > > > in
> > > > > >> our Jira to make sure we do not have anything obsolete when
> it
> > > > > >> comes
> > > > > >>> to
> > > > > > the
> > > > > >> next major version release.
> > > > > >>
> > > > > >> My question for 

Re: Ignite 2.0 tasks/roadmap

2016-08-11 Thread Sergi Vladykin
I'm aware of this issue. My plan was to allow setting the same schema name
to different caches using CacheConfiguration.setSqlSchema(...). This way we
will have separate caches but from SQL point of view respective tables will
reside in the same schema. No need to introduce new concepts.

Sergi


2016-08-11 17:24 GMT+03:00 Sergey Kozlov :

> HI
>
> Due to approach to disable to store more than one type per cache the cache
> use becomes similar the table use for databases.
> So I suppose would be good to introduce a database/schema-similar concept
> for caches. It may be implemented as a non-default behavior for backward
> compatibility.
>
> On Sat, Aug 6, 2016 at 1:04 AM, Dmitriy Setrakyan 
> wrote:
>
> > On Fri, Aug 5, 2016 at 2:46 AM, Alexey Kuznetsov <
> akuznet...@gridgain.com>
> > wrote:
> >
> > > I remember couple more thing for 2.0
> > >
> > > How about to drop **ignite-scalar** module in Ignite 2.0?
> > >
> >
> > Why?
> >
> >
> > > And may be drop **ignite-spark-2.10** module support as of **Spark** 2
> is
> > > on **scala 2.11**.
> > >
> >
> > I would drop it only if it is difficult to support. Do we know what kind
> of
> > impact will it have on the community? Anyone is still using 2.10?
> >
> >
> > >
> > > On Tue, Aug 2, 2016 at 11:09 PM, Denis Magda 
> > wrote:
> > >
> > > > Let’s add this [1] issue to the list because it requires significant
> > work
> > > > on the side of metrics SPI.
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-3495 <
> > > > https://issues.apache.org/jira/browse/IGNITE-3495>
> > > >
> > > > —
> > > > Denis
> > > >
> > > > > On Aug 2, 2016, at 12:45 AM, Yakov Zhdanov 
> > > wrote:
> > > > >
> > > > > Not yet. The thing is that our recent experience showed that it was
> > not
> > > > > very good idea to go with caches. E.g. when you try to deserialize
> > > inside
> > > > > continuous query listener on client side it implicitly calls
> > > cache.get()
> > > > > which in turn may cause deadlock under some circumstances.
> > > > >
> > > > > --Yakov
> > > > >
> > > > > 2016-08-02 2:41 GMT+03:00 Dmitriy Setrakyan  >:
> > > > >
> > > > >> On Mon, Aug 1, 2016 at 3:46 AM, Yakov Zhdanov <
> yzhda...@apache.org>
> > > > wrote:
> > > > >>
> > > > >>> One more point.
> > > > >>>
> > > > >>> I insist on stop using marshaller and meta caches but switch to
> > > > spreading
> > > > >>> this info via custom discovery events.
> > > > >>>
> > > > >>
> > > > >> Do we have a ticket explaining why this needs to be done?
> > > > >>
> > > > >>
> > > > >>>
> > > > >>> --Yakov
> > > > >>>
> > > > >>> 2016-07-27 19:57 GMT+03:00 Dmitriy Setrakyan <
> > dsetrak...@apache.org
> > > >:
> > > > >>>
> > > >  On Wed, Jul 27, 2016 at 11:36 AM, Yakov Zhdanov <
> > > yzhda...@apache.org>
> > > >  wrote:
> > > > 
> > > > > Guys, I think we can also split event notification for user
> > > listeners
> > > > >>> and
> > > > > internal system listeners. I have been seeing a lot of issues
> > > caused
> > > > >> by
> > > > > some heavy or blocking operations in user-defined listeners.
> This
> > > may
> > > >  block
> > > > > internal component notification (e.g. on discovery event)
> causing
> > > >  topology
> > > > > hangings.
> > > > >
> > > > 
> > > >  Sure. There are a lot of features being added. Would be nice to
> > > assign
> > > > >> a
> > > >  release manager for Ignite 2.0 and document all the discussed
> > > features
> > > > >> on
> > > >  the Wiki.
> > > > 
> > > > 
> > > > >
> > > > > --Yakov
> > > > >
> > > > > 2016-06-25 2:42 GMT+03:00 Alexey Goncharuk <
> > > > >> alexey.goncha...@gmail.com
> > > >  :
> > > > >
> > > > >> Folks,
> > > > >>
> > > > >> Recently I have seen a couple of emails suggesting
> > > > >> tasks/improvements
> > > > > that
> > > > >> we cannot do in 1.x releases due to API compatibility reasons,
> > so
> > > > >>> they
> > > > > are
> > > > >> postponed to 2.0. I would like to keep track of these tasks in
> > > some
> > > > >>> way
> > > > > in
> > > > >> our Jira to make sure we do not have anything obsolete when it
> > > > >> comes
> > > > >>> to
> > > > > the
> > > > >> next major version release.
> > > > >>
> > > > >> My question for now is how should we track such tasks? Should
> it
> > > > >> be a
> > > > >> label, a parent task with subtasks, something else?
> > > > >>
> > > > >> I would go with a label + release version.
> > > > >>
> > > > >> --AG
> > > > >>
> > > > >
> > > > 
> > > > >>>
> > > > >>
> > > >
> > > >
> > >
> > >
> > > --
> > > Alexey Kuznetsov
> > > GridGain Systems
> > > www.gridgain.com
> > >
> >
>
>
>
> --
> Sergey Kozlov
> GridGain Systems
> www.gridgain.com
>


Re: Ignite 2.0 tasks/roadmap

2016-08-11 Thread Sergey Kozlov
HI

Due to approach to disable to store more than one type per cache the cache
use becomes similar the table use for databases.
So I suppose would be good to introduce a database/schema-similar concept
for caches. It may be implemented as a non-default behavior for backward
compatibility.

On Sat, Aug 6, 2016 at 1:04 AM, Dmitriy Setrakyan 
wrote:

> On Fri, Aug 5, 2016 at 2:46 AM, Alexey Kuznetsov 
> wrote:
>
> > I remember couple more thing for 2.0
> >
> > How about to drop **ignite-scalar** module in Ignite 2.0?
> >
>
> Why?
>
>
> > And may be drop **ignite-spark-2.10** module support as of **Spark** 2 is
> > on **scala 2.11**.
> >
>
> I would drop it only if it is difficult to support. Do we know what kind of
> impact will it have on the community? Anyone is still using 2.10?
>
>
> >
> > On Tue, Aug 2, 2016 at 11:09 PM, Denis Magda 
> wrote:
> >
> > > Let’s add this [1] issue to the list because it requires significant
> work
> > > on the side of metrics SPI.
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-3495 <
> > > https://issues.apache.org/jira/browse/IGNITE-3495>
> > >
> > > —
> > > Denis
> > >
> > > > On Aug 2, 2016, at 12:45 AM, Yakov Zhdanov 
> > wrote:
> > > >
> > > > Not yet. The thing is that our recent experience showed that it was
> not
> > > > very good idea to go with caches. E.g. when you try to deserialize
> > inside
> > > > continuous query listener on client side it implicitly calls
> > cache.get()
> > > > which in turn may cause deadlock under some circumstances.
> > > >
> > > > --Yakov
> > > >
> > > > 2016-08-02 2:41 GMT+03:00 Dmitriy Setrakyan :
> > > >
> > > >> On Mon, Aug 1, 2016 at 3:46 AM, Yakov Zhdanov 
> > > wrote:
> > > >>
> > > >>> One more point.
> > > >>>
> > > >>> I insist on stop using marshaller and meta caches but switch to
> > > spreading
> > > >>> this info via custom discovery events.
> > > >>>
> > > >>
> > > >> Do we have a ticket explaining why this needs to be done?
> > > >>
> > > >>
> > > >>>
> > > >>> --Yakov
> > > >>>
> > > >>> 2016-07-27 19:57 GMT+03:00 Dmitriy Setrakyan <
> dsetrak...@apache.org
> > >:
> > > >>>
> > >  On Wed, Jul 27, 2016 at 11:36 AM, Yakov Zhdanov <
> > yzhda...@apache.org>
> > >  wrote:
> > > 
> > > > Guys, I think we can also split event notification for user
> > listeners
> > > >>> and
> > > > internal system listeners. I have been seeing a lot of issues
> > caused
> > > >> by
> > > > some heavy or blocking operations in user-defined listeners. This
> > may
> > >  block
> > > > internal component notification (e.g. on discovery event) causing
> > >  topology
> > > > hangings.
> > > >
> > > 
> > >  Sure. There are a lot of features being added. Would be nice to
> > assign
> > > >> a
> > >  release manager for Ignite 2.0 and document all the discussed
> > features
> > > >> on
> > >  the Wiki.
> > > 
> > > 
> > > >
> > > > --Yakov
> > > >
> > > > 2016-06-25 2:42 GMT+03:00 Alexey Goncharuk <
> > > >> alexey.goncha...@gmail.com
> > >  :
> > > >
> > > >> Folks,
> > > >>
> > > >> Recently I have seen a couple of emails suggesting
> > > >> tasks/improvements
> > > > that
> > > >> we cannot do in 1.x releases due to API compatibility reasons,
> so
> > > >>> they
> > > > are
> > > >> postponed to 2.0. I would like to keep track of these tasks in
> > some
> > > >>> way
> > > > in
> > > >> our Jira to make sure we do not have anything obsolete when it
> > > >> comes
> > > >>> to
> > > > the
> > > >> next major version release.
> > > >>
> > > >> My question for now is how should we track such tasks? Should it
> > > >> be a
> > > >> label, a parent task with subtasks, something else?
> > > >>
> > > >> I would go with a label + release version.
> > > >>
> > > >> --AG
> > > >>
> > > >
> > > 
> > > >>>
> > > >>
> > >
> > >
> >
> >
> > --
> > Alexey Kuznetsov
> > GridGain Systems
> > www.gridgain.com
> >
>



-- 
Sergey Kozlov
GridGain Systems
www.gridgain.com


[GitHub] ignite pull request #949: Ignite 2546

2016-08-11 Thread EdShangGG
GitHub user EdShangGG opened a pull request:

https://github.com/apache/ignite/pull/949

Ignite 2546



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/EdShangGG/ignite ignite-2546

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/949.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #949


commit 7d7e1489dbdf8038b8b76a81664e340c82a8d02e
Author: Valentin Kulichenko 
Date:   2016-08-09T03:15:49Z

IGNITE-2546 - Adding transformers to SCAN queries

commit 42666ad72674f2148bc950ca2412fd0237900ca5
Author: EdShangGG 
Date:   2016-08-11T13:41:41Z

IGNITE-2546 Need to introduce transformers to SCAN queries




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Created] (IGNITE-3677) IGFS: Add supported options feature to the secondary FS.

2016-08-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-3677:
---

 Summary: IGFS: Add supported options feature to the secondary FS.
 Key: IGNITE-3677
 URL: https://issues.apache.org/jira/browse/IGNITE-3677
 Project: Ignite
  Issue Type: Task
  Components: IGFS
Affects Versions: 1.6
Reporter: Vladimir Ozerov
 Fix For: 2.0


Various implementations of secondary file systems might or might not support 
certain options. We need to have a method on secondary API to check what is 
possible and what is not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: .Net's QueryExample mixes everything in a single cache

2016-08-11 Thread Pavel Tupitsyn
Thanks Denis, done.

On Thu, Aug 11, 2016 at 2:43 AM, Denis Magda  wrote:

> Pavel, here is the ticket
> https://issues.apache.org/jira/browse/IGNITE-3675
>
> —
> Denis
>
> On Aug 9, 2016, at 11:40 PM, Sergi Vladykin 
> wrote:
>
> Yes, it is a good idea, because I think in 2.0 we will disallow having more
> than a single type in a cache.
>
> Sergi
>
> 2016-08-10 9:35 GMT+03:00 Vladimir Ozerov :
>
> I do not know the reason. Let's split.
>
> On Wed, Aug 10, 2016 at 9:05 AM, Pavel Tupitsyn 
> wrote:
>
> Hi Denis,
>
> It has always been this way, not sure why. Vladimir can probably clarify.
> I agree that this is an anti-pattern, feel free to create a ticket.
>
> Pavel.
>
> On Wed, Aug 10, 2016 at 7:34 AM, Dmitriy Setrakyan <
>
> dsetrak...@apache.org>
>
> wrote:
>
> Denis, good catch! I would also prefer to split Orgs and Employees into
> different caches, as putting them into the same cache is rather an
> anti-pattern.
>
>
> On Tue, Aug 9, 2016 at 6:28 PM, Denis Magda 
>
> wrote:
>
>
> Pavel,
>
> I found out today that our .Net’s QueryExample mixes data for both
> Employee and Organization entities in a single cache. Is there any
> significant reasons for that?
>
> Basically I would prefer demonstrating how data can be logically
>
> divided
>
> among different caches and that cross-cache queries works out of the
>
> box.
>
> You can refer to the same example implemented for Java and C++ for more
> details.
>
> Are you ok with suggested improvement of the example? Do you want me to
> open a JIRA ticket?
>
> —
> Denis
>
>
>
>
>
>
>


[GitHub] ignite pull request #948: IGNITE-3676 Fix the test IgfsDualAbstractSelfTest#...

2016-08-11 Thread tledkov-gridgain
GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/948

IGNITE-3676 Fix the test IgfsDualAbstractSelfTest#testOpenNoPrefetch



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-3676

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/948.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #948


commit 3eb77c06d1dfd174cc00b221cceda4c6d8fc0121
Author: tledkov-gridgain 
Date:   2016-08-11T10:46:01Z

IGNITE-3676 Fix the test IgfsDualAbstractSelfTest#testOpenNoPrefetch




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] ignite pull request #947: Ignite 3676

2016-08-11 Thread tledkov-gridgain
Github user tledkov-gridgain closed the pull request at:

https://github.com/apache/ignite/pull/947


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] ignite pull request #947: Ignite 3676

2016-08-11 Thread tledkov-gridgain
GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/947

Ignite 3676



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-3676

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/947.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #947


commit d0f7252caa1b5b7ec388db6d788e1a05061dd1e8
Author: vozerov-gridgain 
Date:   2016-08-08T10:17:49Z

IGNITE-3590: Initial implementation.

commit c358487b76dfb4662a913ec029814546359eb2b3
Author: vozerov-gridgain 
Date:   2016-08-08T10:27:30Z

Minors.

commit 5930806eef48593c1441bda0d0234376f3f8b34f
Author: vozerov-gridgain 
Date:   2016-08-08T10:28:59Z

Minors.

commit 2f95ce875d8b8eaf86b4ffc73da9fcef13807161
Author: vozerov-gridgain 
Date:   2016-08-08T10:33:44Z

TODOs.

commit b7e2a33570185f7e39c23c57743e7b014c4039bd
Author: vozerov-gridgain 
Date:   2016-08-08T10:40:03Z

TODOs.

commit 569be691e066d1d86eed5fe2599a57629b03f471
Author: vozerov-gridgain 
Date:   2016-08-08T10:42:27Z

TODOs.

commit 24229d450c2bf17ab9f5d74910db5f1cdb49eb1b
Author: vozerov-gridgain 
Date:   2016-08-08T10:50:21Z

TODOs.

commit 1b67b9ce60d83ad653817485130ed76f41c8347c
Author: vozerov-gridgain 
Date:   2016-08-08T15:29:55Z

Fix for work dir.

commit 427955bac52e2c6cedb5fa7eb429c14c503855e3
Author: tledkov-gridgain 
Date:   2016-08-09T07:20:03Z

IGNITE-3638: Implemented rename.

commit 861134bd320d34956933c1475939a9092545b7e8
Author: vozerov-gridgain 
Date:   2016-08-09T07:48:51Z

Merge branch 'ignite-1.7.2' into ignite-1926

commit ba0088e3b977ee006d7621dd4771719c027f1b8b
Author: tledkov-gridgain 
Date:   2016-08-10T07:12:12Z

IGNITE-3649 IGFS: Local secondary: "create" flag is now used for append() 
operation. This closes #938.

commit 4bfa08d0290ccda6a1b638fef21ee551afe709f9
Author: tledkov-gridgain 
Date:   2016-08-10T07:14:37Z

IGNITE-3644 IGFS: Local secondary: Implemented exists() operation. This 
closes #935.

commit 3eb77c06d1dfd174cc00b221cceda4c6d8fc0121
Author: tledkov-gridgain 
Date:   2016-08-11T10:46:01Z

IGNITE-3676 Fix the test IgfsDualAbstractSelfTest#testOpenNoPrefetch




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Created] (IGNITE-3676) Fix the test IgfsDualAbstractSelfTest#testOpenNoPrefetch

2016-08-11 Thread Taras Ledkov (JIRA)
Taras Ledkov created IGNITE-3676:


 Summary: Fix the test IgfsDualAbstractSelfTest#testOpenNoPrefetch
 Key: IGNITE-3676
 URL: https://issues.apache.org/jira/browse/IGNITE-3676
 Project: Ignite
  Issue Type: Sub-task
  Components: IGFS
Affects Versions: 1.6
Reporter: Taras Ledkov
Assignee: Taras Ledkov
 Fix For: 1.8


The test *IgfsDualAbstractSelfTest#testOpenNoPrefetch* should be modified for 
local filesystem. The most local filesystems allow operations with the opened 
file after the file is deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] ignite pull request #946: Ignite 2560

2016-08-11 Thread AMashenkov
GitHub user AMashenkov opened a pull request:

https://github.com/apache/ignite/pull/946

Ignite 2560



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2560

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/946.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #946


commit 9ae083ce24dea51d7d941052e58e38bf879095c6
Author: Andrey V. Mashenkov 
Date:   2016-08-10T14:52:16Z

Ingite-2560 rebased to 1.6.5

commit c2568d5ff1a07a5c3ba15d09105e164870222087
Author: Andrey V. Mashenkov 
Date:   2016-08-10T17:31:56Z

IGNITE-2560: minor changes




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


Re: Distributed Joins Documentation

2016-08-11 Thread Pavel Tupitsyn
Great blog post, Denis!

One nitpick: do not use periods in the headings :)

I'll update the .NET docs shortly.

Pavel.

On Thu, Aug 11, 2016 at 2:31 AM, Denis Magda  wrote:

> Igniters,
>
> As most of you know recently we have released Ignite 1.7.0 that has the
> distributed joins functionality. I’ve added the documentation and blogged
> about this feature
> https://apacheignite.readme.io/docs/sql-queries#distributed-joins <
> https://apacheignite.readme.io/docs/sql-queries#distributed-joins>
> http://dmagda.blogspot.com/2016/08/big-change-in-apache-
> ignite-17-welcome.html  2016/08/big-change-in-apache-ignite-17-welcome.html>
>
> Feel free to provide your comments and fix the documentation if it can be
> improved.
>
> Finally, Pavel and Igor please do the same for .Net and C++ counterparts
> https://issues.apache.org/jira/browse/IGNITE-3673 <
> https://issues.apache.org/jira/browse/IGNITE-3673>
> https://issues.apache.org/jira/browse/IGNITE-3674 <
> https://issues.apache.org/jira/browse/IGNITE-3674>
>
> —
> Denis