Re: GridGain Donates Persistent Distributed Store To ASF (Apache Ignite)

[Dmitriy Setrakyan]

Cos, good questions! My answers are inline...

On Tue, Apr 18, 2017 at 4:17 PM, Konstantin Boudnik  wrote:

> Great news indeed! Thanks for sharing!
>
> Before we jump on the voting and all that, can we have a chance to learn
> more
> about this new feature and its integration points with the rest of the
> platform? A few questions come to mind immediately:
>
> 1. This is an "optional disk layer", so it could be turned off
> _completely_ and
>have no effect on those who don't want nor need to use it, right?
>

Yes


> 2. Does it have any performance implications on the in-memory operations?
>

No, as long as the persistence is turned off, the in-memory operations will
not be impacted.


> 3. When you say it is "fully ... ANSI-99 SQL compliant fault-tolerant"
> does it
>mean that _all_ SQL operations are now now supported through SQL or
> still
>some of them only available through the JAVA APIs? THe fault tolerance
> is
>for the data-center only as before, right? No new WAN-able HA has been
>introduced?
>

Well... I would say most SQL operations are going to be supported,
including CREATE, DROP, INSERT, UPDATE, DELETE, SELECT, and of course,
distributed joins.

And yes, you are right about the fault tolerance.


> 4. With addition of this new model, are there any backward compatibility
>issues that would affect Ignite's application developers?
>

I don't think so. All incompatible changes should have been introduced in
2.0. I will let other community members comment here...


> 5. Can we have a discussion about the design of this new layer so people
> here
>can understand better what's being offered, how to take the advantage of
>it, and - most importantly - to offer their own insights and
> improvements
>into this new subsystems before it's landed in the source code? And it
>would safe a lot of time on Q as well.
>

Yes, good idea. Denis, do you have a high level architecture for the
proposed persistence?


>
> I am confused a little bit by these two slightly controversial statements:
> - "GG... has been developing a unique distributed persistent store...for
> more
>   than a year in-house"
> - "we decided at GridGain that this tremendous feature should be open
> source
>   from the very beginning"
>
> So, it sounds like the code has been under the development for a while and
> it
> isn't opened up "from the very beginning", unless there's a new meaning of
> the
> word beginning I am not aware of just yet :) It feels like this could be a
> significant amount of the code to be digested by the community.
>

Yes, you are right. Many of us wanted to open source this functionality
from the get go. In any case, this makes a great addition to the project. I
hope we will be able to provide enough documentation and feedback on the
dev list to ease up the digestion process.


>
> Appreciate your thoughts on this! Thanks,
>   Cos
>
> On Mon, Apr 17, 2017 at 07:37PM, Denis Magda wrote:
> > Igniters,
> >
> > GridGain, as one of the most active Apache Ignite contributors, has been
> > developing a unique distributed persistent store specifically for Apache
> > Ignite for more than a year in-house. It’s a fully ACID and ANSI-99 SQL
> > compliant fault-tolerant solution.
> >
> > The store transparently integrates with Apache Ignite as an optional disk
> > layer (in addition to the existing RAM layer) via the new page memory
> > architecture that to be released in Apache Ignite 2.0. This allows
> storing
> > supersets of data on disk while having a subset in memory not worrying
> about
> > that you forgot to preload (warmup) your caches!
> >
> > Assuming that the storage goes to ASF as a part of Apache Ignite 2.1
> release
> > the following will be supported by Ignite out-of-the-box:
> >
> > * SQL queries execution over the data that is both in RAM and on disk: no
> > need to preload the whole data set in memory.
> >
> > * Cluster instantaneous restarts: once your cluster ring is recovered
> after
> > a restart your applications are fully operational even if they highly
> > utilize SQL queries.
> >
> > As for the use cases, it means that Apache Ignite will be possible to
> use as
> > a distributed SQL database with memory-first concept.
> >
> > And we decided at GridGain that this tremendous feature should be open
> > source from the very beginning.
> >
> > Guys, could you advise how I can start official donation process?
> >
> > —
> > Denis
> >
> >
> >
> >
> >
> >
>

[jira] [Created] (IGNITE-5021) visorcmd: log command - incorrect chronology in file

[Pavel Konstantinov (JIRA)]

Pavel Konstantinov created IGNITE-5021:
--

 Summary: visorcmd: log command - incorrect chronology in file
 Key: IGNITE-5021
 URL: https://issues.apache.org/jira/browse/IGNITE-5021
 Project: Ignite
  Issue Type: Bug
Reporter: Pavel Konstantinov
Priority: Minor


Events are configured in the grid
{code}











{code}

log -l -t=1
{code}
04/19/17, 11:42:25 | Log started.
04/19/17, 11:42:26 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:27 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:28 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:29 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:30 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:31 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:32 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:33 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:34 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:35 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:35 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:35 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:35 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:35 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:35 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:35 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:35 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:36:57 |  => NODE_LEFT: id8=66e8fb9f, ip=0:0:0:0:0:0:0:1
04/19/17, 11:37:05 |  => NODE_JOINED: id8=c0031666, ip=0:0:0:0:0:0:0:1
04/19/17, 11:38:02 |  => NODE_LEFT: id8=be032f32, ip=0:0:0:0:0:0:0:1
04/19/17, 11:38:07 |  => CLIENT_NODE_DISCONNECTED: id8=12dfc0d3, 
ip=0:0:0:0:0:0:0:1
04/19/17, 11:38:14 |  => CLIENT_NODE_RECONNECTED: id8=12dfc0d3, 
ip=0:0:0:0:0:0:0:1
04/19/17, 11:38:18 |  => NODE_JOINED: id8=f9466d50, ip=0:0:0:0:0:0:0:1
04/19/17, 11:38:19 |  => NODE_JOINED: id8=3a0dadb7, ip=0:0:0:0:0:0:0:1
04/19/17, 11:42:36 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:37 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:38 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:39 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:40 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:41 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:42 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:43 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:44 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:45 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:46 | H/N/C|1   |3   |8   |^...|
04/19/17, 11:42:47 | H/N/C|1   |3   |8   |^...|

{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5020) Client node failed with NPE

[Pavel Konstantinov (JIRA)]

Pavel Konstantinov created IGNITE-5020:
--

 Summary: Client node failed with NPE
 Key: IGNITE-5020
 URL: https://issues.apache.org/jira/browse/IGNITE-5020
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.0
Reporter: Pavel Konstantinov
Priority: Blocker
 Fix For: 2.0


{code}
[11:12:58,181][INFO][tcp-client-disco-msg-worker-#5%tester%][TcpDiscoverySpi] 
Client node disconnected from cluster, will try to reconnect with new id 
[newId=62cb8b71-4fe1-4fc7-b60b-236063ce9571, 
prevId=a48cdc72-d9b2-46db-b07f-3250610df1f8, locNode=TcpDiscoveryNode 
[id=a48cdc72-d9b2-46db-b07f-3250610df1f8, addrs=[0:0:0:0:0:0:0:1, 127.0.0.1, 
192.168.1.40, 192.168.56.1], sockAddrs=[/192.168.1.40:0, /0:0:0:0:0:0:0:1:0, 
/127.0.0.1:0, /192.168.56.1:0], discPort=0, order=4, intOrder=0, 
lastExchangeTime=1492575133264, loc=true, ver=2.0.0#20170418-sha1:6a1d263d, 
isClient=true]]
[11:12:59,580][INFO][sys-#29%tester%][GridCacheProcessor] Stopped cache: 
igfs-data
[11:12:59,586][INFO][sys-#34%tester%][GridCacheProcessor] Stopped cache: 
offheap-sorted-100
[11:12:59,588][INFO][sys-#31%tester%][GridCacheProcessor] Stopped cache: 
c_replicated
[11:12:59,591][INFO][sys-#30%tester%][GridCacheProcessor] Stopped cache: 
c_partitioned2
[11:12:59,591][INFO][sys-#31%tester%][GridCacheProcessor] Stopped cache: 
offheap-fifo-100
[11:12:59,597][INFO][sys-#34%tester%][GridCacheProcessor] Stopped cache: 
oh_partitioned
[11:12:59,597][INFO][sys-#29%tester%][GridCacheProcessor] Stopped cache: 
near-lru-100
[11:12:59,601][INFO][sys-#35%tester%][GridCacheProcessor] Stopped cache: 
ttl_partitioned
[11:12:59,603][INFO][sys-#32%tester%][GridCacheProcessor] Stopped cache: 
c_partitioned3
[11:12:59,603][INFO][sys-#33%tester%][GridCacheProcessor] Stopped cache: 
offheap-lru-100
[11:12:59,604][INFO][sys-#27%tester%][GridCacheProcessor] Stopped cache: 
near-sorted-100
[11:12:59,608][INFO][sys-#32%tester%][GridCacheProcessor] Stopped cache: 
c_partitioned
[11:12:59,608][SEVERE][exchange-worker-#36%tester%][GridDhtPartitionsExchangeFuture]
 Failed to reinitialize local partitions (preloading will be stopped): 
GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=7, 
minorTopVer=0], nodeId=62cb8b71, evt=NODE_JOINED]
java.lang.NullPointerException
at 
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.fetchAffinityOnJoin(CacheAffinitySharedManager.java:953)
at 
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onClientEvent(CacheAffinitySharedManager.java:649)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onClientNodeEvent(GridDhtPartitionsExchangeFuture.java:739)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:549)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1806)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:745)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Stable binary key representation

[Andrey Mashenkov]

Denis, I'll rework PR according new solution.

Alex G, Sergi, what approach is used for keys comparison in ignite 2.0 ?

On Tue, Apr 18, 2017 at 11:11 PM, Denis Magda  wrote:

> At all, guys, BinaryIdentityResolvers were discontinued but the ticket [1]
> that had triggered the discussion has not been fixed yet.
>
> It must be fixed in 2.0 otherwise Hibernate integration can be considered
> broken in 2.0 because the initial workaround was based on the resolvers.
>
> Andrey M., will you finalize it. Alex G. and Sergi can suggest
> non-resolvers based solution.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-3429 <
> https://issues.apache.org/jira/browse/IGNITE-3429>
>
> —
> Denis
>
> > On Apr 11, 2017, at 12:06 PM, Denis Magda  wrote:
> >
> > I don’t see either unless a key’s field is of a float type. However, it
> sounds like an artificial use case.
> >
> > Thanks for the details.
> >
> > —
> > Denis
> >
> >> On Apr 11, 2017, at 11:50 AM, Dmitriy Setrakyan 
> wrote:
> >>
> >> Denis, I think it is important that we know which specific field to use
> for
> >> the affinity resolution, but I don't see any issue in using both,
> primary
> >> and foreign keys, for hashcode and equality. Do you?
> >>
> >> D.
> >>
> >> On Tue, Apr 11, 2017 at 11:46 AM, Igor Sapego 
> wrote:
> >>
> >>> Denis,
> >>>
> >>> The whole binary representation of the object is used now
> >>> for hash code generation and equality comparison. So the
> >>> answer - all fields are used for this.
> >>>
> >>> Best Regards,
> >>> Igor
> >>>
> >>> On Mon, Apr 10, 2017 at 9:50 PM, Denis Magda 
> wrote:
> >>>
>  Considering this simple example
> 
>  INSERT (id, orgId, name, age, address) into Person…
> 
>  where id and orgId define Person’s affinity key - PersonKey(id, orgId)
> 
>  How do we know which fields to use for hash code generation and
> equality
>  comparison? QueryEntity?
> 
>  No, it’s unclear how to document it properly.
> 
>  —
>  Denis
> 
> > On Apr 10, 2017, at 11:14 AM, Vladimir Ozerov 
>  wrote:
> >
> > There is no more such resolver. It was removed.
> >
> > On Mon, Apr 10, 2017 at 8:58 PM, Denis Magda 
> >>> wrote:
> >
> >> Vovan,
> >>
> >> Before I fix the documentation, what’t the replacement for
> >> BinaryFieldIdentiyResolver we used to define field for hash code
> >> calculation and equality comparison when DML statements are used?
> >> https://apacheignite.readme.io/docs/binary-marshaller#
> >> section-binary-field-identity-resolver  .
> >> io/docs/binary-marshaller#section-binary-field-identity-resolver>
> >>
> >> —
> >> Denis
> >>
> >>> On Apr 9, 2017, at 7:39 AM, Vladimir Ozerov 
> >> wrote:
> >>>
> >>> Resolvers were essential for DML because we had broken comparison
> >> semantics
> >>> of binary objects. This is not the case now.
> >>>
> >>> Resolver as a whole is normal practice. E.g. it is implemented in
> >>> .NET
>  on
> >>> core language level and widely used in many cases. Hazelcast has it
> >>> as
> >> well
> >>> AFAIK. So it is wrong to think that the whole idea is useless.
> Think
> >>> of
> >> it
> >>> as a comparator's brother.
> >>>
> >>> The only reason why we need to remove it is missing hash index in
> new
> >>> architecture. It makes sense, as it is better to have AI 2.0
> without
> >> them,
> >>> than no AI 2.0 :-)
> >>>
> >>> 09 апр. 2017 г. 17:31 пользователь "Sergi Vladykin" <
> >>> sergi.vlady...@gmail.com> написал:
> >>>
>  I guess Resolvers were added to DML just because they already
> >>> existed
> >> since
>  1.9 and we were forced to support them in all the parts of our
>  product.
> 
>  We have to stop this practice to add features without clear real
> >>> life
> >> use
>  cases for them.
> 
>  Sergi
> 
>  2017-04-09 17:00 GMT+03:00 Denis Magda :
> 
> > Sergi, Vovan,
> >
> > Sorry for being annoying but I still didn't get an answer on
> >>> whether
> >> the
> > resolvers are the must for DML. The main reason why we made them
> up
> >> some
> > time ago is to support specific DML use cases. However I can't
> >>> recall
> >> the
> > use cases.
> >
> > --
> > Denis
> >
> > On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <
> >> sergi.vlady...@gmail.com
> >
> > wrote:
> >
> >> Ok, we need to do 2 things here:
> >>
> >> 1. Drop the resolvers from the source code.
> >> 2. Write a good page in docs on "What makes a 

Re: Webinar: Apache Ignite as a backbone for microservices-based architectures

[Rishi Yagnik]

Thanks, Denis.

I will surely join the conference.

Take Care,
Rishi

> On Apr 18, 2017, at 7:38 PM, Denis Magda  wrote:
> 
> Igniters,
> 
> Just a final reminder for those who might be interested. The slides are 
> ready, the demo is running..so waiting for all of you tomorrow April 19, 
> 11:00 AM PST on the line ;)
> http://bit.ly/2nEmyv1
> 
> Plus, look at our “news” section for upcoming conferences where the community 
> is going to present Ignite:
> https://ignite.apache.org/news.html
> 
> —
> Denis
> 
>> On Apr 12, 2017, at 6:35 PM, Denis Magda  wrote:
>> 
>> Dear community,
>> 
>> Welcome you to join the next Apache Ignite related webinar where you can 
>> learn more about Service Grid component and how to apply it for 
>> microservices-based solutions.
>> 
>> Description and registration: http://bit.ly/2nEmyv1  
>> 
>> Tweet: https://twitter.com/ApacheIgnite/status/852165280700796928
>> 
>> *Prachi*, would you add the event to the news feed?
>> https://ignite.apache.org/news.html 
>> 
>> —
>> Denis
>> 
>> 
> 

Re: Webinar: Apache Ignite as a backbone for microservices-based architectures

[Denis Magda]

Igniters,

Just a final reminder for those who might be interested. The slides are ready, 
the demo is running..so waiting for all of you tomorrow April 19, 11:00 AM PST 
on the line ;)
http://bit.ly/2nEmyv1

Plus, look at our “news” section for upcoming conferences where the community 
is going to present Ignite:
https://ignite.apache.org/news.html

—
Denis

> On Apr 12, 2017, at 6:35 PM, Denis Magda  wrote:
> 
> Dear community,
> 
> Welcome you to join the next Apache Ignite related webinar where you can 
> learn more about Service Grid component and how to apply it for 
> microservices-based solutions.
> 
> Description and registration: http://bit.ly/2nEmyv1  
> Tweet: https://twitter.com/ApacheIgnite/status/852165280700796928
> 
> *Prachi*, would you add the event to the news feed?
> https://ignite.apache.org/news.html 
> 
> —
> Denis
> 
> 

No direct way to download Ignite Web Agent

[Prachi Garg]

Engineers,

I realized that from the Web Console there is no direct way to download
Ignite Web Agent. The only way to download it is by going to either Model,
Monitoring, or Queries page (It's odd that to download the web agent, you
have to go to model -> import from db-> and then download when prompted).
Since Ignite web agent is required for almost everything on the Web
Console, shouldn't we have an option to download it from somewhere in the
top menu or side menu?

-Prachi

Re: GridGain Donates Persistent Distributed Store To ASF (Apache Ignite)

[Konstantin Boudnik]

Great news indeed! Thanks for sharing!

Before we jump on the voting and all that, can we have a chance to learn more
about this new feature and its integration points with the rest of the
platform? A few questions come to mind immediately:

1. This is an "optional disk layer", so it could be turned off _completely_ and
   have no effect on those who don't want nor need to use it, right?
2. Does it have any performance implications on the in-memory operations?
3. When you say it is "fully ... ANSI-99 SQL compliant fault-tolerant" does it
   mean that _all_ SQL operations are now now supported through SQL or still
   some of them only available through the JAVA APIs? THe fault tolerance is
   for the data-center only as before, right? No new WAN-able HA has been
   introduced?
4. With addition of this new model, are there any backward compatibility
   issues that would affect Ignite's application developers?
5. Can we have a discussion about the design of this new layer so people here
   can understand better what's being offered, how to take the advantage of
   it, and - most importantly - to offer their own insights and improvements
   into this new subsystems before it's landed in the source code? And it
   would safe a lot of time on Q as well.

I am confused a little bit by these two slightly controversial statements:
- "GG... has been developing a unique distributed persistent store...for more
  than a year in-house"
- "we decided at GridGain that this tremendous feature should be open source
  from the very beginning"

So, it sounds like the code has been under the development for a while and it
isn't opened up "from the very beginning", unless there's a new meaning of the
word beginning I am not aware of just yet :) It feels like this could be a
significant amount of the code to be digested by the community.

Appreciate your thoughts on this! Thanks,
  Cos

On Mon, Apr 17, 2017 at 07:37PM, Denis Magda wrote:
> Igniters,
> 
> GridGain, as one of the most active Apache Ignite contributors, has been
> developing a unique distributed persistent store specifically for Apache
> Ignite for more than a year in-house. It’s a fully ACID and ANSI-99 SQL
> compliant fault-tolerant solution. 
> 
> The store transparently integrates with Apache Ignite as an optional disk
> layer (in addition to the existing RAM layer) via the new page memory
> architecture that to be released in Apache Ignite 2.0. This allows storing
> supersets of data on disk while having a subset in memory not worrying about
> that you forgot to preload (warmup) your caches!
> 
> Assuming that the storage goes to ASF as a part of Apache Ignite 2.1 release
> the following will be supported by Ignite out-of-the-box:
> 
> * SQL queries execution over the data that is both in RAM and on disk: no
> need to preload the whole data set in memory.
> 
> * Cluster instantaneous restarts: once your cluster ring is recovered after
> a restart your applications are fully operational even if they highly
> utilize SQL queries.
> 
> As for the use cases, it means that Apache Ignite will be possible to use as
> a distributed SQL database with memory-first concept.
> 
> And we decided at GridGain that this tremendous feature should be open
> source from the very beginning.
> 
> Guys, could you advise how I can start official donation process?
> 
> —
> Denis
> 
> 
>  
> 
>
>  

[GitHub] ignite pull request #1825: lazy qry

[svladykin]

GitHub user svladykin opened a pull request:

https://github.com/apache/ignite/pull/1825

lazy qry



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/svladykin/ignite ignite-lazy-qry

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1825.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1825


commit 3df9268b560f2096aafc1f9994fdb0a6a283645c
Author: Sergi Vladykin 
Date:   2017-04-10T07:08:47Z

ignite-lazy-qry - wip

commit 8d1302891783c6e582caecd93cee7bb739e2381b
Author: Sergi Vladykin 
Date:   2017-04-18T06:49:48Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-lazy-qry

# Conflicts:
#   
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/twostep/GridReduceQueryExecutor.java

commit b1eb1bab62d24799c0a619eb977950bbfaeb9db4
Author: Sergi Vladykin 
Date:   2017-04-18T06:50:12Z

ignite-lazy-qry - minor

commit 553a91d76fb52384ac2f6b561c5f864010fbcd19
Author: Sergi Vladykin 
Date:   2017-04-18T08:52:19Z

ignite-lazy-qry - wip

commit ac94a27c661318bfe14a3ef50fd42c0decda50b7
Author: Sergi Vladykin 
Date:   2017-04-18T09:17:29Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-lazy-qry

commit 6fb564be6da6dfc79d8fb3324afd4fbbaa097a64
Author: Sergi Vladykin 
Date:   2017-04-18T11:43:40Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-lazy-qry

commit b6bfa6b8fee9d9fcb5cca77f5197efe410f9807c
Author: Sergi Vladykin 
Date:   2017-04-18T13:19:07Z

ignite-lazy-qry - wip

commit 59569ed16ae0f31d930bdc469cd5e4603a425f47
Author: Sergi Vladykin 
Date:   2017-04-18T14:33:20Z

ignite-lazy-qry - wippp

commit 2e4b3095fe19d1e60156e092dd156a260d61f297
Author: Sergi Vladykin 
Date:   2017-04-18T14:59:00Z

Merge branch 'ignite-2.0' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-lazy-qry

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/query/GridQueryIndexing.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/query/GridQueryProcessor.java
#   
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/IgniteH2Indexing.java
#   
modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GridH2Table.java
#   
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/h2/opt/GridH2TableSelfTest.java
#   
modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/h2/sql/GridQueryParsingTest.java

commit 9ca8b067e5b62ee9b61de67ddd383f1ff296bfa5
Author: Sergi Vladykin 
Date:   2017-04-18T14:59:50Z

ignite-lazy-qry - merge

commit 92afebb7936118975ae79b1085c91a91e25f6472
Author: Sergi Vladykin 
Date:   2017-04-18T15:09:33Z

ignite-lazy-qry - merge

commit 01b98c39ac5cd6a99479a43d5d01e5d3c0e9a0ed
Author: Sergi Vladykin 
Date:   2017-04-18T21:47:26Z

ignite-lazy-qry - refactor




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Introduce custom executors for compute grid

[Dmitriy Setrakyan]

On Tue, Apr 18, 2017 at 1:49 PM, Vladimir Ozerov 
wrote:

> Yes, task will be routed to public pool with a warning to the log.
>

Let's make sure we throttle these warnings for every computation so we do
not flood the logs.


>
> On Tue, Apr 18, 2017 at 9:21 PM, Dmitriy Setrakyan 
> wrote:
>
> > Christos,
> >
> > I think you are right in general, however, is the executor service is not
> > found, there should not be an exception, we should default to the public
> > pool.
> >
> > D.
> >
> > On Tue, Apr 18, 2017 at 8:53 AM, Christos Erotocritou <
> > chris...@gridgain.com
> > > wrote:
> >
> > > Taras,
> > >
> > > If I’m understanding right:
> > >
> > > Executor is configured local to a node, if same configuration is used
> to
> > > start multiple nodes then they will all have the same executor hence a
> > > broadcast task would execute on both all nodes. If one of those nodes
> > > didn’t have this executor configured then task on that node will throw
> an
> > > exception. Now in a scenario where 2 nodes do not have same executor
> and
> > we
> > > just use compute apply api call to execute task on next available node
> > > chances are it can end up on a node without that executor, correct? So
> we
> > > don’t have a way of routing tasks automatically to nodes with specific
> > > executors, we would have to use cluster groups to ensure that, correct?
> > >
> > > I think its important that we document all these points.
> > >
> > > C.
> > >
> > > > On 18 Apr 2017, at 16:12, Taras Ledkov  wrote:
> > > >
> > > > Hi, Christos.
> > > >
> > > > 1. The custom executor executor is configured for specified node.
> There
> > > is not restriction that guaranties the similar set of executors on the
> > > whole cluster.
> > > >
> > > > 2. I guess You can use ClusterGroup.forPredicate() and custom node
> > > attribute when you want to submit the job only to nodes where the
> > specific
> > > executor is configured.
> > > >
> > > >
> > > > On 18.04.2017 18:00, Christos Erotocritou wrote:
> > > >> I assume its not currently possible to constrain an executor pool to
> > be
> > > started only on a specific node?
> > > >
> > > > --
> > > > Taras Ledkov
> > > > Mail-To: tled...@gridgain.com
> > > >
> > >
> > >
> >
>

Re: GridGain Donates Persistent Distributed Store To ASF (Apache Ignite)

[Denis Magda]

Igniters,

I would encourage everyone interested to share his/her opinion on this donation 
and if there are no objections I will kick off an official vote in a couple of 
ways.

—
Denis

> On Apr 18, 2017, at 4:11 AM, Nikita Ivanov  wrote:
> 
> Combining this with upcoming new SQL features should give Google Spanner a
> run for its money...
> --
> Nikita Ivanov
> 
> 
> On Tue, Apr 18, 2017 at 3:39 AM, christos  wrote:
> 
>> This is fantastic news guys!! Finally a real solution that can cater for
>> BIG
>> AND FAST data!
>> 
>> 
>> 
>> --
>> View this message in context: http://apache-ignite-
>> developers.2346864.n4.nabble.com/GridGain-Donates-
>> Persistent-Distributed-Store-To-ASF-Apache-Ignite-tp16788p16836.html
>> Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
>> 

Re: Introduce custom executors for compute grid

[Vladimir Ozerov]

Yes, task will be routed to public pool with a warning to the log.

On Tue, Apr 18, 2017 at 9:21 PM, Dmitriy Setrakyan 
wrote:

> Christos,
>
> I think you are right in general, however, is the executor service is not
> found, there should not be an exception, we should default to the public
> pool.
>
> D.
>
> On Tue, Apr 18, 2017 at 8:53 AM, Christos Erotocritou <
> chris...@gridgain.com
> > wrote:
>
> > Taras,
> >
> > If I’m understanding right:
> >
> > Executor is configured local to a node, if same configuration is used to
> > start multiple nodes then they will all have the same executor hence a
> > broadcast task would execute on both all nodes. If one of those nodes
> > didn’t have this executor configured then task on that node will throw an
> > exception. Now in a scenario where 2 nodes do not have same executor and
> we
> > just use compute apply api call to execute task on next available node
> > chances are it can end up on a node without that executor, correct? So we
> > don’t have a way of routing tasks automatically to nodes with specific
> > executors, we would have to use cluster groups to ensure that, correct?
> >
> > I think its important that we document all these points.
> >
> > C.
> >
> > > On 18 Apr 2017, at 16:12, Taras Ledkov  wrote:
> > >
> > > Hi, Christos.
> > >
> > > 1. The custom executor executor is configured for specified node. There
> > is not restriction that guaranties the similar set of executors on the
> > whole cluster.
> > >
> > > 2. I guess You can use ClusterGroup.forPredicate() and custom node
> > attribute when you want to submit the job only to nodes where the
> specific
> > executor is configured.
> > >
> > >
> > > On 18.04.2017 18:00, Christos Erotocritou wrote:
> > >> I assume its not currently possible to constrain an executor pool to
> be
> > started only on a specific node?
> > >
> > > --
> > > Taras Ledkov
> > > Mail-To: tled...@gridgain.com
> > >
> >
> >
>

Re: Stable binary key representation

[Denis Magda]

At all, guys, BinaryIdentityResolvers were discontinued but the ticket [1] that 
had triggered the discussion has not been fixed yet.

It must be fixed in 2.0 otherwise Hibernate integration can be considered 
broken in 2.0 because the initial workaround was based on the resolvers.

Andrey M., will you finalize it. Alex G. and Sergi can suggest non-resolvers 
based solution.

[1] https://issues.apache.org/jira/browse/IGNITE-3429 


—
Denis

> On Apr 11, 2017, at 12:06 PM, Denis Magda  wrote:
> 
> I don’t see either unless a key’s field is of a float type. However, it 
> sounds like an artificial use case.
> 
> Thanks for the details.
> 
> —
> Denis
> 
>> On Apr 11, 2017, at 11:50 AM, Dmitriy Setrakyan  
>> wrote:
>> 
>> Denis, I think it is important that we know which specific field to use for
>> the affinity resolution, but I don't see any issue in using both, primary
>> and foreign keys, for hashcode and equality. Do you?
>> 
>> D.
>> 
>> On Tue, Apr 11, 2017 at 11:46 AM, Igor Sapego  wrote:
>> 
>>> Denis,
>>> 
>>> The whole binary representation of the object is used now
>>> for hash code generation and equality comparison. So the
>>> answer - all fields are used for this.
>>> 
>>> Best Regards,
>>> Igor
>>> 
>>> On Mon, Apr 10, 2017 at 9:50 PM, Denis Magda  wrote:
>>> 
 Considering this simple example
 
 INSERT (id, orgId, name, age, address) into Person…
 
 where id and orgId define Person’s affinity key - PersonKey(id, orgId)
 
 How do we know which fields to use for hash code generation and equality
 comparison? QueryEntity?
 
 No, it’s unclear how to document it properly.
 
 —
 Denis
 
> On Apr 10, 2017, at 11:14 AM, Vladimir Ozerov 
 wrote:
> 
> There is no more such resolver. It was removed.
> 
> On Mon, Apr 10, 2017 at 8:58 PM, Denis Magda 
>>> wrote:
> 
>> Vovan,
>> 
>> Before I fix the documentation, what’t the replacement for
>> BinaryFieldIdentiyResolver we used to define field for hash code
>> calculation and equality comparison when DML statements are used?
>> https://apacheignite.readme.io/docs/binary-marshaller#
>> section-binary-field-identity-resolver > io/docs/binary-marshaller#section-binary-field-identity-resolver>
>> 
>> —
>> Denis
>> 
>>> On Apr 9, 2017, at 7:39 AM, Vladimir Ozerov 
>> wrote:
>>> 
>>> Resolvers were essential for DML because we had broken comparison
>> semantics
>>> of binary objects. This is not the case now.
>>> 
>>> Resolver as a whole is normal practice. E.g. it is implemented in
>>> .NET
 on
>>> core language level and widely used in many cases. Hazelcast has it
>>> as
>> well
>>> AFAIK. So it is wrong to think that the whole idea is useless. Think
>>> of
>> it
>>> as a comparator's brother.
>>> 
>>> The only reason why we need to remove it is missing hash index in new
>>> architecture. It makes sense, as it is better to have AI 2.0 without
>> them,
>>> than no AI 2.0 :-)
>>> 
>>> 09 апр. 2017 г. 17:31 пользователь "Sergi Vladykin" <
>>> sergi.vlady...@gmail.com> написал:
>>> 
 I guess Resolvers were added to DML just because they already
>>> existed
>> since
 1.9 and we were forced to support them in all the parts of our
 product.
 
 We have to stop this practice to add features without clear real
>>> life
>> use
 cases for them.
 
 Sergi
 
 2017-04-09 17:00 GMT+03:00 Denis Magda :
 
> Sergi, Vovan,
> 
> Sorry for being annoying but I still didn't get an answer on
>>> whether
>> the
> resolvers are the must for DML. The main reason why we made them up
>> some
> time ago is to support specific DML use cases. However I can't
>>> recall
>> the
> use cases.
> 
> --
> Denis
> 
> On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <
>> sergi.vlady...@gmail.com
> 
> wrote:
> 
>> Ok, we need to do 2 things here:
>> 
>> 1. Drop the resolvers from the source code.
>> 2. Write a good page in docs on "What makes a correct cache key".
>> 
>> Who can do that?
>> 
>> Sergi
>> 
>> 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <
>>> sergi.vlady...@gmail.com
> :
>> 
>>> It is possible to try adding support of comparison to Resolvers,
 but
> the
>>> whole approach looks wrong and for now it is better to get rid of
 it
>> while
>>> we have a chance to break 

Re: Introduce custom executors for compute grid

[Dmitriy Setrakyan]

Christos,

I think you are right in general, however, is the executor service is not
found, there should not be an exception, we should default to the public
pool.

D.

On Tue, Apr 18, 2017 at 8:53 AM, Christos Erotocritou  wrote:

> Taras,
>
> If I’m understanding right:
>
> Executor is configured local to a node, if same configuration is used to
> start multiple nodes then they will all have the same executor hence a
> broadcast task would execute on both all nodes. If one of those nodes
> didn’t have this executor configured then task on that node will throw an
> exception. Now in a scenario where 2 nodes do not have same executor and we
> just use compute apply api call to execute task on next available node
> chances are it can end up on a node without that executor, correct? So we
> don’t have a way of routing tasks automatically to nodes with specific
> executors, we would have to use cluster groups to ensure that, correct?
>
> I think its important that we document all these points.
>
> C.
>
> > On 18 Apr 2017, at 16:12, Taras Ledkov  wrote:
> >
> > Hi, Christos.
> >
> > 1. The custom executor executor is configured for specified node. There
> is not restriction that guaranties the similar set of executors on the
> whole cluster.
> >
> > 2. I guess You can use ClusterGroup.forPredicate() and custom node
> attribute when you want to submit the job only to nodes where the specific
> executor is configured.
> >
> >
> > On 18.04.2017 18:00, Christos Erotocritou wrote:
> >> I assume its not currently possible to constrain an executor pool to be
> started only on a specific node?
> >
> > --
> > Taras Ledkov
> > Mail-To: tled...@gridgain.com
> >
>
>

Re: Adding ML to Ignite, IGNITE-4572

[oignatenko]

Denis,

As far as I can tell renames are now finalized in the way you proposed
yesterday. I re-run my checks for build, module unit tests and examples
(everything went fine) and updated draft doc for readme.io accordingly.

You can find most recent version of the draft doc attached to JIRA
IGNITE-4964

regards, Oleg


Denis Magda wrote
> Thanks Oleg, I’ll review the doc a bit later.
> 
> Yuri, I’ve reopened IGNITE-5000. Please consider my latest notes and
> prepare one more pull-request:
> https://issues.apache.org/jira/browse/IGNITE-5000
> https://issues.apache.org/jira/browse/IGNITE-5000;
> 
> —
> Denis
> 
>> On Apr 17, 2017, at 8:48 AM, oignatenko 

> oignatenko@

>  wrote:
>> 
>> Denis, I prepared preliminary draft doc for readme.io
>> http://readme.io/;, it reflects proposed
>> rename. You can find it attached to JIRA IGNITE-4964
>> 
>> With regards to merge, per my reading of recent mail from Yury it is done
>> except for module rename which is on its way to master now.
>> 
>> regards, Oleg
>> 
>> 
>> Denis Magda-2 wrote
>>> Oleg, Nikita, Yuri,
>>> 
>>> Frankly, I don’t like how the module is named presently which is
>>> 'ignite-math’.
>>> 
>>> I propose to rename it to ‘ignite-ml’. Presently, it holds a small
>>> fraction of all ML functionality we might have in the future and this is
>>> why, thinking of future, it will be a right decision to name it
>>> accordingly.
>>> 
>>> Any concerns?
>>> 
>>> BTW, have the module been fully merged? I want to try to build it and
>>> check the examples.
>>> 
>>> —
>>> Denis
>>> 
 On Apr 14, 2017, at 7:05 AM, oignatenko 
>> 
>>> oignatenko@
>> 
>>>  wrote:
 
 Thank you Denis, that sounds like a good plan. I am working on it now.
 I
 didn't reassign IGNITE-4964, just set a watch on it. Will comment on it
 when
 I have concrete details to share.
 
 Side note my build check completed successfully, although it took quite
 a
 bit of time: I cleaned my local maven repo to make sure that things
 really
 work from the clean state.
 
 
 
 --
 View this message in context:
 http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16677.html
 Sent from the Apache Ignite Developers mailing list archive at
 Nabble.com.
>> 
>> 
>> 
>> 
>> 
>> --
>> View this message in context:
>> http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16758.html
>> http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16758.html;
>> Sent from the Apache Ignite Developers mailing list archive at Nabble.com
>> http://nabble.com/;.





--
View this message in context: 
http://apache-ignite-developers.2346864.n4.nabble.com/Adding-ML-to-Ignite-IGNITE-4572-tp13936p16878.html
Sent from the Apache Ignite Developers mailing list archive at Nabble.com.

[GitHub] ignite pull request #1824: ignite-5019 Deadlock in GridCacheVariableTopology...

[DmitriyGovorukhin]

GitHub user DmitriyGovorukhin opened a pull request:

https://github.com/apache/ignite/pull/1824

ignite-5019 Deadlock in GridCacheVariableTopologySelfTest.testNodeStop



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5019

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1824.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1824


commit 44b1cbc36e0e19c42ecddd98040b8d7494256447
Author: Dmitriy Govorukhin 
Date:   2017-04-18T16:39:13Z

ignite-5019 Potentially  deadlock fix




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-5019) Deadlock in GridCacheVariableTopologySelfTest.testNodeStop

[Dmitriy Govorukhin (JIRA)]

Dmitriy Govorukhin created IGNITE-5019:
--

 Summary: Deadlock in GridCacheVariableTopologySelfTest.testNodeStop
 Key: IGNITE-5019
 URL: https://issues.apache.org/jira/browse/IGNITE-5019
 Project: Ignite
  Issue Type: Bug
  Components: general
Reporter: Dmitriy Govorukhin
Assignee: Dmitriy Govorukhin
 Fix For: 2.0






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5018) review and improve javadocs in ML module

[Oleg Ignatenko (JIRA)]

Oleg Ignatenko created IGNITE-5018:
--

 Summary: review and improve javadocs in ML module
 Key: IGNITE-5018
 URL: https://issues.apache.org/jira/browse/IGNITE-5018
 Project: Ignite
  Issue Type: Task
Reporter: Oleg Ignatenko
Assignee: Oleg Ignatenko
Priority: Minor
 Fix For: 2.0


Review and improve javadocs in Ignite ML module (added per IGNITE-4572). To 
name a few, add descriptions for constructor parameters in classes 
{{CacheMatrix}}, {{CacheMatrixStorage}}, {{RandomVector}}. Etc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Introduce custom executors for compute grid

[Christos Erotocritou]

D, I think this fixed with this ticket: 
https://issues.apache.org/jira/browse/IGNITE-4802 



> On 18 Apr 2017, at 16:30, Dmitriy Setrakyan  wrote:
> 
> Agree, very useful.
> 
> Does the service grid run in its own thread pool? Just wondering if this
> feature can be useful there as well.
> 
> D.
> 
> On Tue, Apr 18, 2017 at 7:55 AM, Denis Magda  wrote:
> 
>> Taras, that’s an excellent addition to the project!
>> 
>> Please don’t forget to document it: https://issues.apache.org/
>> jira/browse/IGNITE-4969 >> 
>> 
>> —
>> Denis
>> 
>>> On Apr 18, 2017, at 1:23 AM, Taras Ledkov  wrote:
>>> 
>>> Igniters,
>>> 
>>> Custom executor (user's thread pool) is added fro compute grid with
>> following semantics:
>>> 
>>> 1. Configuration:
>>> 
>>> IgniteConfiguration cfg;
>>> ...
>>> cfg.setExecutorConfiguration(
>>>   new ExecutorConfiguration().setName("executor0").setSize(2),
>>>   new ExecutorConfiguration().setName("executor1").setSize(4));
>>> 
>>> Where
>>> name - name of executor and thread pool;
>>> size - thread pool size.
>>> 
>>> 2. Usage:
>>> 
>>> Ignite ignite;
>>> ...
>>> IgniteCompute comp = ignite.compute().withExecutor("executor0");
>>> comp.broadcast(new IgniteRunnable() {
>>>   @Override public void run() {
>>>...
>>>   }
>>>   });
>>> 
>>> So, 'withExecutor(String)' returns the compute associated with custom
>> named executor.
>>> All jobs submitted by the components will be processed by thread pool
>> corresponds to named executor.
>>> If the executor isn't configured on the target host the warning will be
>> printed in the log and a job will be processed in the public pool.
>>> e.g.:
>>> [11:20:01,023][WARN ][grid-nio-worker-tcp-comm-0-#27%compute.
>> IgniteComputeCustomExecutorSelfTest1%][GridIoManager] Custom executor
>> 'invalid' doesn't exist. The job will be submit to public pool:
>> b2e85208b51-4fbcb569-07a2-480e-9be1-512bc320
>>> 
>>> Issue: https://issues.apache.org/jira/browse/IGNITE-4699
>>> 
>>> Please share your thoughts or ask questions.
>>> 
>>> --
>>> Taras Ledkov
>>> Mail-To: tled...@gridgain.com
>>> 
>> 
>> 

Re: Introduce custom executors for compute grid

[Christos Erotocritou]

Taras,

If I’m understanding right:

Executor is configured local to a node, if same configuration is used to start 
multiple nodes then they will all have the same executor hence a broadcast task 
would execute on both all nodes. If one of those nodes didn’t have this 
executor configured then task on that node will throw an exception. Now in a 
scenario where 2 nodes do not have same executor and we just use compute apply 
api call to execute task on next available node chances are it can end up on a 
node without that executor, correct? So we don’t have a way of routing tasks 
automatically to nodes with specific executors, we would have to use cluster 
groups to ensure that, correct?

I think its important that we document all these points.

C.

> On 18 Apr 2017, at 16:12, Taras Ledkov  wrote:
> 
> Hi, Christos.
> 
> 1. The custom executor executor is configured for specified node. There is 
> not restriction that guaranties the similar set of executors on the whole 
> cluster.
> 
> 2. I guess You can use ClusterGroup.forPredicate() and custom node attribute 
> when you want to submit the job only to nodes where the specific executor is 
> configured.
> 
> 
> On 18.04.2017 18:00, Christos Erotocritou wrote:
>> I assume its not currently possible to constrain an executor pool to be 
>> started only on a specific node?
> 
> -- 
> Taras Ledkov
> Mail-To: tled...@gridgain.com
> 

[GitHub] ignite pull request #1823: Ignite 4799

[abeliak]

GitHub user abeliak opened a pull request:

https://github.com/apache/ignite/pull/1823

Ignite 4799

Do not merge

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4799

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1823.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1823


commit 3129807585a1a968396d51bd63ff927c0d5c26b5
Author: Yakov Zhdanov 
Date:   2017-04-10T11:08:24Z

ignite-4799 - removed TCP Discovery heartbeat frequency property, fixed 
compilation and logic, fixed xml configs

commit 1a684e9e9856113d4577dd3a655902f80705a455
Author: Denis Magda 
Date:   2017-04-10T18:35:30Z

IGNITE-4799: altered IgniteConfiguration.metricsUpdateFrequency 
documentation.

commit a4ba8a9a18ab5d7f5ad0910b761ef8f46c83e884
Author: Pavel Tupitsyn 
Date:   2017-04-11T07:14:51Z

.NET: Remove TcpDiscoverySpi.HeartbeatFrequency

commit d2910dc5ac942433535dec5689532f2cbc87c277
Author: Alexander Belyak 
Date:   2017-04-17T16:11:39Z

Remove maxMissed*Heartbeats, rename TcpDiscoveryClientHeartbeatMessage to 
TDCMetrics message, add clientFailureDetectionTimeout

commit c72040a6ccb58dcbd1df8a0693c800dfa5c372b3
Author: Alexander Belyak 
Date:   2017-04-18T11:47:38Z

Remove -1 and 0 special values from 
IgniteConfiguration.metricsUpdateFrequency

commit 2fcb7b840816a86de9881795f0c5d22e7f8a68d9
Author: Alexander Belyak 
Date:   2017-04-18T12:52:12Z

Use clientFailureDetectionTimeout in ServerImpl, fix comment

commit cd593c19be65abaa1a1df5fc031f02a07d272799
Author: Alexander Belyak 
Date:   2017-04-18T13:30:19Z

Merge remote-tracking branch 'apache/master' into ignite-4799

commit dc13866824e7dedfc5f9e3b038068b03ec3bc2cf
Author: Yakov Zhdanov 
Date:   2017-04-18T14:19:03Z

ignite-4799 review

commit 3be940080a2564d8995981fb75d6f867b9b4c6a0
Author: Alexander Belyak 
Date:   2017-04-18T15:50:07Z

Rename MetricsMessage to MetricsUpdateMessage, muFreq to metricsUpdateFreq, 
fix metricsUpdateInterval




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: cache query operations

[Denis Magda]

I can’t answer on this question because didn’t develop that part of the system.

Could you try to find this out debugging the code?

—
Denis

> On Apr 18, 2017, at 8:00 AM, ALEKSEY KUZNETSOV  
> wrote:
> 
> So, when GridCacheTtlManager evicted cache entry due to timeout, how H2 got
> notified about it ?
> 
> And as i understand, firstly cacheEntry got evicted, and then H2 data got
> updated(removed corresponding entry)
> 
> вт, 18 апр. 2017 г. в 17:56, Denis Magda :
> 
>> That’s true, SELECT queries are executed right away by H2 while DML are
>> eventually converted to key-value operations.
>> 
>> —
>> Denis
>> 
>>> On Apr 18, 2017, at 1:29 AM, ALEKSEY KUZNETSOV 
>> wrote:
>>> 
>>> if executing sql "select" clause then no cache key-value operations would
>>> be called.
>>> Debbuging cache.query(...) leads
>>> to
>> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing#executeSqlQuery
>>> which doesnt call cache key-value ops
>>> 
>>> вт, 18 апр. 2017 г. в 10:49, ALEKSEY KUZNETSOV >> :
>>> 
 thanx! )
 
 пн, 17 апр. 2017 г. в 21:48, Denis Magda :
 
> Did you have a chance to take a look this documentation that explains
>> how
> DML statements are turned into cache operations?
> https://apacheignite.readme.io/docs/dml#dml-operations <
> https://apacheignite.readme.io/docs/dml#dml-operations>
> 
> —
> Denis
> 
>> On Apr 17, 2017, at 7:20 AM, ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com> wrote:
>> 
>> Hi, Igniters! When one fires query like this : ignite().cache(null
>> ).query(...)
>> The query would be executed against the schema. I wonder, how the
> schema is
>> synchronized with cache, with cache entries?
>> 
>> What i mean is , cache.query(...update operation...) must eventually
> update
>> the cache entry in cache. But how does it happen?(cache API must be
> called,
>> but i cannot realize how)
>> --
>> 
>> *Best Regards,*
>> 
>> *Kuznetsov Aleksey*
> 
> --
 
 *Best Regards,*
 
 *Kuznetsov Aleksey*
 
>>> --
>>> 
>>> *Best Regards,*
>>> 
>>> *Kuznetsov Aleksey*
>> 
>> --
> 
> *Best Regards,*
> 
> *Kuznetsov Aleksey*

Re: Introduce custom executors for compute grid

[Dmitriy Setrakyan]

Agree, very useful.

Does the service grid run in its own thread pool? Just wondering if this
feature can be useful there as well.

D.

On Tue, Apr 18, 2017 at 7:55 AM, Denis Magda  wrote:

> Taras, that’s an excellent addition to the project!
>
> Please don’t forget to document it: https://issues.apache.org/
> jira/browse/IGNITE-4969  >
>
> —
> Denis
>
> > On Apr 18, 2017, at 1:23 AM, Taras Ledkov  wrote:
> >
> > Igniters,
> >
> > Custom executor (user's thread pool) is added fro compute grid with
> following semantics:
> >
> > 1. Configuration:
> >
> > IgniteConfiguration cfg;
> > ...
> > cfg.setExecutorConfiguration(
> >new ExecutorConfiguration().setName("executor0").setSize(2),
> >new ExecutorConfiguration().setName("executor1").setSize(4));
> >
> > Where
> > name - name of executor and thread pool;
> > size - thread pool size.
> >
> > 2. Usage:
> >
> > Ignite ignite;
> > ...
> > IgniteCompute comp = ignite.compute().withExecutor("executor0");
> > comp.broadcast(new IgniteRunnable() {
> >@Override public void run() {
> > ...
> >}
> >});
> >
> > So, 'withExecutor(String)' returns the compute associated with custom
> named executor.
> > All jobs submitted by the components will be processed by thread pool
> corresponds to named executor.
> > If the executor isn't configured on the target host the warning will be
> printed in the log and a job will be processed in the public pool.
> > e.g.:
> > [11:20:01,023][WARN ][grid-nio-worker-tcp-comm-0-#27%compute.
> IgniteComputeCustomExecutorSelfTest1%][GridIoManager] Custom executor
> 'invalid' doesn't exist. The job will be submit to public pool:
> b2e85208b51-4fbcb569-07a2-480e-9be1-512bc320
> >
> > Issue: https://issues.apache.org/jira/browse/IGNITE-4699
> >
> > Please share your thoughts or ask questions.
> >
> > --
> > Taras Ledkov
> > Mail-To: tled...@gridgain.com
> >
>
>

Re: Introduce custom executors for compute grid

[Taras Ledkov]

Hi, Christos.

1. The custom executor executor is configured for specified node. There 
is not restriction that guaranties the similar set of executors on the 
whole cluster.


2. I guess You can use ClusterGroup.forPredicate() and custom node 
attribute when you want to submit the job only to nodes where the 
specific executor is configured.



On 18.04.2017 18:00, Christos Erotocritou wrote:

I assume its not currently possible to constrain an executor pool to be started 
only on a specific node?


--
Taras Ledkov
Mail-To: tled...@gridgain.com

Re: Apache Ignite 2.0 Release

[Vladimir Ozerov]

Denis,

As usual: merge to release branch at the very least. You can merge
ignite-2.0 -> master immediately then if you want, but it is not necessary.
ignite-2.0 will be merged to master eventually anyway.

On Tue, Apr 18, 2017 at 6:00 PM, Denis Magda  wrote:

> Vovan,
>
> Do I need to commit a 2.0 change to ignite-2.0 first and then merge it
> back to the master? What’s the process?
>
> —
> Denis
>
> > On Apr 18, 2017, at 5:13 AM, Vladimir Ozerov 
> wrote:
> >
> > Folks,
> >
> > I created branch for AI 2.0: ignite-2.0. Relevant PR #1819. Let's
> continue
> > 2.0 finalization in this branch.
> >
> > On Wed, Apr 12, 2017 at 7:58 AM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> > wrote:
> >
> >> Awesome! Great addition to the project.
> >>
> >> On Tue, Apr 11, 2017 at 8:27 PM, Denis Magda  wrote:
> >>
> >>> Spring Data integration caught up the train and will be released in
> 2.0:
> >>> https://issues.apache.org/jira/browse/IGNITE-1192 <
> >>> https://issues.apache.org/jira/browse/IGNITE-1192>
> >>>
> >>> —
> >>> Denis
> >>>
>  On Apr 9, 2017, at 6:57 AM, Denis Magda  wrote:
> 
>  Val, yes, as far as I know we still need to merge to the master. The
> >>> branch should be ready later the next week.
> 
>  Denis
> 
>  On Sat, Apr 8, 2017 at 6:14 PM, Valentin Kulichenko <
> >>> valentin.kuliche...@gmail.com >
> >>> wrote:
>  Guys,
> 
>  There is no branch for 2.0 right now, correct? If I want to add
> >> something
>  there, do I just merge to master?
> 
>  -Val
> 
>  On Sat, Apr 8, 2017 at 10:18 AM, Alexander Paschenko <
>  alexander.a.pasche...@gmail.com  >> gmail.com>>
> >>> wrote:
> 
> > I've fixed https://issues.apache.org/jira/browse/IGNITE-4354 <
> >>> https://issues.apache.org/jira/browse/IGNITE-4354>, PR is
> > https://github.com/apache/ignite/pull/1759 <
> >> https://github.com/apache/
> >>> ignite/pull/1759>.
> > Pavel, thank you very much for bringing that to my attention.
> >
> > - Alex
> >
> > 2017-04-07 20:28 GMT+03:00 Sergi Vladykin  >>> >:
> >> A bunch of SQL related tickets is delayed until H2 release on the
> >>> next
> > week.
> >>
> >> Sergi
> >>
> >> 2017-04-07 19:36 GMT+03:00 Alexey Goncharuk <
> >>> alexey.goncha...@gmail.com 
> >> :
> >>
> >>> Folks,
> >>>
> >>> The status of ignite-3477 is as follows: we've fixed almost all
> >>> tests
> > and
> >>> currently waiting for ignite-4535 and ignite-4534 to be merged,
> >>> which
> > will
> >>> happen once TC for those branches is completed. This should happen
> >>> over
> > the
> >>> weekend.
> >>>
> >>> 2017-04-06 19:12 GMT+03:00 Alexey Kuznetsov <
> >> akuznet...@apache.org
> >>> >:
> >>>
>  I've prepared
> 
>  IGNITE-4349  >> <
> >>> https://issues.apache.org/jira/browse/IGNITE-4349>> -
>  Discontinue the schema-import utility
> 
>  Anton, could you review my changes related to Ignite *build*?
>  See: http://reviews.ignite.apache.org/ignite/review/IGNT-CR-163
> >> <
> >>> http://reviews.ignite.apache.org/ignite/review/IGNT-CR-163>
> 
> 
>  --
>  Alexey Kuznetsov
> 
> >>>
> >
> 
> >>>
> >>>
> >>
>
>

Re: Apache Ignite 2.0 Release

[Denis Magda]

Vovan,

Do I need to commit a 2.0 change to ignite-2.0 first and then merge it back to 
the master? What’s the process?

—
Denis

> On Apr 18, 2017, at 5:13 AM, Vladimir Ozerov  wrote:
> 
> Folks,
> 
> I created branch for AI 2.0: ignite-2.0. Relevant PR #1819. Let's continue
> 2.0 finalization in this branch.
> 
> On Wed, Apr 12, 2017 at 7:58 AM, Dmitriy Setrakyan 
> wrote:
> 
>> Awesome! Great addition to the project.
>> 
>> On Tue, Apr 11, 2017 at 8:27 PM, Denis Magda  wrote:
>> 
>>> Spring Data integration caught up the train and will be released in 2.0:
>>> https://issues.apache.org/jira/browse/IGNITE-1192 <
>>> https://issues.apache.org/jira/browse/IGNITE-1192>
>>> 
>>> —
>>> Denis
>>> 
 On Apr 9, 2017, at 6:57 AM, Denis Magda  wrote:
 
 Val, yes, as far as I know we still need to merge to the master. The
>>> branch should be ready later the next week.
 
 Denis
 
 On Sat, Apr 8, 2017 at 6:14 PM, Valentin Kulichenko <
>>> valentin.kuliche...@gmail.com >
>>> wrote:
 Guys,
 
 There is no branch for 2.0 right now, correct? If I want to add
>> something
 there, do I just merge to master?
 
 -Val
 
 On Sat, Apr 8, 2017 at 10:18 AM, Alexander Paschenko <
 alexander.a.pasche...@gmail.com > gmail.com>>
>>> wrote:
 
> I've fixed https://issues.apache.org/jira/browse/IGNITE-4354 <
>>> https://issues.apache.org/jira/browse/IGNITE-4354>, PR is
> https://github.com/apache/ignite/pull/1759 <
>> https://github.com/apache/
>>> ignite/pull/1759>.
> Pavel, thank you very much for bringing that to my attention.
> 
> - Alex
> 
> 2017-04-07 20:28 GMT+03:00 Sergi Vladykin >> >:
>> A bunch of SQL related tickets is delayed until H2 release on the
>>> next
> week.
>> 
>> Sergi
>> 
>> 2017-04-07 19:36 GMT+03:00 Alexey Goncharuk <
>>> alexey.goncha...@gmail.com 
>> :
>> 
>>> Folks,
>>> 
>>> The status of ignite-3477 is as follows: we've fixed almost all
>>> tests
> and
>>> currently waiting for ignite-4535 and ignite-4534 to be merged,
>>> which
> will
>>> happen once TC for those branches is completed. This should happen
>>> over
> the
>>> weekend.
>>> 
>>> 2017-04-06 19:12 GMT+03:00 Alexey Kuznetsov <
>> akuznet...@apache.org
>>> >:
>>> 
 I've prepared
 
 IGNITE-4349 > <
>>> https://issues.apache.org/jira/browse/IGNITE-4349>> -
 Discontinue the schema-import utility
 
 Anton, could you review my changes related to Ignite *build*?
 See: http://reviews.ignite.apache.org/ignite/review/IGNT-CR-163
>> <
>>> http://reviews.ignite.apache.org/ignite/review/IGNT-CR-163>
 
 
 --
 Alexey Kuznetsov
 
>>> 
> 
 
>>> 
>>> 
>> 

Re: cache query operations

[ALEKSEY KUZNETSOV]

So, when GridCacheTtlManager evicted cache entry due to timeout, how H2 got
notified about it ?

And as i understand, firstly cacheEntry got evicted, and then H2 data got
updated(removed corresponding entry)

вт, 18 апр. 2017 г. в 17:56, Denis Magda :

> That’s true, SELECT queries are executed right away by H2 while DML are
> eventually converted to key-value operations.
>
> —
> Denis
>
> > On Apr 18, 2017, at 1:29 AM, ALEKSEY KUZNETSOV 
> wrote:
> >
> > if executing sql "select" clause then no cache key-value operations would
> > be called.
> > Debbuging cache.query(...) leads
> > to
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing#executeSqlQuery
> > which doesnt call cache key-value ops
> >
> > вт, 18 апр. 2017 г. в 10:49, ALEKSEY KUZNETSOV  >:
> >
> >> thanx! )
> >>
> >> пн, 17 апр. 2017 г. в 21:48, Denis Magda :
> >>
> >>> Did you have a chance to take a look this documentation that explains
> how
> >>> DML statements are turned into cache operations?
> >>> https://apacheignite.readme.io/docs/dml#dml-operations <
> >>> https://apacheignite.readme.io/docs/dml#dml-operations>
> >>>
> >>> —
> >>> Denis
> >>>
>  On Apr 17, 2017, at 7:20 AM, ALEKSEY KUZNETSOV <
> >>> alkuznetsov...@gmail.com> wrote:
> 
>  Hi, Igniters! When one fires query like this : ignite().cache(null
>  ).query(...)
>  The query would be executed against the schema. I wonder, how the
> >>> schema is
>  synchronized with cache, with cache entries?
> 
>  What i mean is , cache.query(...update operation...) must eventually
> >>> update
>  the cache entry in cache. But how does it happen?(cache API must be
> >>> called,
>  but i cannot realize how)
>  --
> 
>  *Best Regards,*
> 
>  *Kuznetsov Aleksey*
> >>>
> >>> --
> >>
> >> *Best Regards,*
> >>
> >> *Kuznetsov Aleksey*
> >>
> > --
> >
> > *Best Regards,*
> >
> > *Kuznetsov Aleksey*
>
> --

*Best Regards,*

*Kuznetsov Aleksey*

Re: Introduce custom executors for compute grid

[Christos Erotocritou]

Thanks Taras, this will be very handy.

I assume its not currently possible to constrain an executor pool to be started 
only on a specific node?

C.

> On 18 Apr 2017, at 15:55, Denis Magda  wrote:
> 
> Taras, that’s an excellent addition to the project!
> 
> Please don’t forget to document it: 
> https://issues.apache.org/jira/browse/IGNITE-4969 
> 
> 
> —
> Denis
> 
>> On Apr 18, 2017, at 1:23 AM, Taras Ledkov  wrote:
>> 
>> Igniters,
>> 
>> Custom executor (user's thread pool) is added fro compute grid with 
>> following semantics:
>> 
>> 1. Configuration:
>> 
>> IgniteConfiguration cfg;
>> ...
>> cfg.setExecutorConfiguration(
>>   new ExecutorConfiguration().setName("executor0").setSize(2),
>>   new ExecutorConfiguration().setName("executor1").setSize(4));
>> 
>> Where
>> name - name of executor and thread pool;
>> size - thread pool size.
>> 
>> 2. Usage:
>> 
>> Ignite ignite;
>> ...
>> IgniteCompute comp = ignite.compute().withExecutor("executor0");
>> comp.broadcast(new IgniteRunnable() {
>>   @Override public void run() {
>>...
>>   }
>>   });
>> 
>> So, 'withExecutor(String)' returns the compute associated with custom named 
>> executor.
>> All jobs submitted by the components will be processed by thread pool 
>> corresponds to named executor.
>> If the executor isn't configured on the target host the warning will be 
>> printed in the log and a job will be processed in the public pool.
>> e.g.:
>> [11:20:01,023][WARN 
>> ][grid-nio-worker-tcp-comm-0-#27%compute.IgniteComputeCustomExecutorSelfTest1%][GridIoManager]
>>  Custom executor 'invalid' doesn't exist. The job will be submit to public 
>> pool: b2e85208b51-4fbcb569-07a2-480e-9be1-512bc320
>> 
>> Issue: https://issues.apache.org/jira/browse/IGNITE-4699
>> 
>> Please share your thoughts or ask questions.
>> 
>> -- 
>> Taras Ledkov
>> Mail-To: tled...@gridgain.com
>> 
> 

Re: cache query operations

[Denis Magda]

That’s true, SELECT queries are executed right away by H2 while DML are 
eventually converted to key-value operations.

—
Denis

> On Apr 18, 2017, at 1:29 AM, ALEKSEY KUZNETSOV  
> wrote:
> 
> if executing sql "select" clause then no cache key-value operations would
> be called.
> Debbuging cache.query(...) leads
> to 
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing#executeSqlQuery
> which doesnt call cache key-value ops
> 
> вт, 18 апр. 2017 г. в 10:49, ALEKSEY KUZNETSOV :
> 
>> thanx! )
>> 
>> пн, 17 апр. 2017 г. в 21:48, Denis Magda :
>> 
>>> Did you have a chance to take a look this documentation that explains how
>>> DML statements are turned into cache operations?
>>> https://apacheignite.readme.io/docs/dml#dml-operations <
>>> https://apacheignite.readme.io/docs/dml#dml-operations>
>>> 
>>> —
>>> Denis
>>> 
 On Apr 17, 2017, at 7:20 AM, ALEKSEY KUZNETSOV <
>>> alkuznetsov...@gmail.com> wrote:
 
 Hi, Igniters! When one fires query like this : ignite().cache(null
 ).query(...)
 The query would be executed against the schema. I wonder, how the
>>> schema is
 synchronized with cache, with cache entries?
 
 What i mean is , cache.query(...update operation...) must eventually
>>> update
 the cache entry in cache. But how does it happen?(cache API must be
>>> called,
 but i cannot realize how)
 --
 
 *Best Regards,*
 
 *Kuznetsov Aleksey*
>>> 
>>> --
>> 
>> *Best Regards,*
>> 
>> *Kuznetsov Aleksey*
>> 
> -- 
> 
> *Best Regards,*
> 
> *Kuznetsov Aleksey*

Re: Introduce custom executors for compute grid

[Denis Magda]

Taras, that’s an excellent addition to the project!

Please don’t forget to document it: 
https://issues.apache.org/jira/browse/IGNITE-4969 


—
Denis

> On Apr 18, 2017, at 1:23 AM, Taras Ledkov  wrote:
> 
> Igniters,
> 
> Custom executor (user's thread pool) is added fro compute grid with following 
> semantics:
> 
> 1. Configuration:
> 
> IgniteConfiguration cfg;
> ...
> cfg.setExecutorConfiguration(
>new ExecutorConfiguration().setName("executor0").setSize(2),
>new ExecutorConfiguration().setName("executor1").setSize(4));
> 
> Where
> name - name of executor and thread pool;
> size - thread pool size.
> 
> 2. Usage:
> 
> Ignite ignite;
> ...
> IgniteCompute comp = ignite.compute().withExecutor("executor0");
> comp.broadcast(new IgniteRunnable() {
>@Override public void run() {
> ...
>}
>});
> 
> So, 'withExecutor(String)' returns the compute associated with custom named 
> executor.
> All jobs submitted by the components will be processed by thread pool 
> corresponds to named executor.
> If the executor isn't configured on the target host the warning will be 
> printed in the log and a job will be processed in the public pool.
> e.g.:
> [11:20:01,023][WARN 
> ][grid-nio-worker-tcp-comm-0-#27%compute.IgniteComputeCustomExecutorSelfTest1%][GridIoManager]
>  Custom executor 'invalid' doesn't exist. The job will be submit to public 
> pool: b2e85208b51-4fbcb569-07a2-480e-9be1-512bc320
> 
> Issue: https://issues.apache.org/jira/browse/IGNITE-4699
> 
> Please share your thoughts or ask questions.
> 
> -- 
> Taras Ledkov
> Mail-To: tled...@gridgain.com
> 

[GitHub] ignite pull request #1822: IGNITE-5008 message of IgniteOutOfMemoryException...

[sergey-chugunov-1985]

GitHub user sergey-chugunov-1985 opened a pull request:

https://github.com/apache/ignite/pull/1822

IGNITE-5008 message of IgniteOutOfMemoryException was detailed



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5008

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1822.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1822


commit d6fc207693d0928ef615cc409054af067d5a079b
Author: Sergey Chugunov 
Date:   2017-04-18T14:37:59Z

IGNITE-5008 message of IgniteOutOfMemoryException was detailed




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1821: 8.0.4.ea1 tests run

[glukos]

GitHub user glukos opened a pull request:

https://github.com/apache/ignite/pull/1821

8.0.4.ea1 tests run

Pull request for running ignite tests on 8.0.4.ea1. Please, don't merge.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-8.0.4.ea1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1821.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1821


commit be881725b39926f62df437c2805ac9203fea5737
Author: Ilya Lantukh 
Date:   2016-10-11T16:19:20Z

gg-11595 : WIP.

commit e6d82d8da50b9af9607a914e00da3937d839862a
Author: Ilya Lantukh 
Date:   2016-10-12T16:20:50Z

gg-11595 : WIP.

commit cb9c18ccfc937b4a880eddb5df5d0389de9c7bee
Author: Ilya Lantukh 
Date:   2016-10-13T12:54:39Z

gg-11595 : WIP.

commit ed45f2238361e5bbbe907fdeb4b7a3d7b2b051dd
Author: Ilya Lantukh 
Date:   2016-10-27T13:02:28Z

gg-11595 : Support for restore with concurrent cache operations.

commit e95c68b77cbd3ad30d9f6d0b332137fb26000a41
Author: Ilya Lantukh 
Date:   2016-10-27T13:35:46Z

gg-11595 : Minors.

commit 7a44f1ebccf4fe0f26f54e070bbacbf24fbee3d7
Author: Ilya Lantukh 
Date:   2016-12-16T15:56:24Z

gg-11701 : WIP.

commit 3e2b28075ab5d00dce7fadf4769967f7a0ee2ee8
Author: Ilya Lantukh 
Date:   2016-12-19T12:25:11Z

Merge branches 'ignite-gg-11701' and 'ignite-gg-8.0.2.ea1' of 
https://github.com/gridgain/apache-ignite into ignite-gg-11701

commit cc568f9f6823de04f0beeca45ada4160a58307e4
Author: Ilya Lantukh 
Date:   2016-12-21T13:11:36Z

gg-11701 : WIP

commit e93f8d43a92dd8bde12c4677b5fcc6b0c2de6ce6
Author: Ilya Lantukh 
Date:   2016-12-27T10:45:29Z

Merge branches 'ignite-gg-11701' and 'ignite-gg-8.0.2.ea1' of 
https://github.com/gridgain/apache-ignite into ignite-gg-11701

commit 721f255eb2de1d8207e35328e2dec6514c22500d
Author: Ilya Lantukh 
Date:   2016-12-28T15:58:38Z

gg-11701 : Fixed preloading.

commit bed897dc01960dcbb7219ad948973e0f27bfa564
Author: Ilya Lantukh 
Date:   2016-12-28T16:31:51Z

gg-11701 : Fixed force keys request for single get.

commit 07535d92cd37cc949188434eb7ded2fc8d2e0647
Author: Ilya Lantukh 
Date:   2016-12-28T16:48:15Z

gg-11701 : Added check to multiple get.

commit ff9aa8898445c2a30e80fbc94f668eb5cd29a7d8
Author: Ilya Lantukh 
Date:   2016-12-29T13:04:49Z

Merge branches 'ignite-gg-11701' and 'ignite-gg-8.0.2.ea2' of 
https://github.com/gridgain/apache-ignite into ignite-gg-11701

commit 6cbe8a5a8ea5e4dacb8c5a859cb15aea5b3f38af
Author: Ilya Lantukh 
Date:   2017-01-09T15:22:15Z

gg-11701 : Fixed partition update counters comparison.

commit 3c276c9e092b6f6b4baf0b550ad89cb85ab9c42f
Author: Ilya Lantukh 
Date:   2017-01-10T10:25:09Z

gg-11701 : Implicit lateAffinityAssignment mode when persistence is enabled.

commit b3f7ae3bf903a0a6357163029ed310647877c8fc
Author: Ilya Lantukh 
Date:   2017-01-10T11:25:24Z

gg-11701 : Replaced redundant checks with assertions.

commit 2e55ddb600819dbf4684c0e97cc71a733167a4ce
Author: Ilya Lantukh 
Date:   2017-01-11T15:05:25Z

gg-11701 : Merge with 8.0.2.ea2

commit ecead988090e6a65ffdbb4098252ea26287fe36e
Author: Ilya Lantukh 
Date:   2017-01-11T15:09:25Z

gg-11701 : Merge with 8.0.2.ea2

commit adc0422592a18a7c2635e45185ef17852cd41952
Author: Glukos 
Date:   2017-01-13T17:23:06Z

GG-11595:
Remote exception wrapped on proxy level

commit 4505066481d4fff601a14d344dc8059f5dbe73de
Author: Ivan Rakov 
Date:   2017-01-13T21:32:08Z

GG-11595:
+ serialVersionUid

commit ba847555c86c00a288362e1cfad8c4c30883975c
Author: Ivan Rakov 
Date:   2017-01-16T15:08:35Z

Merge branch 'ignite-gg-8.0.2.ea2#' into ignite-gg-11595

# Conflicts:
#   modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/DynamicCacheChangeRequest.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheProcessor.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheProxy.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/atomic/GridDhtAtomicCache.java
#   

Re: IGNITE-2894 - Binary object inside of Externalizable still serialized with OptimizedMarshaller

[Valentin Kulichenko]

Nikita,

For Externalizable option 1 is the correct one. Externalizable objects
should not be treated as binary objects.

For read/writeObject, you indeed have to extend ObjectOutputStream.
writeObject() is final because you should extend writeObjectOverride()
instead. Take a look at ObjectOutputStream's JavaDoc and on how this is
done in OptimizedObjectOutputStream. Note that ideally we need to implement
everything that is included in Java serialization spec, including some
non-trivial stuff like PutField. I would check if it's possible to somehow
reuse the code that already exists in optimized marshaller as much as
possible.

-Val

On Tue, Apr 18, 2017 at 1:36 PM, Nikita Amelchev 
wrote:

> I see two ways to support the Externalizable in the BM:
> 1. Add a new type constant to the GridBinaryMarshaller class etc and
> read/writeExternal in the BinaryClassDescriptor.
> 2. Make read/writeExternal through the BINARY type without updating
> metadata.
> I don't know how to make a support read/writeObject of the Serializable
> without delegating to the OM. Because read/writeObject methods need the
> Objectoutputstream class argument. One way is to delegate it to the
> OptimizedObjectOutputStream. Second way is to extend the Objectoutputstream
> in the BinaryWriterExImpl. But it is wrong way because the writeObject is
> final.
>
> 2017-01-19 20:46 GMT+03:00 Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
>
> > Nikita,
> >
> > In my view we just need to support Externalizable and
> > writeObject/readObject in BinaryMarshaller and get rid of delegation to
> > optimized marshaller. Once such classes also go through BinaryMarshaller
> > streams, they will be aware of binary configuration and will share the
> same
> > set of handles as well. This should take care of all the issues we have
> > here.
> >
> > -Val
> >
> > On Thu, Jan 19, 2017 at 7:26 AM, Nikita Amelchev 
> > wrote:
> >
> > > I have some questions about single Marshaller.
> > > It seems not easy to merge OptimizedMarshaller with BinaryMarshaller
> and
> > is
> > > there any sense in it?
> > > When Binary object inside Externalizable serialized with optimized it
> > > losing all benefits.
> > > Will OptimizedMarshaller be supported at 2.0 version? Or to merge they
> is
> > > better?
> > > What do you think about it?
> > >
> > > In addition, Vladimir Ozerov, I would like to hear your opinion.
> > >
> > > 2017-01-17 23:32 GMT+03:00 Denis Magda :
> > >
> > > > Someone else added you to the contributors list in JIRA. This is why
> I
> > > > couldn’t add you for the second time. Ignite committers, please reply
> > on
> > > > the dev list if you add someone to the list.
> > > >
> > > > Nikita, yes, this ticket is still relevant. Go ahead and assign it on
> > > > yourself.
> > > >
> > > > Also please you may want to help with approaching 2.0 release and
> take
> > > > care of one of the sub-tasks that must be included in 2.0:
> > > > https://issues.apache.org/jira/browse/IGNITE-4547 <
> > > > https://issues.apache.org/jira/browse/IGNITE-4547>
> > > >
> > > > —
> > > > Denis
> > > >
> > > > > On Jan 15, 2017, at 9:02 PM, Nikita Amelchev  >
> > > > wrote:
> > > > >
> > > > > This issue was created long ago. Is still relevant?
> > > > >
> > > > > JIRA account:
> > > > > Username: NSAmelchev
> > > > > Full Name: Amelchev Nikita
> > > > >
> > > > >
> > > > > 2017-01-14 1:52 GMT+03:00 Denis Magda :
> > > > >
> > > > >> Hi Nikita,
> > > > >>
> > > > >> I can’t find provided account in Ignite JIRA
> > > > >> https://issues.apache.org/jira/browse/IGNITE <
> > > > https://issues.apache.org/
> > > > >> jira/browse/IGNITE>
> > > > >>
> > > > >> Please create an account there and share with me.
> > > > >>
> > > > >> This information might be useful for you as well.
> > > > >>
> > > > >> Subscribe to both dev and user lists:
> > > > >> https://ignite.apache.org/community/resources.html#mail-lists
> > > > >>
> > > > >> Get familiar with Ignite development process described here:
> > > > >> https://cwiki.apache.org/confluence/display/IGNITE/
> > > Development+Process
> > > > >>
> > > > >> Instructions on how to contribute can be found here:
> > > > >> https://cwiki.apache.org/confluence/display/IGNITE/How+
> > to+Contribute
> > > > >>
> > > > >> Project setup in Intellij IDEAL
> > > > >> https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup
> > > > >>
> > > > >> Regards,
> > > > >> Denis
> > > > >>
> > > > >>> On Jan 13, 2017, at 1:37 AM, Nikita Amelchev <
> nsamelc...@gmail.com
> > >
> > > > >> wrote:
> > > > >>>
> > > > >>> Hello everyone.
> > > > >>>
> > > > >>> I'd like to take IGNITE-2894. Can you assign to me?
> > > > >>>
> > > > >>> Username: NSAmelchev
> > > > >>>
> > > > >>> --
> > > > >>> Best wishes,
> > > > >>> Amelchev Nikita
> > > > >>
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > Best wishes,
> > > > > Amelchev Nikita
> > > >
> > > 

When cache topology can be null?

[Дмитрий Рябов]

One case is when node is down. Is there other cases?

[jira] [Created] (IGNITE-5017) HTML formatted releases notes

[Sergey Kozlov (JIRA)]

Sergey Kozlov created IGNITE-5017:
-

 Summary: HTML formatted releases notes
 Key: IGNITE-5017
 URL: https://issues.apache.org/jira/browse/IGNITE-5017
 Project: Ignite
  Issue Type: Bug
  Components: documentation
Affects Versions: 1.9
Reporter: Sergey Kozlov
 Fix For: 2.0






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] ignite pull request #1818: IGNITE-5000 (Rename Ignite Math module to Ignite ...

[ybabak]

Github user ybabak closed the pull request at:

https://github.com/apache/ignite/pull/1818


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-5016) SQL: Support LEFT JOIN from replicated table to a partitioned.

[Sergi Vladykin (JIRA)]

Sergi Vladykin created IGNITE-5016:
--

 Summary: SQL: Support LEFT JOIN from replicated table to a 
partitioned.
 Key: IGNITE-5016
 URL: https://issues.apache.org/jira/browse/IGNITE-5016
 Project: Ignite
  Issue Type: Bug
Reporter: Sergi Vladykin
Assignee: Sergi Vladykin


Now we return duplicates:

IgniteCacheJoinPartitionedAndReplicatedTest.testReplicatedToPartitionedLeftJoin



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5015) In TcpCommunicationSpi use IgniteConfiguration.clientFailureDetectionTimeout

[Alexander Belyak (JIRA)]

Alexander Belyak created IGNITE-5015:


 Summary: In TcpCommunicationSpi use 
IgniteConfiguration.clientFailureDetectionTimeout
 Key: IGNITE-5015
 URL: https://issues.apache.org/jira/browse/IGNITE-5015
 Project: Ignite
  Issue Type: Bug
Reporter: Alexander Belyak


Need to use new IgniteConfiguration.clientFailureDetectionTimeout in 
CommunicationSpi when interacting with client nodes.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5014) Do not used partition exchange worker for CREATE/DROP INDEX

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-5014:
---

 Summary: Do not used partition exchange worker for CREATE/DROP 
INDEX
 Key: IGNITE-5014
 URL: https://issues.apache.org/jira/browse/IGNITE-5014
 Project: Ignite
  Issue Type: Task
  Components: cache, SQL
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
Priority: Minor
 Fix For: 2.1


It was used for historical reasons. But looks like it is not needed anymore.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Apache Ignite 2.0 Release

[Vladimir Ozerov]

CREATE/DROP INDEX feature [1] has been merged to release branch.

[1] https://issues.apache.org/jira/browse/IGNITE-4565

On Tue, Apr 18, 2017 at 4:10 PM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> Hi Roman,
>
> I tried to run tests for RocketMQ streamer, but they failed for me (I've
> added a comment in the ticket). Can you please take a look?
>
> 2017-04-18 15:13 GMT+03:00 Vladimir Ozerov :
>
> > Folks,
> >
> > I created branch for AI 2.0: ignite-2.0. Relevant PR #1819. Let's
> continue
> > 2.0 finalization in this branch.
> >
> > On Wed, Apr 12, 2017 at 7:58 AM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> > wrote:
> >
> > > Awesome! Great addition to the project.
> > >
> > > On Tue, Apr 11, 2017 at 8:27 PM, Denis Magda 
> wrote:
> > >
> > > > Spring Data integration caught up the train and will be released in
> > 2.0:
> > > > https://issues.apache.org/jira/browse/IGNITE-1192 <
> > > > https://issues.apache.org/jira/browse/IGNITE-1192>
> > > >
> > > > —
> > > > Denis
> > > >
> > > > > On Apr 9, 2017, at 6:57 AM, Denis Magda 
> wrote:
> > > > >
> > > > > Val, yes, as far as I know we still need to merge to the master.
> The
> > > > branch should be ready later the next week.
> > > > >
> > > > > Denis
> > > > >
> > > > > On Sat, Apr 8, 2017 at 6:14 PM, Valentin Kulichenko <
> > > > valentin.kuliche...@gmail.com  >>
> > > > wrote:
> > > > > Guys,
> > > > >
> > > > > There is no branch for 2.0 right now, correct? If I want to add
> > > something
> > > > > there, do I just merge to master?
> > > > >
> > > > > -Val
> > > > >
> > > > > On Sat, Apr 8, 2017 at 10:18 AM, Alexander Paschenko <
> > > > > alexander.a.pasche...@gmail.com  > > gmail.com>>
> > > > wrote:
> > > > >
> > > > > > I've fixed https://issues.apache.org/jira/browse/IGNITE-4354 <
> > > > https://issues.apache.org/jira/browse/IGNITE-4354>, PR is
> > > > > > https://github.com/apache/ignite/pull/1759 <
> > > https://github.com/apache/
> > > > ignite/pull/1759>.
> > > > > > Pavel, thank you very much for bringing that to my attention.
> > > > > >
> > > > > > - Alex
> > > > > >
> > > > > > 2017-04-07 20:28 GMT+03:00 Sergi Vladykin <
> > sergi.vlady...@gmail.com
> > > > >:
> > > > > > > A bunch of SQL related tickets is delayed until H2 release on
> the
> > > > next
> > > > > > week.
> > > > > > >
> > > > > > > Sergi
> > > > > > >
> > > > > > > 2017-04-07 19:36 GMT+03:00 Alexey Goncharuk <
> > > > alexey.goncha...@gmail.com 
> > > > > > >:
> > > > > > >
> > > > > > >> Folks,
> > > > > > >>
> > > > > > >> The status of ignite-3477 is as follows: we've fixed almost
> all
> > > > tests
> > > > > > and
> > > > > > >> currently waiting for ignite-4535 and ignite-4534 to be
> merged,
> > > > which
> > > > > > will
> > > > > > >> happen once TC for those branches is completed. This should
> > happen
> > > > over
> > > > > > the
> > > > > > >> weekend.
> > > > > > >>
> > > > > > >> 2017-04-06 19:12 GMT+03:00 Alexey Kuznetsov <
> > > akuznet...@apache.org
> > > > >:
> > > > > > >>
> > > > > > >> >  I've prepared
> > > > > > >> >
> > > > > > >> >  IGNITE-4349  > jira/browse/IGNITE-4349
> > > <
> > > > https://issues.apache.org/jira/browse/IGNITE-4349>> -
> > > > > > >> > Discontinue the schema-import utility
> > > > > > >> >
> > > > > > >> > Anton, could you review my changes related to Ignite
> *build*?
> > > > > > >> > See: http://reviews.ignite.apache.
> > org/ignite/review/IGNT-CR-163
> > > <
> > > > http://reviews.ignite.apache.org/ignite/review/IGNT-CR-163>
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > --
> > > > > > >> > Alexey Kuznetsov
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > > >
> > >
> >
>

[jira] [Created] (IGNITE-5013) Allow pass custom types to Cassandra store

[Dmitry Karachentsev (JIRA)]

Dmitry Karachentsev created IGNITE-5013:
---

 Summary: Allow pass custom types to Cassandra store
 Key: IGNITE-5013
 URL: https://issues.apache.org/jira/browse/IGNITE-5013
 Project: Ignite
  Issue Type: Improvement
  Components: ignite-cassandra
Affects Versions: 2.0
Reporter: Dmitry Karachentsev
 Fix For: 2.1


Any type that is not in 
org.apache.ignite.cache.store.cassandra.common.PropertyMappingHelper#JAVA_TO_CASSANDRA_MAPPING
 will be serialized even if user specified custom codec for Cassandra. 

Should be ability to configure and bypass custom types to Cassandra.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5012) Implement ordinary least squares (OLS) linear regression.

[Artem Malykh (JIRA)]

Artem Malykh created IGNITE-5012:


 Summary: Implement ordinary least squares (OLS) linear regression.
 Key: IGNITE-5012
 URL: https://issues.apache.org/jira/browse/IGNITE-5012
 Project: Ignite
  Issue Type: Task
Reporter: Artem Malykh


Implement ordinary least squares (OLS) linear regression using ignite ml.math. 
This is a useful, but simple basic machine learning algorithm which is a good 
candidate for testing math implemented in ml module on real machine learning 
tasks.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] ignite pull request #1785: IGNITE-4954 - Configurable expiration timeout for...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1785


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1820: fix GridIntList directType

[kdudkov]

GitHub user kdudkov opened a pull request:

https://github.com/apache/ignite/pull/1820

fix GridIntList directType



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite intlist_fix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1820.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1820


commit aa86ac7128da8b68c25861493915e69919fbbb76
Author: Konstantin Dudkov 
Date:   2017-04-18T13:24:30Z

fix GridIntList directType




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Apache Ignite 2.0 Release

[Alexey Goncharuk]

Hi Roman,

I tried to run tests for RocketMQ streamer, but they failed for me (I've
added a comment in the ticket). Can you please take a look?

2017-04-18 15:13 GMT+03:00 Vladimir Ozerov :

> Folks,
>
> I created branch for AI 2.0: ignite-2.0. Relevant PR #1819. Let's continue
> 2.0 finalization in this branch.
>
> On Wed, Apr 12, 2017 at 7:58 AM, Dmitriy Setrakyan 
> wrote:
>
> > Awesome! Great addition to the project.
> >
> > On Tue, Apr 11, 2017 at 8:27 PM, Denis Magda  wrote:
> >
> > > Spring Data integration caught up the train and will be released in
> 2.0:
> > > https://issues.apache.org/jira/browse/IGNITE-1192 <
> > > https://issues.apache.org/jira/browse/IGNITE-1192>
> > >
> > > —
> > > Denis
> > >
> > > > On Apr 9, 2017, at 6:57 AM, Denis Magda  wrote:
> > > >
> > > > Val, yes, as far as I know we still need to merge to the master. The
> > > branch should be ready later the next week.
> > > >
> > > > Denis
> > > >
> > > > On Sat, Apr 8, 2017 at 6:14 PM, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com >
> > > wrote:
> > > > Guys,
> > > >
> > > > There is no branch for 2.0 right now, correct? If I want to add
> > something
> > > > there, do I just merge to master?
> > > >
> > > > -Val
> > > >
> > > > On Sat, Apr 8, 2017 at 10:18 AM, Alexander Paschenko <
> > > > alexander.a.pasche...@gmail.com  > gmail.com>>
> > > wrote:
> > > >
> > > > > I've fixed https://issues.apache.org/jira/browse/IGNITE-4354 <
> > > https://issues.apache.org/jira/browse/IGNITE-4354>, PR is
> > > > > https://github.com/apache/ignite/pull/1759 <
> > https://github.com/apache/
> > > ignite/pull/1759>.
> > > > > Pavel, thank you very much for bringing that to my attention.
> > > > >
> > > > > - Alex
> > > > >
> > > > > 2017-04-07 20:28 GMT+03:00 Sergi Vladykin <
> sergi.vlady...@gmail.com
> > > >:
> > > > > > A bunch of SQL related tickets is delayed until H2 release on the
> > > next
> > > > > week.
> > > > > >
> > > > > > Sergi
> > > > > >
> > > > > > 2017-04-07 19:36 GMT+03:00 Alexey Goncharuk <
> > > alexey.goncha...@gmail.com 
> > > > > >:
> > > > > >
> > > > > >> Folks,
> > > > > >>
> > > > > >> The status of ignite-3477 is as follows: we've fixed almost all
> > > tests
> > > > > and
> > > > > >> currently waiting for ignite-4535 and ignite-4534 to be merged,
> > > which
> > > > > will
> > > > > >> happen once TC for those branches is completed. This should
> happen
> > > over
> > > > > the
> > > > > >> weekend.
> > > > > >>
> > > > > >> 2017-04-06 19:12 GMT+03:00 Alexey Kuznetsov <
> > akuznet...@apache.org
> > > >:
> > > > > >>
> > > > > >> >  I've prepared
> > > > > >> >
> > > > > >> >  IGNITE-4349  jira/browse/IGNITE-4349
> > <
> > > https://issues.apache.org/jira/browse/IGNITE-4349>> -
> > > > > >> > Discontinue the schema-import utility
> > > > > >> >
> > > > > >> > Anton, could you review my changes related to Ignite *build*?
> > > > > >> > See: http://reviews.ignite.apache.
> org/ignite/review/IGNT-CR-163
> > <
> > > http://reviews.ignite.apache.org/ignite/review/IGNT-CR-163>
> > > > > >> >
> > > > > >> >
> > > > > >> > --
> > > > > >> > Alexey Kuznetsov
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> > >
> >
>

Re: Performance vs correctness: I vote fore the second

[Dmitriy Setrakyan]

I agree with Yakov. Let's not swing like a pendulum from one side to the
other. Why not switch to FULL_SYNC only for REPLICATED caches?

D.

On Tue, Apr 18, 2017 at 5:48 AM, Yakov Zhdanov  wrote:

> Sergi, most probably, performance will not be vert much affected since we
> can batch delayed responses and this for primary_sync only. However, I
> agree with your point about overcomplicated code, but I did not state that
> it would be trivial.
>
> So, what is the solution here? Switch replicated cache to full_sync by
> default? This seems very simple from coding standpoint and should work for
> most deployments.
>
> --Yakov
>

Re: Performance vs correctness: I vote fore the second

[Yakov Zhdanov]

Sergi, most probably, performance will not be vert much affected since we
can batch delayed responses and this for primary_sync only. However, I
agree with your point about overcomplicated code, but I did not state that
it would be trivial.

So, what is the solution here? Switch replicated cache to full_sync by
default? This seems very simple from coding standpoint and should work for
most deployments.

--Yakov

Re: Apache Ignite 2.0 Release

[Vladimir Ozerov]

Folks,

I created branch for AI 2.0: ignite-2.0. Relevant PR #1819. Let's continue
2.0 finalization in this branch.

On Wed, Apr 12, 2017 at 7:58 AM, Dmitriy Setrakyan 
wrote:

> Awesome! Great addition to the project.
>
> On Tue, Apr 11, 2017 at 8:27 PM, Denis Magda  wrote:
>
> > Spring Data integration caught up the train and will be released in 2.0:
> > https://issues.apache.org/jira/browse/IGNITE-1192 <
> > https://issues.apache.org/jira/browse/IGNITE-1192>
> >
> > —
> > Denis
> >
> > > On Apr 9, 2017, at 6:57 AM, Denis Magda  wrote:
> > >
> > > Val, yes, as far as I know we still need to merge to the master. The
> > branch should be ready later the next week.
> > >
> > > Denis
> > >
> > > On Sat, Apr 8, 2017 at 6:14 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com >
> > wrote:
> > > Guys,
> > >
> > > There is no branch for 2.0 right now, correct? If I want to add
> something
> > > there, do I just merge to master?
> > >
> > > -Val
> > >
> > > On Sat, Apr 8, 2017 at 10:18 AM, Alexander Paschenko <
> > > alexander.a.pasche...@gmail.com  gmail.com>>
> > wrote:
> > >
> > > > I've fixed https://issues.apache.org/jira/browse/IGNITE-4354 <
> > https://issues.apache.org/jira/browse/IGNITE-4354>, PR is
> > > > https://github.com/apache/ignite/pull/1759 <
> https://github.com/apache/
> > ignite/pull/1759>.
> > > > Pavel, thank you very much for bringing that to my attention.
> > > >
> > > > - Alex
> > > >
> > > > 2017-04-07 20:28 GMT+03:00 Sergi Vladykin  > >:
> > > > > A bunch of SQL related tickets is delayed until H2 release on the
> > next
> > > > week.
> > > > >
> > > > > Sergi
> > > > >
> > > > > 2017-04-07 19:36 GMT+03:00 Alexey Goncharuk <
> > alexey.goncha...@gmail.com 
> > > > >:
> > > > >
> > > > >> Folks,
> > > > >>
> > > > >> The status of ignite-3477 is as follows: we've fixed almost all
> > tests
> > > > and
> > > > >> currently waiting for ignite-4535 and ignite-4534 to be merged,
> > which
> > > > will
> > > > >> happen once TC for those branches is completed. This should happen
> > over
> > > > the
> > > > >> weekend.
> > > > >>
> > > > >> 2017-04-06 19:12 GMT+03:00 Alexey Kuznetsov <
> akuznet...@apache.org
> > >:
> > > > >>
> > > > >> >  I've prepared
> > > > >> >
> > > > >> >  IGNITE-4349  <
> > https://issues.apache.org/jira/browse/IGNITE-4349>> -
> > > > >> > Discontinue the schema-import utility
> > > > >> >
> > > > >> > Anton, could you review my changes related to Ignite *build*?
> > > > >> > See: http://reviews.ignite.apache.org/ignite/review/IGNT-CR-163
> <
> > http://reviews.ignite.apache.org/ignite/review/IGNT-CR-163>
> > > > >> >
> > > > >> >
> > > > >> > --
> > > > >> > Alexey Kuznetsov
> > > > >> >
> > > > >>
> > > >
> > >
> >
> >
>

[GitHub] ignite pull request #1819: Ignite 2.0

[devozerov]

GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/1819

Ignite 2.0



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/ignite ignite-2.0

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1819.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1819


commit 0ccb8569bd3f7b4bbcdb435612867a6723d4a506
Author: devozerov 
Date:   2017-04-06T11:02:09Z

IGNITE-3531: Renamed IgniteFileSystem.format() to clear().

commit a6d518d40b946c23918423e15fd332dc185783b6
Author: devozerov 
Date:   2017-04-06T11:07:46Z

IGNITE-3522: Renamed "streamBufferSize" property of FileSystemConfiguration 
to "bufferSize".

commit 0bad8c670b2b8fad7ba8318a4de908e3c61ceaa0
Author: Ilya Lantukh 
Date:   2017-04-06T13:20:01Z

ignite-4535 : Tests.

commit 6ebddaebc020ccf5dd091f4ea2029395712c73fe
Author: Ilya Lantukh 
Date:   2017-04-06T13:54:45Z

ignite-4535 : Adapted tests to set on-heap cache if they use eviction 
policy.

commit ac8720779b44bf32310e1484d7ca91036c9d67ae
Author: Ilya Lantukh 
Date:   2017-04-06T14:08:51Z

ignite-4535 : Removed cache configuration properties related to synchronous 
eviction.

commit 3b5a74585eb9e5f236fc09e89a2c9e014fe7
Author: Ilya Lantukh 
Date:   2017-04-06T14:16:55Z

ignite-4535 : Javadoc.

commit 2999d99823dd888d17f5b66295a89f6a96f1d076
Author: Ilya Lantukh 
Date:   2017-04-06T15:04:38Z

Merge branches 'ignite-4535-master' and 'ignite-4851' of 
https://github.com/gridgain/apache-ignite into ignite-4851

commit 8499de3e8386e496156846461df37e4fc5f06d37
Author: Ilya Lantukh 
Date:   2017-04-06T15:25:19Z

Fixed IgniteCachePartitionLossPolicySelfTest.

commit e834498134e3d3924514a7bcc635b74130b9064e
Author: Ilya Lantukh 
Date:   2017-04-06T15:29:19Z

Un-commented IgniteCacheTxExpiryPolicyWithStoreTest.testReadThrough.

commit 7a9bd3ef05d23d685694a560424cdc1de7806053
Author: Dmitriy Govorukhin 
Date:   2017-04-06T15:47:00Z

ignite-3477-master fix testClientReconnectRequest

commit 1fb198e806dd7ab6d05837226e9bcc9e838c80aa
Author: Dmitriy Govorukhin 
Date:   2017-04-06T15:47:17Z

Merge remote-tracking branch 'professional/ignite-3477-master' into 
ignite-3477-master

commit 99785f1f5219298bb0b64baa79b21ddd138f38da
Author: Ilya Lantukh 
Date:   2017-04-06T16:00:42Z

ignite-4535 : Fixed new full API tests.

commit 6204d46506508256930ad704d6d0599900403a5b
Author: Alexey Goncharuk 
Date:   2017-04-06T16:02:58Z

IGNITE-3477 - Byte order in SSL decoded buffer

commit d433800fb3b6eb0a6b02757ff674e36dc374f55b
Author: Ilya Lantukh 
Date:   2017-04-06T16:35:29Z

ignite-4535 : Minor fixes in tests.

commit 1970982a4925ed52d5f99006605b69a5b18540a0
Author: Sergi Vladykin 
Date:   2017-04-06T16:59:25Z

Merge branch 'ignite-4811' of https://github.com/gridgain/apache-ignite 
into ignite-3477-master

commit 5c6d5503ef41dea91fac70caa140c1bf798d3c5f
Author: Sergi Vladykin 
Date:   2017-04-07T09:15:05Z

ignite-3477 - destroy checks on api exit

commit 6b62a20323b6ccf424e6531e165bd6b057edc4c1
Author: Dmitriy Govorukhin 
Date:   2017-04-07T11:28:22Z

IGNITE-4889 - Changed Hibernate integration to use custom keys

commit 3be4e00373ec5a2b49788d70eb0aebccc3cb6ccf
Author: Alexander Fedotov 
Date:   2017-04-07T11:59:00Z

Merge branch ignite-1.6.5 into ignite-1.8.5-p1

commit abe711824a3afab6bd3c462f61e01239de910305
Author: Dmitriy Govorukhin 
Date:   2017-04-07T12:21:25Z

Merge remote-tracking branch 'origin/ignite-3477-master' into 
ignite-3477-master

commit d81548d3a4e384e1a9b4adacf1fb487444bbfd33
Author: Alexander Fedotov 
Date:   2017-04-07T12:33:08Z

Merge branch ignite-1.6.6-p1 into ignite-1.8.5-p1

commit ea64bdd8ab4d87cad8c0c018bd9ea437a15e33a8
Author: Ilya Lantukh 
Date:   2017-04-07T12:33:42Z

ignite-4535 : Removed redundant code related to synchronous evictions.

commit 3f90a9c0795c2ba61942f2f379e379e8bcd36d08
Author: Sergey Chugunov 
Date:   2017-04-07T12:39:22Z

IGNITE-4536 - Update metrics for memory policies.

commit 64cad4744f73eb0efbc9e374019fad778553f34f
Author: Alexey Goncharuk 
Date:   2017-04-07T12:41:37Z

Merge branch 'ignite-3477-master' of 
https://github.com/gridgain/apache-ignite into 

[jira] [Created] (IGNITE-5011) Add validation of input params of disco command

[Pavel Konstantinov (JIRA)]

Pavel Konstantinov created IGNITE-5011:
--

 Summary: Add validation of input params of disco command
 Key: IGNITE-5011
 URL: https://issues.apache.org/jira/browse/IGNITE-5011
 Project: Ignite
  Issue Type: Bug
Reporter: Pavel Konstantinov
Priority: Trivial


{code}
visor> disco -t=0h
java.lang.IllegalArgumentException: Time frame size is not positive in: 0h
at org.apache.ignite.visor.visor$.timeFilter(visor.scala:2671)
at 
org.apache.ignite.visor.commands.disco.VisorDiscoveryCommand.disco(VisorDiscoveryCommand.scala:127)
at 
org.apache.ignite.visor.commands.disco.VisorDiscoveryCommand$$anonfun$4.apply(VisorDiscoveryCommand.scala:282)
at 
org.apache.ignite.visor.commands.disco.VisorDiscoveryCommand$$anonfun$4.apply(VisorDiscoveryCommand.scala:282)
at 
org.apache.ignite.visor.commands.VisorConsole.mainLoop(VisorConsole.scala:217)
at 
org.apache.ignite.visor.commands.VisorConsole$.delayedEndpoint$org$apache$ignite$visor$commands$VisorConsole$1(VisorConsole.scala:329)
at 
org.apache.ignite.visor.commands.VisorConsole$delayedInit$body.apply(VisorConsole.scala:318)
at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
at 
scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:381)
at 
scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
at scala.App$class.main(App.scala:76)
at 
org.apache.ignite.visor.commands.VisorConsole$.main(VisorConsole.scala:318)
at 
org.apache.ignite.visor.commands.VisorConsole.main(VisorConsole.scala)
(wrn) : Invalid command name: 'disco -t=0h'
(wrn) : Type 'help' to print commands list.
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] ignite pull request #1818: IGNITE-5000 (Rename Ignite Math module to Ignite ...

[ybabak]

GitHub user ybabak opened a pull request:

https://github.com/apache/ignite/pull/1818

IGNITE-5000 (Rename Ignite Math module to Ignite ML module):

  added missed licenses
  renamed packages
  fixed wrong ml profile activation

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5000

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1818.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1818


commit 4a8cce31ff390fedf2dfef90926e86251e163428
Author: Yury Babak 
Date:   2017-04-18T11:50:55Z

IGNITE-5000 (Rename Ignite Math module to Ignite ML module):
  added missed licenses
  renamed packages
  fixed wrong ml profile activation




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Tickets for work

[Nikita Amelchev]

Hello.
Could you give me any ticket ?  Preferably not time-consuming.

-- 
Best wishes,
Amelchev Nikita

[GitHub] ignite pull request #1817: Ignite 4473 1

[issecorvus]

GitHub user issecorvus opened a pull request:

https://github.com/apache/ignite/pull/1817

Ignite 4473 1



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/ignite ignite-4473-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1817.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1817


commit 9df5e94d5cf14ddd55e29b81989177a7798f7e1a
Author: dkarachentsev 
Date:   2017-02-21T12:34:59Z

IGNITE-4671 - FairAffinityFunction fails on node restart with backupFilter 
set and no backups

commit 8e5a5f9d5cc4e4cd3d7d1e278d9a6a85dacfd706
Author: dkarachentsev 
Date:   2017-03-03T14:28:27Z

IGNITE-4473 - Client should re-try connection attempt in case of concurrent 
network failure.

commit 741ecef20af05d9ab461e043fdda2f5528bbb2bc
Author: dkarachentsev 
Date:   2017-03-05T07:47:39Z

IGNITE-4473 - Compilation.

commit e3df4c7e01f14a4cf44cefc6016373d3386095a5
Author: nikolay_tikhonov 
Date:   2017-03-07T16:16:08Z

temp

commit 33fbe6ace5f4721dd8287032019042150324eb53
Author: nikolay_tikhonov 
Date:   2017-03-10T16:23:02Z

temp




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: IGNITE-2894 - Binary object inside of Externalizable still serialized with OptimizedMarshaller

[Nikita Amelchev]

I see two ways to support the Externalizable in the BM:
1. Add a new type constant to the GridBinaryMarshaller class etc and
read/writeExternal in the BinaryClassDescriptor.
2. Make read/writeExternal through the BINARY type without updating
metadata.
I don't know how to make a support read/writeObject of the Serializable
without delegating to the OM. Because read/writeObject methods need the
Objectoutputstream class argument. One way is to delegate it to the
OptimizedObjectOutputStream. Second way is to extend the Objectoutputstream
in the BinaryWriterExImpl. But it is wrong way because the writeObject is
final.

2017-01-19 20:46 GMT+03:00 Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Nikita,
>
> In my view we just need to support Externalizable and
> writeObject/readObject in BinaryMarshaller and get rid of delegation to
> optimized marshaller. Once such classes also go through BinaryMarshaller
> streams, they will be aware of binary configuration and will share the same
> set of handles as well. This should take care of all the issues we have
> here.
>
> -Val
>
> On Thu, Jan 19, 2017 at 7:26 AM, Nikita Amelchev 
> wrote:
>
> > I have some questions about single Marshaller.
> > It seems not easy to merge OptimizedMarshaller with BinaryMarshaller and
> is
> > there any sense in it?
> > When Binary object inside Externalizable serialized with optimized it
> > losing all benefits.
> > Will OptimizedMarshaller be supported at 2.0 version? Or to merge they is
> > better?
> > What do you think about it?
> >
> > In addition, Vladimir Ozerov, I would like to hear your opinion.
> >
> > 2017-01-17 23:32 GMT+03:00 Denis Magda :
> >
> > > Someone else added you to the contributors list in JIRA. This is why I
> > > couldn’t add you for the second time. Ignite committers, please reply
> on
> > > the dev list if you add someone to the list.
> > >
> > > Nikita, yes, this ticket is still relevant. Go ahead and assign it on
> > > yourself.
> > >
> > > Also please you may want to help with approaching 2.0 release and take
> > > care of one of the sub-tasks that must be included in 2.0:
> > > https://issues.apache.org/jira/browse/IGNITE-4547 <
> > > https://issues.apache.org/jira/browse/IGNITE-4547>
> > >
> > > —
> > > Denis
> > >
> > > > On Jan 15, 2017, at 9:02 PM, Nikita Amelchev 
> > > wrote:
> > > >
> > > > This issue was created long ago. Is still relevant?
> > > >
> > > > JIRA account:
> > > > Username: NSAmelchev
> > > > Full Name: Amelchev Nikita
> > > >
> > > >
> > > > 2017-01-14 1:52 GMT+03:00 Denis Magda :
> > > >
> > > >> Hi Nikita,
> > > >>
> > > >> I can’t find provided account in Ignite JIRA
> > > >> https://issues.apache.org/jira/browse/IGNITE <
> > > https://issues.apache.org/
> > > >> jira/browse/IGNITE>
> > > >>
> > > >> Please create an account there and share with me.
> > > >>
> > > >> This information might be useful for you as well.
> > > >>
> > > >> Subscribe to both dev and user lists:
> > > >> https://ignite.apache.org/community/resources.html#mail-lists
> > > >>
> > > >> Get familiar with Ignite development process described here:
> > > >> https://cwiki.apache.org/confluence/display/IGNITE/
> > Development+Process
> > > >>
> > > >> Instructions on how to contribute can be found here:
> > > >> https://cwiki.apache.org/confluence/display/IGNITE/How+
> to+Contribute
> > > >>
> > > >> Project setup in Intellij IDEAL
> > > >> https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup
> > > >>
> > > >> Regards,
> > > >> Denis
> > > >>
> > > >>> On Jan 13, 2017, at 1:37 AM, Nikita Amelchev  >
> > > >> wrote:
> > > >>>
> > > >>> Hello everyone.
> > > >>>
> > > >>> I'd like to take IGNITE-2894. Can you assign to me?
> > > >>>
> > > >>> Username: NSAmelchev
> > > >>>
> > > >>> --
> > > >>> Best wishes,
> > > >>> Amelchev Nikita
> > > >>
> > > >>
> > > >
> > > >
> > > > --
> > > > Best wishes,
> > > > Amelchev Nikita
> > >
> > >
> >
> >
> > --
> > Best wishes,
> > Amelchev Nikita
> >
>



-- 
Best wishes,
Amelchev Nikita

Re: GridGain Donates Persistent Distributed Store To ASF (Apache Ignite)

[Nikita Ivanov]

Combining this with upcoming new SQL features should give Google Spanner a
run for its money...
--
Nikita Ivanov


On Tue, Apr 18, 2017 at 3:39 AM, christos  wrote:

> This is fantastic news guys!! Finally a real solution that can cater for
> BIG
> AND FAST data!
>
>
>
> --
> View this message in context: http://apache-ignite-
> developers.2346864.n4.nabble.com/GridGain-Donates-
> Persistent-Distributed-Store-To-ASF-Apache-Ignite-tp16788p16836.html
> Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
>

[GitHub] ignite pull request #1811: Ignite 5000

[ybabak]

Github user ybabak closed the pull request at:

https://github.com/apache/ignite/pull/1811


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: GridGain Donates Persistent Distributed Store To ASF (Apache Ignite)

[christos]

This is fantastic news guys!! Finally a real solution that can cater for BIG
AND FAST data! 



--
View this message in context: 
http://apache-ignite-developers.2346864.n4.nabble.com/GridGain-Donates-Persistent-Distributed-Store-To-ASF-Apache-Ignite-tp16788p16836.html
Sent from the Apache Ignite Developers mailing list archive at Nabble.com.

Re: Performance vs correctness: I vote fore the second

[Sergi Vladykin]

Yakov,

The idea of tracking current operations and wait if needed looks
overcomplicated and most probably will result in performance drop.

I think there is no way to have this guarantee with PRIMARY_SYNC in general
case.

Sergi

2017-04-18 13:25 GMT+03:00 Yakov Zhdanov :

> Guys, what if we look at this from another point - we can switch to read
> from primary only if there is any primary_sync operation that is not acked
> by backups yet. Or we can wait until all operations of the kind are acked
> and then proceed with query. This seems to work when we have query after
> sequence of puts, but fails if we have sequence of puts then compute job
> spawning a query from remote node. And this seems to bring lots of
> complications to cache update protocol.
>
> Given this I would vote for switching default (probably, for replicated
> cache only) to full_sync and output a performance warning.
>
> However, there is still an open question - how can I guarantee query
> consistency with primary_sync?
>
> --Yakov
>

Re: Performance vs correctness: I vote fore the second

[Yakov Zhdanov]

Guys, what if we look at this from another point - we can switch to read
from primary only if there is any primary_sync operation that is not acked
by backups yet. Or we can wait until all operations of the kind are acked
and then proceed with query. This seems to work when we have query after
sequence of puts, but fails if we have sequence of puts then compute job
spawning a query from remote node. And this seems to bring lots of
complications to cache update protocol.

Given this I would vote for switching default (probably, for replicated
cache only) to full_sync and output a performance warning.

However, there is still an open question - how can I guarantee query
consistency with primary_sync?

--Yakov

[GitHub] ignite pull request #1816: IGNITE-4990 Remove deprecated properties from Fil...

[tledkov-gridgain]

GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/1816

IGNITE-4990  Remove deprecated properties from FileSystemConfiguration



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4990

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1816.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1816


commit 0b4602e3b6b434951cb303241bc98ec59df79eb4
Author: devozerov 
Date:   2017-04-14T13:28:27Z

Done.

commit 5aeccc03df05a681844b4cd52eed4e4a5ac8a8c1
Author: Alexey Kuznetsov 
Date:   2017-04-17T09:13:04Z

Merge branch master into ignite-4990.

commit 14ba6766e493a8a49201e710ef26129a4da975f1
Author: Alexey Kuznetsov 
Date:   2017-04-17T09:14:32Z

IGNITE-4990 Review.

commit c08b3c856a5d207411174411172103cdd9c031d6
Author: tledkov-gridgain 
Date:   2017-04-18T10:19:04Z

Merge branch '_master' into ignite-4990




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1812: testAllocatedMemory fix

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1812


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1077: Ignite 3920

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1077


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1107: Ignite 3920 1

[devozerov]

Github user devozerov closed the pull request at:

https://github.com/apache/ignite/pull/1107


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Performance vs correctness: I vote fore the second

[Vladimir Ozerov]

Dima,

If change behavior of REPLICATED caches this way, instead of nice
scalability out of the box, users will have slow distributed joins by
default. All we need to do is to set strict FULL_SYNC as default.

On Tue, Apr 18, 2017 at 12:26 PM, Dmitriy Setrakyan 
wrote:

> On Tue, Apr 18, 2017 at 2:21 AM, Sergi Vladykin 
> wrote:
>
> > We never read from backups on partitioned cache, but for replicated we do
> > that to be able to execute the whole query on single node locally.\
> >
>
> But I thought that we agreed to change that behavior and have REPLICATED
> cache work the same as PARTITIONED. I think Valentin provided a link to the
> discussion we had on the dev list.
>
> I would not make FULL_SYNC as default, but I would definitely fix this
> behavior for the REPLICATED caches.
>
> D.
>

Re: Performance vs correctness: I vote fore the second

[Sergi Vladykin]

It only means that we will parse the query always and check if it contains
only replicated tables or not. If it does, then we execute the query on a
single node across all the partitions.

Sergi

2017-04-18 12:26 GMT+03:00 Dmitriy Setrakyan :

> On Tue, Apr 18, 2017 at 2:21 AM, Sergi Vladykin 
> wrote:
>
> > We never read from backups on partitioned cache, but for replicated we do
> > that to be able to execute the whole query on single node locally.\
> >
>
> But I thought that we agreed to change that behavior and have REPLICATED
> cache work the same as PARTITIONED. I think Valentin provided a link to the
> discussion we had on the dev list.
>
> I would not make FULL_SYNC as default, but I would definitely fix this
> behavior for the REPLICATED caches.
>
> D.
>

Re: Performance vs correctness: I vote fore the second

[Dmitriy Setrakyan]

On Tue, Apr 18, 2017 at 2:21 AM, Sergi Vladykin 
wrote:

> We never read from backups on partitioned cache, but for replicated we do
> that to be able to execute the whole query on single node locally.\
>

But I thought that we agreed to change that behavior and have REPLICATED
cache work the same as PARTITIONED. I think Valentin provided a link to the
discussion we had on the dev list.

I would not make FULL_SYNC as default, but I would definitely fix this
behavior for the REPLICATED caches.

D.

[jira] [Created] (IGNITE-5010) Add dynamic index create-drop tests for geo-spacial indexes

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-5010:
---

 Summary: Add dynamic index create-drop tests for geo-spacial 
indexes
 Key: IGNITE-5010
 URL: https://issues.apache.org/jira/browse/IGNITE-5010
 Project: Ignite
  Issue Type: Task
  Components: SQL
Reporter: Vladimir Ozerov
Assignee: Alexander Paschenko
 Fix For: 2.0


Simple create-drop tests should be enough. They should reside in 
{{ignite-geospatial}} module.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] ignite pull request #1815: IGNITE-4188

[SomeFire]

GitHub user SomeFire opened a pull request:

https://github.com/apache/ignite/pull/1815

IGNITE-4188



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/SomeFire/ignite Ignite-4188

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1815.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1815


commit 30430dda81ebdd21b2ccc4aa90529f68af94729c
Author: Dmitrii Ryabov 
Date:   2017-04-13T14:29:41Z

IGNITE-4188 Savepoints support inside of Ignite Transactions

commit 78bfb10ab7bd563a7c4c0eb300939af2509b3ce9
Author: Dmitrii Ryabov 
Date:   2017-04-13T15:23:29Z

Refactoring.

commit d82c8201899a9290bc399fbca1aea8f1c1d1e1c9
Author: Dmitrii Ryabov 
Date:   2017-04-13T15:42:57Z

TxSavepoint javadoc.

commit a3152a839fdba5b40f5b54596d23e1cdf77e0fdd
Author: Dmitrii Ryabov 
Date:   2017-04-14T09:57:56Z

Bug fix.

commit c1eb680ef89a54295a8539739f36fa8483137a32
Author: Dmitrii Ryabov 
Date:   2017-04-14T11:58:43Z

Transaction javadoc.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Performance vs correctness: I vote fore the second

[Sergi Vladykin]

We never read from backups on partitioned cache, but for replicated we do
that to be able to execute the whole query on single node locally.

Sergi

2017-04-18 12:07 GMT+03:00 Dmitriy Setrakyan :

> Sergi, I am confused. If we don't read from backups, then why do we care
> about sync or async backup updates?
>
> On Tue, Apr 18, 2017 at 1:11 AM, Sergi Vladykin 
> wrote:
>
> > Val,
> >
> > There we were not able to run queries against partitioned tables using
> > replicated cache API (I already fixed that in master).
> >
> > Here we are talking about query result inconsistency in case of
> > PRIMARY_SYNC
> > because of async backup update.
> >
> > Sergi
> >
> > 2017-04-18 11:04 GMT+03:00 Valentin Kulichenko <
> > valentin.kuliche...@gmail.com>:
> >
> > > Can you please elaborate then? What is the logic there?
> > >
> > > -Val
> > >
> > > On Tue, Apr 18, 2017 at 9:55 AM, Sergi Vladykin <
> > sergi.vlady...@gmail.com>
> > > wrote:
> > >
> > > > Val,
> > > >
> > > > That discussion has nothing to do with this PRIMARY_SYNC problem.
> > > >
> > > > Sergi
> > > >
> > > > 2017-04-18 10:51 GMT+03:00 Valentin Kulichenko <
> > > > valentin.kuliche...@gmail.com>:
> > > >
> > > > > Sergi,
> > > > >
> > > > > I'm talking about this discussion:
> > > > > http://apache-ignite-developers.2346864.n4.nabble.
> > > > > com/SQL-on-PARTITIONED-vs-REPLICATED-cache-td16478.html
> > > > >
> > > > > -Val
> > > > >
> > > > > On Tue, Apr 18, 2017 at 9:46 AM, Vladimir Ozerov <
> > voze...@gridgain.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Val,
> > > > > >
> > > > > > PRIMARY_SYNC doesn't work correctly with the most common case of
> > SQL
> > > > > query
> > > > > > execution over REPLICATED cache. Also it has weird consequences
> for
> > > > > > continuous queries when coupled with another
> > > > performance-over-correctness
> > > > > > property "readFromBackup=true": user may receive CQ notification
> > with
> > > > new
> > > > > > value, but subsequent GET on local node may return old value.
> > > > > >
> > > > > > On Tue, Apr 18, 2017 at 10:42 AM, Valentin Kulichenko <
> > > > > > valentin.kuliche...@gmail.com> wrote:
> > > > > >
> > > > > > > This sounds more like an issue with query execution, rather
> than
> > > > wrong
> > > > > > > PRIMARY_SYNC
> > > > > > > behavior. We already had a discussion about this optimization
> in
> > > > > > replicated
> > > > > > > cache and decided to switch it off by default.
> > > > > > >
> > > > > > > -Val
> > > > > > >
> > > > > > > On Tue, Apr 18, 2017 at 9:35 AM, Sergi Vladykin <
> > > > > > sergi.vlady...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > With replicated cache we can execute a query against backup
> > > > > partitions
> > > > > > > that
> > > > > > > > were not updated yet because of PRIMARY_SYNC. Thus we do not
> > see
> > > an
> > > > > > > update.
> > > > > > > >
> > > > > > > > Sergi
> > > > > > > >
> > > > > > > > 2017-04-18 10:30 GMT+03:00 Dmitriy Setrakyan <
> > > > dsetrak...@apache.org
> > > > > >:
> > > > > > > >
> > > > > > > > > Vladimir,
> > > > > > > > >
> > > > > > > > > What is wrong with a query in PRIMARY_SYNC mode? Why won't
> it
> > > > work?
> > > > > > > > >
> > > > > > > > > D.
> > > > > > > > >
> > > > > > > > > On Tue, Apr 18, 2017 at 12:25 AM, Vladimir Ozerov <
> > > > > > > voze...@gridgain.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Folks,
> > > > > > > > > >
> > > > > > > > > > I received a number of complaints from users that our
> > default
> > > > > > setting
> > > > > > > > > favor
> > > > > > > > > > performance at the cost of correctness and subtle
> behavior.
> > > > > > > Yesterday I
> > > > > > > > > > faced one such situation on my own.
> > > > > > > > > >
> > > > > > > > > > I started REPLICATED cache on several nodes, put some
> data,
> > > > > > executed
> > > > > > > > > simple
> > > > > > > > > > SQL and got wrong result. No errors, no warnings. The
> > problem
> > > > was
> > > > > > > > caused
> > > > > > > > > by
> > > > > > > > > > default PRIMARY_SYNC mode. WTF, our cache doesn't work
> out
> > of
> > > > the
> > > > > > > box!
> > > > > > > > > >
> > > > > > > > > > Another widely known examples are data streamer behavior,
> > > "read
> > > > > > form
> > > > > > > > > > backups" + continuous queries.
> > > > > > > > > >
> > > > > > > > > > I propose to change our defaults to favor *correctness*
> > over
> > > > > > > > performance,
> > > > > > > > > > and create good documentation and JavaDocs to explain
> users
> > > how
> > > > > to
> > > > > > > tune
> > > > > > > > > our
> > > > > > > > > > product. Proposed changes:
> > > > > > > > > >
> > > > > > > > > > 1) FULL_SYNC as default;
> > > > > > > > > > 2) "readFromBackups=false" as default;
> > > > > > > > > > 3) "IgniteDataStreamer.allowOverwrite=true" as default.
> > > > > > > > > >
> > > > > > > > > > Users should not think how to make Ignite work correctly.
> > It
> > > > > 

Re: TouchedExpiryPolicy works incorrect in some cases IGNITE-4401

[ALEKSEY KUZNETSOV]

Yeah. I've already run the test with two caches. Definately, the bug hidden
in cache.query() method. cache.query() calls
IgniteH2Indexing#queryLocalSql(), which calls executeSqlQueryWithTimer, and
then it sinks into JdbcPreparedStatement.executeQuery(). There is no
key-value operations subsequent

пн, 17 апр. 2017 г. в 21:15, Denis Magda :

> It doesn’t matter who is right and who is wrong unless someone gets to the
> bottom of the issue debugging it.
>
> I would suggest to create a simple unit test with two caches and trying to
> reproduce the following without computations and other redundant stuff.
> Would you like to work on this?
>
> —
> Denis
>
> > On Apr 17, 2017, at 12:44 AM, ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com> wrote:
> >
> > Why do u think so.
> > First of all, the output above is not correct. After 3 iteration
> key-value
> > API strats to return empty value.
> >
> > Every 5 seconds(iteration sleep time)
> repository.getAttributes("1").size()
> > is got called. Which makes an entry "touch" and the entry wont be expired
> > for as long as 10 seconds.
> >
> > Expiry policy says:
> >
> > An {@link ExpiryPolicy} that defines the expiry {@link Duration}
> > * of a Cache Entry based on when it was last touched. A touch includes
> > ** creation, update or **access**.*
> >
> >
> > пт, 14 апр. 2017 г. в 18:42, Denis Magda :
> >
> >> The iteration happens multiple time which means that the key-value API
> had
> >> to return an empty result set on second or third iteration. But this
> never
> >> happens.
> >>
> >> In any case, do you want to find a root of the issue and fix it?
> >> Otherwise, we can update the description and wait while someone else
> fixes
> >> it.
> >>
> >> —
> >> Denis
> >>
> >>> On Apr 14, 2017, at 1:33 AM, ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com>
> >> wrote:
> >>>
> >>> Because expiry time is 10 seconds, while loop iterates every 5 seconds
> >>>
> >>> пт, 14 апр. 2017 г. в 11:32, ALEKSEY KUZNETSOV <
> alkuznetsov...@gmail.com
> >>> :
> >>>
>  No, the bug is in SQL query, not key-value storage.
> 
>  пт, 14 апр. 2017 г. в 11:11, Vladislav Pyatkov  >:
> 
> > Denis, Aleksey,
> >
> > It is correct, remember I have already said something like[1].
> > I have no idea, why this happened in this case with SQL.
> >
> > [1]:
> >
> >
> >>
> http://apache-ignite-developers.2346864.n4.nabble.com/TouchedExpiryPolicy-works-incorrect-in-some-cases-IGNITE-4401-td16349.html#a16356
> >
> > On Fri, Apr 14, 2017 at 4:29 AM, Denis Magda 
> >> wrote:
> >
> >> I could reproduce the issue and this should be what Denis K. meant
> by
> >> saying “expiration policy works incorrectly”.
> >>
> >> If you remove the expiration policy from the caches' configuration
> >> then
> >> the issue disappears. In general, SQL engine processes an expiration
> > event
> >> properly because the SQL queries return an empty result set as
> >> expected
> > but
> >> something doesn’t work well with key-value operations.
> >>
> >> *Denis K*, *Vlad P.*, as creators of the ticket please confirm that
> >> this
> >> is the case.
> >>
> >> Please keep debugging this and switch to the latest Ignite version.
> >>
> >> —
> >> Denis
> >>
> >>
> >>> On Apr 13, 2017, at 4:22 AM, ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com>
> >> wrote:
> >>>
> >>> any feedback?
> >>>
> >>> чт, 13 апр. 2017 г. в 11:51, ALEKSEY KUZNETSOV <
> > alkuznetsov...@gmail.com
> >>> :
> >>>
>  You should run ExpiryPolicyTest. The output should contain strings
> > like
>  contains? new AffinityKey("1", "1"): and contains?2 new
> >> AffinityKey("1", "
>  1"): and empty cursor? =
>  If you look at them you will see, that cache contains affinity key
> > new
>  AffinityKey("1", "1") whereas cursor is empty(on second
> iteration).
> > From
>  this output you can conclude SQL query returns icorrect data(empty
> >> value)
> 
> 
>  чт, 13 апр. 2017 г. в 3:42, Denis Magda :
> 
> > Bluntly speaking I have no idea where to look and what to expect.
> > This
> >> is
> > output of the test execution of my machine:
> >
> > SQL res: [[1], [d]]
> > 2
> > Op consume: 303
> > Value: org.ignite.test.EDU@22db8f4
> > SQL res: []
> > 0
> > Op consume: 9
> > Value: org.ignite.test.EDU@29caf222
> > SQL res: []
> > 0
> > Op consume: 15
> > Value: org.ignite.test.EDU@7cd1ac19
> > SQL res: []
> > 0
> > Op consume: 5
> >
> > Please be more specific, there are too many files in the code.
> >
> > —
> > Denis
> >
> 

[GitHub] ignite pull request #1814: IGNITE-4939 Receive event before cache initialize...

[ezhuravl]

GitHub user ezhuravl reopened a pull request:

https://github.com/apache/ignite/pull/1814

IGNITE-4939 Receive event before cache initialized fix



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4939

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1814.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1814


commit e448a4d011c4137528f7915600fdf32cc5455128
Author: Evgenii Zhuravlev 
Date:   2017-04-18T09:14:07Z

IGNITE-4939 Receive event before cache initialized fix




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1814: IGNITE-4939 Receive event before cache initialize...

[ezhuravl]

Github user ezhuravl closed the pull request at:

https://github.com/apache/ignite/pull/1814


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1814: IGNITE-4939 Receive event before cache initialize...

[ezhuravl]

GitHub user ezhuravl opened a pull request:

https://github.com/apache/ignite/pull/1814

IGNITE-4939 Receive event before cache initialized fix



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4939

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1814.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1814


commit e448a4d011c4137528f7915600fdf32cc5455128
Author: Evgenii Zhuravlev 
Date:   2017-04-18T09:14:07Z

IGNITE-4939 Receive event before cache initialized fix




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Page Memory behavior with default memory policy

[Dmitriy Setrakyan]

On Tue, Apr 18, 2017 at 2:09 AM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> I don't see why not, this is the way our tests are currently running.
> Anyways, we can think about efficient dynamic memory expansion in 2.1, this
> may be feasible if we free up some space in PageId to encode segment
> number. There is a ticket for this:
> https://issues.apache.org/jira/browse/IGNITE-4921


Alexey, if the operating system is already handling this for us, I don't
see a reason to do it manually.

I also like what Denis and Semyon are proposing. However, I would not grab
the full free memory. How about 80% of the free memory?

D.

Re: Page Memory behavior with default memory policy

[Alexey Goncharuk]

I don't see why not, this is the way our tests are currently running.
Anyways, we can think about efficient dynamic memory expansion in 2.1, this
may be feasible if we free up some space in PageId to encode segment
number. There is a ticket for this:
https://issues.apache.org/jira/browse/IGNITE-4921

2017-04-18 12:04 GMT+03:00 Dmitriy Setrakyan :

> Thanks, Alexey, got it. What happens if a user starts multiple nodes on the
> same box? Will it work the way Denis suggested?
>
> On Tue, Apr 18, 2017 at 1:47 AM, Alexey Goncharuk <
> alexey.goncha...@gmail.com> wrote:
>
> > The reason we cannot add another memory region to page memory is
> > performance. Currently, the translation of a page ID to a virtual address
> > is basically a no-op. If we add dynamic memory expansion, we need to
> > maintain some sort of page ID mapping, which adds significant (in my
> view,
> > about 10%) overhead.
> >
> > Agree with Semyon, we can set default size based on available memory on a
> > machine.
> >
> > 2017-04-18 11:38 GMT+03:00 Semyon Boikov :
> >
> > > I think Ignite can behave like JVM: we can have -Xms -Xmx settings with
> > > defaults depending on available memory.
> > >
> > > Thanks
> > >
> > > On Tue, Apr 18, 2017 at 4:56 AM, Dmitriy Setrakyan <
> > dsetrak...@apache.org>
> > > wrote:
> > >
> > > > Denis,
> > > >
> > > > If what you are suggesting is true, then we can always allocate about
> > 80%
> > > > of available memory by default. By the way, it must also work on
> > Windows,
> > > > so we should definitely test it.
> > > >
> > > > Alexey G, can you comment?
> > > >
> > > > D.
> > > >
> > > > On Mon, Apr 17, 2017 at 6:17 PM, Denis Magda 
> > wrote:
> > > >
> > > > > Dmitriy,
> > > > >
> > > > > > Denis, it sounds like with this approach, in case of the
> > > > over-allocation,
> > > > > > the system will just get slower and slower and users will end up
> > > > blaming
> > > > > > Ignite for it. Am I understanding your suggestion correctly?
> > > > >
> > > > >
> > > > > This will not happen (at least in Unix) unless all the nodes really
> > > used
> > > > > all the allocated memory by putting data there or touching all the
> > > memory
> > > > > range somehow else.
> > > > >
> > > > > > How was this handled in Ignite 1.9?
> > > > >
> > > > >
> > > > > If you are talking about the legacy off-heap impl then we requested
> > > small
> > > > > chunks of data from an operating system rather than a continuous
> > memory
> > > > > region as in the page memory. But I would think of the page memory
> as
> > > of
> > > > > Java heap which also can request 8 GB continuous memory region on
> a 8
> > > GB
> > > > > machine following heap settings of an app but an operating system
> > will
> > > > not
> > > > > return the whole range immediately unless Java app occupies the
> whole
> > > > heap
> > > > > or use special parameters.
> > > > >
> > > > > At all, I think it’s safe to use approach suggested by me unless I
> > miss
> > > > > something.
> > > > >
> > > > > —
> > > > > Denis
> > > > >
> > > > > > On Apr 17, 2017, at 6:05 PM, Dmitriy Setrakyan <
> > > dsetrak...@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > On Mon, Apr 17, 2017 at 6:00 PM, Denis Magda 
> > > > wrote:
> > > > > >
> > > > > >> Dmitriy,
> > > > > >>
> > > > > >> All the nodes will request its own continuous memory region that
> > > takes
> > > > > >> 70-80% of all RAM from an underlying operation system. However,
> > the
> > > > > >> operating system will not outfit the nodes with physical pages
> > > mapped
> > > > to
> > > > > >> RAM immediately allowing every node's process to start
> > successfully.
> > > > The
> > > > > >> nodes will communicate to RAM via a virtual memory which in its
> > turn
> > > > > will
> > > > > >> give an access to physical pages whenever is needed applying low
> > > level
> > > > > >> eviction and swapping techniques.
> > > > > >>
> > > > > >
> > > > > > Denis, it sounds like with this approach, in case of the
> > > > over-allocation,
> > > > > > the system will just get slower and slower and users will end up
> > > > blaming
> > > > > > Ignite for it. Am I understanding your suggestion correctly?
> > > > > >
> > > > > > How was this handled in Ignite 1.9?
> > > > > >
> > > > > > D.
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: Performance vs correctness: I vote fore the second

[Dmitriy Setrakyan]

Sergi, I am confused. If we don't read from backups, then why do we care
about sync or async backup updates?

On Tue, Apr 18, 2017 at 1:11 AM, Sergi Vladykin 
wrote:

> Val,
>
> There we were not able to run queries against partitioned tables using
> replicated cache API (I already fixed that in master).
>
> Here we are talking about query result inconsistency in case of
> PRIMARY_SYNC
> because of async backup update.
>
> Sergi
>
> 2017-04-18 11:04 GMT+03:00 Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
>
> > Can you please elaborate then? What is the logic there?
> >
> > -Val
> >
> > On Tue, Apr 18, 2017 at 9:55 AM, Sergi Vladykin <
> sergi.vlady...@gmail.com>
> > wrote:
> >
> > > Val,
> > >
> > > That discussion has nothing to do with this PRIMARY_SYNC problem.
> > >
> > > Sergi
> > >
> > > 2017-04-18 10:51 GMT+03:00 Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com>:
> > >
> > > > Sergi,
> > > >
> > > > I'm talking about this discussion:
> > > > http://apache-ignite-developers.2346864.n4.nabble.
> > > > com/SQL-on-PARTITIONED-vs-REPLICATED-cache-td16478.html
> > > >
> > > > -Val
> > > >
> > > > On Tue, Apr 18, 2017 at 9:46 AM, Vladimir Ozerov <
> voze...@gridgain.com
> > >
> > > > wrote:
> > > >
> > > > > Val,
> > > > >
> > > > > PRIMARY_SYNC doesn't work correctly with the most common case of
> SQL
> > > > query
> > > > > execution over REPLICATED cache. Also it has weird consequences for
> > > > > continuous queries when coupled with another
> > > performance-over-correctness
> > > > > property "readFromBackup=true": user may receive CQ notification
> with
> > > new
> > > > > value, but subsequent GET on local node may return old value.
> > > > >
> > > > > On Tue, Apr 18, 2017 at 10:42 AM, Valentin Kulichenko <
> > > > > valentin.kuliche...@gmail.com> wrote:
> > > > >
> > > > > > This sounds more like an issue with query execution, rather than
> > > wrong
> > > > > > PRIMARY_SYNC
> > > > > > behavior. We already had a discussion about this optimization in
> > > > > replicated
> > > > > > cache and decided to switch it off by default.
> > > > > >
> > > > > > -Val
> > > > > >
> > > > > > On Tue, Apr 18, 2017 at 9:35 AM, Sergi Vladykin <
> > > > > sergi.vlady...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > With replicated cache we can execute a query against backup
> > > > partitions
> > > > > > that
> > > > > > > were not updated yet because of PRIMARY_SYNC. Thus we do not
> see
> > an
> > > > > > update.
> > > > > > >
> > > > > > > Sergi
> > > > > > >
> > > > > > > 2017-04-18 10:30 GMT+03:00 Dmitriy Setrakyan <
> > > dsetrak...@apache.org
> > > > >:
> > > > > > >
> > > > > > > > Vladimir,
> > > > > > > >
> > > > > > > > What is wrong with a query in PRIMARY_SYNC mode? Why won't it
> > > work?
> > > > > > > >
> > > > > > > > D.
> > > > > > > >
> > > > > > > > On Tue, Apr 18, 2017 at 12:25 AM, Vladimir Ozerov <
> > > > > > voze...@gridgain.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Folks,
> > > > > > > > >
> > > > > > > > > I received a number of complaints from users that our
> default
> > > > > setting
> > > > > > > > favor
> > > > > > > > > performance at the cost of correctness and subtle behavior.
> > > > > > Yesterday I
> > > > > > > > > faced one such situation on my own.
> > > > > > > > >
> > > > > > > > > I started REPLICATED cache on several nodes, put some data,
> > > > > executed
> > > > > > > > simple
> > > > > > > > > SQL and got wrong result. No errors, no warnings. The
> problem
> > > was
> > > > > > > caused
> > > > > > > > by
> > > > > > > > > default PRIMARY_SYNC mode. WTF, our cache doesn't work out
> of
> > > the
> > > > > > box!
> > > > > > > > >
> > > > > > > > > Another widely known examples are data streamer behavior,
> > "read
> > > > > form
> > > > > > > > > backups" + continuous queries.
> > > > > > > > >
> > > > > > > > > I propose to change our defaults to favor *correctness*
> over
> > > > > > > performance,
> > > > > > > > > and create good documentation and JavaDocs to explain users
> > how
> > > > to
> > > > > > tune
> > > > > > > > our
> > > > > > > > > product. Proposed changes:
> > > > > > > > >
> > > > > > > > > 1) FULL_SYNC as default;
> > > > > > > > > 2) "readFromBackups=false" as default;
> > > > > > > > > 3) "IgniteDataStreamer.allowOverwrite=true" as default.
> > > > > > > > >
> > > > > > > > > Users should not think how to make Ignite work correctly.
> It
> > > > should
> > > > > > be
> > > > > > > > > correct out of the box.
> > > > > > > > >
> > > > > > > > > Vladimir.
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Page Memory behavior with default memory policy

[Dmitriy Setrakyan]

Thanks, Alexey, got it. What happens if a user starts multiple nodes on the
same box? Will it work the way Denis suggested?

On Tue, Apr 18, 2017 at 1:47 AM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> The reason we cannot add another memory region to page memory is
> performance. Currently, the translation of a page ID to a virtual address
> is basically a no-op. If we add dynamic memory expansion, we need to
> maintain some sort of page ID mapping, which adds significant (in my view,
> about 10%) overhead.
>
> Agree with Semyon, we can set default size based on available memory on a
> machine.
>
> 2017-04-18 11:38 GMT+03:00 Semyon Boikov :
>
> > I think Ignite can behave like JVM: we can have -Xms -Xmx settings with
> > defaults depending on available memory.
> >
> > Thanks
> >
> > On Tue, Apr 18, 2017 at 4:56 AM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> > wrote:
> >
> > > Denis,
> > >
> > > If what you are suggesting is true, then we can always allocate about
> 80%
> > > of available memory by default. By the way, it must also work on
> Windows,
> > > so we should definitely test it.
> > >
> > > Alexey G, can you comment?
> > >
> > > D.
> > >
> > > On Mon, Apr 17, 2017 at 6:17 PM, Denis Magda 
> wrote:
> > >
> > > > Dmitriy,
> > > >
> > > > > Denis, it sounds like with this approach, in case of the
> > > over-allocation,
> > > > > the system will just get slower and slower and users will end up
> > > blaming
> > > > > Ignite for it. Am I understanding your suggestion correctly?
> > > >
> > > >
> > > > This will not happen (at least in Unix) unless all the nodes really
> > used
> > > > all the allocated memory by putting data there or touching all the
> > memory
> > > > range somehow else.
> > > >
> > > > > How was this handled in Ignite 1.9?
> > > >
> > > >
> > > > If you are talking about the legacy off-heap impl then we requested
> > small
> > > > chunks of data from an operating system rather than a continuous
> memory
> > > > region as in the page memory. But I would think of the page memory as
> > of
> > > > Java heap which also can request 8 GB continuous memory region on a 8
> > GB
> > > > machine following heap settings of an app but an operating system
> will
> > > not
> > > > return the whole range immediately unless Java app occupies the whole
> > > heap
> > > > or use special parameters.
> > > >
> > > > At all, I think it’s safe to use approach suggested by me unless I
> miss
> > > > something.
> > > >
> > > > —
> > > > Denis
> > > >
> > > > > On Apr 17, 2017, at 6:05 PM, Dmitriy Setrakyan <
> > dsetrak...@apache.org>
> > > > wrote:
> > > > >
> > > > > On Mon, Apr 17, 2017 at 6:00 PM, Denis Magda 
> > > wrote:
> > > > >
> > > > >> Dmitriy,
> > > > >>
> > > > >> All the nodes will request its own continuous memory region that
> > takes
> > > > >> 70-80% of all RAM from an underlying operation system. However,
> the
> > > > >> operating system will not outfit the nodes with physical pages
> > mapped
> > > to
> > > > >> RAM immediately allowing every node's process to start
> successfully.
> > > The
> > > > >> nodes will communicate to RAM via a virtual memory which in its
> turn
> > > > will
> > > > >> give an access to physical pages whenever is needed applying low
> > level
> > > > >> eviction and swapping techniques.
> > > > >>
> > > > >
> > > > > Denis, it sounds like with this approach, in case of the
> > > over-allocation,
> > > > > the system will just get slower and slower and users will end up
> > > blaming
> > > > > Ignite for it. Am I understanding your suggestion correctly?
> > > > >
> > > > > How was this handled in Ignite 1.9?
> > > > >
> > > > > D.
> > > >
> > > >
> > >
> >
>

Re: Page Memory behavior with default memory policy

[Alexey Goncharuk]

The reason we cannot add another memory region to page memory is
performance. Currently, the translation of a page ID to a virtual address
is basically a no-op. If we add dynamic memory expansion, we need to
maintain some sort of page ID mapping, which adds significant (in my view,
about 10%) overhead.

Agree with Semyon, we can set default size based on available memory on a
machine.

2017-04-18 11:38 GMT+03:00 Semyon Boikov :

> I think Ignite can behave like JVM: we can have -Xms -Xmx settings with
> defaults depending on available memory.
>
> Thanks
>
> On Tue, Apr 18, 2017 at 4:56 AM, Dmitriy Setrakyan 
> wrote:
>
> > Denis,
> >
> > If what you are suggesting is true, then we can always allocate about 80%
> > of available memory by default. By the way, it must also work on Windows,
> > so we should definitely test it.
> >
> > Alexey G, can you comment?
> >
> > D.
> >
> > On Mon, Apr 17, 2017 at 6:17 PM, Denis Magda  wrote:
> >
> > > Dmitriy,
> > >
> > > > Denis, it sounds like with this approach, in case of the
> > over-allocation,
> > > > the system will just get slower and slower and users will end up
> > blaming
> > > > Ignite for it. Am I understanding your suggestion correctly?
> > >
> > >
> > > This will not happen (at least in Unix) unless all the nodes really
> used
> > > all the allocated memory by putting data there or touching all the
> memory
> > > range somehow else.
> > >
> > > > How was this handled in Ignite 1.9?
> > >
> > >
> > > If you are talking about the legacy off-heap impl then we requested
> small
> > > chunks of data from an operating system rather than a continuous memory
> > > region as in the page memory. But I would think of the page memory as
> of
> > > Java heap which also can request 8 GB continuous memory region on a 8
> GB
> > > machine following heap settings of an app but an operating system will
> > not
> > > return the whole range immediately unless Java app occupies the whole
> > heap
> > > or use special parameters.
> > >
> > > At all, I think it’s safe to use approach suggested by me unless I miss
> > > something.
> > >
> > > —
> > > Denis
> > >
> > > > On Apr 17, 2017, at 6:05 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> > > wrote:
> > > >
> > > > On Mon, Apr 17, 2017 at 6:00 PM, Denis Magda 
> > wrote:
> > > >
> > > >> Dmitriy,
> > > >>
> > > >> All the nodes will request its own continuous memory region that
> takes
> > > >> 70-80% of all RAM from an underlying operation system. However, the
> > > >> operating system will not outfit the nodes with physical pages
> mapped
> > to
> > > >> RAM immediately allowing every node's process to start successfully.
> > The
> > > >> nodes will communicate to RAM via a virtual memory which in its turn
> > > will
> > > >> give an access to physical pages whenever is needed applying low
> level
> > > >> eviction and swapping techniques.
> > > >>
> > > >
> > > > Denis, it sounds like with this approach, in case of the
> > over-allocation,
> > > > the system will just get slower and slower and users will end up
> > blaming
> > > > Ignite for it. Am I understanding your suggestion correctly?
> > > >
> > > > How was this handled in Ignite 1.9?
> > > >
> > > > D.
> > >
> > >
> >
>

Re: Page Memory behavior with default memory policy

[Semyon Boikov]

I think Ignite can behave like JVM: we can have -Xms -Xmx settings with
defaults depending on available memory.

Thanks

On Tue, Apr 18, 2017 at 4:56 AM, Dmitriy Setrakyan 
wrote:

> Denis,
>
> If what you are suggesting is true, then we can always allocate about 80%
> of available memory by default. By the way, it must also work on Windows,
> so we should definitely test it.
>
> Alexey G, can you comment?
>
> D.
>
> On Mon, Apr 17, 2017 at 6:17 PM, Denis Magda  wrote:
>
> > Dmitriy,
> >
> > > Denis, it sounds like with this approach, in case of the
> over-allocation,
> > > the system will just get slower and slower and users will end up
> blaming
> > > Ignite for it. Am I understanding your suggestion correctly?
> >
> >
> > This will not happen (at least in Unix) unless all the nodes really used
> > all the allocated memory by putting data there or touching all the memory
> > range somehow else.
> >
> > > How was this handled in Ignite 1.9?
> >
> >
> > If you are talking about the legacy off-heap impl then we requested small
> > chunks of data from an operating system rather than a continuous memory
> > region as in the page memory. But I would think of the page memory as of
> > Java heap which also can request 8 GB continuous memory region on a 8 GB
> > machine following heap settings of an app but an operating system will
> not
> > return the whole range immediately unless Java app occupies the whole
> heap
> > or use special parameters.
> >
> > At all, I think it’s safe to use approach suggested by me unless I miss
> > something.
> >
> > —
> > Denis
> >
> > > On Apr 17, 2017, at 6:05 PM, Dmitriy Setrakyan 
> > wrote:
> > >
> > > On Mon, Apr 17, 2017 at 6:00 PM, Denis Magda 
> wrote:
> > >
> > >> Dmitriy,
> > >>
> > >> All the nodes will request its own continuous memory region that takes
> > >> 70-80% of all RAM from an underlying operation system. However, the
> > >> operating system will not outfit the nodes with physical pages mapped
> to
> > >> RAM immediately allowing every node's process to start successfully.
> The
> > >> nodes will communicate to RAM via a virtual memory which in its turn
> > will
> > >> give an access to physical pages whenever is needed applying low level
> > >> eviction and swapping techniques.
> > >>
> > >
> > > Denis, it sounds like with this approach, in case of the
> over-allocation,
> > > the system will just get slower and slower and users will end up
> blaming
> > > Ignite for it. Am I understanding your suggestion correctly?
> > >
> > > How was this handled in Ignite 1.9?
> > >
> > > D.
> >
> >
>

Re: IGNITE-4799 MetricsUpdate after each job

[Yakov Zhdanov]

Denis, thanks! That makes sense to me.

Sasha, please go ahead and change code to interpret this parameter as exact
frequency in milliseconds.

--
Yakov Zhdanov

Re: cache query operations

[ALEKSEY KUZNETSOV]

if executing sql "select" clause then no cache key-value operations would
be called.
Debbuging cache.query(...) leads
to 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing#executeSqlQuery
which doesnt call cache key-value ops

вт, 18 апр. 2017 г. в 10:49, ALEKSEY KUZNETSOV :

> thanx! )
>
> пн, 17 апр. 2017 г. в 21:48, Denis Magda :
>
>> Did you have a chance to take a look this documentation that explains how
>> DML statements are turned into cache operations?
>> https://apacheignite.readme.io/docs/dml#dml-operations <
>> https://apacheignite.readme.io/docs/dml#dml-operations>
>>
>> —
>> Denis
>>
>> > On Apr 17, 2017, at 7:20 AM, ALEKSEY KUZNETSOV <
>> alkuznetsov...@gmail.com> wrote:
>> >
>> > Hi, Igniters! When one fires query like this : ignite().cache(null
>> > ).query(...)
>> > The query would be executed against the schema. I wonder, how the
>> schema is
>> > synchronized with cache, with cache entries?
>> >
>> > What i mean is , cache.query(...update operation...) must eventually
>> update
>> > the cache entry in cache. But how does it happen?(cache API must be
>> called,
>> > but i cannot realize how)
>> > --
>> >
>> > *Best Regards,*
>> >
>> > *Kuznetsov Aleksey*
>>
>> --
>
> *Best Regards,*
>
> *Kuznetsov Aleksey*
>
-- 

*Best Regards,*

*Kuznetsov Aleksey*

Introduce custom executors for compute grid

[Taras Ledkov]

Igniters,

Custom executor (user's thread pool) is added fro compute grid with 
following semantics:


1. Configuration:

IgniteConfiguration cfg;
...
cfg.setExecutorConfiguration(
new ExecutorConfiguration().setName("executor0").setSize(2),
new ExecutorConfiguration().setName("executor1").setSize(4));

Where
name - name of executor and thread pool;
size - thread pool size.

2. Usage:

Ignite ignite;
...
IgniteCompute comp = ignite.compute().withExecutor("executor0");
comp.broadcast(new IgniteRunnable() {
@Override public void run() {
 ...
}
});

So, 'withExecutor(String)' returns the compute associated with custom 
named executor.
All jobs submitted by the components will be processed by thread pool 
corresponds to named executor.
If the executor isn't configured on the target host the warning will be 
printed in the log and a job will be processed in the public pool.

e.g.:
[11:20:01,023][WARN 
][grid-nio-worker-tcp-comm-0-#27%compute.IgniteComputeCustomExecutorSelfTest1%][GridIoManager] 
Custom executor 'invalid' doesn't exist. The job will be submit to 
public pool: b2e85208b51-4fbcb569-07a2-480e-9be1-512bc320


Issue: https://issues.apache.org/jira/browse/IGNITE-4699

Please share your thoughts or ask questions.

--
Taras Ledkov
Mail-To: tled...@gridgain.com

[GitHub] ignite pull request #1787: IGNITE-2398 .NET: Change default mapper behavior

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1787


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-5009) Backport IGNITE-4932 optimization in 2.x release

[Semen Boikov (JIRA)]

Semen Boikov created IGNITE-5009:


 Summary: Backport IGNITE-4932 optimization in 2.x release
 Key: IGNITE-5009
 URL: https://issues.apache.org/jira/browse/IGNITE-5009
 Project: Ignite
  Issue Type: Task
  Components: cache
Reporter: Semen Boikov
Assignee: Semen Boikov
Priority: Critical
 Fix For: 2.1


In IGNITE-4932 we implemented for 1.9 codebase optimization to avoid cache 
entry creation for cache.get operation, need backport this optimization in 2.x 
release.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] ignite pull request #1774: IGNITE-4946

[gvvinblade]

Github user gvvinblade closed the pull request at:

https://github.com/apache/ignite/pull/1774


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Performance vs correctness: I vote fore the second

[Sergi Vladykin]

Val,

There we were not able to run queries against partitioned tables using
replicated cache API (I already fixed that in master).

Here we are talking about query result inconsistency in case of PRIMARY_SYNC
because of async backup update.

Sergi

2017-04-18 11:04 GMT+03:00 Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Can you please elaborate then? What is the logic there?
>
> -Val
>
> On Tue, Apr 18, 2017 at 9:55 AM, Sergi Vladykin 
> wrote:
>
> > Val,
> >
> > That discussion has nothing to do with this PRIMARY_SYNC problem.
> >
> > Sergi
> >
> > 2017-04-18 10:51 GMT+03:00 Valentin Kulichenko <
> > valentin.kuliche...@gmail.com>:
> >
> > > Sergi,
> > >
> > > I'm talking about this discussion:
> > > http://apache-ignite-developers.2346864.n4.nabble.
> > > com/SQL-on-PARTITIONED-vs-REPLICATED-cache-td16478.html
> > >
> > > -Val
> > >
> > > On Tue, Apr 18, 2017 at 9:46 AM, Vladimir Ozerov  >
> > > wrote:
> > >
> > > > Val,
> > > >
> > > > PRIMARY_SYNC doesn't work correctly with the most common case of SQL
> > > query
> > > > execution over REPLICATED cache. Also it has weird consequences for
> > > > continuous queries when coupled with another
> > performance-over-correctness
> > > > property "readFromBackup=true": user may receive CQ notification with
> > new
> > > > value, but subsequent GET on local node may return old value.
> > > >
> > > > On Tue, Apr 18, 2017 at 10:42 AM, Valentin Kulichenko <
> > > > valentin.kuliche...@gmail.com> wrote:
> > > >
> > > > > This sounds more like an issue with query execution, rather than
> > wrong
> > > > > PRIMARY_SYNC
> > > > > behavior. We already had a discussion about this optimization in
> > > > replicated
> > > > > cache and decided to switch it off by default.
> > > > >
> > > > > -Val
> > > > >
> > > > > On Tue, Apr 18, 2017 at 9:35 AM, Sergi Vladykin <
> > > > sergi.vlady...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > With replicated cache we can execute a query against backup
> > > partitions
> > > > > that
> > > > > > were not updated yet because of PRIMARY_SYNC. Thus we do not see
> an
> > > > > update.
> > > > > >
> > > > > > Sergi
> > > > > >
> > > > > > 2017-04-18 10:30 GMT+03:00 Dmitriy Setrakyan <
> > dsetrak...@apache.org
> > > >:
> > > > > >
> > > > > > > Vladimir,
> > > > > > >
> > > > > > > What is wrong with a query in PRIMARY_SYNC mode? Why won't it
> > work?
> > > > > > >
> > > > > > > D.
> > > > > > >
> > > > > > > On Tue, Apr 18, 2017 at 12:25 AM, Vladimir Ozerov <
> > > > > voze...@gridgain.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Folks,
> > > > > > > >
> > > > > > > > I received a number of complaints from users that our default
> > > > setting
> > > > > > > favor
> > > > > > > > performance at the cost of correctness and subtle behavior.
> > > > > Yesterday I
> > > > > > > > faced one such situation on my own.
> > > > > > > >
> > > > > > > > I started REPLICATED cache on several nodes, put some data,
> > > > executed
> > > > > > > simple
> > > > > > > > SQL and got wrong result. No errors, no warnings. The problem
> > was
> > > > > > caused
> > > > > > > by
> > > > > > > > default PRIMARY_SYNC mode. WTF, our cache doesn't work out of
> > the
> > > > > box!
> > > > > > > >
> > > > > > > > Another widely known examples are data streamer behavior,
> "read
> > > > form
> > > > > > > > backups" + continuous queries.
> > > > > > > >
> > > > > > > > I propose to change our defaults to favor *correctness* over
> > > > > > performance,
> > > > > > > > and create good documentation and JavaDocs to explain users
> how
> > > to
> > > > > tune
> > > > > > > our
> > > > > > > > product. Proposed changes:
> > > > > > > >
> > > > > > > > 1) FULL_SYNC as default;
> > > > > > > > 2) "readFromBackups=false" as default;
> > > > > > > > 3) "IgniteDataStreamer.allowOverwrite=true" as default.
> > > > > > > >
> > > > > > > > Users should not think how to make Ignite work correctly. It
> > > should
> > > > > be
> > > > > > > > correct out of the box.
> > > > > > > >
> > > > > > > > Vladimir.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: IGNITE-4052 ready for review

[Вадим Опольский]

Hi Nikolay!

The comments was added.

Vadim

2017-04-17 15:32 GMT+03:00 Nikolai Tikhonov :

> Vadim,
>
> Thank you for your contribution. I'll look at changes. Can you please post
> the list improvements to jira ticket?
>
> On Mon, Apr 17, 2017 at 3:23 PM, Вадим Опольский 
> wrote:
>
>> Hi, Nikolay!
>>
>> I've made the following improvements ( https://github.com/apache/igni
>> te/pull/1783 ):
>>
>> 1) Moved framework builder related code to separate method, so it make
>> code cleaner and framework testable.
>>
>> 2) Call framework builder method from test to test role and user.
>>
>> 3) Added validating mesos role according with mesos role documentation
>> http://mesos.apache.org/documentation/latest/roles/
>>
>> 4) Still using setEnv method because in IgniteFramework we cannot
>> override static method and make mock static methods (except powermock, but
>> it would require 3 extra dependencies in the module).
>>
>> P.S. setEnv method designed to worked both on Linux and Windows, and it
>> does not left variable in system environment  after testing.
>>
>> Vadim Opolski
>>
>>
>> 2017-04-14 12:07 GMT+03:00 Nikolay Tikhonov (JIRA) :
>>
>>>
>>> [ https://issues.apache.org/jira/browse/IGNITE-4052?page=com.a
>>> tlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&
>>> focusedCommentId=15968814#comment-15968814 ]
>>>
>>> Nikolay Tikhonov commented on IGNITE-4052:
>>> --
>>>
>>> [~javaller]
>>> It meens that lets remove #setEnv mathod and will create mock in test
>>> which will override {{getUser}} and {{getRole}} methods. Also how do you
>>> think might be need to add validation for role? Which valid set of values
>>> for this property?
>>>
>>> > Add ability to set up users for MESOS
>>> > -
>>> >
>>> > Key: IGNITE-4052
>>> > URL: https://issues.apache.org/jira/browse/IGNITE-4052
>>> > Project: Ignite
>>> >  Issue Type: Improvement
>>> >  Components: general
>>> >Affects Versions: 1.7
>>> >Reporter: Nikolay Tikhonov
>>> >Assignee: Vadim Opolski
>>> >Priority: Trivial
>>> >
>>> > In current implementation Ignite Mesos Framework connects to MESOS
>>> cluster via current user. Need to add ability to configure this parameters
>>> via system env properties. Also need to add properties for mesos role.
>>> > See org/apache/ignite/mesos/IgniteFramework.java:537
>>>
>>>
>>>
>>> --
>>> This message was sent by Atlassian JIRA
>>> (v6.3.15#6346)
>>>
>>
>>
>

Re: Performance vs correctness: I vote fore the second

[Valentin Kulichenko]

Can you please elaborate then? What is the logic there?

-Val

On Tue, Apr 18, 2017 at 9:55 AM, Sergi Vladykin 
wrote:

> Val,
>
> That discussion has nothing to do with this PRIMARY_SYNC problem.
>
> Sergi
>
> 2017-04-18 10:51 GMT+03:00 Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
>
> > Sergi,
> >
> > I'm talking about this discussion:
> > http://apache-ignite-developers.2346864.n4.nabble.
> > com/SQL-on-PARTITIONED-vs-REPLICATED-cache-td16478.html
> >
> > -Val
> >
> > On Tue, Apr 18, 2017 at 9:46 AM, Vladimir Ozerov 
> > wrote:
> >
> > > Val,
> > >
> > > PRIMARY_SYNC doesn't work correctly with the most common case of SQL
> > query
> > > execution over REPLICATED cache. Also it has weird consequences for
> > > continuous queries when coupled with another
> performance-over-correctness
> > > property "readFromBackup=true": user may receive CQ notification with
> new
> > > value, but subsequent GET on local node may return old value.
> > >
> > > On Tue, Apr 18, 2017 at 10:42 AM, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com> wrote:
> > >
> > > > This sounds more like an issue with query execution, rather than
> wrong
> > > > PRIMARY_SYNC
> > > > behavior. We already had a discussion about this optimization in
> > > replicated
> > > > cache and decided to switch it off by default.
> > > >
> > > > -Val
> > > >
> > > > On Tue, Apr 18, 2017 at 9:35 AM, Sergi Vladykin <
> > > sergi.vlady...@gmail.com>
> > > > wrote:
> > > >
> > > > > With replicated cache we can execute a query against backup
> > partitions
> > > > that
> > > > > were not updated yet because of PRIMARY_SYNC. Thus we do not see an
> > > > update.
> > > > >
> > > > > Sergi
> > > > >
> > > > > 2017-04-18 10:30 GMT+03:00 Dmitriy Setrakyan <
> dsetrak...@apache.org
> > >:
> > > > >
> > > > > > Vladimir,
> > > > > >
> > > > > > What is wrong with a query in PRIMARY_SYNC mode? Why won't it
> work?
> > > > > >
> > > > > > D.
> > > > > >
> > > > > > On Tue, Apr 18, 2017 at 12:25 AM, Vladimir Ozerov <
> > > > voze...@gridgain.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Folks,
> > > > > > >
> > > > > > > I received a number of complaints from users that our default
> > > setting
> > > > > > favor
> > > > > > > performance at the cost of correctness and subtle behavior.
> > > > Yesterday I
> > > > > > > faced one such situation on my own.
> > > > > > >
> > > > > > > I started REPLICATED cache on several nodes, put some data,
> > > executed
> > > > > > simple
> > > > > > > SQL and got wrong result. No errors, no warnings. The problem
> was
> > > > > caused
> > > > > > by
> > > > > > > default PRIMARY_SYNC mode. WTF, our cache doesn't work out of
> the
> > > > box!
> > > > > > >
> > > > > > > Another widely known examples are data streamer behavior, "read
> > > form
> > > > > > > backups" + continuous queries.
> > > > > > >
> > > > > > > I propose to change our defaults to favor *correctness* over
> > > > > performance,
> > > > > > > and create good documentation and JavaDocs to explain users how
> > to
> > > > tune
> > > > > > our
> > > > > > > product. Proposed changes:
> > > > > > >
> > > > > > > 1) FULL_SYNC as default;
> > > > > > > 2) "readFromBackups=false" as default;
> > > > > > > 3) "IgniteDataStreamer.allowOverwrite=true" as default.
> > > > > > >
> > > > > > > Users should not think how to make Ignite work correctly. It
> > should
> > > > be
> > > > > > > correct out of the box.
> > > > > > >
> > > > > > > Vladimir.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Performance vs correctness: I vote fore the second

[Sergi Vladykin]

Val,

That discussion has nothing to do with this PRIMARY_SYNC problem.

Sergi

2017-04-18 10:51 GMT+03:00 Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Sergi,
>
> I'm talking about this discussion:
> http://apache-ignite-developers.2346864.n4.nabble.
> com/SQL-on-PARTITIONED-vs-REPLICATED-cache-td16478.html
>
> -Val
>
> On Tue, Apr 18, 2017 at 9:46 AM, Vladimir Ozerov 
> wrote:
>
> > Val,
> >
> > PRIMARY_SYNC doesn't work correctly with the most common case of SQL
> query
> > execution over REPLICATED cache. Also it has weird consequences for
> > continuous queries when coupled with another performance-over-correctness
> > property "readFromBackup=true": user may receive CQ notification with new
> > value, but subsequent GET on local node may return old value.
> >
> > On Tue, Apr 18, 2017 at 10:42 AM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > This sounds more like an issue with query execution, rather than wrong
> > > PRIMARY_SYNC
> > > behavior. We already had a discussion about this optimization in
> > replicated
> > > cache and decided to switch it off by default.
> > >
> > > -Val
> > >
> > > On Tue, Apr 18, 2017 at 9:35 AM, Sergi Vladykin <
> > sergi.vlady...@gmail.com>
> > > wrote:
> > >
> > > > With replicated cache we can execute a query against backup
> partitions
> > > that
> > > > were not updated yet because of PRIMARY_SYNC. Thus we do not see an
> > > update.
> > > >
> > > > Sergi
> > > >
> > > > 2017-04-18 10:30 GMT+03:00 Dmitriy Setrakyan  >:
> > > >
> > > > > Vladimir,
> > > > >
> > > > > What is wrong with a query in PRIMARY_SYNC mode? Why won't it work?
> > > > >
> > > > > D.
> > > > >
> > > > > On Tue, Apr 18, 2017 at 12:25 AM, Vladimir Ozerov <
> > > voze...@gridgain.com>
> > > > > wrote:
> > > > >
> > > > > > Folks,
> > > > > >
> > > > > > I received a number of complaints from users that our default
> > setting
> > > > > favor
> > > > > > performance at the cost of correctness and subtle behavior.
> > > Yesterday I
> > > > > > faced one such situation on my own.
> > > > > >
> > > > > > I started REPLICATED cache on several nodes, put some data,
> > executed
> > > > > simple
> > > > > > SQL and got wrong result. No errors, no warnings. The problem was
> > > > caused
> > > > > by
> > > > > > default PRIMARY_SYNC mode. WTF, our cache doesn't work out of the
> > > box!
> > > > > >
> > > > > > Another widely known examples are data streamer behavior, "read
> > form
> > > > > > backups" + continuous queries.
> > > > > >
> > > > > > I propose to change our defaults to favor *correctness* over
> > > > performance,
> > > > > > and create good documentation and JavaDocs to explain users how
> to
> > > tune
> > > > > our
> > > > > > product. Proposed changes:
> > > > > >
> > > > > > 1) FULL_SYNC as default;
> > > > > > 2) "readFromBackups=false" as default;
> > > > > > 3) "IgniteDataStreamer.allowOverwrite=true" as default.
> > > > > >
> > > > > > Users should not think how to make Ignite work correctly. It
> should
> > > be
> > > > > > correct out of the box.
> > > > > >
> > > > > > Vladimir.
> > > > > >
> > > > >
> > > >
> > >
> >
>

[GitHub] ignite pull request #1810: MemoryConfiguration javadoc updated

[sergey-chugunov-1985]

Github user sergey-chugunov-1985 closed the pull request at:

https://github.com/apache/ignite/pull/1810


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Performance vs correctness: I vote fore the second

[Valentin Kulichenko]

Sergi,

I'm talking about this discussion:
http://apache-ignite-developers.2346864.n4.nabble.com/SQL-on-PARTITIONED-vs-REPLICATED-cache-td16478.html

-Val

On Tue, Apr 18, 2017 at 9:46 AM, Vladimir Ozerov 
wrote:

> Val,
>
> PRIMARY_SYNC doesn't work correctly with the most common case of SQL query
> execution over REPLICATED cache. Also it has weird consequences for
> continuous queries when coupled with another performance-over-correctness
> property "readFromBackup=true": user may receive CQ notification with new
> value, but subsequent GET on local node may return old value.
>
> On Tue, Apr 18, 2017 at 10:42 AM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > This sounds more like an issue with query execution, rather than wrong
> > PRIMARY_SYNC
> > behavior. We already had a discussion about this optimization in
> replicated
> > cache and decided to switch it off by default.
> >
> > -Val
> >
> > On Tue, Apr 18, 2017 at 9:35 AM, Sergi Vladykin <
> sergi.vlady...@gmail.com>
> > wrote:
> >
> > > With replicated cache we can execute a query against backup partitions
> > that
> > > were not updated yet because of PRIMARY_SYNC. Thus we do not see an
> > update.
> > >
> > > Sergi
> > >
> > > 2017-04-18 10:30 GMT+03:00 Dmitriy Setrakyan :
> > >
> > > > Vladimir,
> > > >
> > > > What is wrong with a query in PRIMARY_SYNC mode? Why won't it work?
> > > >
> > > > D.
> > > >
> > > > On Tue, Apr 18, 2017 at 12:25 AM, Vladimir Ozerov <
> > voze...@gridgain.com>
> > > > wrote:
> > > >
> > > > > Folks,
> > > > >
> > > > > I received a number of complaints from users that our default
> setting
> > > > favor
> > > > > performance at the cost of correctness and subtle behavior.
> > Yesterday I
> > > > > faced one such situation on my own.
> > > > >
> > > > > I started REPLICATED cache on several nodes, put some data,
> executed
> > > > simple
> > > > > SQL and got wrong result. No errors, no warnings. The problem was
> > > caused
> > > > by
> > > > > default PRIMARY_SYNC mode. WTF, our cache doesn't work out of the
> > box!
> > > > >
> > > > > Another widely known examples are data streamer behavior, "read
> form
> > > > > backups" + continuous queries.
> > > > >
> > > > > I propose to change our defaults to favor *correctness* over
> > > performance,
> > > > > and create good documentation and JavaDocs to explain users how to
> > tune
> > > > our
> > > > > product. Proposed changes:
> > > > >
> > > > > 1) FULL_SYNC as default;
> > > > > 2) "readFromBackups=false" as default;
> > > > > 3) "IgniteDataStreamer.allowOverwrite=true" as default.
> > > > >
> > > > > Users should not think how to make Ignite work correctly. It should
> > be
> > > > > correct out of the box.
> > > > >
> > > > > Vladimir.
> > > > >
> > > >
> > >
> >
>

Re: cache query operations

[ALEKSEY KUZNETSOV]

thanx! )

пн, 17 апр. 2017 г. в 21:48, Denis Magda :

> Did you have a chance to take a look this documentation that explains how
> DML statements are turned into cache operations?
> https://apacheignite.readme.io/docs/dml#dml-operations <
> https://apacheignite.readme.io/docs/dml#dml-operations>
>
> —
> Denis
>
> > On Apr 17, 2017, at 7:20 AM, ALEKSEY KUZNETSOV 
> wrote:
> >
> > Hi, Igniters! When one fires query like this : ignite().cache(null
> > ).query(...)
> > The query would be executed against the schema. I wonder, how the schema
> is
> > synchronized with cache, with cache entries?
> >
> > What i mean is , cache.query(...update operation...) must eventually
> update
> > the cache entry in cache. But how does it happen?(cache API must be
> called,
> > but i cannot realize how)
> > --
> >
> > *Best Regards,*
> >
> > *Kuznetsov Aleksey*
>
> --

*Best Regards,*

*Kuznetsov Aleksey*

Re: Performance vs correctness: I vote fore the second

[Sergi Vladykin]

Val,

I'm not sure I understand what optimization you are talking about and what
exactly did you decide to switch off, can you explain please?

Sergi

2017-04-18 10:42 GMT+03:00 Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> This sounds more like an issue with query execution, rather than wrong
> PRIMARY_SYNC
> behavior. We already had a discussion about this optimization in replicated
> cache and decided to switch it off by default.
>
> -Val
>
> On Tue, Apr 18, 2017 at 9:35 AM, Sergi Vladykin 
> wrote:
>
> > With replicated cache we can execute a query against backup partitions
> that
> > were not updated yet because of PRIMARY_SYNC. Thus we do not see an
> update.
> >
> > Sergi
> >
> > 2017-04-18 10:30 GMT+03:00 Dmitriy Setrakyan :
> >
> > > Vladimir,
> > >
> > > What is wrong with a query in PRIMARY_SYNC mode? Why won't it work?
> > >
> > > D.
> > >
> > > On Tue, Apr 18, 2017 at 12:25 AM, Vladimir Ozerov <
> voze...@gridgain.com>
> > > wrote:
> > >
> > > > Folks,
> > > >
> > > > I received a number of complaints from users that our default setting
> > > favor
> > > > performance at the cost of correctness and subtle behavior.
> Yesterday I
> > > > faced one such situation on my own.
> > > >
> > > > I started REPLICATED cache on several nodes, put some data, executed
> > > simple
> > > > SQL and got wrong result. No errors, no warnings. The problem was
> > caused
> > > by
> > > > default PRIMARY_SYNC mode. WTF, our cache doesn't work out of the
> box!
> > > >
> > > > Another widely known examples are data streamer behavior, "read form
> > > > backups" + continuous queries.
> > > >
> > > > I propose to change our defaults to favor *correctness* over
> > performance,
> > > > and create good documentation and JavaDocs to explain users how to
> tune
> > > our
> > > > product. Proposed changes:
> > > >
> > > > 1) FULL_SYNC as default;
> > > > 2) "readFromBackups=false" as default;
> > > > 3) "IgniteDataStreamer.allowOverwrite=true" as default.
> > > >
> > > > Users should not think how to make Ignite work correctly. It should
> be
> > > > correct out of the box.
> > > >
> > > > Vladimir.
> > > >
> > >
> >
>

Re: Performance vs correctness: I vote fore the second

[Vladimir Ozerov]

Val,

PRIMARY_SYNC doesn't work correctly with the most common case of SQL query
execution over REPLICATED cache. Also it has weird consequences for
continuous queries when coupled with another performance-over-correctness
property "readFromBackup=true": user may receive CQ notification with new
value, but subsequent GET on local node may return old value.

On Tue, Apr 18, 2017 at 10:42 AM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> This sounds more like an issue with query execution, rather than wrong
> PRIMARY_SYNC
> behavior. We already had a discussion about this optimization in replicated
> cache and decided to switch it off by default.
>
> -Val
>
> On Tue, Apr 18, 2017 at 9:35 AM, Sergi Vladykin 
> wrote:
>
> > With replicated cache we can execute a query against backup partitions
> that
> > were not updated yet because of PRIMARY_SYNC. Thus we do not see an
> update.
> >
> > Sergi
> >
> > 2017-04-18 10:30 GMT+03:00 Dmitriy Setrakyan :
> >
> > > Vladimir,
> > >
> > > What is wrong with a query in PRIMARY_SYNC mode? Why won't it work?
> > >
> > > D.
> > >
> > > On Tue, Apr 18, 2017 at 12:25 AM, Vladimir Ozerov <
> voze...@gridgain.com>
> > > wrote:
> > >
> > > > Folks,
> > > >
> > > > I received a number of complaints from users that our default setting
> > > favor
> > > > performance at the cost of correctness and subtle behavior.
> Yesterday I
> > > > faced one such situation on my own.
> > > >
> > > > I started REPLICATED cache on several nodes, put some data, executed
> > > simple
> > > > SQL and got wrong result. No errors, no warnings. The problem was
> > caused
> > > by
> > > > default PRIMARY_SYNC mode. WTF, our cache doesn't work out of the
> box!
> > > >
> > > > Another widely known examples are data streamer behavior, "read form
> > > > backups" + continuous queries.
> > > >
> > > > I propose to change our defaults to favor *correctness* over
> > performance,
> > > > and create good documentation and JavaDocs to explain users how to
> tune
> > > our
> > > > product. Proposed changes:
> > > >
> > > > 1) FULL_SYNC as default;
> > > > 2) "readFromBackups=false" as default;
> > > > 3) "IgniteDataStreamer.allowOverwrite=true" as default.
> > > >
> > > > Users should not think how to make Ignite work correctly. It should
> be
> > > > correct out of the box.
> > > >
> > > > Vladimir.
> > > >
> > >
> >
>

Re: Performance vs correctness: I vote fore the second

[Valentin Kulichenko]

This sounds more like an issue with query execution, rather than wrong
PRIMARY_SYNC
behavior. We already had a discussion about this optimization in replicated
cache and decided to switch it off by default.

-Val

On Tue, Apr 18, 2017 at 9:35 AM, Sergi Vladykin 
wrote:

> With replicated cache we can execute a query against backup partitions that
> were not updated yet because of PRIMARY_SYNC. Thus we do not see an update.
>
> Sergi
>
> 2017-04-18 10:30 GMT+03:00 Dmitriy Setrakyan :
>
> > Vladimir,
> >
> > What is wrong with a query in PRIMARY_SYNC mode? Why won't it work?
> >
> > D.
> >
> > On Tue, Apr 18, 2017 at 12:25 AM, Vladimir Ozerov 
> > wrote:
> >
> > > Folks,
> > >
> > > I received a number of complaints from users that our default setting
> > favor
> > > performance at the cost of correctness and subtle behavior. Yesterday I
> > > faced one such situation on my own.
> > >
> > > I started REPLICATED cache on several nodes, put some data, executed
> > simple
> > > SQL and got wrong result. No errors, no warnings. The problem was
> caused
> > by
> > > default PRIMARY_SYNC mode. WTF, our cache doesn't work out of the box!
> > >
> > > Another widely known examples are data streamer behavior, "read form
> > > backups" + continuous queries.
> > >
> > > I propose to change our defaults to favor *correctness* over
> performance,
> > > and create good documentation and JavaDocs to explain users how to tune
> > our
> > > product. Proposed changes:
> > >
> > > 1) FULL_SYNC as default;
> > > 2) "readFromBackups=false" as default;
> > > 3) "IgniteDataStreamer.allowOverwrite=true" as default.
> > >
> > > Users should not think how to make Ignite work correctly. It should be
> > > correct out of the box.
> > >
> > > Vladimir.
> > >
> >
>

[GitHub] ignite pull request #1732: IGNITE-4856 .NET: StopAll on AppDomain unload

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/1732


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Performance vs correctness: I vote fore the second

[Sergi Vladykin]

With replicated cache we can execute a query against backup partitions that
were not updated yet because of PRIMARY_SYNC. Thus we do not see an update.

Sergi

2017-04-18 10:30 GMT+03:00 Dmitriy Setrakyan :

> Vladimir,
>
> What is wrong with a query in PRIMARY_SYNC mode? Why won't it work?
>
> D.
>
> On Tue, Apr 18, 2017 at 12:25 AM, Vladimir Ozerov 
> wrote:
>
> > Folks,
> >
> > I received a number of complaints from users that our default setting
> favor
> > performance at the cost of correctness and subtle behavior. Yesterday I
> > faced one such situation on my own.
> >
> > I started REPLICATED cache on several nodes, put some data, executed
> simple
> > SQL and got wrong result. No errors, no warnings. The problem was caused
> by
> > default PRIMARY_SYNC mode. WTF, our cache doesn't work out of the box!
> >
> > Another widely known examples are data streamer behavior, "read form
> > backups" + continuous queries.
> >
> > I propose to change our defaults to favor *correctness* over performance,
> > and create good documentation and JavaDocs to explain users how to tune
> our
> > product. Proposed changes:
> >
> > 1) FULL_SYNC as default;
> > 2) "readFromBackups=false" as default;
> > 3) "IgniteDataStreamer.allowOverwrite=true" as default.
> >
> > Users should not think how to make Ignite work correctly. It should be
> > correct out of the box.
> >
> > Vladimir.
> >
>

Re: Performance vs correctness: I vote fore the second

[Dmitriy Setrakyan]

Vladimir,

What is wrong with a query in PRIMARY_SYNC mode? Why won't it work?

D.

On Tue, Apr 18, 2017 at 12:25 AM, Vladimir Ozerov 
wrote:

> Folks,
>
> I received a number of complaints from users that our default setting favor
> performance at the cost of correctness and subtle behavior. Yesterday I
> faced one such situation on my own.
>
> I started REPLICATED cache on several nodes, put some data, executed simple
> SQL and got wrong result. No errors, no warnings. The problem was caused by
> default PRIMARY_SYNC mode. WTF, our cache doesn't work out of the box!
>
> Another widely known examples are data streamer behavior, "read form
> backups" + continuous queries.
>
> I propose to change our defaults to favor *correctness* over performance,
> and create good documentation and JavaDocs to explain users how to tune our
> product. Proposed changes:
>
> 1) FULL_SYNC as default;
> 2) "readFromBackups=false" as default;
> 3) "IgniteDataStreamer.allowOverwrite=true" as default.
>
> Users should not think how to make Ignite work correctly. It should be
> correct out of the box.
>
> Vladimir.
>

Performance vs correctness: I vote fore the second

[Vladimir Ozerov]

Folks,

I received a number of complaints from users that our default setting favor
performance at the cost of correctness and subtle behavior. Yesterday I
faced one such situation on my own.

I started REPLICATED cache on several nodes, put some data, executed simple
SQL and got wrong result. No errors, no warnings. The problem was caused by
default PRIMARY_SYNC mode. WTF, our cache doesn't work out of the box!

Another widely known examples are data streamer behavior, "read form
backups" + continuous queries.

I propose to change our defaults to favor *correctness* over performance,
and create good documentation and JavaDocs to explain users how to tune our
product. Proposed changes:

1) FULL_SYNC as default;
2) "readFromBackups=false" as default;
3) "IgniteDataStreamer.allowOverwrite=true" as default.

Users should not think how to make Ignite work correctly. It should be
correct out of the box.

Vladimir.

Re: question: How data are stored in IgniteCache?

[Vyacheslav Daradur]

Denis, thanks a lot for your explanations.

2017-04-17 21:20 GMT+03:00 Denis Magda :

> It depends on a storage. If it’s a relational database and
> isStoreKeepBinary is false then an object will be deserialized upon the
> store invocation and the storage will insert or update a record using
> “INSERT * ..” or “UPDATE * …” statements. Take a look at the code.
>
> —
> Denis
>
> > On Apr 17, 2017, at 5:04 AM, Vyacheslav Daradur 
> wrote:
> >
> >>> When you transfer an object over the wire or put it into a persistent
> > store (withKeepBinary property enabled) then only the byte array is used.
> > If we put objects into persistent store and withKeepBinary property is
> > DISABLED, in wich form it will be stored?
> >
> > 2017-04-16 18:29 GMT+03:00 Denis Magda :
> >
> >> 1. When you transfer an object over the wire or put it into a persistent
> >> store (withKeepBinary property enabled) then only the byte array is
> used.
> >> 2. No
> >>
> >> —
> >> Denis
> >>
> >>> On Apr 15, 2017, at 1:51 PM, Vyacheslav Daradur 
> >> wrote:
> >>>
> > If we use a cache which is configured to use binary mashaller:
> > 1. In which cases a placed in the cache object (serialized object)
> >> won't
> >>> be wrapped?
> > 2. In which cases a placed in the cache object won't be serialized
> (to
> >>> byte array) and will be stored as is?
> >>>
> >>> I meant: Are there any cases of described (1,2) situations.
> >>> If yes, which cases?
> >>>
> >>> 2017-04-15 23:44 GMT+03:00 Vyacheslav Daradur :
> >>>
>  If we use a cache which is configured to use binary mashaller:
>  1. In which cases a placed in the cache object (serialized object)
> won't
>  be wrapped?
>  2. In which cases a placed in the cache object won't be serialized (to
>  byte array) and will be stored as is?
> 
>  2017-04-14 20:03 GMT+03:00 Denis Magda :
> 
> > If a serialized object is stored in an on-heap cache then it will be
> > wrapped by BinaryObjectImp but if it’s an off-heap cache then
> > BinaryObjectOffHeapImpl is used instead.
> >
> > —
> > Denis
> >
> >> On Apr 14, 2017, at 12:32 AM, Vyacheslav Daradur <
> daradu...@gmail.com
> >>>
> > wrote:
> >>
>  when it’s stored in memory in a specific cache partition
> >> Does that mean that any serialized object is always stored IN MEMORY
> >> as
> > is
> >> wrapped by BinaryObjectImpl?
> >>
> >>
> >>
> >> 2017-04-14 3:42 GMT+03:00 Denis Magda :
> >>
> >>> A serialized object is always wrapped by BinaryObjectImpl when it’s
> > stored
> >>> in memory in a specific cache partition or you access it from your
> >>> application in a form of BinaryObject. However, when you transfer
> the
> >>> object over the wire or put it into a persistent store
> >> (withKeepBinary
> >>> property enabled) then only the byte array is used.
> >>>
> >>> —
> >>> Denis
> >>>
>  On Apr 13, 2017, at 12:21 AM, Vyacheslav Daradur <
> >> daradu...@gmail.com
> >>
> >>> wrote:
> 
>  Denis, thank you for answers.
> 
>  I meant another.
> 
>  For example:
>  Cache queries use a BinaryObjectImpl and a withKeepBinary-mode use
> > it, so
>  looks like all actions on serialized object are make via a
> >>> BinaryObjectImpl.
> 
>  Does a serialized object always is stored as BinaryObjectImpl or
> it
> > will
> >>> be
>  wrapped on demand?
> 
>  2017-04-12 22:34 GMT+03:00 Denis Magda :
> 
> > A Java wrapper around an actual binary byte array with some
> > additional
> > fields and methods to work with the serialized data.
> >
> > —
> > Denis
> >
> >> On Apr 12, 2017, at 8:33 AM, Vyacheslav Daradur <
> > daradu...@gmail.com>
> > wrote:
> >>
> >> In what cases BinaryObjecImpl is used?
> >>
> >> 2017-04-12 18:08 GMT+03:00 Denis Magda :
> >>
> >>> Hi,
> >>>
> >>> A cache entry is always stored in a binary format (byte array)
> >> in a
> > cache.
> >>> Even when you transfer an entry from one node to another, as a
> > result
> >>> of
> >>> cache.put(…), operation the entry will be serialized into the
> > binary
> > format
> >>> and transferred over the wire.
> >>>
> >>> —
> >>> Denis
> >>>
>  On Apr 12, 2017, at 1:11 AM, Vyacheslav Daradur <
> > daradu...@gmail.com
> 
> >>> wrote:
> 
>  Hello Igniters!
> 
>  I have one conceptual question:
> 
>  When we put an object in