[jira] [Created] (IGNITE-6953) OOM on the client node freezes the whole cluster

2017-11-17 Thread Denis Magda (JIRA)
Denis Magda created IGNITE-6953:
---

 Summary: OOM on the client node freezes the whole cluster
 Key: IGNITE-6953
 URL: https://issues.apache.org/jira/browse/IGNITE-6953
 Project: Ignite
  Issue Type: Bug
Reporter: Denis Magda
 Fix For: 2.4


It's reported that if an OOM happens on the client side the whole cluster 
becomes unresponsive:
http://apache-ignite-users.70518.x6.nabble.com/Out-of-memory-in-client-node-freezes-complete-cluster-td18044.html

Let's reproduce and prevent this by bringing the client node down 
automatically.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (IGNITE-6952) Deserialization failures on AIX

2017-11-17 Thread Denis Magda (JIRA)
Denis Magda created IGNITE-6952:
---

 Summary: Deserialization failures on AIX 
 Key: IGNITE-6952
 URL: https://issues.apache.org/jira/browse/IGNITE-6952
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.3
Reporter: Denis Magda
Assignee: Vladimir Ozerov
 Fix For: 2.4


Ignite fails on AIX with the following stack trace:

{code}
GridConnectionBytesVerifyFilter], accepted=true]]]
java.lang.IllegalArgumentException: null
at java.nio.Buffer.position(Buffer.java:255) ~[?:1.8.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readArrayLE(DirectByteBufferStreamImplV2.java:1587)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readArrayLE(DirectByteBufferStreamImplV2.java:1542)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readLongArray(DirectByteBufferStreamImplV2.java:1013)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.DirectMessageReader.readLongArray(DirectMessageReader.java:212)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.GridLongList.readFrom(GridLongList.java:558)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readMessage(DirectByteBufferStreamImplV2.java:1165)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.read(DirectByteBufferStreamImplV2.java:1785)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readCollection(DirectByteBufferStreamImplV2.java:1244)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.DirectMessageReader.readCollection(DirectMessageReader.java:333)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.CacheGroupAffinityMessage.readFrom(CacheGroupAffinityMessage.java:292)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readMessage(DirectByteBufferStreamImplV2.java:1165)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.read(DirectByteBufferStreamImplV2.java:1785)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readMap(DirectByteBufferStreamImplV2.java:1294)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.DirectMessageReader.readMap(DirectMessageReader.java:345)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsFullMessage.readFrom(GridDhtPartitionsFullMessage.java:645)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.stream.v2.DirectByteBufferStreamImplV2.readMessage(DirectByteBufferStreamImplV2.java:1165)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.direct.DirectMessageReader.readMessage(DirectMessageReader.java:311)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.managers.communication.GridIoMessage.readFrom(GridIoMessage.java:262)
~[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridDirectParser.decode(GridDirectParser.java:90)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridNioCodecFilter.onMessageReceived(GridNioCodecFilter.java:114)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridConnectionBytesVerifyFilter.onMessageReceived(GridConnectionBytesVerifyFilter.java:133)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridNioServer$HeadFilter.onMessageReceived(GridNioServer.java:3388)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridNioFilterChain.onMessageReceived(GridNioFilterChain.java:175)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1261)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2272)
[ignite-core-2.3.0.jar:2.3.0]
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2048)
[ignite-core-2.3.0.jar:2.3.0]
at

Data eviction/expiration from Ignite persistence

2017-11-17 Thread Denis Magda
Igniters,

I’ve been talking to many Ignite users here and there who are already on Ignite 
persistence or consider to turn it on. The majority of them are more than 
satisfied with its current state and provided capabilities. That’s is really 
good news for us.

However, I tend to come across the people who ask about eviction/expiration 
policies for the persistence itself. Had around 6 conversation about the topic 
this month only.

Usually the requirement is connected with a streaming use case. When an 
application streams a lot of data (IoT, metrics, etc.) to the cluster but the 
data becomes stale in some period of time (day, couple of days, etc.). The user 
doesn’t want to waste the disk space and needs to simple purge the data from 
there.

My suggestion here is to create a timer task that will remove the stale data 
from the cluster. However, since the demand is growing probably it’s a good 
time to discuss a feasibility of this feature.

Alex G, as the main architect of the persistence, could you share your thoughts 
on this? What will it cost to us to support eviction/expiration for the 
persistence?

—
Denis 

Re: IGNITE-6745. Status

2017-11-17 Thread Denis Magda
Igniters,

Who is going to take a lead of Java 9 support and can do thorough review of all 
the related changes? Here is a set of the tickets and Cergey solved one of them:
https://issues.apache.org/jira/browse/IGNITE-6728

—
Denis

> On Nov 16, 2017, at 3:12 PM, Cergey  wrote:
> 
> Hi, igniters
> 
> 
> 
> Why no one commented on the patch and pull request
> (https://github.com/apache/ignite/pull/2970) ?  What should I do ?
> 
> 
> 
> Regards,
> 
> Cergey Chaulin
> 
> 
> 



Re: Deserialization error with Ignite 2.3.0

2017-11-17 Thread Valentin Kulichenko
Andrey,

This ticket seems to be about the same issue:
https://issues.apache.org/jira/browse/IGNITE-6944

Can you please take a look and provide your thoughts?

-Val

On Tue, Nov 7, 2017 at 4:02 PM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> *cross-posting to dev*
>
> Hi Chris,
>
> This is a regression due to this fix: https://issues.apache.
> org/jira/browse/IGNITE-6485. For some reason, since 2.3 it is looking for
> readResolve/writeReplace methods only in exact class that is being
> serialized, but not in the whole hierarchy.
>
> *Andrey G*, since you did this change, can you please clarify it? Why
> isn't it looking into parent classes anymore? Generally,
> readResolve/writeReplace can reside in parent classes, so previous behavior
> seems to be the correct one. Can you create a ticket to fix it back?
>
> -Val
>
> On Mon, Nov 6, 2017 at 7:59 PM, Chris Berry  wrote:
>
>> Hi Val
>>
>> I have found that if I switch from
>>
>> igniteConfig.setMarshaller(new BinaryMarshaller());
>>
>> To:
>>
>> igniteConfig.setMarshaller(new
>> OptimizedMarshaller().setRequireSerializable(false));
>>
>> Then everything works again.
>>
>> Although I am worried that this will have a terrible effect on performance
>> (I switched to the BinaryMarshaller to get a good performance bump at the
>> release of 2.0.0)
>>
>> I believe that all of this is because the concrete forms of the
>> ImmutableList (e.g. RegularImmutableList)
>> all use:
>>
>>   final transient Object[] array;
>>
>> to keep their data.
>> And that the handling of transients changed for the BinaryMarshaller in
>> 2.3.0
>>
>> Thanks for your help!!
>> Cheers,
>> -- Chris
>>
>>
>>
>>
>> Thanks,
>> -- Chris
>>
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
>


Re: Thin client protocol message format

2017-11-17 Thread Denis Magda
The concern has been solved after I had a swift private conversation with 
Vladimir Ozerov.

We just need to document how to write and read all the types supported by 
binary format. That’s more than enough for the beginning.

—
Denis

> On Nov 17, 2017, at 10:40 AM, Denis Magda  wrote:
> 
> Igniters,
> 
> Let me reiterate on the protocol details until it becomes publicly available.
> 
> If I’m not mistaken one of the reasons we got down to this task was to give 
> an efficient low-level protocol that will simplify and boost Ignite 
> clients/connectors development for various programming languages *avoiding* 
> any Ignite related dependancies such as binary marshaller. 
> 
> However, by some reason we agreed to stick to the binary marshaller:
>> * Ignite binary format is used for value serialization (via
>> GridBinaryMarshaller/BinaryWriter/BinaryReader). Ignite binary protocol has
>> to be implemented by clients anyway to work with cache values, so it makes
>> sense to use for all data exchange.
> 
> This suggests me that if a developer wants to exchange messages with the 
> cluster from Python, Node.js, Objective-C, embedded hardware such as smart 
> meters being power by real-time operating systems he/she has to support the 
> binary marshaller first. This sounds like a bar for the protocol adoption. 
> 
> Question:
> 
> - What exactly has to be supported on the binary protocol side?
> - Any concerns on why we can’t switch to widely used format such as JSON? 
> 
> —
> Denis
> 
> 
>> On Aug 1, 2017, at 9:10 AM, Pavel Tupitsyn  wrote:
>> 
>> Igniters,
>> 
>> Below is a proposed design for thin client protocol [1] [2] socket data
>> exchange format.
>> 
>> * Values are little-endian
>> * Every request and response message starts with 4-byte length (including
>> handshake)
>> * Ignite binary format is used for value serialization (via
>> GridBinaryMarshaller/BinaryWriter/BinaryReader). Ignite binary protocol has
>> to be implemented by clients anyway to work with cache values, so it makes
>> sense to use for all data exchange.
>> 
>> 
>> 1) Socket connection is established on a port according
>> to ConnectorConfiguration.port
>> 
>> 2) Handshake is performed.
>> Request:
>>  int32 length = 8 // message length
>>  byte opCode = 1   // handshake command
>>  int16 verMajor
>>  int16 verMinor
>>  int16 verMaintenance
>>  byte clientCode = 2// client type code (odbc, jdbc, platform)
>> 
>> Response:
>>  uint32 length = 1
>>  byte success
>> 
>> 3) Execute command. Request starts with a command code, then goes
>> command-specific data.
>> For example, IgniteCache.put(1, "foo") will look like this:
>> Request:
>>  int16 opCode// OP_CACHE_GET
>>  int32 cacheId// GridCacheUtils.cacheId
>>  byte flags  // skipStore, noRetry, etc
>>  binobject key
>> 
>> Response:
>>  byte success
>>  binobject value
>> 
>> Where binobject corresponds to Ignite BinaryMarshaller format, which is
>> produced by BinaryWriter.writeObject method. Integer will be represented as
>> byte typeCode = 3  // GridBinaryMarshaller.INT
>> int32 value = 1
>> 
>> 4) Goto (3)
>> 
>> 
>> Comments are welcome.
>> 
>> Pavel
>> 
>> [1]
>> http://apache-ignite-developers.2346864.n4.nabble.com/Support-for-Ignite-clients-in-any-language-thin-client-protocol-td20297.html
>> [2] https://issues.apache.org/jira/browse/IGNITE-5896
> 



Re: Thin client protocol message format

2017-11-17 Thread Denis Magda
Igniters,

Let me reiterate on the protocol details until it becomes publicly available.

If I’m not mistaken one of the reasons we got down to this task was to give an 
efficient low-level protocol that will simplify and boost Ignite 
clients/connectors development for various programming languages *avoiding* any 
Ignite related dependancies such as binary marshaller. 

However, by some reason we agreed to stick to the binary marshaller:
> * Ignite binary format is used for value serialization (via
> GridBinaryMarshaller/BinaryWriter/BinaryReader). Ignite binary protocol has
> to be implemented by clients anyway to work with cache values, so it makes
> sense to use for all data exchange.

This suggests me that if a developer wants to exchange messages with the 
cluster from Python, Node.js, Objective-C, embedded hardware such as smart 
meters being power by real-time operating systems he/she has to support the 
binary marshaller first. This sounds like a bar for the protocol adoption. 

Question:

- What exactly has to be supported on the binary protocol side?
- Any concerns on why we can’t switch to widely used format such as JSON? 

—
Denis


> On Aug 1, 2017, at 9:10 AM, Pavel Tupitsyn  wrote:
> 
> Igniters,
> 
> Below is a proposed design for thin client protocol [1] [2] socket data
> exchange format.
> 
> * Values are little-endian
> * Every request and response message starts with 4-byte length (including
> handshake)
> * Ignite binary format is used for value serialization (via
> GridBinaryMarshaller/BinaryWriter/BinaryReader). Ignite binary protocol has
> to be implemented by clients anyway to work with cache values, so it makes
> sense to use for all data exchange.
> 
> 
> 1) Socket connection is established on a port according
> to ConnectorConfiguration.port
> 
> 2) Handshake is performed.
> Request:
>   int32 length = 8 // message length
>   byte opCode = 1   // handshake command
>   int16 verMajor
>   int16 verMinor
>   int16 verMaintenance
>   byte clientCode = 2// client type code (odbc, jdbc, platform)
> 
> Response:
>   uint32 length = 1
>   byte success
> 
> 3) Execute command. Request starts with a command code, then goes
> command-specific data.
> For example, IgniteCache.put(1, "foo") will look like this:
> Request:
>   int16 opCode// OP_CACHE_GET
>   int32 cacheId// GridCacheUtils.cacheId
>   byte flags  // skipStore, noRetry, etc
>   binobject key
> 
> Response:
>   byte success
>   binobject value
> 
> Where binobject corresponds to Ignite BinaryMarshaller format, which is
> produced by BinaryWriter.writeObject method. Integer will be represented as
>  byte typeCode = 3  // GridBinaryMarshaller.INT
>  int32 value = 1
> 
> 4) Goto (3)
> 
> 
> Comments are welcome.
> 
> Pavel
> 
> [1]
> http://apache-ignite-developers.2346864.n4.nabble.com/Support-for-Ignite-clients-in-any-language-thin-client-protocol-td20297.html
> [2] https://issues.apache.org/jira/browse/IGNITE-5896



[GitHub] ignite pull request #3066: IGNITE-5938 Try to enable v2 serializer

2017-11-17 Thread glukos
GitHub user glukos opened a pull request:

https://github.com/apache/ignite/pull/3066

IGNITE-5938 Try to enable v2 serializer



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite master-v2ser

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3066.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3066


commit 6e3f4043c6ab58f6ad5109bbdaf4832685bc7df8
Author: Ivan Rakov 
Date:   2017-11-17T18:34:54Z

IGNITE-5938 Try to enable v2 serializer




---


[GitHub] ignite pull request #3065: Ignite 5938 noarch

2017-11-17 Thread glukos
GitHub user glukos opened a pull request:

https://github.com/apache/ignite/pull/3065

Ignite 5938 noarch



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5938-noarch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3065.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3065


commit 2022020738912ce5b79500228722fe5a38c08a4d
Author: Ivan Rakov 
Date:   2017-11-14T21:36:57Z

IGNITE-5938 Working prototype.

commit e1ba31aa81ac08bc5da106d6c71ab05da6796c90
Author: Ivan Rakov 
Date:   2017-11-15T10:24:00Z

IGNITE-5938 Get rid of deserialization during compaction.

commit 1de758dafd59a40e9af5f0bce28ca4aac97c7648
Author: Ivan Rakov 
Date:   2017-11-15T16:01:05Z

IGNITE-5938 Several bugs fixed

commit fe6efc382a10e7c214966a3b8a8abe202a6a0741
Author: Ivan Rakov 
Date:   2017-11-16T08:44:23Z

Merge branch 'master' into ignite-5938-master

commit f890e7ac628f714f62ce6461b151ec92f1a2
Author: Ivan Rakov 
Date:   2017-11-16T10:22:30Z

IGNITE-5938 Several bugs fixed

commit c5f761307af3fc02cd2af83fb2c325a653abead5
Author: Ivan Rakov 
Date:   2017-11-16T11:32:23Z

IGNITE-5938 Minor refactor

commit 8f10daa3b76bc7a5b266a0f6dbc1e62153308a84
Author: Ivan Rakov 
Date:   2017-11-16T11:59:59Z

IGNITE-5938 serialVUID added

commit 1d29758b8f380c8400d625976f2d2d22a319179c
Author: Ivan Rakov 
Date:   2017-11-16T12:59:40Z

IGNITE-5938 Javadocs

commit 2e26a8ee69538c2ca29fe012a0d1d982ab9c7f0d
Author: Ivan Rakov 
Date:   2017-11-16T13:19:35Z

IGNITE-5938 Exception handling

commit e29e370bc344e5c0ab4f962f62c0351937c56458
Author: Ivan Rakov 
Date:   2017-11-16T13:21:54Z

IGNITE-5938 Javadoc

commit dc5b554dae568c543b70ac386a3fcffd00dcfda8
Author: Ivan Rakov 
Date:   2017-11-16T15:09:46Z

IGNITE-5938 bugs fixed

commit 2eaaa39c27b0fbcee5e25dec4e70a3088fb975c7
Author: Ivan Rakov 
Date:   2017-11-17T09:46:33Z

IGNITE-5938 Bug fixed

commit 535bbf0ab7890d4188534acc635de501a96fb0c6
Author: Ivan Rakov 
Date:   2017-11-17T15:24:48Z

IGNITE-5938 Bug fixed

commit 203b685ac6fcea0bdd779063752ba61f6e67ea81
Author: Ivan Rakov 
Date:   2017-11-17T17:18:54Z

IGNITE-5938 debug info

commit 77a2c756a86887457386d188343a2038554e482c
Author: Ivan Rakov 
Date:   2017-11-17T18:16:39Z

IGNITE-5938 debug info

commit 89ad24fecb9e46eed2c2af1aee235a9e9c3c9408
Author: Ivan Rakov 
Date:   2017-11-17T18:20:11Z

IGNITE-5938 Disabling archivation.




---


Re: Losing data during restarting cluster with persistence enabled

2017-11-17 Thread Valentin Kulichenko
Vyacheslav,

CacheStore design assumes that the underlying storage is shared by all the
nodes in topology. Even if you delay rebalancing on node stop (which is
possible via CacheConfiguration#rebalanceDelay), I doubt it will solve all
your consistency issues.

Why don't you use Ignite persistence [1]?

[1] https://apacheignite.readme.io/docs/distributed-persistent-store

-Val

On Fri, Nov 17, 2017 at 4:24 AM, Vyacheslav Daradur 
wrote:

> Hi Andrey! Thank you for answering.
>
> >> Key to partition mapping shouldn't depends on topology, and shouldn't
> changed unstable topology.
> Key to partition mapping doesn't depend on topology in my test
> affinity function. It only depends on partitions number.
> But partition to node mapping depends on topology and at cluster stop,
> when one node left topology, some partitions may be moved to other
> nodes.
>
> >> Does all nodes share same RockDB database or each node has its own copy?
> Each Ignite node has own RocksDB instance.
>
> >> Would you please share configuration?
> It's pretty simple:
> IgniteConfiguration cfg = new IgniteConfiguration();
> cfg.setIgniteInstanceName(instanceName);
>
> CacheConfiguration cacheCfg = new
> CacheConfiguration<>();
> cacheCfg.setName(TEST_CACHE_NAME);
> cacheCfg.setCacheMode(CacheMode.PARTITIONED);
> cacheCfg.setWriteSynchronizationMode(
> CacheWriteSynchronizationMode.PRIMARY_SYNC);
> cacheCfg.setBackups(1);
> cacheCfg.setAffinity(new
> TestAffinityFunction(partitionsNumber, backupsNumber));
> cacheCfg.setWriteThrough(true);
> cacheCfg.setReadThrough(true);
> cacheCfg.setRebalanceMode(CacheRebalanceMode.SYNC);
> cacheCfg.setCacheStoreFactory(new
> RocksDBCacheStoreFactory<>("/test/path/to/persistence",
> TEST_CACHE_NAME, cfg));
>
> cfg.setCacheConfiguration(cacheCfg);
>
> Could you give me advice on places which I need to pay attention?
>
>
> On Wed, Nov 15, 2017 at 3:02 PM, Andrey Mashenkov
>  wrote:
> > Hi Vyacheslav,
> >
> > Key to partition mapping shouldn't depends on topology, and shouldn't
> > changed unstable topology.
> > Looks like you've missed smth.
> >
> > Would you please share configuration?
> > Does all nodes share same RockDB database or each node has its own copy?
> >
> >
> >
> > On Wed, Nov 15, 2017 at 12:22 AM, Vyacheslav Daradur <
> daradu...@gmail.com>
> > wrote:
> >
> >> Hi, Igniters!
> >>
> >> I’m using partitioned Ignite cache with RocksDB as 3rd party persistence
> >> store.
> >> I've got an issue: if cache rebalancing is switched on, then it’s
> >> possible to lose some data.
> >>
> >> Basic scenario:
> >> 1) Start Ignite cluster and fill a cache with RocksDB persistence;
> >> 2) Stop all nodes
> >> 3) Start Ignite cluster and validate data
> >>
> >> This works fine while rebalancing is switched off.
> >>
> >> If rebalancing switched on: when I call Ignition#stopAll, some nodes
> >> go down sequentially and while one node having gone down another start
> >> rebalancing. When nodes started affinity function works with a full
> >> set of nodes and may define a wrong partition for a key because the
> >> previous state was changed at rebalancing.
> >>
> >> Maybe I'm doing something wrong. How can I avoid rebalancing while
> >> stopping all nodes in the cluster?
> >>
> >> Could you give me any advice, please?
> >>
> >> --
> >> Best Regards, Vyacheslav D.
> >>
> >
> >
> >
> > --
> > Best regards,
> > Andrey V. Mashenkov
>
>
>
> --
> Best Regards, Vyacheslav D.
>


[GitHub] ignite pull request #3064: Ignite 1.9.9

2017-11-17 Thread sk0x50
GitHub user sk0x50 opened a pull request:

https://github.com/apache/ignite/pull/3064

Ignite 1.9.9



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sk0x50/ignite ignite-1.9.9

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3064.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3064


commit d427021f329292fb69d348ba949ad1f8f1e9089e
Author: Andrey V. Mashenkov 
Date:   2017-06-21T16:30:27Z

IGNITE-5552: ServiceProcessor recalculates all service assignments even if 
there is a pending topology change.

commit f1b9cdc0716a1b23f54d68ce0fe19eb85107567d
Author: Alexey Goncharuk 
Date:   2017-06-14T18:37:54Z

GG-12354: Partial fix of IGNITE-5473: Introduce troubleshooting logger.

commit beb2409cfe2045789443d47de735d879961d371e
Author: Andrey V. Mashenkov 
Date:   2017-06-23T09:26:06Z

GG-12352: Forcible node drop makes cluster instable in some cases.
Disable forcible node drop by default.

commit 802f18fc250cbae8959192c78bb28dc525ed3cf7
Author: AMRepo 
Date:   2017-06-22T21:24:57Z

Fix compilation

commit 39d2dec85a3c571dfdb1cd6189b53ae2413a5d22
Author: Andrey V. Mashenkov 
Date:   2017-06-23T10:41:30Z

Merge branch 'ignite-1.7.12-b2' into ignite-1.8.8

# Conflicts:
#   modules/core/src/main/java/org/apache/ignite/internal/GridTopic.java
#   
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoManager.java
#   
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoMessageFactory.java
#   
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/IgniteIoTestMessage.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAdapter.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/atomic/GridDhtAtomicCache.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/service/GridServiceProcessor.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/task/GridTaskThreadContextKey.java
#   
modules/core/src/main/java/org/apache/ignite/spi/communication/tcp/TcpCommunicationSpi.java
#   
modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheTestSuite5.java

commit 96445156143b46b664f785b353463dd435ac342d
Author: Andrey V. Mashenkov 
Date:   2017-06-23T10:42:44Z

Merge remote-tracking branch 'origin/ignite-1.8.8' into ignite-1.8.8

# Conflicts:
#   
modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheTestSuite5.java

commit 7c569f15a80ab9301c21011d2c9159e311e46cef
Author: Andrey V. Mashenkov 
Date:   2017-06-23T10:48:10Z

Merge remote-tracking branch 'origin/ignite-1.8.7.p1' into ignite-1.8.8

commit a64339449be8fa602cab3f2868c5f74004a7b747
Author: Igor Sapego 
Date:   2017-06-23T13:57:49Z

IGNITE-4370: Implemented writing of batch of parameters for ODBC.

(cherry picked from commit c10be5780589cc84e7929e234e4411d515166e0b)
(cherry picked from commit d268b32cb252a5f06887d2b803d27ddc20ded95f)

commit cbe5df51c423568782e31245c7f1aa06c9ba3be1
Author: Igor Sapego 
Date:   2017-06-13T16:47:00Z

IGNITE-5478: ODBC: SQLNumParams now returns number of required parameters.

(cherry picked from commit b1c56a1)

(cherry picked from commit 4a8f295)

commit 9ad513e68b89e907f7db36a3f3f0daca0e5986e6
Author: Igor Sapego 
Date:   2017-06-23T14:06:40Z

IGNITE-5478: Fix for cherry pick

(cherry picked from commit a2a4ec1ee9794cb542f146a07c6c67002cad444e)

commit bfec212b1ece0e9e791de6dfb642324834fa77ca
Author: AMRepo 
Date:   2017-06-22T21:24:57Z

Partially reverted GG-12352.

commit 1abc14fdc4e39d8245c3e50fb2cf3d183df08021
Author: AMRepo 
Date:   2017-06-22T21:24:57Z

Partially reverted GG-12352.

commit cb95f7aa729fda19b4c3b7a5ada903a1e1bc2eab
Author: Andrey V. Mashenkov 
Date:   2017-06-23T16:08:00Z

Merge remote-tracking branch 'origin/ignite-1.7.12' into ignite-1.7.12

commit f59007f2c1221d8dd22abb0a9692b4abb31e87ad
Author: AMRepo 
Date:   2017-06-22T21:24:57Z

Partially reverted GG-12352.

commit 2c7d3c03cd36aee4f0985e674f550a62fc64d2db
Author: Andrey V. Mashenkov 
Date:   2017-06-23T17:11:00Z

Merge branch 'ignite-1.8.8' into ignite-1.9.4

# Conflicts:
#   

[GitHub] ignite pull request #3063: IGNITE-6949: Add support of distributed matrices ...

2017-11-17 Thread zaleslaw
GitHub user zaleslaw opened a pull request:

https://github.com/apache/ignite/pull/3063

IGNITE-6949: Add support of distributed matrices for OLS regression



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6949

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3063.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3063


commit b84d1b6ae284edaa510f7eefa84f5594dd1cb83e
Author: zaleslaw 
Date:   2017-11-17T14:44:05Z

Removed incorrect patch

commit 401a464c6bde19bbf1f11d0d32a2aea9facc
Author: zaleslaw 
Date:   2017-11-17T15:11:09Z

Added OLS Distributed support

commit 9701adc7957ef20cc84eeef3394634dabb6a9af0
Author: zaleslaw 
Date:   2017-11-17T15:55:07Z

Fixed code style




---


[GitHub] ignite pull request #3062: Backport IGNITE-6818, fix test

2017-11-17 Thread alamar
GitHub user alamar opened a pull request:

https://github.com/apache/ignite/pull/3062

Backport IGNITE-6818, fix test



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-13069

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3062.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3062


commit 4a1415ad01ff9fde30d5c7c02e6d938f1515178d
Author: Andrey V. Mashenkov 
Date:   2017-04-12T10:01:25Z

IGNITE-4907: Fixed excessive service instances can be started with dynamic 
deployment. This closes #1766.

(cherry picked from commit 0f7ef74)

commit e206b9f1fd3dbf927f940d558144a4796479ed5d
Author: vsisko 
Date:   2017-04-14T11:32:30Z

IGNITE-4871 Added Kubernetes IP finder to Cluster configuration screen.
(cherry picked from commit f978ff2)

commit b22738080101536a8af1ed60e70d693897e9bc7c
Author: dkarachentsev 
Date:   2017-04-14T14:54:02Z

ignite-4173 Fix test. Permits must be released on node fail.

(cherry picked from commit 1f867c6)

commit 41c5288606710b9c42983780ac7464a746d09eb0
Author: dkarachentsev 
Date:   2017-04-14T14:56:25Z

Merge remote-tracking branch 'origin/ignite-1.8.6' into ignite-1.8.6

commit 1985496ea98f4d7112a0b99727297427f343e9b2
Author: dkarachentsev 
Date:   2017-04-17T07:20:41Z

Add missed license headers.

commit ae794ab45b22abe1e570d5d29661396ea7c0f951
Author: dkarachentsev 
Date:   2017-04-17T10:40:28Z

IGNITE-4159 Fix version.

commit bf1049741f7a64728bd433f78262ba273f969848
Author: Andrey V. Mashenkov 
Date:   2017-04-17T16:00:30Z

IGNITE-4954 - Configurable expiration timeout for Cassandra session. This 
closes #1785.

commit aca67066ceaf762fcd44ef0715a4ba0f941226a8
Author: dkarachentsev 
Date:   2017-04-18T09:23:28Z

Merge branch 'ignite-1.8.5-p1' into ignite-1.8.6

commit a4c70f12305c15cbf15a6e5eb7c500f883a8b317
Author: agura 
Date:   2017-04-20T17:45:58Z

ignite-5041 NPE in deadlock detection fixed

commit d94172e7bb7ff4ebab422997e5c70f8cd03c0ae4
Author: agura 
Date:   2017-04-26T12:16:46Z

Redundant test removed

commit f9ecacc625b458539775e6550bd9b7613ed38f21
Author: dkarachentsev 
Date:   2017-04-28T08:46:23Z

IGNITE-5077 - Support service security permissions

backport from master
(cherry picked from commit 6236b5f)

commit 91c899b909383c78b78b9bf0c8f233b8c75ef29e
Author: Valentin Kulichenko 
Date:   2017-04-28T12:48:57Z

IGNITE-5081 - Removed redundant duplication of permissions in 
SecurityPermissionSetBuilder

commit b48a26b9b1e97fb8eb52c2a2f36005770922ac3d
Author: Valentin Kulichenko 
Date:   2017-04-28T12:53:33Z

IGNITE-5080 - Fixes in SecurityBasicPermissionSet

commit f66c23cbb9a6f2c923ebf75c58f00afaf1c0b5f3
Author: Evgenii Zhuravlev 
Date:   2017-05-03T14:47:45Z

IGNITE-4939 Receive event before cache initialized fix

commit 45b4d6316145d0b4b46713409f5e8fbe55ff4c41
Author: Evgenii Zhuravlev 
Date:   2017-05-04T09:11:37Z

IGNITE-4939 Receive event before cache initialized fix

commit 075bcfca0ea22633be13cd02647e359ad6fdca16
Author: Andrey V. Mashenkov 
Date:   2017-05-04T09:21:04Z

Fix flacky service deployment tests.

commit 25c06b50d46937cb39534cdf4147b862217289a2
Author: rfqu 
Date:   2017-05-02T16:46:44Z

ignite-4220 Support statements for JDBC and Cassandra store

commit 987c182686962673e70398395cb27e94f894713b
Author: nikolay_tikhonov 
Date:   2017-05-15T08:54:16Z

Fixed "IGNITE-5214 ConcurrentModificationException with enable DEBUG log 
level"

Signed-off-by: nikolay_tikhonov 

commit a5d2b9ba3ed1e72ef3970ecfed499a97f925ff2c
Author: Andrey V. Mashenkov 
Date:   2017-05-15T14:39:52Z

Fixed thread pools incorrect shutdown.

(cherry picked from commit dacf973)

commit 5e08c2951522a640bf84b6c60f03f29edd970ffc
Author: Andrey V. Mashenkov 
Date:   2017-05-15T17:24:10Z

IGNITE-5225: Fix NPE caused by changes in IGNITE-4577.

(cherry picked from commit d463840)

commit 62765b5b7598ab2a82b9c595e4a1276d9f2f5860
Author: sboikov 
Date:   2017-05-16T08:30:29Z

DirectByteBufferStreamImpl: converted asserts into exceptions.

(cherry picked from commit 560ef60)

commit ebc4a1648a80fbbd485e4c351fce9bee163318f9
Author: 

[GitHub] ignite pull request #2894: Ignite-6669: CacheStoreSessionListener#onSessionS...

2017-11-17 Thread sk0x50
Github user sk0x50 closed the pull request at:

https://github.com/apache/ignite/pull/2894


---


[GitHub] ignite pull request #3040: ignite-6924: Fixed missed CacheStoreSessionListen...

2017-11-17 Thread sk0x50
Github user sk0x50 closed the pull request at:

https://github.com/apache/ignite/pull/3040


---


[jira] [Created] (IGNITE-6951) .NET: BinaryWriter should throw exception on pointer fields

2017-11-17 Thread Pavel Tupitsyn (JIRA)
Pavel Tupitsyn created IGNITE-6951:
--

 Summary: .NET: BinaryWriter should throw exception on pointer 
fields
 Key: IGNITE-6951
 URL: https://issues.apache.org/jira/browse/IGNITE-6951
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
  Components: platforms
Reporter: Pavel Tupitsyn
 Fix For: 2.4


Serializing pointers does not make sense. We already throw on {{IntPtr}} 
fields, should do the same for any other pointer ({{long*}}, {{void*}}, etc).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[GitHub] ignite pull request #2832: IGNITE-6337 .NET: Thin client: SQL queries

2017-11-17 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2832


---


[jira] [Created] (IGNITE-6950) Update documentation for user facing CREATE TABLE params

2017-11-17 Thread Alexander Paschenko (JIRA)
Alexander Paschenko created IGNITE-6950:
---

 Summary: Update documentation for user facing CREATE TABLE params
 Key: IGNITE-6950
 URL: https://issues.apache.org/jira/browse/IGNITE-6950
 Project: Ignite
  Issue Type: Task
  Security Level: Public (Viewable by anyone)
  Components: documentation, sql
Affects Versions: 2.4
Reporter: Alexander Paschenko
Assignee: Alexander Paschenko
 Fix For: 2.4


Changes to documentation should be made to reflect additional spelling for some 
{{CREATE TABLE}} params added in IGNITE-6270.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[GitHub] ignite pull request #2831: IGNITE-6270 Renamed user facing CREATE TABLE

2017-11-17 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2831


---


[jira] [Created] (IGNITE-6949) Cleanup OLS code

2017-11-17 Thread Yury Babak (JIRA)
Yury Babak created IGNITE-6949:
--

 Summary: Cleanup OLS code
 Key: IGNITE-6949
 URL: https://issues.apache.org/jira/browse/IGNITE-6949
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
Reporter: Yury Babak
Assignee: Aleksey Zinoviev
 Fix For: 2.4


We want fix wrong styles like wildcards in imports, unnecessary empty lines, 
missed empty lines and if-else blocks format in OLS related files.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Re: IGNITE-6802 is ready for review (NullPointerException when WAL store and WAL archive paths are same )

2017-11-17 Thread Alexey Goncharuk
Dmitriy,

I added comments to the upsource review, please take a look.

2017-11-17 15:57 GMT+03:00 Дмитрий Рябов :

> Hello, Igniters! Please review.
>
> JIRA: https://issues.apache.org/jira/browse/IGNITE-6802
> PR: https://github.com/apache/ignite/pull/3021/files
> Upsource: https://reviews.ignite.apache.org/ignite/review/IGNT-CR-396
> CI tests: https://ci.ignite.apache.org/viewLog.html?buildId=945125;
> tab=buildResultsDiv=Ignite20Tests_RunAll
>


[jira] [Created] (IGNITE-6948) SQL: Document new INLINE_SIZE option in CREATE INDEX command

2017-11-17 Thread Kirill Shirokov (JIRA)
Kirill Shirokov created IGNITE-6948:
---

 Summary: SQL: Document new INLINE_SIZE option in CREATE INDEX 
command
 Key: IGNITE-6948
 URL: https://issues.apache.org/jira/browse/IGNITE-6948
 Project: Ignite
  Issue Type: Task
  Security Level: Public (Viewable by anyone)
  Components: documentation
Reporter: Kirill Shirokov
 Fix For: 2.4


INLINE_SIZE is optional and accepts positive integer values. To specify the 
default inline size user should omit the option.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Re: Stop Ignite opening 47500 and 47100

2017-11-17 Thread Alexey Goncharuk
Hi,

You can implement your own no-op discovery and communication SPIs and plug
them in IgniteConfiguration, then no network ports will be opened.

--AG

2017-11-17 16:05 GMT+03:00 karthik :

> In our Project we are not allowed to open any port. Is it possible to
> disable
> TcpDiscoveySpi and TcpCommunicationSpi . I use Ignite only on single node
> in
> sever mode.
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>


[GitHub] ignite pull request #3061: IGNITE-2662 .NET Core support (run on Linux)

2017-11-17 Thread ptupitsyn
GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/3061

IGNITE-2662 .NET Core support (run on Linux)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ptupitsyn/ignite ignite-2662

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3061.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3061


commit dc2ad8777d6c11fe7f5b457f073c2d1a5d7f9933
Author: Pavel Tupitsyn 
Date:   2017-11-15T18:43:42Z

IGNITE-2662 .NET Core support (run on Linux)

commit 79ad83dc4dcbb9baa16ef3ec569f91c592cb8103
Author: Pavel Tupitsyn 
Date:   2017-11-15T18:48:11Z

Add solution file

commit 90888b0385b87ef3ff0b171224f318a5abb3a32a
Author: Pavel Tupitsyn 
Date:   2017-11-15T18:56:30Z

First test implemented

commit c900fc5a91b39b629f6643f457515db626de772e
Author: Pavel Tupitsyn 
Date:   2017-11-15T18:58:04Z

wip

commit 1349c71a3b2bb4b91612426d4b09fd3b5d8341af
Author: Pavel Tupitsyn 
Date:   2017-11-15T19:00:06Z

wip

commit 8489bfdb0887f59e81265a8c49289f680b5105c0
Author: Pavel Tupitsyn 
Date:   2017-11-15T19:10:37Z

Fix callbacks initialization on Core

commit 6c0e6c7b204d2e8c886e0129c09b116770368848
Author: Pavel Tupitsyn 
Date:   2017-11-15T19:11:38Z

wip

commit a6d7081f90bd02e56f3b1e1eb91f7a7ce6d63186
Author: Pavel Tupitsyn 
Date:   2017-11-15T19:12:08Z

Ignore exe files

commit 6f618ff83b2f8b74a5f640d4bce782fc0b89edb8
Author: Pavel Tupitsyn 
Date:   2017-11-15T19:20:31Z

Add unix check

commit 9a660799d8d706a93deeb235a46a38061366cdae
Author: Pavel Tupitsyn 
Date:   2017-11-15T19:47:54Z

Fix NuGet version issues

commit a56fd81e41f0d4969bf06d03c98df75910cec797
Author: Pavel Tupitsyn 
Date:   2017-11-15T19:53:38Z

Fix readme

commit 0c434e7622affd119ee77fa9904f765825c7188a
Author: Pavel Tupitsyn 
Date:   2017-11-15T19:58:58Z

Adding test scripts

commit 0684ff1b5484918b2be46d8fa58d4c24bf8cc828
Author: Pavel Tupitsyn 
Date:   2017-11-15T20:04:53Z

Fix line breaks

commit 3373839cf9de4fce7b435ee7f4b0686332c5f313
Author: Pavel Tupitsyn 
Date:   2017-11-15T20:05:38Z

wip

commit 8f0212cf8c5fb5fc06dcd1a0e1de7fbf5d063dd4
Author: Pavel Tupitsyn 
Date:   2017-11-15T20:09:19Z

Fixing test script

commit ed5f0508e8ad02c0f4807f5ecb6cbe5ed284c5cf
Author: Pavel Tupitsyn 
Date:   2017-11-15T20:17:59Z

WSL script works!

commit 4f851234b9f90ee07d2f4f55010cb00b5334dcb0
Author: Pavel Tupitsyn 
Date:   2017-11-16T06:33:13Z

Fix test script

commit 2e9ee68622c70dc21b17a66877dae45e1c1e96ae
Author: Pavel Tupitsyn 
Date:   2017-11-16T09:00:38Z

Merge branch 'master' into ignite-2662

commit 6241a66735456aa2b22fe8f7c6d2298cdb348349
Author: Pavel Tupitsyn 
Date:   2017-11-16T09:21:22Z

xplat dll loader

commit 18037a44809ffe7f9474db23358cffa55db84583
Author: Pavel Tupitsyn 
Date:   2017-11-16T09:38:14Z

Implementing Java detection on Linux

commit 1e674877d25df345e0c258d275f009416eb48aa2
Author: Pavel Tupitsyn 
Date:   2017-11-16T09:54:01Z

Refactoring dll loading

commit 19699d72bfdda806d464f9515078327703bda93c
Author: Pavel Tupitsyn 
Date:   2017-11-16T09:55:01Z

wip dll loading

commit aba0381c265075c44aeeff76f03c75c945ab44d2
Author: Pavel Tupitsyn 
Date:   2017-11-16T10:03:55Z

Fix config copy

commit 999d1536d2475f501a846950dc1dd2a3a57f984e
Author: Pavel Tupitsyn 
Date:   2017-11-16T10:11:18Z

Fix JVM class to be xplat

commit 323c8708e68eb47fdd6b87d867a6726dc13ff006
Author: Pavel Tupitsyn 
Date:   2017-11-16T10:15:20Z

Add logging to test

commit 0e05f74922336c48a52a5ae64fb7b82cf256f3cb
Author: Pavel Tupitsyn 
Date:   2017-11-16T11:20:57Z

Fix NuGet home detection

commit 910747328e48664364ade24ae1740a5960afaa73
Author: Pavel Tupitsyn 
Date:   2017-11-16T11:21:47Z

fix home detection on Linux

commit 881ed862b29851262a8c563a170cdae33f521a23
Author: Pavel Tupitsyn 
Date:   2017-11-16T11:28:52Z

Fix NuGet folder name casing

commit 4bbc940eb39f335038d5d8b3bba69818c93283b0
Author: Pavel Tupitsyn 
Date:   2017-11-16T11:38:57Z

Fixing dir separator handling

commit 5c0c9d32b98e752911bf1ed4434261b7be720265
Author: Pavel Tupitsyn 

[GitHub] ignite pull request #3060: Ignite 2.1.7.b3

2017-11-17 Thread apopovgg
GitHub user apopovgg opened a pull request:

https://github.com/apache/ignite/pull/3060

Ignite 2.1.7.b3



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.1.7.b3

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3060.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3060


commit ad38f7b4b5e6845b2d7eb888f805484504f5
Author: Ilya Lantukh 
Date:   2017-08-23T11:12:42Z

gg-12662 : Fixed JDBC backward compatibility.

commit 28c906e3e0c51e6f1a4a95b2027d248f9b5035c2
Author: Sergey Chugunov 
Date:   2017-08-02T15:14:46Z

IGNITE-5542 CacheGroup configuration from cluster is merged with local 
settings

(cherry picked from commit 88818ec)

commit caeb11936fa3534b9468d443c11744362044cae5
Author: sboikov 
Date:   2017-08-23T12:19:52Z

ignite-6124 Guard logging with isInfoEnabled

(cherry picked from commit bebe4d8)

commit 6f407ebabb9dc27459fdbee6423640132b995b1d
Author: tledkov-gridgain 
Date:   2017-08-23T12:46:23Z

IGNITE-6169: Fixed thin JDBC driver compatibility problem.

commit 9dac636c4eef494fe612389c19218eec92057fc0
Author: Ilya Kasnacheev 
Date:   2017-08-23T13:26:58Z

IGNITE-4643: Fixed NPE in  JdbcDatabaseMetadata.getIndexInfo(). This closes 
#2481.

commit 77241cdc45c90ee9bab4a7a0f3d5a1a7664e3426
Author: sboikov 
Date:   2017-08-23T13:45:34Z

Merge remote-tracking branch 'community/ignite-2.1.4' into ignite-2.1.4

commit a5e376f63886696331e5be0c457dc0624c49e3d4
Author: sboikov 
Date:   2017-08-23T13:44:04Z

ignite-6124 Added missed initialization of merged join exchanges in 
GridDhtPartitionsExchangeFuture.onBecomeCoordinator

(cherry picked from commit 0c5dca9)

commit be5589db9e0600b295b745ddab5e7aae390ac7ae
Author: Ilya Lantukh 
Date:   2017-08-23T14:25:33Z

ignite-5986 : Fixed failing .NET test.

commit 43e4ff2c0ecd1ef30d18cf1fbc9052f5ba703d05
Author: sboikov 
Date:   2017-07-18T14:52:51Z

Fixed test IgniteClusterActivateDeactivateTestWithPersistence.

(cherry picked from commit 54585ab)

commit d596b7806db3f002f83da5a02bc882d03dae3dfd
Author: Ilya Lantukh 
Date:   2017-08-23T15:23:06Z

Updated classnames.properties.

commit 3e08cd401d598a34832e72afc5e6c94a3a9ab081
Author: sboikov 
Date:   2017-08-23T15:29:52Z

ignite-6174 Temporary changed test until issue not fixed

(cherry picked from commit 4fe8f76)

commit 44e0b4cd62142dce8cf39f826449b9a04e22e1cf
Author: Alexey Kuznetsov 
Date:   2017-08-24T07:57:36Z

IGNITE-6136 Fixed version for demo.
(cherry picked from commit e1bf8d7)

commit 8d1838b03d6c1e5f86dfbb7f41c59895775e20c1
Author: Dmitry Pavlov 
Date:   2017-07-27T11:51:25Z

Adjusted memory policy to prevent OOM.

commit a3ec54b16bce1a569fbefba17188ccb4702b82a4
Author: sboikov 
Date:   2017-08-24T11:09:12Z

ignite-6124 DataStreamerImpl: do not wait for exchange future inside cache 
gateway.

(cherry picked from commit 3ab523c)

commit 30e6d019a21f4a045a50d7d95a04507e3b646e69
Author: sboikov 
Date:   2017-08-24T11:10:34Z

Merge remote-tracking branch 'community/ignite-2.1.4' into ignite-2.1.4

commit 41f574a7372ffc04b69809298798f24fb34c161f
Author: Dmitriy Govorukhin 
Date:   2017-08-24T12:58:27Z

Fixed test.

commit 943736b36d67381157fc2807cd7af4b03d44fef3
Author: nikolay_tikhonov 
Date:   2017-08-24T15:58:16Z

Revert "IGNITE-5947 Fixed "ClassCastException when two-dimensional array is 
fetched from cache".
* Due to this changes break compatibility with .NET;
* This fix doesn't cover all cases.

Signed-off-by: nikolay_tikhonov 

commit c2e836b5b9b183404f4507c64c13ab5c05653d24
Author: EdShangGG 
Date:   2017-08-24T16:15:24Z

ignite-6175 JVM Crash in Ignite Binary Objects Simple Mapper Basic suite

Signed-off-by: Andrey Gura 

commit b2b596b4f59bcf7a1b7397a6fd681a0ae47092db
Author: Andrey Novikov 
Date:   2017-08-25T03:48:15Z

IGNITE-5200 Web Console: Don't cache generated chunks in production.
(cherry picked from commit e1eb1b9)

commit 9399610d2dd4b67b1da6475ce2141787fb8dbb0e
Author: Ilya Lantukh 
Date:   2017-08-25T10:12:32Z

ignite-6180: restoring marshaller mappings on node start is implemented

commit 5bda4090f1580ea7b6557c8716e57a12572c322f
Author: Ivan Rakov 

Stop Ignite opening 47500 and 47100

2017-11-17 Thread karthik
In our Project we are not allowed to open any port. Is it possible to disable
TcpDiscoveySpi and TcpCommunicationSpi . I use Ignite only on single node in
sever mode. 



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/


request for contributors permissions

2017-11-17 Thread Dan
Hello!
 
I want to resolve bugs.
My GutHub account: Denis-Log

My Jira account: da.loginov

Please, provide me with necessary permissions.
 
Thanks in advance.
 

 
Best regards,
Denis


[GitHub] ignite pull request #3013: IGNITE-6850 SQL: integrate index inline size to C...

2017-11-17 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3013


---


[GitHub] ignite pull request #3059: IGNITE-358 add InputSplit to HadoopV1Reporter

2017-11-17 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3059


---


[GitHub] ignite pull request #3030: IGNITE-5846: Add support of distributed matrices ...

2017-11-17 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3030


---


Facility to detect long STW pauses and other system response degradations

2017-11-17 Thread Dmitriy_Sorokin
Hi, Igniters!

This discussion thread related to
https://issues.apache.org/jira/browse/IGNITE-6171.

Currently there are no JVM performance monitoring tools in AI, for example
the impact of GC (eg STW) on the operation of the node. I think we should
add this functionality.

1) It is useful to know that STW duration increased or any other situations
leads to similar consequences.
This will allow system administrators to solve issues prior they become
problems.

I propose to add a special thread that will record current time every N
milliseconds and check the difference with the latest recorded value. 
The maximum and total pause values for a certain period can be published in
the special metrics available through JMX.

2) If the pause reaches a critical value, we need to stop the node, without
waiting for end of the pause.

The thread (from the first part of the proposed solution) is able to
estimate the pause duration, but only after its completion. 
So, we need an external thread (in another JVM or native) that is able to
recognize that the pause duration has passed the critical mark.

We can estimate (STW or similar) pause duration by
 a) reading value updated by the first thread, somehow (eg via JMX, shmem or
shared file)
 or
 b) by using JVM diagnostic tools. Does anybody know crossplatform
solutions?

Feel free to suggest ideas or tips, especially about second part of
proposal.

Thoughts?



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/


[GitHub] ignite pull request #3048: IGNITE-6932 GridQueryProcessor.querySqlFieldsNoCa...

2017-11-17 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3048


---


IGNITE-6802 is ready for review (NullPointerException when WAL store and WAL archive paths are same )

2017-11-17 Thread Дмитрий Рябов
Hello, Igniters! Please review.

JIRA: https://issues.apache.org/jira/browse/IGNITE-6802
PR: https://github.com/apache/ignite/pull/3021/files
Upsource: https://reviews.ignite.apache.org/ignite/review/IGNT-CR-396
CI tests: 
https://ci.ignite.apache.org/viewLog.html?buildId=945125=buildResultsDiv=Ignite20Tests_RunAll


[GitHub] ignite pull request #2981: Ignite 6496: Client node does not reconnect to se...

2017-11-17 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/2981


---


[jira] [Created] (IGNITE-6947) Near optimistic prepare future should abandon remap after failed map attempt

2017-11-17 Thread Alexey Goncharuk (JIRA)
Alexey Goncharuk created IGNITE-6947:


 Summary: Near optimistic prepare future should abandon remap after 
failed map attempt
 Key: IGNITE-6947
 URL: https://issues.apache.org/jira/browse/IGNITE-6947
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
Reporter: Alexey Goncharuk
Assignee: Alexey Goncharuk


Otherwise, it leads to an NPE if there are no affinity nodes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Re: Losing data during restarting cluster with persistence enabled

2017-11-17 Thread Vyacheslav Daradur
Hi Andrey! Thank you for answering.

>> Key to partition mapping shouldn't depends on topology, and shouldn't 
>> changed unstable topology.
Key to partition mapping doesn't depend on topology in my test
affinity function. It only depends on partitions number.
But partition to node mapping depends on topology and at cluster stop,
when one node left topology, some partitions may be moved to other
nodes.

>> Does all nodes share same RockDB database or each node has its own copy?
Each Ignite node has own RocksDB instance.

>> Would you please share configuration?
It's pretty simple:
IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setIgniteInstanceName(instanceName);

CacheConfiguration cacheCfg = new
CacheConfiguration<>();
cacheCfg.setName(TEST_CACHE_NAME);
cacheCfg.setCacheMode(CacheMode.PARTITIONED);

cacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.PRIMARY_SYNC);
cacheCfg.setBackups(1);
cacheCfg.setAffinity(new
TestAffinityFunction(partitionsNumber, backupsNumber));
cacheCfg.setWriteThrough(true);
cacheCfg.setReadThrough(true);
cacheCfg.setRebalanceMode(CacheRebalanceMode.SYNC);
cacheCfg.setCacheStoreFactory(new
RocksDBCacheStoreFactory<>("/test/path/to/persistence",
TEST_CACHE_NAME, cfg));

cfg.setCacheConfiguration(cacheCfg);

Could you give me advice on places which I need to pay attention?


On Wed, Nov 15, 2017 at 3:02 PM, Andrey Mashenkov
 wrote:
> Hi Vyacheslav,
>
> Key to partition mapping shouldn't depends on topology, and shouldn't
> changed unstable topology.
> Looks like you've missed smth.
>
> Would you please share configuration?
> Does all nodes share same RockDB database or each node has its own copy?
>
>
>
> On Wed, Nov 15, 2017 at 12:22 AM, Vyacheslav Daradur 
> wrote:
>
>> Hi, Igniters!
>>
>> I’m using partitioned Ignite cache with RocksDB as 3rd party persistence
>> store.
>> I've got an issue: if cache rebalancing is switched on, then it’s
>> possible to lose some data.
>>
>> Basic scenario:
>> 1) Start Ignite cluster and fill a cache with RocksDB persistence;
>> 2) Stop all nodes
>> 3) Start Ignite cluster and validate data
>>
>> This works fine while rebalancing is switched off.
>>
>> If rebalancing switched on: when I call Ignition#stopAll, some nodes
>> go down sequentially and while one node having gone down another start
>> rebalancing. When nodes started affinity function works with a full
>> set of nodes and may define a wrong partition for a key because the
>> previous state was changed at rebalancing.
>>
>> Maybe I'm doing something wrong. How can I avoid rebalancing while
>> stopping all nodes in the cluster?
>>
>> Could you give me any advice, please?
>>
>> --
>> Best Regards, Vyacheslav D.
>>
>
>
>
> --
> Best regards,
> Andrey V. Mashenkov



-- 
Best Regards, Vyacheslav D.


[GitHub] ignite pull request #3059: IGNITE-358 add InputSplit to HadoopV1Reporter

2017-11-17 Thread kdudkov
GitHub user kdudkov opened a pull request:

https://github.com/apache/ignite/pull/3059

IGNITE-358 add InputSplit to HadoopV1Reporter



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-358

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3059.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3059


commit 23253093e26a875602c0617d8b1e2e7a873cda77
Author: Konstantin Dudkov 
Date:   2017-11-17T12:09:43Z

IGNITE-358 add InputSplit to HadoopV1Reporter




---


[GitHub] ignite pull request #3058: Ignite 6927

2017-11-17 Thread oleg-ostanin
GitHub user oleg-ostanin opened a pull request:

https://github.com/apache/ignite/pull/3058

Ignite 6927



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6927

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3058.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3058


commit 0f92af4e44515608107495bd87f88bb50e1d6fa7
Author: Andrey Novikov 
Date:   2017-11-16T04:27:48Z

IGNITE-6927 Added Web Console direct install build.

commit 87bf305bfae3bb39c634468233e62a2a20c0a406
Author: Andrey Novikov 
Date:   2017-11-17T06:30:39Z

Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite 
into ignite-6927

commit b707a160eae67e263ec370d47ea08271eb135f4e
Author: Andrey Novikov 
Date:   2017-11-17T06:37:40Z

IGNITE-6927 Added build instructions.

commit 58b8be58e420bad9157503aff1035510ecb70283
Author: Andrey Novikov 
Date:   2017-11-17T06:52:19Z

IGNITE-6927 Added build instructions.

commit 5bdf0a012daa8a71143f4c76f1a195e381e4c73f
Author: Andrey Novikov 
Date:   2017-11-17T06:52:29Z

Version changed to 2.3.0-SNAPSHOT




---


[GitHub] ignite pull request #3057: Ignite gg 13072

2017-11-17 Thread dkarachentsev
GitHub user dkarachentsev opened a pull request:

https://github.com/apache/ignite/pull/3057

Ignite gg 13072



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-13072

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3057.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3057






---


[GitHub] ignite pull request #3056: Ignite gg 13072

2017-11-17 Thread dkarachentsev
Github user dkarachentsev closed the pull request at:

https://github.com/apache/ignite/pull/3056


---


[GitHub] ignite pull request #3056: Ignite gg 13072

2017-11-17 Thread dkarachentsev
GitHub user dkarachentsev opened a pull request:

https://github.com/apache/ignite/pull/3056

Ignite gg 13072



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-13072

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3056.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3056






---


[jira] [Created] (IGNITE-6946) Change Ignite Logger configuration on the fly

2017-11-17 Thread Alexey Popov (JIRA)
Alexey Popov created IGNITE-6946:


 Summary: Change Ignite Logger configuration on the fly
 Key: IGNITE-6946
 URL: https://issues.apache.org/jira/browse/IGNITE-6946
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
  Components: general
Affects Versions: 2.3
Reporter: Alexey Popov


It would be great if we can change Ignite Logger configuration on the fly 
without restarting the cluster node. I will really help to debug an issue while 
it is reproducible at the current cluster state.
It should be done within the configured type of a logger, i.e. it is enough to 
change logging levels without changing the Logger type itself.

Such configuration change should be done for all supported logger types (JUL, 
log4j, log4i2 and others).

Also it should be done easily, for instance, via Visor, WebConsole or any other 
user-friendly tool ).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (IGNITE-6945) SQL: optionally do not copy offheap rows for local SqlQuery

2017-11-17 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-6945:
---

 Summary: SQL: optionally do not copy offheap rows for local 
SqlQuery
 Key: IGNITE-6945
 URL: https://issues.apache.org/jira/browse/IGNITE-6945
 Project: Ignite
  Issue Type: Task
  Security Level: Public (Viewable by anyone)
  Components: sql
Reporter: Vladimir Ozerov
Assignee: Taras Ledkov
 Fix For: 2.4


Currently when iterating over rows we eagerly materialize them [1]. If key or 
value are large enough, we could loose a lot of time on offheap-heap copying. 
To partially mitigate this, we can do the following:
1) Add new flag {{SqlQuery.localNoCopy}} which is applicable only for local 
queries.
2) When enabled we will not copy final {{_KEY}} and {{_VAL}} columns to heap. 
but rather wrap them into {{BinaryOffheapObjectImpl}}
3) These rows must be released when query iterator switches to the next row.

[1] {{H2RowFactory.getRow}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[GitHub] ignite pull request #3055: IGNITE-5731: curWaitingJobs and avgJobWaitTime ca...

2017-11-17 Thread Denis-Log
GitHub user Denis-Log opened a pull request:

https://github.com/apache/ignite/pull/3055

IGNITE-5731: curWaitingJobs and avgJobWaitTime calculation fixed



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/Denis-Log/ignite IGNITE-5731

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3055.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3055


commit 71b01e53ba0987027c82666080700839bd656671
Author: Denis Loginov 
Date:   2017-11-17T08:24:50Z

IGNITE-5731: curWaitingJobs and avgJobWaitTime calculation fixed




---