Re: IgniteDataStreamer silently fails on a server node

2018-02-22 Thread Nikolay Izhikov 

> I set 
> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl#DFLT_MAX_REMAP_CNT
>  to zero and see test fails now.

Anton, thank you very much!

I've prepared PR [1] to fix issue [2]. TC looks OK.

Looks like it was introduced by commit 53802d8b46f8ee09d51ba84267f65dc6f7e73a2f 
[3]

Can, anyone do review of my fix?
It literally consists of 1 LOC.

[1] https://github.com/apache/ignite/pull/3550
[2] https://issues.apache.org/jira/browse/IGNITE-7756
[3] 
https://github.com/SomeFire/ignite/commit/53802d8b46f8ee09d51ba84267f65dc6f7e73a2f

В Ср, 21/02/2018 в 15:49 +0300, Anton Vinogradov пишет:
> Nikolay,
> 
> I checked your reproducer and it looks like retry helps data streamer to
> solve this.
> I
> set 
> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl#DFLT_MAX_REMAP_CNT
> to zero and see test fails now.
> 
> Looks like second attempt is successful for such case.
> 
> On Wed, Feb 21, 2018 at 1:36 PM, Dmitriy Setrakyan 
> wrote:
> 
> > On Wed, Feb 21, 2018 at 12:10 AM, Nikolay Izhikov 
> > wrote:
> > 
> > > Hello, Igniters.
> > > 
> > > > Does this exception happen because IgniteUuid class is not correctly
> > > > handled?
> > > 
> > > Yes. I've fixed it.
> > > But story doesn't end :)
> > > 
> > > > As far as propagating exceptions, addData(...) is asynchronous
> > 
> > operation
> > > > and returns IgniteFuture.
> > > 
> > > Thank you, my bad.
> > > 
> > > But, I can't understand when returned future should complete?
> > > 
> > > Javadocs or documentation doesn't explain it.
> > > Also, I can't find any tests that checks future returned from addData.
> > > 
> > > Can someone explan it to me?
> > > 
> > 
> > The returned future should complete either when the data is successfully
> > stored in cache, or whenever the failure occurs.
> > 

signature.asc
Description: This is a digitally signed message part

Re: IgniteDataStreamer silently fails on a server node

2018-02-21 Thread Anton Vinogradov 
Nikolay,

I checked your reproducer and it looks like retry helps data streamer to
solve this.
I
set 
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl#DFLT_MAX_REMAP_CNT
to zero and see test fails now.

Looks like second attempt is successful for such case.

On Wed, Feb 21, 2018 at 1:36 PM, Dmitriy Setrakyan 
wrote:

> On Wed, Feb 21, 2018 at 12:10 AM, Nikolay Izhikov 
> wrote:
>
> > Hello, Igniters.
> >
> > > Does this exception happen because IgniteUuid class is not correctly
> > > handled?
> >
> > Yes. I've fixed it.
> > But story doesn't end :)
> >
> > > As far as propagating exceptions, addData(...) is asynchronous
> operation
> > > and returns IgniteFuture.
> >
> > Thank you, my bad.
> >
> > But, I can't understand when returned future should complete?
> >
> > Javadocs or documentation doesn't explain it.
> > Also, I can't find any tests that checks future returned from addData.
> >
> > Can someone explan it to me?
> >
>
> The returned future should complete either when the data is successfully
> stored in cache, or whenever the failure occurs.
>

Re: IgniteDataStreamer silently fails on a server node

2018-02-21 Thread Dmitriy Setrakyan 
On Wed, Feb 21, 2018 at 12:10 AM, Nikolay Izhikov 
wrote:

> Hello, Igniters.
>
> > Does this exception happen because IgniteUuid class is not correctly
> > handled?
>
> Yes. I've fixed it.
> But story doesn't end :)
>
> > As far as propagating exceptions, addData(...) is asynchronous operation
> > and returns IgniteFuture.
>
> Thank you, my bad.
>
> But, I can't understand when returned future should complete?
>
> Javadocs or documentation doesn't explain it.
> Also, I can't find any tests that checks future returned from addData.
>
> Can someone explan it to me?
>

The returned future should complete either when the data is successfully
stored in cache, or whenever the failure occurs.

Re: IgniteDataStreamer silently fails on a server node

2018-02-21 Thread Nikolay Izhikov 
Hello, Igniters.

> Does this exception happen because IgniteUuid class is not correctly
> handled? 

Yes. I've fixed it. 
But story doesn't end :)

> As far as propagating exceptions, addData(...) is asynchronous operation
> and returns IgniteFuture.

Thank you, my bad.

But, I can't understand when returned future should complete?

Javadocs or documentation doesn't explain it.
Also, I can't find any tests that checks future returned from addData.

Can someone explan it to me?

В Пн, 19/02/2018 в 21:47 -0600, Dmitriy Setrakyan пишет:
> Nikolay,
> 
> Does this exception happen because IgniteUuid class is not correctly
> handled? If that's the case, we should fix it. Would be great if you could
> do it.
> 
> As far as propagating exceptions, addData(...) is asynchronous operation
> and returns IgniteFuture. The exception should be propagated to that
> future. Do you not see it there?
> 
> D.
> 
> On Mon, Feb 19, 2018 at 1:00 PM, Nikolay Izhikov 
> wrote:
> 
> > Hello, Igniters.
> > 
> > While working on IGNITE-7727 I found strange behavior of
> > IgniteDataStreamer:
> > 
> > If we have IgniteUuid as an indexed type update silently brokes on a
> > server node.
> > Client doesn't have any notification about fails.
> > All calls of `addData`, `close`, etc. succeed on a client side but fails
> > on server side.
> > 
> > I see 2 issue here:
> > 
> > 1. The fail itself - it certainly a bug, I think I can fix it.
> > 
> > 2. Lack of client notification. Is it OK when client doesn't know about
> > fails of streamer updates?
> > Do we have this documented? I briefly looked at streamer docs but can't
> > find description of such behavior.
> > 
> > 
> > Reproducer [1]
> > 
> > ```
> > public void testStreamer() throws Exception {
> > Ignite client = grid("client");
> > 
> > CacheConfiguration ccfg = new CacheConfiguration("UUID_CACHE");
> > 
> > ccfg.setIndexedTypes(IgniteUuid.class, String.class);
> > 
> > client.createCache(ccfg);
> > 
> > try(IgniteDataStreamer cache =
> > client.dataStreamer("UUID_CACHE")) {
> > 
> > for(Integer i=0; i<2; i++)
> > cache.addData(IgniteUuid.randomUuid().toString(),
> > i.toString());
> > }
> > }
> > ```
> > 
> > Server node stack trace [2]:
> > 
> > ```
> > Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> > update index, incorrect key class [expCls=org.apache.ignite.
> > lang.IgniteUuid,
> > actualCls=org.apache.ignite.internal.binary.BinaryObjectImpl]
> > at org.apache.ignite.internal.processors.query.GridQueryProcessor.
> > typeByValue(GridQueryProcessor.java:1954)
> > at org.apache.ignite.internal.processors.query.
> > GridQueryProcessor.store(GridQueryProcessor.java:1877)
> > at org.apache.ignite.internal.processors.cache.query.
> > GridCacheQueryManager.store(GridCacheQueryManager.java:403)
> > at org.apache.ignite.internal.processors.cache.
> > IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.finishUpdate(
> > IgniteCacheOffheapManagerImpl.java:1343)
> > at org.apache.ignite.internal.processors.cache.
> > IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(
> > IgniteCacheOffheapManagerImpl.java:1207)
> > at org.apache.ignite.internal.processors.cache.
> > IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.
> > java:345)
> > at org.apache.ignite.internal.processors.cache.
> > GridCacheMapEntry.storeValue(GridCacheMapEntry.java:3527)
> > at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.
> > initialValue(GridCacheMapEntry.java:2735)
> > at org.apache.ignite.internal.processors.datastreamer.
> > DataStreamerImpl$IsolatedUpdater.receive(DataStreamerImpl.java:2113)
> > ... 11 more
> > ```
> > 
> > [1] https://gist.github.com/nizhikov/2e70a73c7b74a50fc89d270e9af1e1ca
> > 
> > [2] https://gist.github.com/nizhikov/c491c8f2b45aa59458b37b42b4b8dab4

signature.asc
Description: This is a digitally signed message part

Re: IgniteDataStreamer silently fails on a server node

2018-02-19 Thread Dmitriy Setrakyan 
Nikolay,

Does this exception happen because IgniteUuid class is not correctly
handled? If that's the case, we should fix it. Would be great if you could
do it.

As far as propagating exceptions, addData(...) is asynchronous operation
and returns IgniteFuture. The exception should be propagated to that
future. Do you not see it there?

D.

On Mon, Feb 19, 2018 at 1:00 PM, Nikolay Izhikov 
wrote:

> Hello, Igniters.
>
> While working on IGNITE-7727 I found strange behavior of
> IgniteDataStreamer:
>
> If we have IgniteUuid as an indexed type update silently brokes on a
> server node.
> Client doesn't have any notification about fails.
> All calls of `addData`, `close`, etc. succeed on a client side but fails
> on server side.
>
> I see 2 issue here:
>
> 1. The fail itself - it certainly a bug, I think I can fix it.
>
> 2. Lack of client notification. Is it OK when client doesn't know about
> fails of streamer updates?
> Do we have this documented? I briefly looked at streamer docs but can't
> find description of such behavior.
>
>
> Reproducer [1]
>
> ```
> public void testStreamer() throws Exception {
> Ignite client = grid("client");
>
> CacheConfiguration ccfg = new CacheConfiguration("UUID_CACHE");
>
> ccfg.setIndexedTypes(IgniteUuid.class, String.class);
>
> client.createCache(ccfg);
>
> try(IgniteDataStreamer cache =
> client.dataStreamer("UUID_CACHE")) {
>
> for(Integer i=0; i<2; i++)
> cache.addData(IgniteUuid.randomUuid().toString(),
> i.toString());
> }
> }
> ```
>
> Server node stack trace [2]:
>
> ```
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> update index, incorrect key class [expCls=org.apache.ignite.
> lang.IgniteUuid,
> actualCls=org.apache.ignite.internal.binary.BinaryObjectImpl]
> at org.apache.ignite.internal.processors.query.GridQueryProcessor.
> typeByValue(GridQueryProcessor.java:1954)
> at org.apache.ignite.internal.processors.query.
> GridQueryProcessor.store(GridQueryProcessor.java:1877)
> at org.apache.ignite.internal.processors.cache.query.
> GridCacheQueryManager.store(GridCacheQueryManager.java:403)
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.finishUpdate(
> IgniteCacheOffheapManagerImpl.java:1343)
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(
> IgniteCacheOffheapManagerImpl.java:1207)
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.
> java:345)
> at org.apache.ignite.internal.processors.cache.
> GridCacheMapEntry.storeValue(GridCacheMapEntry.java:3527)
> at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.
> initialValue(GridCacheMapEntry.java:2735)
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerImpl$IsolatedUpdater.receive(DataStreamerImpl.java:2113)
> ... 11 more
> ```
>
> [1] https://gist.github.com/nizhikov/2e70a73c7b74a50fc89d270e9af1e1ca
>
> [2] https://gist.github.com/nizhikov/c491c8f2b45aa59458b37b42b4b8dab4

IgniteDataStreamer silently fails on a server node

2018-02-19 Thread Nikolay Izhikov 
Hello, Igniters.

While working on IGNITE-7727 I found strange behavior of IgniteDataStreamer:

If we have IgniteUuid as an indexed type update silently brokes on a server 
node.
Client doesn't have any notification about fails.
All calls of `addData`, `close`, etc. succeed on a client side but fails on 
server side.

I see 2 issue here:

1. The fail itself - it certainly a bug, I think I can fix it.

2. Lack of client notification. Is it OK when client doesn't know about fails 
of streamer updates?
Do we have this documented? I briefly looked at streamer docs but can't find 
description of such behavior.


Reproducer [1]

```
public void testStreamer() throws Exception {
Ignite client = grid("client");

CacheConfiguration ccfg = new CacheConfiguration("UUID_CACHE");

ccfg.setIndexedTypes(IgniteUuid.class, String.class);

client.createCache(ccfg);

try(IgniteDataStreamer cache =
client.dataStreamer("UUID_CACHE")) {

for(Integer i=0; i<2; i++)
cache.addData(IgniteUuid.randomUuid().toString(), i.toString());
}
}
```

Server node stack trace [2]:

```
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to update 
index, incorrect key class [expCls=org.apache.ignite.lang.IgniteUuid,
actualCls=org.apache.ignite.internal.binary.BinaryObjectImpl]
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.typeByValue(GridQueryProcessor.java:1954)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.store(GridQueryProcessor.java:1877)
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.store(GridCacheQueryManager.java:403)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.finishUpdate(IgniteCacheOffheapManagerImpl.java:1343)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1207)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:345)
at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.storeValue(GridCacheMapEntry.java:3527)
at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.initialValue(GridCacheMapEntry.java:2735)
at 
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$IsolatedUpdater.receive(DataStreamerImpl.java:2113)
... 11 more
```

[1] https://gist.github.com/nizhikov/2e70a73c7b74a50fc89d270e9af1e1ca

[2] https://gist.github.com/nizhikov/c491c8f2b45aa59458b37b42b4b8dab4

signature.asc
Description: This is a digitally signed message part