[jira] [Updated] (IGNITE-12842) Cache "IGNITE_DAEMON" system properties in GirdKernalContextImpl.isDaemon call

[Sunny Chan (Jira)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-12842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunny Chan updated IGNITE-12842:

Summary: Cache "IGNITE_DAEMON" system properties in 
GirdKernalContextImpl.isDaemon call  (was: Cache "IGNITE_DAEMON" system 
properties in isDaemon call)

> Cache "IGNITE_DAEMON" system properties in GirdKernalContextImpl.isDaemon call
> --
>
> Key: IGNITE-12842
> URL: https://issues.apache.org/jira/browse/IGNITE-12842
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.7.6
>Reporter: Sunny Chan
>Priority: Major
> Attachments: isDaemon.jpg
>
>
> In our performance tuning exercise using JFR, we have observed that when 
> accessing the cache, Ignite will repeatedly called System.getProperty in 
> GridKernalContextImpl.isDaemon()
>   !isDaemon.jpg!
> This is generated with the earlier version of Ignite, and when I examine the 
> latest code it also behaving the same way



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-12842) Cache "IGNITE_DAEMON" system properties in isDaemon call

[Sunny Chan (Jira)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-12842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunny Chan updated IGNITE-12842:

Attachment: isDaemon.jpg

> Cache "IGNITE_DAEMON" system properties in isDaemon call
> 
>
> Key: IGNITE-12842
> URL: https://issues.apache.org/jira/browse/IGNITE-12842
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.7.6
>Reporter: Sunny Chan
>Priority: Major
> Attachments: isDaemon.jpg
>
>
> In our performance tuning exercise using JFR, we have observed that when 
> accessing the cache, Ignite will repeatedly called System.getProperty in 
> GridKernalContextImpl.isDaemon()
>  
> This is generated with the earlier version of Ignite, and when I examine the 
> latest code it also behaving the same way



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-12842) Cache "IGNITE_DAEMON" system properties in isDaemon call

[Sunny Chan (Jira)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-12842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunny Chan updated IGNITE-12842:

Attachment: (was: isDaemon.jpg)

> Cache "IGNITE_DAEMON" system properties in isDaemon call
> 
>
> Key: IGNITE-12842
> URL: https://issues.apache.org/jira/browse/IGNITE-12842
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.7.6
>Reporter: Sunny Chan
>Priority: Major
> Attachments: isDaemon.jpg
>
>
> In our performance tuning exercise using JFR, we have observed that when 
> accessing the cache, Ignite will repeatedly called System.getProperty in 
> GridKernalContextImpl.isDaemon()
>  
> This is generated with the earlier version of Ignite, and when I examine the 
> latest code it also behaving the same way



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-12842) Cache "IGNITE_DAEMON" system properties in isDaemon call

[Sunny Chan (Jira)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-12842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunny Chan updated IGNITE-12842:

Description: 
In our performance tuning exercise using JFR, we have observed that when 
accessing the cache, Ignite will repeatedly called System.getProperty in 
GridKernalContextImpl.isDaemon()

  !isDaemon.jpg!

This is generated with the earlier version of Ignite, and when I examine the 
latest code it also behaving the same way

  was:
In our performance tuning exercise using JFR, we have observed that when 
accessing the cache, Ignite will repeatedly called System.getProperty in 
GridKernalContextImpl.isDaemon()

 

This is generated with the earlier version of Ignite, and when I examine the 
latest code it also behaving the same way


> Cache "IGNITE_DAEMON" system properties in isDaemon call
> 
>
> Key: IGNITE-12842
> URL: https://issues.apache.org/jira/browse/IGNITE-12842
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.7.6
>Reporter: Sunny Chan
>Priority: Major
> Attachments: isDaemon.jpg
>
>
> In our performance tuning exercise using JFR, we have observed that when 
> accessing the cache, Ignite will repeatedly called System.getProperty in 
> GridKernalContextImpl.isDaemon()
>   !isDaemon.jpg!
> This is generated with the earlier version of Ignite, and when I examine the 
> latest code it also behaving the same way



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-12842) Cache "IGNITE_DAEMON" system properties in isDaemon call

[Sunny Chan (Jira)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-12842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunny Chan updated IGNITE-12842:

Description: 
In our performance tuning exercise using JFR, we have observed that when 
accessing the cache, Ignite will repeatedly called System.getProperty in 
GridKernalContextImpl.isDaemon()

 

This is generated with the earlier version of Ignite, and when I examine the 
latest code it also behaving the same way

  was:
In our performance tuning exercise using JFR, we have observed that when 
accessing the cache, Ignite will repeatedly called System.getProperty in 
GridKernalContextImpl.isDaemon()

This is generated with the earlier version of Ignite, and when I examine the 
latest code it also behaving the same way


> Cache "IGNITE_DAEMON" system properties in isDaemon call
> 
>
> Key: IGNITE-12842
> URL: https://issues.apache.org/jira/browse/IGNITE-12842
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.7.6
>Reporter: Sunny Chan
>Priority: Major
> Attachments: isDaemon.jpg
>
>
> In our performance tuning exercise using JFR, we have observed that when 
> accessing the cache, Ignite will repeatedly called System.getProperty in 
> GridKernalContextImpl.isDaemon()
>  
> This is generated with the earlier version of Ignite, and when I examine the 
> latest code it also behaving the same way



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-12842) Cache "IGNITE_DAEMON" system properties in isDaemon call

[Sunny Chan (Jira)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-12842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunny Chan updated IGNITE-12842:

Description: 
In our performance tuning exercise using JFR, we have observed that when 
accessing the cache, Ignite will repeatedly called System.getProperty in 
GridKernalContextImpl.isDaemon()

This is generated with the earlier version of Ignite, and when I examine the 
latest code it also behaving the same way

  was:
In our performance tuning exercise using JFR, we have observed that when 
accessing the cache, Ignite will repeatedly called System.getProperty in 
GridKernalContextImpl.isDaemon()

!isDaemon.jpg! This is generated with the earlier version of Ignite, and when I 
examine the latest code it also behaving the same way


> Cache "IGNITE_DAEMON" system properties in isDaemon call
> 
>
> Key: IGNITE-12842
> URL: https://issues.apache.org/jira/browse/IGNITE-12842
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.7.6
>Reporter: Sunny Chan
>Priority: Major
> Attachments: isDaemon.jpg
>
>
> In our performance tuning exercise using JFR, we have observed that when 
> accessing the cache, Ignite will repeatedly called System.getProperty in 
> GridKernalContextImpl.isDaemon()
> This is generated with the earlier version of Ignite, and when I examine the 
> latest code it also behaving the same way



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-12842) Cache "IGNITE_DAEMON" system properties in isDaemon call

[Sunny Chan (Jira)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-12842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunny Chan updated IGNITE-12842:

Attachment: isDaemon.jpg

> Cache "IGNITE_DAEMON" system properties in isDaemon call
> 
>
> Key: IGNITE-12842
> URL: https://issues.apache.org/jira/browse/IGNITE-12842
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.7.6
>Reporter: Sunny Chan
>Priority: Major
> Attachments: isDaemon.jpg
>
>
> In our performance tuning exercise using JFR, we have observed that when 
> accessing the cache, Ignite will repeatedly called System.getProperty in 
> GridKernalContextImpl.isDaemon()
> !isDaemon.jpg! This is generated with the earlier version of Ignite, and when 
> I examine the latest code it also behaving the same way



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-12842) Cache "IGNITE_DAEMON" system properties in isDaemon call

[Sunny Chan (Jira)]

Sunny Chan created IGNITE-12842:
---

 Summary: Cache "IGNITE_DAEMON" system properties in isDaemon call
 Key: IGNITE-12842
 URL: https://issues.apache.org/jira/browse/IGNITE-12842
 Project: Ignite
  Issue Type: Improvement
  Components: cache
Affects Versions: 2.7.6
Reporter: Sunny Chan


In our performance tuning exercise using JFR, we have observed that when 
accessing the cache, Ignite will repeatedly called System.getProperty in 
GridKernalContextImpl.isDaemon()

!isDaemon.jpg! This is generated with the earlier version of Ignite, and when I 
examine the latest code it also behaving the same way



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-12719) Allow users to configuring Ignite Thread Pool's Core thread count/max thread count/etc

[Sunny Chan (Jira)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-12719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunny Chan updated IGNITE-12719:

Affects Version/s: 2.7.6

> Allow users to configuring Ignite Thread Pool's Core thread count/max thread 
> count/etc
> --
>
> Key: IGNITE-12719
> URL: https://issues.apache.org/jira/browse/IGNITE-12719
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.7.6
>Reporter: Sunny Chan
>Priority: Minor
>
> We are running Ignite cluster on bare metal on a relatively high core count 
> machine (4x10 cores 20 threads), and looking some of the thread pool 
> initialization code: 
> {{(IgnitionEx.java)}}
> {{sysExecSvc = *new* IgniteThreadPoolExecutor(}}{{"sys", 
> }}{{cfg.getIgniteInstanceName(),}}{{ }}
> {{cfg.getSystemThreadPoolSize(),}}{{ }}
> {{cfg.getSystemThreadPoolSize(),}}
> {{*_DFLT_THREAD_KEEP_ALIVE_TIME_*, }}{{*new* 
> LinkedBlockingQueue(),}}
> {{GridIoPolicy.*_SYSTEM_POOL_*);}}
> Notice that the core thread pool size is equals to the max thread pool 
> settings, which is by default same as the number of CPU cores. And in our 
> cases, we won’t be reusing any threads until we have enough request coming in 
> to fill 80 threads. Also, we might want to tune the thread keep alive time to 
> improve thread reuse.
> We would like to propose to change ignite so that users can configure the 
> core thread pool size in these Ignite thread pools.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-12719) Allow users to configuring Ignite Thread Pool's Core thread count/max thread count/etc

[Sunny Chan (Jira)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-12719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunny Chan updated IGNITE-12719:

Priority: Minor  (was: Major)

> Allow users to configuring Ignite Thread Pool's Core thread count/max thread 
> count/etc
> --
>
> Key: IGNITE-12719
> URL: https://issues.apache.org/jira/browse/IGNITE-12719
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sunny Chan
>Priority: Minor
>
> We are running Ignite cluster on bare metal on a relatively high core count 
> machine (4x10 cores 20 threads), and looking some of the thread pool 
> initialization code: 
> {{(IgnitionEx.java)}}
> {{sysExecSvc = *new* IgniteThreadPoolExecutor(}}{{"sys", 
> }}{{cfg.getIgniteInstanceName(),}}{{ }}
> {{cfg.getSystemThreadPoolSize(),}}{{ }}
> {{cfg.getSystemThreadPoolSize(),}}
> {{*_DFLT_THREAD_KEEP_ALIVE_TIME_*, }}{{*new* 
> LinkedBlockingQueue(),}}
> {{GridIoPolicy.*_SYSTEM_POOL_*);}}
> Notice that the core thread pool size is equals to the max thread pool 
> settings, which is by default same as the number of CPU cores. And in our 
> cases, we won’t be reusing any threads until we have enough request coming in 
> to fill 80 threads. Also, we might want to tune the thread keep alive time to 
> improve thread reuse.
> We would like to propose to change ignite so that users can configure the 
> core thread pool size in these Ignite thread pools.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-12719) Allow users to configuring Ignite Thread Pool's Core thread count/max thread count/etc

[Sunny Chan (Jira)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-12719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunny Chan updated IGNITE-12719:

Description: 
We are running Ignite cluster on bare metal on a relatively high core count 
machine (4x10 cores 20 threads), and looking some of the thread pool 
initialization code: 

{{(IgnitionEx.java)}}

{{sysExecSvc = *new* IgniteThreadPoolExecutor(}}{{"sys", 
}}{{cfg.getIgniteInstanceName(),}}{{ }}

{{cfg.getSystemThreadPoolSize(),}}{{ }}

{{cfg.getSystemThreadPoolSize(),}}

{{*_DFLT_THREAD_KEEP_ALIVE_TIME_*, }}{{*new* LinkedBlockingQueue(),}}

{{GridIoPolicy.*_SYSTEM_POOL_*);}}

Notice that the core thread pool size is equals to the max thread pool 
settings, which is by default same as the number of CPU cores. And in our 
cases, we won’t be reusing any threads until we have enough request coming in 
to fill 80 threads. Also, we might want to tune the thread keep alive time to 
improve thread reuse.

We would like to propose to change ignite so that users can configure the core 
thread pool size in these Ignite thread pools.

> Allow users to configuring Ignite Thread Pool's Core thread count/max thread 
> count/etc
> --
>
> Key: IGNITE-12719
> URL: https://issues.apache.org/jira/browse/IGNITE-12719
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sunny Chan
>Priority: Major
>
> We are running Ignite cluster on bare metal on a relatively high core count 
> machine (4x10 cores 20 threads), and looking some of the thread pool 
> initialization code: 
> {{(IgnitionEx.java)}}
> {{sysExecSvc = *new* IgniteThreadPoolExecutor(}}{{"sys", 
> }}{{cfg.getIgniteInstanceName(),}}{{ }}
> {{cfg.getSystemThreadPoolSize(),}}{{ }}
> {{cfg.getSystemThreadPoolSize(),}}
> {{*_DFLT_THREAD_KEEP_ALIVE_TIME_*, }}{{*new* 
> LinkedBlockingQueue(),}}
> {{GridIoPolicy.*_SYSTEM_POOL_*);}}
> Notice that the core thread pool size is equals to the max thread pool 
> settings, which is by default same as the number of CPU cores. And in our 
> cases, we won’t be reusing any threads until we have enough request coming in 
> to fill 80 threads. Also, we might want to tune the thread keep alive time to 
> improve thread reuse.
> We would like to propose to change ignite so that users can configure the 
> core thread pool size in these Ignite thread pools.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (IGNITE-12719) Allow users to configuring Ignite Thread Pool's Core thread count/max thread count/etc

[Sunny Chan (Jira)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-12719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunny Chan updated IGNITE-12719:

Environment: (was: We are running Ignite cluster on bare metal on a 
relatively high core count machine (4x10 cores 20 threads), and looking some of 
the thread pool initialization code: 

{{(IgnitionEx.java)}}

{{sysExecSvc = *new* IgniteThreadPoolExecutor(}}{{                          
"sys", }}{{cfg.getIgniteInstanceName(),}}{{    
cfg.getSystemThreadPoolSize(),}}{{    
cfg.getSystemThreadPoolSize(),}}{{    
*_DFLT_THREAD_KEEP_ALIVE_TIME_*, }}{{*new* LinkedBlockingQueue(),}}{{ 
         GridIoPolicy.*_SYSTEM_POOL_*);}}

Notice that the core thread pool size is equals to the max thread pool 
settings, which is by default same as the number of CPU cores. And in our 
cases, we won’t be reusing any threads until we have enough request coming in 
to fill 80 threads. Also, we might want to tune the thread keep alive time to 
improve thread reuse.

We would like to propose to change ignite so that users can configure the core 
thread pool size in these Ignite thread pools.)

> Allow users to configuring Ignite Thread Pool's Core thread count/max thread 
> count/etc
> --
>
> Key: IGNITE-12719
> URL: https://issues.apache.org/jira/browse/IGNITE-12719
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sunny Chan
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-12719) Allow users to configuring Ignite Thread Pool's Core thread count/max thread count/etc

[Sunny Chan (Jira)]

Sunny Chan created IGNITE-12719:
---

 Summary: Allow users to configuring Ignite Thread Pool's Core 
thread count/max thread count/etc
 Key: IGNITE-12719
 URL: https://issues.apache.org/jira/browse/IGNITE-12719
 Project: Ignite
  Issue Type: Bug
 Environment: We are running Ignite cluster on bare metal on a 
relatively high core count machine (4x10 cores 20 threads), and looking some of 
the thread pool initialization code: 

{{(IgnitionEx.java)}}

{{sysExecSvc = *new* IgniteThreadPoolExecutor(}}{{                          
"sys", }}{{cfg.getIgniteInstanceName(),}}{{    
cfg.getSystemThreadPoolSize(),}}{{    
cfg.getSystemThreadPoolSize(),}}{{    
*_DFLT_THREAD_KEEP_ALIVE_TIME_*, }}{{*new* LinkedBlockingQueue(),}}{{ 
         GridIoPolicy.*_SYSTEM_POOL_*);}}

Notice that the core thread pool size is equals to the max thread pool 
settings, which is by default same as the number of CPU cores. And in our 
cases, we won’t be reusing any threads until we have enough request coming in 
to fill 80 threads. Also, we might want to tune the thread keep alive time to 
improve thread reuse.

We would like to propose to change ignite so that users can configure the core 
thread pool size in these Ignite thread pools.
Reporter: Sunny Chan






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Comment Edited] (IGNITE-12120) Change log level in GridCacheWritebehindStore

[Sunny Chan (Jira)]

[ 
https://issues.apache.org/jira/browse/IGNITE-12120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000670#comment-17000670
 ] 

Sunny Chan edited comment on IGNITE-12120 at 12/20/19 6:49 AM:
---

I have also added an extra patch for CassandraSessionImpl which I believe 
should be a warning rather than an error - this is due to the fact that when 
the exception occurs is throw it will get retried by 
GridCacheWriteBehindStore::updateStore and therefore it is not strictly an 
"error". But I am happy to restore it if we have decided that it should be an 
error.


was (Author: sunnychanclsa):
I have also added an extra patch for CassandraSessionImpl which also seems to 
be a warning rather than an error - this is due to the fact that when the 
exception occurs is throw it will get retried by 
GridCacheWriteBehindStore::updateStore and therefore it is not strictly an 
"error". But I am happy to restore it if we have decided that it should be an 
error.

> Change log level in GridCacheWritebehindStore
> -
>
> Key: IGNITE-12120
> URL: https://issues.apache.org/jira/browse/IGNITE-12120
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 2.7.5
>Reporter: Sunny Chan
>Priority: Trivial
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In the 
> [GridCacheWriteBehindStore|https://github.com/apache/ignite/blob/7e73098d4d6e3d5f78326cb11dac7e083a2312dd/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/store/GridCacheWriteBehindStore.java#L893],
>  when the updateStore failed to write to underlying store, it logs this as 
> error:
> {{LT.error(log, e, "Unable to update underlying store: " + store);}}
> After this line logged the error, it would return false so that it would 
> retry the store (by returning false).  
> While later on in the updatStore function, when the writeCache overflows, it 
> would log this:
> {{log.warning("Failed to update store (value will be lost as current buffer 
> size is greater " + …}}
> then it will remove the failed entry.
> I think the severity of the log messages is not right, as the fail update 
> would still be retried.
> So I propose to change the log severity level so that the first one would be 
> a warn, and second one would be error



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (IGNITE-12120) Change log level in GridCacheWritebehindStore

[Sunny Chan (Jira)]

[ 
https://issues.apache.org/jira/browse/IGNITE-12120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000670#comment-17000670
 ] 

Sunny Chan commented on IGNITE-12120:
-

I have also added an extra patch for CassandraSessionImpl which also seems to 
be a warning rather than an error - this is due to the fact that when the 
exception occurs is throw it will get retried by 
GridCacheWriteBehindStore::updateStore and therefore it is not strictly an 
"error". But I am happy to restore it if we have decided that it should be an 
error.

> Change log level in GridCacheWritebehindStore
> -
>
> Key: IGNITE-12120
> URL: https://issues.apache.org/jira/browse/IGNITE-12120
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 2.7.5
>Reporter: Sunny Chan
>Priority: Trivial
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In the 
> [GridCacheWriteBehindStore|https://github.com/apache/ignite/blob/7e73098d4d6e3d5f78326cb11dac7e083a2312dd/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/store/GridCacheWriteBehindStore.java#L893],
>  when the updateStore failed to write to underlying store, it logs this as 
> error:
> {{LT.error(log, e, "Unable to update underlying store: " + store);}}
> After this line logged the error, it would return false so that it would 
> retry the store (by returning false).  
> While later on in the updatStore function, when the writeCache overflows, it 
> would log this:
> {{log.warning("Failed to update store (value will be lost as current buffer 
> size is greater " + …}}
> then it will remove the failed entry.
> I think the severity of the log messages is not right, as the fail update 
> would still be retried.
> So I propose to change the log severity level so that the first one would be 
> a warn, and second one would be error



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-12120) Change log level in GridCacheWritebehindStore

[Sunny Chan (Jira)]

Sunny Chan created IGNITE-12120:
---

 Summary: Change log level in GridCacheWritebehindStore
 Key: IGNITE-12120
 URL: https://issues.apache.org/jira/browse/IGNITE-12120
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.7.5
Reporter: Sunny Chan


In the 
[GridCacheWriteBehindStore|https://github.com/apache/ignite/blob/7e73098d4d6e3d5f78326cb11dac7e083a2312dd/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/store/GridCacheWriteBehindStore.java#L893],
 when the updateStore failed to write to underlying store, it logs this as 
error:

{{LT.error(log, e, "Unable to update underlying store: " + store);}}

After this line logged the error, it would return false so that it would retry 
the store (by returning false).  

While later on in the updatStore function, when the writeCache overflows, it 
would log this:

{{log.warning("Failed to update store (value will be lost as current buffer 
size is greater " + …}}

then it will remove the failed entry.

I think the severity of the log messages is not right, as the fail update would 
still be retried.

So I propose to change the log severity level so that the first one would be a 
warn, and second one would be error



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Resolved] (IGNITE-7382) Unknown Pair exception if work directory is removed after cluster shutdown

[Sunny Chan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunny Chan resolved IGNITE-7382.

Resolution: Fixed

> Unknown Pair exception if work directory is removed after cluster shutdown
> --
>
> Key: IGNITE-7382
> URL: https://issues.apache.org/jira/browse/IGNITE-7382
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.3
> Environment: RHEL 7.4
> Docker
>Reporter: Sunny Chan
>Priority: Major
>
> To reproduce, try this:
> 1) Start a server node
> 2) Start a node in client mode, connect to server node
> 3) shutdown the server node and _leave the client node running_
> 4) remove Ignite work directory for the shutdown server node
> 5) Client node reconnects automatically, send a service call request
> 6) Exception occurs on Server, unable to deserialize 
> org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2
> {noformat}
> 2018-01-05 00:00:26.027 processors.job.GridJobWorker [svc-#273%clog%] ERROR - 
> Failed to initialize job 
> [jobId=839fef0c061-ad77f485-d677-4694-8d0c-e780f16be9b7, 
> ses=GridJobSessionImpl [ses=GridTaskSessionImpl [taskName=o.a.i.i.processors.
> service.GridServiceProxy$ServiceProxyCallable, dep=LocalDeployment 
> [super=GridDeployment [ts=1515076515280, depMode=SHARED, 
> clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, 
> clsLdrId=3e4f891c061-5235b03b-87c3-4c29-a576-1b44936b0c11, user
> Ver=0, loc=true, sampleClsName=java.lang.String, pendingUndeploy=false, 
> undeployed=false, usage=0]], 
> taskClsName=o.a.i.i.processors.service.GridServiceProxy$ServiceProxyCallable, 
> sesId=739fef0c061-ad77f485-d677-4694-8d0c-e780f16be9b7, st
> artTime=1515078026015, endTime=1515078036021, 
> taskNodeId=ad77f485-d677-4694-8d0c-e780f16be9b7, 
> clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, closed=false, cpSpi=null, 
> failSpi=null, loadSpi=null, usage=1, fullSup=false, internal=false
> , subjId=ad77f485-d677-4694-8d0c-e780f16be9b7, mapFut=IgniteFuture 
> [orig=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, 
> hash=360535376]], execName=null], 
> jobId=839fef0c061-ad77f485-d677-4694-8d0c-e780f16be9b7]]
> org.apache.ignite.IgniteCheckedException: Failed to deserialize object 
> [typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2]
> at 
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9859) 
> [logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.processors.job.GridJobWorker.initialize(GridJobWorker.java:438)
>  [logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1109)
>  [logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1913)
>  [logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
>  [logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
>  [logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
>  [logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
>  [logic.jar:1.1.0]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [?:1.8.0_121]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [?:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to 
> deserialize object 
> [typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2]
> at 
> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
>  ~[logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1762)
>  ~[logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1714)
>  ~[logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:310)
>  ~[logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:99)
>  ~[logic.jar:1.1.0]
> at 
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>  ~[logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9853) 
> 

[jira] [Commented] (IGNITE-5960) Ignite Continuous Query (Queries 3): CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic is flaky

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16507764#comment-16507764
 ] 

Sunny Chan commented on IGNITE-5960:


I have updated the patch:
 # Merge with master to refresh the code
 # The patch now use StripedCompositeReadWriteLock
 # Refactor and put docs around the new code

Please review and let me know if there is anything else I need to change.

> Ignite Continuous Query (Queries 3): 
> CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic
>  is flaky
> -
>
> Key: IGNITE-5960
> URL: https://issues.apache.org/jira/browse/IGNITE-5960
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Sergey Chugunov
>Assignee: Alexey Kuznetsov
>Priority: Major
>  Labels: MakeTeamcityGreenAgain, test-failure
> Fix For: 2.6
>
>
> According to [TC 
> history|http://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=6546112007182082024=testDetails_Ignite20Tests=%3Cdefault%3E]
>  test is flaky.
> It is possible to reproduce it locally, sample run shows 9 failed tests out 
> of 30 overall executed.
> Test fails with jUnit assertion check: 
> {noformat}
> junit.framework.AssertionFailedError: 
> Expected :1
> Actual   :0
>  
>   at junit.framework.Assert.fail(Assert.java:57)
>   at junit.framework.Assert.failNotEquals(Assert.java:329)
>   at junit.framework.Assert.assertEquals(Assert.java:78)
>   at junit.framework.Assert.assertEquals(Assert.java:234)
>   at junit.framework.Assert.assertEquals(Assert.java:241)
>   at junit.framework.TestCase.assertEquals(TestCase.java:409)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.concurrentUpdatesAndQueryStart(CacheContinuousQueryConcurrentPartitionUpdateTest.java:385)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.testConcurrentUpdatesAndQueryStartTx(CacheContinuousQueryConcurrentPartitionUpdateTest.java:245)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at junit.framework.TestCase.runTest(TestCase.java:176)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2000)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:132)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1915)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-5960) Ignite Continuous Query (Queries 3): CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic is flaky

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505892#comment-16505892
 ] 

Sunny Chan commented on IGNITE-5960:


[~agoncharuk] Sure I will get the patch updated and get back to you shortly.

> Ignite Continuous Query (Queries 3): 
> CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic
>  is flaky
> -
>
> Key: IGNITE-5960
> URL: https://issues.apache.org/jira/browse/IGNITE-5960
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Sergey Chugunov
>Assignee: Alexey Kuznetsov
>Priority: Major
>  Labels: MakeTeamcityGreenAgain, test-failure
> Fix For: 2.6
>
>
> According to [TC 
> history|http://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=6546112007182082024=testDetails_Ignite20Tests=%3Cdefault%3E]
>  test is flaky.
> It is possible to reproduce it locally, sample run shows 9 failed tests out 
> of 30 overall executed.
> Test fails with jUnit assertion check: 
> {noformat}
> junit.framework.AssertionFailedError: 
> Expected :1
> Actual   :0
>  
>   at junit.framework.Assert.fail(Assert.java:57)
>   at junit.framework.Assert.failNotEquals(Assert.java:329)
>   at junit.framework.Assert.assertEquals(Assert.java:78)
>   at junit.framework.Assert.assertEquals(Assert.java:234)
>   at junit.framework.Assert.assertEquals(Assert.java:241)
>   at junit.framework.TestCase.assertEquals(TestCase.java:409)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.concurrentUpdatesAndQueryStart(CacheContinuousQueryConcurrentPartitionUpdateTest.java:385)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.testConcurrentUpdatesAndQueryStartTx(CacheContinuousQueryConcurrentPartitionUpdateTest.java:245)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at junit.framework.TestCase.runTest(TestCase.java:176)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2000)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:132)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1915)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-6252) Cassandra Cache Store Session does not retry if prepare statement failed

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333942#comment-16333942
 ] 

Sunny Chan commented on IGNITE-6252:


Okay branch for PR updated: https://github.com/apache/ignite/pull/2583

> Cassandra Cache Store Session does not retry if prepare statement failed
> 
>
> Key: IGNITE-6252
> URL: https://issues.apache.org/jira/browse/IGNITE-6252
> Project: Ignite
>  Issue Type: Bug
>  Components: cassandra
>Affects Versions: 2.0, 2.1
>Reporter: Sunny Chan
>Assignee: Igor Rudyak
>Priority: Major
>
> During our testing, we have found that certain warning about prepared 
> statement:
> 2017-08-31 11:27:19.479 
> org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
> flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
> error detected, refreshing Cassandra session
> com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
> unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have 
> used a PreparedStatement that was created with another Cluster instance.
> We notice that after this warning occurs some of the data didn't persist 
> properly in cassandra cache. After further examining the Ignite's 
> CassandraSessionImpl code in method 
> execute(BatchExecutionAssistance,Iterable), we found that at around [line 
> 283|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L283],
>  if the prepare statement fails in the asnyc call, it will not retry the 
> operation as the error is stored in [line 
> 269|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L269]
>  and cleared in [line 
> 277|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L277]
>  but it was not checked again after going through the [ResultSetFuture 
> |https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L307].
> I believe in line 307 you should check for error != null such that any 
> failure will be retry.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-6252) Cassandra Cache Store Session does not retry if prepare statement failed

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333909#comment-16333909
 ] 

Sunny Chan commented on IGNITE-6252:


[~irudyak] I think that could work too but my knowledge of this component is 
limited so I can't tell whether processedCount == dataSize is too strong a 
statement (e.g. I don't know whether there is a case such that datasize might 
be less due to error processing in other places).

[~ntikhonov] has already reviewed my previous patch and he seems to be okay 
with this - so I wonder whether we can merge it as it is and then consider the 
processedCount() == dataSize check as a future enhancement? 

> Cassandra Cache Store Session does not retry if prepare statement failed
> 
>
> Key: IGNITE-6252
> URL: https://issues.apache.org/jira/browse/IGNITE-6252
> Project: Ignite
>  Issue Type: Bug
>  Components: cassandra
>Affects Versions: 2.0, 2.1
>Reporter: Sunny Chan
>Assignee: Igor Rudyak
>Priority: Major
>
> During our testing, we have found that certain warning about prepared 
> statement:
> 2017-08-31 11:27:19.479 
> org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
> flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
> error detected, refreshing Cassandra session
> com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
> unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have 
> used a PreparedStatement that was created with another Cluster instance.
> We notice that after this warning occurs some of the data didn't persist 
> properly in cassandra cache. After further examining the Ignite's 
> CassandraSessionImpl code in method 
> execute(BatchExecutionAssistance,Iterable), we found that at around [line 
> 283|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L283],
>  if the prepare statement fails in the asnyc call, it will not retry the 
> operation as the error is stored in [line 
> 269|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L269]
>  and cleared in [line 
> 277|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L277]
>  but it was not checked again after going through the [ResultSetFuture 
> |https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L307].
> I believe in line 307 you should check for error != null such that any 
> failure will be retry.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-6252) Cassandra Cache Store Session does not retry if prepare statement failed

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333880#comment-16333880
 ] 

Sunny Chan commented on IGNITE-6252:


[~irudyak]
 # The "Prepare Statement exception" is thrown during executeAsync call, so the 
future for the specific item is never added to the futResults.
 # The handlePreparedStatmenetClusterError(e) does not cause the retry of the 
specific item - only re init the prepared statement

As a result, if we do not check for retry in 
[309|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L309]
 for any error occurs in the for block, the item failed in 1 is never retried.

 

> Cassandra Cache Store Session does not retry if prepare statement failed
> 
>
> Key: IGNITE-6252
> URL: https://issues.apache.org/jira/browse/IGNITE-6252
> Project: Ignite
>  Issue Type: Bug
>  Components: cassandra
>Affects Versions: 2.0, 2.1
>Reporter: Sunny Chan
>Assignee: Igor Rudyak
>Priority: Major
>
> During our testing, we have found that certain warning about prepared 
> statement:
> 2017-08-31 11:27:19.479 
> org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
> flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
> error detected, refreshing Cassandra session
> com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
> unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have 
> used a PreparedStatement that was created with another Cluster instance.
> We notice that after this warning occurs some of the data didn't persist 
> properly in cassandra cache. After further examining the Ignite's 
> CassandraSessionImpl code in method 
> execute(BatchExecutionAssistance,Iterable), we found that at around [line 
> 283|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L283],
>  if the prepare statement fails in the asnyc call, it will not retry the 
> operation as the error is stored in [line 
> 269|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L269]
>  and cleared in [line 
> 277|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L277]
>  but it was not checked again after going through the [ResultSetFuture 
> |https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L307].
> I believe in line 307 you should check for error != null such that any 
> failure will be retry.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (IGNITE-6252) Cassandra Cache Store Session does not retry if prepare statement failed

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333849#comment-16333849
 ] 

Sunny Chan edited comment on IGNITE-6252 at 1/22/18 4:09 AM:
-

[~irudyak] Let me walk you through this:
 # block around 
[231|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L231]
 throws InvalidQueryException. data item is not processed and not added to the 
list of future
 # prepStatEx record the exception in 
[251|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L251]
 # error set to keep the exception throw (line 
[276|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L276])
 # line 
[282|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L282]
 resets the prepStatEx to null
 # line 
[309|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L309]
 didn't check the cached "error", so the error in 231 is ignored and it will 
not retry and return 
[310|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L310]

[~ntikhonov] has previously updated my patch - I have merged his changes along 
with updated unit test provided in [PR 
3088|https://github.com/apache/ignite/pull/3088] into my PR branch


was (Author: sunnychanclsa):
[~irudyak] Let me walk you through this:
 # block around 
[231|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L231]
 throws InvalidQueryException. data item is not processed and not added to the 
list of future
 # prepStatEx record the exception in 
[251|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L251]
 # error set to keep the exception throw (line 
[276|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L276])
 # line 
[282|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L282]
 resets the prepStatEx to null
 # line 
[309|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L309]
 didn't check the cached "error", so the error in 231 is ignored and it will 
not retry and return 
[310|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L310]

[~ntikhonov] has previously updated my patch - I have merged his changes along 
with updated unit test provided in [PR 
3088|https://github.com/apache/ignite/pull/3088]

> Cassandra Cache Store Session does not retry if prepare statement failed
> 
>
> Key: IGNITE-6252
> URL: https://issues.apache.org/jira/browse/IGNITE-6252
> Project: Ignite
>  Issue Type: Bug
>  Components: cassandra
>Affects Versions: 2.0, 2.1
>Reporter: Sunny Chan
>Assignee: Igor Rudyak
>Priority: Major
>
> During our testing, we have found that certain warning about prepared 
> statement:
> 2017-08-31 11:27:19.479 
> org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
> flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
> error detected, refreshing Cassandra session
> com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
> unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have 
> used a PreparedStatement that was created with another Cluster instance.
> We notice that after this warning occurs some of the data didn't persist 
> properly in cassandra cache. 

[jira] [Commented] (IGNITE-6252) Cassandra Cache Store Session does not retry if prepare statement failed

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333849#comment-16333849
 ] 

Sunny Chan commented on IGNITE-6252:


[~irudyak] Let me walk you through this:
 # block around 
[231|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L231]
 throws InvalidQueryException. data item is not processed and not added to the 
list of future
 # prepStatEx record the exception in 
[251|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L251]
 # error set to keep the exception throw (line 
[276|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L276])
 # line 
[282|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L282]
 resets the prepStatEx to null
 # line 
[309|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L309]
 didn't check the cached "error", so the error in 231 is ignored and it will 
not retry and return 
[310|https://github.com/apache/ignite/blob/6330f0bb74d6b986d040a8017f3dfbde361d5457/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L310]

[~ntikhonov] has previously updated my patch - I have merged his changes along 
with updated unit test provided in [PR 
3088|https://github.com/apache/ignite/pull/3088]

> Cassandra Cache Store Session does not retry if prepare statement failed
> 
>
> Key: IGNITE-6252
> URL: https://issues.apache.org/jira/browse/IGNITE-6252
> Project: Ignite
>  Issue Type: Bug
>  Components: cassandra
>Affects Versions: 2.0, 2.1
>Reporter: Sunny Chan
>Assignee: Igor Rudyak
>Priority: Major
>
> During our testing, we have found that certain warning about prepared 
> statement:
> 2017-08-31 11:27:19.479 
> org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
> flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
> error detected, refreshing Cassandra session
> com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
> unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have 
> used a PreparedStatement that was created with another Cluster instance.
> We notice that after this warning occurs some of the data didn't persist 
> properly in cassandra cache. After further examining the Ignite's 
> CassandraSessionImpl code in method 
> execute(BatchExecutionAssistance,Iterable), we found that at around [line 
> 283|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L283],
>  if the prepare statement fails in the asnyc call, it will not retry the 
> operation as the error is stored in [line 
> 269|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L269]
>  and cleared in [line 
> 277|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L277]
>  but it was not checked again after going through the [ResultSetFuture 
> |https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L307].
> I believe in line 307 you should check for error != null such that any 
> failure will be retry.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (IGNITE-6252) Cassandra Cache Store Session does not retry if prepare statement failed

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333772#comment-16333772
 ] 

Sunny Chan edited comment on IGNITE-6252 at 1/22/18 1:45 AM:
-

[~irudyak] Jason's patch only address the fact the prepare statement needs to 
be re-initialized but it did not address the fact that the batch will need to 
retry as well. In fact, the unit test provided by Jason Man there is an mistake 
in there:

{{for (int i = 0; i < 10; i++) {}}
 {{    data.add(String.valueOf(i));}}
 {{}}}
 {{cassandraSession.execute(batchExecutionAssistant, data);}}
 {{}}
 {{assertEquals(9, batchExecutionAssistant.processedCount());}}

Notice that there are 10 items of data, but the test only asserts that there 
are 9 items being processed. The test should verify for 10 items. With my patch 
this should account for all the items.


was (Author: sunnychanclsa):
[~irudyak] Jason's patch only address the fact the prepare statement needs to 
be re-initialized but it did not address the fact that the batch will need to 
retry as well. In fact, the unit test provided by Jason Man there is an mistake 
in there:

{{for (int i = 0; i < 10; i++) {}}
{{    data.add(String.valueOf(i));}}
{{}}}
{{cassandraSession.execute(batchExecutionAssistant, data);}}
{{}}
{{assertEquals(9, batchExecutionAssistant.processedCount());}}

Notice that there are 10 items of data, but the test only asserts that there 
are 9 items being processed. The test should verify for 10 items. With my patch 
this should account for all the items.

> Cassandra Cache Store Session does not retry if prepare statement failed
> 
>
> Key: IGNITE-6252
> URL: https://issues.apache.org/jira/browse/IGNITE-6252
> Project: Ignite
>  Issue Type: Bug
>  Components: cassandra
>Affects Versions: 2.0, 2.1
>Reporter: Sunny Chan
>Assignee: Igor Rudyak
>Priority: Major
>
> During our testing, we have found that certain warning about prepared 
> statement:
> 2017-08-31 11:27:19.479 
> org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
> flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
> error detected, refreshing Cassandra session
> com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
> unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have 
> used a PreparedStatement that was created with another Cluster instance.
> We notice that after this warning occurs some of the data didn't persist 
> properly in cassandra cache. After further examining the Ignite's 
> CassandraSessionImpl code in method 
> execute(BatchExecutionAssistance,Iterable), we found that at around [line 
> 283|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L283],
>  if the prepare statement fails in the asnyc call, it will not retry the 
> operation as the error is stored in [line 
> 269|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L269]
>  and cleared in [line 
> 277|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L277]
>  but it was not checked again after going through the [ResultSetFuture 
> |https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L307].
> I believe in line 307 you should check for error != null such that any 
> failure will be retry.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (IGNITE-6252) Cassandra Cache Store Session does not retry if prepare statement failed

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333772#comment-16333772
 ] 

Sunny Chan edited comment on IGNITE-6252 at 1/22/18 1:45 AM:
-

[~irudyak] Jason's patch only address the fact the prepare statement needs to 
be re-initialized but it did not address the fact that the batch will need to 
retry as well. In fact, the unit test provided by Jason Man there is an mistake 
in there:

{{for (int i = 0; i < 10; i++) {}}
 {{    data.add(String.valueOf( i ));}}
 {{}}}
 {{cassandraSession.execute(batchExecutionAssistant, data);}}
 {{}}
 {{assertEquals(9, batchExecutionAssistant.processedCount());}}

Notice that there are 10 items of data, but the test only asserts that there 
are 9 items being processed. The test should verify for 10 items. With my patch 
this should account for all the items.


was (Author: sunnychanclsa):
[~irudyak] Jason's patch only address the fact the prepare statement needs to 
be re-initialized but it did not address the fact that the batch will need to 
retry as well. In fact, the unit test provided by Jason Man there is an mistake 
in there:

{{for (int i = 0; i < 10; i++) {}}
 {{    data.add(String.valueOf(i));}}
 {{}}}
 {{cassandraSession.execute(batchExecutionAssistant, data);}}
 {{}}
 {{assertEquals(9, batchExecutionAssistant.processedCount());}}

Notice that there are 10 items of data, but the test only asserts that there 
are 9 items being processed. The test should verify for 10 items. With my patch 
this should account for all the items.

> Cassandra Cache Store Session does not retry if prepare statement failed
> 
>
> Key: IGNITE-6252
> URL: https://issues.apache.org/jira/browse/IGNITE-6252
> Project: Ignite
>  Issue Type: Bug
>  Components: cassandra
>Affects Versions: 2.0, 2.1
>Reporter: Sunny Chan
>Assignee: Igor Rudyak
>Priority: Major
>
> During our testing, we have found that certain warning about prepared 
> statement:
> 2017-08-31 11:27:19.479 
> org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
> flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
> error detected, refreshing Cassandra session
> com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
> unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have 
> used a PreparedStatement that was created with another Cluster instance.
> We notice that after this warning occurs some of the data didn't persist 
> properly in cassandra cache. After further examining the Ignite's 
> CassandraSessionImpl code in method 
> execute(BatchExecutionAssistance,Iterable), we found that at around [line 
> 283|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L283],
>  if the prepare statement fails in the asnyc call, it will not retry the 
> operation as the error is stored in [line 
> 269|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L269]
>  and cleared in [line 
> 277|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L277]
>  but it was not checked again after going through the [ResultSetFuture 
> |https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L307].
> I believe in line 307 you should check for error != null such that any 
> failure will be retry.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IGNITE-6252) Cassandra Cache Store Session does not retry if prepare statement failed

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333772#comment-16333772
 ] 

Sunny Chan commented on IGNITE-6252:


[~irudyak] Jason's patch only address the fact the prepare statement needs to 
be re-initialized but it did not address the fact that the batch will need to 
retry as well. In fact, the unit test provided by Jason Man there is an mistake 
in there:

{{for (int i = 0; i < 10; i++) {}}
{{    data.add(String.valueOf(i));}}
{{}}}
{{cassandraSession.execute(batchExecutionAssistant, data);}}
{{}}
{{assertEquals(9, batchExecutionAssistant.processedCount());}}

Notice that there are 10 items of data, but the test only asserts that there 
are 9 items being processed. The test should verify for 10 items. With my patch 
this should account for all the items.

> Cassandra Cache Store Session does not retry if prepare statement failed
> 
>
> Key: IGNITE-6252
> URL: https://issues.apache.org/jira/browse/IGNITE-6252
> Project: Ignite
>  Issue Type: Bug
>  Components: cassandra
>Affects Versions: 2.0, 2.1
>Reporter: Sunny Chan
>Assignee: Igor Rudyak
>Priority: Major
>
> During our testing, we have found that certain warning about prepared 
> statement:
> 2017-08-31 11:27:19.479 
> org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
> flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
> error detected, refreshing Cassandra session
> com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
> unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have 
> used a PreparedStatement that was created with another Cluster instance.
> We notice that after this warning occurs some of the data didn't persist 
> properly in cassandra cache. After further examining the Ignite's 
> CassandraSessionImpl code in method 
> execute(BatchExecutionAssistance,Iterable), we found that at around [line 
> 283|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L283],
>  if the prepare statement fails in the asnyc call, it will not retry the 
> operation as the error is stored in [line 
> 269|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L269]
>  and cleared in [line 
> 277|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L277]
>  but it was not checked again after going through the [ResultSetFuture 
> |https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L307].
> I believe in line 307 you should check for error != null such that any 
> failure will be retry.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7382) Unknown Pair exception if work directory is removed after cluster shutdown

[Sunny Chan (JIRA)]

Sunny Chan created IGNITE-7382:
--

 Summary: Unknown Pair exception if work directory is removed after 
cluster shutdown
 Key: IGNITE-7382
 URL: https://issues.apache.org/jira/browse/IGNITE-7382
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.3
 Environment: Redhat Linux 7.4

Reporter: Sunny Chan


To reproduce, try this:

1) Start a server node
2) Start a node in client mode, connect to server node
3) shutdown the server node and _leave the client node running_
4) remove Ignite work directory for the shutdown server node
5) Client node reconnects automatically, send a service call request
6) Exception occurs on Server, unable to deserialize 
org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2

{noformat}
2018-01-05 00:00:26.027 processors.job.GridJobWorker [svc-#273%clog%] ERROR - 
Failed to initialize job 
[jobId=839fef0c061-ad77f485-d677-4694-8d0c-e780f16be9b7, ses=GridJobSessionImpl 
[ses=GridTaskSessionImpl [taskName=o.a.i.i.processors.
service.GridServiceProxy$ServiceProxyCallable, dep=LocalDeployment 
[super=GridDeployment [ts=1515076515280, depMode=SHARED, 
clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, 
clsLdrId=3e4f891c061-5235b03b-87c3-4c29-a576-1b44936b0c11, user
Ver=0, loc=true, sampleClsName=java.lang.String, pendingUndeploy=false, 
undeployed=false, usage=0]], 
taskClsName=o.a.i.i.processors.service.GridServiceProxy$ServiceProxyCallable, 
sesId=739fef0c061-ad77f485-d677-4694-8d0c-e780f16be9b7, st
artTime=1515078026015, endTime=1515078036021, 
taskNodeId=ad77f485-d677-4694-8d0c-e780f16be9b7, 
clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, closed=false, cpSpi=null, 
failSpi=null, loadSpi=null, usage=1, fullSup=false, internal=false
, subjId=ad77f485-d677-4694-8d0c-e780f16be9b7, mapFut=IgniteFuture 
[orig=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, 
hash=360535376]], execName=null], 
jobId=839fef0c061-ad77f485-d677-4694-8d0c-e780f16be9b7]]
org.apache.ignite.IgniteCheckedException: Failed to deserialize object 
[typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2]
at 
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9859) 
[logic.jar:1.1.0]
at 
org.apache.ignite.internal.processors.job.GridJobWorker.initialize(GridJobWorker.java:438)
 [logic.jar:1.1.0]
at 
org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1109)
 [logic.jar:1.1.0]
at 
org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1913)
 [logic.jar:1.1.0]
at 
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
 [logic.jar:1.1.0]
at 
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
 [logic.jar:1.1.0]
at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
 [logic.jar:1.1.0]
at 
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
 [logic.jar:1.1.0]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[?:1.8.0_121]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[?:1.8.0_121]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to 
deserialize object 
[typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2]
at 
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
 ~[logic.jar:1.1.0]
at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1762)
 ~[logic.jar:1.1.0]
at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1714)
 ~[logic.jar:1.1.0]
at 
org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:310)
 ~[logic.jar:1.1.0]
at 
org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:99)
 ~[logic.jar:1.1.0]
at 
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
 ~[logic.jar:1.1.0]
at 
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9853) 
[logic.jar:1.1.0]
... 10 more
Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to unmarshal 
object with optimized marshaller
at 
org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1786)
 ~[logic.jar:1.1.0]
at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1962)
 ~[logic.jar:1.1.0]
at 

[jira] [Updated] (IGNITE-7382) Unknown Pair exception if work directory is removed after cluster shutdown

[Sunny Chan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-7382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunny Chan updated IGNITE-7382:
---
Environment: 
RHEL 7.4
Docker


  was:
Redhat Linux 7.4



> Unknown Pair exception if work directory is removed after cluster shutdown
> --
>
> Key: IGNITE-7382
> URL: https://issues.apache.org/jira/browse/IGNITE-7382
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.3
> Environment: RHEL 7.4
> Docker
>Reporter: Sunny Chan
>
> To reproduce, try this:
> 1) Start a server node
> 2) Start a node in client mode, connect to server node
> 3) shutdown the server node and _leave the client node running_
> 4) remove Ignite work directory for the shutdown server node
> 5) Client node reconnects automatically, send a service call request
> 6) Exception occurs on Server, unable to deserialize 
> org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2
> {noformat}
> 2018-01-05 00:00:26.027 processors.job.GridJobWorker [svc-#273%clog%] ERROR - 
> Failed to initialize job 
> [jobId=839fef0c061-ad77f485-d677-4694-8d0c-e780f16be9b7, 
> ses=GridJobSessionImpl [ses=GridTaskSessionImpl [taskName=o.a.i.i.processors.
> service.GridServiceProxy$ServiceProxyCallable, dep=LocalDeployment 
> [super=GridDeployment [ts=1515076515280, depMode=SHARED, 
> clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, 
> clsLdrId=3e4f891c061-5235b03b-87c3-4c29-a576-1b44936b0c11, user
> Ver=0, loc=true, sampleClsName=java.lang.String, pendingUndeploy=false, 
> undeployed=false, usage=0]], 
> taskClsName=o.a.i.i.processors.service.GridServiceProxy$ServiceProxyCallable, 
> sesId=739fef0c061-ad77f485-d677-4694-8d0c-e780f16be9b7, st
> artTime=1515078026015, endTime=1515078036021, 
> taskNodeId=ad77f485-d677-4694-8d0c-e780f16be9b7, 
> clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, closed=false, cpSpi=null, 
> failSpi=null, loadSpi=null, usage=1, fullSup=false, internal=false
> , subjId=ad77f485-d677-4694-8d0c-e780f16be9b7, mapFut=IgniteFuture 
> [orig=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, 
> hash=360535376]], execName=null], 
> jobId=839fef0c061-ad77f485-d677-4694-8d0c-e780f16be9b7]]
> org.apache.ignite.IgniteCheckedException: Failed to deserialize object 
> [typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2]
> at 
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9859) 
> [logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.processors.job.GridJobWorker.initialize(GridJobWorker.java:438)
>  [logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1109)
>  [logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1913)
>  [logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
>  [logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
>  [logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
>  [logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
>  [logic.jar:1.1.0]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [?:1.8.0_121]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [?:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
> Caused by: org.apache.ignite.binary.BinaryObjectException: Failed to 
> deserialize object 
> [typeName=org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2]
> at 
> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
>  ~[logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1762)
>  ~[logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1714)
>  ~[logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:310)
>  ~[logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:99)
>  ~[logic.jar:1.1.0]
> at 
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>  ~[logic.jar:1.1.0]
> at 
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9853) 

[jira] [Commented] (IGNITE-7083) Reduce memory usage of CachePartitionFullCountersMap

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279600#comment-16279600
 ] 

Sunny Chan commented on IGNITE-7083:


Pull request: https://github.com/apache/ignite/pull/3152

> Reduce memory usage of CachePartitionFullCountersMap
> 
>
> Key: IGNITE-7083
> URL: https://issues.apache.org/jira/browse/IGNITE-7083
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.3
> Environment: Any
>Reporter: Sunny Chan
>Assignee: Alexey Goncharuk
> Fix For: 2.4
>
>
> The Cache Partition Exchange Manager kept a copy of the already completed 
> exchange. However, we have found that it uses a significant amount of memory. 
> Upon further investigation using heap dump we have found that a large amount 
> of memory is used by the CachePartitionFullCountersMap. We have also observed 
> in most cases, these maps contains only 0s.
> Therefore I propose an optimization for this: Initially the long arrays to 
> store initial update counter and update counter in the CPFCM will be null, 
> and when you get the value and see these tables are null then we will return 
> 0 for the counter. We only allocate the long arrays when there is any 
> non-zero updates to the the map.
> In our tests, the amount of heap used by GridCachePartitionExchangeManager 
> was around 70MB (67 copies of these CPFCM), after we apply the optimization 
> it drops to around 9MB.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-7083) Reduce memory usage of CachePartitionFullCountersMap

[Sunny Chan (JIRA)]

Sunny Chan created IGNITE-7083:
--

 Summary: Reduce memory usage of CachePartitionFullCountersMap
 Key: IGNITE-7083
 URL: https://issues.apache.org/jira/browse/IGNITE-7083
 Project: Ignite
  Issue Type: Improvement
  Components: cache
Affects Versions: 2.3
 Environment: Any
Reporter: Sunny Chan


The Cache Partition Exchange Manager kept a copy of the already completed 
exchange. However, we have found that it uses a significant amount of memory. 
Upon further investigation using heap dump we have found that a large amount of 
memory is used by the CachePartitionFullCountersMap. We have also observed in 
most cases, these maps contains only 0s.

Therefore I propose an optimization for this: Initially the long arrays to 
store initial update counter and update counter in the CPFCM will be null, and 
when you get the value and see these tables are null then we will return 0 for 
the counter. We only allocate the long arrays when there is any non-zero 
updates to the the map.

In our tests, the amount of heap used by GridCachePartitionExchangeManager was 
around 70MB (67 copies of these CPFCM), after we apply the optimization it 
drops to around 9MB.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (IGNITE-5960) Ignite Continuous Query (Queries 3): CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic is flaky

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269874#comment-16269874
 ] 

Sunny Chan edited comment on IGNITE-5960 at 11/29/17 1:24 AM:
--

[~akuznetsov] The CacheContinuousQueryConcurrentPartitionUpdateTest is a redux 
of our problem. From our application prospective we did this:

# Setup a cache with a pair of node
# Setup continuous query, and update entries in the cache that would trigger 
the CQ
# Remove one node forcefully (e.g. kill -9). The thread that update entries 
should continue in the background
# Now restart the killed node and you will find the continuous query no longer 
gets any event even the update continuous.


was (Author: sunnychanclsa):
[~akuznetsov] The CacheContinuousQueryConcurrentPartitionUpdateTest is a redux 
of our problem. From our application prospective we did this:

# Setup a cache with a pair of node
# Setup continuous query, and update entries in the cache that would trigger 
the CQ
# Remove one node forcefully (e.g. kill -9). The entries update to cache should 
continue in the background
# Now restart the killed node and you will find the continuous query no longer 
gets any event even the update continuous.

> Ignite Continuous Query (Queries 3): 
> CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic
>  is flaky
> -
>
> Key: IGNITE-5960
> URL: https://issues.apache.org/jira/browse/IGNITE-5960
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Sergey Chugunov
>Assignee: Alexey Kuznetsov
>  Labels: MakeTeamcityGreenAgain, test-failure
> Fix For: 2.4
>
>
> According to [TC 
> history|http://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=6546112007182082024=testDetails_Ignite20Tests=%3Cdefault%3E]
>  test is flaky.
> It is possible to reproduce it locally, sample run shows 9 failed tests out 
> of 30 overall executed.
> Test fails with jUnit assertion check: 
> {noformat}
> junit.framework.AssertionFailedError: 
> Expected :1
> Actual   :0
>  
>   at junit.framework.Assert.fail(Assert.java:57)
>   at junit.framework.Assert.failNotEquals(Assert.java:329)
>   at junit.framework.Assert.assertEquals(Assert.java:78)
>   at junit.framework.Assert.assertEquals(Assert.java:234)
>   at junit.framework.Assert.assertEquals(Assert.java:241)
>   at junit.framework.TestCase.assertEquals(TestCase.java:409)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.concurrentUpdatesAndQueryStart(CacheContinuousQueryConcurrentPartitionUpdateTest.java:385)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.testConcurrentUpdatesAndQueryStartTx(CacheContinuousQueryConcurrentPartitionUpdateTest.java:245)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at junit.framework.TestCase.runTest(TestCase.java:176)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2000)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:132)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1915)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-5960) Ignite Continuous Query (Queries 3): CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic is flaky

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269874#comment-16269874
 ] 

Sunny Chan commented on IGNITE-5960:


[~akuznetsov] The CacheContinuousQueryConcurrentPartitionUpdateTest is a redux 
of our problem. From our application prospective we did this:

# Setup a cache with a pair of node
# Setup continuous query, and update entries in the cache that would trigger 
the CQ
# Remove one node forcefully (e.g. kill -9). The entries update to cache should 
continue in the background
# Now restart the killed node and you will find the continuous query no longer 
gets any event even the update continuous.

> Ignite Continuous Query (Queries 3): 
> CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic
>  is flaky
> -
>
> Key: IGNITE-5960
> URL: https://issues.apache.org/jira/browse/IGNITE-5960
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Sergey Chugunov
>Assignee: Alexey Kuznetsov
>  Labels: MakeTeamcityGreenAgain, test-failure
> Fix For: 2.4
>
>
> According to [TC 
> history|http://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=6546112007182082024=testDetails_Ignite20Tests=%3Cdefault%3E]
>  test is flaky.
> It is possible to reproduce it locally, sample run shows 9 failed tests out 
> of 30 overall executed.
> Test fails with jUnit assertion check: 
> {noformat}
> junit.framework.AssertionFailedError: 
> Expected :1
> Actual   :0
>  
>   at junit.framework.Assert.fail(Assert.java:57)
>   at junit.framework.Assert.failNotEquals(Assert.java:329)
>   at junit.framework.Assert.assertEquals(Assert.java:78)
>   at junit.framework.Assert.assertEquals(Assert.java:234)
>   at junit.framework.Assert.assertEquals(Assert.java:241)
>   at junit.framework.TestCase.assertEquals(TestCase.java:409)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.concurrentUpdatesAndQueryStart(CacheContinuousQueryConcurrentPartitionUpdateTest.java:385)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.testConcurrentUpdatesAndQueryStartTx(CacheContinuousQueryConcurrentPartitionUpdateTest.java:245)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at junit.framework.TestCase.runTest(TestCase.java:176)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2000)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:132)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1915)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-5960) Ignite Continuous Query (Queries 3): CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic is flaky

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268184#comment-16268184
 ] 

Sunny Chan commented on IGNITE-5960:


Pull request: https://github.com/apache/ignite/pull/3100

> Ignite Continuous Query (Queries 3): 
> CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic
>  is flaky
> -
>
> Key: IGNITE-5960
> URL: https://issues.apache.org/jira/browse/IGNITE-5960
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Sergey Chugunov
>Assignee: Alexey Kuznetsov
>  Labels: MakeTeamcityGreenAgain, test-failure
> Fix For: 2.4
>
>
> According to [TC 
> history|http://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=6546112007182082024=testDetails_Ignite20Tests=%3Cdefault%3E]
>  test is flaky.
> It is possible to reproduce it locally, sample run shows 9 failed tests out 
> of 30 overall executed.
> Test fails with jUnit assertion check: 
> {noformat}
> junit.framework.AssertionFailedError: 
> Expected :1
> Actual   :0
>  
>   at junit.framework.Assert.fail(Assert.java:57)
>   at junit.framework.Assert.failNotEquals(Assert.java:329)
>   at junit.framework.Assert.assertEquals(Assert.java:78)
>   at junit.framework.Assert.assertEquals(Assert.java:234)
>   at junit.framework.Assert.assertEquals(Assert.java:241)
>   at junit.framework.TestCase.assertEquals(TestCase.java:409)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.concurrentUpdatesAndQueryStart(CacheContinuousQueryConcurrentPartitionUpdateTest.java:385)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.testConcurrentUpdatesAndQueryStartTx(CacheContinuousQueryConcurrentPartitionUpdateTest.java:245)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at junit.framework.TestCase.runTest(TestCase.java:176)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2000)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:132)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1915)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-5960) Ignite Continuous Query (Queries 3): CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic is flaky

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268181#comment-16268181
 ] 

Sunny Chan commented on IGNITE-5960:


[~akuznetsov] I have tried your patch but I can still reproduce the issue on my 
machine, using the unit test above and our own application.



> Ignite Continuous Query (Queries 3): 
> CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic
>  is flaky
> -
>
> Key: IGNITE-5960
> URL: https://issues.apache.org/jira/browse/IGNITE-5960
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Sergey Chugunov
>Assignee: Alexey Kuznetsov
>  Labels: MakeTeamcityGreenAgain, test-failure
> Fix For: 2.4
>
>
> According to [TC 
> history|http://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=6546112007182082024=testDetails_Ignite20Tests=%3Cdefault%3E]
>  test is flaky.
> It is possible to reproduce it locally, sample run shows 9 failed tests out 
> of 30 overall executed.
> Test fails with jUnit assertion check: 
> {noformat}
> junit.framework.AssertionFailedError: 
> Expected :1
> Actual   :0
>  
>   at junit.framework.Assert.fail(Assert.java:57)
>   at junit.framework.Assert.failNotEquals(Assert.java:329)
>   at junit.framework.Assert.assertEquals(Assert.java:78)
>   at junit.framework.Assert.assertEquals(Assert.java:234)
>   at junit.framework.Assert.assertEquals(Assert.java:241)
>   at junit.framework.TestCase.assertEquals(TestCase.java:409)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.concurrentUpdatesAndQueryStart(CacheContinuousQueryConcurrentPartitionUpdateTest.java:385)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.testConcurrentUpdatesAndQueryStartTx(CacheContinuousQueryConcurrentPartitionUpdateTest.java:245)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at junit.framework.TestCase.runTest(TestCase.java:176)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2000)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:132)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1915)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-5960) Ignite Continuous Query (Queries 3): CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic is flaky

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16266441#comment-16266441
 ] 

Sunny Chan commented on IGNITE-5960:


This issue has affected our application preventing live restart for our node. I 
have further investigated the issue and tired the propose fix but it didn't 
completely resolved the issue.

Using my own extra debugging statements, I have determine that the missing 
events is down to the fact that if you have lots of concurrent threads updating 
the cache entries, the listener updates could potentially be out of sequence as 
there is no lock, like this:

1) T1 test lsnrs!=null, assume no listener and obtain update counter 1
2) T2 test lsnrs==null, assume no listener and obtain update counter 2
3) T3 test lsnrs!=null, so there is a listener and obtain update counter 3

So we end up with T1, T3 will send update event but T2 won't and this caused a 
gap in the update sequence.

I will propose a fix which introduces a ReadWriteLock in the CCQM, and when we 
update the listener list we will obtain the write lock. Then in 
GridCacheMapEntry before we enter the synchronized block for the GridEntry, we 
obtain the Read lock. This way we ensure that the before the listener is 
updated all update/set cache entry will be completed. I have ran the unit test 
repeatedly using this fix and it seems to pass 100% of the time.

> Ignite Continuous Query (Queries 3): 
> CacheContinuousQueryConcurrentPartitionUpdateTest::testConcurrentUpdatesAndQueryStartAtomic
>  is flaky
> -
>
> Key: IGNITE-5960
> URL: https://issues.apache.org/jira/browse/IGNITE-5960
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.1
>Reporter: Sergey Chugunov
>Assignee: Alexey Kuznetsov
>  Labels: MakeTeamcityGreenAgain, test-failure
> Fix For: 2.4
>
>
> According to [TC 
> history|http://ci.ignite.apache.org/project.html?projectId=Ignite20Tests=6546112007182082024=testDetails_Ignite20Tests=%3Cdefault%3E]
>  test is flaky.
> It is possible to reproduce it locally, sample run shows 9 failed tests out 
> of 30 overall executed.
> Test fails with jUnit assertion check: 
> {noformat}
> junit.framework.AssertionFailedError: 
> Expected :1
> Actual   :0
>  
>   at junit.framework.Assert.fail(Assert.java:57)
>   at junit.framework.Assert.failNotEquals(Assert.java:329)
>   at junit.framework.Assert.assertEquals(Assert.java:78)
>   at junit.framework.Assert.assertEquals(Assert.java:234)
>   at junit.framework.Assert.assertEquals(Assert.java:241)
>   at junit.framework.TestCase.assertEquals(TestCase.java:409)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.concurrentUpdatesAndQueryStart(CacheContinuousQueryConcurrentPartitionUpdateTest.java:385)
>   at 
> org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryConcurrentPartitionUpdateTest.testConcurrentUpdatesAndQueryStartTx(CacheContinuousQueryConcurrentPartitionUpdateTest.java:245)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at junit.framework.TestCase.runTest(TestCase.java:176)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2000)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:132)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1915)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (IGNITE-6252) Cassandra Cache Store Session does not retry if prepare statement failed

[Sunny Chan (JIRA)]

[ 
https://issues.apache.org/jira/browse/IGNITE-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233551#comment-16233551
 ] 

Sunny Chan commented on IGNITE-6252:


It's good thank you.

> Cassandra Cache Store Session does not retry if prepare statement failed
> 
>
> Key: IGNITE-6252
> URL: https://issues.apache.org/jira/browse/IGNITE-6252
> Project: Ignite
>  Issue Type: Bug
>  Components: cassandra
>Affects Versions: 2.0, 2.1
>Reporter: Sunny Chan
>Priority: Major
>
> During our testing, we have found that certain warning about prepared 
> statement:
> 2017-08-31 11:27:19.479 
> org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
> flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
> error detected, refreshing Cassandra session
> com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
> unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have 
> used a PreparedStatement that was created with another Cluster instance.
> We notice that after this warning occurs some of the data didn't persist 
> properly in cassandra cache. After further examining the Ignite's 
> CassandraSessionImpl code in method 
> execute(BatchExecutionAssistance,Iterable), we found that at around [line 
> 283|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L283],
>  if the prepare statement fails in the asnyc call, it will not retry the 
> operation as the error is stored in [line 
> 269|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L269]
>  and cleared in [line 
> 277|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L277]
>  but it was not checked again after going through the [ResultSetFuture 
> |https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L307].
> I believe in line 307 you should check for error != null such that any 
> failure will be retry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Issue Comment Deleted] (IGNITE-6252) Cassandra Cache Store Session does not retry if prepare statement failed

[Sunny Chan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunny Chan updated IGNITE-6252:
---
Comment: was deleted

(was: https://github.com/apache/ignite/pull/2583)

> Cassandra Cache Store Session does not retry if prepare statement failed
> 
>
> Key: IGNITE-6252
> URL: https://issues.apache.org/jira/browse/IGNITE-6252
> Project: Ignite
>  Issue Type: Bug
>  Components: cassandra
>Affects Versions: 2.0, 2.1
>Reporter: Sunny Chan
>
> During our testing, we have found that certain warning about prepared 
> statement:
> 2017-08-31 11:27:19.479 
> org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
> flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
> error detected, refreshing Cassandra session
> com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
> unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have 
> used a PreparedStatement that was created with another Cluster instance.
> We notice that after this warning occurs some of the data didn't persist 
> properly in cassandra cache. After further examining the Ignite's 
> CassandraSessionImpl code in method 
> execute(BatchExecutionAssistance,Iterable), we found that at around [line 
> 283|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L283],
>  if the prepare statement fails in the asnyc call, it will not retry the 
> operation as the error is stored in [line 
> 269|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L269]
>  and cleared in [line 
> 277|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L277]
>  but it was not checked again after going through the [ResultSetFuture 
> |https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L307].
> I believe in line 307 you should check for error != null such that any 
> failure will be retry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (IGNITE-6252) Cassandra Cache Store Session does not retry if prepare statement failed

[Sunny Chan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunny Chan updated IGNITE-6252:
---
Description: 
During our testing, we have found that certain warning about prepared statement:

2017-08-31 11:27:19.479 
org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
error detected, refreshing Cassandra session
com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have used 
a PreparedStatement that was created with another Cluster instance.

We notice that after this warning occurs some of the data didn't persist 
properly in cassandra cache. After further examining the Ignite's 
CassandraSessionImpl code in method execute(BatchExecutionAssistance,Iterable), 
we found that at around [line 
283|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L283],
 if the prepare statement fails in the asnyc call, it will not retry the 
operation as the error is stored in [line 
269|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L269]
 and cleared in [line 
277|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L277]
 but it was not checked again after going through the [ResultSetFuture 
|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L307].

I believe in line 307 you should check for error != null such that any failure 
will be retry.




  was:
During our testing, we have found that certain warning about prepared statement:

{{2017-08-31 11:27:19.479 
org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
error detected, refreshing Cassandra session
com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have used 
a PreparedStatement that was created with another Cluster instance.}}

We notice that after this warning occurs some of the data didn't persist 
properly in cassandra cache. After further examining the Ignite's 
CassandraSessionImpl code in method execute(BatchExecutionAssistance,Iterable), 
we found that at around [line 
283|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L283],
 if the prepare statement fails in the asnyc call, it will not retry the 
operation as the error is stored in [line 
269|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L269]
 and cleared in [line 
277|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L277]
 but it was not checked again after going through the [ResultSetFuture 
|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L307].

I believe in line 307 you should check for error != null such that any failure 
will be retry.





> Cassandra Cache Store Session does not retry if prepare statement failed
> 
>
> Key: IGNITE-6252
> URL: https://issues.apache.org/jira/browse/IGNITE-6252
> Project: Ignite
>  Issue Type: Bug
>  Components: cassandra
>Affects Versions: 2.0, 2.1
>Reporter: Sunny Chan
>
> During our testing, we have found that certain warning about prepared 
> statement:
> 2017-08-31 11:27:19.479 
> org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
> flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
> error detected, refreshing Cassandra session
> com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
> unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have 
> used a PreparedStatement that was created with another Cluster instance.
> We notice that after this warning occurs some of the data 

[jira] [Updated] (IGNITE-6252) Cassandra Cache Store Session does not retry if prepare statement failed

[Sunny Chan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IGNITE-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunny Chan updated IGNITE-6252:
---
Description: 
During our testing, we have found that certain warning about prepared statement:

{{2017-08-31 11:27:19.479 
org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
error detected, refreshing Cassandra session
com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have used 
a PreparedStatement that was created with another Cluster instance.}}

We notice that after this warning occurs some of the data didn't persist 
properly in cassandra cache. After further examining the Ignite's 
CassandraSessionImpl code in method execute(BatchExecutionAssistance,Iterable), 
we found that at around [line 
283|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L283],
 if the prepare statement fails in the asnyc call, it will not retry the 
operation as the error is stored in [line 
269|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L269]
 and cleared in [line 
277|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L277]
 but it was not checked again after going through the [ResultSetFuture 
|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L307].

I believe in line 307 you should check for error != null such that any failure 
will be retry.




  was:
During our testing, we have found that certain warning about prepared statement:

2017-08-31 11:27:19.479 
org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
error detected, refreshing Cassandra session
com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have used 
a PreparedStatement that was created with another Cluster instance.

We notice that after this warning occurs some of the data didn't persist 
properly in cassandra cache. After further examining the Ignite's 
CassandraSessionImpl code in method execute(BatchExecutionAssistance,Iterable), 
we found that at around [line 
283|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L283],
 if the prepare statement fails in the asnyc call, it will not retry the 
operation as the error is stored in [line 
269|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L269]
 and cleared in [line 
277|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L277]
 but it was not checked again after going through the [ResultSetFuture 
|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L307].

I believe in line 307 you should check for error != null such that any failure 
will be retry. Also potentially in line 312 we will need to check 
isTableAbsenceError(error).






> Cassandra Cache Store Session does not retry if prepare statement failed
> 
>
> Key: IGNITE-6252
> URL: https://issues.apache.org/jira/browse/IGNITE-6252
> Project: Ignite
>  Issue Type: Bug
>  Components: cassandra
>Affects Versions: 2.0, 2.1
>Reporter: Sunny Chan
>
> During our testing, we have found that certain warning about prepared 
> statement:
> {{2017-08-31 11:27:19.479 
> org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
> flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
> error detected, refreshing Cassandra session
> com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
> unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have 
> used a PreparedStatement that was created with 

[jira] [Created] (IGNITE-6252) Cassandra Cache Store Session does not retry if prepare statement failed

[Sunny Chan (JIRA)]

Sunny Chan created IGNITE-6252:
--

 Summary: Cassandra Cache Store Session does not retry if prepare 
statement failed
 Key: IGNITE-6252
 URL: https://issues.apache.org/jira/browse/IGNITE-6252
 Project: Ignite
  Issue Type: Bug
  Components: cassandra
Affects Versions: 2.1, 2.0
Reporter: Sunny Chan


During our testing, we have found that certain warning about prepared statement:

2017-08-31 11:27:19.479 
org.apache.ignite.cache.store.cassandra.CassandraCacheStore 
flusher-0-#265%% WARN CassandraCacheStore - Prepared statement cluster 
error detected, refreshing Cassandra session
com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute 
unknown prepared query : 0xc7647611fd755386ef63478ee7de577b. You may have used 
a PreparedStatement that was created with another Cluster instance.

We notice that after this warning occurs some of the data didn't persist 
properly in cassandra cache. After further examining the Ignite's 
CassandraSessionImpl code in method execute(BatchExecutionAssistance,Iterable), 
we found that at around [line 
283|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L283],
 if the prepare statement fails in the asnyc call, it will not retry the 
operation as the error is stored in [line 
269|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L269]
 and cleared in [line 
277|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L277]
 but it was not checked again after going through the [ResultSetFuture 
|https://github.com/apache/ignite/blob/86bd544a557663bce497134f7826be6b24d53330/modules/cassandra/store/src/main/java/org/apache/ignite/cache/store/cassandra/session/CassandraSessionImpl.java#L307].

I believe in line 307 you should check for error != null such that any failure 
will be retry. Also potentially in line 312 we will need to check 
isTableAbsenceError(error).







--
This message was sent by Atlassian JIRA
(v6.4.14#64029)