[jira] [Created] (IGNITE-13477) Fix NPE in SQL tracing implementation.

2020-09-22 Thread Mikhail Petrov (Jira)
Mikhail Petrov created IGNITE-13477:
---

 Summary: Fix NPE in SQL tracing implementation.
 Key: IGNITE-13477
 URL: https://issues.apache.org/jira/browse/IGNITE-13477
 Project: Ignite
  Issue Type: Improvement
Reporter: Mikhail Petrov


MTC.support() can return null. So it's needed to check result of this method 
while in not try/catch block for null. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13186) Commands for control.sh to manage properties at the distributed metastore

2020-09-22 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200598#comment-17200598
 ] 

Ignite TC Bot commented on IGNITE-13186:


{panel:title=Branch: [pull/8208/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/8208/head] Base: [master] : New Tests 
(4)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Control Utility{color} [[tests 
4|https://ci.ignite.apache.org/viewLog.html?buildId=5604379]]
* {color:#013220}IgniteControlUtilityTestSuite: 
GridCommandHandlerPropertiesTest.testPropertyDefaultQueryTimeout - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite: 
GridCommandHandlerPropertiesTest.testPropertyDisabledSqlFunctions - 
PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite: 
GridCommandHandlerPropertiesTest.testList - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite: 
GridCommandHandlerPropertiesTest.testGet - PASSED{color}

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5604252&buildTypeId=IgniteTests24Java8_RunAll]

> Commands for control.sh to manage properties at the distributed metastore
> -
>
> Key: IGNITE-13186
> URL: https://issues.apache.org/jira/browse/IGNITE-13186
> Project: Ignite
>  Issue Type: Improvement
>  Components: control.sh, sql
>Affects Versions: 2.8.1
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Now we store some properties (e.g. all properties of the 
> {{DistributedSqlConfiguration}}) at the distributed metastore.
> Now there is not way to change this properties for user.
> *Proposed fix:*
> adds command for {{CommandHandler}} to manage properties stored at the 
> distibuted metastore. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-12968) Create cluster snapshot documentation pages

2020-09-22 Thread Denis A. Magda (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200488#comment-17200488
 ] 

Denis A. Magda commented on IGNITE-12968:
-

Here you go: https://ignite.apache.org/docs/latest/persistence/snapshots

Just in case, clear your browser caches if you don't see the latest changes. 
There is some issue with the caches invalidation that will be fixed a bit later.

> Create cluster snapshot documentation pages
> ---
>
> Key: IGNITE-12968
> URL: https://issues.apache.org/jira/browse/IGNITE-12968
> Project: Ignite
>  Issue Type: Task
>  Components: documentation
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Blocker
>  Labels: iep-43, important
> Fix For: 2.9
>
>  Time Spent: 7h 40m
>  Remaining Estimate: 0h
>
> Add the following to the Apache Ignite documentation:
> 1. How to create a cluster snapshot (describe API, limitations)
> 2. How to configure a destination directory
> 3. Manual steps for a snapshot restore
> 4. Examples



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Comment Edited] (IGNITE-13465) Ignite cluster falls apart if two nodes segmented sequentially

2020-09-22 Thread Vladimir Steshin (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17199980#comment-17199980
 ] 

Vladimir Steshin edited comment on IGNITE-13465 at 9/22/20, 7:43 PM:
-

The problem is that connectionRecoveryTimeout can be wholly spent on one next 
node. If two fails in a row at the same time, previous nodes may become 
segmented one by one. 

I suggest to slice connectionRecoveryTimeout in order to traverse several next 
nodes in attempt to reconnect to the ring. 
To avoid too small timeouts per one node we should introduce a constant like 
100ms as minimal timeout on attempt to connect to one next node in the ring.



was (Author: vladsz83):
The problem is that connectionRecoveryTimeout can be wholly spent on one next 
node. If two fails at the same time, previous nodes may become segmented one by 
one. 

I suggest to slice connectionRecoveryTimeout in order to traverse several next 
nodes in attempt to reconnect to the ring. 
To avoid too small timeouts per one node I suggest to introduce a constant like 
100ms as minimal timeout on attempt to connect to one next node in the ring.


> Ignite cluster falls apart if two nodes segmented sequentially
> --
>
> Key: IGNITE-13465
> URL: https://issues.apache.org/jira/browse/IGNITE-13465
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.9
>Reporter: Aleksey Plekhanov
>Assignee: Vladimir Steshin
>Priority: Blocker
> Fix For: 2.9
>
> Attachments: GridSequentionNodesFailureTest.java
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> After ticket IGNITE-13134 sequential nodes segmentation leads to segmentation 
> of other nodes in the cluster.
> Reproducer attached.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-12489) Error during purges by expiration: Unknown page type

2020-09-22 Thread Kevin Yan (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200321#comment-17200321
 ] 

Kevin Yan commented on IGNITE-12489:


Is this issue fixed in 2.8.1? Thanks.

> Error during purges by expiration: Unknown page type
> 
>
> Key: IGNITE-12489
> URL: https://issues.apache.org/jira/browse/IGNITE-12489
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.7, 2.7.6
>Reporter: Ruslan Kamashev
>Assignee: Anton Kalashnikov
>Priority: Blocker
> Fix For: 2.10
>
>
> {{*logger*}}
> {code:java}
> org.apache.ignite.internal.processors.cache.GridCacheIoManager
> {code}
> {{*message*}}
> {code:java}
> Failed to process message [senderId=969d56ba-4b46-40cf-886e-ac445cf6a95d, 
> messageType=class 
> o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicUpdateRequest]{code}
> {{*thread*}}
> {code:java}
> sys-stripe-19-#20{code}
> {{*trace*}}
> {code:java}
> java.lang.IllegalStateException: Unknown page type: 1 pageId: 00010303117d
>   at 
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.io(BPlusTree.java:5058)
>   at 
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$200(BPlusTree.java:90)
>   at 
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.nextPage(BPlusTree.java:5330)
>   at 
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.next(BPlusTree.java:5566)
>   at 
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2232)
>   at 
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2157)
>   at 
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:845)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:207)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheUtils.unwindEvicts(GridCacheUtils.java:888)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessageProcessed(GridCacheIoManager.java:1103)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1076)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
>   at 
> org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:505)
>   at 
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
>   at java.lang.Thread.run(Thread.java:748)
>   Dec 23, 2019 @ 18:28:28.457 {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-12451) Introduce deadlock detection for cache entry reentrant locks

2020-09-22 Thread Aleksey Plekhanov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200262#comment-17200262
 ] 

Aleksey Plekhanov commented on IGNITE-12451:


[~maliev], when an old thread can't lock entry, {{hasLockCollisions}} returns 
{{false}} for this thread (since there isn't another older thread exists which 
holds this lock) and an old thread tries to lock this entry again. 
When a new thread can't lock entry, {{hasLockCollisions}} returns {{true}} for 
this thread (since there is another older thread exists which holds this lock) 
and a new thread release all acquired locks and retry the whole operation. 
After locks have been released by a new thread - an old thread can acquire the 
lock it was waited for and proceed further. So, all deadlocks will be 
eventually resolved.  

> Introduce deadlock detection for cache entry reentrant locks
> 
>
> Key: IGNITE-12451
> URL: https://issues.apache.org/jira/browse/IGNITE-12451
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.7.6
>Reporter: Ivan Rakov
>Assignee: Mirza Aliev
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Aside from IGNITE-12365, we still have possible threat of cache-entry-level 
> deadlock in case of careless usage of JCache mass operations (putAll, 
> removeAll):
> 1. If two different user threads will perform putAll on the same two keys in 
> reverse order (primary node for which is the same), there's a chance that 
> sys-stripe threads will be deadlocked.
> 2. Even without direct contract violation from user side, HashMap can be 
> passed as argument for putAll. Even if user threads have called mass 
> operations with two keys in the same order, HashMap iteration order is not 
> strictly defined, which may cause the same deadlock. 
> Local deadlock detection should mitigate this issue. We can create a wrapper 
> for ReentrantLock with logic that performs cycle detection in wait-for graph 
> in case we are waiting for lock acquisition for too long. Exception will be 
> thrown from one of the threads in such case, failing user operation, but 
> letting the system make progress.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Comment Edited] (IGNITE-12451) Introduce deadlock detection for cache entry reentrant locks

2020-09-22 Thread Mirza Aliev (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200206#comment-17200206
 ] 

Mirza Aliev edited comment on IGNITE-12451 at 9/22/20, 4:42 PM:


[~alex_pl]

> Since only younger threads retry all locks

Could you please explain in more detail this phrase? As far as I can 
understand, only _older_ threads retry all locks, because the timeout has 
passed for old threads, while young threads still try to lock entry because 
their timeout has not happened. This means that there is a chance that the 
older thread will be deadlocked again with a new putAll thread that comes at 
the moment when the old thread started to take locks for its entries, and so 
on, this means that the older thread won't do any progress. 


was (Author: maliev):
> Since only younger threads retry all locks

Could you please explain in more detail this phrase? As far as I can 
understand, only _older_ threads retry all locks, because the timeout has 
passed for old threads, while young threads still try to lock entry because 
their timeout has not happened. This means that there is a chance that the 
older thread will be deadlocked again with a new putAll thread that comes at 
the moment when the old thread started to take locks for its entries, and so 
on, this means that the older thread won't do any progress. 

> Introduce deadlock detection for cache entry reentrant locks
> 
>
> Key: IGNITE-12451
> URL: https://issues.apache.org/jira/browse/IGNITE-12451
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.7.6
>Reporter: Ivan Rakov
>Assignee: Mirza Aliev
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Aside from IGNITE-12365, we still have possible threat of cache-entry-level 
> deadlock in case of careless usage of JCache mass operations (putAll, 
> removeAll):
> 1. If two different user threads will perform putAll on the same two keys in 
> reverse order (primary node for which is the same), there's a chance that 
> sys-stripe threads will be deadlocked.
> 2. Even without direct contract violation from user side, HashMap can be 
> passed as argument for putAll. Even if user threads have called mass 
> operations with two keys in the same order, HashMap iteration order is not 
> strictly defined, which may cause the same deadlock. 
> Local deadlock detection should mitigate this issue. We can create a wrapper 
> for ReentrantLock with logic that performs cycle detection in wait-for graph 
> in case we are waiting for lock acquisition for too long. Exception will be 
> thrown from one of the threads in such case, failing user operation, but 
> letting the system make progress.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-12451) Introduce deadlock detection for cache entry reentrant locks

2020-09-22 Thread Mirza Aliev (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200206#comment-17200206
 ] 

Mirza Aliev commented on IGNITE-12451:
--

> Since only younger threads retry all locks

Could you please explain in more detail this phrase? As far as I can 
understand, only _older_ threads retry all locks, because the timeout has 
passed for old threads, while young threads still try to lock entry because 
their timeout has not happened. This means that there is a chance that the 
older thread will be deadlocked again with a new putAll thread that comes at 
the moment when the old thread started to take locks for its entries, and so 
on, this means that the older thread won't do any progress. 

> Introduce deadlock detection for cache entry reentrant locks
> 
>
> Key: IGNITE-12451
> URL: https://issues.apache.org/jira/browse/IGNITE-12451
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.7.6
>Reporter: Ivan Rakov
>Assignee: Mirza Aliev
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Aside from IGNITE-12365, we still have possible threat of cache-entry-level 
> deadlock in case of careless usage of JCache mass operations (putAll, 
> removeAll):
> 1. If two different user threads will perform putAll on the same two keys in 
> reverse order (primary node for which is the same), there's a chance that 
> sys-stripe threads will be deadlocked.
> 2. Even without direct contract violation from user side, HashMap can be 
> passed as argument for putAll. Even if user threads have called mass 
> operations with two keys in the same order, HashMap iteration order is not 
> strictly defined, which may cause the same deadlock. 
> Local deadlock detection should mitigate this issue. We can create a wrapper 
> for ReentrantLock with logic that performs cycle detection in wait-for graph 
> in case we are waiting for lock acquisition for too long. Exception will be 
> thrown from one of the threads in such case, failing user operation, but 
> letting the system make progress.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-7369) .NET: Thin client: Transactions

2020-09-22 Thread Pavel Tupitsyn (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200185#comment-17200185
 ] 

Pavel Tupitsyn commented on IGNITE-7369:


[~GuruStron] looks good to me. Merged to master: 
6633df5d2fa98714dcc377f38f7b8e331928a3ec

Thank you!

> .NET: Thin client: Transactions
> ---
>
> Key: IGNITE-7369
> URL: https://issues.apache.org/jira/browse/IGNITE-7369
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Affects Versions: 2.4
>Reporter: Pavel Tupitsyn
>Assignee: Sergey Stronchinskiy
>Priority: Major
>  Labels: .NET, iep-34
> Fix For: 2.10
>
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> Implement transactions in thin client protocol and .NET thin client.
> Main issue: Ignite transactions are tied to a specific thread.
> See how JDBC works around this by starting a dedicated thread.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Comment Edited] (IGNITE-12451) Introduce deadlock detection for cache entry reentrant locks

2020-09-22 Thread Aleksey Plekhanov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200168#comment-17200168
 ] 

Aleksey Plekhanov edited comment on IGNITE-12451 at 9/22/20, 3:58 PM:
--

[~maliev], {{hasLockCollisions}} gives sometimes false-positive results, but I 
think it's not very often. If we already waited some time for lock most likely 
this thread already in deadlock and most likely thread that holds the lock we 
are trying to acquire blocking us. If not, we just retry and nothing bad 
happens, user operation is not affected by this retry.

Fair cycles searching is more resource consuming (affects performance and 
footprint). This solution doesn't add any footprint and has no performance 
impact, but I see no advantages of fair cycles searching here. 

This algorithm does garantee that system won't hang. Since only younger threads 
retry all locks, older threads don't retry and will always advance in progress 
(there will be no infinite retries).   

Throwing exception to user leads to bad user experience. If we can perform this 
operation without exception, why should we throw it? 



was (Author: alex_pl):
{{hasLockCollisions}} gives sometimes false-positive results, but I think it's 
not very often. If we already waited some time for lock most likely this thread 
already in deadlock and most likely thread that holds the lock we are trying to 
acquire blocking us. If not, we just retry and nothing bad happens, user 
operation is not affected by this retry.

Fair cycles searching is more resource consuming (affects performance and 
footprint). This solution doesn't add any footprint and has no performance 
impact, but I see no advantages of fair cycles searching here. 

This algorithm does garantee that system won't hang. Since only younger threads 
retry all locks, older threads don't retry and will always advance in progress 
(there will be no infinite retries).   

Throwing exception to user leads to bad user experience. If we can perform this 
operation without exception, why should we throw it? 


> Introduce deadlock detection for cache entry reentrant locks
> 
>
> Key: IGNITE-12451
> URL: https://issues.apache.org/jira/browse/IGNITE-12451
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.7.6
>Reporter: Ivan Rakov
>Assignee: Mirza Aliev
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Aside from IGNITE-12365, we still have possible threat of cache-entry-level 
> deadlock in case of careless usage of JCache mass operations (putAll, 
> removeAll):
> 1. If two different user threads will perform putAll on the same two keys in 
> reverse order (primary node for which is the same), there's a chance that 
> sys-stripe threads will be deadlocked.
> 2. Even without direct contract violation from user side, HashMap can be 
> passed as argument for putAll. Even if user threads have called mass 
> operations with two keys in the same order, HashMap iteration order is not 
> strictly defined, which may cause the same deadlock. 
> Local deadlock detection should mitigate this issue. We can create a wrapper 
> for ReentrantLock with logic that performs cycle detection in wait-for graph 
> in case we are waiting for lock acquisition for too long. Exception will be 
> thrown from one of the threads in such case, failing user operation, but 
> letting the system make progress.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-12451) Introduce deadlock detection for cache entry reentrant locks

2020-09-22 Thread Aleksey Plekhanov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200168#comment-17200168
 ] 

Aleksey Plekhanov commented on IGNITE-12451:


{{hasLockCollisions}} gives sometimes false-positive results, but I think it's 
not very often. If we already waited some time for lock most likely this thread 
already in deadlock and most likely thread that holds the lock we are trying to 
acquire blocking us. If not, we just retry and nothing bad happens, user 
operation is not affected by this retry.

Fair cycles searching is more resource consuming (affects performance and 
footprint). This solution doesn't add any footprint and has no performance 
impact, but I see no advantages of fair cycles searching here. 

This algorithm does garantee that system won't hang. Since only younger threads 
retry all locks, older threads don't retry and will always advance in progress 
(there will be no infinite retries).   

Throwing exception to user leads to bad user experience. If we can perform this 
operation without exception, why should we throw it? 


> Introduce deadlock detection for cache entry reentrant locks
> 
>
> Key: IGNITE-12451
> URL: https://issues.apache.org/jira/browse/IGNITE-12451
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.7.6
>Reporter: Ivan Rakov
>Assignee: Mirza Aliev
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Aside from IGNITE-12365, we still have possible threat of cache-entry-level 
> deadlock in case of careless usage of JCache mass operations (putAll, 
> removeAll):
> 1. If two different user threads will perform putAll on the same two keys in 
> reverse order (primary node for which is the same), there's a chance that 
> sys-stripe threads will be deadlocked.
> 2. Even without direct contract violation from user side, HashMap can be 
> passed as argument for putAll. Even if user threads have called mass 
> operations with two keys in the same order, HashMap iteration order is not 
> strictly defined, which may cause the same deadlock. 
> Local deadlock detection should mitigate this issue. We can create a wrapper 
> for ReentrantLock with logic that performs cycle detection in wait-for graph 
> in case we are waiting for lock acquisition for too long. Exception will be 
> thrown from one of the threads in such case, failing user operation, but 
> letting the system make progress.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Comment Edited] (IGNITE-12451) Introduce deadlock detection for cache entry reentrant locks

2020-09-22 Thread Mirza Aliev (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200154#comment-17200154
 ] 

Mirza Aliev edited comment on IGNITE-12451 at 9/22/20, 3:30 PM:


I see that your solution is based on a _tryLock_ idea, we also came to that 
because real-time locks graph analyzing brought a significant performance drop. 

I see that you introduced _hasLockCollisions_ method, but IMHO it gives a lot 
of false-positive results, this method does not search for cycles in locks 
acquisition order. Also, it does not guarantee that system won't hang because 
there is a chance that the retry-lock-acquisition loop also will lead to 
deadlocks. 

So, I believe that the right way to handle deadlock and let the system continue 
to work is the following: we need to throw an exception when _tryLock_ timeouts 
and a user will see _CachePartialUpdateCheckedException_. This solution is 
almost developed in my PR and will be ready in a few days.

 

[~alex_pl] what do you think? 


was (Author: maliev):
I see that your solution is based on a _tryLock_ idea, we also came to that 
because real-time locks graph analyzing brought a significant performance drop. 

I see that you introduced _hasLockCollisions_ method, but IMHO it gives a lot 
of false-positive results, this method does not search for cycles in locks 
acquisition order. Also, it does not guarantee that system won't hang because 
there is a chance that the retry-lock-acquisition loop also will lead to 
deadlocks. 

So, I believe that the right way to handle deadlock and let the system continue 
to work is the following: we need to throw an exception when _tryLock_ timeouts 
and a user will see _CachePartialUpdateCheckedException_. This solution is 
almost developed in my PR and will be ready in a few days.

 

[~alex_pl] what do you think? 

 

 

 

> Introduce deadlock detection for cache entry reentrant locks
> 
>
> Key: IGNITE-12451
> URL: https://issues.apache.org/jira/browse/IGNITE-12451
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.7.6
>Reporter: Ivan Rakov
>Assignee: Mirza Aliev
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Aside from IGNITE-12365, we still have possible threat of cache-entry-level 
> deadlock in case of careless usage of JCache mass operations (putAll, 
> removeAll):
> 1. If two different user threads will perform putAll on the same two keys in 
> reverse order (primary node for which is the same), there's a chance that 
> sys-stripe threads will be deadlocked.
> 2. Even without direct contract violation from user side, HashMap can be 
> passed as argument for putAll. Even if user threads have called mass 
> operations with two keys in the same order, HashMap iteration order is not 
> strictly defined, which may cause the same deadlock. 
> Local deadlock detection should mitigate this issue. We can create a wrapper 
> for ReentrantLock with logic that performs cycle detection in wait-for graph 
> in case we are waiting for lock acquisition for too long. Exception will be 
> thrown from one of the threads in such case, failing user operation, but 
> letting the system make progress.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-12451) Introduce deadlock detection for cache entry reentrant locks

2020-09-22 Thread Mirza Aliev (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200154#comment-17200154
 ] 

Mirza Aliev commented on IGNITE-12451:
--

I see that your solution is based on a _tryLock_ idea, we also came to that 
because real-time locks graph analyzing brought a significant performance drop. 

I see that you introduced _hasLockCollisions_ method, but IMHO it gives a lot 
of false-positive results, this method does not search for cycles in locks 
acquisition order. Also, it does not guarantee that system won't hang because 
there is a chance that the retry-lock-acquisition loop also will lead to 
deadlocks. 

So, I believe that the right way to handle deadlock and let the system continue 
to work is the following: we need to throw an exception when _tryLock_ timeouts 
and a user will see _CachePartialUpdateCheckedException_. __ This solution is 
almost developed in my PR and will be ready in a few days.

 

[~alex_pl] what do you think? 

 

 

 

> Introduce deadlock detection for cache entry reentrant locks
> 
>
> Key: IGNITE-12451
> URL: https://issues.apache.org/jira/browse/IGNITE-12451
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.7.6
>Reporter: Ivan Rakov
>Assignee: Mirza Aliev
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Aside from IGNITE-12365, we still have possible threat of cache-entry-level 
> deadlock in case of careless usage of JCache mass operations (putAll, 
> removeAll):
> 1. If two different user threads will perform putAll on the same two keys in 
> reverse order (primary node for which is the same), there's a chance that 
> sys-stripe threads will be deadlocked.
> 2. Even without direct contract violation from user side, HashMap can be 
> passed as argument for putAll. Even if user threads have called mass 
> operations with two keys in the same order, HashMap iteration order is not 
> strictly defined, which may cause the same deadlock. 
> Local deadlock detection should mitigate this issue. We can create a wrapper 
> for ReentrantLock with logic that performs cycle detection in wait-for graph 
> in case we are waiting for lock acquisition for too long. Exception will be 
> thrown from one of the threads in such case, failing user operation, but 
> letting the system make progress.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Comment Edited] (IGNITE-12451) Introduce deadlock detection for cache entry reentrant locks

2020-09-22 Thread Mirza Aliev (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200154#comment-17200154
 ] 

Mirza Aliev edited comment on IGNITE-12451 at 9/22/20, 3:29 PM:


I see that your solution is based on a _tryLock_ idea, we also came to that 
because real-time locks graph analyzing brought a significant performance drop. 

I see that you introduced _hasLockCollisions_ method, but IMHO it gives a lot 
of false-positive results, this method does not search for cycles in locks 
acquisition order. Also, it does not guarantee that system won't hang because 
there is a chance that the retry-lock-acquisition loop also will lead to 
deadlocks. 

So, I believe that the right way to handle deadlock and let the system continue 
to work is the following: we need to throw an exception when _tryLock_ timeouts 
and a user will see _CachePartialUpdateCheckedException_. This solution is 
almost developed in my PR and will be ready in a few days.

 

[~alex_pl] what do you think? 

 

 

 


was (Author: maliev):
I see that your solution is based on a _tryLock_ idea, we also came to that 
because real-time locks graph analyzing brought a significant performance drop. 

I see that you introduced _hasLockCollisions_ method, but IMHO it gives a lot 
of false-positive results, this method does not search for cycles in locks 
acquisition order. Also, it does not guarantee that system won't hang because 
there is a chance that the retry-lock-acquisition loop also will lead to 
deadlocks. 

So, I believe that the right way to handle deadlock and let the system continue 
to work is the following: we need to throw an exception when _tryLock_ timeouts 
and a user will see _CachePartialUpdateCheckedException_. __ This solution is 
almost developed in my PR and will be ready in a few days.

 

[~alex_pl] what do you think? 

 

 

 

> Introduce deadlock detection for cache entry reentrant locks
> 
>
> Key: IGNITE-12451
> URL: https://issues.apache.org/jira/browse/IGNITE-12451
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.7.6
>Reporter: Ivan Rakov
>Assignee: Mirza Aliev
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Aside from IGNITE-12365, we still have possible threat of cache-entry-level 
> deadlock in case of careless usage of JCache mass operations (putAll, 
> removeAll):
> 1. If two different user threads will perform putAll on the same two keys in 
> reverse order (primary node for which is the same), there's a chance that 
> sys-stripe threads will be deadlocked.
> 2. Even without direct contract violation from user side, HashMap can be 
> passed as argument for putAll. Even if user threads have called mass 
> operations with two keys in the same order, HashMap iteration order is not 
> strictly defined, which may cause the same deadlock. 
> Local deadlock detection should mitigate this issue. We can create a wrapper 
> for ReentrantLock with logic that performs cycle detection in wait-for graph 
> in case we are waiting for lock acquisition for too long. Exception will be 
> thrown from one of the threads in such case, failing user operation, but 
> letting the system make progress.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (IGNITE-640) Implement IgniteMultimap data structures

2020-09-22 Thread Pavel Pereslegin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Pereslegin reassigned IGNITE-640:
---

Assignee: Pavel Pereslegin

> Implement IgniteMultimap data structures
> 
>
> Key: IGNITE-640
> URL: https://issues.apache.org/jira/browse/IGNITE-640
> Project: Ignite
>  Issue Type: Sub-task
>  Components: data structures
>Reporter: Dmitriy Setrakyan
>Assignee: Pavel Pereslegin
>Priority: Major
>
> We need to add {{IgniteMultimap}} data structure in addition to other data 
> structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
> {{java.util.Map}} class in JDK, but support the semantics of multiple values 
> per key, similar to [Guava 
> Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
>  
> However, unlike in Guava, our multi-map should work with Lists, not 
> Collections. Lists should make it possible to support the following methods:
> {code}
> // Gets value at a certain index for a key.
> V get(K, index);
> // Gets all values for a collection of keys at a certain index.
> Map getAll(Collection, index);
> // Gets values for specified indexes for a key.
> List get(K, Iterable indexes);
> // Gets all values for a collection of keys at specified indexes.
> Map> getAll(Collection, Iterable indexes);
> // Gets values for specified range of indexes, between min and max.
> List get(K, int min, int max);
> // Gets all values for a collection of keys for a specified index range, 
> between min and max.
> Map> getAll(Collection, int min, int max);
> // Gets all values for a specific key.
> List get(K);
> // Gets all values for a collection of keys.
> Map> getAll(Collection);
> // Iterate through all elements with a certain index.
> Iterator> iterate(int idx);
> // Do we need this?
> Collection> get(K, IgniteBiPredicate)
> {code}
> Multimap should also support colocated and non-colocated modes, similar to 
> [IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
>  and its implementation, 
> [GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].
> h2. Design Details
> The most natural way to implement such map, would be to store every value 
> under a separate key in an Ignite cache. For example, let's say that we have 
> a key {{K}} with multiple values: {{V0, V1, V2, ...}}. Then the cache should 
> end up with the following values {{K0, V0}}, {{K1, V1}}, {{K2, V2}}, etc. 
> This means that we need to wrap user key into our own, internal key, which 
> will also have {{index}} field. 
> Also note that we need to collocate all the values for the same key on the 
> same node, which means that we need to define user key K as the affinity key, 
> like so:
> {code}
> class MultiKey {
> @CacheAffinityMapped
> private K key;
> int index;
> }
> {code}
> Look ups of values at specific indexes becomes very simple. Just attach a 
> specific index to a key and do a cache lookup. Look ups for all values for a 
> key should work as following:
> {code}
> MultiKey key;
> V v = null;
> int index = 0;
> List res = new LinkedList<>();
> do {
> v = cache.get(MultiKey(K, index));
> if (v != null)
> res.add(v);
> index++;
> }
> while (v != null);
> return res;
> {code}
> We could also use batching for performance reason. In this case the batch 
> size should be configurable.
> {code}
> int index = 0;
> List res = new LinkedList<>();
> while (true) {
> List batch = new ArrayList<>(batchSize);
> // Populate batch.
> for (; index < batchSize; index++)
> batch.add(new MultiKey(K, index % batchSize);
> Map batchRes = cache.getAll(batch);
> // Potentially need to properly sort values, based on the key order,
> // if the returning map does not do it automatically.
> res.addAll(batchRes.values());
> if (res.size() < batch.size())
> break;
> }
> return res;
> {code}
> h2. Evictions
> Evictions in the {{IgniteMultiMap}} should have 2 levels: maximum number of 
> keys, and maximum number of values for a key. The maximum number of keys 
> should be controlled by Ignite standard eviction policy. The maximum number 
> of values for a key should be controlled by the implementation of the 
> multi-map. Either eviction parameter should be configurable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (IGNITE-640) Implement IgniteMultimap data structures

2020-09-22 Thread Pavel Pereslegin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Pereslegin reassigned IGNITE-640:
---

Assignee: (was: Pavel Pereslegin)

> Implement IgniteMultimap data structures
> 
>
> Key: IGNITE-640
> URL: https://issues.apache.org/jira/browse/IGNITE-640
> Project: Ignite
>  Issue Type: Sub-task
>  Components: data structures
>Reporter: Dmitriy Setrakyan
>Priority: Major
>
> We need to add {{IgniteMultimap}} data structure in addition to other data 
> structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
> {{java.util.Map}} class in JDK, but support the semantics of multiple values 
> per key, similar to [Guava 
> Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
>  
> However, unlike in Guava, our multi-map should work with Lists, not 
> Collections. Lists should make it possible to support the following methods:
> {code}
> // Gets value at a certain index for a key.
> V get(K, index);
> // Gets all values for a collection of keys at a certain index.
> Map getAll(Collection, index);
> // Gets values for specified indexes for a key.
> List get(K, Iterable indexes);
> // Gets all values for a collection of keys at specified indexes.
> Map> getAll(Collection, Iterable indexes);
> // Gets values for specified range of indexes, between min and max.
> List get(K, int min, int max);
> // Gets all values for a collection of keys for a specified index range, 
> between min and max.
> Map> getAll(Collection, int min, int max);
> // Gets all values for a specific key.
> List get(K);
> // Gets all values for a collection of keys.
> Map> getAll(Collection);
> // Iterate through all elements with a certain index.
> Iterator> iterate(int idx);
> // Do we need this?
> Collection> get(K, IgniteBiPredicate)
> {code}
> Multimap should also support colocated and non-colocated modes, similar to 
> [IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
>  and its implementation, 
> [GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].
> h2. Design Details
> The most natural way to implement such map, would be to store every value 
> under a separate key in an Ignite cache. For example, let's say that we have 
> a key {{K}} with multiple values: {{V0, V1, V2, ...}}. Then the cache should 
> end up with the following values {{K0, V0}}, {{K1, V1}}, {{K2, V2}}, etc. 
> This means that we need to wrap user key into our own, internal key, which 
> will also have {{index}} field. 
> Also note that we need to collocate all the values for the same key on the 
> same node, which means that we need to define user key K as the affinity key, 
> like so:
> {code}
> class MultiKey {
> @CacheAffinityMapped
> private K key;
> int index;
> }
> {code}
> Look ups of values at specific indexes becomes very simple. Just attach a 
> specific index to a key and do a cache lookup. Look ups for all values for a 
> key should work as following:
> {code}
> MultiKey key;
> V v = null;
> int index = 0;
> List res = new LinkedList<>();
> do {
> v = cache.get(MultiKey(K, index));
> if (v != null)
> res.add(v);
> index++;
> }
> while (v != null);
> return res;
> {code}
> We could also use batching for performance reason. In this case the batch 
> size should be configurable.
> {code}
> int index = 0;
> List res = new LinkedList<>();
> while (true) {
> List batch = new ArrayList<>(batchSize);
> // Populate batch.
> for (; index < batchSize; index++)
> batch.add(new MultiKey(K, index % batchSize);
> Map batchRes = cache.getAll(batch);
> // Potentially need to properly sort values, based on the key order,
> // if the returning map does not do it automatically.
> res.addAll(batchRes.values());
> if (res.size() < batch.size())
> break;
> }
> return res;
> {code}
> h2. Evictions
> Evictions in the {{IgniteMultiMap}} should have 2 levels: maximum number of 
> keys, and maximum number of values for a key. The maximum number of keys 
> should be controlled by Ignite standard eviction policy. The maximum number 
> of values for a key should be controlled by the implementation of the 
> multi-map. Either eviction parameter should be configurable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-12451) Introduce deadlock detection for cache entry reentrant locks

2020-09-22 Thread Aleksey Plekhanov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200107#comment-17200107
 ] 

Aleksey Plekhanov commented on IGNITE-12451:


[~maliev], we've faced this issue and I want to fix it fast.
Actually I have a simple working prototype that affects only putAll operation 
(single-entry operation not affected), without performance impact (I hope): 
https://github.com/apache/ignite/pull/8268 . 

> Introduce deadlock detection for cache entry reentrant locks
> 
>
> Key: IGNITE-12451
> URL: https://issues.apache.org/jira/browse/IGNITE-12451
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.7.6
>Reporter: Ivan Rakov
>Assignee: Mirza Aliev
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Aside from IGNITE-12365, we still have possible threat of cache-entry-level 
> deadlock in case of careless usage of JCache mass operations (putAll, 
> removeAll):
> 1. If two different user threads will perform putAll on the same two keys in 
> reverse order (primary node for which is the same), there's a chance that 
> sys-stripe threads will be deadlocked.
> 2. Even without direct contract violation from user side, HashMap can be 
> passed as argument for putAll. Even if user threads have called mass 
> operations with two keys in the same order, HashMap iteration order is not 
> strictly defined, which may cause the same deadlock. 
> Local deadlock detection should mitigate this issue. We can create a wrapper 
> for ReentrantLock with logic that performs cycle detection in wait-for graph 
> in case we are waiting for lock acquisition for too long. Exception will be 
> thrown from one of the threads in such case, failing user operation, but 
> letting the system make progress.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Comment Edited] (IGNITE-13174) C++: Add Windows support to CMake build system

2020-09-22 Thread Ivan Daschinskiy (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200096#comment-17200096
 ] 

Ivan Daschinskiy edited comment on IGNITE-13174 at 9/22/20, 1:58 PM:
-

[~isapego] Good news! I see, that there is some problems with ODBC tests in 
debug configuration, but in release everything is ok. I suppose, that debug 
configuration is a littlbe bit outdated in TC. 

PR with your changes looks good to me!




was (Author: ivandasch):
[~isapego] Good news! I see, that there is some problems with ODBC tests in 
debug configuration, but in release everything is ok. I suppose, that debug 
configuration is a littlbe bit outdated in TC. 



> C++: Add Windows support to CMake build system
> --
>
> Key: IGNITE-13174
> URL: https://issues.apache.org/jira/browse/IGNITE-13174
> Project: Ignite
>  Issue Type: Bug
>  Components: platforms
>Affects Versions: 2.8.1
>Reporter: Igor Sapego
>Assignee: Igor Sapego
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Ticket IGNITE-13078 adds CMake build system support, but only for Linux. Need 
> make sure it works on Windows and create TC job for it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13174) C++: Add Windows support to CMake build system

2020-09-22 Thread Ivan Daschinskiy (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200096#comment-17200096
 ] 

Ivan Daschinskiy commented on IGNITE-13174:
---

[~isapego] Good news! I see, that there is some problems with ODBC tests in 
debug configuration, but in release everything is ok. I suppose, that debug 
configuration is a littlbe bit outdated in TC. 



> C++: Add Windows support to CMake build system
> --
>
> Key: IGNITE-13174
> URL: https://issues.apache.org/jira/browse/IGNITE-13174
> Project: Ignite
>  Issue Type: Bug
>  Components: platforms
>Affects Versions: 2.8.1
>Reporter: Igor Sapego
>Assignee: Igor Sapego
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Ticket IGNITE-13078 adds CMake build system support, but only for Linux. Need 
> make sure it works on Windows and create TC job for it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-12451) Introduce deadlock detection for cache entry reentrant locks

2020-09-22 Thread Mirza Aliev (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200051#comment-17200051
 ] 

Mirza Aliev commented on IGNITE-12451:
--

[~alex_pl] yes, currently I'm working on the ticket, in a few days I'll prepare 
PR 

> Introduce deadlock detection for cache entry reentrant locks
> 
>
> Key: IGNITE-12451
> URL: https://issues.apache.org/jira/browse/IGNITE-12451
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.7.6
>Reporter: Ivan Rakov
>Assignee: Mirza Aliev
>Priority: Major
> Fix For: 2.10
>
>
> Aside from IGNITE-12365, we still have possible threat of cache-entry-level 
> deadlock in case of careless usage of JCache mass operations (putAll, 
> removeAll):
> 1. If two different user threads will perform putAll on the same two keys in 
> reverse order (primary node for which is the same), there's a chance that 
> sys-stripe threads will be deadlocked.
> 2. Even without direct contract violation from user side, HashMap can be 
> passed as argument for putAll. Even if user threads have called mass 
> operations with two keys in the same order, HashMap iteration order is not 
> strictly defined, which may cause the same deadlock. 
> Local deadlock detection should mitigate this issue. We can create a wrapper 
> for ReentrantLock with logic that performs cycle detection in wait-for graph 
> in case we are waiting for lock acquisition for too long. Exception will be 
> thrown from one of the threads in such case, failing user operation, but 
> letting the system make progress.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-12451) Introduce deadlock detection for cache entry reentrant locks

2020-09-22 Thread Aleksey Plekhanov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200046#comment-17200046
 ] 

Aleksey Plekhanov commented on IGNITE-12451:


[~maliev], are you working on this ticket? Can I take it?

> Introduce deadlock detection for cache entry reentrant locks
> 
>
> Key: IGNITE-12451
> URL: https://issues.apache.org/jira/browse/IGNITE-12451
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.7.6
>Reporter: Ivan Rakov
>Assignee: Mirza Aliev
>Priority: Major
> Fix For: 2.10
>
>
> Aside from IGNITE-12365, we still have possible threat of cache-entry-level 
> deadlock in case of careless usage of JCache mass operations (putAll, 
> removeAll):
> 1. If two different user threads will perform putAll on the same two keys in 
> reverse order (primary node for which is the same), there's a chance that 
> sys-stripe threads will be deadlocked.
> 2. Even without direct contract violation from user side, HashMap can be 
> passed as argument for putAll. Even if user threads have called mass 
> operations with two keys in the same order, HashMap iteration order is not 
> strictly defined, which may cause the same deadlock. 
> Local deadlock detection should mitigate this issue. We can create a wrapper 
> for ReentrantLock with logic that performs cycle detection in wait-for graph 
> in case we are waiting for lock acquisition for too long. Exception will be 
> thrown from one of the threads in such case, failing user operation, but 
> letting the system make progress.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-12968) Create cluster snapshot documentation pages

2020-09-22 Thread Maxim Muzafarov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200044#comment-17200044
 ] 

Maxim Muzafarov commented on IGNITE-12968:
--

[~dmagda]

I've fixed the comments as you suggested. The snapshot directory structure has 
been also updated since {{binary}} and {{marshaller}} directories moved under 
{{db}} directory.

Can we update the page on the website?

> Create cluster snapshot documentation pages
> ---
>
> Key: IGNITE-12968
> URL: https://issues.apache.org/jira/browse/IGNITE-12968
> Project: Ignite
>  Issue Type: Task
>  Components: documentation
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Blocker
>  Labels: iep-43, important
> Fix For: 2.9
>
>  Time Spent: 7h 40m
>  Remaining Estimate: 0h
>
> Add the following to the Apache Ignite documentation:
> 1. How to create a cluster snapshot (describe API, limitations)
> 2. How to configure a destination directory
> 3. Manual steps for a snapshot restore
> 4. Examples



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13476) Calcite improvements. Implement ProjectionNode.

2020-09-22 Thread Stanilovsky Evgeny (Jira)
Stanilovsky Evgeny created IGNITE-13476:
---

 Summary: Calcite improvements. Implement ProjectionNode.
 Key: IGNITE-13476
 URL: https://issues.apache.org/jira/browse/IGNITE-13476
 Project: Ignite
  Issue Type: Improvement
  Components: sql
Affects Versions: 2.8.1
Reporter: Stanilovsky Evgeny
Assignee: Stanilovsky Evgeny


Currently we have no functionality for filtering only useful columns in inner 
representations. For further heap allocations reduction and as a consequence: 
gc pressure, we need to implement such approach.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Comment Edited] (IGNITE-13006) Apache Ignite spring libs upgrade from version 4x to spring 5.2 version or later..

2020-09-22 Thread Hemambara (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200019#comment-17200019
 ] 

Hemambara edited comment on IGNITE-13006 at 9/22/20, 11:44 AM:
---

I am planning to pick this story and work on it. Will initially come up with 
analysis. Please let me know for any issues with picking it now and resolve it 
for 2.10.

I do not see Ignite 2.10 branch in GitHub. Do we have plan to create the branch 
2.10 ? or we are not there yet 


was (Author: kotari):
I am planning to pick this story and work on it. Will initially come up with 
analysis. Please let me know for any issues with picking it now and resolve it 
for 2.10

> Apache Ignite spring libs upgrade from version 4x to spring 5.2 version or 
> later..
> --
>
> Key: IGNITE-13006
> URL: https://issues.apache.org/jira/browse/IGNITE-13006
> Project: Ignite
>  Issue Type: Improvement
>  Components: spring
>Affects Versions: 2.8
> Environment: We are trying this on linux env.
>Reporter: Devendra Jain
>Priority: Blocker
> Fix For: 2.10
>
>
> With Apache Ignite Release 2.8, Spring libs still using spring older version 
> 4.3x. Do you have a plan to upgrade to Spring libs with 5.2x version or 
> latter by end of this year.
> Since we are looking to use Apache ignite but due to some policy issue we 
> can't be able to use Ignite 2.8 version which is still using older spring 
> libarary version 4.3x.
> Please let us know as soon as possible. How you are planning to address this 
> issue and what is the time new release including this fix.
>  
> Thanks,
> Devendra Jain
> deven...@gmail.com



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13006) Apache Ignite spring libs upgrade from version 4x to spring 5.2 version or later..

2020-09-22 Thread Hemambara (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200019#comment-17200019
 ] 

Hemambara commented on IGNITE-13006:


I am planning to pick this story and work on it. Will initially come up with 
analysis. Please let me know for any issues with picking it now and resolve it 
for 2.10

> Apache Ignite spring libs upgrade from version 4x to spring 5.2 version or 
> later..
> --
>
> Key: IGNITE-13006
> URL: https://issues.apache.org/jira/browse/IGNITE-13006
> Project: Ignite
>  Issue Type: Improvement
>  Components: spring
>Affects Versions: 2.8
> Environment: We are trying this on linux env.
>Reporter: Devendra Jain
>Priority: Blocker
> Fix For: 2.10
>
>
> With Apache Ignite Release 2.8, Spring libs still using spring older version 
> 4.3x. Do you have a plan to upgrade to Spring libs with 5.2x version or 
> latter by end of this year.
> Since we are looking to use Apache ignite but due to some policy issue we 
> can't be able to use Ignite 2.8 version which is still using older spring 
> libarary version 4.3x.
> Please let us know as soon as possible. How you are planning to address this 
> issue and what is the time new release including this fix.
>  
> Thanks,
> Devendra Jain
> deven...@gmail.com



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13174) C++: Add Windows support to CMake build system

2020-09-22 Thread Igor Sapego (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1729#comment-1729
 ] 

Igor Sapego commented on IGNITE-13174:
--

[~ivandasch] everything seems ready. Please, review suites and changes in PR 
(i've created a new one, as I can not modify yours).

> C++: Add Windows support to CMake build system
> --
>
> Key: IGNITE-13174
> URL: https://issues.apache.org/jira/browse/IGNITE-13174
> Project: Ignite
>  Issue Type: Bug
>  Components: platforms
>Affects Versions: 2.8.1
>Reporter: Igor Sapego
>Assignee: Igor Sapego
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Ticket IGNITE-13078 adds CMake build system support, but only for Linux. Need 
> make sure it works on Windows and create TC job for it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Comment Edited] (IGNITE-13465) Ignite cluster falls apart if two nodes segmented sequentially

2020-09-22 Thread Vladimir Steshin (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17199980#comment-17199980
 ] 

Vladimir Steshin edited comment on IGNITE-13465 at 9/22/20, 11:24 AM:
--

The problem is that connectionRecoveryTimeout can be wholly spent on one next 
node. If two fails at the same time, previous nodes may become segmented one by 
one. 

I suggest to slice connectionRecoveryTimeout in order to traverse several next 
nodes in attempt to reconnect to the ring. 
To avoid too small timeouts per one node I suggest to introduce a constant like 
100ms as minimal timeout on attempt to connect to one next node in the ring.



was (Author: vladsz83):
The problem is that connectionRecoveryTimeout can be wholly spent on one next 
node. If two fails at the same time, previous nodes may become segmented one by 
one. 

I suggest to slice connectionRecoveryTimeout in order to traverse several next 
nodes in attempt to reconnect to the ring. We should consider maximum 
reasonable nodes number to reconnect to as `servers/2 + 1`. If we cannot 
connect to half of the ring, this can be considered as major malfunction of the 
network and segmentation.
To avoid too small timeouts per one node I suggest to introduce a constant like 
100ms as minimal timeout on attempt to connect to one next node in the ring.


> Ignite cluster falls apart if two nodes segmented sequentially
> --
>
> Key: IGNITE-13465
> URL: https://issues.apache.org/jira/browse/IGNITE-13465
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.9
>Reporter: Aleksey Plekhanov
>Assignee: Vladimir Steshin
>Priority: Blocker
> Fix For: 2.9
>
> Attachments: GridSequentionNodesFailureTest.java
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> After ticket IGNITE-13134 sequential nodes segmentation leads to segmentation 
> of other nodes in the cluster.
> Reproducer attached.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-11312) JDBC: Thin driver reports incorrect property names

2020-09-22 Thread Stanislav Lukyanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stanislav Lukyanov updated IGNITE-11312:

Summary: JDBC: Thin driver reports incorrect property names  (was: JDBC: 
Thin driver doesn't reports incorrect property names)

> JDBC: Thin driver reports incorrect property names
> --
>
> Key: IGNITE-11312
> URL: https://issues.apache.org/jira/browse/IGNITE-11312
> Project: Ignite
>  Issue Type: Improvement
>  Components: jdbc
>Reporter: Stanislav Lukyanov
>Assignee: Lev Agafonov
>Priority: Major
>  Labels: newbie
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> JDBC driver reports the properties it supports via getPropertyInfo method. It 
> currently reports the property names as simple strings, like 
> "enforceJoinOrder". However, when the properties are processed on connect 
> they are looked up with prefix "ignite.jdbc", e.g. 
> "ignite.jdbc.enforceJoinOrder".
> Because of this UI tools like DBeaver can't properly pass the properties to 
> Ignite. For example, when "enforceJoinOrder" is set to true in "Connection 
> settings" -> "Driver properties" menu of DBeaver it has no effect.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13475) NPE on IgniteTxHandler.finishDhtLocal

2020-09-22 Thread Ilya Kasnacheev (Jira)
Ilya Kasnacheev created IGNITE-13475:


 Summary: NPE on IgniteTxHandler.finishDhtLocal
 Key: IGNITE-13475
 URL: https://issues.apache.org/jira/browse/IGNITE-13475
 Project: Ignite
  Issue Type: Bug
  Components: cache
Reporter: Ilya Kasnacheev
 Fix For: 2.8.1


{code}
[05:57:16,193][SEVERE][sys-stripe-15-#16][] Critical system error detected. 
Will be handled accordingly to configured handler 
[hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, 
super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet 
[SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], 
failureCtx=FailureContext [type=CRITICAL_ERROR, 
err=java.lang.NullPointerException]]
java.lang.NullPointerException
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:1064)
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:953)
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:909)
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$200(IgniteTxHandler.java:123)
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:217)
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:215)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1142)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:591)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1847)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1472)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$5200(GridIoManager.java:229)
at 
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1367)
at 
org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:565)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:745)
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13465) Ignite cluster falls apart if two nodes segmented sequentially

2020-09-22 Thread Vladimir Steshin (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17199980#comment-17199980
 ] 

Vladimir Steshin commented on IGNITE-13465:
---

The problem is that connectionRecoveryTimeout can be wholly spent on one next 
node. If two fails at the same time, previous nodes may become segmented one by 
one. 

I suggest to slice connectionRecoveryTimeout in order to traverse several next 
nodes in attempt to reconnect to the ring. We should consider maximum 
reasonable nodes number to reconnect to as `servers/2 + 1`. If we cannot 
connect to half of the ring, this can be considered as major malfunction of the 
network and segmentation.
To avoid too small timeouts per one node I suggest to introduce a constant like 
100ms as minimal timeout on attempt to connect to one next node in the ring.


> Ignite cluster falls apart if two nodes segmented sequentially
> --
>
> Key: IGNITE-13465
> URL: https://issues.apache.org/jira/browse/IGNITE-13465
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.9
>Reporter: Aleksey Plekhanov
>Assignee: Vladimir Steshin
>Priority: Blocker
> Fix For: 2.9
>
> Attachments: GridSequentionNodesFailureTest.java
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> After ticket IGNITE-13134 sequential nodes segmentation leads to segmentation 
> of other nodes in the cluster.
> Reproducer attached.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13463) Calcite improvements. Rework RootNode rows exchange.

2020-09-22 Thread Igor Seliverstov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17199966#comment-17199966
 ] 

Igor Seliverstov commented on IGNITE-13463:
---

[~zstan], LGTM, merged to the feature branch.

> Calcite improvements. Rework RootNode rows exchange.
> 
>
> Key: IGNITE-13463
> URL: https://issues.apache.org/jira/browse/IGNITE-13463
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.8.1
> Environment: In current implementation calcite.exec.rel.RootNode 
> generates huge number of context switches, looks like we have a field for 
> optimizations here. Need additional investigations and benchmarks.
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-13474) Client node consistentId uniqueness is not checked

2020-09-22 Thread Ilya Kasnacheev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Kasnacheev updated IGNITE-13474:
-
Attachment: ServerNode.java
ClientNode.java

> Client node consistentId uniqueness is not checked
> --
>
> Key: IGNITE-13474
> URL: https://issues.apache.org/jira/browse/IGNITE-13474
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 2.8.1
>Reporter: Ilya Kasnacheev
>Priority: Major
> Attachments: ClientNode.java, ServerNode.java
>
>
> Please see attached server and client nodes, as well as SO discussion.
> consistentId uniqueness is not checked on client node join, leading to 
> multiple client nodes with same consistentId as on server node.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13474) Client node consistentId uniqueness is not checked

2020-09-22 Thread Ilya Kasnacheev (Jira)
Ilya Kasnacheev created IGNITE-13474:


 Summary: Client node consistentId uniqueness is not checked
 Key: IGNITE-13474
 URL: https://issues.apache.org/jira/browse/IGNITE-13474
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 2.8.1
Reporter: Ilya Kasnacheev


Please see attached server and client nodes, as well as SO discussion.

consistentId uniqueness is not checked on client node join, leading to multiple 
client nodes with same consistentId as on server node.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-13472) JDBC bulkload operations are processed with wrong security context

2020-09-22 Thread Ryabov Dmitrii (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryabov Dmitrii updated IGNITE-13472:

Description: 
{{JdbcAuthorizationTest.testCopyFrom()}} have many exceptions like

{code:java}
[12:02:56] (err) Failed to execute compound future reducer: GridCompoundFuture 
[rdc=null, initFlag=1, lsnrCalls=0, done=false, cancelled=false, err=null, 
futs=TransformCollectionView [true]]class 
org.apache.ignite.IgniteCheckedException: Authorization failed [perm=CACHE_PUT, 
name=test-bulkload-cache, 
subject=TestSecuritySubject{id=ca0e2ed9-f877-4c49-a80e-11a1c7a0, 
type=REMOTE_NODE, login=jdbc.JdbcAuthorizationTest0}]
at 
org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7589)
at 
org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:979)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at 
org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:569)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.plugin.security.SecurityException: 
Authorization failed [perm=CACHE_PUT, name=test-bulkload-cache, 
subject=TestSecuritySubject{id=ca0e2ed9-f877-4c49-a80e-11a1c7a0, 
type=REMOTE_NODE, login=jdbc.JdbcAuthorizationTest0}]
at 
org.apache.ignite.internal.processors.security.impl.TestSecurityProcessor.authorize(TestSecurityProcessor.java:153)
at 
org.apache.ignite.internal.processors.security.IgniteSecurityProcessor.authorize(IgniteSecurityProcessor.java:207)
at 
org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.checkSecurityPermission(DataStreamerUpdateJob.java:170)
at 
org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:123)
at 
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7087)
at 
org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:971)
... 4 more
{code}
This exception means the put operation is processed with node context instead 
of user. Also, operation doesn't finish successfully.

  was:
{{JdbcAuthorizationTest.testCopyFrom()}} have many exceptions like

{code:java}
[12:02:56] (err) Failed to execute compound future reducer: GridCompoundFuture 
[rdc=null, initFlag=1, lsnrCalls=0, done=false, cancelled=false, err=null, 
futs=TransformCollectionView [true]]class 
org.apache.ignite.IgniteCheckedException: Authorization failed [perm=CACHE_PUT, 
name=test-bulkload-cache, 
subject=TestSecuritySubject{id=ca0e2ed9-f877-4c49-a80e-11a1c7a0, 
type=REMOTE_NODE, login=jdbc.JdbcAuthorizationTest0}]
at 
org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7589)
at 
org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:979)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at 
org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:569)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.plugin.security.SecurityException: 
Authorization failed [perm=CACHE_PUT, name=test-bulkload-cache, 
subject=TestSecuritySubject{id=ca0e2ed9-f877-4c49-a80e-11a1c7a0, 
type=REMOTE_NODE, login=jdbc.JdbcAuthorizationTest0}]
at 
org.apache.ignite.internal.processors.security.impl.TestSecurityProcessor.authorize(TestSecurityProcessor.java:153)
at 
org.apache.ignite.internal.processors.security.IgniteSecurityProcessor.authorize(IgniteSecurityProcessor.java:207)
at 
org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.checkSecurityPermission(DataStreamerUpdateJob.java:170)
at 
org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:123)
at 
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7087)
at 
org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:971)
... 4 more
{code}


> JDBC bulkload operations are processed with wrong security context
> --
>
> Key: IGNITE-13472
> URL: https://issues.apache.org/jira/browse/IGNITE-13472
> Project: Ignite
>  Issue Type: Bug
>Reporter: Ryabov Dmitrii
>Assignee: Ryabov Dmitrii
>Priority: Minor
>
> {{JdbcAuthorizationTest.testCopyFrom()}} have many exceptions like
> {code:java}
> [12:02:56] (err) Failed to execute

[jira] [Created] (IGNITE-13473) Snapshot tests for ducktape

2020-09-22 Thread Sergei Ryzhov (Jira)
Sergei Ryzhov created IGNITE-13473:
--

 Summary: Snapshot tests for ducktape
 Key: IGNITE-13473
 URL: https://issues.apache.org/jira/browse/IGNITE-13473
 Project: Ignite
  Issue Type: Task
Reporter: Sergei Ryzhov
Assignee: Sergei Ryzhov


Snapshot tests for ducktape



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13472) JDBC bulkload operations are processed with wrong security context

2020-09-22 Thread Ryabov Dmitrii (Jira)
Ryabov Dmitrii created IGNITE-13472:
---

 Summary: JDBC bulkload operations are processed with wrong 
security context
 Key: IGNITE-13472
 URL: https://issues.apache.org/jira/browse/IGNITE-13472
 Project: Ignite
  Issue Type: Bug
Reporter: Ryabov Dmitrii
Assignee: Ryabov Dmitrii


{{JdbcAuthorizationTest.testCopyFrom()}} have many exceptions like

{code:java}
[12:02:56] (err) Failed to execute compound future reducer: GridCompoundFuture 
[rdc=null, initFlag=1, lsnrCalls=0, done=false, cancelled=false, err=null, 
futs=TransformCollectionView [true]]class 
org.apache.ignite.IgniteCheckedException: Authorization failed [perm=CACHE_PUT, 
name=test-bulkload-cache, 
subject=TestSecuritySubject{id=ca0e2ed9-f877-4c49-a80e-11a1c7a0, 
type=REMOTE_NODE, login=jdbc.JdbcAuthorizationTest0}]
at 
org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7589)
at 
org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:979)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at 
org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:569)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.plugin.security.SecurityException: 
Authorization failed [perm=CACHE_PUT, name=test-bulkload-cache, 
subject=TestSecuritySubject{id=ca0e2ed9-f877-4c49-a80e-11a1c7a0, 
type=REMOTE_NODE, login=jdbc.JdbcAuthorizationTest0}]
at 
org.apache.ignite.internal.processors.security.impl.TestSecurityProcessor.authorize(TestSecurityProcessor.java:153)
at 
org.apache.ignite.internal.processors.security.IgniteSecurityProcessor.authorize(IgniteSecurityProcessor.java:207)
at 
org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.checkSecurityPermission(DataStreamerUpdateJob.java:170)
at 
org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:123)
at 
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7087)
at 
org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:971)
... 4 more
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13471) Execute user-defined compute jobs asynchronously when CompletionStage is returned

2020-09-22 Thread Pavel Tupitsyn (Jira)
Pavel Tupitsyn created IGNITE-13471:
---

 Summary: Execute user-defined compute jobs asynchronously when 
CompletionStage is returned
 Key: IGNITE-13471
 URL: https://issues.apache.org/jira/browse/IGNITE-13471
 Project: Ignite
  Issue Type: Improvement
  Components: compute
Reporter: Pavel Tupitsyn


When user-defined Compute jobs (callables, runnables) return CompletionStage:

1. Wait for completion (in a non-blocking way if possible)
2. Extract the result and return to the initiator node
3. Construct completed CompletionStage with the result on the initiator node 
and return to the user code



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-13454) Thin Client Ping API

2020-09-22 Thread Pavel Tupitsyn (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-13454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn updated IGNITE-13454:

Labels: .NET newbie  (was: .NET)

> Thin Client Ping API
> 
>
> Key: IGNITE-13454
> URL: https://issues.apache.org/jira/browse/IGNITE-13454
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: .NET, newbie
> Fix For: 2.10
>
>
> Add ping API to thin clients: IgniteClient.ping() (Java), 
> IIgniteClient.Ping() (.NET) and async counterparts.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (IGNITE-13422) Explicit command.sh option to enable experimental commands

2020-09-22 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17199918#comment-17199918
 ] 

Ignite TC Bot commented on IGNITE-13422:


{panel:title=Branch: [pull/8233/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/8233/head] Base: [master] : No new tests 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5616422&buildTypeId=IgniteTests24Java8_RunAll]

> Explicit command.sh option to enable experimental commands
> --
>
> Key: IGNITE-13422
> URL: https://issues.apache.org/jira/browse/IGNITE-13422
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Fix For: 2.10
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Right now experimental commands can be enabled only via an environment 
> variable(IGNITE_ENABLE_EXPERIMENTAL_COMMAND).
> We need to add an explicit option to command.sh "--enable-experimental" which 
> will turn on experimental commands.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)