Hi Sergey, you are right - I can reproduce this also. It seems to me that this is caused because we treat the same both EVT_NODE_LEFT and EVT_NODE_FAILED events. In this case, node leaves the topology without failure, but fails to release the semaphore before EVT_NODE_LEFT event occurs on other nodes, this really is a bug.
Thanks! Vladisav On Wed, Nov 9, 2016 at 11:23 AM, Sergey Chugunov <sergey.chugu...@gmail.com> wrote: > Hello Vladisav, > > I found this behavior in a very simple environment: I had two nodes on my > local machine started by *ExampleNodeStartup* class and another node > started with *IgniteSemaphoreExample* class. > > No modifications were made to any code or configuration and I used latest > version of code available in master branch. > No node failures occurred during test execution as well. > > As far as I understood from short investigation synchronization semaphore > of name "IgniteSemaphoreExample" goes to broken state when > *IgniteSemaphoreExample* node normally finishes and disconnects from the > cluster. > After that reusing of this semaphore becomes impossible and leads to > hanging of new nodes doing so. > > Can you reproduce this? If so I will submit a ticket and share with you. > > Thank you, > Sergey. > > > On Wed, Nov 9, 2016 at 10:55 AM, Vladisav Jelisavcic <vladis...@gmail.com> > wrote: > > > Hi Sergey, > > > > can you please provide more information? > > Have you changed the example (if so, can you provide the changes you > made?) > > Is the example executed normally (without node failures)? > > > > In the example, semaphore is created in non-failover safe mode, > > which means it is not safe to use it once it is broken (something like > > CyclicBarrier in java.util.concurrent), > > and the semaphore is preserved in spite of the first node failing (if the > > backups are configured), > > so if the first node failed, then (broken) semaphore with the same name > > should still be in the cache, > > and this is expected behavior. > > > > If this is not the case (test was executed normally) then please submit a > > ticket describing more your setup, > > how many nodes, how many backups configured, etc.. > > > > Thanks! > > Vladisav > > > > On Tue, Nov 8, 2016 at 10:37 AM, Sergey Chugunov < > > sergey.chugu...@gmail.com> > > wrote: > > > > > Hello folks, > > > > > > I found a reason why *IgniteSemaphoreExample* hangs when started twice > > > without restarting a cluster; and it doesn't seem minor to me anymore. > > > > > > From here I'm going to refer to example's code so please have it > opened. > > > > > > So, when the first instance of node running example code finishes and > > > leaves the cluster, synchronization semaphore named > > > "IgniteSemaphoreExample" goes to broken state on all other cluster > nodes. > > > If I restart example without restarting all nodes of the cluster, final > > > *acquire *call on the semaphore on client side hangs because all other > > > nodes treat it as broken and don't increase permits with their *release > > > *calls > > > on it. > > > > > > There is an interesting comment inside its *tryReleaseShared* > > > implementation > > > (BTW it is implemented in *GridCacheSemaphoreImpl*): > > > > > > "// If broken, return immediately, exception will be thrown anyway. > > > if (broken) > > > return true;" > > > > > > It seems that no exceptions are thrown neither on client side calling > > > *acquire > > > *or on server side calling *release *methods on a broken semaphore. > > > > > > Does anybody know why it behaves in that way? Is it expected behavior > at > > > all and if yes where is it documented? > > > > > > Thanks, > > > Sergey Chugunov. > > > > > > > > > -- > С уважением, > Сергей Чугунов. >
