Re: [GridCachePartitionExchangeManager] Pending transaction deadlock detection futures

[ght230]

I have created a JIRA for that.
https://issues.apache.org/jira/browse/IGNITE-4924.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/GridCachePartitionExchangeManager-Pending-transaction-deadlock-detection-futures-tp11362p11767.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: [GridCachePartitionExchangeManager] Pending transaction deadlock detection futures

[Alexey Kuznetsov]

Hi ght230,

I think that Visor CMD should work in client mode, but it is not
implemented yet.
Please create issue in Ignite JIRA for that.

On Sat, Apr 1, 2017 at 1:41 PM, ght230 <ght...@163.com> wrote:

> I want to know should Visor CMD be worked in client mode or daemon mode?
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/GridCachePartitionExchangeManager-Pending-transaction-
> deadlock-detection-futures-tp11362p11638.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Alexey Kuznetsov

Re: [GridCachePartitionExchangeManager] Pending transaction deadlock detection futures

[ght230]

I want to know should Visor CMD be worked in client mode or daemon mode?



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/GridCachePartitionExchangeManager-Pending-transaction-deadlock-detection-futures-tp11362p11638.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: [GridCachePartitionExchangeManager] Pending transaction deadlock detection futures

[Alexey Kuznetsov]

Val,

>> Does anyone have an idea why client mode in Visor affects behavior? I
thought we already forced client mode there, no?

Visor CMD was NOT reworked to client mode.
So Visor CMD starts server node in daemon mode.


On Wed, Mar 29, 2017 at 2:28 AM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> cross-posting to dev list.
>
> Guys,
>
> Does anyone have an idea why client mode in Visor affects behavior? I
> thought we already forced client mode there, no?
>
> Alexey, you should know the answer. Can you please take a look at this
> thread?
>
> -Val
>
>
> On Tue, Mar 28, 2017 at 7:02 AM, ght230  wrote:
>
>> Yes,I use custom build.
>>
>> Today I tried to set Ignition.setClientMode(true) for command "open" of
>> the
>> visorcmd.
>>
>> It seems the visorcmd will not stuck the whole cluster again.
>>
>> Is there anything wrong in "ClientMode" of the visorcmd?
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/GridCachePartitionExchangeManager-Pending-t
>> ransaction-deadlock-detection-futures-tp11362p11501.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>


-- 
Alexey Kuznetsov

Re: [GridCachePartitionExchangeManager] Pending transaction deadlock detection futures

[Valentin Kulichenko]

cross-posting to dev list.

Guys,

Does anyone have an idea why client mode in Visor affects behavior? I
thought we already forced client mode there, no?

Alexey, you should know the answer. Can you please take a look at this
thread?

-Val

On Tue, Mar 28, 2017 at 7:02 AM, ght230 <ght...@163.com> wrote:

> Yes,I use custom build.
>
> Today I tried to set Ignition.setClientMode(true) for command "open" of the
> visorcmd.
>
> It seems the visorcmd will not stuck the whole cluster again.
>
> Is there anything wrong in "ClientMode" of the visorcmd?
>
>
>
> --
> View this message in context: http://apache-ignite-users.705
> 18.x6.nabble.com/GridCachePartitionExchangeManager-Pending-
> transaction-deadlock-detection-futures-tp11362p11501.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: [GridCachePartitionExchangeManager] Pending transaction deadlock detection futures

[ght230]

Yes,I use custom build.

Today I tried to set Ignition.setClientMode(true) for command "open" of the
visorcmd.

It seems the visorcmd will not stuck the whole cluster again.

Is there anything wrong in "ClientMode" of the visorcmd?



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/GridCachePartitionExchangeManager-Pending-transaction-deadlock-detection-futures-tp11362p11501.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: [GridCachePartitionExchangeManager] Pending transaction deadlock detection futures

[dkarachentsev]

Hi,

Do you use custom build? In logs Ignite version 1.6.12, but it uses stripped
pool, which was introduced only in 1.8.2 and 1.9.
Also there was a node stall, because of many disk operations (swap file).
This doesn't lead to hang, but may reduce cache performance.

Besides log files also thread dumps required, because they help to find out
the reason of the hang. Please, attach them as well.

-Dmitry.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/GridCachePartitionExchangeManager-Pending-transaction-deadlock-detection-futures-tp11362p11485.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: [GridCachePartitionExchangeManager] Pending transaction deadlock detection futures

[dkarachentsev]

Hi,

What operations in visorcmd you did?
Please attach thread dumps (when cluster hangs) and logs from all nodes.

Thanks!

-Dmitry.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/GridCachePartitionExchangeManager-Pending-transaction-deadlock-detection-futures-tp11362p11386.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

[GridCachePartitionExchangeManager] Pending transaction deadlock detection futures

[ght230]

I start 4 Ignite servers, then I start ignitevisorcmd to view the cache.

But sometimes the ignitevisorcmd will stop at 
"Security status [authentication=off, tls/ssl=off]"
and it will stuck the whole cluster.

>From the log of ignitevisorcmd, we can find the following info.

"[16:52:23,640][WARN
][exchange-worker-#300%null%][GridCachePartitionExchangeManager] Pending
exchange futures:
[16:52:23,641][WARN
][exchange-worker-#300%null%][GridCachePartitionExchangeManager] Last 10
exchange futures (total: 1):
[16:52:23,646][WARN
][exchange-worker-#300%null%][GridCachePartitionExchangeManager] >>>
GridDhtPartitionsExchangeFuture [dummy=false, forcePreload=false,
reassign=false, discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode
[id=c0256629-1950-45cf-98cb-e1d874e907a0, nodeName=null, addrs=[10.10.10.81,
127.0.0.1, 192.168.14.81], sockAddrs=[/127.0.0.1:47504, /10.10.10.81:47504,
dn81.dcs.com/192.168.14.81:47504], discPort=47504, order=8, intOrder=7,
lastExchangeTime=1490172142809, loc=true, ver=1.6.12#19700101-sha1:,
isClient=false], topVer=8, nodeId8=c0256629, msg=null, type=NODE_JOINED,
tstamp=1490172143132], crd=TcpDiscoveryNode
[id=b0b8409c-36de-42c2-92c2-65f583941e07, nodeName=null, addrs=[10.10.10.81,
127.0.0.1, 192.168.14.81], sockAddrs=[dn81.dcs.com/192.168.14.81:47500,
/127.0.0.1:47500, /10.10.10.81:47500], discPort=47500, order=1, intOrder=1,
lastExchangeTime=1490172143021, loc=false,
ver=1.6.12#19700101-sha1:, isClient=false],
exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=8,
minorTopVer=0], nodeId=c0256629, evt=NODE_JOINED], added=false,
initFut=GridFutureAdapter [resFlag=2, res=true, startTime=1490172143567,
endTime=1490172143617, ignoreInterrupts=false, state=DONE], init=true,
topSnapshot=null, lastVer=null, partReleaseFut=null, affChangeMsg=null,
skipPreload=true, clientOnlyExchange=true, initTs=1490172143567,
centralizedAff=false, forcedRebFut=null, evtLatch=0,
remaining=[b0b8409c-36de-42c2-92c2-65f583941e07,
d55c68e1-ddbd-435b-9c74-f7b59912cd49, b208aa0c-08bd-4f93-9c59-ac5ea49bdfda,
6fdfdedc-d797-4c62-902e-26df6aa97b57], srvNodes=[TcpDiscoveryNode
[id=b0b8409c-36de-42c2-92c2-65f583941e07, nodeName=null, addrs=[10.10.10.81,
127.0.0.1, 192.168.14.81], sockAddrs=[dn81.dcs.com/192.168.14.81:47500,
/127.0.0.1:47500, /10.10.10.81:47500], discPort=47500, order=1, intOrder=1,
lastExchangeTime=1490172143021, loc=false,
ver=1.6.12#19700101-sha1:, isClient=false], TcpDiscoveryNode
[id=d55c68e1-ddbd-435b-9c74-f7b59912cd49, nodeName=null, addrs=[10.10.10.81,
127.0.0.1, 192.168.14.81], sockAddrs=[/10.10.10.81:47501, /127.0.0.1:47501,
dn81.dcs.com/192.168.14.81:47501], discPort=47501, order=2, intOrder=2,
lastExchangeTime=1490172143031, loc=false,
ver=1.6.12#19700101-sha1:, isClient=false], TcpDiscoveryNode
[id=6fdfdedc-d797-4c62-902e-26df6aa97b57, nodeName=null, addrs=[10.10.10.81,
127.0.0.1, 192.168.14.81], sockAddrs=[/10.10.10.81:47502,
dn81.dcs.com/192.168.14.81:47502, /127.0.0.1:47502], discPort=47502,
order=4, intOrder=4, lastExchangeTime=1490172143041, loc=false,
ver=1.6.12#19700101-sha1:, isClient=false], TcpDiscoveryNode
[id=b208aa0c-08bd-4f93-9c59-ac5ea49bdfda, nodeName=null, addrs=[10.10.10.81,
127.0.0.1, 192.168.14.81], sockAddrs=[/127.0.0.1:47503, /10.10.10.81:47503,
dn81.dcs.com/192.168.14.81:47503], discPort=47503, order=5, intOrder=5,
lastExchangeTime=1490172143051, loc=false,
ver=1.6.12#19700101-sha1:, isClient=false]], super=GridFutureAdapter
[resFlag=0, res=null, startTime=1490172143567, endTime=0,
ignoreInterrupts=false, state=INIT]]
[16:52:23,650][WARN
][exchange-worker-#300%null%][GridCachePartitionExchangeManager] Pending
transactions:
[16:52:23,651][WARN
][exchange-worker-#300%null%][GridCachePartitionExchangeManager] Pending
explicit locks:
[16:52:23,651][WARN
][exchange-worker-#300%null%][GridCachePartitionExchangeManager] Pending
cache futures:
[16:52:23,652][WARN
][exchange-worker-#300%null%][GridCachePartitionExchangeManager] Pending
atomic cache futures:
[16:52:23,653][WARN
][exchange-worker-#300%null%][GridCachePartitionExchangeManager] Pending
data streamer futures:
[16:52:23,654][WARN
][exchange-worker-#300%null%][GridCachePartitionExchangeManager] Pending
transaction deadlock detection futures:
[16:52:23,655][WARN ][exchange-worker-#300%null%][TcpCommunicationSpi]
Communication SPI recovery descriptors: 
[key=ConnectionKey [nodeId=b0b8409c-36de-42c2-92c2-65f583941e07, idx=0,
connCnt=-1], msgsSent=1, msgsAckedByRmt=1, msgsRcvd=0, lastAcked=0,
reserveCnt=1, descIdHash=1052107904]
Communication SPI clients: "

Does anyone know about it?




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/GridCachePartitionExchangeManager-Pending-transaction-deadlock-detection-futures-tp11362.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.