Hi,I have download 1.5.0-b1 and make it happened again.For I want to use
Ignite in our production environment, I want to make sure whether it's
caused by not correct configuration."
"The follow is the exception logs from node 10.1.XX.XX which left the grid.
XX_[hadoop-s1]_logs.txt
<http://apache-ignite-users.70518.x6.nabble.com/file/n2219/XX_%5Bhadoop-s1%5D_logs.txt>
The follow is the logs from the other 2 nodes. gridlogs.rar
<http://apache-ignite-users.70518.x6.nabble.com/file/n2219/gridlogs.rar>
2015-12-14T16:25:34+0000 DEBUG [grid-time-coordinator-#62%SCHEDULE_NEW%]
org.apache.ignite.internal.processors.clock.GridClockSyncProcessor -
Collected time sync results:
{9e22645c-52ce-404f-bd42-b82e98a1b593=0}2015-12-14T16:25:34+0000 DEBUG
[ignite-#16%pub-SCHEDULE_NEW%]
org.apache.ignite.spi.failover.always.AlwaysFailoverSpi - Received failed
job result: GridJobResultImpl [job=C2 [], sib=GridJobSiblingImpl
[sesId=47ec4de9151-0d8281ae-7bfd-4a19-bf6f-35128bef7c8c,
jobId=57ec4de9151-148fc364-78cd-475f-81bd-b3188a47f5f6,
nodeId=148fc364-78cd-475f-81bd-b3188a47f5f6, isJobDone=false],
jobCtx=GridJobContextImpl
[jobId=57ec4de9151-148fc364-78cd-475f-81bd-b3188a47f5f6, timeoutObj=null,
attrs={}], node=TcpDiscoveryNode [id=148fc364-78cd-475f-81bd-b3188a47f5f6,
addrs=[0:0:0:0:0:0:0:1%1, 10.1.XX.XX, 127.0.0.1],
sockAddrs=[hadoop-s1/10.1.XX.XX:47500, /0:0:0:0:0:0:0:1%1:47500,
/10.1.XX.XX:47500, /127.0.0.1:47500], discPort=47500, order=1, intOrder=1,
lastExchangeTime=1450068071263, loc=false, ver=1.5.0#20151201-sha1:062d440c,
isClient=false], ex=class
org.apache.ignite.cluster.ClusterTopologyException: Node has left grid:
148fc364-78cd-475f-81bd-b3188a47f5f6, hasRes=true, isCancelled=false,
isOccupied=true]2015-12-14T16:25:34+0000 WARN [ignite-#16%pub-SCHEDULE_NEW%]
org.apache.ignite.spi.failover.always.AlwaysFailoverSpi - Job failover
failed because number of maximum failover attempts for affinity call is
exceeded [failedJob=C2 [], maxFailoverAttempts=1]2015-12-14T16:25:34+0000
DEBUG [ignite-#16%pub-SCHEDULE_NEW%]
org.apache.ignite.internal.processors.task.GridTaskWorker - Failed to
failover a job to another node (failover SPI returned null) [job=C2 [],
node=TcpDiscoveryNode [id=148fc364-78cd-475f-81bd-b3188a47f5f6,
addrs=[0:0:0:0:0:0:0:1%1, 10.1.XX.XX, 127.0.0.1],
sockAddrs=[hadoop-s1/10.1.XX.XX:47500, /0:0:0:0:0:0:0:1%1:47500,
/10.1.XX.XX:47500, /127.0.0.1:47500], discPort=47500, order=1, intOrder=1,
lastExchangeTime=1450068071263, loc=false, ver=1.5.0#20151201-sha1:062d440c,
isClient=false]]2015-12-14T16:25:34+0000 DEBUG
[ignite-#16%pub-SCHEDULE_NEW%]
org.apache.ignite.internal.processors.task.GridTaskWorker - Creating fake
response because node left grid [job=C2 [],
nodeId=148fc364-78cd-475f-81bd-b3188a47f5f6]2015-12-14T16:25:34+0000 DEBUG
[ignite-#16%pub-SCHEDULE_NEW%]
org.apache.ignite.internal.processors.task.GridTaskWorker - Simulating
fake response from left node [res=GridJobExecuteResponse
[nodeId=148fc364-78cd-475f-81bd-b3188a47f5f6,
sesId=eaec4de9151-0d8281ae-7bfd-4a19-bf6f-35128bef7c8c,
jobId=faec4de9151-148fc364-78cd-475f-81bd-b3188a47f5f6, gridEx=null,
isCancelled=false],
nodeId=148fc364-78cd-475f-81bd-b3188a47f5f6]2015-12-14T16:25:34+0000 DEBUG
[ignite-#16%pub-SCHEDULE_NEW%]
org.apache.ignite.internal.processors.task.GridTaskWorker - Obtained job
result policy [policy=FAILOVER, ses=GridTaskSessionImpl
[taskName=com.ukelink.demo.schedule.service.ScheduleCompute$2,
dep=LocalDeployment [super=GridDeployment [ts=1450068069718, depMode=SHARED,
clsLdr=sun.misc.Launcher$AppClassLoader@35e5ebbf,
clsLdrId=65dcace9151-0d8281ae-7bfd-4a19-bf6f-35128bef7c8c, userVer=0,
loc=true, sampleClsName=java.lang.String, pendingUndeploy=false,
undeployed=false, usage=0]],
taskClsName=com.ukelink.demo.schedule.service.ScheduleCompute$2,
sesId=eaec4de9151-0d8281ae-7bfd-4a19-bf6f-35128bef7c8c,
startTime=1450110329992, endTime=9223372036854775807,
taskNodeId=0d8281ae-7bfd-4a19-bf6f-35128bef7c8c,
clsLdr=sun.misc.Launcher$AppClassLoader@35e5ebbf, closed=false, cpSpi=null,
failSpi=null, loadSpi=null, usage=1, fullSup=false,
subjId=0d8281ae-7bfd-4a19-bf6f-35128bef7c8c, mapFut=IgniteFuture
[orig=GridFutureAdapter [resFlag=2, res=null, startTime=1450110329992,
endTime=1450110329992, ignoreInterrupts=false, lsnr=null,
state=DONE]]]]2015-12-14T16:25:34+0000 DEBUG [ignite-#16%pub-SCHEDULE_NEW%]
org.apache.ignite.spi.failover.always.AlwaysFailoverSpi - Received failed
job result: GridJobResultImpl [job=C2 [], sib=GridJobSiblingImpl
[sesId=eaec4de9151-0d8281ae-7bfd-4a19-bf6f-35128bef7c8c,
jobId=faec4de9151-148fc364-78cd-475f-81bd-b3188a47f5f6,
nodeId=148fc364-78cd-475f-81bd-b3188a47f5f6, isJobDone=false],
jobCtx=GridJobContextImpl
[jobId=faec4de9151-148fc364-78cd-475f-81bd-b3188a47f5f6, timeoutObj=null,
attrs={}], node=TcpDiscoveryNode [id=148fc364-78cd-475f-81bd-b3188a47f5f6,
addrs=[0:0:0:0:0:0:0:1%1, 10.1.XX.XX, 127.0.0.1],
sockAddrs=[hadoop-s1/10.1.XX.XX:47500, /0:0:0:0:0:0:0:1%1:47500,
/10.1.XX.XX:47500, /127.0.0.1:47500], discPort=47500, order=1, intOrder=1,
lastExchangeTime=1450068071263, loc=false, ver=1.5.0#20151201-sha1:062d440c,
isClient=false], ex=class
org.apache.ignite.cluster.ClusterTopologyException: Node has left grid:
148fc364-78cd-475f-81bd-b3188a47f5f6, hasRes=true, isCancelled=false,
isOccupied=true]2015-12-14T16:25:34+0000 DEBUG [ignite-#1%pub-SCHEDULE_NEW%]
org.apache.ignite.internal.processors.closure.GridClosureProcessor - Grid
runnable started: closure-proc-worker2015-12-14T16:25:34+0000 WARN
[ignite-#16%pub-SCHEDULE_NEW%]
org.apache.ignite.spi.failover.always.AlwaysFailoverSpi - Job failover
failed because number of maximum failover attempts for affinity call is
exceeded [failedJob=C2 [], maxFailoverAttempts=1]2015-12-14T16:25:34+0000
DEBUG [ignite-#1%pub-SCHEDULE_NEW%]
org.apache.ignite.internal.processors.closure.GridClosureProcessor - Grid
runnable finished normally: closure-proc-worker2015-12-14T16:25:34+0000
DEBUG [ignite-#16%pub-SCHEDULE_NEW%]
org.apache.ignite.internal.processors.task.GridTaskWorker - Failed to
failover a job to another node (failover SPI returned null) [job=C2 [],
node=TcpDiscoveryNode [id=148fc364-78cd-475f-81bd-b3188a47f5f6,
addrs=[0:0:0:0:0:0:0:1%1, 10.1.XX.XX, 127.0.0.1],
sockAddrs=[hadoop-s1/10.1.XX.XX:47500, /0:0:0:0:0:0:0:1%1:47500,
/10.1.XX.XX:47500, /127.0.0.1:47500], discPort=47500, order=1, intOrder=1,
lastExchangeTime=1450068071263, loc=false, ver=1.5.0#20151201-sha1:062d440c,
isClient=false]]2015-12-14T16:25:34+0000 DEBUG
[ignite-#16%pub-SCHEDULE_NEW%]
org.apache.ignite.internal.processors.closure.GridClosureProcessor - Grid
runnable finished normally: closure-proc-worker2015-12-14T16:25:34+0000
DEBUG [tcp-disco-msg-worker-#2%SCHEDULE_NEW]
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - Message has been
sent to next node [msg=TcpDiscoveryDiscardMessage
[msgId=80fc4de9151-0d8281ae-7bfd-4a19-bf6f-35128bef7c8c,
customMsgDiscard=true, super=TcpDiscoveryAbstractMessage [sndNodeId=null,
id=a1fc4de9151-0d8281ae-7bfd-4a19-bf6f-35128bef7c8c,
verifierNodeId=0d8281ae-7bfd-4a19-bf6f-35128bef7c8c, topVer=0, pendingIdx=0,
failedNodes=null, isClient=false]],
next=9e22645c-52ce-404f-bd42-b82e98a1b593, res=1]2015-12-14T16:25:34+0000
DEBUG [tcp-disco-msg-worker-#2%SCHEDULE_NEW]
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - Processing message
[cls=TcpDiscoveryDiscardMessage,
id=01fc4de9151-0d8281ae-7bfd-4a19-bf6f-35128bef7c8c]2015-12-14T16:25:34+0000
DEBUG [tcp-disco-msg-worker-#2%SCHEDULE_NEW]
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - Processing message
[cls=TcpDiscoveryDiscardMessage,
id=31fc4de9151-0d8281ae-7bfd-4a19-bf6f-35128bef7c8c]2015-12-14T16:25:34+0000
DEBUG [tcp-disco-sock-reader-#6%SCHEDULE_NEW]
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - Message has been
received: TcpDiscoveryDiscardMessage
[msgId=80fc4de9151-0d8281ae-7bfd-4a19-bf6f-35128bef7c8c,
customMsgDiscard=true, super=TcpDiscoveryAbstractMessage
[sndNodeId=9e22645c-52ce-404f-bd42-b82e98a1b593,
id=a1fc4de9151-0d8281ae-7bfd-4a19-bf6f-35128bef7c8c,
verifierNodeId=0d8281ae-7bfd-4a19-bf6f-35128bef7c8c, topVer=0, pendingIdx=0,
failedNodes=null, isClient=false]]2015-12-14T16:25:34+0000 DEBUG
[tcp-disco-msg-worker-#2%SCHEDULE_NEW]
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - Processing message
[cls=TcpDiscoveryDiscardMessage,
id=a1fc4de9151-0d8281ae-7bfd-4a19-bf6f-35128bef7c8c]2015-12-14T16:25:34+0000
DEBUG [tcp-disco-sock-reader-#6%SCHEDULE_NEW]
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - Message has been
added to queue: TcpDiscoveryDiscardMessage
[msgId=80fc4de9151-0d8281ae-7bfd-4a19-bf6f-35128bef7c8c,
customMsgDiscard=true, super=TcpDiscoveryAbstractMessage
[sndNodeId=9e22645c-52ce-404f-bd42-b82e98a1b593,
id=a1fc4de9151-0d8281ae-7bfd-4a19-bf6f-35128bef7c8c,
verifierNodeId=0d8281ae-7bfd-4a19-bf6f-35128bef7c8c, topVer=0, pendingIdx=0,
failedNodes=null, isClient=false]]2015-12-14T16:25:34+0000 DEBUG
[disco-event-worker-#46%SCHEDULE_NEW%]
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi - Forcing NIO
client close since node has left
[nodeId=148fc364-78cd-475f-81bd-b3188a47f5f6,
client=GridTcpNioCommunicationClient [ses=GridSelectorNioSessionImpl
[selectorIdx=0, queueSize=0, writeBuf=java.nio.DirectByteBuffer[pos=0
lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768
cap=32768], recovery=GridNioRecoveryDescriptor [acked=412368, resendCnt=0,
rcvCnt=304545, reserved=true, lastAck=304544, nodeLeft=false,
node=TcpDiscoveryNode [id=148fc364-78cd-475f-81bd-b3188a47f5f6,
addrs=[0:0:0:0:0:0:0:1%1, 10.1.XX.XX, 127.0.0.1],
sockAddrs=[hadoop-s1/10.1.XX.XX:47500, /0:0:0:0:0:0:0:1%1:47500,
/10.1.XX.XX:47500, /127.0.0.1:47500], discPort=47500, order=1, intOrder=1,
lastExchangeTime=1450068070384, loc=false, ver=1.5.0#20151201-sha1:062d440c,
isClient=false], connected=true, connectCnt=0, queueLimit=5120],
super=GridNioSessionImpl [locAddr=/10.1.XX.ZZ:47100,
rmtAddr=/10.1.XX.XX:59033, createTime=1450068070495, closeTime=0,
bytesSent=497671887, bytesRcvd=503364967, sndSchedTime=1450110333152,
lastSndTime=1450110333152, lastRcvTime=1450110322496, readsPaused=false,
filterChain=FilterChain[filters=[GridNioCodecFilter
[parser=org.apache.ignite.internal.util.nio.GridDirectParser@20fa7c47,
directMode=true], GridConnectionBytesVerifyFilter], accepted=true]],
super=GridAbstractCommunicationClient [lastUsed=1450068070505,
reserves=0]]]2015-12-14T16:25:34+0000 DEBUG
[disco-event-worker-#46%SCHEDULE_NEW%]
org.apache.ignite.internal.managers.communication.GridIoManager - Removed
message set due to node leaving grid: GridCommunicationMessageSet
[nodeId=148fc364-78cd-475f-81bd-b3188a47f5f6, endTime=1450110336341,
timeoutId=f8dcace9151-c1a9e05a-67c3-4ee0-8cea-7fb5cf78a5a8, topic=T6
[topic=TOPIC_CACHE, id1=83e8ca36-2305-3266-8e65-1463be879baa, id2=0], plc=6,
msgs=[], reserved=false, timeout=10000, skipOnTimeout=false,
lastTs=1450068072063]2015-12-14T16:25:34+0000 DEBUG
[disco-event-worker-#46%SCHEDULE_NEW%]
org.apache.ignite.internal.managers.communication.GridIoManager - Removed
messages from discovery startup delay list (sender node left topology):
null2015-12-14T16:25:34+0000 DEBUG [disco-event-worker-#46%SCHEDULE_NEW%]
org.apache.ignite.internal.processors.cache.GridCacheMvccManager -
Processing node left
[nodeId=148fc364-78cd-475f-81bd-b3188a47f5f6]2015-12-14T16:25:34+0000 DEBUG
[disco-event-worker-#46%SCHEDULE_NEW%]
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicUpdateFuture
- Processing node leave event [fut=GridDhtAtomicUpdateFuture
[futVer=GridCacheVersion [topVer=61547991, nodeOrderDrId=2,
globalTime=1450110331961, order=1450069365377], writeVer=GridCacheVersion
[topVer=61547991, nodeOrderDrId=2, globalTime=1450110331961,
order=1450069365376], forceTransformBackups=false,
mappings={148fc364-78cd-475f-81bd-b3188a47f5f6=GridDhtAtomicUpdateRequest
[nodeId=148fc364-78cd-475f-81bd-b3188a47f5f6, futVer=GridCacheVersion
[topVer=61547991, nodeOrderDrId=2, globalTime=1450110331961,
order=1450069365377], writeVer=GridCacheVersion [topVer=61547991,
nodeOrderDrId=2, globalTime=1450110331961, order=1450069365376],
topVer=AffinityTopologyVersion [topVer=3, minorTopVer=0],
keys=[KeyCacheObjectImpl [val=AffinityKey [key=119522, affKey=Groupid8],
hasValBytes=true]], vals=[CacheObjectImpl [val=ID:119522 VsimID:867743
FlowByte:1014838 Amount:67.0 BuyTime:2015-12-14T16:25:31.962Z Guid:Groupid8
Remark:Just a
test-3370460822845607988!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!,
hasValBytes=true]], prevVals=null, ttls=null, conflictExpireTimes=null,
nearTtls=null, nearExpireTimes=null, syncMode=PRIMARY_SYNC, nearKeys=null,
nearVals=null, forceTransformBackups=false,
subjId=0d8281ae-7bfd-4a19-bf6f-35128bef7c8c, taskNameHash=0,
updateCntrs=GridLongList [idx=1, arr=[152308]], onRes=false,
keepBinary=false, super=GridCacheMessage [msgId=397580, depInfo=null,
err=null, skipPrepare=false, cacheId=-1699403857, cacheId=-1699403857]]},
updateReq=GridNearAtomicUpdateRequest
[nodeId=0d8281ae-7bfd-4a19-bf6f-35128bef7c8c, futVer=GridCacheVersion
[topVer=61547991, nodeOrderDrId=2, globalTime=1450110331961,
order=1450069365375], fastMap=false, updateVer=null,
topVer=AffinityTopologyVersion [topVer=3, minorTopVer=0], topLocked=false,
syncMode=PRIMARY_SYNC, op=UPDATE, keys=[UserKeyCacheObjectImpl
[val=AffinityKey [key=119522, affKey=Groupid8], hasValBytes=false]],
conflictTtls=null, conflictExpireTimes=null, retval=false, expiryPlc=null,
hasPrimary=true, subjId=0d8281ae-7bfd-4a19-bf6f-35128bef7c8c,
taskNameHash=0, skipStore=false, clientReq=false, keepBinary=false,
filter=[CacheEntryPredicateContainsValue [val=UserCacheObjectImpl
[val=ID:119522 VsimID:867743 FlowByte:1014838 Amount:67.0
BuyTime:2015-12-14T15:57:21.865Z Guid:Groupid8 Remark:Just a
test-3370460822845607988!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!,
hasValBytes=false]]], parent=GridCacheMessage [msgId=-1, depInfo=null,
err=null, skipPrepare=false, cacheId=-1699403857, cacheId=-1699403857]],
updateRes=GridNearAtomicUpdateResponse
[nodeId=0d8281ae-7bfd-4a19-bf6f-35128bef7c8c, futVer=GridCacheVersion
[topVer=61547991, nodeOrderDrId=2, globalTime=1450110331961,
order=1450069365375], err=null, ret=GridCacheReturn [v=null, cacheObj=null,
success=true, invokeRes=false, loc=true, cacheId=0], failedKeys=null,
remapKeys=null, nearVals=null, nearVer=null, nearTtls=null,
nearExpireTimes=null, super=parent], waitForExchange=true, resCnt=0],
nodeId=148fc364-78cd-475f-81bd-b3188a47f5f6]
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Grid-stopped-with-IgniteCheckedException-Failed-to-deserialize-object-with-given-class-loader-sun-mir-tp2123p2219.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
