[ https://issues.apache.org/jira/browse/IGNITE-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy Pavlov resolved IGNITE-7878. ------------------------------------ Resolution: Fixed [~vosipov], [~ptupitsyn], thank you. Now test is passing https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_IgnitePlatformNetLongRunning&tab=buildTypeHistoryList&branch_IgniteTests24Java8=%3Cdefault%3E (checked 20 runs) > Ignite Platform .NET Long Running: Stable failed with execution timeout > ------------------------------------------------------------------------- > > Key: IGNITE-7878 > URL: https://issues.apache.org/jira/browse/IGNITE-7878 > Project: Ignite > Issue Type: Task > Reporter: Dmitriy Pavlov > Assignee: Dmitriy Pavlov > Priority: Critical > Labels: MakeTeamcityGreenAgain > > https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_IgnitePlatformNetLongRunning&tab=buildTypeHistoryList&branch_IgniteTests24Java8=%3Cdefault%3E > First failure changes > https://ci.ignite.apache.org/viewLog.html?buildId=1120409&tab=buildChangesDiv&buildTypeId=IgniteTests24Java8_IgnitePlatformNetLongRunning > {noformat} > 15:10:35] : [Apache.Ignite.Core.Tests.exe] > Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault (1h:19m:50s) > [15:10:35] : > [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Error > Output] > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] > >>> __________ ________________ > >>> / _/ ___/ |/ / _/_ __/ __/ > >>> _/ // (7 7 // / / / / _/ > >>> /___/\___/_/|_/___/ /_/ /___/ > >>> > >>> ver. 2.5.0-SNAPSHOT#19700101-sha1:DEV > >>> 2018 Copyright(C) Apache Software Foundation > >>> > >>> Ignite documentation: http://ignite.apache.org > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Config URL: > file:/C:/BuildAgent/work/bd85361428dcdb1/modules/platforms/dotnet/Apache.Ignite.Core.Tests/bin/Debug/Config/Compute/compute-grid2.xml > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] IgniteConfiguration > [igniteInstanceName=null, pubPoolSize=8, svcPoolSize=8, callbackPoolSize=8, > stripedPoolSize=8, sysPoolSize=8, mgmtPoolSize=4, igfsPoolSize=8, > dataStreamerPoolSize=8, utilityCachePoolSize=8, > utilityCacheKeepAliveTime=60000, p2pPoolSize=2, qryPoolSize=8, > igniteHome=C:\BuildAgent\work\bd85361428dcdb1, > igniteWorkDir=C:\BuildAgent\work\bd85361428dcdb1\work, > mbeanSrv=com.sun.jmx.mbeanserver.JmxMBeanServer@5965d37, > nodeId=7cb3b3f7-bd11-4f02-b579-7871ca2063ca, > marsh=org.apache.ignite.internal.binary.BinaryMarshaller@f7d76a5, > marshLocJobs=false, daemon=false, p2pEnabled=false, netTimeout=5000, > sndRetryDelay=1000, sndRetryCnt=3, metricsHistSize=10000, > metricsUpdateFreq=2000, metricsExpTime=9223372036854775807, > discoSpi=TcpDiscoverySpi [addrRslvr=null, sockTimeout=0, ackTimeout=0, > marsh=null, reconCnt=10, reconDelay=2000, maxAckTimeout=600000, > forceSrvMode=false, clientReconnectDisabled=false], segPlc=STOP, > segResolveAttempts=2, waitForSegOnStart=true, allResolversPassReq=true, > segChkFreq=10000, commSpi=TcpCommunicationSpi [connectGate=null, > connPlc=null, enableForcibleNodeKill=false, enableTroubleshootingLog=false, > srvLsnr=org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$2@75d9f436, > locAddr=null, locHost=null, locPort=47100, locPortRange=100, shmemPort=-1, > directBuf=true, directSndBuf=false, idleConnTimeout=600000, connTimeout=5000, > maxConnTimeout=600000, reconCnt=10, sockSndBuf=32768, sockRcvBuf=32768, > msgQueueLimit=0, slowClientQueueLimit=0, nioSrvr=null, shmemSrv=null, > usePairedConnections=false, connectionsPerNode=1, tcpNoDelay=true, > filterReachableAddresses=false, ackSndThreshold=32, unackedMsgsBufSize=0, > sockWriteTimeout=2000, lsnr=null, boundTcpPort=-1, boundTcpShmemPort=-1, > selectorsCnt=4, selectorSpins=0, addrRslvr=null, > ctxInitLatch=java.util.concurrent.CountDownLatch@760bcec6[Count = 1], > stopping=false, > metricsLsnr=org.apache.ignite.spi.communication.tcp.TcpCommunicationMetricsListener@51912f3a], > evtSpi=org.apache.ignite.spi.eventstorage.NoopEventStorageSpi@35bda3f3, > colSpi=NoopCollisionSpi [], deploySpi=LocalDeploymentSpi [lsnr=null], > indexingSpi=org.apache.ignite.spi.indexing.noop.NoopIndexingSpi@fff9936, > addrRslvr=null, clientMode=false, rebalanceThreadPoolSize=1, > txCfg=org.apache.ignite.configuration.TransactionConfiguration@272a1bf1, > cacheSanityCheckEnabled=true, discoStartupDelay=60000, deployMode=SHARED, > p2pMissedCacheSize=100, locHost=null, timeSrvPortBase=31100, > timeSrvPortRange=100, failureDetectionTimeout=10000, > clientFailureDetectionTimeout=30000, metricsLogFreq=60000, hadoopCfg=null, > connectorCfg=org.apache.ignite.configuration.ConnectorConfiguration@7f9a18d1, > odbcCfg=null, warmupClos=null, atomicCfg=AtomicConfiguration > [seqReserveSize=1000, cacheMode=PARTITIONED, backups=1, aff=null, > grpName=null], classLdr=null, sslCtxFactory=null, > platformCfg=PlatformDotNetConfiguration [binaryCfg=null], binaryCfg=null, > memCfg=null, pstCfg=null, dsCfg=DataStorageConfiguration > [sysRegionInitSize=41943040, sysCacheMaxSize=104857600, pageSize=0, > concLvl=0, dfltDataRegConf=DataRegionConfiguration [name=default, > maxSize=5153866547, initSize=268435456, swapPath=null, > pageEvictionMode=DISABLED, evictionThreshold=0.9, emptyPagesPoolSize=100, > metricsEnabled=false, metricsSubIntervalCount=5, > metricsRateTimeInterval=60000, persistenceEnabled=false, > checkpointPageBufSize=0], storagePath=null, checkpointFreq=180000, > lockWaitTime=10000, checkpointThreads=4, checkpointWriteOrder=SEQUENTIAL, > walHistSize=20, walSegments=10, walSegmentSize=67108864, walPath=db/wal, > walArchivePath=db/wal/archive, metricsEnabled=false, walMode=LOG_ONLY, > walTlbSize=131072, walBuffSize=0, walFlushFreq=2000, walFsyncDelay=1000, > walRecordIterBuffSize=67108864, alwaysWriteFullPages=false, > fileIOFactory=org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIOFactory@27e09eea, > metricsSubIntervalCnt=5, metricsRateTimeInterval=60000, > walAutoArchiveAfterInactivity=-1, writeThrottlingEnabled=false, > walCompactionEnabled=false], activeOnStart=true, autoActivation=true, > longQryWarnTimeout=3000, sqlConnCfg=null, > cliConnCfg=ClientConnectorConfiguration [host=null, port=10800, > portRange=100, sockSndBufSize=0, sockRcvBufSize=0, tcpNoDelay=true, > maxOpenCursorsPerConn=128, threadPoolSize=8, idleTimeout=0, jdbcEnabled=true, > odbcEnabled=true, thinCliEnabled=true, sslEnabled=false, > useIgniteSslCtxFactory=true, sslClientAuth=false, sslCtxFactory=null]] > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Daemon mode: off > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] OS: Windows 10 10.0 amd64 > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] OS user: PUBLICAGENT02_0$ > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] PID: 13396 > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Language runtime: Java > Platform API Specification ver. 1.8 > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] VM information: Java(TM) SE > Runtime Environment 1.8.0_161-b12 Oracle Corporation Java HotSpot(TM) 64-Bit > Server VM 25.161-b12 > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] VM total memory: 3.6GB > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Remote Management [restart: > off, REST: on, JMX (remote: off)] > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Logger: Log4JLogger > [quiet=false, config=null] > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] > IGNITE_HOME=C:\BuildAgent\work\bd85361428dcdb1 > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] VM arguments: > [-XX:+HeapDumpOnOutOfMemoryError, -Xms1g, -Xmx4g, -ea, -DIGNITE_QUIET=true, > -Duser.timezone=UTC, -Xdebug, -Xnoagent, -Djava.compiler=NONE, > -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005] > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] System cache's DataRegion > size is configured to 40 MB. Use > DataStorageConfiguration.systemCacheMemorySize property to change the setting. > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Configured caches [in > 'sysMemPlc' dataRegion: ['ignite-sys-cache', 'ignite-hadoop-mr-sys-cache']] > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] 3-rd party licenses can be > found at: C:\BuildAgent\work\bd85361428dcdb1\libs\licenses > [15:10:35] : > [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Output] > [04-Mar-2018 12:10:35][WARN ][pub-#11985][GridDiagnostic] This operating > system has been tested less rigorously: Windows 10 10.0 amd64. Our team will > appreciate the feedback if you experience any problems running ignite in this > environment. > [15:10:35] : > [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Error > Output] > [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] Configured plugins: > [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] ^-- > TestReconnectPlugin 1.0 > [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] ^-- > [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] > [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] ^-- > StanByClusterTestProvider 1.0 > [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] ^-- null > [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] > [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] ^-- Ignite Native > I/O Plugin [Direct I/O] > [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] ^-- Copyright(C) > Apache Software Foundation > [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] > [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] ^-- > TestPlatformPlugin 1.0 > [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] ^-- - > [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] > [04-03-2018 12:10:35][INFO ][main][TcpCommunicationSpi] Successfully bound > communication NIO server to TCP port [port=47100, locHost=0.0.0.0/0.0.0.0, > selectorsCnt=4, selectorSpins=0, pairedConn=false] > [15:10:35] : > [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Output] > [04-Mar-2018 12:10:35][WARN ][main][TcpCommunicationSpi] Message queue limit > is set to 0 which may lead to potential OOMEs when running cache operations > in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender > and receiver sides. > [04-Mar-2018 12:10:35][WARN ][main][NoopCheckpointSpi] Checkpoints are > disabled (to enable configure any GridCheckpointSpi implementation) > [04-Mar-2018 12:10:35][WARN ][main][GridCollisionManager] Collision > resolution is disabled (all jobs will be activated upon arrival). > [15:10:35] : > [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Error > Output] > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Security status > [authentication=off, tls/ssl=off] > [15:10:35] : > [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Output] > [04-Mar-2018 12:10:35][WARN ][main][IgniteH2Indexing] Custom H2 serialization > is already configured, will override. > [15:10:35] : > [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Error > Output] > [04-03-2018 12:10:35][INFO ][main][ClientListenerProcessor] Client connector > processor has started on TCP port 10800 > [04-03-2018 12:10:35][INFO ][main][GridTcpRestProtocol] Command protocol > successfully started [name=TCP binary, host=0.0.0.0/0.0.0.0, port=11211] > [15:10:35] : > [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Output] > [04-Mar-2018 12:10:35][WARN ][main][IgniteKernal] Hadoop module will not > start due to exception: Failed to resolve Hadoop classpath (please define > HADOOP_HOME environment variable and point it to your Hadoop distribution). > [04-Mar-2018 12:10:35][WARN ][main][PlatformProcessorImpl] Marshaller is > automatically set to o.a.i.i.binary.BinaryMarshaller (other nodes must have > the same marshaller type). > [15:10:35] : > [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Error > Output] > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Non-loopback local IPs: > 172.25.1.165, 192.168.254.1, fe80:0:0:0:255f:a5bb:3481:818%eth3, > fe80:0:0:0:74b3:f73e:adf3:1960%eth2 > [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Enabled local MACs: > 00155D019801, 8215F57D6461 > [04-03-2018 12:10:35][INFO ][main][TcpDiscoverySpi] Successfully bound to TCP > port [port=47500, localHost=0.0.0.0/0.0.0.0, > locNodeId=7cb3b3f7-bd11-4f02-b579-7871ca2063ca] > [15:10:35] : > [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Output] > [04-Mar-2018 12:10:35][WARN ][main][TcpDiscoveryMulticastIpFinder] > TcpDiscoveryMulticastIpFinder has no pre-configured addresses (it is > recommended in production to specify at least one address in > TcpDiscoveryMulticastIpFinder.getAddresses() configuration property) > [04-Mar-2018 12:11:02][WARN ][main][TcpDiscoverySpi] Node has not been > connected to topology and will repeat join process. Check remote nodes logs > for possible error messages. Note that large topology may require significant > time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration > property if getting this message on the starting nodes [networkTimeout=5000] > [15:11:35] : > [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Error > Output] > [04-03-2018 12:11:35][INFO > ][ignite-update-notifier-timer][GridUpdateNotifier] Update status is not > available. > [04-03-2018 12:11:35][INFO > ][ignite-update-notifier-timer][GridUpdateNotifier] Update status is not > available. > [15:16:26] : > [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Output] > [04-Mar-2018 12:16:26][WARN ][main][TcpDiscoverySpi] Node has not been > connected to topology and will repeat join process. Check remote nodes logs > for possible error messages. Note that large topology may require significant > time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration > property if getting this message on the starting nodes [networkTimeout=5000] > [04-Mar-2018 12:22:14][WARN ][main][TcpDiscoverySpi] Node has not been > connected to topology and will repeat join process. Check remote nodes logs > for possible error messages. Note that large topology may require significant > time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration > property if getting this message on the starting nodes [networkTimeout=5000] > [04-Mar-2018 12:27:19][WARN ][main][TcpDiscoverySpi] Node has not been > connected to topology and will repeat join process. Check remote nodes logs > for possible error messages. Note that large topology may require significant > time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration > property if getting this message on the starting nodes [networkTimeout=5000] > [04-Mar-2018 12:32:57][WARN ][main][TcpDiscoverySpi] Node has not been > connected to topology and will repeat join process. Check remote nodes logs > for possible error messages. Note that large topology may require significant > time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration > property if getting this message on the starting nodes [networkTimeout=5000] > [04-Mar-2018 12:39:12][WARN ][main][TcpDiscoverySpi] Node has not been > connected to topology and will repeat join process. Check remote nodes logs > for possible error messages. Note that large topology may require significant > time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration > property if getting this message on the starting nodes [networkTimeout=5000] > [04-Mar-2018 12:44:47][WARN ][main][TcpDiscoverySpi] Node has not been > connected to topology and will repeat join process. Check remote nodes logs > for possible error messages. Note that large topology may require significant > time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration > property if getting this message on the starting nodes [networkTimeout=5000] > [04-Mar-2018 12:50:37][WARN ][main][TcpDiscoverySpi] Node has not been > connected to topology and will repeat join process. Check remote nodes logs > for possible error messages. Note that large topology may require significant > time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration > property if getting this message on the starting nodes [networkTimeout=5000] > [04-Mar-2018 12:57:11][WARN ][main][TcpDiscoverySpi] Node has not been > connected to topology and will repeat join process. Check remote nodes logs > for possible error messages. Note that large topology may require significant > time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration > property if getting this message on the starting nodes [networkTimeout=5000] > [04-Mar-2018 13:02:22][WARN ][main][TcpDiscoverySpi] Node has not been > connected to topology and will repeat join process. Check remote nodes logs > for possible error messages. Note that large topology may require significant > time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration > property if getting this message on the starting nodes [networkTimeout=5000] > [04-Mar-2018 13:07:49][WARN ][main][TcpDiscoverySpi] Node has not been > connected to topology and will repeat join process. Check remote nodes logs > for possible error messages. Note that large topology may require significant > time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration > property if getting this message on the starting nodes [networkTimeout=5000] > [04-Mar-2018 13:13:28][WARN ][main][TcpDiscoverySpi] Node has not been > connected to topology and will repeat join process. Check remote nodes logs > for possible error messages. Note that large topology may require significant > time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration > property if getting this message on the starting nodes [networkTimeout=5000] > [04-Mar-2018 13:19:04][WARN ][main][TcpDiscoverySpi] Node has not been > connected to topology and will repeat join process. Check remote nodes logs > for possible error messages. Note that large topology may require significant > time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration > property if getting this message on the starting nodes [networkTimeout=5000] > [04-Mar-2018 13:24:35][WARN ][main][TcpDiscoverySpi] Node has not been > connected to topology and will repeat join process. Check remote nodes logs > for possible error messages. Note that large topology may require significant > time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration > property if getting this message on the starting nodes [networkTimeout=5000] > [04-Mar-2018 13:29:56][WARN ][main][TcpDiscoverySpi] Node has not been > connected to topology and will repeat join process. Check remote nodes logs > for possible error messages. Note that large topology may require significant > time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration > property if getting this message on the starting nodes [networkTimeout=5000] > [16:30:23]E: [Step 10/12] The build Ignite Tests 2.4+ (Java 8)::Ignite > Platform .NET Long Running #622 {buildId=1123041} has been running for more > than 120 minutes. Terminating... > [16:30:20]E: [Step 10/12] Execution timeout > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)