[
https://issues.apache.org/jira/browse/IGNITE-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitriy Pavlov updated IGNITE-7878:
-----------------------------------
Priority: Critical (was: Major)
> Ignite Platform .NET Long Running: Stable failed with execution timeout
> -------------------------------------------------------------------------
>
> Key: IGNITE-7878
> URL: https://issues.apache.org/jira/browse/IGNITE-7878
> Project: Ignite
> Issue Type: Task
> Reporter: Dmitriy Pavlov
> Priority: Critical
> Labels: MakeTeamcityGreenAgain
>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_IgnitePlatformNetLongRunning&tab=buildTypeHistoryList&branch_IgniteTests24Java8=%3Cdefault%3E
> First failure changes
> https://ci.ignite.apache.org/viewLog.html?buildId=1120409&tab=buildChangesDiv&buildTypeId=IgniteTests24Java8_IgnitePlatformNetLongRunning
> {noformat}
> 15:10:35] : [Apache.Ignite.Core.Tests.exe]
> Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault (1h:19m:50s)
> [15:10:35] :
> [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Error
> Output]
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal]
> >>> __________ ________________
> >>> / _/ ___/ |/ / _/_ __/ __/
> >>> _/ // (7 7 // / / / / _/
> >>> /___/\___/_/|_/___/ /_/ /___/
> >>>
> >>> ver. 2.5.0-SNAPSHOT#19700101-sha1:DEV
> >>> 2018 Copyright(C) Apache Software Foundation
> >>>
> >>> Ignite documentation: http://ignite.apache.org
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Config URL:
> file:/C:/BuildAgent/work/bd85361428dcdb1/modules/platforms/dotnet/Apache.Ignite.Core.Tests/bin/Debug/Config/Compute/compute-grid2.xml
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] IgniteConfiguration
> [igniteInstanceName=null, pubPoolSize=8, svcPoolSize=8, callbackPoolSize=8,
> stripedPoolSize=8, sysPoolSize=8, mgmtPoolSize=4, igfsPoolSize=8,
> dataStreamerPoolSize=8, utilityCachePoolSize=8,
> utilityCacheKeepAliveTime=60000, p2pPoolSize=2, qryPoolSize=8,
> igniteHome=C:\BuildAgent\work\bd85361428dcdb1,
> igniteWorkDir=C:\BuildAgent\work\bd85361428dcdb1\work,
> mbeanSrv=com.sun.jmx.mbeanserver.JmxMBeanServer@5965d37,
> nodeId=7cb3b3f7-bd11-4f02-b579-7871ca2063ca,
> marsh=org.apache.ignite.internal.binary.BinaryMarshaller@f7d76a5,
> marshLocJobs=false, daemon=false, p2pEnabled=false, netTimeout=5000,
> sndRetryDelay=1000, sndRetryCnt=3, metricsHistSize=10000,
> metricsUpdateFreq=2000, metricsExpTime=9223372036854775807,
> discoSpi=TcpDiscoverySpi [addrRslvr=null, sockTimeout=0, ackTimeout=0,
> marsh=null, reconCnt=10, reconDelay=2000, maxAckTimeout=600000,
> forceSrvMode=false, clientReconnectDisabled=false], segPlc=STOP,
> segResolveAttempts=2, waitForSegOnStart=true, allResolversPassReq=true,
> segChkFreq=10000, commSpi=TcpCommunicationSpi [connectGate=null,
> connPlc=null, enableForcibleNodeKill=false, enableTroubleshootingLog=false,
> srvLsnr=org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$2@75d9f436,
> locAddr=null, locHost=null, locPort=47100, locPortRange=100, shmemPort=-1,
> directBuf=true, directSndBuf=false, idleConnTimeout=600000, connTimeout=5000,
> maxConnTimeout=600000, reconCnt=10, sockSndBuf=32768, sockRcvBuf=32768,
> msgQueueLimit=0, slowClientQueueLimit=0, nioSrvr=null, shmemSrv=null,
> usePairedConnections=false, connectionsPerNode=1, tcpNoDelay=true,
> filterReachableAddresses=false, ackSndThreshold=32, unackedMsgsBufSize=0,
> sockWriteTimeout=2000, lsnr=null, boundTcpPort=-1, boundTcpShmemPort=-1,
> selectorsCnt=4, selectorSpins=0, addrRslvr=null,
> ctxInitLatch=java.util.concurrent.CountDownLatch@760bcec6[Count = 1],
> stopping=false,
> metricsLsnr=org.apache.ignite.spi.communication.tcp.TcpCommunicationMetricsListener@51912f3a],
> evtSpi=org.apache.ignite.spi.eventstorage.NoopEventStorageSpi@35bda3f3,
> colSpi=NoopCollisionSpi [], deploySpi=LocalDeploymentSpi [lsnr=null],
> indexingSpi=org.apache.ignite.spi.indexing.noop.NoopIndexingSpi@fff9936,
> addrRslvr=null, clientMode=false, rebalanceThreadPoolSize=1,
> txCfg=org.apache.ignite.configuration.TransactionConfiguration@272a1bf1,
> cacheSanityCheckEnabled=true, discoStartupDelay=60000, deployMode=SHARED,
> p2pMissedCacheSize=100, locHost=null, timeSrvPortBase=31100,
> timeSrvPortRange=100, failureDetectionTimeout=10000,
> clientFailureDetectionTimeout=30000, metricsLogFreq=60000, hadoopCfg=null,
> connectorCfg=org.apache.ignite.configuration.ConnectorConfiguration@7f9a18d1,
> odbcCfg=null, warmupClos=null, atomicCfg=AtomicConfiguration
> [seqReserveSize=1000, cacheMode=PARTITIONED, backups=1, aff=null,
> grpName=null], classLdr=null, sslCtxFactory=null,
> platformCfg=PlatformDotNetConfiguration [binaryCfg=null], binaryCfg=null,
> memCfg=null, pstCfg=null, dsCfg=DataStorageConfiguration
> [sysRegionInitSize=41943040, sysCacheMaxSize=104857600, pageSize=0,
> concLvl=0, dfltDataRegConf=DataRegionConfiguration [name=default,
> maxSize=5153866547, initSize=268435456, swapPath=null,
> pageEvictionMode=DISABLED, evictionThreshold=0.9, emptyPagesPoolSize=100,
> metricsEnabled=false, metricsSubIntervalCount=5,
> metricsRateTimeInterval=60000, persistenceEnabled=false,
> checkpointPageBufSize=0], storagePath=null, checkpointFreq=180000,
> lockWaitTime=10000, checkpointThreads=4, checkpointWriteOrder=SEQUENTIAL,
> walHistSize=20, walSegments=10, walSegmentSize=67108864, walPath=db/wal,
> walArchivePath=db/wal/archive, metricsEnabled=false, walMode=LOG_ONLY,
> walTlbSize=131072, walBuffSize=0, walFlushFreq=2000, walFsyncDelay=1000,
> walRecordIterBuffSize=67108864, alwaysWriteFullPages=false,
> fileIOFactory=org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIOFactory@27e09eea,
> metricsSubIntervalCnt=5, metricsRateTimeInterval=60000,
> walAutoArchiveAfterInactivity=-1, writeThrottlingEnabled=false,
> walCompactionEnabled=false], activeOnStart=true, autoActivation=true,
> longQryWarnTimeout=3000, sqlConnCfg=null,
> cliConnCfg=ClientConnectorConfiguration [host=null, port=10800,
> portRange=100, sockSndBufSize=0, sockRcvBufSize=0, tcpNoDelay=true,
> maxOpenCursorsPerConn=128, threadPoolSize=8, idleTimeout=0, jdbcEnabled=true,
> odbcEnabled=true, thinCliEnabled=true, sslEnabled=false,
> useIgniteSslCtxFactory=true, sslClientAuth=false, sslCtxFactory=null]]
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Daemon mode: off
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] OS: Windows 10 10.0 amd64
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] OS user: PUBLICAGENT02_0$
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] PID: 13396
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Language runtime: Java
> Platform API Specification ver. 1.8
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] VM information: Java(TM) SE
> Runtime Environment 1.8.0_161-b12 Oracle Corporation Java HotSpot(TM) 64-Bit
> Server VM 25.161-b12
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] VM total memory: 3.6GB
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Remote Management [restart:
> off, REST: on, JMX (remote: off)]
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Logger: Log4JLogger
> [quiet=false, config=null]
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal]
> IGNITE_HOME=C:\BuildAgent\work\bd85361428dcdb1
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] VM arguments:
> [-XX:+HeapDumpOnOutOfMemoryError, -Xms1g, -Xmx4g, -ea, -DIGNITE_QUIET=true,
> -Duser.timezone=UTC, -Xdebug, -Xnoagent, -Djava.compiler=NONE,
> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005]
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] System cache's DataRegion
> size is configured to 40 MB. Use
> DataStorageConfiguration.systemCacheMemorySize property to change the setting.
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Configured caches [in
> 'sysMemPlc' dataRegion: ['ignite-sys-cache', 'ignite-hadoop-mr-sys-cache']]
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] 3-rd party licenses can be
> found at: C:\BuildAgent\work\bd85361428dcdb1\libs\licenses
> [15:10:35] :
> [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Output]
> [04-Mar-2018 12:10:35][WARN ][pub-#11985][GridDiagnostic] This operating
> system has been tested less rigorously: Windows 10 10.0 amd64. Our team will
> appreciate the feedback if you experience any problems running ignite in this
> environment.
> [15:10:35] :
> [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Error
> Output]
> [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] Configured plugins:
> [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] ^--
> TestReconnectPlugin 1.0
> [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] ^--
> [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor]
> [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] ^--
> StanByClusterTestProvider 1.0
> [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] ^-- null
> [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor]
> [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] ^-- Ignite Native
> I/O Plugin [Direct I/O]
> [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] ^-- Copyright(C)
> Apache Software Foundation
> [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor]
> [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] ^--
> TestPlatformPlugin 1.0
> [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor] ^-- -
> [04-03-2018 12:10:35][INFO ][main][IgnitePluginProcessor]
> [04-03-2018 12:10:35][INFO ][main][TcpCommunicationSpi] Successfully bound
> communication NIO server to TCP port [port=47100, locHost=0.0.0.0/0.0.0.0,
> selectorsCnt=4, selectorSpins=0, pairedConn=false]
> [15:10:35] :
> [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Output]
> [04-Mar-2018 12:10:35][WARN ][main][TcpCommunicationSpi] Message queue limit
> is set to 0 which may lead to potential OOMEs when running cache operations
> in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender
> and receiver sides.
> [04-Mar-2018 12:10:35][WARN ][main][NoopCheckpointSpi] Checkpoints are
> disabled (to enable configure any GridCheckpointSpi implementation)
> [04-Mar-2018 12:10:35][WARN ][main][GridCollisionManager] Collision
> resolution is disabled (all jobs will be activated upon arrival).
> [15:10:35] :
> [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Error
> Output]
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Security status
> [authentication=off, tls/ssl=off]
> [15:10:35] :
> [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Output]
> [04-Mar-2018 12:10:35][WARN ][main][IgniteH2Indexing] Custom H2 serialization
> is already configured, will override.
> [15:10:35] :
> [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Error
> Output]
> [04-03-2018 12:10:35][INFO ][main][ClientListenerProcessor] Client connector
> processor has started on TCP port 10800
> [04-03-2018 12:10:35][INFO ][main][GridTcpRestProtocol] Command protocol
> successfully started [name=TCP binary, host=0.0.0.0/0.0.0.0, port=11211]
> [15:10:35] :
> [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Output]
> [04-Mar-2018 12:10:35][WARN ][main][IgniteKernal] Hadoop module will not
> start due to exception: Failed to resolve Hadoop classpath (please define
> HADOOP_HOME environment variable and point it to your Hadoop distribution).
> [04-Mar-2018 12:10:35][WARN ][main][PlatformProcessorImpl] Marshaller is
> automatically set to o.a.i.i.binary.BinaryMarshaller (other nodes must have
> the same marshaller type).
> [15:10:35] :
> [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Error
> Output]
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Non-loopback local IPs:
> 172.25.1.165, 192.168.254.1, fe80:0:0:0:255f:a5bb:3481:818%eth3,
> fe80:0:0:0:74b3:f73e:adf3:1960%eth2
> [04-03-2018 12:10:35][INFO ][main][IgniteKernal] Enabled local MACs:
> 00155D019801, 8215F57D6461
> [04-03-2018 12:10:35][INFO ][main][TcpDiscoverySpi] Successfully bound to TCP
> port [port=47500, localHost=0.0.0.0/0.0.0.0,
> locNodeId=7cb3b3f7-bd11-4f02-b579-7871ca2063ca]
> [15:10:35] :
> [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Output]
> [04-Mar-2018 12:10:35][WARN ][main][TcpDiscoveryMulticastIpFinder]
> TcpDiscoveryMulticastIpFinder has no pre-configured addresses (it is
> recommended in production to specify at least one address in
> TcpDiscoveryMulticastIpFinder.getAddresses() configuration property)
> [04-Mar-2018 12:11:02][WARN ][main][TcpDiscoverySpi] Node has not been
> connected to topology and will repeat join process. Check remote nodes logs
> for possible error messages. Note that large topology may require significant
> time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration
> property if getting this message on the starting nodes [networkTimeout=5000]
> [15:11:35] :
> [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Error
> Output]
> [04-03-2018 12:11:35][INFO
> ][ignite-update-notifier-timer][GridUpdateNotifier] Update status is not
> available.
> [04-03-2018 12:11:35][INFO
> ][ignite-update-notifier-timer][GridUpdateNotifier] Update status is not
> available.
> [15:16:26] :
> [Apache.Ignite.Core.Tests.IgniteStartStopTest.TestStartDefault] [Test Output]
> [04-Mar-2018 12:16:26][WARN ][main][TcpDiscoverySpi] Node has not been
> connected to topology and will repeat join process. Check remote nodes logs
> for possible error messages. Note that large topology may require significant
> time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration
> property if getting this message on the starting nodes [networkTimeout=5000]
> [04-Mar-2018 12:22:14][WARN ][main][TcpDiscoverySpi] Node has not been
> connected to topology and will repeat join process. Check remote nodes logs
> for possible error messages. Note that large topology may require significant
> time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration
> property if getting this message on the starting nodes [networkTimeout=5000]
> [04-Mar-2018 12:27:19][WARN ][main][TcpDiscoverySpi] Node has not been
> connected to topology and will repeat join process. Check remote nodes logs
> for possible error messages. Note that large topology may require significant
> time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration
> property if getting this message on the starting nodes [networkTimeout=5000]
> [04-Mar-2018 12:32:57][WARN ][main][TcpDiscoverySpi] Node has not been
> connected to topology and will repeat join process. Check remote nodes logs
> for possible error messages. Note that large topology may require significant
> time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration
> property if getting this message on the starting nodes [networkTimeout=5000]
> [04-Mar-2018 12:39:12][WARN ][main][TcpDiscoverySpi] Node has not been
> connected to topology and will repeat join process. Check remote nodes logs
> for possible error messages. Note that large topology may require significant
> time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration
> property if getting this message on the starting nodes [networkTimeout=5000]
> [04-Mar-2018 12:44:47][WARN ][main][TcpDiscoverySpi] Node has not been
> connected to topology and will repeat join process. Check remote nodes logs
> for possible error messages. Note that large topology may require significant
> time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration
> property if getting this message on the starting nodes [networkTimeout=5000]
> [04-Mar-2018 12:50:37][WARN ][main][TcpDiscoverySpi] Node has not been
> connected to topology and will repeat join process. Check remote nodes logs
> for possible error messages. Note that large topology may require significant
> time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration
> property if getting this message on the starting nodes [networkTimeout=5000]
> [04-Mar-2018 12:57:11][WARN ][main][TcpDiscoverySpi] Node has not been
> connected to topology and will repeat join process. Check remote nodes logs
> for possible error messages. Note that large topology may require significant
> time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration
> property if getting this message on the starting nodes [networkTimeout=5000]
> [04-Mar-2018 13:02:22][WARN ][main][TcpDiscoverySpi] Node has not been
> connected to topology and will repeat join process. Check remote nodes logs
> for possible error messages. Note that large topology may require significant
> time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration
> property if getting this message on the starting nodes [networkTimeout=5000]
> [04-Mar-2018 13:07:49][WARN ][main][TcpDiscoverySpi] Node has not been
> connected to topology and will repeat join process. Check remote nodes logs
> for possible error messages. Note that large topology may require significant
> time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration
> property if getting this message on the starting nodes [networkTimeout=5000]
> [04-Mar-2018 13:13:28][WARN ][main][TcpDiscoverySpi] Node has not been
> connected to topology and will repeat join process. Check remote nodes logs
> for possible error messages. Note that large topology may require significant
> time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration
> property if getting this message on the starting nodes [networkTimeout=5000]
> [04-Mar-2018 13:19:04][WARN ][main][TcpDiscoverySpi] Node has not been
> connected to topology and will repeat join process. Check remote nodes logs
> for possible error messages. Note that large topology may require significant
> time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration
> property if getting this message on the starting nodes [networkTimeout=5000]
> [04-Mar-2018 13:24:35][WARN ][main][TcpDiscoverySpi] Node has not been
> connected to topology and will repeat join process. Check remote nodes logs
> for possible error messages. Note that large topology may require significant
> time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration
> property if getting this message on the starting nodes [networkTimeout=5000]
> [04-Mar-2018 13:29:56][WARN ][main][TcpDiscoverySpi] Node has not been
> connected to topology and will repeat join process. Check remote nodes logs
> for possible error messages. Note that large topology may require significant
> time to start. Increase 'TcpDiscoverySpi.networkTimeout' configuration
> property if getting this message on the starting nodes [networkTimeout=5000]
> [16:30:23]E: [Step 10/12] The build Ignite Tests 2.4+ (Java 8)::Ignite
> Platform .NET Long Running #622 {buildId=1123041} has been running for more
> than 120 minutes. Terminating...
> [16:30:20]E: [Step 10/12] Execution timeout
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
