If you want to bring your cluster into a valid state again, you can remove the WAL and db directories and restart nodes. But you will lose all data in this case, obviously.
What configuration properties did you change? Denis вт, 3 июл. 2018 г. в 4:23, 剑剑 <[email protected]>: > The node has not happened fault, is I modify configuration problem appears > after the restart, now I need to how to correct the nodes offline, and then > again in the form of a new node to join the cluster? All node profiles for > the cluster are the same. > > 发自我的 iPhone > > 在 2018年7月3日，00:16，Denis Mekhanikov <[email protected]> 写道： > > Looks like your persistence files are corrupted. > You configured *LOG_ONLY* WAL mode. It doesn't guarantee survival of OS > crushes and power failures. > How did you restart your node? > > Denis > > пн, 2 июл. 2018 г. в 16:40, NO <[email protected]>: > >> When I restart the node, I get the following error, >> The problem persists after restarting the machine。 >> >> ========================================================================== >> [2018-07-02T21:25:52,932][INFO >> ][exchange-worker-#190][GridCacheDatabaseSharedManager] Read checkpoint >> status >> [startMarker=/data3/apache-ignite-persistence/node00-8c6172fa-0543-4b8d-937e-75ac27ba21ff/cp/1530535766680-f62c2aa7-4a26-45ad-b311-5b5e9ddc3f0e-START.bin, >> endMarker=/data3/apache-ignite-persistence/node00-8c6172fa-0543-4b8d-937e-75ac27ba21ff/cp/1530535612596-2ccb2f7a-9578-44a7-ad29-ff5d6e990ae4-END.bin] >> [2018-07-02T21:25:52,933][INFO >> ][exchange-worker-#190][GridCacheDatabaseSharedManager] Checking memory >> state [lastValidPos=FileWALPointer [idx=845169, fileOff=32892207, >> len=7995], lastMarked=FileWALPointer [idx=845199, fileOff=43729777, >> len=7995], lastCheckpointId=f62c2aa7-4a26-45ad-b311-5b5e9ddc3f0e] >> [2018-07-02T21:25:52,933][WARN >> ][exchange-worker-#190][GridCacheDatabaseSharedManager] Ignite node stopped >> in the middle of checkpoint. Will restore memory state and finish >> checkpoint on node start. >> [2018-07-02T21:25:52,949][INFO >> ][grid-nio-worker-tcp-comm-0-#153][TcpCommunicationSpi] Accepted incoming >> communication connection [locAddr=/10.16.133.187:47100, rmtAddr=/ >> 10.16.133.186:22315] >> [2018-07-02T21:25:53,131][INFO >> ][grid-nio-worker-tcp-comm-1-#154][TcpCommunicationSpi] Accepted incoming >> communication connection [locAddr=/10.16.133.187:47100, rmtAddr=/ >> 10.16.133.185:32502] >> [2018-07-02T21:25:56,112][ERROR][exchange-worker-#190][GridDhtPartitionsExchangeFuture] >> Failed to reinitialize local partitions (preloading will be stopped): >> GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=4917, >> minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode >> [id=3c06c945-de21-4b7f-8830-344306327643, addrs=[10.16.133.187, 127.0.0.1], >> sockAddrs=[/127.0.0.1:47500, /10.16.133.187:47500], discPort=47500, >> order=4917, intOrder=2496, lastExchangeTime=1530537954950, loc=true, >> ver=2.4.0#20180305-sha1:aa342270, isClient=false], topVer=4917, >> nodeId8=3c06c945, msg=null, type=NODE_JOINED, tstamp=1530537952291], >> nodeId=3c06c945, evt=NODE_JOINED] >> org.apache.ignite.IgniteCheckedException: Unknown page IO type: 0 >> at >> org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getBPlusIO(PageIO.java:567) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getPageIO(PageIO.java:478) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getPageIO(PageIO.java:438) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.pagemem.wal.record.delta.DataPageInsertFragmentRecord.applyDelta(DataPageInsertFragmentRecord.java:58) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1967) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1827) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readCheckpointAndRestoreMemory(GridCacheDatabaseSharedManager.java:725) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:741) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:626) >> [ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2337) >> [ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) >> [ignite-core-2.4.0.jar:2.4.0] >> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_45] >> [2018-07-02T21:25:56,116][INFO >> ][exchange-worker-#190][GridDhtPartitionsExchangeFuture] Finish exchange >> future [startVer=AffinityTopologyVersion [topVer=4917, minorTopVer=0], >> resVer=null, err=class org.apache.ignite.IgniteCheckedException: Unknown >> page IO type: 0] >> [2018-07-02T21:25:56,117][ERROR][main][IgniteKernal] Got exception while >> starting (will rollback startup routine). >> org.apache.ignite.IgniteCheckedException: Unknown page IO type: 0 >> at >> org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getBPlusIO(PageIO.java:567) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getPageIO(PageIO.java:478) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getPageIO(PageIO.java:438) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.pagemem.wal.record.delta.DataPageInsertFragmentRecord.applyDelta(DataPageInsertFragmentRecord.java:58) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1967) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1827) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readCheckpointAndRestoreMemory(GridCacheDatabaseSharedManager.java:725) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:741) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:626) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2337) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_45] >> [2018-07-02T21:25:56,120][ERROR][exchange-worker-#190][GridCachePartitionExchangeManager] >> Failed to wait for completion of partition map exchange (preloading will >> not start): GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent >> [evtNode=TcpDiscoveryNode [id=3c06c945-de21-4b7f-8830-344306327643, >> addrs=[10.16.133.187, 127.0.0.1], sockAddrs=[/127.0.0.1:47500, / >> 10.16.133.187:47500], discPort=47500, order=4917, intOrder=2496, >> lastExchangeTime=1530537954950, loc=true, ver=2.4.0#20180305-sha1:aa342270, >> isClient=false], topVer=4917, nodeId8=3c06c945, msg=null, type=NODE_JOINED, >> tstamp=1530537952291], crd=TcpDiscoveryNode >> [id=a8bf945b-981c-46f6-baae-b31626755e09, addrs=[10.16.133.188, 127.0.0.1], >> sockAddrs=[/10.16.133.188:47500, /127.0.0.1:47500], discPort=47500, >> order=3093, intOrder=1555, lastExchangeTime=1530537952080, loc=false, >> ver=2.4.0#20180305-sha1:aa342270, isClient=false], >> exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion >> [topVer=4917, minorTopVer=0], discoEvt=DiscoveryEvent >> [evtNode=TcpDiscoveryNode [id=3c06c945-de21-4b7f-8830-344306327643, >> addrs=[10.16.133.187, 127.0.0.1], sockAddrs=[/127.0.0.1:47500, / >> 10.16.133.187:47500], discPort=47500, order=4917, intOrder=2496, >> lastExchangeTime=1530537954950, loc=true, ver=2.4.0#20180305-sha1:aa342270, >> isClient=false], topVer=4917, nodeId8=3c06c945, msg=null, type=NODE_JOINED, >> tstamp=1530537952291], nodeId=3c06c945, evt=NODE_JOINED], added=true, >> initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=false, >> hash=1338265304], init=false, lastVer=null, partReleaseFut=null, >> exchActions=null, affChangeMsg=null, initTs=1530537952341, >> centralizedAff=false, changeGlobalStateE=null, done=true, state=SRV, >> evtLatch=0, remaining=[a8fdbf0a-6956-40ad-b0bd-ca85bc1e3668, >> f24a14f3-8720-4723-9130-0af8803abd5d, 25e9bb3d-6a64-4463-bcc9-fb413b2fa231, >> 311efb23-c4ad-49a3-9356-f2423538b319, 7851df66-cbd6-4a94-8acd-5d86437ea4d8, >> 12aaa4f7-0a3f-4b3f-842f-964837f6cae5, a8bf945b-981c-46f6-baae-b31626755e09, >> c77aef15-7234-4b16-9bdb-09fc6ce26422, >> 1a9f815c-8b2a-41c8-af99-70bdf0706e7e], super=GridFutureAdapter >> [ignoreInterrupts=false, state=DONE, res=class >> o.a.i.IgniteCheckedException: Unknown page IO type: 0, hash=454207607]] >> org.apache.ignite.IgniteCheckedException: Unknown page IO type: 0 >> at >> org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getBPlusIO(PageIO.java:567) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getPageIO(PageIO.java:478) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getPageIO(PageIO.java:438) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.pagemem.wal.record.delta.DataPageInsertFragmentRecord.applyDelta(DataPageInsertFragmentRecord.java:58) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1967) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1827) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readCheckpointAndRestoreMemory(GridCacheDatabaseSharedManager.java:725) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:741) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:626) >> ~[ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2337) >> [ignite-core-2.4.0.jar:2.4.0] >> at >> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) >> [ignite-core-2.4.0.jar:2.4.0] >> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_45] >> >> ========================================================================== >> >> config.xml >> >> ============================================================================================ >> <?xml version="1.0" encoding="UTF-8"?> >> <beans xmlns="http://www.springframework.org/schema/beans"; >> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >> xmlns:util="http://www.springframework.org/schema/util"; >> xsi:schemaLocation="http://www.springframework.org/schema/beans >> http://www.springframework.org/schema/beans/spring-beans.xsd >> http://www.springframework.org/schema/util >> http://www.springframework.org/schema/util/spring-util.xsd >> "> >> <bean id="ignite.cfg" >> class="org.apache.ignite.configuration.IgniteConfiguration"> >> >> <property name="failureDetectionTimeout" value="60000"/> >> >> <property name="clientFailureDetectionTimeout" value="60000"/> >> >> <property name="segmentationPolicy" value="RESTART_JVM"/> >> >> <!-- <property name="publicThreadPoolSize" value="192"/> --> >> >> <!-- <property name="systemThreadPoolSize" value="192"/> --> >> >> <property name="dataStreamerThreadPoolSize" value="96"/> >> >> <property name="rebalanceThreadPoolSize" value="4" /> >> >> <property name="dataStorageConfiguration"> >> <bean >> class="org.apache.ignite.configuration.DataStorageConfiguration"> >> <property name="defaultDataRegionConfiguration"> >> <bean >> class="org.apache.ignite.configuration.DataRegionConfiguration"> >> <property name="name" >> value="qipu_entity_cache_data_region"/> >> >> <property name="initialSize" value="#{10L * 1024 >> * 1024 * 1024}"/> >> >> <property name="maxSize" value="#{100L * 1024 * >> 1024 * 1024}"/> >> >> <property name="persistenceEnabled" value="true"/> >> >> <property name="metricsEnabled" value="true"/> >> >> <property name="checkpointPageBufferSize" >> value="#{1 * 1024 * 1024 * 1024}"/> >> </bean> >> </property> >> >> <property name="storagePath" >> value="/data3/apache-ignite-persistence"/> >> >> <property name="walPath" >> value="/data2/apache-ignite-wal"/> >> >> <property name="walArchivePath" >> value="/data2/apache-ignite-wal/archive"/> >> >> <property name="walSegmentSize" value="#{64 * 1024 * >> 1024}"/> >> >> <property name="pageSize" value="#{4 * 1024}"/> >> >> <property name="walSegments" value="#{20}"/> >> >> <property name="walMode" value="LOG_ONLY"/> >> >> <property name="metricsEnabled" value="true"/> >> >> <property name="writeThrottlingEnabled" value="false"/> >> >> <property name="checkpointThreads" value="8"/> >> >> <property name="walThreadLocalBufferSize" value="#{1 * >> 1024 * 1024}"/> >> </bean> >> </property> >> >> <property name="cacheConfiguration"> >> <bean >> class="org.apache.ignite.configuration.CacheConfiguration"> >> <property name="dataRegionName" >> value="qipu_entity_cache_data_region"/> >> >> <property name="name" value="qipu_entity_cache"/> >> >> <property name="cacheMode" value="PARTITIONED"/> >> >> <property name="partitionLossPolicy" value="IGNORE"/> >> >> <property name="atomicityMode" value="ATOMIC"/> >> >> <property name="backups" value="2"/> >> >> <property name="writeSynchronizationMode" >> value="FULL_SYNC"/> >> >> <property name="statisticsEnabled" value="true"/> >> >> <property name="rebalanceBatchSize" value="#{20 * 1024 * >> 1024}"/> >> >> <property name="rebalanceThrottle" value="0"/> >> >> <property name="rebalanceMode" value="ASYNC"/> >> >> <property name="rebalanceBatchesPrefetchCount" value="4"/> >> >> <property name="rebalanceTimeout" value="20000"/> >> >> <property name="maxConcurrentAsyncOperations" value="#{4 >> * 500}"/> >> </bean> >> </property> >> >> <property name="communicationSpi"> >> <bean >> class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi"> >> <property name="socketWriteTimeout" value="6000"/> >> <property name="messageQueueLimit" value="102400"/> >> </bean> >> </property> >> >> <property name="discoverySpi"> >> <bean >> class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> >> <property name="forceServerMode" value="true"/> >> <property name="ipFinder"> >> <bean >> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder"> >> <property name="addresses"> >> <list> >> <!-- In distributed environment, replace >> with actual host IP address. --> >> <value>10.16.133.179:47500..47509</value> >> <value>10.16.133.180:47500..47509</value> >> <value>10.16.133.181:47500..47509</value> >> <value>10.16.133.182:47500..47509</value> >> <value>10.16.133.183:47500..47509</value> >> <value>10.16.133.184:47500..47509</value> >> <value>10.16.133.185:47500..47509</value> >> <value>10.16.133.186:47500..47509</value> >> <value>10.16.133.187:47500..47509</value> >> <value>10.16.133.188:47500..47509</value> >> </list> >> </property> >> </bean> >> </property> >> </bean> >> </property> >> <property name="gridLogger"> >> <bean class="org.apache.ignite.logger.log4j2.Log4J2Logger"> >> <constructor-arg type="java.lang.String" >> value="/config/ignite-log4j2.xml"/> >> </bean> >> </property> >> </bean> >> </beans> >> >> >> ============================================================================================ >> >> The current node is unable to provide services. Please help check where >> the problem is. thank you very much。 >> >>
