Hi Mimmo, Local node's value of 'java.net.preferIPv4Stack' system property differs from remote node's (all nodes in topology should have identical value) [locPreferIpV4=true, rmtPreferIpV4=null, locId8=2df6b461, rmtId8=927af8bb, rmtAddrs=[slave-1/0:0:0:0:0:0:0:1%lo, /127.0.0.1, /192.168.30.22]]
Run both nodes with java.net.preferIPv4Stack=true option Evgenii 2017-06-23 12:51 GMT+03:00 Michael Cherkasov <[email protected]>: > Hi Mimmo, > > Could you please run all nodes with -DIGNITE_QUIET=false flag and send > logs from both, server and client nodes? > > Does it work for you if you run only 1 server node and 1 client node from > the same host? I think it's better to check this simple > configuration first, so to be sure that there's no network problem. > > Thanks, > Mikhail. > > > > 2017-06-21 17:32 GMT+03:00 mimmo_c <[email protected]>: > >> Hi, I'm trying to configure Ignite for hadoop. I would like to store the >> data >> of a wordcount inside a grid cache between two server node. I'm on a >> Cluster >> of 33 nodes but i want try with 3 node. >> When I start two node in server mode they connect itselfef with ignite.sh >> <hadoop/default-configuration.xml>. From the master node I run hadoop >> with >> the jar of wordcount in wich just start ignite in client mode and try to >> start a cache . For configuration i use default-configuration.xml in >> config/hadoop. >> >> When i start hadoop from master it gives this error >> >> >> >>> __________ ________________ >> >>> / _/ ___/ |/ / _/_ __/ __/ >> >>> _/ // (7 7 // / / / / _/ >> >>> /___/\___/_/|_/___/ /_/ /___/ >> >>> >> >>> ver. 2.0.0#20170430-sha1:d4eef3c6 >> >>> 2017 Copyright(C) Apache Software Foundation >> >>> >> >>> Ignite documentation: http://ignite.apache.org >> >> 2017-06-21 13:53:47,737 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - Config URL: >> file:/home/hduser/apache-ignite-2.0.0-src/config/hadoop/ >> default-config.xml >> 2017-06-21 13:53:47,737 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - Daemon mode: off >> 2017-06-21 13:53:47,737 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - OS: Linux 4.4.0-79-generic amd64 >> 2017-06-21 13:53:47,737 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - OS user: hduser >> 2017-06-21 13:53:47,739 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - PID: 22611 >> 2017-06-21 13:53:47,739 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - Language runtime: Java Platform API >> Specification ver. 1.8 >> 2017-06-21 13:53:47,739 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - VM information: Java(TM) SE Runtime >> Environment 1.8.0_131-b11 Oracle Corporation Java HotSpot(TM) 64-Bit >> Server >> VM 25.131-b11 >> 2017-06-21 13:53:47,740 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - VM total memory: 0.87GB >> 2017-06-21 13:53:47,740 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - Remote Management [restart: off, REST: off, >> JMX (remote: off)] >> 2017-06-21 13:53:47,740 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - >> IGNITE_HOME=/home/hduser/apache-ignite-2.0.0-src >> 2017-06-21 13:53:47,741 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - VM arguments: [-Xmx1000m, >> -Dhadoop.log.dir=/usr/local/hadoop/logs, -Dhadoop.log.file=hadoop.log, >> -Dhadoop.home.dir=/usr/local/hadoop, -Dhadoop.id.str=, >> -Dhadoop.root.logger=INFO,console, >> -Djava.library.path=/usr/local/hadoop/lib/native, >> -Dhadoop.policy.file=hadoop-policy.xml, -Djava.net.preferIPv4Stack=true, >> -Dhadoop.security.logger=INFO,NullAppender] >> 2017-06-21 13:53:47,741 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - Configured caches [in 'default' >> memoryPolicy: >> ['ignite-sys-cache', 'ignite-hadoop-mr-sys-cache', >> 'ignite-atomics-sys-cache', 'igfs-internal-igfs-meta', >> 'igfs-internal-igfs-data']] >> 2017-06-21 13:53:47,744 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - 3-rd party licenses can be found at: >> /home/hduser/apache-ignite-2.0.0-src/libs/licenses >> 2017-06-21 13:53:47,746 WARN [pub-#14%null%] internal.GridDiagnostic >> (Log4JLogger.java:warning(480)) - Initial heap size is 504MB (should be >> no >> less than 512MB, use -Xms512m -Xmx512m). >> 2017-06-21 13:53:47,794 INFO [main] plugin.IgnitePluginProcessor >> (Log4JLogger.java:info(475)) - Configured plugins: >> 2017-06-21 13:53:47,794 INFO [main] plugin.IgnitePluginProcessor >> (Log4JLogger.java:info(475)) - ^-- None >> 2017-06-21 13:53:47,794 INFO [main] plugin.IgnitePluginProcessor >> (Log4JLogger.java:info(475)) - >> 2017-06-21 13:53:47,842 INFO [main] tcp.TcpCommunicationSpi >> (Log4JLogger.java:info(475)) - Successfully bound communication NIO server >> to TCP port [port=47100, locHost=0.0.0.0/0.0.0.0, selectorsCnt=4, >> selectorSpins=0, pairedConn=false] >> 2017-06-21 13:53:47,846 WARN [main] tcp.TcpCommunicationSpi >> (Log4JLogger.java:warning(480)) - Message queue limit is set to 0 which >> may >> lead to potential OOMEs when running cache operations in FULL_ASYNC or >> PRIMARY_SYNC modes due to message queues growth on sender and receiver >> sides. >> 2017-06-21 13:53:47,864 WARN [main] noop.NoopCheckpointSpi >> (Log4JLogger.java:warning(480)) - Checkpoints are disabled (to enable >> configure any GridCheckpointSpi implementation) >> 2017-06-21 13:53:47,889 WARN [main] collision.GridCollisionManager >> (Log4JLogger.java:warning(480)) - Collision resolution is disabled (all >> jobs >> will be activated upon arrival). >> 2017-06-21 13:53:47,890 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - Security status [authentication=off, >> tls/ssl=off] >> 2017-06-21 13:53:48,181 INFO [main] rest.GridRestProcessor >> (Log4JLogger.java:info(475)) - REST protocols do not start on client node. >> To start the protocols on client node set >> '-DIGNITE_REST_START_ON_CLIENT=true' system property. >> 2017-06-21 13:53:48,230 INFO [main] loopback.IpcServerTcpEndpoint >> (Log4JLogger.java:info(475)) - IPC server loopback endpoint started >> [port=10500] >> 2017-06-21 13:53:48,232 INFO [main] loopback.IpcServerTcpEndpoint >> (Log4JLogger.java:info(475)) - IPC server loopback endpoint started >> [port=11400] >> 2017-06-21 13:53:48,246 INFO [main] hadoop.HadoopProcessor >> (Log4JLogger.java:info(475)) - HADOOP_HOME is set to /usr/local/hadoop >> 2017-06-21 13:53:48,246 INFO [main] hadoop.HadoopProcessor >> (Log4JLogger.java:info(475)) - Resolved Hadoop classpath locations: >> /usr/local/hadoop, /usr/local/hadoop, /usr/local/hadoop >> 2017-06-21 13:53:48,248 WARN [main] hadoop.HadoopProcessor >> (Log4JLogger.java:warning(480)) - Hadoop libraries are found in Ignite >> classpath, this could lead to class loading errors (please remove all >> Hadoop >> libraries from Ignite classpath) >> [path=file:/usr/local/hadoop/share/hadoop/common/hadoop-common-2.7.3.jar] >> 2017-06-21 13:53:48,281 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - Non-loopback local IPs: 192.168.30.5 >> 2017-06-21 13:53:48,281 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - Enabled local MACs: FA163E27B794 >> 2017-06-21 13:53:48,394 WARN [main] discovery.GridDiscoveryManager >> (Log4JLogger.java:warning(480)) - Local node's value of >> 'java.net.preferIPv4Stack' system property differs from remote node's (all >> nodes in topology should have identical value) [locPreferIpV4=true, >> rmtPreferIpV4=null, locId8=2df6b461, rmtId8=927af8bb, >> rmtAddrs=[slave-1/0:0:0:0:0:0:0:1%lo, /127.0.0.1, /192.168.30.22]] >> 2017-06-21 13:53:48,596 INFO [main] cache.GridCacheProcessor >> (Log4JLogger.java:info(475)) - Started cache >> [name=ignite-hadoop-mr-sys-cache, memoryPolicyName=sysMemPlc, >> mode=REPLICATED] >> 2017-06-21 13:53:48,607 INFO [main] cache.GridCacheProcessor >> (Log4JLogger.java:info(475)) - Started cache [name=ignite-sys-cache, >> memoryPolicyName=sysMemPlc, mode=REPLICATED] >> 2017-06-21 13:53:48,609 INFO [main] cache.GridCacheProcessor >> (Log4JLogger.java:info(475)) - Started cache >> [name=ignite-atomics-sys-cache, >> memoryPolicyName=sysMemPlc, mode=PARTITIONED] >> 2017-06-21 13:53:48,618 INFO [main] cache.GridCacheProcessor >> (Log4JLogger.java:info(475)) - Started cache >> [name=igfs-internal-igfs-data, >> memoryPolicyName=sysMemPlc, mode=PARTITIONED] >> 2017-06-21 13:53:48,620 INFO [main] cache.GridCacheProcessor >> (Log4JLogger.java:info(475)) - Started cache >> [name=igfs-internal-igfs-meta, >> memoryPolicyName=sysMemPlc, mode=PARTITIONED] >> 2017-06-21 13:53:50,666 WARN [exchange-worker-#25%null%] >> dht.GridDhtAssignmentFetchFuture (Log4JLogger.java:warning(480)) - >> Failed to >> request affinity assignment from remote node (will continue to another >> node): TcpDiscoveryNode [id=927af8bb-e105-4332-baf3-072f921d09cd, >> addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 192.168.30.22], >> sockAddrs=[0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500, >> slave-1/192.168.30.22:47500], discPort=47500, order=1, intOrder=1, >> lastExchangeTime=1498053228353, loc=false, ver=2.0.0#20170430-sha1:d4eef3 >> c6, >> isClient=false] >> 2017-06-21 13:53:52,686 WARN [exchange-worker-#25%null%] >> dht.GridDhtAssignmentFetchFuture (Log4JLogger.java:warning(480)) - >> Failed to >> request affinity assignment from remote node (will continue to another >> node): TcpDiscoveryNode [id=927af8bb-e105-4332-baf3-072f921d09cd, >> addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 192.168.30.22], >> sockAddrs=[0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500, >> slave-1/192.168.30.22:47500], discPort=47500, order=1, intOrder=1, >> lastExchangeTime=1498053228353, loc=false, ver=2.0.0#20170430-sha1:d4eef3 >> c6, >> isClient=false] >> 2017-06-21 13:53:54,706 WARN [exchange-worker-#25%null%] >> dht.GridDhtAssignmentFetchFuture (Log4JLogger.java:warning(480)) - >> Failed to >> request affinity assignment from remote node (will continue to another >> node): TcpDiscoveryNode [id=927af8bb-e105-4332-baf3-072f921d09cd, >> addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 192.168.30.22], >> sockAddrs=[0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500, >> slave-1/192.168.30.22:47500], discPort=47500, order=1, intOrder=1, >> lastExchangeTime=1498053228353, loc=false, ver=2.0.0#20170430-sha1:d4eef3 >> c6, >> isClient=false] >> 2017-06-21 13:53:56,724 WARN [exchange-worker-#25%null%] >> dht.GridDhtAssignmentFetchFuture (Log4JLogger.java:warning(480)) - >> Failed to >> request affinity assignment from remote node (will continue to another >> node): TcpDiscoveryNode [id=927af8bb-e105-4332-baf3-072f921d09cd, >> addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 192.168.30.22], >> sockAddrs=[0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500, >> slave-1/192.168.30.22:47500], discPort=47500, order=1, intOrder=1, >> lastExchangeTime=1498053228353, loc=false, ver=2.0.0#20170430-sha1:d4eef3 >> c6, >> isClient=false] >> 2017-06-21 13:53:58,744 WARN [exchange-worker-#25%null%] >> dht.GridDhtAssignmentFetchFuture (Log4JLogger.java:warning(480)) - >> Failed to >> request affinity assignment from remote node (will continue to another >> node): TcpDiscoveryNode [id=927af8bb-e105-4332-baf3-072f921d09cd, >> addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 192.168.30.22], >> sockAddrs=[0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500, >> slave-1/192.168.30.22:47500], discPort=47500, order=1, intOrder=1, >> lastExchangeTime=1498053228353, loc=false, ver=2.0.0#20170430-sha1:d4eef3 >> c6, >> isClient=false] >> 2017-06-21 13:54:00,789 ERROR [exchange-worker-#25%null%] >> preloader.GridDhtPartitionsExchangeFuture (Log4JLogger.java:error(495)) - >> Failed to reinitialize local partitions (preloading will be stopped): >> GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=4, >> minorTopVer=0], nodeId=2df6b461, evt=NODE_JOINED] >> class org.apache.ignite.IgniteCheckedException: Failed to send message >> (node >> may have left the grid or TCP connection cannot be established due to >> firewall issues) [node=TcpDiscoveryNode >> [id=927af8bb-e105-4332-baf3-072f921d09cd, addrs=[0:0:0:0:0:0:0:1%lo, >> 127.0.0.1, 192.168.30.22], sockAddrs=[0:0:0:0:0:0:0:1%lo:47500, >> /127.0.0.1:47500, slave-1/192.168.30.22:47500], discPort=47500, order=1, >> intOrder=1, lastExchangeTime=1498053228353, loc=false, >> ver=2.0.0#20170430-sha1:d4eef3c6, isClient=false], topic=TOPIC_CACHE, >> msg=GridDhtPartitionsSingleMessage [parts={-1448311745=GridDhtPar >> titionMap >> [moving=0, size=0], -2100569601=GridDhtPartitionMap [moving=0, size=0], >> 689859866=GridDhtPartitionMap [moving=0, size=0], >> -313790114=GridDhtPartitionMap [moving=0, size=0], >> -313518151=GridDhtPartitionMap [moving=0, size=0]}, >> partCntrs={-1448311745={}, -2100569601={}, 689859866={}, -313790114={}, >> -313518151={}}, ex=null, client=true, compress=true, >> super=GridDhtPartitionsAbstractMessage [exchId=GridDhtPartitionExchangeId >> [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], >> nodeId=2df6b461, >> evt=NODE_JOINED], lastVer=GridCacheVersion [topVer=0, order=1498053227978, >> nodeOrder=0], flags=1, super=GridCacheMessage [msgId=6, depInfo=null, >> err=null, skipPrepare=false, cacheId=0, cacheId=0]]], policy=2] >> at >> org.apache.ignite.internal.managers.communication.GridIoMana >> ger.send(GridIoManager.java:1334) >> at >> org.apache.ignite.internal.managers.communication.GridIoMana >> ger.sendToGridTopic(GridIoManager.java:1398) >> at >> org.apache.ignite.internal.processors.cache.GridCacheIoManag >> er.send(GridCacheIoManager.java:946) >> at >> org.apache.ignite.internal.processors.cache.distributed.dht. >> preloader.GridDhtPartitionsExchangeFuture.sendLocalPartitions(GridDhtPar >> titionsExchangeFuture.java:1094) >> at >> org.apache.ignite.internal.processors.cache.distributed.dht. >> preloader.GridDhtPartitionsExchangeFuture.clientOnlyExchange(GridDhtPart >> itionsExchangeFuture.java:793) >> at >> org.apache.ignite.internal.processors.cache.distributed.dht. >> preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartit >> ionsExchangeFuture.java:581) >> at >> org.apache.ignite.internal.processors.cache.GridCachePartiti >> onExchangeManager$ExchangeWorker.body(GridCacheP >> artitionExchangeManager.java:1806) >> at >> org.apache.ignite.internal.util.worker.GridWorker.run(GridWo >> rker.java:110) >> at java.lang.Thread.run(Thread.java:748) >> Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to send >> message to remote node: TcpDiscoveryNode >> [id=927af8bb-e105-4332-baf3-072f921d09cd, addrs=[0:0:0:0:0:0:0:1%lo, >> 127.0.0.1, 192.168.30.22], sockAddrs=[0:0:0:0:0:0:0:1%lo:47500, >> /127.0.0.1:47500, slave-1/192.168.30.22:47500], discPort=47500, order=1, >> intOrder=1, lastExchangeTime=1498053228353, loc=false, >> ver=2.0.0#20170430-sha1:d4eef3c6, isClient=false] >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> sendMessage0(TcpCommunicationSpi.java:2483) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> sendMessage(TcpCommunicationSpi.java:2419) >> at >> org.apache.ignite.internal.managers.communication.GridIoMana >> ger.send(GridIoManager.java:1329) >> ... 8 more >> Caused by: class org.apache.ignite.IgniteCheckedException: >> java.lang.NullPointerException >> at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils >> .java:7242) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter.res >> olve(GridFutureAdapter.java:258) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter.get >> 0(GridFutureAdapter.java:170) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter.get >> (GridFutureAdapter.java:139) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> reserveClient(TcpCommunicationSpi.java:2630) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> sendMessage0(TcpCommunicationSpi.java:2455) >> ... 10 more >> Caused by: java.util.concurrent.ExecutionException: >> java.lang.NullPointerException >> at java.util.concurrent.FutureTask.report(FutureTask.java:122) >> at java.util.concurrent.FutureTask.get(FutureTask.java:192) >> at >> org.apache.ignite.internal.util.IgniteUtils.filterReachable( >> IgniteUtils.java:1895) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> createTcpClient(TcpCommunicationSpi.java:2891) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> createNioClient(TcpCommunicationSpi.java:2702) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> reserveClient(TcpCommunicationSpi.java:2594) >> ... 11 more >> Caused by: java.lang.NullPointerException >> at >> org.apache.ignite.internal.util.IgniteUtils.reachable(Ignite >> Utils.java:2102) >> at >> org.apache.ignite.internal.util.IgniteUtils$18.run(IgniteUtils.java:1884) >> at java.util.concurrent.Executors$RunnableAdapter.call( >> Executors.java:511) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool >> Executor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo >> lExecutor.java:617) >> ... 1 more >> 2017-06-21 13:54:00,798 ERROR [main] internal.IgniteKernal >> (Log4JLogger.java:error(495)) - Got exception while starting (will >> rollback >> startup routine). >> class org.apache.ignite.IgniteCheckedException: Failed to send message >> (node >> may have left the grid or TCP connection cannot be established due to >> firewall issues) [node=TcpDiscoveryNode >> [id=927af8bb-e105-4332-baf3-072f921d09cd, addrs=[0:0:0:0:0:0:0:1%lo, >> 127.0.0.1, 192.168.30.22], sockAddrs=[0:0:0:0:0:0:0:1%lo:47500, >> /127.0.0.1:47500, slave-1/192.168.30.22:47500], discPort=47500, order=1, >> intOrder=1, lastExchangeTime=1498053228353, loc=false, >> ver=2.0.0#20170430-sha1:d4eef3c6, isClient=false], topic=TOPIC_CACHE, >> msg=GridDhtPartitionsSingleMessage [parts={-1448311745=GridDhtPar >> titionMap >> [moving=0, size=0], -2100569601=GridDhtPartitionMap [moving=0, size=0], >> 689859866=GridDhtPartitionMap [moving=0, size=0], >> -313790114=GridDhtPartitionMap [moving=0, size=0], >> -313518151=GridDhtPartitionMap [moving=0, size=0]}, >> partCntrs={-1448311745={}, -2100569601={}, 689859866={}, -313790114={}, >> -313518151={}}, ex=null, client=true, compress=true, >> super=GridDhtPartitionsAbstractMessage [exchId=GridDhtPartitionExchangeId >> [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], >> nodeId=2df6b461, >> evt=NODE_JOINED], lastVer=GridCacheVersion [topVer=0, order=1498053227978, >> nodeOrder=0], flags=1, super=GridCacheMessage [msgId=6, depInfo=null, >> err=null, skipPrepare=false, cacheId=0, cacheId=0]]], policy=2] >> at >> org.apache.ignite.internal.managers.communication.GridIoMana >> ger.send(GridIoManager.java:1334) >> at >> org.apache.ignite.internal.managers.communication.GridIoMana >> ger.sendToGridTopic(GridIoManager.java:1398) >> at >> org.apache.ignite.internal.processors.cache.GridCacheIoManag >> er.send(GridCacheIoManager.java:946) >> at >> org.apache.ignite.internal.processors.cache.distributed.dht. >> preloader.GridDhtPartitionsExchangeFuture.sendLocalPartitions(GridDhtPar >> titionsExchangeFuture.java:1094) >> at >> org.apache.ignite.internal.processors.cache.distributed.dht. >> preloader.GridDhtPartitionsExchangeFuture.clientOnlyExchange(GridDhtPart >> itionsExchangeFuture.java:793) >> at >> org.apache.ignite.internal.processors.cache.distributed.dht. >> preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartit >> ionsExchangeFuture.java:581) >> at >> org.apache.ignite.internal.processors.cache.GridCachePartiti >> onExchangeManager$ExchangeWorker.body(GridCacheP >> artitionExchangeManager.java:1806) >> at >> org.apache.ignite.internal.util.worker.GridWorker.run(GridWo >> rker.java:110) >> at java.lang.Thread.run(Thread.java:748) >> Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to send >> message to remote node: TcpDiscoveryNode >> [id=927af8bb-e105-4332-baf3-072f921d09cd, addrs=[0:0:0:0:0:0:0:1%lo, >> 127.0.0.1, 192.168.30.22], sockAddrs=[0:0:0:0:0:0:0:1%lo:47500, >> /127.0.0.1:47500, slave-1/192.168.30.22:47500], discPort=47500, order=1, >> intOrder=1, lastExchangeTime=1498053228353, loc=false, >> ver=2.0.0#20170430-sha1:d4eef3c6, isClient=false] >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> sendMessage0(TcpCommunicationSpi.java:2483) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> sendMessage(TcpCommunicationSpi.java:2419) >> at >> org.apache.ignite.internal.managers.communication.GridIoMana >> ger.send(GridIoManager.java:1329) >> ... 8 more >> Caused by: class org.apache.ignite.IgniteCheckedException: >> java.lang.NullPointerException >> at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils >> .java:7242) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter.res >> olve(GridFutureAdapter.java:258) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter.get >> 0(GridFutureAdapter.java:170) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter.get >> (GridFutureAdapter.java:139) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> reserveClient(TcpCommunicationSpi.java:2630) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> sendMessage0(TcpCommunicationSpi.java:2455) >> ... 10 more >> Caused by: java.util.concurrent.ExecutionException: >> java.lang.NullPointerException >> at java.util.concurrent.FutureTask.report(FutureTask.java:122) >> at java.util.concurrent.FutureTask.get(FutureTask.java:192) >> at >> org.apache.ignite.internal.util.IgniteUtils.filterReachable( >> IgniteUtils.java:1895) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> createTcpClient(TcpCommunicationSpi.java:2891) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> createNioClient(TcpCommunicationSpi.java:2702) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> reserveClient(TcpCommunicationSpi.java:2594) >> ... 11 more >> Caused by: java.lang.NullPointerException >> at >> org.apache.ignite.internal.util.IgniteUtils.reachable(Ignite >> Utils.java:2102) >> at >> org.apache.ignite.internal.util.IgniteUtils$18.run(IgniteUtils.java:1884) >> at java.util.concurrent.Executors$RunnableAdapter.call( >> Executors.java:511) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool >> Executor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo >> lExecutor.java:617) >> ... 1 more >> 2017-06-21 13:54:00,804 ERROR [exchange-worker-#25%null%] >> cache.GridCachePartitionExchangeManager (Log4JLogger.java:error(495)) - >> Failed to wait for completion of partition map exchange (preloading will >> not >> start): GridDhtPartitionsExchangeFuture [dummy=false, forcePreload=false, >> reassign=false, discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode >> [id=2df6b461-723c-45d4-9bcc-02daab91fb74, addrs=[127.0.0.1, >> 192.168.30.5], >> sockAddrs=[Master/192.168.30.5:0, /127.0.0.1:0], discPort=0, order=4, >> intOrder=0, lastExchangeTime=1498053228293, loc=true, >> ver=2.0.0#20170430-sha1:d4eef3c6, isClient=true], topVer=4, >> nodeId8=2df6b461, msg=null, type=NODE_JOINED, tstamp=1498053228384], >> crd=TcpDiscoveryNode [id=927af8bb-e105-4332-baf3-072f921d09cd, >> addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 192.168.30.22], >> sockAddrs=[0:0:0:0:0:0:0:1%lo:47500, /127.0.0.1:47500, >> slave-1/192.168.30.22:47500], discPort=47500, order=1, intOrder=1, >> lastExchangeTime=1498053228353, loc=false, ver=2.0.0#20170430-sha1:d4eef3 >> c6, >> isClient=false], exchId=GridDhtPartitionExchangeId >> [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], >> nodeId=2df6b461, >> evt=NODE_JOINED], added=false, initFut=GridFutureAdapter >> [ignoreInterrupts=false, state=DONE, res=false, hash=1930617560], >> init=false, lastVer=null, partReleaseFut=null, affChangeMsg=null, >> skipPreload=true, clientOnlyExchange=true, initTs=1498053228616, >> centralizedAff=false, changeGlobalStateE=null, >> exchangeOnChangeGlobalState=false, forcedRebFut=null, evtLatch=0, >> remaining=[927af8bb-e105-4332-baf3-072f921d09cd], >> srvNodes=[TcpDiscoveryNode >> [id=927af8bb-e105-4332-baf3-072f921d09cd, addrs=[0:0:0:0:0:0:0:1%lo, >> 127.0.0.1, 192.168.30.22], sockAddrs=[0:0:0:0:0:0:0:1%lo:47500, >> /127.0.0.1:47500, slave-1/192.168.30.22:47500], discPort=47500, order=1, >> intOrder=1, lastExchangeTime=1498053228353, loc=false, >> ver=2.0.0#20170430-sha1:d4eef3c6, isClient=false]], >> super=GridFutureAdapter >> [ignoreInterrupts=false, state=DONE, res=class >> o.a.i.IgniteCheckedException: >> Failed to send message (node may have left the grid or TCP connection >> cannot >> be established due to firewall issues) [node=TcpDiscoveryNode >> [id=927af8bb-e105-4332-baf3-072f921d09cd, addrs=[0:0:0:0:0:0:0:1%lo, >> 127.0.0.1, 192.168.30.22], sockAddrs=[0:0:0:0:0:0:0:1%lo:47500, >> /127.0.0.1:47500, slave-1/192.168.30.22:47500], discPort=47500, order=1, >> intOrder=1, lastExchangeTime=1498053228353, loc=false, >> ver=2.0.0#20170430-sha1:d4eef3c6, isClient=false], topic=TOPIC_CACHE, >> msg=GridDhtPartitionsSingleMessage [parts={-1448311745=GridDhtPar >> titionMap >> [moving=0, size=0], -2100569601=GridDhtPartitionMap [moving=0, size=0], >> 689859866=GridDhtPartitionMap [moving=0, size=0], >> -313790114=GridDhtPartitionMap [moving=0, size=0], >> -313518151=GridDhtPartitionMap [moving=0, size=0]}, >> partCntrs={-1448311745={}, -2100569601={}, 689859866={}, -313790114={}, >> -313518151={}}, ex=null, client=true, compress=true, >> super=GridDhtPartitionsAbstractMessage [exchId=GridDhtPartitionExchangeId >> [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], >> nodeId=2df6b461, >> evt=NODE_JOINED], lastVer=GridCacheVersion [topVer=0, order=1498053227978, >> nodeOrder=0], flags=1, super=GridCacheMessage [msgId=6, depInfo=null, >> err=null, skipPrepare=false, cacheId=0, cacheId=0]]], policy=2], >> hash=1242579201]] >> class org.apache.ignite.IgniteCheckedException: Failed to send message >> (node >> may have left the grid or TCP connection cannot be established due to >> firewall issues) [node=TcpDiscoveryNode >> [id=927af8bb-e105-4332-baf3-072f921d09cd, addrs=[0:0:0:0:0:0:0:1%lo, >> 127.0.0.1, 192.168.30.22], sockAddrs=[0:0:0:0:0:0:0:1%lo:47500, >> /127.0.0.1:47500, slave-1/192.168.30.22:47500], discPort=47500, order=1, >> intOrder=1, lastExchangeTime=1498053228353, loc=false, >> ver=2.0.0#20170430-sha1:d4eef3c6, isClient=false], topic=TOPIC_CACHE, >> msg=GridDhtPartitionsSingleMessage [parts={-1448311745=GridDhtPar >> titionMap >> [moving=0, size=0], -2100569601=GridDhtPartitionMap [moving=0, size=0], >> 689859866=GridDhtPartitionMap [moving=0, size=0], >> -313790114=GridDhtPartitionMap [moving=0, size=0], >> -313518151=GridDhtPartitionMap [moving=0, size=0]}, >> partCntrs={-1448311745={}, -2100569601={}, 689859866={}, -313790114={}, >> -313518151={}}, ex=null, client=true, compress=true, >> super=GridDhtPartitionsAbstractMessage [exchId=GridDhtPartitionExchangeId >> [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], >> nodeId=2df6b461, >> evt=NODE_JOINED], lastVer=GridCacheVersion [topVer=0, order=1498053227978, >> nodeOrder=0], flags=1, super=GridCacheMessage [msgId=6, depInfo=null, >> err=null, skipPrepare=false, cacheId=0, cacheId=0]]], policy=2] >> at >> org.apache.ignite.internal.managers.communication.GridIoMana >> ger.send(GridIoManager.java:1334) >> at >> org.apache.ignite.internal.managers.communication.GridIoMana >> ger.sendToGridTopic(GridIoManager.java:1398) >> at >> org.apache.ignite.internal.processors.cache.GridCacheIoManag >> er.send(GridCacheIoManager.java:946) >> at >> org.apache.ignite.internal.processors.cache.distributed.dht. >> preloader.GridDhtPartitionsExchangeFuture.sendLocalPartitions(GridDhtPar >> titionsExchangeFuture.java:1094) >> at >> org.apache.ignite.internal.processors.cache.distributed.dht. >> preloader.GridDhtPartitionsExchangeFuture.clientOnlyExchange(GridDhtPart >> itionsExchangeFuture.java:793) >> at >> org.apache.ignite.internal.processors.cache.distributed.dht. >> preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartit >> ionsExchangeFuture.java:581) >> at >> org.apache.ignite.internal.processors.cache.GridCachePartiti >> onExchangeManager$ExchangeWorker.body(GridCacheP >> artitionExchangeManager.java:1806) >> at >> org.apache.ignite.internal.util.worker.GridWorker.run(GridWo >> rker.java:110) >> at java.lang.Thread.run(Thread.java:748) >> Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to send >> message to remote node: TcpDiscoveryNode >> [id=927af8bb-e105-4332-baf3-072f921d09cd, addrs=[0:0:0:0:0:0:0:1%lo, >> 127.0.0.1, 192.168.30.22], sockAddrs=[0:0:0:0:0:0:0:1%lo:47500, >> /127.0.0.1:47500, slave-1/192.168.30.22:47500], discPort=47500, order=1, >> intOrder=1, lastExchangeTime=1498053228353, loc=false, >> ver=2.0.0#20170430-sha1:d4eef3c6, isClient=false] >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> sendMessage0(TcpCommunicationSpi.java:2483) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> sendMessage(TcpCommunicationSpi.java:2419) >> at >> org.apache.ignite.internal.managers.communication.GridIoMana >> ger.send(GridIoManager.java:1329) >> ... 8 more >> Caused by: class org.apache.ignite.IgniteCheckedException: >> java.lang.NullPointerException >> at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils >> .java:7242) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter.res >> olve(GridFutureAdapter.java:258) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter.get >> 0(GridFutureAdapter.java:170) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter.get >> (GridFutureAdapter.java:139) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> reserveClient(TcpCommunicationSpi.java:2630) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> sendMessage0(TcpCommunicationSpi.java:2455) >> ... 10 more >> Caused by: java.util.concurrent.ExecutionException: >> java.lang.NullPointerException >> at java.util.concurrent.FutureTask.report(FutureTask.java:122) >> at java.util.concurrent.FutureTask.get(FutureTask.java:192) >> at >> org.apache.ignite.internal.util.IgniteUtils.filterReachable( >> IgniteUtils.java:1895) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> createTcpClient(TcpCommunicationSpi.java:2891) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> createNioClient(TcpCommunicationSpi.java:2702) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> reserveClient(TcpCommunicationSpi.java:2594) >> ... 11 more >> Caused by: java.lang.NullPointerException >> at >> org.apache.ignite.internal.util.IgniteUtils.reachable(Ignite >> Utils.java:2102) >> at >> org.apache.ignite.internal.util.IgniteUtils$18.run(IgniteUtils.java:1884) >> at java.util.concurrent.Executors$RunnableAdapter.call( >> Executors.java:511) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool >> Executor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo >> lExecutor.java:617) >> ... 1 more >> 2017-06-21 13:54:00,825 INFO [main] cache.GridCacheProcessor >> (Log4JLogger.java:info(475)) - Stopped cache: ignite-sys-cache >> 2017-06-21 13:54:00,825 INFO [main] cache.GridCacheProcessor >> (Log4JLogger.java:info(475)) - Stopped cache: ignite-hadoop-mr-sys-cache >> 2017-06-21 13:54:00,826 INFO [main] cache.GridCacheProcessor >> (Log4JLogger.java:info(475)) - Stopped cache: ignite-atomics-sys-cache >> 2017-06-21 13:54:00,826 INFO [main] cache.GridCacheProcessor >> (Log4JLogger.java:info(475)) - Stopped cache: igfs-internal-igfs-meta >> 2017-06-21 13:54:00,827 INFO [main] cache.GridCacheProcessor >> (Log4JLogger.java:info(475)) - Stopped cache: igfs-internal-igfs-data >> 2017-06-21 13:54:00,836 INFO [main] internal.IgniteKernal >> (Log4JLogger.java:info(475)) - >> >> >>> +----------------------------------------------------------- >> ----------------------+ >> >>> Ignite ver. 2.0.0#20170430-sha1:d4eef3c68f >> f116ee34bc13648cd82c640b3ea072 >> >>> stopped OK >> >>> +----------------------------------------------------------- >> ----------------------+ >> >>> Grid uptime: 00:00:14:004 >> >> >> Exception in thread "main" class org.apache.ignite.spi.IgniteSp >> iException: >> Failed to send message to remote node: TcpDiscoveryNode >> [id=927af8bb-e105-4332-baf3-072f921d09cd, addrs=[0:0:0:0:0:0:0:1%lo, >> 127.0.0.1, 192.168.30.22], sockAddrs=[0:0:0:0:0:0:0:1%lo:47500, >> /127.0.0.1:47500, slave-1/192.168.30.22:47500], discPort=47500, order=1, >> intOrder=1, lastExchangeTime=1498053228353, loc=false, >> ver=2.0.0#20170430-sha1:d4eef3c6, isClient=false] >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> sendMessage0(TcpCommunicationSpi.java:2483) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> sendMessage(TcpCommunicationSpi.java:2419) >> at >> org.apache.ignite.internal.managers.communication.GridIoMana >> ger.send(GridIoManager.java:1329) >> at >> org.apache.ignite.internal.managers.communication.GridIoMana >> ger.sendToGridTopic(GridIoManager.java:1398) >> at >> org.apache.ignite.internal.processors.cache.GridCacheIoManag >> er.send(GridCacheIoManager.java:946) >> at >> org.apache.ignite.internal.processors.cache.distributed.dht. >> preloader.GridDhtPartitionsExchangeFuture.sendLocalPartitions(GridDhtPar >> titionsExchangeFuture.java:1094) >> at >> org.apache.ignite.internal.processors.cache.distributed.dht. >> preloader.GridDhtPartitionsExchangeFuture.clientOnlyExchange(GridDhtPart >> itionsExchangeFuture.java:793) >> at >> org.apache.ignite.internal.processors.cache.distributed.dht. >> preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartit >> ionsExchangeFuture.java:581) >> at >> org.apache.ignite.internal.processors.cache.GridCachePartiti >> onExchangeManager$ExchangeWorker.body(GridCacheP >> artitionExchangeManager.java:1806) >> at >> org.apache.ignite.internal.util.worker.GridWorker.run(GridWo >> rker.java:110) >> at java.lang.Thread.run(Thread.java:748) >> Caused by: class org.apache.ignite.IgniteCheckedException: >> java.lang.NullPointerException >> at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils >> .java:7242) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter.res >> olve(GridFutureAdapter.java:258) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter.get >> 0(GridFutureAdapter.java:170) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter.get >> (GridFutureAdapter.java:139) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> reserveClient(TcpCommunicationSpi.java:2630) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> sendMessage0(TcpCommunicationSpi.java:2455) >> ... 10 more >> Caused by: java.util.concurrent.ExecutionException: >> java.lang.NullPointerException >> at java.util.concurrent.FutureTask.report(FutureTask.java:122) >> at java.util.concurrent.FutureTask.get(FutureTask.java:192) >> at >> org.apache.ignite.internal.util.IgniteUtils.filterReachable( >> IgniteUtils.java:1895) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> createTcpClient(TcpCommunicationSpi.java:2891) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> createNioClient(TcpCommunicationSpi.java:2702) >> at >> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi. >> reserveClient(TcpCommunicationSpi.java:2594) >> ... 11 more >> Caused by: java.lang.NullPointerException >> at >> org.apache.ignite.internal.util.IgniteUtils.reachable(Ignite >> Utils.java:2102) >> at >> org.apache.ignite.internal.util.IgniteUtils$18.run(IgniteUtils.java:1884) >> at java.util.concurrent.Executors$RunnableAdapter.call( >> Executors.java:511) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool >> Executor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo >> lExecutor.java:617) >> ... 1 more >> >> >> The IPDiscovery found the two node server try to connect and for a little >> bit of time they connect with him but it failed to send message with >> server >> node.... I don't think it's a problem of Port because they are open... Any >> suggestion? >> >> the code of WordCount >> >> public static void main(String args[]) throws Exception { >> >> //TcpDiscoverySpi spi = new TcpDiscoverySpi(); >> >> //TcpDiscoveryVmIpFinder ipFinder = new >> TcpDiscoveryVmIpFinder(); >> >> // Set initial IP addresses. >> // Note that you can optionally specify a port or a port >> range. >> //ipFinder.setAddresses(Arrays.asList("192.168.30.99", >> "192.168.30.22")); >> >> //spi.setIpFinder(ipFinder); >> >> //IgniteConfiguration cfg = new IgniteConfiguration(); >> >> // Override default discovery SPI. >> //cfg.setDiscoverySpi(spi); >> >> // Start Ignite node. >> Ignition.setClientMode(true); >> >> >> >> try(Ignite ignite = >> Ignition.start("/home/hduser/apache-ignite-2.0.0-src/config/ >> hadoop/default-config.xml")){ >> >> CacheConfiguration<Integer ,String> cfg2 = new >> CacheConfiguration<>(); >> >> IgniteCache<Integer, String> cache; >> >> cfg2.setName("demoCache"); >> cfg2.setAtomicityMode(CacheAto >> micityMode.TRANSACTIONAL); >> >> //cache = ignite.getOrCreateCache(cfg2); >> >> //cache.put(1,"uno"); >> //cache.put(2,"due"); >> >> //System.out.println(cache.get(1)); >> >> //System.out.println(cache.get(2)); >> >> >> int res = ToolRunner.run(new WordCountIgnite(), >> args); >> System.exit(res); >> } >> } >> >> Else i try to put manually IP of server but it's the same. It crash before >> start cache, i try just to connect without create cache but it's the same. >> Any suggestion? >> >> Thanks >> >> >> >> -- >> View this message in context: http://apache-ignite-users.705 >> 18.x6.nabble.com/Hadoop-and-Ignite-Problem-grid-cache-tp14028.html >> Sent from the Apache Ignite Users mailing list archive at Nabble.com. >> > >
