Sergey Karpushin created IGNITE-10388:
-----------------------------------------
Summary: Prevent Ignite from "dying" if first attempt to join the
cluster failed
Key: IGNITE-10388
URL: https://issues.apache.org/jira/browse/IGNITE-10388
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: 2.6
Environment: * Linux OS (I'm using Manjaro 1.18)
* Jdk 1.8
* Docker-compose 3.7
Reporter: Sergey Karpushin
I'm creating this issue as suggested on
[stackoverflow|https://stackoverflow.com/questions/53433749/how-to-prevent-ignite-from-dying-if-first-attempt-to-join-the-cluster-failed]
I've also created a simple [github
project|https://github.com/skarpushin/issue20181122] to help reproduce the
issue (please check README file before running the project).
I'm using {{docker-compose}} to start several containers with java web
application with embedded Ignite 2.6.
I configured Ignite so it will use SSL.
If I start node1 and wait it to fully start, and then start node2, they will
join cluster no problem.
But if I start them simultaneously (that's how it works by default with
docker-compose) they both will fail with exception and will never retry or get
back.
{{{{class org.apache.ignite.IgniteException: Unable to establish secure
connection. Was remote cluster configured with SSL? [rmtAddr=/10.9.9.1:47501,
errMsg="Remote host closed connection during handshake"] at
org.apache.ignite.spi.discovery.tcp.ServerImpl.sendMessageDirectly(ServerImpl.java:1293)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.sendJoinRequestMessage(ServerImpl.java:1046)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:890)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:373) at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:1948)
at
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
at
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:915)
at
org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1721) at
org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1028) at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2014)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1723)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1151) at
org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:649) at
org.apache.ignite.IgniteSpring.start(IgniteSpring.java:66) at
org.apache.ignite.IgniteSpringBean.afterSingletonsInstantiated(IgniteSpringBean.java:172)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:777)
at
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:869)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
at
org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:409)
at
org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:291)
at
org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4853)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5314)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145) at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:753)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729) at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717) at
org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:976) at
org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1853) at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
java.util.concurrent.FutureTask.run(FutureTask.java:266) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) Caused by:
javax.net.ssl.SSLHandshakeException: Remote host closed connection during
handshake at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1002)
at
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:757) at
sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123) at
java.io.OutputStream.write(OutputStream.java:75) at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.writeToSocket(TcpDiscoverySpi.java:1501)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.openSocket(TcpDiscoverySpi.java:1452)
at
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.openSocket(TcpDiscoverySpi.java:1413)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl.sendMessageDirectly(ServerImpl.java:1199)
... 33 more Caused by: java.io.EOFException: SSL peer shut down incorrectly at
sun.security.ssl.InputRecord.read(InputRecord.java:505) at
sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983) ... 41 more
2018-11-22 14:47:06.727 INFO localhost-startStop-1 Log4JLogger Command protocol
successfully stopped: TCP binary 2018-11-22 14:47:11.733 INFO
localhost-startStop-1 Log4JLogger No verification for local node leave has been
received from coordinator (will stop node anyway).}}}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
