RE: Baseline topology issue when restarting server nodes one by one
Sorry, I guess my wording was a bit confusing. Not all traffic goes through the gateway server. There are two networking subsystems in Ignite: Discovery and Communication. Discovery is responsible for clustering (node discovery, heartbeats, etc). Default implementation (TcpDiscoverySpi) connects all servers in a ring topology, and each client has it’s “gateway” server responsible for connecting the client to the cluster, so all Discovery events go through the gateway. Communication is responsible for the actual load (cache operations, messaging, etc). Default implementation (TcpCommunicationSpi) connects all nodes, servers and clients, in a fully connected topology and clients communicate with the servers directly, bypassing their Discovery gateway. Thanks, Stan From: Olexandr K Sent: 14 июня 2018 г. 14:56 To: user@ignite.apache.org Subject: Re: Baseline topology issue when restarting server nodes one by one No, it is reconnecting fine. I just wondered that client node is kicked off from cluster when we have one of server nodes available. It is clear after your explanation. Just curious - why client node is connected to one server node and not to both? All my caches are partitioned and client should go to server A or B depending on which partition was resolved for the key, right? Did you mean that if client C is using server A as gateway, all traffic goes through server A? On Wed, Jun 13, 2018 at 4:43 PM, Stanislav Lukyanov wrote: On the client disconnection: The client is connected to one of the server nodes, using it as a sort of gateway to the cluster. If the gateway server fails, the client supposed to attempt to reconnect to other IPs. Right now I can’t say for sure whether it does so indefinitely, or has some timeout for it. Perhaps you’ve changed networkTimeout or something, and after that it doesn’t reconnect? On the IDs in Visor: Somewhat confusingly, ConsistentID is NOT NodeID. Visor shows NodeID and not ConsistentID. NodeID is regenerated each time a node restarts and it is always a UUID. ConsistentID is a UUID by default, but it doesn’t have to be the same as NodeID and doesn’t have to have a form of UUID – any string works (and even any Object with an idempotent toString(), but tsss - don’t tell anyone!). Stan From: Olexandr K Sent: 13 июня 2018 г. 0:58 To: user@ignite.apache.org Subject: Re: Baseline topology issue when restarting server nodes one by one I configured ConsistentId equal to hostname for each node and this issue is not reproduced anymore One more strange behaviour I noticed is that one of client nodes gets disconnected after one of server nodes goes down. I have reconnect logic in place so it comes back later but is such behaviour expected? Not sure whether it is related with consistent IDs but I didn't see it earlier... BTW, after configuring consistent IDs I see them in "control.bat --baseline" output only. Visor output and server logs still show generated IDs That looks confusing... Cluster state: active Current topology version: 7 Baseline nodes: ConsistentID=V-HP-LK-DCN01, STATE=ONLINE ConsistentID=V-HP-LK-DCN02, STATE=ONLINE 9871EAFF(@n0) | Server BBA63A1F(@n2) | Server 1DEDB701(@n1) | Client 5931AF53(@n3) | Client logs\v-hp-lk-dcn01\ignite.log:383:>>> Local node [ID=9871EAFF-73AF-4E2E-99A7-8F5DF58A3C40, order=1, clientMode=false] logs\v-hp-lk-dcn02\ignite.log:274:>>> Local node [ID=BBA63A1F-559E-461C-B7ED-B10CE3DE33CC, order=7, clientMode=false] On Tue, Jun 12, 2018 at 9:48 PM, Olexandr K wrote: Hi, Dmitry server nodes start with ignite-server.xml and client nodes with ignite-client.xml server node hosts: v-hp-lk-dcn01, v-hp-lk-dcn02 http://www.springframework.org/schema/beans; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframe
Re: Baseline topology issue when restarting server nodes one by one
No, it is reconnecting fine. I just wondered that client node is kicked off from cluster when we have one of server nodes available. It is clear after your explanation. Just curious - why client node is connected to one server node and not to both? All my caches are partitioned and client should go to server A or B depending on which partition was resolved for the key, right? Did you mean that if client C is using server A as gateway, all traffic goes through server A? On Wed, Jun 13, 2018 at 4:43 PM, Stanislav Lukyanov wrote: > On the client disconnection: > > The client is connected to one of the server nodes, using it as a sort of > gateway to the cluster. > > If the gateway server fails, the client supposed to attempt to reconnect > to other IPs. > > Right now I can’t say for sure whether it does so indefinitely, or has > some timeout for it. > > Perhaps you’ve changed networkTimeout or something, and after that it > doesn’t reconnect? > > > > On the IDs in Visor: > > Somewhat confusingly, ConsistentID is NOT NodeID. > > Visor shows NodeID and not ConsistentID. > > > > NodeID is regenerated each time a node restarts and it is always a UUID. > > ConsistentID is a UUID by default, but it doesn’t have to be the same as > NodeID and doesn’t have to have a form of UUID – any string works (and even > any Object with an idempotent toString(), but tsss - don’t tell anyone!). > > > > Stan > > > > *From: *Olexandr K > *Sent: *13 июня 2018 г. 0:58 > *To: *user@ignite.apache.org > *Subject: *Re: Baseline topology issue when restarting server nodes one > by one > > > > I configured ConsistentId equal to hostname for each node and this issue > is not reproduced anymore > > > > One more strange behaviour I noticed is that one of client nodes gets > disconnected after one of server nodes goes down. > > I have reconnect logic in place so it comes back later but is such > behaviour expected? > > Not sure whether it is related with consistent IDs but I didn't see it > earlier... > > > > BTW, after configuring consistent IDs I see them in "control.bat > --baseline" output only. > > Visor output and server logs still show generated IDs > > That looks confusing... > > > > > > Cluster state: active > Current topology version: 7 > Baseline nodes: > ConsistentID=V-HP-LK-DCN01, STATE=ONLINE > ConsistentID=V-HP-LK-DCN02, STATE=ONLINE > > > > > > 9871EAFF(@n0) | Server > > BBA63A1F(@n2) | Server > > 1DEDB701(@n1) | Client > > 5931AF53(@n3) | Client > > > > > > logs\v-hp-lk-dcn01\ignite.log:383:>>> Local node > [ID=9871EAFF-73AF-4E2E-99A7-8F5DF58A3C40, order=1, clientMode=false] > logs\v-hp-lk-dcn02\ignite.log:274:>>> Local node > [ID=BBA63A1F-559E-461C-B7ED-B10CE3DE33CC, order=7, clientMode=false] > > > > > > On Tue, Jun 12, 2018 at 9:48 PM, Olexandr K > wrote: > > Hi, Dmitry > > > > server nodes start with ignite-server.xml and client nodes with > ignite-client.xml > > server node hosts: v-hp-lk-dcn01, v-hp-lk-dcn02 > > > > > > > > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > > class="org.apache.ignite.configuration.IgniteConfiguration"> > > > /> > > > > > > > > > > > > > value="s-hp-fs01\\dev$\\config\\keystore.jks" > /> > > > factory-method="getDisabledTrustManager" /> > > > > > > > value="S-hp-fs01\\dev$\\config\\log4j2.xml"/> > > > > > > > > value="auth_durable_region"/> > > value="FULL_ASYNC"/> > > > > > > > > > > > > > > > > > > > >
RE: Baseline topology issue when restarting server nodes one by one
On the client disconnection: The client is connected to one of the server nodes, using it as a sort of gateway to the cluster. If the gateway server fails, the client supposed to attempt to reconnect to other IPs. Right now I can’t say for sure whether it does so indefinitely, or has some timeout for it. Perhaps you’ve changed networkTimeout or something, and after that it doesn’t reconnect? On the IDs in Visor: Somewhat confusingly, ConsistentID is NOT NodeID. Visor shows NodeID and not ConsistentID. NodeID is regenerated each time a node restarts and it is always a UUID. ConsistentID is a UUID by default, but it doesn’t have to be the same as NodeID and doesn’t have to have a form of UUID – any string works (and even any Object with an idempotent toString(), but tsss - don’t tell anyone!). Stan From: Olexandr K Sent: 13 июня 2018 г. 0:58 To: user@ignite.apache.org Subject: Re: Baseline topology issue when restarting server nodes one by one I configured ConsistentId equal to hostname for each node and this issue is not reproduced anymore One more strange behaviour I noticed is that one of client nodes gets disconnected after one of server nodes goes down. I have reconnect logic in place so it comes back later but is such behaviour expected? Not sure whether it is related with consistent IDs but I didn't see it earlier... BTW, after configuring consistent IDs I see them in "control.bat --baseline" output only. Visor output and server logs still show generated IDs That looks confusing... Cluster state: active Current topology version: 7 Baseline nodes: ConsistentID=V-HP-LK-DCN01, STATE=ONLINE ConsistentID=V-HP-LK-DCN02, STATE=ONLINE 9871EAFF(@n0) | Server BBA63A1F(@n2) | Server 1DEDB701(@n1) | Client 5931AF53(@n3) | Client logs\v-hp-lk-dcn01\ignite.log:383:>>> Local node [ID=9871EAFF-73AF-4E2E-99A7-8F5DF58A3C40, order=1, clientMode=false] logs\v-hp-lk-dcn02\ignite.log:274:>>> Local node [ID=BBA63A1F-559E-461C-B7ED-B10CE3DE33CC, order=7, clientMode=false] On Tue, Jun 12, 2018 at 9:48 PM, Olexandr K wrote: Hi, Dmitry server nodes start with ignite-server.xml and client nodes with ignite-client.xml server node hosts: v-hp-lk-dcn01, v-hp-lk-dcn02 http://www.springframework.org/schema/beans; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd;> v-hp-lk-dcn01:47500..47504 v-hp-lk-dcn02:47500..47504 http://www.springframework.org/schema/beans; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://www.springframework.org/schema/beans http://ww
Re: Baseline topology issue when restarting server nodes one by one
I configured ConsistentId equal to hostname for each node and this issue is not reproduced anymore One more strange behaviour I noticed is that one of client nodes gets disconnected after one of server nodes goes down. I have reconnect logic in place so it comes back later but is such behaviour expected? Not sure whether it is related with consistent IDs but I didn't see it earlier... BTW, after configuring consistent IDs I see them in "control.bat --baseline" output only. Visor output and server logs still show generated IDs That looks confusing... Cluster state: active Current topology version: 7 Baseline nodes: ConsistentID=V-HP-LK-DCN01, STATE=ONLINE ConsistentID=V-HP-LK-DCN02, STATE=ONLINE 9871EAFF(@n0) | Server BBA63A1F(@n2) | Server 1DEDB701(@n1) | Client 5931AF53(@n3) | Client logs\v-hp-lk-dcn01\ignite.log:383:>>> Local node [ID=9871EAFF-73AF-4E2E-99A7-8F5DF58A3C40, order=1, clientMode=false] logs\v-hp-lk-dcn02\ignite.log:274:>>> Local node [ID=BBA63A1F-559E-461C-B7ED-B10CE3DE33CC, order=7, clientMode=false] On Tue, Jun 12, 2018 at 9:48 PM, Olexandr K wrote: > Hi, Dmitry > > server nodes start with ignite-server.xml and client nodes with > ignite-client.xml > server node hosts: v-hp-lk-dcn01, v-hp-lk-dcn02 > > > > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > > class="org.apache.ignite.configuration.IgniteConfiguration"> > > > /> > > > > > > > > > > > > > value="s-hp-fs01\\dev$\\config\\keystore.jks" > /> > > > factory-method="getDisabledTrustManager" /> > > > > > > > value="S-hp-fs01\\dev$\\config\\log4j2.xml"/> > > > > > > > > value="auth_durable_region"/> > > value="FULL_ASYNC"/> > > > > > > > > > > > > > > > > > > > > > > > > > > value="auth_durable_region"/> > value="true"/> > > /> > > > > > > > > > > > > > > > v-hp-lk-dcn01:47500..47504 > v-hp-lk-dcn02:47500..47504 > > > > > > > > > > > > > > http://www.springframework.org/schema/beans; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; > xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > > > > /> > > > > value="s-hp-fs01\\dev$\\config\\keystore.jks" /> > > > factory-method="getDisabledTrustManager" /> > > > > > > > value="S-hp-fs01\\dev$\\config\\ignite-log4j2.xml" > /> > > > > > > > > value="auth_durable_region" /> > > value="FULL_ASYNC" /> > > > > > > > > > > > > > > value="f:\\ignite-wal\\archive" > /> > > > > /> > > > > > > > > value="auth_durable_region" /> > value="true" /> > > >
Re: Baseline topology issue when restarting server nodes one by one
Hi, Dmitry server nodes start with ignite-server.xml and client nodes with ignite-client.xml server node hosts: v-hp-lk-dcn01, v-hp-lk-dcn02 http://www.springframework.org/schema/beans; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd;> v-hp-lk-dcn01:47500..47504 v-hp-lk-dcn02:47500..47504 http://www.springframework.org/schema/beans; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd;> v-hp-lk-dcn01:47500..47504 v-hp-lk-dcn02:47500..47504 On Tue, Jun 12, 2018 at 7:03 PM, dkarachentsev wrote: > Hi, > > What IgniteConfiguration do you use? Could you please share it? > > Thanks! > -Dmitry > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Baseline topology issue when restarting server nodes one by one
Hi, What IgniteConfiguration do you use? Could you please share it? Thanks! -Dmitry -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
