Hello! I'll start in the reverse order:
3. If everything is OK, the process should take around a second (for non-persistent cluster). 2. As soon you see "Topology snapshot" in the log, the process is complete. 1. From your configuration I see suspicious that it only lists two other nodes and not itself. It is recommended that any node can discover itself and start a cluster as 1st node. Try adding <value>[own ip address]</value> Regards, -- Ilya Kasnacheev пт, 21 февр. 2020 г. в 14:52, Devin Anderson <[email protected]>: > Hi Ilya, > > That certainly makes sense, but I'm not totally sure how to act on that > information yet. A couple questions: > > 1.) Given the configuration file I posted (which is basically the same on > each > node in the cluster, save that the IP addresses of the nodes to discover > are > different), should each of the nodes eventually join the cluster? Am I > missing > anything? > 2.) Is there a message that I should see in the logs that identifies that > the > node has successfully joined the cluster? > 3.) How long should I expect the process to take with three nodes (just a > loose > estimate)? > > Thanks for your ongoing help. It's much appreciated. :) > > Devin > > On 2/21/20 3:23 AM, Ilya Kasnacheev wrote: > > Hello! > > > > Your node has never finished joining to cluster nor was able to > self-discover > > to form a cluster of its own, as evident by: > > > > > > "main" #1 prio=5 os_prio=0 tid=0x00007f7e8000d000 nid=0x5fed waiting on > > condition [0x00007f7e8875f000] > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > at java.lang.Thread.sleep(Native Method) > > at > org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:7778) > > at > > > org.apache.ignite.spi.discovery.tcp.*ServerImpl.sendJoinRequestMessage*(ServerImpl.java:1131) > > at > > > org.apache.ignite.spi.discovery.tcp.*ServerImpl.joinTopology*(ServerImpl.java:910) > > at > org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:391) > > at > > > org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2020) > > at > > > org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297) > > at > > > org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:939) > > at > org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1682) > > at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1066) > > at > > > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038) > > at > > > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730) > > - locked <0x00000005e8cca5b8> (a > > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance) > > at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158) > > at > > > org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1076) > > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:962) > > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:861) > > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:731) > > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:700) > > at org.apache.ignite.Ignition.start(Ignition.java:348) > > at > > > org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:301) > > > > Until node is in topology, obviously it can't serve any requests. > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > пт, 21 февр. 2020 г. в 01:55, Devin Anderson <[email protected] > > <mailto:[email protected]>>: > > > > Hi Ilya, > > > > I'm attaching the `jstack` dump to this message. When I took the > dump, > > there > > were ten requests that had been made to the Apache Ignite REST API > using > > POST > > data of the form: > > > > cmd=add&key=[key]&value=[value] > > > > ... where [key] and [value] were proper URI encoded data. > > > > Thanks in advance for any help. I'll take a look as well and try to > > figure out > > what's going on. > > > > Devin > > > > On 2/20/20 5:43 AM, Ilya Kasnacheev wrote: > > > Hello! > > > > > > Please collect thread dump (jstack) from affected node, share it > with us. > > > > > > Regards, > > > -- > > > Ilya Kasnacheev > > > > > > > > > чт, 20 февр. 2020 г. в 16:17, Devin Anderson <[email protected] > > <mailto:[email protected]> > > > <mailto:[email protected] <mailto:[email protected]>>>: > > > > > > Hi all, > > > > > > I'm seeing issues wherein the Apache Ignite REST API appears > to accept > > > requests, but doesn't ever reply. This doesn't always happen; > for > > > example, if > > > I make a request that I expect the API to reject, I get back a > > response: > > > > > > ---------- > > > > > > # curl -v -X GET 'http://127.0.0.1:8080/ignite > > < > https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080_ignite&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=O-a0zsxmvZZBVCsKS7owS9oPa3AKjd7Xpau8laEriDM&m=nxds_HJL4Wp8JgHdOCFCwfFGfocNDpGEKHbOigV9HgY&s=Hhk_7aUuf0SyczRr-esByr4-rhF7d6_AdK2FyS_MjFo&e= > > > > > > > < > https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080_ignite&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=O-a0zsxmvZZBVCsKS7owS9oPa3AKjd7Xpau8laEriDM&m=0D1iM7MuHAG3uQxX5V8Iv_9kNyO25FNACjcqQnHaaLs&s=Q3T6HlQSi3TnKwaRcPsb0MWMRbFNexaNfrucVQSrTkI&e= > >' > > > * Hostname was NOT found in DNS cache > > > * Trying 127.0.0.1... > > > * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0) > > > > GET /ignite HTTP/1.1 > > > > User-Agent: curl/7.35.0 > > > > Host: 127.0.0.1:8080 > > < > https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=O-a0zsxmvZZBVCsKS7owS9oPa3AKjd7Xpau8laEriDM&m=nxds_HJL4Wp8JgHdOCFCwfFGfocNDpGEKHbOigV9HgY&s=VEZ1YiACosFkygqRYUfQg4SBv12OI3TTYzpQDOhT8bE&e= > > > > > > > < > https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=O-a0zsxmvZZBVCsKS7owS9oPa3AKjd7Xpau8laEriDM&m=0D1iM7MuHAG3uQxX5V8Iv_9kNyO25FNACjcqQnHaaLs&s=gS2lm03jyjMVp9ZTWC259JI_RG9y4l9QsuvxX4Cz85c&e= > > > > > > Accept: */* > > > > > > > < HTTP/1.1 400 Bad Request > > > < Date: Thu, 20 Feb 2020 11:21:22 GMT > > > < Content-Type: application/json;charset=utf-8 > > > < Content-Length: 0 > > > * Server Jetty(9.4.11.v20180605) is not blacklisted > > > < Server: Jetty(9.4.11.v20180605) > > > < > > > * Connection #0 to host 127.0.0.1 left intact > > > > > > ---------- > > > > > > I expect the (above) request above to fail because I don't > supply the > > > required > > > `cmd` parameter. > > > > > > However, if I make a request that I believe should succeed, I > never > > > receive a > > > response: > > > > > > ---------- > > > > > > # curl -v 'http://127.0.0.1:8080/ignite?cmd=version > > < > https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080_ignite-3Fcmd-3Dversion&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=O-a0zsxmvZZBVCsKS7owS9oPa3AKjd7Xpau8laEriDM&m=nxds_HJL4Wp8JgHdOCFCwfFGfocNDpGEKHbOigV9HgY&s=b1MZBDEsOiVfh5l7ltu2DgXaYnh_5XtnKPg1RgsDfTI&e= > > > > > > > < > https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080_ignite-3Fcmd-3Dversion&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=O-a0zsxmvZZBVCsKS7owS9oPa3AKjd7Xpau8laEriDM&m=0D1iM7MuHAG3uQxX5V8Iv_9kNyO25FNACjcqQnHaaLs&s=USDfyc0TCRCJRx4vzXe9ezgsCMJ8YfFh2IK983DMgrc&e= > >' > > > * Hostname was NOT found in DNS cache > > > * Trying 127.0.0.1... > > > * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0) > > > > GET /ignite?cmd=version HTTP/1.1 > > > > User-Agent: curl/7.35.0 > > > > Host: 127.0.0.1:8080 > > < > https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=O-a0zsxmvZZBVCsKS7owS9oPa3AKjd7Xpau8laEriDM&m=nxds_HJL4Wp8JgHdOCFCwfFGfocNDpGEKHbOigV9HgY&s=VEZ1YiACosFkygqRYUfQg4SBv12OI3TTYzpQDOhT8bE&e= > > > > > > > < > https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=O-a0zsxmvZZBVCsKS7owS9oPa3AKjd7Xpau8laEriDM&m=0D1iM7MuHAG3uQxX5V8Iv_9kNyO25FNACjcqQnHaaLs&s=gS2lm03jyjMVp9ZTWC259JI_RG9y4l9QsuvxX4Cz85c&e= > > > > > > Accept: */* > > > > > > > > > > ---------- > > > > > > `curl` will wait forever for a response that never arrives. > > > > > > I tried looking for logs output by the REST API so that I could > > debug the > > > issue, but failed to find any log entries emitted from the > REST API. > > > > > > Given that I'm new to Apache Ignite, I suspect that I'm almost > > certainly > > > doing > > > something wrong, but I have absolutely no idea what that could > be. > > > > > > I have a few questions: > > > > > > 1.) Has anyone encountered an issue similar to this with the > REST API? > > > If so, > > > how did you solve it? > > > 2.) Can anyone tell me where I might find REST API log entries? > > > 3.) I have persistence enabled in the XML configuration. Would > that > > conflict > > > with the REST API? > > > 4.) I have additional nodes (3 nodes in total) in the XML > > configuration. > > > AFAICT from log entries and some liberal use of `ss`, the > nodes are > > > communicating with each other. Would having multiple nodes > > conflict with > > > the > > > REST API? > > > 5.) Is there anything else I'm missing that might not be > obvious to me > > > because > > > I'm new to Apache Ignite? > > > > > > Other than (3) and (4), I think the configuration file I'm > using is > > > rather bland: > > > > > > ---------- > > > > > > <beans xmlns="http://www.springframework.org/schema/beans > > < > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.springframework.org_schema_beans&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=O-a0zsxmvZZBVCsKS7owS9oPa3AKjd7Xpau8laEriDM&m=nxds_HJL4Wp8JgHdOCFCwfFGfocNDpGEKHbOigV9HgY&s=QS1N4uN0i6B10p6B-YPORLbXGqiH4ve3DC9FLR-9HH4&e= > > > > > > > < > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.springframework.org_schema_beans&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=O-a0zsxmvZZBVCsKS7owS9oPa3AKjd7Xpau8laEriDM&m=0D1iM7MuHAG3uQxX5V8Iv_9kNyO25FNACjcqQnHaaLs&s=uORaM4cyX6R3dKSjnm0_K8A4230IGy7LPWUzRAG_IV4&e= > >" > > > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance > > < > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.w3.org_2001_XMLSchema-2Dinstance&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=O-a0zsxmvZZBVCsKS7owS9oPa3AKjd7Xpau8laEriDM&m=nxds_HJL4Wp8JgHdOCFCwfFGfocNDpGEKHbOigV9HgY&s=UyopUd5Ot09OGVA6KNi_oTS8JCxjI42wdeGJQbrC2UQ&e= > > > > > > > < > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.w3.org_2001_XMLSchema-2Dinstance&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=O-a0zsxmvZZBVCsKS7owS9oPa3AKjd7Xpau8laEriDM&m=0D1iM7MuHAG3uQxX5V8Iv_9kNyO25FNACjcqQnHaaLs&s=vdA3V10EDd-kLU4fjTYawM1o1NL-UvHO1iiW4VxHUyw&e= > >" > > > xsi:schemaLocation=" > > > http://www.springframework.org/schema/beans > > < > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.springframework.org_schema_beans&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=O-a0zsxmvZZBVCsKS7owS9oPa3AKjd7Xpau8laEriDM&m=nxds_HJL4Wp8JgHdOCFCwfFGfocNDpGEKHbOigV9HgY&s=QS1N4uN0i6B10p6B-YPORLbXGqiH4ve3DC9FLR-9HH4&e= > > > > > > > < > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.springframework.org_schema_beans&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=O-a0zsxmvZZBVCsKS7owS9oPa3AKjd7Xpau8laEriDM&m=0D1iM7MuHAG3uQxX5V8Iv_9kNyO25FNACjcqQnHaaLs&s=uORaM4cyX6R3dKSjnm0_K8A4230IGy7LPWUzRAG_IV4&e= > > > > > http://www.springframework.org/schema/beans/spring-beans.xsd > > < > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.springframework.org_schema_beans_spring-2Dbeans.xsd&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=O-a0zsxmvZZBVCsKS7owS9oPa3AKjd7Xpau8laEriDM&m=nxds_HJL4Wp8JgHdOCFCwfFGfocNDpGEKHbOigV9HgY&s=1W2X_AlApjVwM2r9Q9I8hQniVWhjpGvzeGWSbA7z0Oc&e= > > > > > > > < > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.springframework.org_schema_beans_spring-2Dbeans.xsd&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=O-a0zsxmvZZBVCsKS7owS9oPa3AKjd7Xpau8laEriDM&m=0D1iM7MuHAG3uQxX5V8Iv_9kNyO25FNACjcqQnHaaLs&s=Reg5uEVfbW-0vucPqzp0oUT2MSnUoevTyriLXqXkAkY&e= > >"> > > > <bean > class="org.apache.ignite.configuration.IgniteConfiguration"> > > > > > > <property name="dataStorageConfiguration"> > > > <bean > > > class="org.apache.ignite.configuration.DataStorageConfiguration"> > > > <!-- > > > Default memory region that grows endlessly. A cache > is bound > > > to this > > > memory region unless it sets another one in its > > > CacheConfiguration/ > > > --> > > > <property name="defaultDataRegionConfiguration"> > > > <bean > > > class="org.apache.ignite.configuration.DataRegionConfiguration"> > > > <property name="persistenceEnabled" value="true"/> > > > <property name="name" value="Default_Region"/> > > > <!-- 100 MB memory region with disabled eviction > --> > > > <property name="initialSize" value="#{100 * 1024 * > > 1024}"/> > > > </bean> > > > </property> > > > </bean> > > > </property> > > > > > > <property name="discoverySpi"> > > > <bean > > class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> > > > <property name="ipFinder"> > > > <bean > > > > > > > class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder"> > > > <property name="addresses"> > > > <list> > > > <value>[other node > address]</value><value>[other node > > > address]</value> > > > </list> > > > </property> > > > </bean> > > > </property> > > > </bean> > > > </property> > > > > > > </bean> > > > </beans> > > > > > > ---------- > > > > > > (Obviously, each instance of '[other node address]' is > substituted > > with a > > > real > > > IP address in the actual files I'm using.) > > > > > > Thanks in advance for any help. > > > > > > -- > > > Devin > > > > > > >
