I got it to work without my iptables rules loaded by changing my discovery settings on both my host and container to:
discovery.zen.ping.unicast.hosts: ["172.17.42.1:9300", "172.17.42.1:9301"] Thanks for all of your help! Knowing Docker needed UDP access helped quite a bit. On Monday, July 14, 2014 12:01:52 PM UTC-4, Tony P. wrote: > > Ah, learned something new. I've updated my iptables to accept tcp and udp > for 9200:9400. However, I'm still running into the same error. I'm still > getting the earlier mentioned error message when using the container's IP > (not ideal). When I use the following unicast list on the host: > > host: discovery.zen.ping.unicast.hosts: ["172.17.42.1:9200", " > 172.17.42.1:9201"] > > I get the following on the host and the associated exception on the > container: > > [2014-07-14 11:48:50,357][WARN ][discovery.zen.ping.unicast] > [elasticsearch-node-00] failed to send ping to > [[#zen_unicast_2#][elasticsearch-node-00][inet[/172.17.42.1:9201]]] > org.elasticsearch.transport.ReceiveTimeoutTransportException: > [][inet[/172.17.42.1:9201]][discovery/zen/unicast] request_id [0] timed out > after [3751ms] > at > org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:356) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744) > > I'm pretty sure this is an issue with my iptables config. I don't get the > stack traces on the container after flushing my iptable rules but I do > still receive the above error message on the host. I'm wondering if it has > something to do with these two lines that docker adds to my iptables rules > > 0 0 ACCEPT tcp -- !docker0 docker0 0.0.0.0/0 > 172.17.0.2 tcp dpt:9300 > 0 0 ACCEPT tcp -- !docker0 docker0 0.0.0.0/0 > 172.17.0.2 tcp dpt:9200 > > Do I need to manually add additional rules to allow the host to discover > the container? > > On Monday, July 14, 2014 9:48:58 AM UTC-4, Jörg Prante wrote: >> >> curl uses TCP, but for multicast/unicast, you need also UDP. >> >> Jörg >> >> >> On Mon, Jul 14, 2014 at 3:13 PM, Tony P. <[email protected]> wrote: >> >>> I've tried a few different things to no avail. >>> >>> Current setup: >>> # vain effort to attempt to have the host find the node in the docker >>> container >>> host: discovery.zen.ping.unicast.hosts: ["127.0.0.1", "172.17.0.14"] >>> container: discovery.zen.ping.unicast.hosts: ["127.0.0.1"] >>> # Which won't be particularly effective since the next time I restart >>> the docker container its IP will change. >>> >>> I've also tried: >>> # accessing the host from the docker container >>> host: discovery.zen.ping.unicast.hosts: ["127.0.0.1"] >>> container: discovery.zen.ping.unicast.hosts: ["127.0.0.1"] >>> >>> # having the host use the docker0 bridge which theoretically should have >>> worked since I mapped 9201 on docker0 to 9200 on the docker container >>> host: discovery.zen.ping.unicast.hosts: ["172.17.42.1:9200", " >>> 172.17.42.1:9201"] >>> >>> # same as above except using the host's actual IP and the same port >>> mapping across docker0 >>> host: discovery.zen.ping.unicast.hosts: ["xxx.xxx.222.206:9200", >>> "xxx.xxx.222.206:9201"] >>> >>> I can curl all of these IPs at 9200/9201 and get a response from the >>> desired ES node. >>> >>> On Friday, July 11, 2014 7:16:17 PM UTC-4, Mark Walkom wrote: >>> >>>> ES will try to connect via unicast based on whatever you have in your >>>> config. >>>> What does the discovery.zen.ping.unicast line look like? >>>> >>>> Regards, >>>> Mark Walkom >>>> >>>> Infrastructure Engineer >>>> Campaign Monitor >>>> email: [email protected] >>>> web: www.campaignmonitor.com >>>> >>>> >>>> On 12 July 2014 05:00, Tony P. <[email protected]> wrote: >>>> >>>>> I've been playing with Elasticsearch and had a working cluster in a >>>>> multicast environment using VMs. I recently tried adapting that to work >>>>> within Docker and I'm running into a wall with the unicast configuration. >>>>> >>>>> Current setup is two nodes: one on the host and another in a docker >>>>> container (dockerfile/elasticsearch) >>>>> >>>>> I'm running the container with: >>>>> $ docker run -d -h "elasticsearch-node-01" >>>>> --name="elasticsearch-node-01" \ >>>>> -p 9201:9200 -p 9301:9300 -v \ >>>>> /etc/elasticsearch/cluster/:/data \ >>>>> dockerfile/elasticsearch /elasticsearch/bin/elasticsearch \ >>>>> -Des.config=/data/elasticsearch.yml >>>>> >>>>> It spins up on the docker0 bridge with some IP, 172.17.0.xx >>>>> docker0 is bound to 172.17.42.1 >>>>> 127.0.0.1 refers to the host in this example >>>>> >>>>> I can access the elasticsearch node in the container with any of the >>>>> commands: >>>>> $ curl 127.0.0.1:9201 >>>>> $ curl 172.17.42.1:9201 >>>>> $ curl 172.17.0.xx:9200 >>>>> >>>>> But when I add it to a cluster via unicast, I see an exception thrown >>>>> with "No route to host" being the reason. >>>>> >>>>> After trying a few different IPs to see which could be accessed >>>>> through the bridge, it seems that the container doesn't know how to speak >>>>> to the host to join the host node's cluster or tell the host node that it >>>>> has a cluster that can be joined. The host node logs also shows a similar >>>>> error: >>>>> >>>>> [2014-07-10 12:00:37,264][INFO][discovery.zen] >>>>> [elasticsearch-node-test] failed to send join request to master >>>>> [[elasticsearch-node-01][I3LiEOyeSome3djzr37uuQ][ >>>>> elasticsearch-node-01][inet[/172.17.0.14:9300]]], reason >>>>> [org.elasticsearch.transport.RemoteTransportException: >>>>> [elasticsearch-node-01][inet[/172.17.0.14:9300]][discovery/zen/join]; >>>>> org.elasticsearch.transport.ConnectTransportException: >>>>> [elasticsearch-node-test][inet[/128.59.222.215:9300]] >>>>> connect_timeout[30s]; java.net.NoRouteToHostException: No route to >>>>> host] >>>>> >>>>> There are two conversations I've found here with docker, elasticsearch >>>>> and unicast but neither provide an answer to my issue: >>>>> https://groups.google.com/d/msg/elasticsearch/OsGJcxuW1vI/qybPOrgE4fMJ >>>>> https://groups.google.com/d/msg/elasticsearch/2p9jXbCwRC8/mm4BPt5iQfgJ >>>>> >>>>> Any ideas on what I'm doing wrong? Is it because elasticsearch is >>>>> attempting to search based on the node's hostname (elasticsearch-node-01) >>>>> instead of the IP address? The hostname, elasticsearch-node-01 isn't >>>>> valid >>>>> since it's just generated by passing it to docker. Should I use the IP as >>>>> the hostname if that's what ES is using to add the node to the cluster? >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "elasticsearch" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> >>>>> To view this discussion on the web visit https://groups.google.com/d/ >>>>> msgid/elasticsearch/d52a92e3-0956-48b6-8816-cea5ed31a412% >>>>> 40googlegroups.com >>>>> <https://groups.google.com/d/msgid/elasticsearch/d52a92e3-0956-48b6-8816-cea5ed31a412%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/a6f26f35-9b9e-44e6-9392-8f1bdfb2b1ea%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/elasticsearch/a6f26f35-9b9e-44e6-9392-8f1bdfb2b1ea%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ac7e5c06-9c8e-4cc7-acc2-6f75bf672fb9%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
