[jira] [Commented] (MESOS-9269) Mesos UCR with Docker only Works on Host
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632654#comment-16632654
]
z s commented on MESOS-9269:
Investigating further, I decided to deploy a DC/OS cluster to view how those
iptables are configured. I'm not sure what exactly the diff is between the two
but the MESOS UCR works perfectly on DC/OS.
The DC/OS cluster iptables seem to have the same original configuration as the
Mesos/Marathon-only cluster:
DC/OS Cluster:
{code:java}
$ sudo iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy DROP)
target prot opt source destination
DOCKER-USER all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-1 all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate
RELATED,ESTABLISHED
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate
RELATED,ESTABLISHED
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate
RELATED,ESTABLISHED
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain DOCKER (3 references)
target prot opt source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
RETURN all -- anywhere anywhere
Chain DOCKER-ISOLATION-STAGE-2 (3 references)
target prot opt source destination
DROP all -- anywhere anywhere
DROP all -- anywhere anywhere
DROP all -- anywhere anywhere
RETURN all -- anywhere anywhere
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere{code}
{code:java}
$ sudo iptables -L -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
DOCKER all -- anywhere anywhere ADDRTYPE match
dst-type LOCAL
UCR-DEFAULT-BRIDGE all -- anywhere anywhere ADDRTYPE
match dst-type LOCAL
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
DOCKER all -- anywhere
!ip-127-0-0-0.us-west-2.compute.internal/8 ADDRTYPE match dst-type LOCAL
UCR-DEFAULT-BRIDGE all -- anywhere
!ip-127-0-0-0.us-west-2.compute.internal/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
MASQUERADE all -- anywhere anywhere vdir ORIGINAL
vmethod MASQ /* Minuteman-IPVS-IPTables-masquerade-rule */
MASQUERADE all -- ip-172-17-0-0.us-west-2.compute.internal/16 anywhere
MASQUERADE all -- 9.0.0.0/8 anywhere match-set
overlay dst
CNI-1ca4fce35f5dae9dad10d9ba all --
ip-172-31-254-0.us-west-2.compute.internal/24 anywhere /* name:
"mesos-bridge" id: "6e424731-3d67-4d37-8f67-fc94972af19c" */
Chain CNI-1ca4fce35f5dae9dad10d9ba (1 references)
target prot opt source destination
ACCEPT all -- anywhere
ip-172-31-254-0.us-west-2.compute.internal/24 /* name: "mesos-bridge" id:
"6e424731-3d67-4d37-8f67-fc94972af19c" */
MASQUERADE all -- anywhere !base-address.mcast.net/4 /* name:
"mesos-bridge" id: "6e424731-3d67-4d37-8f67-fc94972af19c" */
Chain DOCKER (2 references)
target prot opt source destination
RETURN all -- anywhere anywhere
Chain UCR-DEFAULT-BRIDGE (2 references)
target prot opt source destination
DNAT
[jira] [Comment Edited] (MESOS-9269) Mesos UCR with Docker only Works on Host
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631316#comment-16631316
]
z s edited comment on MESOS-9269 at 9/28/18 3:57 AM:
-
[~dgoel] are you suggesting that the port-mapper plugin currently can be
configured to add the iptables? Or are you proposing a new feature/binary
change?
Any suggestions on a workaround so that the iptable rules just work out of the
box?
was (Author: dkjs):
[~dgoel] are you suggesting that the port-mapper plugin currently can be
configured to add the iptables? Or are you proposing a new feature/binary
change?
Any suggesting on a workaround so that the iptable rules just work out of the
box?
> Mesos UCR with Docker only Works on Host
>
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9269) Mesos UCR with Docker only Works on Host
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631316#comment-16631316
]
z s commented on MESOS-9269:
[~dgoel] are you suggesting that the port-mapper plugin currently can be
configured to add the iptables? Or are you proposing a new feature/binary
change?
Any suggesting on a workaround so that the iptable rules just work out of the
box?
> Mesos UCR with Docker only Works on Host
>
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9269) Mesos UCR with Docker only Works on Host
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631227#comment-16631227
]
z s commented on MESOS-9269:
Any suggestions as to how to permanently fix these rules? Not sure why the
docker default rules are incompatible with the CNI/Mesos rules.
> Mesos UCR with Docker only Works on Host
>
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (MESOS-9269) Mesos UCR with Docker only Works on Host
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
z s updated MESOS-9269:
---
Comment: was deleted
(was: Setting the following rules seems to have helped:
# sudo iptables -D DOCKER-ISOLATION-STAGE-2 -j RETURN
# sudo iptables -I DOCKER-ISOLATION-STAGE-2 1 -j RETURN
See [Marathon
Jira|https://issues.apache.org/jira/browse/MESOS-9269?focusedCommentId=16631202=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16631202])
> Mesos UCR with Docker only Works on Host
>
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9269) Mesos UCR with Docker only Works on Host
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631209#comment-16631209
]
z s commented on MESOS-9269:
Setting the following rules seems to have helped:
# sudo iptables -D DOCKER-ISOLATION-STAGE-2 -j RETURN
# sudo iptables -I DOCKER-ISOLATION-STAGE-2 1 -j RETURN
See [Marathon
Jira|https://issues.apache.org/jira/browse/MESOS-9269?focusedCommentId=16631202=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16631202]
> Mesos UCR with Docker only Works on Host
>
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9269) Mesos UCR with Docker only Works on Host
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631205#comment-16631205
]
z s commented on MESOS-9269:
Thanks! [~dgoel]
That solves the issue for external connectivity!
However, we still cannot curl localhost:
{code:java}
$ curl localhost:26036
curl: (7) Failed to connect to localhost port 26036: Connection refused
ubuntu@ip-172-27-
$ curl 127.0.0.1:26036
curl: (7) Failed to connect to 127.0.0.1 port 26036: Connection refused
{code}
> Mesos UCR with Docker only Works on Host
>
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (MESOS-9269) Mesos UCR with Docker only Works on Localhost
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16630874#comment-16630874
]
z s edited comment on MESOS-9269 at 9/27/18 6:37 PM:
-
There must be something trivial that's incorrect with the routing table
configuration.
I still see that on the same host the following works:
{code:java}
$ curl :[port]
[http response]{code}
but none of these work:
{code:java}
$ curl localhost:[port]
curl: (7) Failed to connect to localhost port [port]: Connection refused
$ curl 172.0.0.1:[port]
curl: (7) Failed to connect to localhost port [port]: Connection refused
{code}
And (obviously) curl does not work if it's from a remote host.
was (Author: dkjs):
There must be something trivial that's incorrect with the routing table
configuration.
I still see that on the same host the following works:
{code:java}
$ curl :[port]
[http response]{code}
but none of these work:
{code:java}
$ curl localhost:[port]
curl: (7) Failed to connect to 172.0.0.1 port 10081: Connection timed out
$ curl 172.0.0.1:[port]
curl: (7) Failed to connect to 172.0.0.1 port 10081: Connection timed out
{code}
And (obviously) curl does not work if it's from a remote host.
> Mesos UCR with Docker only Works on Localhost
> -
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9269) Mesos UCR with Docker only Works on Localhost
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16630874#comment-16630874
]
z s commented on MESOS-9269:
There must be something trivial that's incorrect with the routing table
configuration.
I still see that on the same host the following works:
{code:java}
$ curl :[port]
[http response]{code}
but none of these work:
{code:java}
$ curl localhost:[port]
curl: (7) Failed to connect to 172.0.0.1 port 10081: Connection timed out
$ curl 172.0.0.1:[port]
curl: (7) Failed to connect to 172.0.0.1 port 10081: Connection timed out
{code}
And (obviously) curl does not work if it's from a remote host.
> Mesos UCR with Docker only Works on Localhost
> -
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (MESOS-9269) Mesos UCR with Docker only Works on Localhost
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629747#comment-16629747
]
z s edited comment on MESOS-9269 at 9/27/18 4:43 AM:
-
The last line seems to imply that the nat rule is correct. The service is
assigned to port 22555 on the host machine.
{code:java}
Chain MESOS-BRIDGE-PORT-MAPPER (2 references) target prot opt source
destination DNAT tcp -- anywhere anywhere
tcp dpt:22555 /* container_id: 48d3e3f9-9d37-4da7-9011-586138cd5e74
*/ to:10.1.0.3:22555
{code}
ifconfig on host:
{code:java}
mesos-cni0 Link encap:Ethernet HWaddr e6:96:93:6a:f4:1d
inet addr:10.1.0.1 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::e496:93ff:fe6a:f41d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:196 errors:0 dropped:0 overruns:0 frame:0
TX packets:31 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:17473 (17.4 KB) TX bytes:6322 (6.3 KB)
{code}
ifconfig from inside the container also seems to be consistent:
{code:java}
eth0 Link encap:Ethernet HWaddr 5E:43:B2:17:A3:2F
inet addr:10.1.0.3 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::5c43:b2ff:fe17:a32f/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:24 errors:0 dropped:0 overruns:0 frame:0
TX packets:17 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:5832 (5.6 KiB) TX bytes:2975 (2.9 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
{code}
was (Author: dkjs):
The last line seems to imply that the nat rule is correct:
{code:java}
Chain MESOS-BRIDGE-PORT-MAPPER (2 references) target prot opt source
destination DNAT tcp -- anywhere anywhere
tcp dpt:22555 /* container_id: 48d3e3f9-9d37-4da7-9011-586138cd5e74
*/ to:10.1.0.3:22555
{code}
The service is assigned to port 22555 on the host machine. ifconfig from inside
the container also seems to be consistent:
{code:java}
eth0 Link encap:Ethernet HWaddr 5E:43:B2:17:A3:2F
inet addr:10.1.0.3 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::5c43:b2ff:fe17:a32f/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:24 errors:0 dropped:0 overruns:0 frame:0
TX packets:17 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:5832 (5.6 KiB) TX bytes:2975 (2.9 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
{code}
> Mesos UCR with Docker only Works on Localhost
> -
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9269) Mesos UCR with Docker only Works on Localhost
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629747#comment-16629747
]
z s commented on MESOS-9269:
The last line seems to imply that the nat rule is correct:
{code:java}
Chain MESOS-BRIDGE-PORT-MAPPER (2 references) target prot opt source
destination DNAT tcp -- anywhere anywhere
tcp dpt:22555 /* container_id: 48d3e3f9-9d37-4da7-9011-586138cd5e74
*/ to:10.1.0.3:22555
{code}
The service is assigned to port 22555 on the host machine. ifconfig from inside
the container also seems to be consistent:
{code:java}
eth0 Link encap:Ethernet HWaddr 5E:43:B2:17:A3:2F
inet addr:10.1.0.3 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::5c43:b2ff:fe17:a32f/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:24 errors:0 dropped:0 overruns:0 frame:0
TX packets:17 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:5832 (5.6 KiB) TX bytes:2975 (2.9 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
{code}
> Mesos UCR with Docker only Works on Localhost
> -
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9269) Mesos UCR with Docker only Works on Localhost
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629744#comment-16629744
]
z s commented on MESOS-9269:
I also see that if I change the command to ping the external internet, that
network request "hangs":
{code:java}
...
"cmd": "ping google.com",
...{code}
[~jieyu] here's the iptables nat
{code:java}
$ sudo iptables -t nat -L -n -v
Chain PREROUTING (policy ACCEPT 33 packets, 2732 bytes)
pkts bytes target prot opt in out source destination
72 4344 DOCKER all -- * * 0.0.0.0/0 0.0.0.0/0
ADDRTYPE match dst-type LOCAL
46 2784 MESOS-BRIDGE-PORT-MAPPER all -- * * 0.0.0.0/0
0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT 2 packets, 128 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 4 packets, 335 bytes)
pkts bytes target prot opt in out source destination
81 6005 DOCKER all -- * * 0.0.0.0/0 !127.0.0.0/8
ADDRTYPE match dst-type LOCAL
5 406 MESOS-BRIDGE-PORT-MAPPER all -- * * 0.0.0.0/0
!127.0.0.0/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT 4 packets, 335 bytes)
pkts bytes target prot opt in out source destination
9 540 MASQUERADE all -- * !docker0 172.17.0.0/16
0.0.0.0/0
0 0 MASQUERADE tcp -- * * 172.17.0.2 172.17.0.2
tcp dpt:3888
0 0 MASQUERADE tcp -- * * 172.17.0.2 172.17.0.2
tcp dpt:2888
0 0 MASQUERADE tcp -- * * 172.17.0.2 172.17.0.2
tcp dpt:2181
0 0 MASQUERADE tcp -- * * 172.17.0.3 172.17.0.3
tcp dpt:8081
0 0 MASQUERADE tcp -- * * 172.17.0.3 172.17.0.3
tcp dpt:8080
0 0 CNI-82e39e8d2e77928aff7dd8f0 all -- * * 10.1.0.0/16
0.0.0.0/0 /* name: "mesos-bridge" id:
"48d3e3f9-9d37-4da7-9011-586138cd5e74" */
Chain CNI-82e39e8d2e77928aff7dd8f0 (1 references)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all -- * * 0.0.0.0/0 10.1.0.0/16
/* name: "mesos-bridge" id: "48d3e3f9-9d37-4da7-9011-586138cd5e74" */
0 0 MASQUERADE all -- * * 0.0.0.0/0
!224.0.0.0/4 /* name: "mesos-bridge" id:
"48d3e3f9-9d37-4da7-9011-586138cd5e74" */
Chain DOCKER (2 references)
pkts bytes target prot opt in out source destination
0 0 RETURN all -- docker0 * 0.0.0.0/0 0.0.0.0/0
15 900 DNAT tcp -- !docker0 * 0.0.0.0/0 0.0.0.0/0
tcp dpt:3888 to:172.17.0.2:3888
0 0 DNAT tcp -- !docker0 * 0.0.0.0/0 0.0.0.0/0
tcp dpt:2888 to:172.17.0.2:2888
11 660 DNAT tcp -- !docker0 * 0.0.0.0/0 0.0.0.0/0
tcp dpt:2181 to:172.17.0.2:2181
0 0 DNAT tcp -- !docker0 * 0.0.0.0/0 0.0.0.0/0
tcp dpt:8081 to:172.17.0.3:8081
0 0 DNAT tcp -- !docker0 * 0.0.0.0/0 0.0.0.0/0
tcp dpt:8080 to:172.17.0.3:8080
Chain MESOS-BRIDGE-PORT-MAPPER (2 references)
pkts bytes target prot opt in out source destination
0 0 DNAT tcp -- !mesos-cni0 * 0.0.0.0/0
0.0.0.0/0 tcp dpt:22555 /* container_id:
48d3e3f9-9d37-4da7-9011-586138cd5e74 */ to:10.1.0.3:22555
{code}
> Mesos UCR with Docker only Works on Localhost
> -
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type":
[jira] [Commented] (MESOS-9269) Mesos UCR with Docker only Works on Localhost
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629732#comment-16629732
]
z s commented on MESOS-9269:
[~jieyu] It looks like the iptables on my host machines are not correct. I
don't see anything specific for the Mesos UCR:
{code:java}
$ sudo iptables -nL
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
DOCKER-USER all -- 0.0.0.0/0 0.0.0.0/0
DOCKER-ISOLATION-STAGE-1 all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate
RELATED,ESTABLISHED
DOCKER all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain DOCKER (1 references)
target prot opt source destination
ACCEPT tcp -- 0.0.0.0/0 172.17.0.2 tcp dpt:3888
ACCEPT tcp -- 0.0.0.0/0 172.17.0.2 tcp dpt:2888
ACCEPT tcp -- 0.0.0.0/0 172.17.0.2 tcp dpt:2181
ACCEPT tcp -- 0.0.0.0/0 172.17.0.3 tcp dpt:8081
ACCEPT tcp -- 0.0.0.0/0 172.17.0.3 tcp dpt:8080
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all -- 0.0.0.0/0 0.0.0.0/0 {code}
> Mesos UCR with Docker only Works on Localhost
> -
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (MESOS-9269) Mesos UCR with Docker only Works on Localhost
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
z s updated MESOS-9269:
---
Comment: was deleted
(was: It seems like it's actually an application issue with the IP binding, not
sure the reason why though.
Explicitly binding to 0.0.0.0 does not work
{code:java}
python3 -m http.server --bind 0.0.0.0 $PORT0
{code}
vs
{code:java}
python3 -m http.server $PORT0
{code}
(the later appears to work without issue).
Is there an explaination for this? By default all of our docker services bind
to 0.0.0.0 to accept all incoming traffic. After trying to switch over to the
Mesos UCR we see this strange behavior. Are the UCR interfaces configurable or
documented somewhere?)
> Mesos UCR with Docker only Works on Localhost
> -
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (MESOS-9269) Mesos UCR with Docker only Works on Localhost
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629620#comment-16629620
]
z s edited comment on MESOS-9269 at 9/27/18 1:31 AM:
-
It seems like it's actually an application issue with the IP binding, not sure
the reason why though.
Explicitly binding to 0.0.0.0 does not work
{code:java}
python3 -m http.server --bind 0.0.0.0 $PORT0
{code}
vs
{code:java}
python3 -m http.server $PORT0
{code}
(the later appears to work without issue).
Is there an explaination for this? By default all of our docker services bind
to 0.0.0.0 to accept all incoming traffic. After trying to switch over to the
Mesos UCR we see this strange behavior. Are the UCR interfaces configurable or
documented somewhere?
was (Author: dkjs):
It seems like it's actually an application issue with the IP binding, not sure
the reason why though.
Explicitly binding to 0.0.0.0 does not work
{code:java}
python3 -m http.server --bind 0.0.0.0 $PORT0
{code}
vs
{code:java}
python3 -m http.server $PORT0
{code}
(the later appears to work without issue).
Is there an explaination for this? By default all of our docker services bind
to 0.0.0.0 to accept all incoming traffic. After trying to switch over to the
Mesos UCR we see this strange behavior.
> Mesos UCR with Docker only Works on Localhost
> -
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (MESOS-9269) Mesos UCR with Docker only Works on Localhost
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629620#comment-16629620
]
z s edited comment on MESOS-9269 at 9/27/18 1:26 AM:
-
It seems like it's actually an application issue with the IP binding, not sure
the reason why though.
Explicitly binding to 0.0.0.0 does not work
{code:java}
python3 -m http.server --bind 0.0.0.0 $PORT0
{code}
vs
{code:java}
python3 -m http.server $PORT0
{code}
(the later appears to work without issue).
Is there an explaination for this? By default all of our docker services bind
to 0.0.0.0 to accept all incoming traffic. After trying to switch over to the
Mesos UCR we see this strange behavior.
was (Author: dkjs):
It seems like it's actually an application issue with the IP binding, not sure
the reason why though.
Explicitly binding to 0.0.0.0 does not work
{code:java}
python3 -m http.server --bind 0.0.0.0 $PORT0
{code}
vs
{code:java}
python3 -m http.server $PORT0
{code}
(the later appears to work without issue)
> Mesos UCR with Docker only Works on Localhost
> -
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9269) Mesos UCR with Docker only Works on Localhost
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629620#comment-16629620
]
z s commented on MESOS-9269:
It seems like it's actually an application issue with the IP binding, not sure
the reason why though.
Explicitly binding to 0.0.0.0 does not work
{code:java}
python3 -m http.server --bind 0.0.0.0 $PORT0
{code}
vs
{code:java}
python3 -m http.server $PORT0
{code}
(the later appears to work without issue)
> Mesos UCR with Docker only Works on Localhost
> -
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (MESOS-9269) Mesos UCR with Docker only Works on Localhost
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629578#comment-16629578
]
z s edited comment on MESOS-9269 at 9/27/18 1:22 AM:
-
[~jieyu] , here's a simple configuration. I also noticed that it only works if
I specify the IP address of the host and not localhost. It sill only works when
i curl from the local host. This seems to suggest an issue with my configured
routes.
h3. Curl Command (from same host):
{code:java}
$ curl localhost:9873
curl: (7) Failed to connect to localhost port 9873: Connection refused
{code}
{code:java}
$ curl 172.27.1.35:9873
http://www.w3.org/TR/html4/strict.dtd;>
Directory listing for /
Directory listing for /
stderr
stdout
{code}
h3.
Marathon Configuration:
{code:java}
{
"id": "/my-app",
"cmd": "python3 -m http.server --bind 0.0.0.0 $PORT0",
"cpus": 1,
"mem": 128,
"disk": 0,
"instances": 1,
"acceptedResourceRoles": ["*"],
"container": {
"type": "MESOS",
"docker": {
"forcePullImage": false,
"image": "python:alpine3.7",
"parameters": [],
"privileged": false
},
"volumes": [],
"portMappings": [
{
"containerPort": 0,
"hostPort": 0,
"labels": {},
"name": "http",
"protocol": "tcp",
"servicePort": 10001
}
]
},
"networks": [
{
"mode": "container/bridge"
}
],
"portDefinitions": []
}{code}
was (Author: dkjs):
[~jieyu] , here's a simple configuration. I also noticed that it only works if
I specify the IP address of the host and not localhost. It sill only works when
i curl from the local host. This seems to suggest an issue with my configured
routes.
h3. Curl Command (from same host):
{code:java}
$ curl localhost:9873
curl: (7) Failed to connect to localhost port 9873: Connection refused
{code}
{code:java}
$ curl 172.27.1.35:9873
http://www.w3.org/TR/html4/strict.dtd;>
Directory listing for /
Directory listing for /
stderr
stdout
{code}
h3.
Marathon Configuration:
{code:java}
{
"id": "/my-app",
"cmd": "python3 -m http.server $PORT0",
"cpus": 1,
"mem": 128,
"disk": 0,
"instances": 1,
"acceptedResourceRoles": ["*"],
"container": {
"type": "MESOS",
"docker": {
"forcePullImage": false,
"image": "python:alpine3.7",
"parameters": [],
"privileged": false
},
"volumes": [],
"portMappings": [
{
"containerPort": 0,
"hostPort": 0,
"labels": {},
"name": "http",
"protocol": "tcp",
"servicePort": 10001
}
]
},
"networks": [
{
"mode": "container/bridge"
}
],
"portDefinitions": []
}{code}
> Mesos UCR with Docker only Works on Localhost
> -
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (MESOS-9269) Mesos UCR with Docker only Works on Localhost
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
z s updated MESOS-9269:
---
Comment: was deleted
(was: It looks like the iptables on my host machines are not correct. I don't
see anything for the Mesos UCR:
{code:java}
$ sudo iptables -nL
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
DOCKER-USER all -- 0.0.0.0/0 0.0.0.0/0
DOCKER-ISOLATION-STAGE-1 all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate
RELATED,ESTABLISHED
DOCKER all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain DOCKER (1 references)
target prot opt source destination
ACCEPT tcp -- 0.0.0.0/0 172.17.0.2 tcp dpt:3888
ACCEPT tcp -- 0.0.0.0/0 172.17.0.2 tcp dpt:2888
ACCEPT tcp -- 0.0.0.0/0 172.17.0.2 tcp dpt:2181
ACCEPT tcp -- 0.0.0.0/0 172.17.0.3 tcp dpt:8081
ACCEPT tcp -- 0.0.0.0/0 172.17.0.3 tcp dpt:8080
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all -- 0.0.0.0/0 0.0.0.0/0
{code}
)
> Mesos UCR with Docker only Works on Localhost
> -
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9269) Mesos UCR with Docker only Works on Localhost
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629587#comment-16629587
]
z s commented on MESOS-9269:
It looks like the iptables on my host machines are not correct. I don't see
anything for the Mesos UCR:
{code:java}
$ sudo iptables -nL
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
DOCKER-USER all -- 0.0.0.0/0 0.0.0.0/0
DOCKER-ISOLATION-STAGE-1 all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate
RELATED,ESTABLISHED
DOCKER all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain DOCKER (1 references)
target prot opt source destination
ACCEPT tcp -- 0.0.0.0/0 172.17.0.2 tcp dpt:3888
ACCEPT tcp -- 0.0.0.0/0 172.17.0.2 tcp dpt:2888
ACCEPT tcp -- 0.0.0.0/0 172.17.0.2 tcp dpt:2181
ACCEPT tcp -- 0.0.0.0/0 172.17.0.3 tcp dpt:8081
ACCEPT tcp -- 0.0.0.0/0 172.17.0.3 tcp dpt:8080
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all -- 0.0.0.0/0 0.0.0.0/0
{code}
> Mesos UCR with Docker only Works on Localhost
> -
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9269) Mesos UCR with Docker only Works on Localhost
[
https://issues.apache.org/jira/browse/MESOS-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629578#comment-16629578
]
z s commented on MESOS-9269:
[~jieyu] , here's a simple configuration. I also noticed that it only works if
I specify the IP address of the host and not localhost. It sill only works when
i curl from the local host. This seems to suggest an issue with my configured
routes.
h3. Curl Command (from same host):
{code:java}
$ curl localhost:9873
curl: (7) Failed to connect to localhost port 9873: Connection refused
{code}
{code:java}
$ curl 172.27.1.35:9873
http://www.w3.org/TR/html4/strict.dtd;>
Directory listing for /
Directory listing for /
stderr
stdout
{code}
h3.
Marathon Configuration:
{code:java}
{
"id": "/my-app",
"cmd": "python3 -m http.server $PORT0",
"cpus": 1,
"mem": 128,
"disk": 0,
"instances": 1,
"acceptedResourceRoles": ["*"],
"container": {
"type": "MESOS",
"docker": {
"forcePullImage": false,
"image": "python:alpine3.7",
"parameters": [],
"privileged": false
},
"volumes": [],
"portMappings": [
{
"containerPort": 0,
"hostPort": 0,
"labels": {},
"name": "http",
"protocol": "tcp",
"servicePort": 10001
}
]
},
"networks": [
{
"mode": "container/bridge"
}
],
"portDefinitions": []
}{code}
> Mesos UCR with Docker only Works on Localhost
> -
>
> Key: MESOS-9269
> URL: https://issues.apache.org/jira/browse/MESOS-9269
> Project: Mesos
> Issue Type: Bug
> Components: agent, docker
>Affects Versions: 1.7.0
> Environment: Ubuntu 16.04
> Mesos 1.7.0
> Marathon 1.7.111
>Reporter: z s
>Priority: Major
>
> I'm having an issue setting up the `mesos-cni-port-mapper` to allow remote
> connectivity.
> When I `curl :` from the machine I get a response but from a
> remote machine the `curl` connection timesout. I'm not sure what's wrong with
> my route settings.
>
> */var/lib/mesos/cni/config/mesos-bridge.json*
>
> {code:java}
> {
> "name" : "mesos-bridge",
> "type" : "mesos-cni-port-mapper",
> "excludeDevices" : ["mesos-cni0"],
> "chain": "MESOS-BRIDGE-PORT-MAPPER",
> "delegate": {
> "type": "bridge",
> "bridge": "mesos-cni0",
> "isGateway": true,
> "ipMasq": true,
> "ipam": {
> "type": "host-local",
> "subnet": "10.1.0.0/16",
> "routes": [
> { "dst":
> "0.0.0.0/0" }
> ]
> }
> }
> }
> {code}
>
> {code:java}
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 0.0.0.0 172.27.1.1 0.0.0.0 UG 0 0 0 ens3
> 10.1.0.0 0.0.0.0 255.255.0.0 U 0 0 0 mesos-cni0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
> 172.27.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
> {code}
> Any suggestions?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
