[jira] [Commented] (MESOS-6417) Introduce an extra 'unknown' health check state.

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446407#comment-16446407
 ] 

Avinash Sridharan commented on MESOS-6417:
--

[~alexr] any plans on introducing this state into Mesos. This is pretty 
important for entities that are pegging against the Mesos state and trying to 
ascertain if task is unhealthy vs health check hasn't been defined for the 
task. DC/OS minuteman being a consumer of this feature.

> Introduce an extra 'unknown' health check state.
> 
>
> Key: MESOS-6417
> URL: https://issues.apache.org/jira/browse/MESOS-6417
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Alexander Rukletsov
>Assignee: Alexander Rukletsov
>Priority: Major
>  Labels: health-check, mesosphere
>
> There are three logical states regarding health checks:
> 1) no health checks;
> 2) a health check is defined, but no result is available yet;
> 3) a health check is defined, it is either healthy or not.
> Currently, we do not distinguish between 1) and 2), which can be problematic 
> for framework authors.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (MESOS-6417) Introduce an extra 'unknown' health check state.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan reassigned MESOS-6417:


Assignee: Alexander Rukletsov

> Introduce an extra 'unknown' health check state.
> 
>
> Key: MESOS-6417
> URL: https://issues.apache.org/jira/browse/MESOS-6417
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Alexander Rukletsov
>Assignee: Alexander Rukletsov
>Priority: Major
>  Labels: health-check, mesosphere
>
> There are three logical states regarding health checks:
> 1) no health checks;
> 2) a health check is defined, but no result is available yet;
> 3) a health check is defined, it is either healthy or not.
> Currently, we do not distinguish between 1) and 2), which can be problematic 
> for framework authors.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-8367) The Mesos configuration guide documents the `--ip6` and `--ip6_discovery_command` flags incorrectly

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16443256#comment-16443256
 ] 

Avinash Sridharan commented on MESOS-8367:
--

This is a documentation fix so we don't necessarily need to backport it to 
1.5.1 since we don't have versioned documentation.

> The Mesos configuration guide documents the `--ip6` and 
> `--ip6_discovery_command` flags incorrectly
> ---
>
> Key: MESOS-8367
> URL: https://issues.apache.org/jira/browse/MESOS-8367
> Project: Mesos
>  Issue Type: Bug
>  Components: documentation
>Reporter: Avinash Sridharan
>Assignee: Gilbert Song
>Priority: Major
>
> The Apache Mesos configuration guide lists the `ip6_discovery_command` and 
> the `ip6` flags being common to both the master and the agent:
> http://mesos.apache.org/documentation/latest/configuration/master-and-agent/
> However these flags have been introduced only in the agent and not the master:
> ```
> [vagrant@centos8 bin]$ ./mesos-master.sh --help | grep discovery
>   
>with `--ip_discovery_command`.
>   --ip_discovery_command=VALUE
>Optional IP discovery binary: if set, it is expected to emit
> [vagrant@centos8 bin]$ ./mesos-agent.sh --help | grep discovery
>   --appc_simple_discovery_uri_prefix=VALUE URI prefix 
> to be used for simple discovery of appc images,
>with 
> `--ip_discovery_command`.
>with 
> '--ip6_discovery_command'.
>   --ip6_discovery_command=VALUEOptional 
> IPv6 discovery binary: if set, it is expected to emit
>   --ip_discovery_command=VALUE Optional 
> IP discovery binary: if set, it is expected to emit
> [vagrant@centos8 bin]$ cd ~/dev/
> [vagrant@centos8 dev]$ ls
> ```
> So we need to fix the documentation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (MESOS-8367) The Mesos configuration guide documents the `--ip6` and `--ip6_discovery_command` flags incorrectly

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan reassigned MESOS-8367:


Assignee: Gilbert Song  (was: Avinash Sridharan)

> The Mesos configuration guide documents the `--ip6` and 
> `--ip6_discovery_command` flags incorrectly
> ---
>
> Key: MESOS-8367
> URL: https://issues.apache.org/jira/browse/MESOS-8367
> Project: Mesos
>  Issue Type: Bug
>  Components: documentation
>Reporter: Avinash Sridharan
>Assignee: Gilbert Song
>Priority: Major
>
> The Apache Mesos configuration guide lists the `ip6_discovery_command` and 
> the `ip6` flags being common to both the master and the agent:
> http://mesos.apache.org/documentation/latest/configuration/master-and-agent/
> However these flags have been introduced only in the agent and not the master:
> ```
> [vagrant@centos8 bin]$ ./mesos-master.sh --help | grep discovery
>   
>with `--ip_discovery_command`.
>   --ip_discovery_command=VALUE
>Optional IP discovery binary: if set, it is expected to emit
> [vagrant@centos8 bin]$ ./mesos-agent.sh --help | grep discovery
>   --appc_simple_discovery_uri_prefix=VALUE URI prefix 
> to be used for simple discovery of appc images,
>with 
> `--ip_discovery_command`.
>with 
> '--ip6_discovery_command'.
>   --ip6_discovery_command=VALUEOptional 
> IPv6 discovery binary: if set, it is expected to emit
>   --ip_discovery_command=VALUE Optional 
> IP discovery binary: if set, it is expected to emit
> [vagrant@centos8 bin]$ cd ~/dev/
> [vagrant@centos8 dev]$ ls
> ```
> So we need to fix the documentation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-8367) The Mesos configuration guide documents the `--ip6` and `--ip6_discovery_command` flags incorrectly

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-8367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16443252#comment-16443252
 ] 

Avinash Sridharan commented on MESOS-8367:
--

[~gilbert] you seem to have already fixed this :

commit 62d392704c499e06da0323e50dfd016cdac06f33
Author: Gilbert Song 
Date: Mon Jan 22 14:09:36 2018 -0800

Fixed --ip6 and --ip6_discovery_command document location.

Review: [https://reviews.apache.org/r/65274]

 

:)

> The Mesos configuration guide documents the `--ip6` and 
> `--ip6_discovery_command` flags incorrectly
> ---
>
> Key: MESOS-8367
> URL: https://issues.apache.org/jira/browse/MESOS-8367
> Project: Mesos
>  Issue Type: Bug
>  Components: documentation
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
>Priority: Major
>
> The Apache Mesos configuration guide lists the `ip6_discovery_command` and 
> the `ip6` flags being common to both the master and the agent:
> http://mesos.apache.org/documentation/latest/configuration/master-and-agent/
> However these flags have been introduced only in the agent and not the master:
> ```
> [vagrant@centos8 bin]$ ./mesos-master.sh --help | grep discovery
>   
>with `--ip_discovery_command`.
>   --ip_discovery_command=VALUE
>Optional IP discovery binary: if set, it is expected to emit
> [vagrant@centos8 bin]$ ./mesos-agent.sh --help | grep discovery
>   --appc_simple_discovery_uri_prefix=VALUE URI prefix 
> to be used for simple discovery of appc images,
>with 
> `--ip_discovery_command`.
>with 
> '--ip6_discovery_command'.
>   --ip6_discovery_command=VALUEOptional 
> IPv6 discovery binary: if set, it is expected to emit
>   --ip_discovery_command=VALUE Optional 
> IP discovery binary: if set, it is expected to emit
> [vagrant@centos8 bin]$ cd ~/dev/
> [vagrant@centos8 dev]$ ls
> ```
> So we need to fix the documentation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (MESOS-8424) Test that operations are correctly reported following a master failover

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-8424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-8424:
-
Fix Version/s: 1.6.0

> Test that operations are correctly reported following a master failover
> ---
>
> Key: MESOS-8424
> URL: https://issues.apache.org/jira/browse/MESOS-8424
> Project: Mesos
>  Issue Type: Task
>  Components: master
>Reporter: Jan Schlicht
>Assignee: Jan Schlicht
>Priority: Major
>  Labels: mesosphere
> Fix For: 1.6.0
>
>
> As the master keeps track of operations running on a resource provider, it 
> needs to be updated on these operations when agents reregister after a master 
> failover. E.g., an operation that has finished during the failover should be 
> reported as finished by the master after the agent on which the resource 
> provider is running has reregistered.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-8424) Test that operations are correctly reported following a master failover

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-8424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348666#comment-16348666
 ] 

Avinash Sridharan commented on MESOS-8424:
--

[~nfnt] closing this out since this seems to have been completed.

> Test that operations are correctly reported following a master failover
> ---
>
> Key: MESOS-8424
> URL: https://issues.apache.org/jira/browse/MESOS-8424
> Project: Mesos
>  Issue Type: Task
>  Components: master
>Reporter: Jan Schlicht
>Assignee: Jan Schlicht
>Priority: Major
>  Labels: mesosphere
>
> As the master keeps track of operations running on a resource provider, it 
> needs to be updated on these operations when agents reregister after a master 
> failover. E.g., an operation that has finished during the failover should be 
> reported as finished by the master after the agent on which the resource 
> provider is running has reregistered.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (MESOS-8424) Test that operations are correctly reported following a master failover

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-8424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-8424:
-
Shepherd: Benjamin Bannier

> Test that operations are correctly reported following a master failover
> ---
>
> Key: MESOS-8424
> URL: https://issues.apache.org/jira/browse/MESOS-8424
> Project: Mesos
>  Issue Type: Task
>  Components: master
>Reporter: Jan Schlicht
>Assignee: Jan Schlicht
>Priority: Major
>  Labels: mesosphere
>
> As the master keeps track of operations running on a resource provider, it 
> needs to be updated on these operations when agents reregister after a master 
> failover. E.g., an operation that has finished during the failover should be 
> reported as finished by the master after the agent on which the resource 
> provider is running has reregistered.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (MESOS-8367) The Mesos configuration guide documents the `--ip6` and `--ip6_discovery_command` flags incorrectly

[Avinash Sridharan (JIRA)]

Avinash Sridharan created MESOS-8367:


 Summary: The Mesos configuration guide documents the `--ip6` and 
`--ip6_discovery_command` flags incorrectly
 Key: MESOS-8367
 URL: https://issues.apache.org/jira/browse/MESOS-8367
 Project: Mesos
  Issue Type: Bug
  Components: documentation
Reporter: Avinash Sridharan
Assignee: Avinash Sridharan


The Apache Mesos configuration guide lists the `ip6_discovery_command` and the 
`ip6` flags being common to both the master and the agent:
http://mesos.apache.org/documentation/latest/configuration/master-and-agent/

However these flags have been introduced only in the agent and not the master:
```
[vagrant@centos8 bin]$ ./mesos-master.sh --help | grep discovery

 with `--ip_discovery_command`.
  --ip_discovery_command=VALUE  
 Optional IP discovery binary: if set, it is expected to emit
[vagrant@centos8 bin]$ ./mesos-agent.sh --help | grep discovery
  --appc_simple_discovery_uri_prefix=VALUE URI prefix 
to be used for simple discovery of appc images,
   with 
`--ip_discovery_command`.
   with 
'--ip6_discovery_command'.
  --ip6_discovery_command=VALUEOptional 
IPv6 discovery binary: if set, it is expected to emit
  --ip_discovery_command=VALUE Optional IP 
discovery binary: if set, it is expected to emit
[vagrant@centos8 bin]$ cd ~/dev/
[vagrant@centos8 dev]$ ls
```

So we need to fix the documentation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-8050) Mesos HTTP/HTTPS health checks for IPv6 docker containers.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-8050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-8050:
-
  Sprint: Mesosphere Sprint 67
Story Points: 5

> Mesos HTTP/HTTPS health checks for IPv6 docker containers.
> --
>
> Key: MESOS-8050
> URL: https://issues.apache.org/jira/browse/MESOS-8050
> Project: Mesos
>  Issue Type: Task
>Reporter: Avinash Sridharan
>Assignee: Qian Zhang
>
> Currently the MESOS HTTP/HTTPS health checks hardcode the IP address to 
> 127.0.0.1 while performing the pings on the containers. With IPv6 containers, 
> even dual stack kernels the container will have both the IPv4 and IPv6 
> loopback interfaces (127.0.0.1 and ::1). Further, its upto the application's 
> discretion to either open a INET or an INET6 socket which would imply that to 
> support IPv6 containers the MESOS HTTP/HTTPS health checks need to be 
> configurable to perform health checks on 127.0.0.1 or ::1. 
> A proposal here would be to introduce the concept of a transport on which 
> MESOS HTTP/HTTPS health checks work. That is the framework specifies whether 
> MESOS HTTP healthchecks work over TCP or TCP6. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-8050) Mesos HTTP/HTTPS health checks for IPv6 docker containers.

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-8050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226011#comment-16226011
 ] 

Avinash Sridharan commented on MESOS-8050:
--

Protobuf changes for `HTTP` and `TCP` health checks:
https://reviews.apache.org/r/63434/

> Mesos HTTP/HTTPS health checks for IPv6 docker containers.
> --
>
> Key: MESOS-8050
> URL: https://issues.apache.org/jira/browse/MESOS-8050
> Project: Mesos
>  Issue Type: Task
>Reporter: Avinash Sridharan
>Assignee: Qian Zhang
>
> Currently the MESOS HTTP/HTTPS health checks hardcode the IP address to 
> 127.0.0.1 while performing the pings on the containers. With IPv6 containers, 
> even dual stack kernels the container will have both the IPv4 and IPv6 
> loopback interfaces (127.0.0.1 and ::1). Further, its upto the application's 
> discretion to either open a INET or an INET6 socket which would imply that to 
> support IPv6 containers the MESOS HTTP/HTTPS health checks need to be 
> configurable to perform health checks on 127.0.0.1 or ::1. 
> A proposal here would be to introduce the concept of a transport on which 
> MESOS HTTP/HTTPS health checks work. That is the framework specifies whether 
> MESOS HTTP healthchecks work over TCP or TCP6. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-8050) Mesos HTTP/HTTPS health checks for IPv6 docker containers.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-8050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-8050:
-
Shepherd: Alexander Rukletsov  (was: Qian Zhang)

> Mesos HTTP/HTTPS health checks for IPv6 docker containers.
> --
>
> Key: MESOS-8050
> URL: https://issues.apache.org/jira/browse/MESOS-8050
> Project: Mesos
>  Issue Type: Task
>Reporter: Avinash Sridharan
>Assignee: Qian Zhang
>
> Currently the MESOS HTTP/HTTPS health checks hardcode the IP address to 
> 127.0.0.1 while performing the pings on the containers. With IPv6 containers, 
> even dual stack kernels the container will have both the IPv4 and IPv6 
> loopback interfaces (127.0.0.1 and ::1). Further, its upto the application's 
> discretion to either open a INET or an INET6 socket which would imply that to 
> support IPv6 containers the MESOS HTTP/HTTPS health checks need to be 
> configurable to perform health checks on 127.0.0.1 or ::1. 
> A proposal here would be to introduce the concept of a transport on which 
> MESOS HTTP/HTTPS health checks work. That is the framework specifies whether 
> MESOS HTTP healthchecks work over TCP or TCP6. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (MESOS-8050) Mesos HTTP/HTTPS health checks for IPv6 docker containers.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-8050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan reassigned MESOS-8050:


Assignee: Qian Zhang  (was: Vinod Kone)

> Mesos HTTP/HTTPS health checks for IPv6 docker containers.
> --
>
> Key: MESOS-8050
> URL: https://issues.apache.org/jira/browse/MESOS-8050
> Project: Mesos
>  Issue Type: Task
>Reporter: Avinash Sridharan
>Assignee: Qian Zhang
>
> Currently the MESOS HTTP/HTTPS health checks hardcode the IP address to 
> 127.0.0.1 while performing the pings on the containers. With IPv6 containers, 
> even dual stack kernels the container will have both the IPv4 and IPv6 
> loopback interfaces (127.0.0.1 and ::1). Further, its upto the application's 
> discretion to either open a INET or an INET6 socket which would imply that to 
> support IPv6 containers the MESOS HTTP/HTTPS health checks need to be 
> configurable to perform health checks on 127.0.0.1 or ::1. 
> A proposal here would be to introduce the concept of a transport on which 
> MESOS HTTP/HTTPS health checks work. That is the framework specifies whether 
> MESOS HTTP healthchecks work over TCP or TCP6. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-8050) Mesos HTTP/HTTPS health checks for IPv6 docker containers.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-8050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-8050:
-
Shepherd: Qian Zhang  (was: Vinod Kone)

> Mesos HTTP/HTTPS health checks for IPv6 docker containers.
> --
>
> Key: MESOS-8050
> URL: https://issues.apache.org/jira/browse/MESOS-8050
> Project: Mesos
>  Issue Type: Task
>Reporter: Avinash Sridharan
>Assignee: Qian Zhang
>
> Currently the MESOS HTTP/HTTPS health checks hardcode the IP address to 
> 127.0.0.1 while performing the pings on the containers. With IPv6 containers, 
> even dual stack kernels the container will have both the IPv4 and IPv6 
> loopback interfaces (127.0.0.1 and ::1). Further, its upto the application's 
> discretion to either open a INET or an INET6 socket which would imply that to 
> support IPv6 containers the MESOS HTTP/HTTPS health checks need to be 
> configurable to perform health checks on 127.0.0.1 or ::1. 
> A proposal here would be to introduce the concept of a transport on which 
> MESOS HTTP/HTTPS health checks work. That is the framework specifies whether 
> MESOS HTTP healthchecks work over TCP or TCP6. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-8121) Unified Containerizer Auto backend should check xfs ftype for overlayfs backend.

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-8121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16213278#comment-16213278
 ] 

Avinash Sridharan commented on MESOS-8121:
--

[~gilbert] [~jieyu] we should return a failure if we see an incompatibility on 
the ftype instead of selecting another backend by default. I am just concerned 
that this is something that needs to be boiled up to the operator instead of 
silently failing and selecting a less performant backend?



> Unified Containerizer Auto backend should check xfs ftype for overlayfs 
> backend.
> 
>
> Key: MESOS-8121
> URL: https://issues.apache.org/jira/browse/MESOS-8121
> Project: Mesos
>  Issue Type: Bug
>  Components: containerization
>Reporter: Gilbert Song
>  Labels: backend, provisioner
>
> when using xfs as the backing filesystem in unified containerizer, the 
> `ftype` has to be equal to 1 if we are using the overlay fs backend. we 
> should add the detection in auto backend logic because some OS (like centos 
> 7.2) has xfs ftype=0 by default.
> https://docs.docker.com/engine/userguide/storagedriver/overlayfs-driver/



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (MESOS-6240) Allow executor/agent communication over non-TCP/IP stream socket.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan reassigned MESOS-6240:


Assignee: Benjamin Hindman

> Allow executor/agent communication over non-TCP/IP stream socket.
> -
>
> Key: MESOS-6240
> URL: https://issues.apache.org/jira/browse/MESOS-6240
> Project: Mesos
>  Issue Type: Improvement
>  Components: containerization
> Environment: Linux and Windows
>Reporter: Avinash Sridharan
>Assignee: Benjamin Hindman
>Priority: Critical
>  Labels: mesosphere
>
> Currently, the executor agent communication happens specifically over TCP 
> sockets. This works fine in most cases, but specifically for the 
> `MesosContainerizer` when containers are running on CNI networks, this mode 
> of communication starts imposing constraints on the CNI network. Since, now 
> there has to connectivity between the CNI network  (on which the executor is 
> running) and the agent. Introducing paths from a CNI network to the 
> underlying agent, at best, creates headaches for operators and at worst 
> introduces serious security holes in the network, since it is breaking the 
> isolation between the container CNI network and the host network (on which 
> the agent is running).
> In order to simplify/strengthen deployment of Mesos containers on CNI 
> networks we therefore need to move away from using TCP/IP sockets for 
> executor/agent communication. Since, executor and agent are guaranteed to run 
> on the same host, the above problems can be resolved if, for the 
> `MesosContainerizer`, we use UNIX domain sockets or named pipes instead of 
> TCP/IP sockets for the executor/agent communication.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-6240) Allow executor/agent communication over non-TCP/IP stream socket.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-6240:
-
Target Version/s: 1.5.0

> Allow executor/agent communication over non-TCP/IP stream socket.
> -
>
> Key: MESOS-6240
> URL: https://issues.apache.org/jira/browse/MESOS-6240
> Project: Mesos
>  Issue Type: Improvement
>  Components: containerization
> Environment: Linux and Windows
>Reporter: Avinash Sridharan
>Assignee: Benjamin Hindman
>Priority: Critical
>  Labels: mesosphere
>
> Currently, the executor agent communication happens specifically over TCP 
> sockets. This works fine in most cases, but specifically for the 
> `MesosContainerizer` when containers are running on CNI networks, this mode 
> of communication starts imposing constraints on the CNI network. Since, now 
> there has to connectivity between the CNI network  (on which the executor is 
> running) and the agent. Introducing paths from a CNI network to the 
> underlying agent, at best, creates headaches for operators and at worst 
> introduces serious security holes in the network, since it is breaking the 
> isolation between the container CNI network and the host network (on which 
> the agent is running).
> In order to simplify/strengthen deployment of Mesos containers on CNI 
> networks we therefore need to move away from using TCP/IP sockets for 
> executor/agent communication. Since, executor and agent are guaranteed to run 
> on the same host, the above problems can be resolved if, for the 
> `MesosContainerizer`, we use UNIX domain sockets or named pipes instead of 
> TCP/IP sockets for the executor/agent communication.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-8050) Mesos HTTP/HTTPS health checks for IPv6 docker containers.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-8050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-8050:
-
Target Version/s: 1.5.0

> Mesos HTTP/HTTPS health checks for IPv6 docker containers.
> --
>
> Key: MESOS-8050
> URL: https://issues.apache.org/jira/browse/MESOS-8050
> Project: Mesos
>  Issue Type: Task
>Reporter: Avinash Sridharan
>Assignee: Vinod Kone
>
> Currently the MESOS HTTP/HTTPS health checks hardcode the IP address to 
> 127.0.0.1 while performing the pings on the containers. With IPv6 containers, 
> even dual stack kernels the container will have both the IPv4 and IPv6 
> loopback interfaces (127.0.0.1 and ::1). Further, its upto the application's 
> discretion to either open a INET or an INET6 socket which would imply that to 
> support IPv6 containers the MESOS HTTP/HTTPS health checks need to be 
> configurable to perform health checks on 127.0.0.1 or ::1. 
> A proposal here would be to introduce the concept of a transport on which 
> MESOS HTTP/HTTPS health checks work. That is the framework specifies whether 
> MESOS HTTP healthchecks work over TCP or TCP6. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (MESOS-8050) Mesos HTTP/HTTPS health checks for IPv6 docker containers.

[Avinash Sridharan (JIRA)]

Avinash Sridharan created MESOS-8050:


 Summary: Mesos HTTP/HTTPS health checks for IPv6 docker containers.
 Key: MESOS-8050
 URL: https://issues.apache.org/jira/browse/MESOS-8050
 Project: Mesos
  Issue Type: Task
Reporter: Avinash Sridharan
Assignee: Vinod Kone


Currently the MESOS HTTP/HTTPS health checks hardcode the IP address to 
127.0.0.1 while performing the pings on the containers. With IPv6 containers, 
even dual stack kernels the container will have both the IPv4 and IPv6 loopback 
interfaces (127.0.0.1 and ::1). Further, its upto the application's discretion 
to either open a INET or an INET6 socket which would imply that to support IPv6 
containers the MESOS HTTP/HTTPS health checks need to be configurable to 
perform health checks on 127.0.0.1 or ::1. 

A proposal here would be to introduce the concept of a transport on which MESOS 
HTTP/HTTPS health checks work. That is the framework specifies whether MESOS 
HTTP healthchecks work over TCP or TCP6. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7923) Make args optional in mesos port mapper plugin

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149245#comment-16149245
 ] 

Avinash Sridharan commented on MESOS-7923:
--

[~qianzhang] could you kindly shepherd this ticket?

> Make args optional in mesos port mapper plugin
> --
>
> Key: MESOS-7923
> URL: https://issues.apache.org/jira/browse/MESOS-7923
> Project: Mesos
>  Issue Type: Bug
>  Components: network
>Reporter: Deepak Goel
>Assignee: Deepak Goel
>
> Current implementation of the mesos-port-mapper plugin fails if the args 
> field is absent in the cni config which makes it very specific to mesos. 
> Instead, if args could be optional then this plugin could be used in a more 
> generic environment. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7923) Make args optional in mesos port mapper plugin

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7923:
-
Shepherd: Qian Zhang  (was: Avinash Sridharan)

> Make args optional in mesos port mapper plugin
> --
>
> Key: MESOS-7923
> URL: https://issues.apache.org/jira/browse/MESOS-7923
> Project: Mesos
>  Issue Type: Bug
>  Components: network
>Reporter: Deepak Goel
>Assignee: Deepak Goel
>
> Current implementation of the mesos-port-mapper plugin fails if the args 
> field is absent in the cni config which makes it very specific to mesos. 
> Instead, if args could be optional then this plugin could be used in a more 
> generic environment. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7807) Docker executor needs to return multiple IP addresses for the container

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7807:
-
Fix Version/s: 1.5.0

> Docker executor needs to return multiple IP addresses for the container
> ---
>
> Key: MESOS-7807
> URL: https://issues.apache.org/jira/browse/MESOS-7807
> Project: Mesos
>  Issue Type: Task
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
>  Labels: Mesosphere
> Fix For: 1.5.0
>
>
> `Docker executor` currently returns only a single IP address for each docker 
> container. In a world where container has a v4 and v6 address the executor 
> needs to return all the addresses it sees for the container else we won't be 
> able to support dual-stack containers.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7806) Add copy assignment operator to `net::IP::Network`

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112934#comment-16112934
 ] 

Avinash Sridharan commented on MESOS-7806:
--

commit cd3380c4e9521b4b26f9030658816eee7a4b89a1
Author: Avinash sridharan 
Date:   Mon Jul 24 18:24:51 2017 -0700

Added a test to check for copy assignment of `net::IP::Network`.

Review: https://reviews.apache.org/r/61005/


> Add copy assignment operator to `net::IP::Network`
> --
>
> Key: MESOS-7806
> URL: https://issues.apache.org/jira/browse/MESOS-7806
> Project: Mesos
>  Issue Type: Task
>  Components: stout
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
> Fix For: 1.4.0
>
>
> Currently, we can't extend the class `net::IP::Network` with out adding a 
> copy assignment operator in the derived class, due to the use of 
> `std::unique_ptr` in the base class. Hence, need to introduce a copy 
> assignment operator into the base class.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7176) Add versioning support to network/cni isolator

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7176:
-
Shepherd: Qian Zhang  (was: Jie Yu)

> Add versioning support to network/cni isolator
> --
>
> Key: MESOS-7176
> URL: https://issues.apache.org/jira/browse/MESOS-7176
> Project: Mesos
>  Issue Type: Task
>  Components: containerization
>Reporter: Avinash Sridharan
>
> Currently the network/cni isolator support CNI SPEC version 0.2 . The CNI 
> SPEC version 0.3 has already been ratified and introduces new features such 
> as CNI service chaining and CNI plugin capabilities. However, CNI spec 
> version 0.3 is incompatible with CNI spec 0.2. Hence we need to introduce 
> versioning support in `network/cni` isolator in order to make it backward 
> compatible.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (MESOS-7176) Add versioning support to network/cni isolator

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan reassigned MESOS-7176:


Assignee: Avinash Sridharan

> Add versioning support to network/cni isolator
> --
>
> Key: MESOS-7176
> URL: https://issues.apache.org/jira/browse/MESOS-7176
> Project: Mesos
>  Issue Type: Task
>  Components: containerization
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
>
> Currently the network/cni isolator support CNI SPEC version 0.2 . The CNI 
> SPEC version 0.3 has already been ratified and introduces new features such 
> as CNI service chaining and CNI plugin capabilities. However, CNI spec 
> version 0.3 is incompatible with CNI spec 0.2. Hence we need to introduce 
> versioning support in `network/cni` isolator in order to make it backward 
> compatible.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-3389) Assign UID to Master/Slave modules

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099231#comment-16099231
 ] 

Avinash Sridharan commented on MESOS-3389:
--

[~qianzhang] closing this out, since this seems to have been required because 
of an `ip-per-container` solution which is already solved by the `network/cni` 
isolator. Please re-open if this is still something we plan to do.

> Assign UID to Master/Slave modules
> --
>
> Key: MESOS-3389
> URL: https://issues.apache.org/jira/browse/MESOS-3389
> Project: Mesos
>  Issue Type: Bug
>Reporter: Kapil Arya
>Assignee: Qian Zhang
>
> This is to allow an Agent identify a certain module. The UID can be used to 
> find the ckpt directory during ckpt/restart. For example, one can imagine 
> putting module-specific state in `/tmp/mesos/...//modules/ UID>`.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-3389) Assign UID to Master/Slave modules

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099226#comment-16099226
 ] 

Avinash Sridharan commented on MESOS-3389:
--

[~qianzhang] are you still ddoing this? Is there still a use-case for this 
ticket?

> Assign UID to Master/Slave modules
> --
>
> Key: MESOS-3389
> URL: https://issues.apache.org/jira/browse/MESOS-3389
> Project: Mesos
>  Issue Type: Bug
>Reporter: Kapil Arya
>Assignee: Qian Zhang
>
> This is to allow an Agent identify a certain module. The UID can be used to 
> find the ckpt directory during ckpt/restart. For example, one can imagine 
> putting module-specific state in `/tmp/mesos/...//modules/ UID>`.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-3018) A mechanism for messages between Master modules and Slave modules

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099202#comment-16099202
 ] 

Avinash Sridharan commented on MESOS-3018:
--

[~karya] do we still need this? Don't think we are doing this anytime soon. Can 
we go ahead and close this out as "Won't fix"?

> A mechanism for messages between Master modules and Slave modules
> -
>
> Key: MESOS-3018
> URL: https://issues.apache.org/jira/browse/MESOS-3018
> Project: Mesos
>  Issue Type: Task
>Reporter: Kapil Arya
>Assignee: Kapil Arya
>  Labels: mesosphere
>
> A slave module should be able to send a message to a master module and 
> vice-versa to allow out-of-band communication between master/slave modules.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (MESOS-7807) Docker executor needs to return multiple IP addresses for the container

[Avinash Sridharan (JIRA)]

Avinash Sridharan created MESOS-7807:


 Summary: Docker executor needs to return multiple IP addresses for 
the container
 Key: MESOS-7807
 URL: https://issues.apache.org/jira/browse/MESOS-7807
 Project: Mesos
  Issue Type: Task
Reporter: Avinash Sridharan
Assignee: Avinash Sridharan


`Docker executor` currently returns only a single IP address for each docker 
container. In a world where container has a v4 and v6 address the executor 
needs to return all the addresses it sees for the container else we won't be 
able to support dual-stack containers.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (MESOS-7806) Add copy assignment operator to `net::IP::Network`

[Avinash Sridharan (JIRA)]

Avinash Sridharan created MESOS-7806:


 Summary: Add copy assignment operator to `net::IP::Network`
 Key: MESOS-7806
 URL: https://issues.apache.org/jira/browse/MESOS-7806
 Project: Mesos
  Issue Type: Task
  Components: stout
Reporter: Avinash Sridharan
Assignee: Avinash Sridharan


Currently, we can't extend the class `net::IP::Network` with out adding a copy 
assignment operator in the derived class, due to the use of `std::unique_ptr` 
in the base class. Hence, need to introduce a copy assignment operator into the 
base class.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7769) libprocess initializes to bind to random port if --ip is not specified

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093493#comment-16093493
 ] 

Avinash Sridharan commented on MESOS-7769:
--

commit 78671375662f286128cd80d3b1f6586d0ec35cf8
Author: Avinash sridharan avin...@mesosphere.io
Date:   Fri Jul 7 16:34:23 2017 -0700

Fixed initialization of __address__ in the abscense of --ip flag.

When we introduced the __address6__ optional IPv6 storage into
libprocess we also introduced a regression because of which the port
doesn't get initialized to whatever is specified in the --port flag
until the --ip flag is specified.

This change fixes the initialization of the __address__ in the
absence of the --ip flag.

Review: https://reviews.apache.org/r/60720/

> libprocess initializes to bind to random port if --ip is not specified
> --
>
> Key: MESOS-7769
> URL: https://issues.apache.org/jira/browse/MESOS-7769
> Project: Mesos
>  Issue Type: Bug
>  Components: libprocess
>Affects Versions: 1.4.0
>Reporter: Yan Xu
>Assignee: Avinash Sridharan
>Priority: Blocker
>  Labels: mesosphere
> Fix For: 1.4.0
>
>
> When running current 
> [HEAD|https://github.com/apache/mesos/commit/c90bea80486c089e933bef64aca341e4cfaaef25],
> {noformat:title=without --ip}
> ./mesos-master.sh --work_dir=/tmp/mesos-test1
> ...
> I0707 14:14:05.927870  5820 master.cpp:438] Master 
> db2a2d26-a9a9-4e6f-9909-b9eca47a2862 () started on :36839
> {noformat}
> {noformat:title=with --ip}
> ./mesos-master.sh --ip= --work_dir=/tmp/mesos-test1
> I0707 14:09:56.851483  5729 master.cpp:438] Master 
> 963e0f42-9767-4629-8e3d-02c6ab6ad225 () started on :5050
> {noformat}
> It would be great this is caught by tests/CI.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (MESOS-7776) Document `MESOS_CONTAINER_IP`

[Avinash Sridharan (JIRA)]

Avinash Sridharan created MESOS-7776:


 Summary: Document `MESOS_CONTAINER_IP` 
 Key: MESOS-7776
 URL: https://issues.apache.org/jira/browse/MESOS-7776
 Project: Mesos
  Issue Type: Documentation
  Components: containerization
Reporter: Avinash Sridharan
Assignee: Avinash Sridharan


We introduced `MESOS_CONTAINER_IP` to inform tasks launched by the 
default-executor to inform the tasks about their container IP. This was done 
primarily to break the dependency of the containers on `LIBPROCESS_IP` to learn 
their IP addresses which was misleading. 

This change need to be documented.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7772) Copy-n-paste error in slave/main.cpp

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7772:
-
Story Points: 1

> Copy-n-paste error in slave/main.cpp
> 
>
> Key: MESOS-7772
> URL: https://issues.apache.org/jira/browse/MESOS-7772
> Project: Mesos
>  Issue Type: Bug
>  Components: agent
>Affects Versions: 1.4.0
>Reporter: Benjamin Bannier
>Assignee: Avinash Sridharan
>Priority: Blocker
>  Labels: coverity, mesosphere
>
> Coverity diagnosed a copy-n-paste error in {{slave/main.cpp}} 
> (https://scan5.coverity.com/reports.htm#v10074/p10429/fileInstanceId=120155401=33592186=1414687+1+Comment),
> {noformat}
> 323  } else if (flags.ip6.isSome()) {
> CID 1414687 (#1 of 1): Copy-paste error (COPY_PASTE_ERROR)
> copy_paste_error: ip in flags.ip looks like a copy-paste error.
>   Should it say ip6 instead?
> 324os::setenv("LIBPROCESS_IP6", flags.ip.get());
> 325  }
> {noformat}
> We check the incorrect IP for some value here (check on {{ip6}}, but use of 
> {{ip}}), and it seems extremely likely we intended to use {{flags.ip6}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7772) Copy-n-paste error in slave/main.cpp

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7772:
-
Priority: Blocker  (was: Major)

> Copy-n-paste error in slave/main.cpp
> 
>
> Key: MESOS-7772
> URL: https://issues.apache.org/jira/browse/MESOS-7772
> Project: Mesos
>  Issue Type: Bug
>  Components: agent
>Affects Versions: 1.4.0
>Reporter: Benjamin Bannier
>Assignee: Avinash Sridharan
>Priority: Blocker
>  Labels: coverity, mesosphere
>
> Coverity diagnosed a copy-n-paste error in {{slave/main.cpp}} 
> (https://scan5.coverity.com/reports.htm#v10074/p10429/fileInstanceId=120155401=33592186=1414687+1+Comment),
> {noformat}
> 323  } else if (flags.ip6.isSome()) {
> CID 1414687 (#1 of 1): Copy-paste error (COPY_PASTE_ERROR)
> copy_paste_error: ip in flags.ip looks like a copy-paste error.
>   Should it say ip6 instead?
> 324os::setenv("LIBPROCESS_IP6", flags.ip.get());
> 325  }
> {noformat}
> We check the incorrect IP for some value here (check on {{ip6}}, but use of 
> {{ip}}), and it seems extremely likely we intended to use {{flags.ip6}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7772) Copy-n-paste error in slave/main.cpp

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078872#comment-16078872
 ] 

Avinash Sridharan commented on MESOS-7772:
--

https://reviews.apache.org/r/60720/
https://reviews.apache.org/r/60723/

> Copy-n-paste error in slave/main.cpp
> 
>
> Key: MESOS-7772
> URL: https://issues.apache.org/jira/browse/MESOS-7772
> Project: Mesos
>  Issue Type: Bug
>  Components: agent
>Affects Versions: 1.4.0
>Reporter: Benjamin Bannier
>Assignee: Avinash Sridharan
>  Labels: coverity, mesosphere
>
> Coverity diagnosed a copy-n-paste error in {{slave/main.cpp}} 
> (https://scan5.coverity.com/reports.htm#v10074/p10429/fileInstanceId=120155401=33592186=1414687+1+Comment),
> {noformat}
> 323  } else if (flags.ip6.isSome()) {
> CID 1414687 (#1 of 1): Copy-paste error (COPY_PASTE_ERROR)
> copy_paste_error: ip in flags.ip looks like a copy-paste error.
>   Should it say ip6 instead?
> 324os::setenv("LIBPROCESS_IP6", flags.ip.get());
> 325  }
> {noformat}
> We check the incorrect IP for some value here (check on {{ip6}}, but use of 
> {{ip}}), and it seems extremely likely we intended to use {{flags.ip6}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Issue Comment Deleted] (MESOS-7772) Copy-n-paste error in slave/main.cpp

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7772:
-
Comment: was deleted

(was: https://reviews.apache.org/r/60720/
https://reviews.apache.org/r/60723/)

> Copy-n-paste error in slave/main.cpp
> 
>
> Key: MESOS-7772
> URL: https://issues.apache.org/jira/browse/MESOS-7772
> Project: Mesos
>  Issue Type: Bug
>  Components: agent
>Affects Versions: 1.4.0
>Reporter: Benjamin Bannier
>Assignee: Avinash Sridharan
>  Labels: coverity, mesosphere
>
> Coverity diagnosed a copy-n-paste error in {{slave/main.cpp}} 
> (https://scan5.coverity.com/reports.htm#v10074/p10429/fileInstanceId=120155401=33592186=1414687+1+Comment),
> {noformat}
> 323  } else if (flags.ip6.isSome()) {
> CID 1414687 (#1 of 1): Copy-paste error (COPY_PASTE_ERROR)
> copy_paste_error: ip in flags.ip looks like a copy-paste error.
>   Should it say ip6 instead?
> 324os::setenv("LIBPROCESS_IP6", flags.ip.get());
> 325  }
> {noformat}
> We check the incorrect IP for some value here (check on {{ip6}}, but use of 
> {{ip}}), and it seems extremely likely we intended to use {{flags.ip6}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7769) libprocess initializes to bind to random port if --ip is not specified

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7769:
-
  Sprint: Mesosphere Sprint 59
Story Points: 1
  Labels: mesosphere  (was: )

> libprocess initializes to bind to random port if --ip is not specified
> --
>
> Key: MESOS-7769
> URL: https://issues.apache.org/jira/browse/MESOS-7769
> Project: Mesos
>  Issue Type: Bug
>  Components: libprocess
>Affects Versions: 1.4.0
>Reporter: Yan Xu
>Assignee: Avinash Sridharan
>Priority: Blocker
>  Labels: mesosphere
>
> When running current 
> [HEAD|https://github.com/apache/mesos/commit/c90bea80486c089e933bef64aca341e4cfaaef25],
> {noformat:title=without --ip}
> ./mesos-master.sh --work_dir=/tmp/mesos-test1
> ...
> I0707 14:14:05.927870  5820 master.cpp:438] Master 
> db2a2d26-a9a9-4e6f-9909-b9eca47a2862 () started on :36839
> {noformat}
> {noformat:title=with --ip}
> ./mesos-master.sh --ip= --work_dir=/tmp/mesos-test1
> I0707 14:09:56.851483  5729 master.cpp:438] Master 
> 963e0f42-9767-4629-8e3d-02c6ab6ad225 () started on :5050
> {noformat}
> It would be great this is caught by tests/CI.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7772) Copy-n-paste error in slave/main.cpp

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7772:
-
Shepherd: Benjamin Hindman
  Sprint: Mesosphere Sprint 59
Target Version/s: 1.4.0
  Labels: coverity mesosphere  (was: coverity)

> Copy-n-paste error in slave/main.cpp
> 
>
> Key: MESOS-7772
> URL: https://issues.apache.org/jira/browse/MESOS-7772
> Project: Mesos
>  Issue Type: Bug
>  Components: agent
>Affects Versions: 1.4.0
>Reporter: Benjamin Bannier
>Assignee: Avinash Sridharan
>  Labels: coverity, mesosphere
>
> Coverity diagnosed a copy-n-paste error in {{slave/main.cpp}} 
> (https://scan5.coverity.com/reports.htm#v10074/p10429/fileInstanceId=120155401=33592186=1414687+1+Comment),
> {noformat}
> 323  } else if (flags.ip6.isSome()) {
> CID 1414687 (#1 of 1): Copy-paste error (COPY_PASTE_ERROR)
> copy_paste_error: ip in flags.ip looks like a copy-paste error.
>   Should it say ip6 instead?
> 324os::setenv("LIBPROCESS_IP6", flags.ip.get());
> 325  }
> {noformat}
> We check the incorrect IP for some value here (check on {{ip6}}, but use of 
> {{ip}}), and it seems extremely likely we intended to use {{flags.ip6}}.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (MESOS-7769) libprocess initializes to bind to random port if --ip is not specified

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan reassigned MESOS-7769:


Assignee: Avinash Sridharan

> libprocess initializes to bind to random port if --ip is not specified
> --
>
> Key: MESOS-7769
> URL: https://issues.apache.org/jira/browse/MESOS-7769
> Project: Mesos
>  Issue Type: Bug
>  Components: libprocess
>Affects Versions: 1.4.0
>Reporter: Yan Xu
>Assignee: Avinash Sridharan
>Priority: Blocker
>
> When running current 
> [HEAD|https://github.com/apache/mesos/commit/c90bea80486c089e933bef64aca341e4cfaaef25],
> {noformat:title=without --ip}
> ./mesos-master.sh --work_dir=/tmp/mesos-test1
> ...
> I0707 14:14:05.927870  5820 master.cpp:438] Master 
> db2a2d26-a9a9-4e6f-9909-b9eca47a2862 () started on :36839
> {noformat}
> {noformat:title=with --ip}
> ./mesos-master.sh --ip= --work_dir=/tmp/mesos-test1
> I0707 14:09:56.851483  5729 master.cpp:438] Master 
> 963e0f42-9767-4629-8e3d-02c6ab6ad225 () started on :5050
> {noformat}
> It would be great this is caught by tests/CI.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (MESOS-7767) Make `net::IP` fields protected to allow for inheritance

[Avinash Sridharan (JIRA)]

Avinash Sridharan created MESOS-7767:


 Summary: Make `net::IP` fields protected to allow for inheritance
 Key: MESOS-7767
 URL: https://issues.apache.org/jira/browse/MESOS-7767
 Project: Mesos
  Issue Type: Task
Reporter: Avinash Sridharan
Assignee: Avinash Sridharan
Priority: Minor


Correctly the properties of `net::IP` are `private` making it hard for classes 
to inherit `net::IP`. Although `net::IPv4` and `net::IPv6` already inherit 
`net::IP` they don't have access to the storage structures stored within 
`net::IP`. While this works for the current API this is not very extensible. 

Hence, we should make the properties of `net::IP` protected instead of private.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (MESOS-7488) Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077568#comment-16077568
 ] 

Avinash Sridharan edited comment on MESOS-7488 at 7/7/17 4:51 AM:
--

commit 114bd98a04e05665c647c4a9c94e595ef104ec64
Author: Avinash sridharan 
Date:   Thu Jul 6 17:01:37 2017 -0700

Updated v6 address for containers running on host network.

Currently the agent is populating only the v4 address for containers
running on the host network. However, clusters running on a dual stack
network need to report v4 and v6 address for containers running on the
host network. This change allows v6 address specified by operators to be
advertised for containers running on the host network.

Review: https://reviews.apache.org/r/59233/

commit 8d9929a5fe51c866aabac04c764d4e9ceca98d06
Author: Avinash sridharan 
Date:   Thu Jul 6 17:01:23 2017 -0700

Added storage for IPv6 in a `libprocess` process.

Review: https://reviews.apache.org/r/59130/

commit 2c1be9ced126fce008c626df63c110169e5b60fd
Author: Avinash sridharan 
Date:   Wed Jul 5 12:38:07 2017 -0700

Added support for `net::IPv4` and `net::IPv6` in `FlagsBase`.

Review: https://reviews.apache.org/r/60149/

commit 355861cd9209a4dc915fe47a8d29a0aff289611b
Author: Avinash sridharan 
Date:   Wed Jul 5 12:36:33 2017 -0700

Mesos updates for new inet4::Address.

Review: https://reviews.apache.org/r/60148/

commit 58d70f773d83799619db9981292327e7aa70cacd
Author: Avinash sridharan 
Date:   Wed Jul 5 12:31:23 2017 -0700

Refactored inet::Address to base of inet4::Address and inet6::Address.

Review: https://reviews.apache.org/r/60136/

commit 5ea1549273e02975cac7241cac3707b01ecd5bd0
Author: Avinash sridharan 
Date:   Wed Jul 5 12:29:14 2017 -0700

Refactored net::IP and added net::IPv4 and net::IPv6.

Classes can now initialize the specialized classes whenever they need to
create explicit v4 and v6 addresses, rather than relying on the
`create/Try` semantics of net::IP.

Review: https://reviews.apache.org/r/59721/

commit 9b69c09310cdb6d7cfca1284f60c3f1b422c77cc
Author: Avinash sridharan 
Date:   Wed Jul 5 11:38:14 2017 -0700

Replaced use of `net::IPNetwork` by `net::IP::Network` in Mesos.

Review: https://reviews.apache.org/r/59739/


commit ec1f15e3acc07dbcfeadfe39fa0d252cbb46ba4e
Author: Avinash sridharan 
Date:   Wed Jul 5 11:39:31 2017 -0700

Replaced use of `net::IPNetwork` by `net::IP::Network` in stout.

Review: https://reviews.apache.org/r/59739/

commit 36b97ed73dce0b43ef8639d90132c2752669dc25
Author: Avinash sridharan 
Date:   Wed Jul 5 11:34:12 2017 -0700

Moved `net::IPNetwork` to `net::IP:Network`.

Review: https://reviews.apache.org/r/59688/






was (Author: avin...@mesosphere.io):
commit 114bd98a04e05665c647c4a9c94e595ef104ec64
Author: Avinash sridharan 
Date:   Thu Jul 6 17:01:37 2017 -0700

Updated v6 address for containers running on host network.

Currently the agent is populating only the v4 address for containers
running on the host network. However, clusters running on a dual stack
network need to report v4 and v6 address for containers running on the
host network. This change allows v6 address specified by operators to be
advertised for containers running on the host network.

Review: https://reviews.apache.org/r/59233/

commit 8d9929a5fe51c866aabac04c764d4e9ceca98d06
Author: Avinash sridharan 
Date:   Thu Jul 6 17:01:23 2017 -0700

Added storage for IPv6 in a `libprocess` process.

Review: https://reviews.apache.org/r/59130/

commit 952a7422c4f97289ccf633c746d12b54a36aea45
Author: Avinash sridharan 
Date:   Thu Jul 6 16:50:36 2017 -0700

Fixed `net::IPv4` to return `INADDR_ANY` for `ANY()`.

Review: https://reviews.apache.org/r/60692/

commit 2c1be9ced126fce008c626df63c110169e5b60fd
Author: Avinash sridharan 
Date:   Wed Jul 5 12:38:07 2017 -0700

Added support for `net::IPv4` and `net::IPv6` in `FlagsBase`.

Review: https://reviews.apache.org/r/60149/

commit 355861cd9209a4dc915fe47a8d29a0aff289611b
Author: Avinash sridharan 
Date:   Wed Jul 5 12:36:33 2017 -0700

Mesos updates for new inet4::Address.

Review: https://reviews.apache.org/r/60148/

commit 58d70f773d83799619db9981292327e7aa70cacd
Author: Avinash sridharan 
Date:   Wed Jul 5 12:31:23 2017 -0700

Refactored inet::Address to base of inet4::Address and inet6::Address.

Review: https://reviews.apache.org/r/60136/

commit 5ea1549273e02975cac7241cac3707b01ecd5bd0

[jira] [Updated] (MESOS-7488) Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7488:
-
Target Version/s: 1.4.0

> Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent
> -
>
> Key: MESOS-7488
> URL: https://issues.apache.org/jira/browse/MESOS-7488
> Project: Mesos
>  Issue Type: Task
>  Components: agent
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
>  Labels: mesosphere
> Fix For: 1.4.0
>
>
> As a first step to support IPv6 containers on Mesos, we need to provide 
> `--ip6` and `--ip6_discovery_command` flags to the agent so that the operator 
> can  specify an IPv6 address for the `libprocess` actor on the agent. In this 
> ticket we will not aim to add IPv6 communication support for Mesos but will 
> aim to use the IPv6 address provided by the operator to fill in the v6 
> address for any containers running on the host network in a dual stack 
> environment.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7765) MasterTest.KillUnknownTask is failling due to a bug in `net::IPv4::ANY()`

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7765:
-
Fix Version/s: 1.4.0

> MasterTest.KillUnknownTask is failling due to a bug in `net::IPv4::ANY()`
> -
>
> Key: MESOS-7765
> URL: https://issues.apache.org/jira/browse/MESOS-7765
> Project: Mesos
>  Issue Type: Bug
> Environment: Linux
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
>Priority: Blocker
>  Labels: mesosphere
> Fix For: 1.4.0
>
>
> Seeing the following failure when running `MasterTest.KillUnknownTask`:
> ```
> I0706 14:08:20.724071 25596 sched.cpp:1041] Scheduler::statusUpdate took 
> 19411ns
> [libprotobuf FATAL google/protobuf/message_lite.cc:294] CHECK failed: 
> IsInitialized(): Can't serialize message of type "mesos.scheduler.Call" 
> because it is missing required fields: acknowledge.slave_id.value
> libprocess: scheduler-5cca230e-e4c9-466e-b2cd-bde7b7d7ed71@127.0.0.1:44650 
> terminating due to CHECK failed: IsInitialized(): Can't serialize message of 
> type "mesos.scheduler.Call" because it is missing required fields: 
> acknowledge.slave_id.valueI0706 14:08:20.724196 25570 sched.cpp:2021] Asked 
> to stop the driver
> ```
> Looks we introduced a bug when we create the `net::IPv4` class. The `ANY` 
> method of this class returns `INADDR_LOOPBACK` instead of `INADDR_ANY`. This 
> ends up causing weird issues in terms of connectivity. We need to fix 
> `net::IPv4::ANY` to return `INADDR_ANY`.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7488) Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077568#comment-16077568
 ] 

Avinash Sridharan commented on MESOS-7488:
--

commit 114bd98a04e05665c647c4a9c94e595ef104ec64
Author: Avinash sridharan 
Date:   Thu Jul 6 17:01:37 2017 -0700

Updated v6 address for containers running on host network.

Currently the agent is populating only the v4 address for containers
running on the host network. However, clusters running on a dual stack
network need to report v4 and v6 address for containers running on the
host network. This change allows v6 address specified by operators to be
advertised for containers running on the host network.

Review: https://reviews.apache.org/r/59233/

commit 8d9929a5fe51c866aabac04c764d4e9ceca98d06
Author: Avinash sridharan 
Date:   Thu Jul 6 17:01:23 2017 -0700

Added storage for IPv6 in a `libprocess` process.

Review: https://reviews.apache.org/r/59130/

commit 952a7422c4f97289ccf633c746d12b54a36aea45
Author: Avinash sridharan 
Date:   Thu Jul 6 16:50:36 2017 -0700

Fixed `net::IPv4` to return `INADDR_ANY` for `ANY()`.

Review: https://reviews.apache.org/r/60692/

commit 2c1be9ced126fce008c626df63c110169e5b60fd
Author: Avinash sridharan 
Date:   Wed Jul 5 12:38:07 2017 -0700

Added support for `net::IPv4` and `net::IPv6` in `FlagsBase`.

Review: https://reviews.apache.org/r/60149/

commit 355861cd9209a4dc915fe47a8d29a0aff289611b
Author: Avinash sridharan 
Date:   Wed Jul 5 12:36:33 2017 -0700

Mesos updates for new inet4::Address.

Review: https://reviews.apache.org/r/60148/

commit 58d70f773d83799619db9981292327e7aa70cacd
Author: Avinash sridharan 
Date:   Wed Jul 5 12:31:23 2017 -0700

Refactored inet::Address to base of inet4::Address and inet6::Address.

Review: https://reviews.apache.org/r/60136/

commit 5ea1549273e02975cac7241cac3707b01ecd5bd0
Author: Avinash sridharan 
Date:   Wed Jul 5 12:29:14 2017 -0700

Refactored net::IP and added net::IPv4 and net::IPv6.

Classes can now initialize the specialized classes whenever they need to
create explicit v4 and v6 addresses, rather than relying on the
`create/Try` semantics of net::IP.

Review: https://reviews.apache.org/r/59721/

commit 9b69c09310cdb6d7cfca1284f60c3f1b422c77cc
Author: Avinash sridharan 
Date:   Wed Jul 5 11:38:14 2017 -0700

Replaced use of `net::IPNetwork` by `net::IP::Network` in Mesos.

Review: https://reviews.apache.org/r/59739/


commit ec1f15e3acc07dbcfeadfe39fa0d252cbb46ba4e
Author: Avinash sridharan 
Date:   Wed Jul 5 11:39:31 2017 -0700

Replaced use of `net::IPNetwork` by `net::IP::Network` in stout.

Review: https://reviews.apache.org/r/59739/

commit 36b97ed73dce0b43ef8639d90132c2752669dc25
Author: Avinash sridharan 
Date:   Wed Jul 5 11:34:12 2017 -0700

Moved `net::IPNetwork` to `net::IP:Network`.

Review: https://reviews.apache.org/r/59688/





> Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent
> -
>
> Key: MESOS-7488
> URL: https://issues.apache.org/jira/browse/MESOS-7488
> Project: Mesos
>  Issue Type: Task
>  Components: agent
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
>  Labels: mesosphere
> Fix For: 1.4.0
>
>
> As a first step to support IPv6 containers on Mesos, we need to provide 
> `--ip6` and `--ip6_discovery_command` flags to the agent so that the operator 
> can  specify an IPv6 address for the `libprocess` actor on the agent. In this 
> ticket we will not aim to add IPv6 communication support for Mesos but will 
> aim to use the IPv6 address provided by the operator to fill in the v6 
> address for any containers running on the host network in a dual stack 
> environment.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7488) Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077565#comment-16077565
 ] 

Avinash Sridharan commented on MESOS-7488:
--

commit 9b9114741a192fdbc7b2ab2729b1a00aa200e9fc
Author: Avinash sridharan 
Date:   Thu Jul 6 18:05:54 2017 -0700

Added IPv6 flags for Mesos agent.

Review: https://reviews.apache.org/r/59127/

> Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent
> -
>
> Key: MESOS-7488
> URL: https://issues.apache.org/jira/browse/MESOS-7488
> Project: Mesos
>  Issue Type: Task
>  Components: agent
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
>  Labels: mesosphere
> Fix For: 1.4.0
>
>
> As a first step to support IPv6 containers on Mesos, we need to provide 
> `--ip6` and `--ip6_discovery_command` flags to the agent so that the operator 
> can  specify an IPv6 address for the `libprocess` actor on the agent. In this 
> ticket we will not aim to add IPv6 communication support for Mesos but will 
> aim to use the IPv6 address provided by the operator to fill in the v6 
> address for any containers running on the host network in a dual stack 
> environment.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (MESOS-7765) MasterTest.KillUnknownTask is failling due to a bug in `net::IPv4::ANY()`

[Avinash Sridharan (JIRA)]

Avinash Sridharan created MESOS-7765:


 Summary: MasterTest.KillUnknownTask is failling due to a bug in 
`net::IPv4::ANY()`
 Key: MESOS-7765
 URL: https://issues.apache.org/jira/browse/MESOS-7765
 Project: Mesos
  Issue Type: Bug
 Environment: Linux
Reporter: Avinash Sridharan
Assignee: Avinash Sridharan
Priority: Blocker


Seeing the following failure when running `MasterTest.KillUnknownTask`:
```
I0706 14:08:20.724071 25596 sched.cpp:1041] Scheduler::statusUpdate took 19411ns
[libprotobuf FATAL google/protobuf/message_lite.cc:294] CHECK failed: 
IsInitialized(): Can't serialize message of type "mesos.scheduler.Call" because 
it is missing required fields: acknowledge.slave_id.value
libprocess: scheduler-5cca230e-e4c9-466e-b2cd-bde7b7d7ed71@127.0.0.1:44650 
terminating due to CHECK failed: IsInitialized(): Can't serialize message of 
type "mesos.scheduler.Call" because it is missing required fields: 
acknowledge.slave_id.valueI0706 14:08:20.724196 25570 sched.cpp:2021] Asked to 
stop the driver
```

Looks we introduced a bug when we create the `net::IPv4` class. The `ANY` 
method of this class returns `INADDR_LOOPBACK` instead of `INADDR_ANY`. This 
ends up causing weird issues in terms of connectivity. We need to fix 
`net::IPv4::ANY` to return `INADDR_ANY`.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-1027) IPv6 support

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-1027:
-
Labels: mesosphere  (was: )

> IPv6 support
> 
>
> Key: MESOS-1027
> URL: https://issues.apache.org/jira/browse/MESOS-1027
> Project: Mesos
>  Issue Type: Epic
>  Components: agent, framework, libprocess, master
>Reporter: Dominic Hamon
>Assignee: Avinash Sridharan
>  Labels: mesosphere
>
> From the CLI down through the various layers of tech we should support IPv6.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (MESOS-1027) IPv6 support

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan reassigned MESOS-1027:


Assignee: Avinash Sridharan

> IPv6 support
> 
>
> Key: MESOS-1027
> URL: https://issues.apache.org/jira/browse/MESOS-1027
> Project: Mesos
>  Issue Type: Epic
>  Components: agent, framework, libprocess, master
>Reporter: Dominic Hamon
>Assignee: Avinash Sridharan
>
> From the CLI down through the various layers of tech we should support IPv6.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7709) Add --dns flag to the agent.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7709:
-
Sprint: Mesosphere Sprint 58
Labels: mesosphere  (was: )

> Add --dns flag to the agent.
> 
>
> Key: MESOS-7709
> URL: https://issues.apache.org/jira/browse/MESOS-7709
> Project: Mesos
>  Issue Type: Task
>  Components: containerization
>Reporter: Avinash Sridharan
>Assignee: Qian Zhang
>  Labels: mesosphere
>
> Mesos support both CNI (through `network/cni` isolator) and CNM (through 
> docker) specification. Both these specifications allow for DNS entries for 
> containers to be set on a per-container, and per-network basis. 
> Currently, the behavior of the agent is to use the DNS nameservers set in 
> /etc/resolv.conf when the CNI or CNM plugin that is used to attached the 
> container to the CNI/CNM network doesnt' explicitly set the DNS for the 
> container. This is a bit inflexible especially when we have a mix of v4 and 
> v6 networks. 
> The operator should be able to specify DNS nameservers for the networks he 
> installs either the override the ones provided by the plugin or as defaults 
> when the plugins are not going to specify DNS name servers.
> In order to achieve the above goal we need to introduce a `\--dns` flag to 
> the agent. The `\--dns` flag should support a JSON (or a JSON file) with the 
> following schema:
> {code}
> {
>   "mesos": [
> {
>   "network" : ,
>   "nameservers": []
> }
>   ],
>   "docker": [
> {
>   "network" : ,
>   "nameservers": []
> }
>   ]
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (MESOS-7709) Add --dns flag to the agent.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan reassigned MESOS-7709:


Assignee: Qian Zhang  (was: Avinash Sridharan)

> Add --dns flag to the agent.
> 
>
> Key: MESOS-7709
> URL: https://issues.apache.org/jira/browse/MESOS-7709
> Project: Mesos
>  Issue Type: Task
>  Components: containerization
>Reporter: Avinash Sridharan
>Assignee: Qian Zhang
>
> Mesos support both CNI (through `network/cni` isolator) and CNM (through 
> docker) specification. Both these specifications allow for DNS entries for 
> containers to be set on a per-container, and per-network basis. 
> Currently, the behavior of the agent is to use the DNS nameservers set in 
> /etc/resolv.conf when the CNI or CNM plugin that is used to attached the 
> container to the CNI/CNM network doesnt' explicitly set the DNS for the 
> container. This is a bit inflexible especially when we have a mix of v4 and 
> v6 networks. 
> The operator should be able to specify DNS nameservers for the networks he 
> installs either the override the ones provided by the plugin or as defaults 
> when the plugins are not going to specify DNS name servers.
> In order to achieve the above goal we need to introduce a `\--dns` flag to 
> the agent. The `\--dns` flag should support a JSON (or a JSON file) with the 
> following schema:
> {code}
> {
>   "mesos": [
> {
>   "network" : ,
>   "nameservers": []
> }
>   ],
>   "docker": [
> {
>   "network" : ,
>   "nameservers": []
> }
>   ]
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7709) Add --dns flag to the agent.

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064291#comment-16064291
 ] 

Avinash Sridharan commented on MESOS-7709:
--

@qian zhang yeah that's what I was referring to. We should take the options 
provided with `--dns` only if `ContainerInfo.docker.parameters` doesn't have a 
`--dns` option set.

> Add --dns flag to the agent.
> 
>
> Key: MESOS-7709
> URL: https://issues.apache.org/jira/browse/MESOS-7709
> Project: Mesos
>  Issue Type: Task
>  Components: containerization
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
>
> Mesos support both CNI (through `network/cni` isolator) and CNM (through 
> docker) specification. Both these specifications allow for DNS entries for 
> containers to be set on a per-container, and per-network basis. 
> Currently, the behavior of the agent is to use the DNS nameservers set in 
> /etc/resolv.conf when the CNI or CNM plugin that is used to attached the 
> container to the CNI/CNM network doesnt' explicitly set the DNS for the 
> container. This is a bit inflexible especially when we have a mix of v4 and 
> v6 networks. 
> The operator should be able to specify DNS nameservers for the networks he 
> installs either the override the ones provided by the plugin or as defaults 
> when the plugins are not going to specify DNS name servers.
> In order to achieve the above goal we need to introduce a `\--dns` flag to 
> the agent. The `\--dns` flag should support a JSON (or a JSON file) with the 
> following schema:
> {code}
> {
>   "mesos": {
> [ 
>   {
> "network" : ,
> "nameservers": []
>   }
> ]
>   },
>   "docker": {
> [ 
>   {
> "network" : ,
> "nameservers": []
>   }
> ]
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7709) Add --dns flag to the agent.

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064080#comment-16064080
 ] 

Avinash Sridharan commented on MESOS-7709:
--

[~qianzhang] I think the proposal makes sense. The only point I would like to 
add is for the CNM case, if no DNS is provided in the `docker run` options we 
should take the `--dns` option specified in the agent flags. CNM plugins 
generally don't provide DNS option (not aware of libnetwork taking that as an 
option).

> Add --dns flag to the agent.
> 
>
> Key: MESOS-7709
> URL: https://issues.apache.org/jira/browse/MESOS-7709
> Project: Mesos
>  Issue Type: Task
>  Components: containerization
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
>
> Mesos support both CNI (through `network/cni` isolator) and CNM (through 
> docker) specification. Both these specifications allow for DNS entries for 
> containers to be set on a per-container, and per-network basis. 
> Currently, the behavior of the agent is to use the DNS nameservers set in 
> /etc/resolv.conf when the CNI or CNM plugin that is used to attached the 
> container to the CNI/CNM network doesnt' explicitly set the DNS for the 
> container. This is a bit inflexible especially when we have a mix of v4 and 
> v6 networks. 
> The operator should be able to specify DNS nameservers for the networks he 
> installs either the override the ones provided by the plugin or as defaults 
> when the plugins are not going to specify DNS name servers.
> In order to achieve the above goal we need to introduce a `\--dns` flag to 
> the agent. The `\--dns` flag should support a JSON (or a JSON file) with the 
> following schema:
> {code}
> {
>   "mesos": {
> [ 
>   {
> "network" : ,
> "nameservers": []
>   }
> ]
>   },
>   "docker": {
> [ 
>   {
> "network" : ,
> "nameservers": []
>   }
> ]
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7675) Isolate network ports.

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062520#comment-16062520
 ] 

Avinash Sridharan commented on MESOS-7675:
--

[~jpe...@apache.org] I am assuming this would work only for tasks on the host 
network. Also, this seems like we need to perform the algorithm for the 
lifetime of every task running on the agent? How do you propose we do this. By 
doing a periodic scan?

PS: By group isolation, did you mean cgroup isolation?

> Isolate network ports.
> --
>
> Key: MESOS-7675
> URL: https://issues.apache.org/jira/browse/MESOS-7675
> Project: Mesos
>  Issue Type: Improvement
>  Components: agent
>Reporter: James Peach
>Assignee: James Peach
>Priority: Minor
>
> If a task uses network ports, there is no isolator that can enforce that it 
> only listens on the ports that it has resources for. Implement a ports 
> isolator that can limit tasks to listen only on allocated TCP ports.
> Roughly, the algorithm for this follows what standard tools like {{lsof}} and 
> {{ss}} do.
> * Find all the listening TCP sockets (using netlink)
> * Index the sockets by their node (from the netlink information)
> * Find all the open sockets on the system (by scanning {{/proc/\*/fd/\*}} 
> links)
> * For each open socket, check whether its node (given in the link target) in 
> the set of listen sockets that we scanned
> * If the socket is a listening socket and the corresponding PID is in the 
> task, send a resource limitation for the task
> Matching pids to tasks depends on using group isolation, otherwise we would 
> have to build a full process tree, which would be nice to avoid.
> Scanning all the open sockets can be avoided by using the {{net_cls}} 
> isolator with kernel + libnl3 patches to publish the socket classid when we 
> find the listening socket.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7709) Add --dns flag to the agent.

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061815#comment-16061815
 ] 

Avinash Sridharan commented on MESOS-7709:
--

Currently, if the underlying plugins do not return a DNS nameserver than the 
network/cni isolator for CNI and docker daemon (for CNM) default to whatever is 
configured in the hosts `/etc/resolv.conf`. The problem with this mechanism is 
that `/etc/resolv.conf` allows only 3 nameservers to the be specified. So its 
pretty inflexible for the operators to setup defaults for networks where the 
operator knows the plugins are not going to return DNS nameservers. The problem 
becomes even more acerbated when you have a mix of v4 and v6 containers, since 
if now you rely only on `/etc/resolv.conf` to provide the default you will have 
pick some of the 3 possible nameservers to v4 and some to v6 again making it 
inflexible.

> Add --dns flag to the agent.
> 
>
> Key: MESOS-7709
> URL: https://issues.apache.org/jira/browse/MESOS-7709
> Project: Mesos
>  Issue Type: Task
>  Components: containerization
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
>
> Mesos support both CNI (through `network/cni` isolator) and CNM (through 
> docker) specification. Both these specifications allow for DNS entries for 
> containers to be set on a per-container, and per-network basis. 
> Currently, the behavior of the agent is to use the DNS nameservers set in 
> /etc/resolv.conf when the CNI or CNM plugin that is used to attached the 
> container to the CNI/CNM network doesnt' explicitly set the DNS for the 
> container. This is a bit inflexible especially when we have a mix of v4 and 
> v6 networks. 
> The operator should be able to specify DNS nameservers for the networks he 
> installs either the override the ones provided by the plugin or as defaults 
> when the plugins are not going to specify DNS name servers.
> In order to achieve the above goal we need to introduce a `--dns` flag to the 
> agent. The `--dns` flag should support a JSON (or a JSON file) with the 
> following schema:
> {
>   "mesos": {
>  [ 
>{ "network" : ,
>  "nameservers": []
>}
>  ]
>   },
>   "docker": {
> [ 
>{ "network" : ,
>  "nameservers": []
>}
>  ]
>   }
> }



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (MESOS-7631) DefautlExecutor needs to inform tasks about IP addresses

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan reassigned MESOS-7631:


Assignee: Avinash Sridharan  (was: Anand Mazumdar)

> DefautlExecutor needs to inform tasks about IP addresses
> 
>
> Key: MESOS-7631
> URL: https://issues.apache.org/jira/browse/MESOS-7631
> Project: Mesos
>  Issue Type: Task
>  Components: executor
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
> Fix For: 1.4.0
>
>
> Currently, when the DefaultExecutor launches tasks it doesn’t pass any 
> networking information (host or CNI network IP addresses) to the tasks that 
> it launches. This becomes problematic for the applications which need to know 
> the IP address that they need to bind to and cannot rely on binding to 
> INADDR_ANY. The tasks launched by the DefaultExecutor can potentially look at 
> all the available interface IP addresses and select a non-loopback IP address 
> but this is very error prone when there are multiple interfaces on a host or 
> a container is attached to multiple networks.
> Possible solution:
> The DefaultExecutor sets a variable MESOS_TASK_IP into the task’s shell. The 
> task will always read the variable to determine the IP address it should use. 
> Below we describe the algorithm to set MESOS_CONTAINER_IP for various cases.
> Setting MESOS_CONTAINER_IP for `host` network:
> For host network the `LIBPROCESS_IP` in the DefaultExecutor is always set to 
> the agent IP. The DefaultExecutor can therefore set `MESOS_TASK_IP` to the 
> `LIBPROCESS_IP` whenever it sees that the LIBPROCESS_IP is anything other 
> than “0.0.0.0” or “::” (for IPv6).
> Setting MESOS_TASK_IP for a single “CNI” network:
> For CNI network the `LIBPROCESS_IP` is set to “0.0.0.0” and the `hostname` of 
> the container is set to the CNI ip address. The DefaultExecutor already 
> resolves the hostname for CNI network (to register with agent), so it can 
> just set MESOS_TASK_IP to the IP address it learnt by resolving the hostname.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (MESOS-7712) CNI port-mapper plugin should allow only TCP or UDP in port-mapping fields.

[Avinash Sridharan (JIRA)]

Avinash Sridharan created MESOS-7712:


 Summary: CNI port-mapper plugin should allow only TCP or UDP in 
port-mapping fields.
 Key: MESOS-7712
 URL: https://issues.apache.org/jira/browse/MESOS-7712
 Project: Mesos
  Issue Type: Task
Reporter: Avinash Sridharan
Assignee: Avinash Sridharan


We should allow only TCP or UDP as protocols in the port-mapping protobuf 
passed on to the port-mapper CNI plugin.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-1958) Intercontainer IPv6 routing table

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-1958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16059618#comment-16059618
 ] 

Avinash Sridharan commented on MESOS-1958:
--

v6/v4 routing is taken care off by CNI so we shouldn't need this.

> Intercontainer IPv6 routing table
> -
>
> Key: MESOS-1958
> URL: https://issues.apache.org/jira/browse/MESOS-1958
> Project: Mesos
>  Issue Type: Task
>  Components: containerization
>Reporter: Evelina Dumitrescu
>Assignee: Evelina Dumitrescu
>
> Try table() should dump routes for AF_INET6 (maybe use 
> AF_UNSPEC ?)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-1956) Add IPv6 & ICMPv6 libnl traffic control U32 filters

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16059615#comment-16059615
 ] 

Avinash Sridharan commented on MESOS-1956:
--

Since these changes would be need only for the port-mapping isolator and not 
for CNI, and we are not introducing IPv6 support in the port-mapping network 
isolator closing this out.

> Add IPv6 & ICMPv6 libnl traffic control U32 filters
> ---
>
> Key: MESOS-1956
> URL: https://issues.apache.org/jira/browse/MESOS-1956
> Project: Mesos
>  Issue Type: Task
>  Components: containerization
>Reporter: Evelina Dumitrescu
>Assignee: Evelina Dumitrescu
>
> For IPv6, the filtering should be done by source and destination ports, 
> destination IP, destination MAC.
> For ICMPv6, the filtering should be done by protocol and destination IP.
> The IPv6/IPv4 difference could be done by the source/destination IP type from 
> the classifier.
> IPv4 packets with options in the header are currently ignored due to a bug in 
> libnl. It should be investigated if the problem occurs in the case of IPv6.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-1957) Port mapping IPv6 support

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-1957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16059589#comment-16059589
 ] 

Avinash Sridharan commented on MESOS-1957:
--

[~jieyu] the goal here seems to be to introduce IPv6 support in the 
port-mapping network isolator. I am assuming we are not going to be pursing 
this anymore?

> Port mapping IPv6 support
> -
>
> Key: MESOS-1957
> URL: https://issues.apache.org/jira/browse/MESOS-1957
> Project: Mesos
>  Issue Type: Task
>  Components: containerization
>Reporter: Evelina Dumitrescu
>Assignee: Evelina Dumitrescu
>
> It has to be investigated the best way for the port mapping module to know 
> that it uses and IPv6 environment. Possible ways are:
>   - by the IP address of the host eth0
>   - passing a flag
> A question that needs special attention is what should be done if eth0 or lo 
> have both IPv4 and IPv6 addresses.
> Also, it may be necessary to keep separate counters for IPv4/IPv6.
> The LOOPBACK_IP is currently hardcoded to 127.0.0.1/8. In the case of an IPv6 
> environment, owing to the fact that this variable is used only for services 
> that run on a single node and might not be used in routing scenarios, it 
> should be investigated keeping this value or adding another variable 
> LOOPBACK_IPv6. This will affect the add/removeContainerIPFilters and 
> add/removeHostIPFilters methods. 
> Socket diagnosis needs to be done also for AF_INET6 family.
> IPv6 disabling needs to be removed (eg:   script << "echo 1 > 
> /proc/sys/net/ipv6/conf/all/disable_ipv6\n"; from 
> PortMappingIsolatorProcess::scripts).
> For IPv4 some functionalities are disabled(eg: 'rp_filter' and 
> 'send_redirects' for lo) or enabled(eg: 'route_localnet' and 'accept_local' 
> for lo). Other ones are inherited by the container from the host(eg: 
> tcp_rmem, tcp_wmem).
> The ephemeral port ranges can be different between IPv6 and IPv4(eg: 
> os::read("/proc/sys/net/ipv4/ip_local_port_range"); ).
> The same thing should be done for IPv6.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-2249) Mesos entities should be able to use IPv6 and IPv4 in the same time

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-2249:
-
Shepherd: Jie Yu  (was: Dominic Hamon)

> Mesos entities should be able to use IPv6 and IPv4 in the same time
> ---
>
> Key: MESOS-2249
> URL: https://issues.apache.org/jira/browse/MESOS-2249
> Project: Mesos
>  Issue Type: Task
>Reporter: Evelina Dumitrescu
>Assignee: Evelina Dumitrescu
>
> Each Mesos entity should be able to bind on both Ipv4 and Ipv6 and let the 
> enitity that wants to connect to decide which protocol to use.
> For example, we can have a slave that wants to use IPv4 and another one that 
> wants to use IPv6, so the master should bind on both.
> In consequence, I want to propose in process.cpp to have two Node fields, one 
> for each type of endpoint. It might be better that the field for Ipv6 to be 
> an Option, because the stack might not support IPv6(eg: the kernel si not 
> compiled with Ipv6 support). Also, UPID will contain two fields of Node, for 
> each type of protocol.
> For the HTTP endpoints, whenever a request is done, the entities should try 
> firstly to connect on IPv4 and if the connection fails, to try to use IPv6, 
> or vice versa. We could let the user set up which policy to use. I think in 
> this context it does not matter which protocol is used. I saw this approach 
> in various projects:
> http://www.perforce.com/perforce/r13.1/manuals/cmdref/env.P4PORT.html 
> (tcp4to6(Attempt to listen/connect to an IPv4 address. If this fails, try 
> IPv6.) and tcp6to4(Attempt to listen/connect to an IPv6 address. If this 
> fails, try IPv4.)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (MESOS-2249) Mesos entities should be able to use IPv6 and IPv4 in the same time

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan reassigned MESOS-2249:


Assignee: Benjamin Hindman  (was: Evelina Dumitrescu)

> Mesos entities should be able to use IPv6 and IPv4 in the same time
> ---
>
> Key: MESOS-2249
> URL: https://issues.apache.org/jira/browse/MESOS-2249
> Project: Mesos
>  Issue Type: Task
>Reporter: Evelina Dumitrescu
>Assignee: Benjamin Hindman
>
> Each Mesos entity should be able to bind on both Ipv4 and Ipv6 and let the 
> enitity that wants to connect to decide which protocol to use.
> For example, we can have a slave that wants to use IPv4 and another one that 
> wants to use IPv6, so the master should bind on both.
> In consequence, I want to propose in process.cpp to have two Node fields, one 
> for each type of endpoint. It might be better that the field for Ipv6 to be 
> an Option, because the stack might not support IPv6(eg: the kernel si not 
> compiled with Ipv6 support). Also, UPID will contain two fields of Node, for 
> each type of protocol.
> For the HTTP endpoints, whenever a request is done, the entities should try 
> firstly to connect on IPv4 and if the connection fails, to try to use IPv6, 
> or vice versa. We could let the user set up which policy to use. I think in 
> this context it does not matter which protocol is used. I saw this approach 
> in various projects:
> http://www.perforce.com/perforce/r13.1/manuals/cmdref/env.P4PORT.html 
> (tcp4to6(Attempt to listen/connect to an IPv4 address. If this fails, try 
> IPv6.) and tcp6to4(Attempt to listen/connect to an IPv6 address. If this 
> fails, try IPv4.)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (MESOS-7709) Add --dns flag to the agent.

[Avinash Sridharan (JIRA)]

Avinash Sridharan created MESOS-7709:


 Summary: Add --dns flag to the agent.
 Key: MESOS-7709
 URL: https://issues.apache.org/jira/browse/MESOS-7709
 Project: Mesos
  Issue Type: Task
  Components: containerization
Reporter: Avinash Sridharan
Assignee: Avinash Sridharan


Mesos support both CNI (through `network/cni` isolator) and CNM (through 
docker) specification. Both these specifications allow for DNS entries for 
containers to be set on a per-container, and per-network basis. 

Currently, the behavior of the agent is to use the DNS nameservers set in 
/etc/resolv.conf when the CNI or CNM plugin that is used to attached the 
container to the CNI/CNM network doesnt' explicitly set the DNS for the 
container. This is a bit inflexible especially when we have a mix of v4 and v6 
networks. 

The operator should be able to specify DNS nameservers for the networks he 
installs either the override the ones provided by the plugin or as defaults 
when the plugins are not going to specify DNS name servers.

In order to achieve the above goal we need to introduce a `--dns` flag to the 
agent. The `--dns` flag should support a JSON (or a JSON file) with the 
following schema:
{
  "mesos": {
 [ 
   { "network" : ,
 "nameservers": []
   }
 ]
  },
  "docker": {
[ 
   { "network" : ,
 "nameservers": []
   }
 ]
  }
}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (MESOS-7488) Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013267#comment-16013267
 ] 

Avinash Sridharan edited comment on MESOS-7488 at 6/16/17 4:52 PM:
---

https://reviews.apache.org/r/59127/
https://reviews.apache.org/r/59128/
https://reviews.apache.org/r/59688/
https://reviews.apache.org/r/59739/
https://reviews.apache.org/r/59721/
https://reviews.apache.org/r/60136/
https://reviews.apache.org/r/60148/
https://reviews.apache.org/r/59131/
https://reviews.apache.org/r/60149/
https://reviews.apache.org/r/59130/
https://reviews.apache.org/r/59233/


was (Author: avin...@mesosphere.io):
https://reviews.apache.org/r/59127/
https://reviews.apache.org/r/59128/
https://reviews.apache.org/r/59129/
https://reviews.apache.org/r/59130/
https://reviews.apache.org/r/59131/
https://reviews.apache.org/r/59233/

> Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent
> -
>
> Key: MESOS-7488
> URL: https://issues.apache.org/jira/browse/MESOS-7488
> Project: Mesos
>  Issue Type: Task
>  Components: agent
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
>  Labels: mesosphere
> Fix For: 1.4.0
>
>
> As a first step to support IPv6 containers on Mesos, we need to provide 
> `--ip6` and `--ip6_discovery_command` flags to the agent so that the operator 
> can  specify an IPv6 address for the `libprocess` actor on the agent. In this 
> ticket we will not aim to add IPv6 communication support for Mesos but will 
> aim to use the IPv6 address provided by the operator to fill in the v6 
> address for any containers running on the host network in a dual stack 
> environment.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (MESOS-7631) DefautlExecutor needs to inform tasks about IP addresses

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7631:
-
Sprint: Mesosphere Sprint 57

> DefautlExecutor needs to inform tasks about IP addresses
> 
>
> Key: MESOS-7631
> URL: https://issues.apache.org/jira/browse/MESOS-7631
> Project: Mesos
>  Issue Type: Task
>  Components: executor
>Reporter: Avinash Sridharan
>Assignee: Anand Mazumdar
>
> Currently, when the DefaultExecutor launches tasks it doesn’t pass any 
> networking information (host or CNI network IP addresses) to the tasks that 
> it launches. This becomes problematic for the applications which need to know 
> the IP address that they need to bind to and cannot rely on binding to 
> INADDR_ANY. The tasks launched by the DefaultExecutor can potentially look at 
> all the available interface IP addresses and select a non-loopback IP address 
> but this is very error prone when there are multiple interfaces on a host or 
> a container is attached to multiple networks.
> Possible solution:
> The DefaultExecutor sets a variable MESOS_TASK_IP into the task’s shell. The 
> task will always read the variable to determine the IP address it should use. 
> Below we describe the algorithm to set MESOS_CONTAINER_IP for various cases.
> Setting MESOS_CONTAINER_IP for `host` network:
> For host network the `LIBPROCESS_IP` in the DefaultExecutor is always set to 
> the agent IP. The DefaultExecutor can therefore set `MESOS_TASK_IP` to the 
> `LIBPROCESS_IP` whenever it sees that the LIBPROCESS_IP is anything other 
> than “0.0.0.0” or “::” (for IPv6).
> Setting MESOS_TASK_IP for a single “CNI” network:
> For CNI network the `LIBPROCESS_IP` is set to “0.0.0.0” and the `hostname` of 
> the container is set to the CNI ip address. The DefaultExecutor already 
> resolves the hostname for CNI network (to register with agent), so it can 
> just set MESOS_TASK_IP to the IP address it learnt by resolving the hostname.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7631) DefautlExecutor needs to inform tasks about IP addresses

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7631:
-
Shepherd: Anand Mazumdar  (was: Vinod Kone)

> DefautlExecutor needs to inform tasks about IP addresses
> 
>
> Key: MESOS-7631
> URL: https://issues.apache.org/jira/browse/MESOS-7631
> Project: Mesos
>  Issue Type: Task
>  Components: executor
>Reporter: Avinash Sridharan
>Assignee: Anand Mazumdar
>
> Currently, when the DefaultExecutor launches tasks it doesn’t pass any 
> networking information (host or CNI network IP addresses) to the tasks that 
> it launches. This becomes problematic for the applications which need to know 
> the IP address that they need to bind to and cannot rely on binding to 
> INADDR_ANY. The tasks launched by the DefaultExecutor can potentially look at 
> all the available interface IP addresses and select a non-loopback IP address 
> but this is very error prone when there are multiple interfaces on a host or 
> a container is attached to multiple networks.
> Possible solution:
> The DefaultExecutor sets a variable MESOS_TASK_IP into the task’s shell. The 
> task will always read the variable to determine the IP address it should use. 
> Below we describe the algorithm to set MESOS_CONTAINER_IP for various cases.
> Setting MESOS_CONTAINER_IP for `host` network:
> For host network the `LIBPROCESS_IP` in the DefaultExecutor is always set to 
> the agent IP. The DefaultExecutor can therefore set `MESOS_TASK_IP` to the 
> `LIBPROCESS_IP` whenever it sees that the LIBPROCESS_IP is anything other 
> than “0.0.0.0” or “::” (for IPv6).
> Setting MESOS_TASK_IP for a single “CNI” network:
> For CNI network the `LIBPROCESS_IP` is set to “0.0.0.0” and the `hostname` of 
> the container is set to the CNI ip address. The DefaultExecutor already 
> resolves the hostname for CNI network (to register with agent), so it can 
> just set MESOS_TASK_IP to the IP address it learnt by resolving the hostname.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7631) DefautlExecutor needs to inform tasks about IP addresses

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7631:
-
Description: 
Currently, when the DefaultExecutor launches tasks it doesn’t pass any 
networking information (host or CNI network IP addresses) to the tasks that it 
launches. This becomes problematic for the applications which need to know the 
IP address that they need to bind to and cannot rely on binding to INADDR_ANY. 
The tasks launched by the DefaultExecutor can potentially look at all the 
available interface IP addresses and select a non-loopback IP address but this 
is very error prone when there are multiple interfaces on a host or a container 
is attached to multiple networks.

Possible solution:
The DefaultExecutor sets a variable MESOS_TASK_IP into the task’s shell. The 
task will always read the variable to determine the IP address it should use. 
Below we describe the algorithm to set MESOS_CONTAINER_IP for various cases.

Setting MESOS_CONTAINER_IP for `host` network:
For host network the `LIBPROCESS_IP` in the DefaultExecutor is always set to 
the agent IP. The DefaultExecutor can therefore set `MESOS_TASK_IP` to the 
`LIBPROCESS_IP` whenever it sees that the LIBPROCESS_IP is anything other than 
“0.0.0.0” or “::” (for IPv6).
Setting MESOS_TASK_IP for a single “CNI” network:
For CNI network the `LIBPROCESS_IP` is set to “0.0.0.0” and the `hostname` of 
the container is set to the CNI ip address. The DefaultExecutor already 
resolves the hostname for CNI network (to register with agent), so it can just 
set MESOS_TASK_IP to the IP address it learnt by resolving the hostname.



> DefautlExecutor needs to inform tasks about IP addresses
> 
>
> Key: MESOS-7631
> URL: https://issues.apache.org/jira/browse/MESOS-7631
> Project: Mesos
>  Issue Type: Task
>  Components: executor
>Reporter: Avinash Sridharan
>Assignee: Anand Mazumdar
>
> Currently, when the DefaultExecutor launches tasks it doesn’t pass any 
> networking information (host or CNI network IP addresses) to the tasks that 
> it launches. This becomes problematic for the applications which need to know 
> the IP address that they need to bind to and cannot rely on binding to 
> INADDR_ANY. The tasks launched by the DefaultExecutor can potentially look at 
> all the available interface IP addresses and select a non-loopback IP address 
> but this is very error prone when there are multiple interfaces on a host or 
> a container is attached to multiple networks.
> Possible solution:
> The DefaultExecutor sets a variable MESOS_TASK_IP into the task’s shell. The 
> task will always read the variable to determine the IP address it should use. 
> Below we describe the algorithm to set MESOS_CONTAINER_IP for various cases.
> Setting MESOS_CONTAINER_IP for `host` network:
> For host network the `LIBPROCESS_IP` in the DefaultExecutor is always set to 
> the agent IP. The DefaultExecutor can therefore set `MESOS_TASK_IP` to the 
> `LIBPROCESS_IP` whenever it sees that the LIBPROCESS_IP is anything other 
> than “0.0.0.0” or “::” (for IPv6).
> Setting MESOS_TASK_IP for a single “CNI” network:
> For CNI network the `LIBPROCESS_IP` is set to “0.0.0.0” and the `hostname` of 
> the container is set to the CNI ip address. The DefaultExecutor already 
> resolves the hostname for CNI network (to register with agent), so it can 
> just set MESOS_TASK_IP to the IP address it learnt by resolving the hostname.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (MESOS-7631) DefautlExecutor needs to inform tasks about IP addresses

[Avinash Sridharan (JIRA)]

Avinash Sridharan created MESOS-7631:


 Summary: DefautlExecutor needs to inform tasks about IP addresses
 Key: MESOS-7631
 URL: https://issues.apache.org/jira/browse/MESOS-7631
 Project: Mesos
  Issue Type: Task
  Components: executor
Reporter: Avinash Sridharan
Assignee: Anand Mazumdar






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7488) Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7488:
-
Labels: mesosphere  (was: )

> Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent
> -
>
> Key: MESOS-7488
> URL: https://issues.apache.org/jira/browse/MESOS-7488
> Project: Mesos
>  Issue Type: Task
>  Components: agent
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
>  Labels: mesosphere
> Fix For: 1.4.0
>
>
> As a first step to support IPv6 containers on Mesos, we need to provide 
> `--ip6` and `--ip6_discovery_command` flags to the agent so that the operator 
> can  specify an IPv6 address for the `libprocess` actor on the agent. In this 
> ticket we will not aim to add IPv6 communication support for Mesos but will 
> aim to use the IPv6 address provided by the operator to fill in the v6 
> address for any containers running on the host network in a dual stack 
> environment.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7488) Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7488:
-
  Sprint: Mesosphere Sprint 57
Story Points: 5

> Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent
> -
>
> Key: MESOS-7488
> URL: https://issues.apache.org/jira/browse/MESOS-7488
> Project: Mesos
>  Issue Type: Task
>  Components: agent
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
>  Labels: mesosphere
> Fix For: 1.4.0
>
>
> As a first step to support IPv6 containers on Mesos, we need to provide 
> `--ip6` and `--ip6_discovery_command` flags to the agent so that the operator 
> can  specify an IPv6 address for the `libprocess` actor on the agent. In this 
> ticket we will not aim to add IPv6 communication support for Mesos but will 
> aim to use the IPv6 address provided by the operator to fill in the v6 
> address for any containers running on the host network in a dual stack 
> environment.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7488) Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7488:
-
Fix Version/s: 1.4.0

> Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent
> -
>
> Key: MESOS-7488
> URL: https://issues.apache.org/jira/browse/MESOS-7488
> Project: Mesos
>  Issue Type: Task
>  Components: agent
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
> Fix For: 1.4.0
>
>
> As a first step to support IPv6 containers on Mesos, we need to provide 
> `--ip6` and `--ip6_discovery_command` flags to the agent so that the operator 
> can  specify an IPv6 address for the `libprocess` actor on the agent. In this 
> ticket we will not aim to add IPv6 communication support for Mesos but will 
> aim to use the IPv6 address provided by the operator to fill in the v6 
> address for any containers running on the host network in a dual stack 
> environment.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7488) Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7488:
-
Description: As a first step to support IPv6 containers on Mesos, we need 
to provide `--ip6` and `--ip6_discovery_command` flags to the agent so that the 
operator can  specify an IPv6 address for the `libprocess` actor on the agent. 
In this ticket we will not aim to add IPv6 communication support for Mesos but 
will aim to use the IPv6 address provided by the operator to fill in the v6 
address for any containers running on the host network in a dual stack 
environment.

> Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent
> -
>
> Key: MESOS-7488
> URL: https://issues.apache.org/jira/browse/MESOS-7488
> Project: Mesos
>  Issue Type: Task
>  Components: agent
>Reporter: Avinash Sridharan
>Assignee: Avinash Sridharan
>
> As a first step to support IPv6 containers on Mesos, we need to provide 
> `--ip6` and `--ip6_discovery_command` flags to the agent so that the operator 
> can  specify an IPv6 address for the `libprocess` actor on the agent. In this 
> ticket we will not aim to add IPv6 communication support for Mesos but will 
> aim to use the IPv6 address provided by the operator to fill in the v6 
> address for any containers running on the host network in a dual stack 
> environment.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (MESOS-7488) Add `--ip6` and `--ip6_discovery_command` flag to Mesos agent

[Avinash Sridharan (JIRA)]

Avinash Sridharan created MESOS-7488:


 Summary: Add `--ip6` and `--ip6_discovery_command` flag to Mesos 
agent
 Key: MESOS-7488
 URL: https://issues.apache.org/jira/browse/MESOS-7488
 Project: Mesos
  Issue Type: Task
  Components: agent
Reporter: Avinash Sridharan
Assignee: Avinash Sridharan






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MESOS-4606) Add IPv6 support to net::IP and net::IPNetwork

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15982053#comment-15982053
 ] 

Avinash Sridharan commented on MESOS-4606:
--

[~jieyu] we already have this patch from [~bennoe] can we look at committing 
this patch to Mesos.

> Add IPv6 support to net::IP and net::IPNetwork
> --
>
> Key: MESOS-4606
> URL: https://issues.apache.org/jira/browse/MESOS-4606
> Project: Mesos
>  Issue Type: Improvement
>  Components: stout
>Reporter: Benno Evers
>Assignee: Benno Evers
>Priority: Minor
>  Labels: network, stout
>
> The classes net::IP and net::IPNetwork should to be able to store IPv6 
> addresses.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (MESOS-7304) Fetcher should not depends on SlaveID.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan reassigned MESOS-7304:


Assignee: Joseph Wu

> Fetcher should not depends on SlaveID.
> --
>
> Key: MESOS-7304
> URL: https://issues.apache.org/jira/browse/MESOS-7304
> Project: Mesos
>  Issue Type: Task
>  Components: containerization, fetcher
>Reporter: Jie Yu
>Assignee: Joseph Wu
>
> Currently, various Fetcher interfaces depends on SlaveID, which is an 
> unnecessary coupling. For instance:
> {code}
> Try Fetcher::recover(const SlaveID& slaveId, const Flags& flags);
> Future Fetcher::fetch(
> const ContainerID& containerId,
> const CommandInfo& commandInfo,
> const string& sandboxDirectory,
> const Option& user,
> const SlaveID& slaveId,
> const Flags& flags);
> {code}
> Looks like the only reason we need a SlaveID is because we need to calculate 
> the fetcher cache directory based on that. We should calculate the fetcher 
> cache directory in the caller and pass that directory to Fetcher.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (MESOS-7302) Support launching standalone containers.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan reassigned MESOS-7302:


Assignee: Joseph Wu

> Support launching standalone containers.
> 
>
> Key: MESOS-7302
> URL: https://issues.apache.org/jira/browse/MESOS-7302
> Project: Mesos
>  Issue Type: Epic
>  Components: containerization
>Reporter: Jie Yu
>Assignee: Joseph Wu
>
> Containerizer should support launching containers (both top level and nested) 
> that are not tied to a particular Mesos task or executor. This is for the 
> case where the agent wants to launch some system containers (e.g., for CSI 
> plugin) that will be managed by Mesos.
> More specifically, the Containerizer interfaces should be refactored so that 
> they do not depend on TaskInfo or ExecutorInfo. Currently, the `launch` 
> interface depends on them. Instead, we should consistently use ContainerInfo 
> and CommandInfo in Containerizer and isolators.
> This is also one necessary step towards running MesosContainerizer in 
> standalone mode.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7326) ship a sample CNI config that provides a mesos-bridge network, analogous to the default bridge that ships with Docker

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7326:
-
Shepherd: Jie Yu

> ship a sample CNI config that provides a mesos-bridge network, analogous to 
> the default bridge that ships with Docker
> -
>
> Key: MESOS-7326
> URL: https://issues.apache.org/jira/browse/MESOS-7326
> Project: Mesos
>  Issue Type: Task
>  Components: documentation, network
>Reporter: James DeFelice
>Assignee: Avinash Sridharan
>  Labels: mesosphere
>
> UCR supports port mapping and marathon now ships an API that lets users 
> specify a `container/bridge` networking mode. by default, this maps to a CNI 
> network called `mesos-bridge`. Mesos should, at least:
> # ship a sample CNI configuration file that requires minimal edits, if any, 
> to enable a `mesos-bridge` CNI network for a vanilla mesos install; like 
> docker, the default bridge defaults to a "host-private" mode of operation
> # clearly document the steps required to enable this default bridge network 
> for simple mesos clusters
> /cc [~jieyu]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (MESOS-7326) ship a sample CNI config that provides a mesos-bridge network, analogous to the default bridge that ships with Docker

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan reassigned MESOS-7326:


Assignee: Avinash Sridharan

> ship a sample CNI config that provides a mesos-bridge network, analogous to 
> the default bridge that ships with Docker
> -
>
> Key: MESOS-7326
> URL: https://issues.apache.org/jira/browse/MESOS-7326
> Project: Mesos
>  Issue Type: Task
>  Components: documentation, network
>Reporter: James DeFelice
>Assignee: Avinash Sridharan
>  Labels: mesosphere
>
> UCR supports port mapping and marathon now ships an API that lets users 
> specify a `container/bridge` networking mode. by default, this maps to a CNI 
> network called `mesos-bridge`. Mesos should, at least:
> # ship a sample CNI configuration file that requires minimal edits, if any, 
> to enable a `mesos-bridge` CNI network for a vanilla mesos install; like 
> docker, the default bridge defaults to a "host-private" mode of operation
> # clearly document the steps required to enable this default bridge network 
> for simple mesos clusters
> /cc [~jieyu]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MESOS-7302) Support launching standalone containers.

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943468#comment-15943468
 ] 

Avinash Sridharan commented on MESOS-7302:
--

[~jieyu]
How would the standalone containers report task updates to the agent? They 
would still need to have the default/command executor semantics to monitor the 
container right?

Just a thought, in order to launch stand alone containers why can we make the 
agent act as a first class framework that can launch containers directly on the 
agent?

> Support launching standalone containers.
> 
>
> Key: MESOS-7302
> URL: https://issues.apache.org/jira/browse/MESOS-7302
> Project: Mesos
>  Issue Type: Epic
>  Components: containerization
>Reporter: Jie Yu
>
> Containerizer should support launching containers (both top level and nested) 
> that are not tied to a particular Mesos task or executor. This is for the 
> case where the agent wants to launch some system containers (e.g., for CSI 
> plugin) that will be managed by Mesos.
> More specifically, the Containerizer interfaces should be refactored so that 
> they do not depend on TaskInfo or ExecutorInfo. Currently, the `launch` 
> interface depends on them. Instead, we should consistently use ContainerInfo 
> and CommandInfo in Containerizer and isolators.
> This is also one necessary step towards running MesosContainerizer in 
> standalone mode.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (MESOS-7290) make fails at protobuf stage

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan reassigned MESOS-7290:


Assignee: Kapil Arya

> make fails at protobuf stage
> 
>
> Key: MESOS-7290
> URL: https://issues.apache.org/jira/browse/MESOS-7290
> Project: Mesos
>  Issue Type: Bug
>Affects Versions: 1.2.0
> Environment: CentOS 7.3 (built from 1611 image)
>Reporter: Raul Harnasch
>Assignee: Kapil Arya
>
> {noformat}
> Building protobuf Python egg ...
> cd ../3rdparty/protobuf-2.6.1/python &&   \
>   CC="gcc"\
>   CXX="g++"   \
>   CFLAGS="-g1 -O0 -Wno-unused-local-typedefs" \
>   CXXFLAGS="-g1 -O0 -Wno-unused-local-typedefs -std=c++11"
> \
>   PYTHONPATH=/opt/mesos/build/3rdparty/setuptools-20.9.0  \
>   /bin/python setup.py build bdist_egg
> Installed 
> /opt/mesos/build/3rdparty/protobuf-2.6.1/python/.eggs/google_apputils-0.4.2-py2.7.egg
> Traceback (most recent call last):
>   File "setup.py", line 200, in 
> "Protocol Buffers are Google's data interchange format.",
>   File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
> _setup_distribution = dist = klass(attrs)
>   File "/opt/mesos/build/3rdparty/setuptools-20.9.0/setuptools/dist.py", line 
> 269, in __init__
> self.fetch_build_eggs(attrs['setup_requires'])
>   File "/opt/mesos/build/3rdparty/setuptools-20.9.0/setuptools/dist.py", line 
> 313, in fetch_build_eggs
> replace_conflicting=True,
>   File 
> "/opt/mesos/build/3rdparty/setuptools-20.9.0/pkg_resources/__init__.py", line 
> 826, in resolve
> dist = best[req.key] = env.best_match(req, ws, installer)
>   File 
> "/opt/mesos/build/3rdparty/setuptools-20.9.0/pkg_resources/__init__.py", line 
> 1085, in best_match
> dist = working_set.find(req)
>   File 
> "/opt/mesos/build/3rdparty/setuptools-20.9.0/pkg_resources/__init__.py", line 
> 695, in find
> raise VersionConflict(dist, req)
> pkg_resources.VersionConflict: (pytz 2012d 
> (/usr/lib/python2.7/site-packages), Requirement.parse('pytz>=2010'))
> make[2]: *** 
> [../3rdparty/protobuf-2.6.1/python/dist/protobuf-2.6.1-py2.7.egg] Error 1
> make[2]: *** Waiting for unfinished jobs
> make[2]: Leaving directory `/opt/mesos/build/src'
> make[1]: *** [all] Error 2
> make[1]: Leaving directory `/opt/mesos/build/src'
> make: *** [all-recursive] Error 1
> {noformat}
> Looks like a dependency issue, but as the error suggests, I have pytz 2012d 
> when the minimum requirement is 2010. Pip confirms this:
> {noformat}
> $ pip freeze | grep pytz
> pytz===2012d
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MESOS-7290) make fails at protobuf stage

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939773#comment-15939773
 ] 

Avinash Sridharan commented on MESOS-7290:
--

[~rharnasch] have you installed all the dependencies in 
http://mesos.apache.org/gettingstarted/
For CentOS 7.1. Should be the same.



> make fails at protobuf stage
> 
>
> Key: MESOS-7290
> URL: https://issues.apache.org/jira/browse/MESOS-7290
> Project: Mesos
>  Issue Type: Bug
>Affects Versions: 1.2.0
> Environment: CentOS 7.3 (built from 1611 image)
>Reporter: Raul Harnasch
>
> {noformat}
> Building protobuf Python egg ...
> cd ../3rdparty/protobuf-2.6.1/python &&   \
>   CC="gcc"\
>   CXX="g++"   \
>   CFLAGS="-g1 -O0 -Wno-unused-local-typedefs" \
>   CXXFLAGS="-g1 -O0 -Wno-unused-local-typedefs -std=c++11"
> \
>   PYTHONPATH=/opt/mesos/build/3rdparty/setuptools-20.9.0  \
>   /bin/python setup.py build bdist_egg
> Installed 
> /opt/mesos/build/3rdparty/protobuf-2.6.1/python/.eggs/google_apputils-0.4.2-py2.7.egg
> Traceback (most recent call last):
>   File "setup.py", line 200, in 
> "Protocol Buffers are Google's data interchange format.",
>   File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
> _setup_distribution = dist = klass(attrs)
>   File "/opt/mesos/build/3rdparty/setuptools-20.9.0/setuptools/dist.py", line 
> 269, in __init__
> self.fetch_build_eggs(attrs['setup_requires'])
>   File "/opt/mesos/build/3rdparty/setuptools-20.9.0/setuptools/dist.py", line 
> 313, in fetch_build_eggs
> replace_conflicting=True,
>   File 
> "/opt/mesos/build/3rdparty/setuptools-20.9.0/pkg_resources/__init__.py", line 
> 826, in resolve
> dist = best[req.key] = env.best_match(req, ws, installer)
>   File 
> "/opt/mesos/build/3rdparty/setuptools-20.9.0/pkg_resources/__init__.py", line 
> 1085, in best_match
> dist = working_set.find(req)
>   File 
> "/opt/mesos/build/3rdparty/setuptools-20.9.0/pkg_resources/__init__.py", line 
> 695, in find
> raise VersionConflict(dist, req)
> pkg_resources.VersionConflict: (pytz 2012d 
> (/usr/lib/python2.7/site-packages), Requirement.parse('pytz>=2010'))
> make[2]: *** 
> [../3rdparty/protobuf-2.6.1/python/dist/protobuf-2.6.1-py2.7.egg] Error 1
> make[2]: *** Waiting for unfinished jobs
> make[2]: Leaving directory `/opt/mesos/build/src'
> make[1]: *** [all] Error 2
> make[1]: Leaving directory `/opt/mesos/build/src'
> make: *** [all-recursive] Error 1
> {noformat}
> Looks like a dependency issue, but as the error suggests, I have pytz 2012d 
> when the minimum requirement is 2010. Pip confirms this:
> {noformat}
> $ pip freeze | grep pytz
> pytz===2012d
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MESOS-7271) JNI SIGSEGV failed when connecting spark to mesos master

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939757#comment-15939757
 ] 

Avinash Sridharan commented on MESOS-7271:
--

[~mgummelt] [~anandmazumdar] is this an issue with the JNI or an 
incompatibility between the Spark and Mesos versions?

> JNI SIGSEGV failed when connecting spark to mesos master
> 
>
> Key: MESOS-7271
> URL: https://issues.apache.org/jira/browse/MESOS-7271
> Project: Mesos
>  Issue Type: Bug
>  Components: java api
>Affects Versions: 1.1.0, 1.2.0
> Environment: Ubuntu 16.04, OpenJDK 8, Spark 2.1.1
>Reporter: Qi Cui
>
> Run starting. Expected test count is: 1
> SampleDataFrameTest:
> 17/03/20 11:53:16 WARN NativeCodeLoader: Unable to load native-hadoop library 
> for your platform... using builtin-java classes where applicable
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0320 11:53:19.775842  4679 process.cpp:1071] libprocess is initialized on 
> 192.168.0.99:38293 with 8 worker threads
> I0320 11:53:19.775975  4679 logging.cpp:199] Logging to STDERR
> I0320 11:53:19.789871  4725 sched.cpp:226] Version: 1.1.0
> I0320 11:53:19.832826  4717 sched.cpp:330] New master detected at 
> master@192.168.0.50:5050
> I0320 11:53:19.838253  4717 sched.cpp:341] No credentials provided. 
> Attempting to register without authentication
> I0320 11:53:19.838337  4717 sched.cpp:820] Sending SUBSCRIBE call to 
> master@192.168.0.50:5050
> I0320 11:53:19.840265  4717 sched.cpp:853] Will retry registration in 
> 32.354951ms if necessary
> I0320 11:53:19.844734  4717 sched.cpp:743] Framework registered with 
> 6e147824-5d88-411b-9c09-a7137565c309-0001
> I0320 11:53:19.864850  4717 sched.cpp:757] Scheduler::registered took 
> 20.022604ms
> ERROR: exception pending on entry to FindMesosClass()
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x7ffa06fea4a6, pid=4677, tid=0x7ff9a1a46700
> #
> # JRE version: OpenJDK Runtime Environment (8.0_121-b13) (build 
> 1.8.0_121-8u121-b13-0ubuntu1.16.04.2-b13)
> # Java VM: OpenJDK 64-Bit Server VM (25.121-b13 mixed mode linux-amd64 
> compressed oops)
> # Problematic frame:
> # V  [libjvm.so+0x6744a6]
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core 
> dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # /media/sf_G_DRIVE/src/spark-testing-base/hs_err_pid4677.log
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> #



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (MESOS-7281) Backwards incompatible UpdateFrameworkMessage handling

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939753#comment-15939753
 ] 

Avinash Sridharan edited comment on MESOS-7281 at 3/24/17 4:00 AM:
---

[~mcypark] is this because of the multi-role changes. I am assuming it won't 
effect 1.2 ? 

Nevermind this is only on the master.


was (Author: avin...@mesosphere.io):
[~mcypark] is this because of the multi-role changes. I am assuming it won't 
effect 1.2 ?

> Backwards incompatible UpdateFrameworkMessage handling
> --
>
> Key: MESOS-7281
> URL: https://issues.apache.org/jira/browse/MESOS-7281
> Project: Mesos
>  Issue Type: Bug
>  Components: agent
>Reporter: Ilya Pronin
>Assignee: Ilya Pronin
>Priority: Blocker
>
> Patch in [r/57108|https://reviews.apache.org/r/57108/] introduced framework 
> info updates. Agents are using a new {{framework_info}} field without 
> checking that it's present. If a patched agent is used with not patched 
> master it will get a default-initialized {{framework_info}}. This will cause 
> agent failures later. E.g abort on framework ID validation when it tries to 
> launch a new task for the updated framework.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MESOS-7281) Backwards incompatible UpdateFrameworkMessage handling

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939753#comment-15939753
 ] 

Avinash Sridharan commented on MESOS-7281:
--

[~mcypark] is this because of the multi-role changes. I am assuming it won't 
effect 1.2 ?

> Backwards incompatible UpdateFrameworkMessage handling
> --
>
> Key: MESOS-7281
> URL: https://issues.apache.org/jira/browse/MESOS-7281
> Project: Mesos
>  Issue Type: Bug
>  Components: agent
>Reporter: Ilya Pronin
>Assignee: Ilya Pronin
>Priority: Blocker
>
> Patch in [r/57108|https://reviews.apache.org/r/57108/] introduced framework 
> info updates. Agents are using a new {{framework_info}} field without 
> checking that it's present. If a patched agent is used with not patched 
> master it will get a default-initialized {{framework_info}}. This will cause 
> agent failures later. E.g abort on framework ID validation when it tries to 
> launch a new task for the updated framework.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7272) Unified containerizer does not support docker registry version < 2.3.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7272:
-
Shepherd: Jie Yu

> Unified containerizer does not support docker registry version < 2.3.
> -
>
> Key: MESOS-7272
> URL: https://issues.apache.org/jira/browse/MESOS-7272
> Project: Mesos
>  Issue Type: Bug
>  Components: containerization, docker
>Affects Versions: 1.0.2, 1.1.1, 1.2.0
>Reporter: depay
>Assignee: Gilbert Song
>  Labels: easyfix
>
> in file `src/uri/fetchers/docker.cpp`
> ```
> Option contentType = response.headers.get("Content-Type");  
> if (contentType.isSome() &&  
> !strings::startsWith(  
> contentType.get(),  
> "application/vnd.docker.distribution.manifest.v1")) {  
>   return Failure(  
>   "Unsupported manifest MIME type: " + contentType.get());  
> }  
> ```
> Docker fetcher check the contentType strictly, while docker registry with 
> version < 2.3 returns manifests with contentType `application/json`, that 
> leading failure like `E0321 13:27:27.572402 40370 slave.cpp:4650] Container 
> 'xxx' for executor 'xxx' of framework xxx failed to start: Unsupported 
> manifest MIME type: application/json; charset=utf-8`.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MESOS-7272) Unified containerizer does not support docker registry version < 2.3.

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939750#comment-15939750
 ] 

Avinash Sridharan commented on MESOS-7272:
--

[~jieyu] assuming you are the shepherd here?

> Unified containerizer does not support docker registry version < 2.3.
> -
>
> Key: MESOS-7272
> URL: https://issues.apache.org/jira/browse/MESOS-7272
> Project: Mesos
>  Issue Type: Bug
>  Components: containerization, docker
>Affects Versions: 1.0.2, 1.1.1, 1.2.0
>Reporter: depay
>Assignee: Gilbert Song
>  Labels: easyfix
>
> in file `src/uri/fetchers/docker.cpp`
> ```
> Option contentType = response.headers.get("Content-Type");  
> if (contentType.isSome() &&  
> !strings::startsWith(  
> contentType.get(),  
> "application/vnd.docker.distribution.manifest.v1")) {  
>   return Failure(  
>   "Unsupported manifest MIME type: " + contentType.get());  
> }  
> ```
> Docker fetcher check the contentType strictly, while docker registry with 
> version < 2.3 returns manifests with contentType `application/json`, that 
> leading failure like `E0321 13:27:27.572402 40370 slave.cpp:4650] Container 
> 'xxx' for executor 'xxx' of framework xxx failed to start: Unsupported 
> manifest MIME type: application/json; charset=utf-8`.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7285) Implement a plugin to list container's on a given agent.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7285:
-
Description: We need the CLI to support a `list` command to enumerate the 
containers running on a given agent. To achieve this we will need to implement 
a container plugin that will be implement this list method.

> Implement a plugin to list container's on a given agent.
> 
>
> Key: MESOS-7285
> URL: https://issues.apache.org/jira/browse/MESOS-7285
> Project: Mesos
>  Issue Type: Task
>Reporter: Avinash Sridharan
>Assignee: Armand Grillet
>
> We need the CLI to support a `list` command to enumerate the containers 
> running on a given agent. To achieve this we will need to implement a 
> container plugin that will be implement this list method.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (MESOS-7285) Implement a plugin to list container's on a given agent.

[Avinash Sridharan (JIRA)]

Avinash Sridharan created MESOS-7285:


 Summary: Implement a plugin to list container's on a given agent.
 Key: MESOS-7285
 URL: https://issues.apache.org/jira/browse/MESOS-7285
 Project: Mesos
  Issue Type: Task
Reporter: Avinash Sridharan
Assignee: Armand Grillet






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7284) Allow Mesos CLI to take an agent IP

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7284:
-
Description: Allow the Mesos CLI to take an agent IP. This will allow the 
CLI to remotely connect to an agent and run commands on that agent.

> Allow Mesos CLI to take an agent IP
> ---
>
> Key: MESOS-7284
> URL: https://issues.apache.org/jira/browse/MESOS-7284
> Project: Mesos
>  Issue Type: Task
>Reporter: Avinash Sridharan
>Assignee: Armand Grillet
>
> Allow the Mesos CLI to take an agent IP. This will allow the CLI to remotely 
> connect to an agent and run commands on that agent.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (MESOS-7284) Allow Mesos CLI to take an agent IP

[Avinash Sridharan (JIRA)]

Avinash Sridharan created MESOS-7284:


 Summary: Allow Mesos CLI to take an agent IP
 Key: MESOS-7284
 URL: https://issues.apache.org/jira/browse/MESOS-7284
 Project: Mesos
  Issue Type: Task
Reporter: Avinash Sridharan
Assignee: Armand Grillet






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (MESOS-7283) Add ability to initialize a test cluster for Mesos CLI unit-test infrastructure

[Avinash Sridharan (JIRA)]

Avinash Sridharan created MESOS-7283:


 Summary: Add ability to initialize a test cluster for Mesos CLI 
unit-test infrastructure
 Key: MESOS-7283
 URL: https://issues.apache.org/jira/browse/MESOS-7283
 Project: Mesos
  Issue Type: Task
Reporter: Avinash Sridharan
Assignee: Armand Grillet


Similar to the Mesos unit-tests we need to have the ability to bring up a test 
cluster against which we can run the python unit-tests for Mesos CLI.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-7282) Create a table abstraction for the Mesos CLI

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-7282:
-
Description: Add a an abstraction for printing and formatting tables in the 
Mesos CLI. This will very useful for all commands that need to print their 
output in a table format.

> Create a table abstraction for the Mesos CLI
> 
>
> Key: MESOS-7282
> URL: https://issues.apache.org/jira/browse/MESOS-7282
> Project: Mesos
>  Issue Type: Task
>Reporter: Avinash Sridharan
>Assignee: Armand Grillet
>
> Add a an abstraction for printing and formatting tables in the Mesos CLI. 
> This will very useful for all commands that need to print their output in a 
> table format.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (MESOS-7282) Create a table abstraction for the Mesos CLI

[Avinash Sridharan (JIRA)]

Avinash Sridharan created MESOS-7282:


 Summary: Create a table abstraction for the Mesos CLI
 Key: MESOS-7282
 URL: https://issues.apache.org/jira/browse/MESOS-7282
 Project: Mesos
  Issue Type: Task
Reporter: Avinash Sridharan
Assignee: Armand Grillet






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-6032) Add infrastructure for unit tests in the new python-based CLI.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-6032:
-
Shepherd: Kevin Klues  (was: Joseph Wu)

> Add infrastructure for unit tests in the new python-based CLI.
> --
>
> Key: MESOS-6032
> URL: https://issues.apache.org/jira/browse/MESOS-6032
> Project: Mesos
>  Issue Type: Task
>  Components: cli, tests
>Reporter: Kevin Klues
>Assignee: Armand Grillet
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (MESOS-6032) Add infrastructure for unit tests in the new python-based CLI.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan reassigned MESOS-6032:


Assignee: Armand Grillet  (was: Kevin Klues)

> Add infrastructure for unit tests in the new python-based CLI.
> --
>
> Key: MESOS-6032
> URL: https://issues.apache.org/jira/browse/MESOS-6032
> Project: Mesos
>  Issue Type: Task
>  Components: cli, tests
>Reporter: Kevin Klues
>Assignee: Armand Grillet
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (MESOS-7252) Need to fix resource check in long-lived framework and inverse-offer framework

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-7252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan reassigned MESOS-7252:


Assignee: Kapil Arya

> Need to fix resource check in long-lived framework and inverse-offer framework
> --
>
> Key: MESOS-7252
> URL: https://issues.apache.org/jira/browse/MESOS-7252
> Project: Mesos
>  Issue Type: Bug
>  Components: framework
>Reporter: Avinash Sridharan
>Assignee: Kapil Arya
>  Labels: Mesosphere
>
> The multi-role changes in Mesos changed the implementation of 
> `Resources::contains`. This results in the search for a given resource to be 
> performed only for unallocated resources. For allocated resources the search 
> is actually performed only for a given role. 
> Due to this change the resource check in both the long-lived framework and 
> the inverse-offer framework are both failing leading to these frameworks not 
> launching any tasks. 
> The fix would be to unallocate all resources in a given offer and than do the 
> `contains` check.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (MESOS-6240) Allow executor/agent communication over non-TCP/IP stream socket.

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15762620#comment-15762620
 ] 

Avinash Sridharan edited comment on MESOS-6240 at 3/21/17 12:07 AM:


Libprocess has support for domain socket, but the agent/executor communication 
is still over TCP/IP sockets. We still need to evaluate changes required to 
switch the agent/executor communication to domain socket.


was (Author: avin...@mesosphere.io):
Libprocess has support for domain socket, but the agent <-> executor 
communication is still over TCP/IP sockets. We still need to evaluate changes 
required to switch the agent <-> executor communication to domain socket.

> Allow executor/agent communication over non-TCP/IP stream socket.
> -
>
> Key: MESOS-6240
> URL: https://issues.apache.org/jira/browse/MESOS-6240
> Project: Mesos
>  Issue Type: Improvement
>  Components: containerization
> Environment: Linux and Windows
>Reporter: Avinash Sridharan
>  Labels: mesosphere
>
> Currently, the executor agent communication happens specifically over TCP 
> sockets. This works fine in most cases, but specifically for the 
> `MesosContainerizer` when containers are running on CNI networks, this mode 
> of communication starts imposing constraints on the CNI network. Since, now 
> there has to connectivity between the CNI network  (on which the executor is 
> running) and the agent. Introducing paths from a CNI network to the 
> underlying agent, at best, creates headaches for operators and at worst 
> introduces serious security holes in the network, since it is breaking the 
> isolation between the container CNI network and the host network (on which 
> the agent is running).
> In order to simplify/strengthen deployment of Mesos containers on CNI 
> networks we therefore need to move away from using TCP/IP sockets for 
> executor/agent communication. Since, executor and agent are guaranteed to run 
> on the same host, the above problems can be resolved if, for the 
> `MesosContainerizer`, we use UNIX domain sockets or named pipes instead of 
> TCP/IP sockets for the executor/agent communication.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (MESOS-6240) Allow executor/agent communication over non-TCP/IP stream socket.

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15762620#comment-15762620
 ] 

Avinash Sridharan edited comment on MESOS-6240 at 3/21/17 12:06 AM:


Libprocess has support for domain socket, but the agent <-> executor 
communication is still over TCP/IP sockets. We still need to evaluate changes 
required to switch the agent <-> executor communication to domain socket.


was (Author: avin...@mesosphere.io):
Libprocess has support for domain socket, but the agent<->executor 
communication is still over TCP/IP sockets. We still need to evaluate changes 
required to switch the agent<->executor communication to domain socket.

> Allow executor/agent communication over non-TCP/IP stream socket.
> -
>
> Key: MESOS-6240
> URL: https://issues.apache.org/jira/browse/MESOS-6240
> Project: Mesos
>  Issue Type: Improvement
>  Components: containerization
> Environment: Linux and Windows
>Reporter: Avinash Sridharan
>  Labels: mesosphere
>
> Currently, the executor agent communication happens specifically over TCP 
> sockets. This works fine in most cases, but specifically for the 
> `MesosContainerizer` when containers are running on CNI networks, this mode 
> of communication starts imposing constraints on the CNI network. Since, now 
> there has to connectivity between the CNI network  (on which the executor is 
> running) and the agent. Introducing paths from a CNI network to the 
> underlying agent, at best, creates headaches for operators and at worst 
> introduces serious security holes in the network, since it is breaking the 
> isolation between the container CNI network and the host network (on which 
> the agent is running).
> In order to simplify/strengthen deployment of Mesos containers on CNI 
> networks we therefore need to move away from using TCP/IP sockets for 
> executor/agent communication. Since, executor and agent are guaranteed to run 
> on the same host, the above problems can be resolved if, for the 
> `MesosContainerizer`, we use UNIX domain sockets or named pipes instead of 
> TCP/IP sockets for the executor/agent communication.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (MESOS-7269) Migrate setting in config.py to a TOML file

[Avinash Sridharan (JIRA)]

Avinash Sridharan created MESOS-7269:


 Summary: Migrate setting in config.py to a TOML file
 Key: MESOS-7269
 URL: https://issues.apache.org/jira/browse/MESOS-7269
 Project: Mesos
  Issue Type: Task
  Components: cli
Reporter: Avinash Sridharan
Assignee: Armand Grillet






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MESOS-4945) Garbage collect unused docker layers in the store.

[Avinash Sridharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932786#comment-15932786
 ] 

Avinash Sridharan commented on MESOS-4945:
--

[~gilbert] [~zhitao] are there any more tickets in this epic. Looks a bit 
incomplete at this point.

> Garbage collect unused docker layers in the store.
> --
>
> Key: MESOS-4945
> URL: https://issues.apache.org/jira/browse/MESOS-4945
> Project: Mesos
>  Issue Type: Epic
>Reporter: Jie Yu
>Assignee: Zhitao Li
>
> Right now, we don't have any garbage collection in place for docker layers. 
> It's not straightforward to implement because we don't know what container is 
> currently using the layer. We probably need a way to track the current usage 
> of layers.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (MESOS-4945) Garbage collect unused docker layers in the store.

[Avinash Sridharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Avinash Sridharan updated MESOS-4945:
-
Epic Name: UCR GC

> Garbage collect unused docker layers in the store.
> --
>
> Key: MESOS-4945
> URL: https://issues.apache.org/jira/browse/MESOS-4945
> Project: Mesos
>  Issue Type: Epic
>Reporter: Jie Yu
>Assignee: Zhitao Li
>
> Right now, we don't have any garbage collection in place for docker layers. 
> It's not straightforward to implement because we don't know what container is 
> currently using the layer. We probably need a way to track the current usage 
> of layers.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)