Default value of Filter.refuse_seconds

[Erb, Stephan]

Hi everyone,

within this Aurora review request https://reviews.apache.org/r/46603/ we are 
wondering about the current Filter.refuse_seconds default value of 5 seconds 
[1]. 

Is there any reason why this particular value was chosen? Would we have to 
expect increased load for the leading Mesos master of extremely large clusters 
if we set it to 0? 

Thanks for your feedback!,
Stephan


[1] 
https://github.com/apache/mesos/blob/3d2d3edb96c9c428f988653cdd4428a05690e747/include/mesos/mesos.proto#L1423

Re: LIBPROCSES_IP

[Erb, Stephan]

>Framework should be the one that sets
>LIBPROCESS_ADVERTISE_IP and LIBPROCESS_ADVERTISE_PORT appropriately if it
>tries to launch another Mesos framework so that Master can reach the new
>framework.

As a framework/executor author this is not possible in all scenarios: There is 
no way to discover IP addresses assigned via CNI before the first StatusUpdate 
has been received. It is therefore not possible to set LIBPROCESS_ADVERTISE_IP 
appropriately at launch time. 

Please see https://issues.apache.org/jira/browse/MESOS-6281 for details.


On 12/10/16 06:42, "Avinash Sridharan"  wrote:

Valid point. Makes sense to drive this decision from the user and the
framework.

On Tue, Oct 11, 2016 at 9:32 PM, Jie Yu  wrote:

> >
> > While I believe this particular logic of setting LIBPROCESS_ADVERTISE_IP
> > to agent IP can be done in the agent (it could look at the port mapping
> > as well)
>
>
> What if there are multiple port mappings? How can the agent decide which
> port to be used as  LIBPROCESS_ADVERTISE_PORT?
>
> On Tue, Oct 11, 2016 at 9:27 PM, Avinash Sridharan 
> wrote:
>
> > Definitely a +1 for executor binding to 0.0.0.0, instead of doing a
> > `gethostname` and `getaddrinfo`. But I am assuming this semantics would
> > kick in only if LIBPROCESS_IP is not set, which should be the norm.
> >
> > +1 for LIBPROCESS_ADVERTISE_IP and LIBPROCESS_ADVERTISE_PORT and the 
onus
> > being on the frameworks to set these variables. I guess the framework 
can
> > set the LIBPROCESS_ADVERTISE_IP to the agent IP and
> > LIBPROCESS_ADVERTISE_PORT to the host port when it specifies a
> > port-mapping. While I believe this particular logic of
> > setting LIBPROCESS_ADVERTISE_IP to agent IP can be done in the agent (it
> > could look at the port mapping as well), when to actually set these
> > variables (whether the executors even need to advertise their IP
> addresses,
> > is a decision that the Frameworks should be privy too and not left to 
the
> > agent.
> >
> > On Tue, Oct 11, 2016 at 7:31 PM, haosdent  wrote:
> >
> > > > libprocess should always bind to 0.0.0.0
> > > + 1 for this
> > >
> > > On Wed, Oct 12, 2016 at 2:33 AM, Jie Yu  wrote:
> > >
> > > > Hi folks,
> > > >
> > > > I was in the process of cleaning up some tech debt related to env
> > > variables
> > > > in our code base. I created an epic ticket
> > > >  to track. I
> > searched
> > > > relevant tickets fired previously, and found MESOS-3740
> > > > . I did some
> digging
> > > on
> > > > how we handle LIBPROCESS_IP currently, and here are my findings:
> > > >
> > > > 1) We always set LIBPROCESS_IP in the executor environment 
variables:
> > > > https://github.com/apache/mesos/blob/master/src/slave/
> > > > slave.cpp#L6793-L6796
> > > >
> > > > This is not an issue for an executor that runs on host network.
> > However,
> > > if
> > > > the executor wants to run on non-host network (e.g., overlay), this
> > might
> > > > be problematic, because libprocess for the executor will try to bind
> to
> > > > LIBPROCESS_IP, but the IP is not valid inside the container.
> > > >
> > > > 2) As mentioned in MESOS-3740
> > > > , some user wants
> to
> > > run
> > > > a Mesos framework in a Mesos container. The old style framework
> driver
> > > > assumes a 2 way communication channel between the framework and the
> > Mesos
> > > > master. In order for the master to reach the framework running
> inside a
> > > > Mesos container, the framework's libprocess should advertise its ip
> and
> > > > port properly. This problem gets tricky because the networking for
> the
> > > > Mesos container:
> > > >
> > > > 2.a) If the container uses host network, libprocess should bind to
> > > 0.0.0.0,
> > > > and advertise itself using the agent ip and the relevant port
> > > > 2.b) If the container has a routable ip (e.g., using calico or
> > overlay),
> > > > libprocess should still bind to 0.0.0.0, and advertise itself using
> the
> > > > container ip and the relevant port. Currently, it binds to agent ip
> > > (which
> > > > will fail), and advertise itself using agnet ip and the port in the
> > > > container (which will fail as well)
> > > > 2.c) If the container has a private ip (e.g., bridge), libprocess
> > should
> > > > still bind to 0.0.0.0, and advertise itself using the agent ip and
> > > _mapped_
> > > > host port. Currently, it 

mesos.interface==1.2.0

[Erb, Stephan]

The mesos.interface==1.2.0 Python package is still missing on pypi 
(https://pypi.python.org/pypi/mesos.interface/). Would be great if one of you 
could upload it.

Thanks,
Stephan

Re: mesos.interface==1.4.0

[Erb, Stephan]

Thanks!

On 01.11.17, 13:04, "Kapil Arya" <ka...@mesosphere.io> wrote:

> Kapil was this missed during the 1.4.0 release?


Yeah, it was missed due to PyPi deprecating the legacy API which our CI job
depended on. The failure to publish went unnoticed unfortunately and hence
the missing package. I have uploaded it manually for now and will fix the
CI script as well.

It seems to be in the
> release guide:
>
> 
http://mesos.apache.org/documentation/latest/release-guide/#updating-external-tooling
>
> On Sun, Oct 22, 2017 at 1:38 AM, Erb, Stephan <stephan@blue-yonder.com
> >
> wrote:
>
> > Hi,
> >
> > the mesos.interface==1.4.0 Python package is currently missing on PyPi (
> > https://pypi.python.org/pypi/mesos.interface/).
> >
> > Would be great if one of you could upload it.
> >
> > Thanks a lot,
> > Stephan
> >
>

mesos.interface==1.4.0

[Erb, Stephan]

Hi,

the mesos.interface==1.4.0 Python package is currently missing on PyPi 
(https://pypi.python.org/pypi/mesos.interface/).

Would be great if one of you could upload it.

Thanks a lot,
Stephan