Re: [API WG] Proposals for dealing with master subscriber leaks.

2018-11-14 Thread Joseph Wu
Heartbeats are currently the least-liked solution, for precisely the reason
BenM stated.  Clients of the API, such as the maintainers of the DC/OS UI,
would also like to avoid making more connections than necessary and/or
keeping additional state between connections.


Currently, I am leaning towards keeping subscribers in a circular buffer.
This solution is minimal in the code footprint and requires no client-side
changes besides heavily incentivizing retry logic (which we already expect
in most cases).
One potential downside is having more subscribers than the (master flag)
configured maximum.  In this case, each client would kick out the first
few; which would then retry and kick out the next few, etc.  Each retry is
equivalent to a GET /master/state, and the extra calls would basically
erase the performance gains we have from streaming the events.

Nevertheless, I think a reasonably high default would have minimal impact
on both master performance and client connectivity.  The code for this
proposal can be found here:
https://reviews.apache.org/r/69307/  (Just one review)

On Sun, Nov 11, 2018 at 9:22 AM Benjamin Mahler  wrote:

> >- We can add heartbeats to the SUBSCRIBE call.
> > This would need to be
> >  part of a separate operator Call, because one platform (browsers) that
> > might subscribe to the master does not support two-way streaming.
>
> This doesn't make sense to me, the heartbeats should still be part of the
> same connection (request and response are infinite and heartbeating) by
> default. Splitting into a separate call is messy and shouldn't be what we
> force everyone to do, it should only be done in cases that it's impossible
> to use a single connection (e.g. browsers).
>
> On Sat, Nov 10, 2018 at 12:03 AM Joseph Wu  wrote:
>
>> Hi all,
>>
>> During some internal scale testing, we noticed that, when Mesos streaming
>> endpoints are accessed via certain proxies (or load balancers), the
>> proxies
>> might not close connections after they are complete.  For the Mesos
>> master,
>> which only has the /api/v1 SUBSCRIBE streaming endpoint, this can generate
>> unnecessary authorization requests and affects performance.
>>
>> We are considering a few potential solutions:
>>
>>- We can add heartbeats to the SUBSCRIBE call.  This would need to be
>>part of a separate operator Call, because one platform (browsers) that
>>might subscribe to the master does not support two-way streaming.
>>- We can add (optional) arguments to the SUBSCRIBE call, which tells
>> the
>>master to disconnect it after a while.  And the client would have to
>> remake
>>the connection every so often.
>>- We can change the master to hold subscribers in a circular buffer,
>> and
>>disconnect the oldest ones if there are too many connections.
>>
>> We're tracking progress on this issue here:
>> https://issues.apache.org/jira/browse/MESOS-9258
>> Some prototypes of the code changes involved are also linked in the JIRA.
>>
>> Please chime in if you have any suggestions or if any of these options
>> would be undesirable/bad,
>> ~Joseph
>>
>


Mesos Infrastructure - Intern

2018-11-14 Thread Thodoris Zois
Hello list,

I am sorry about the irrelevant content, but I think this is the best place to 
ask! 

Currently I am a postgraduate student at Vrije University of Amsterdam and I am 
looking for a summer internship. In my BSc. thesis I got a lot of hands-on 
experience on Mesos among other interesting stuff. Since many people here come 
from companies that use Mesos and I would really like to continue working on 
that direction, if there is any open position in Europe, please let me know so 
I can apply!

Thank you very much,
Thodoris Zois