[Proposal] Use jemalloc as default memory allocator for Mesos

2017-08-18 Thread Benno Evers
change. Users who compile Mesos themselves would of course have the option to disable jemalloc at configuration time or replace it with their memory allocator of choice. I'm looking forward to hear any thoughts and comments. Thanks, -- Benno Evers Software Engineer, Mesosphere

Re: [Proposal] Use jemalloc as default memory allocator for Mesos

2017-08-22 Thread Benno Evers
dvantages of both options. > > Alexander Rojas > alexan...@mesosphere.io > > > > > > On 18. Aug 2017, at 12:49, Benno Evers wrote: > > > > Hi all, > > > > I would like to propose bundling jemalloc as a new dependency > > under `3rdpart

Sending TASK_STARTING in the built-in executors

2017-08-23 Thread Benno Evers
of the built-in mesos executors, and 2. it doesn't handle the possibility of receiving TASK_STARTING update, and 3. it reports an error whenever it encounters an unexpected task states in an update. If you are aware of any such framework, please speak up so we can consider it. Thanks, --

Re: Sending TASK_STARTING in the built-in executors

2017-08-23 Thread Benno Evers
t of the `docker inspect` command for the started container. On Wed, Aug 23, 2017 at 4:54 PM, James Peach wrote: > > > On Aug 23, 2017, at 2:38 AM, Benno Evers wrote: > > > > Hi all, > > > > when starting a task, an executor can send out the following status &g

Re: [Proposal] Use jemalloc as default memory allocator for Mesos

2017-08-25 Thread Benno Evers
e are solutions that I know work on Windows (from past work I've > done). I'm unsure about either jemalloc and tcmalloc, however. > > Thanks, > > /Jeff > > -Original Message- > From: Benno Evers [mailto:bev...@mesosphere.com] > Sent: Tuesday, August 22,

Re: CMake build refactoring

2017-08-28 Thread Benno Evers
s to stout >>> is >>> >>> `target_link_libraries(process stout)`, and the boost dependency is >>> >>> understood transitively. >>> >>> >>> >>> The second phase was refactoring the Mesos build itself (that is, not &g

Re: [Proposal] Use jemalloc as default memory allocator for Mesos

2017-09-05 Thread Benno Evers
packages. > > Seems that without trying it, we won’t find out if jemalloc works as hoped > on Windows for us - the Firefox project results however are encouraging. On > the other hand, if it doesn’t work, we could simply decide to disable it on > Windows just like some other Mesos f

Re: strange behaviour: Task status -> error-> finished

2017-09-19 Thread Benno Evers
"reason" of the ERROR, so I do not know > what occured, and cannot at this stage reproduce manually the use case. > > Can we have "non terminal" errors, from mesos point of view, where task > should not be considered as over? > > Thanks > > Olivier > -- Benno Evers Software Engineer, Mesosphere

Re: [Proposal] Use jemalloc as default memory allocator for Mesos

2017-09-20 Thread Benno Evers
lloc. If worse > comes to worse, we'll maintain our own cmake build process for jemalloc > against a specific branch. We already maintain local changes for some other > 3rd party dependencies. > > /Jeff > > -Original Message- > From: Benno Evers [mailto:bev...@mesos

Re: Shedding light on libmesos and libprocess threading models?

2017-09-26 Thread Benno Evers
appears > (without OMP) that libmesos or libprocess still spawn an additional 12 > threads. So what are those 12 threads used for? > > Oh - this is the (ancient) 0.28.3-2.0.1 release for Ubuntu 14.04 LTS, in > case that matters. > > Cheers, > > Jim > > -- > Senior Production Engineer > Industrial Light & Magic (ILM) > -- Benno Evers Software Engineer, Mesosphere

Re: Sending TASK_STARTING in the built-in executors

2017-10-20 Thread Benno Evers
cover that some framework you depend on would choke on this, now would be a good time to update it before upgrading to Mesos 1.5.0 ;) Best regards, -- Benno Evers Software Engineer, Mesosphere On Wed, Aug 23, 2017 at 11:38 AM, Benno Evers wrote: > Hi all, > > when starting a task, an exe

Design doc: Relaxing Agent State Recovery on Restart

2017-11-04 Thread Benno Evers
are welcome and highly valued. Best regards, -- Benno Evers Software Engineer, Mesosphere

The --recover=cleanup flag

2017-11-06 Thread Benno Evers
e executors and exit. Use this option when doing an incompatible agent or executor upgrade!). (default: reconnect) Best regards, -- Benno Evers Software Engineer, Mesosphere

Re: Design doc: Relaxing Agent State Recovery on Restart

2017-11-08 Thread Benno Evers
up another, new > > proposal to relax this strict behaviour: > > > > > > https://docs.google.com/document/d/1iOENs0JoXPc7sf1NDBCR2tPJ_ > > KxwU4lLtr53SrE5U3Q/edit?usp=sharing > > > > As always, any comments and suggestions are welcome and highly valued. > > > > Best regards, > > -- > > Benno Evers > > Software Engineer, Mesosphere > > > > > > -- > Cheers, > > Zhitao Li > -- Benno Evers Software Engineer, Mesosphere

Re: Design doc: Relaxing Agent State Recovery on Restart

2017-11-09 Thread Benno Evers
Whoops, sorry, done. On Thu, Nov 9, 2017 at 6:41 AM, Zhitao Li wrote: > Can you allow viewers to comment on the doc? Thanks > > On Wed, Nov 8, 2017 at 5:06 PM, Benno Evers wrote: > > > Thanks to everyone for the great comments. > > > > After thinking about the

Mesos commit hooks requiring internet access

2017-11-10 Thread Benno Evers
t be a better way to implement this. My proposal would be to move most checks to post-reviews.py, which is a slow operation anyways and which actually marks the point in time where the commits should be cleaned up enough to pass all checks. What do you think? Best regards, -- Benno Evers Software Engineer, Mesosphere

Re: Mesos commit hooks requiring internet access

2017-11-13 Thread Benno Evers
not a common event as you can see from the history of the CLI > pip-requirements.txt: > https://github.com/apache/mesos/commits/master/src/python/cli_new/pip- > requirements.txt > and > https://github.com/apache/mesos/commits/2d19111e4852aed25161e4549ff704 > f9d4c2f37b/src/cli_ne

Agent reconfiguration without losing tasks

2017-12-06 Thread Benno Evers
pply https://reviews.apache.org/r/64384/ and share the feedback about any issues it causes (or doesn't cause). Best regards, -- Benno Evers Software Engineer, Mesosphere

Boost version upgrade

2018-01-09 Thread Benno Evers
m the community about this? Best regards, -- Benno Evers Software Engineer, Mesosphere

Re: Replacing ad hoc virtualenvs for testing and linting with tox

2018-01-11 Thread Benno Evers
>>>> The biggest concern here would probably the change in dependencies, >>>> since >>>> it may seem like we're adding an additional dependency to mesos. However >>>> since virtualenv is a dependency of tox, we will not break any existing >>>> dependencies, as requiring tox will automatically require virtualenv. >>>> Otherwise I don't really see any downside in making the switch. >>>> >>>> Please let me know what you think! >>>> >>>> Eric >>>> >>> >>> > > > -- > Armand Grillet > Software Engineer, Mesosphere > -- Benno Evers Software Engineer, Mesosphere

Re: Replacing ad hoc virtualenvs for testing and linting with tox

2018-01-17 Thread Benno Evers
n > bindings, at least initially. The tests that I aim to run with tox are > mostly CLI-related. In the long term though, it may be worth > considering > using tox to perform all python-related build/test tasks. > > Eric > > On Thu, Jan 11, 2018 at 6:3

Re: Boost version upgrade

2018-02-02 Thread Benno Evers
Just a quick follow-up, the version was bumped to 1.65 in 469363d. On Tue, Jan 9, 2018 at 6:51 PM, Vinod Kone wrote: > If it’s backwards compatible then let’s go for it. > > @vinodkone > > > On Jan 9, 2018, at 9:21 AM, Benno Evers wrote: > > > > Hi folks, >

Re: Follow up on providing `--reconfiguration_policy=any` in future versions

2018-02-19 Thread Benno Evers
tr53SrE5U3Q/edit#> > will > really help our operation. > > Do we want to have a follow up meeting to see what's blockers to fully > implement that? > > Thanks. > > > -- > Cheers, > > Zhitao Li > -- Benno Evers Software Engineer, Mesosphere

Re: Follow up on providing `--reconfiguration_policy=any` in future versions

2018-02-22 Thread Benno Evers
cussions? > > On Mon, Feb 19, 2018 at 9:59 AM, Benno Evers > wrote: > > > Hi Zhitao, > > > > great to see that there's interest in this. > > > > The most specific concern that we had at the time was that we were not > > sure about the best way to

Adding a `FLAKY` label to flaky unit tests

2018-03-29 Thread Benno Evers
." What do you think? Best regards, -- Benno Evers Software Engineer, Mesosphere

Re: Adding a `FLAKY` label to flaky unit tests

2018-04-03 Thread Benno Evers
ther reasons tests are DISABLED today? > > On Thu, Mar 29, 2018 at 10:35 AM, Meng Zhu wrote: > > > +1, the advantages are appealing. > > > > Though I am afraid that this will probably reduce the incentive to fix > > flaky tests. > > > > -Meng > >

Re: Follow up on providing `--reconfiguration_policy=any` in future versions

2018-06-18 Thread Benno Evers
n MESOS-8600 about whether we should overwrite > checkpointed data upon permitted changed. > > > On Thu, Feb 22, 2018 at 9:44 AM, Benno Evers > wrote: > > > Hi, > > > > I'm afraid these were mostly informal lunch-time discussions, but I just > > opene

Using jemalloc as default allocator

2018-08-10 Thread Benno Evers
ent - The bundled jemalloc version (5.0.1) has been released as stable for over a year and has not seen any severe bugs - Our own Mesos builds with jemalloc don't show any issues so far What do you think? Best regards, -- Benno Evers Software Engineer, Mesosphere

Re: Using jemalloc as default allocator

2018-08-13 Thread Benno Evers
Ok then, let's not do it for now. Best regards, On Fri, Aug 10, 2018 at 6:10 PM, James Peach wrote: > > > > On Aug 10, 2018, at 8:56 AM, Benno Evers wrote: > > > > Hi guys, > > > > it's quite late in the release cycle, but I've been t

Mesos Flakiness Statistics

2018-10-12 Thread Benno Evers
ail percentage isn't as different between now and then as we might have hoped. Hope this was interesting, and best regards, -- Benno Evers Software Engineer, Mesosphere

Re: Mesos Flakiness Statistics

2018-10-15 Thread Benno Evers
disabled tests, which is better than I expected, but I hope there's good > tracking on getting these un-disabled again: > > $ grep -R DISABLED src/tests | grep -v DISABLED_ON_WINDOWS | grep -v > NestedQuota | grep -v ChildRole | grep -v NestedRoles | grep -v > environment.cpp | w

Re: Welcome Meng Zhu as PMC member and committer!

2018-11-05 Thread Benno Evers
ive in the project for almost a year and has been >> very productive and collaborative. He is now one of the few people of >> understands the allocator code well, as well as the roadmap for this area >> of the project. He has also found and fixed bugs, and helped users in slac

Re: Discussion: Scheduler API for Operation Reconciliation

2019-01-03 Thread Benno Evers
t reconciliation. In other > words, this call plays the role of a trigger of the > operation status updates. This approach is the simplest in terms of the > implementation, > but the trade-off is that the framework needs to live with (1). > > > So far we haven't discussed much about (2) for operation reconciliation, so > let's also briefly talk > about it. Potentially (2) can be addressed by making the agent *actively > push * > *operation statusupdates to the framework when an LRP is resubscribed*, so > the framework won't need to do > periodic operation reconciliation. If we do this in the future, it would > also be more aligned with > proposal II or III. > > So the question again: is it worth the complexity to keep > `RECONCILE_OPERATIONS` > synchronous? I'd like to hear the opinions from the community so we can > drive towards a better > API design! > > Best, > Chun-Hung > -- Benno Evers Software Engineer, Mesosphere

Apache ReviewBoard not accepting new users

2019-03-06 Thread Benno Evers
switching to LDAP logins. Since reviewboard supports only exactly one sign-up backend, so when they enabled LDAP, "classic" signup was disabled in the process. We're still trying to get some background on this decision from the ASF Infra team. Best regards, -- Benno Evers So

Design Doc: Metrics subset access

2019-03-14 Thread Benno Evers
rd to hear your thoughts, ideas, etc. Best regards, -- Benno Evers Software Engineer, Mesosphere

Re: Mesos on ssl

2019-04-05 Thread Benno Evers
> and it just hangs. No logs nothing... > I'm testing this on ubuntu 18.04. > > Any tipps ? > thanks > Jorge > > > Jorge Machado > www.jmachado.me > > > > > > -- Benno Evers Software Engineer, Mesosphere

Preparing for the 1.8.0 Release

2019-04-07 Thread Benno Evers
ueue that were held up by the pending release, feel free to go wild now ;) Best regards, -- Benno Evers Software Engineer, Mesosphere

Re: Bundled glog update from 0.3.3 to 0.4.0

2019-04-11 Thread Benno Evers
> > > > > Hi all, > > > We are intending to update the bundled glog from 0.3.3 to 0.4.0. > > > > > > If you have any objections/concerns, or know about any issues > introduced > > > into glog between 0.3.3 and 0.4.0, please raise them. >

Subject: [VOTE] Release Apache Mesos 1.8.0 (rc1)

2019-04-15 Thread Benno Evers
Hi all, Please vote on releasing the following candidate as Apache Mesos 1.8.0. 1.8.0 includes the following: * Operation feedback for v1 schedulers. * Per-framework minimum allocatable resources. * New CLI subco

Re: Subject: [VOTE] Release Apache Mesos 1.8.0 (rc1)

2019-04-16 Thread Benno Evers
CHANGELOG on master? > > On Mon, Apr 15, 2019 at 2:26 PM Benno Evers wrote: > > > Hi all, > > > > Please vote on releasing the following candidate as Apache Mesos 1.8.0. > > > > > > 1.8.0 includes the following: > > > > > ---

[RESULT][VOTE] Release Apache Mesos 1.8.0 (rc1)

2019-04-18 Thread Benno Evers
Hi all, the vote for Mesos 1.8.0 (rc1) has *not* passed, with one +1 vote and one -1 vote. Since the issue that caused the -1 vote has already been fixed on master and been backported to the release branch, a vote for rc2 will promptly follow. Best regards, Benno

[VOTE] Release Apache Mesos 1.8.0 (rc2)

2019-04-18 Thread Benno Evers
Hi all, Please vote on releasing the following candidate as Apache Mesos 1.8.0. 1.8.0 includes the following: * Greatly reduced allocator cycle time. * Operation feedback for v1 schedulers. * Per-framework minimu

Re: [VOTE] Release Apache Mesos 1.8.0 (rc2)

2019-04-25 Thread Benno Evers
iling: >> [ FAILED ] VolumeGidManagerTest.ROOT_UNPRIVILEGED_USER_SlaveReboot >> [ FAILED ] CniIsolatorTest.VETH_VerifyResourceStatistics >> [ FAILED ] >> DockerVolumeIsolatorTest.ROOT_EmptyCheckpointFileSlaveRecovery >> >> >> On Thu, Apr 18, 20

[VOTE] Release Apache Mesos 1.8.0 (rc3)

2019-04-26 Thread Benno Evers
Hi all, Please vote on releasing the following candidate as Apache Mesos 1.8.0. 1.8.0 includes the following: * Greatly reduced allocator cycle time. * Operation feedback for v1 schedulers. * Per-framework minimu

Re: [VOTE] Release Apache Mesos 1.8.0 (rc3)

2019-04-26 Thread Benno Evers
Addendum: The vote is open until Thursday, May 2nd. On Fri, Apr 26, 2019 at 6:28 PM Benno Evers wrote: > Hi all, > > Please vote on releasing the following candidate as Apache Mesos 1.8.0. > > > 1.8.0 incl

Re: [VOTE] Release Apache Mesos 1.8.0 (rc3)

2019-04-29 Thread Benno Evers
r: 'tensorflow.python.framework.errors_impl.InvalidArgumentError'>, Default > MaxPoolingOp only supports NHWC on device type CPU >[[node tower_0/v/cg/mpool0/MaxPool (defined at > /user/tf-benchmarks-113/scripts/tf_cnn_benchmarks/convnet_builder.py:261) ] > > runnin

[RESULT][VOTE] Release Apache Mesos 1.8.0 (rc3)

2019-05-02 Thread Benno Evers
Hi all, The vote for Mesos 1.8.0 (rc3) has passed with the following votes. +1 (Binding) -- Vinod Kone Alex Rukletsov Meng Zhu +1 (Non-binding) -- Jorge Machado There were no 0 or -1 votes. Please find the release at: https://dist.apache.

Re: On adding a debug endpoint for Mesos containerizer

2019-06-07 Thread Benno Evers
cking pending libprocess futures > [2]. > > > > Please review both of them. > > > > [1] Container debug endpoint: > https://docs.google.com/document/d/1VtlKD6b8a22HzSdaJUeI7cPGuKd01vLwBJT4XfkeUDI > > [2] Tracking libprocess futures: > https://docs.google.com/document/d/1Unu2pe0dRq3Z6XQ5S8lWZm2cU2REjfkUj0xk2ePQ0MY > > -- Benno Evers Software Engineer, Mesosphere

Re: On augmenting TLS configuration options in libprocess

2019-06-19 Thread Benno Evers
e using OpenSSL calls. > > Please review. > > [1] > https://docs.google.com/document/d/1O3q7UOXVGNw81xOkRNFPzrtbC__D-N_D_mwV6D--y0k/edit > -- Benno Evers Software Engineer, Mesosphere

[VOTE] Release Apache Mesos 1.8.1 (rc1)

2019-07-10 Thread Benno Evers
Hi all, Please vote on releasing the following candidate as Apache Mesos 1.8.1. We had a lot of good bugfixes that only very slightly missed the 1.8.0 release, some just by a few days. By now, the 1.8.x branch seems to have calmed down a bit, so it seems to be a good opportunity to do an 1.8.1 re

[RESULT][VOTE] Release Apache Mesos 1.8.1 (rc1)

2019-07-18 Thread Benno Evers
Hi all, The vote for Mesos 1.8.1 (rc1) has passed with the following votes. +1 (Binding) -- Vinod Kone Meng Zhu Greg Mann There were no 0 or -1 votes. Please find the release at: https://dist.apache.org/repos/dist/release/mesos/1.8.1 It is recommended to use a mirro