RE: Looking for Shepherd for MESOS-4033 (commit hook for non-ascii characters)

[Yong Tang]

Hi Vinod,

Thanks for the help. I updated the mesos-style.py and added the non-ascii check 
there.

Please let me know if there is anything else that needs to do to move forward 
this issue.


Thanks
Yong

> From: vinodk...@apache.org
> Date: Fri, 1 Apr 2016 15:47:29 -0700
> Subject: Re: Looking for Shepherd for MESOS-4033 (commit hook for non-ascii 
> characters)
> To: dev@mesos.apache.org
> 
> @AleR do you want to be a shepherd for this, since you originally filed
> this ticket?
> 
> @Yong: Just took a quick look at the review. It's unfortunate that we need
> a whole new script for checking non-ascii characters. Can we update
> mesos-style.py to catch this?
> 
> On Fri, Apr 1, 2016 at 7:36 AM, Yong Tang 
> wrote:
> 
> > Ping again to find a shepherd for MESOS-4033. A review request has been
> > created with good discussions from many reviewers:
> >
> > https://reviews.apache.org/r/45033/
> >
> > It would be really good if a shepherd could provide some guidance so that
> > this ticket could move forward.
> >
> > Thanks
> > Yong
> >
> > > Date: Sat, 26 Mar 2016 08:13:28 +0800
> > > Subject: Re: Looking for Shepherd for MESOS-4033 (commit hook for
> > non-ascii characters)
> > > From: xia...@gmail.com
> > > To: dev@mesos.apache.org
> > >
> > > +1
> > >
> > > 2016-03-25 23:26 GMT+08:00 Yong Tang :
> > >
> > > > Hi
> > > >
> > > > Just bump the email to look for shepherd for MESOS-4033.
> > > >
> > > > This issue (commit hook for non ascii characters) has already been
> > fairly
> > > > discussed on the review board. Thanks Deshi, Neil, and haosdent for the
> > > > great inputs.
> > > >
> > > > It really would be nice if there is anyone that could shepherd so that
> > the
> > > > issue could move forward.
> > > >
> > > > Thanks
> > > > Yong
> > > >
> > > >
> > > > > From: yong.tang.git...@outlook.com
> > > > > To: dev@mesos.apache.org
> > > > > Subject: Looking for Shepherd for MESOS-4033 (commit hook for
> > non-ascii
> > > > characters)
> > > > > Date: Tue, 22 Mar 2016 08:31:40 -0700
> > > > >
> > > > > Hi All
> > > > >
> > > > > Can anyone help shepherd MESOS-4033 - Add a commit hook for non-ascii
> > > > characters?
> > > > >
> > > > > https://issues.apache.org/jira/browse/MESOS-4033
> > > > >
> > > > > This issue is about adding a commit hook to check for non-ascii
> > > > characters. The issue has been accepted sometime ago.
> > > > >
> > > > > Thanks a lot for the help
> > > > > Yong
> > > > >
> > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Deshi Xiao
> > > Twitter: xds2000
> > > E-mail: xiaods(AT)gmail.com
> >
> >
  

Design doc: Securing Mesos Containers Using Capabilities

[Jojy Varghese]

Hi all,
As Mesos is adding more features to its Unified Containerizer[1], 
ability to run Mesos containers in a secure environment has been one of the top 
priorities. As an initial step, we could use Posix capabilities[2] to create a 
secure sandbox to run the Mesos containers.
Please review the design doc that proposes how we can secure Mesos 
containers using capabilities:

https://docs.google.com/document/d/1YiTift8TQla2vq3upQr7K-riQ_pQ-FKOCOsysQJROGc/edit?usp=sharing
 



Thanks,
Jojy


[1] http://mesos.apache.org/documentation/latest/mesos-containerizer/ 

[2] 
https://www.kernel.org/pub/linux/libs/security/linux-privs/kernel-2.2/capfaq-0.2.txt
 

Re: Ordering guarantee of future.onAny callbacks

[haosdent]

I fill a tech-debt ticket to record this in
https://issues.apache.org/jira/browse/MESOS-5079

On Wed, Mar 30, 2016 at 5:25 PM, Joris Van Remoortere 
wrote:

> I think the API of future intuitively suggests parts of the chain will
> execute in a specific order. The type system even enforces some of these
> IIUC.
>
> For example:
> func().then([](T) -> X {...}).repair([](Future) -> X {...}).then([](X)
> -> Y {...}).onAny([](Future){...})
>
> As you can see, the .onAny(Future) can't just start executing on
> the T returned
> by func().
>
> I think it would be confusing and error prone to remember that sibling `.
> onAny` pairs could execute in arbitrary order, but members along the chain
> such as `.then` and `.repair` could not.
>
> I'm not sure the benefits of executing multiple `.onAny` calls in parallel
> in the future (if we ever need to) outweigh the inconsistency in the API
> regarding ordered execution of certain chains (At least as the API is
> currently designed).
>
> —
> *Joris Van Remoortere*
> Mesosphere
>
> On Wed, Mar 30, 2016 at 12:25 AM, Benjamin Mahler 
> wrote:
>
> > I don't believe we have made this assumption often, and I'm not convinced
> > it's a problem in Future.
> >
> > Why would one make such an assumption in the first place? It seems
> brittle,
> > especially when callbacks are executed in deferred contexts that Future
> is
> > unaware of. It also limits our flexibility within Future if we add these
> > ordering semantics.
> >
> > On Tue, Mar 29, 2016 at 2:41 AM, Joris Van Remoortere <
> jo...@mesosphere.io
> > >
> > wrote:
> >
> > > Rather than saying we shouldn't make this assumption (which I assume
> has
> > > already been made in places, and will likely be made again), can we fix
> > the
> > > problem?
> > > There's a `TODO` already suggesting running the callback in a separate
> > > execution environment.
> > > We could also synchronize with thread setting the future to true.
> > >
> > > I'd rather discuss and fix the problem, than try to enforce avoiding
> this
> > > case.
> > >
> > > Thoughts?
> > >
> > > —
> > > *Joris Van Remoortere*
> > > Mesosphere
> > >
> > > On Tue, Mar 29, 2016 at 2:50 AM, Jie Yu  wrote:
> > >
> > > > Hi,
> > > >
> > > > While digging a bug reported in, I realized an assumption we
> shouldn't
> > > make
> > > > in our code.
> > > > https://issues.apache.org/jira/browse/MESOS-5023
> > > >
> > > > Say you have the following code:
> > > >
> > > > void some_func()
> > > > {
> > > >   future
> > > > .onAny(callback_A)
> > > > .onAny(callback_B);
> > > > }
> > > >
> > > > Question: will callback_A already be executed before callback_B?
> > > >
> > > > The answer is NO. We should never assume that. Under the following
> > > > interleaving, callback_B can be invoked first:
> > > >
> > > > Thread-1   Thread-2
> > > >
> > > > onAny(callback_A) {
> > > >   onAnyCallbacks.push_back(callback_A);
> > > > }
> > > >   set() {
> > > > lock()
> > > > if (state ==
> > > > PENDING) {
> > > >   state =
> > READY;
> > > >   result =
> > true;
> > > > }
> > > > unlock();
> > > >
> > > > onAny(callback_B) {
> > > >   lock()
> > > >   if (state != PENDING) {
> > > > run = true
> > > >   }
> > > >   unlock()
> > > >
> > > >   if (run) {
> > > > callback_B()
> > > >   }
> > > >
> > > >  if (result)
> {
> > > >
> > > > internal::run(data->onAnyCallbacks,
> > > > *this);
> > > >  }
> > > >
> > > > - Jie
> > > >
> > >
> >
>



-- 
Best Regards,
Haosdent Huang

Re: Correctly handle MOUNT type disk

[Jie Yu]

Zhitao,

Typically, you need to enable this feature in a two phase manner:

Phase 1: upgrade Mesos and scheduler, but not use this feature yet.
Phase 2: turn on this feature

To be safe, IMO i think the scheduler driver (or the scheduler itself if
using HTTP api) should check for unknown tag number in the protobuf message
and decline the offer if unknown tag is found.

- Jie

On Fri, Apr 1, 2016 at 10:14 AM, Zhitao Li  wrote:

> After multiple disk support is introduced, I tried to perform some initial
> testing of the feature. In the process, I noticed that some existing
> frameworks (e.g. Apache Aurora) is not safe to be used.
>
> What happens is because the framework has not been upgraded to the proper
> version binding, the scheduler cannot see the type/mount part yet, and
> would incorrectly thought this just another slice of scalar disk resource.
> What's worse, in the process of handling static reserved resources, the
> scheduler usually merges multiple scalar resources.
>
> The net result could be that the scheduler try to tap onto these MOUNT
> disks and launch task with such resource, but Mesos master would consider
> the resource as invalid (because the machine doesn't really have disk
> resources usable for sandbox), the launch task action triggers an error.
>
> I'm not exactly sure whether this is a problem on Mesos side or framework
> side, but until the framework is upgraded and correctly identifies new
> protobuf fields, are there better ways to handle this?
>
> Thanks.
>
> --
> Cheers,
>
> Zhitao Li
>

RE: Looking for Shepherd for MESOS-4033 (commit hook for non-ascii characters)

[Yong Tang]

Ping again to find a shepherd for MESOS-4033. A review request has been created 
with good discussions from many reviewers:

https://reviews.apache.org/r/45033/

It would be really good if a shepherd could provide some guidance so that this 
ticket could move forward.

Thanks
Yong

> Date: Sat, 26 Mar 2016 08:13:28 +0800
> Subject: Re: Looking for Shepherd for MESOS-4033 (commit hook for non-ascii 
> characters)
> From: xia...@gmail.com
> To: dev@mesos.apache.org
> 
> +1
> 
> 2016-03-25 23:26 GMT+08:00 Yong Tang :
> 
> > Hi
> >
> > Just bump the email to look for shepherd for MESOS-4033.
> >
> > This issue (commit hook for non ascii characters) has already been fairly
> > discussed on the review board. Thanks Deshi, Neil, and haosdent for the
> > great inputs.
> >
> > It really would be nice if there is anyone that could shepherd so that the
> > issue could move forward.
> >
> > Thanks
> > Yong
> >
> >
> > > From: yong.tang.git...@outlook.com
> > > To: dev@mesos.apache.org
> > > Subject: Looking for Shepherd for MESOS-4033 (commit hook for non-ascii
> > characters)
> > > Date: Tue, 22 Mar 2016 08:31:40 -0700
> > >
> > > Hi All
> > >
> > > Can anyone help shepherd MESOS-4033 - Add a commit hook for non-ascii
> > characters?
> > >
> > > https://issues.apache.org/jira/browse/MESOS-4033
> > >
> > > This issue is about adding a commit hook to check for non-ascii
> > characters. The issue has been accepted sometime ago.
> > >
> > > Thanks a lot for the help
> > > Yong
> > >
> >
> >
> 
> 
> 
> -- 
> Deshi Xiao
> Twitter: xds2000
> E-mail: xiaods(AT)gmail.com
  

On launching command tasks

[Alex Rukletsov]

When launching a command task without wrapping it in `/bin/sh -c` (i.e.
CommandInfo.shell=false), Mesos expects the first argument to be the same
as the command itself [1]. Though this is similar to how UNIX exec* calls
operate, it can be unclear to a user. Moreover, we do not validate this on
the master side, but rather let the command executor crash with a "bad
address" error. Docker, for example, requires the command only once in
their entrypoint specification [2].

My suggestion is to change the command executor so that it ensures that the
first argument is always the command itself.

Alternatively, if we prefer to keep the current behaviour, I would propose
to adjust the documentation to be more explicit and introduce a validation
check on the master.

[1] Example snippet in C++

   commandInfo->set_value(command);

   commandInfo->add_arguments()->assign(command);


[2] https://docs.docker.com/engine/reference/builder/#entrypoint