Re: Blog posts for 0.28.1, 0.28.2 releases?

2016-06-10 Thread Jörg Schad
In the same area:
Should we have blogpost for the MesosCon EU/ASIA CFP similar as we had for
MesosCon NA?

On Fri, Jun 10, 2016 at 11:50 AM, Neil Conway  wrote:

> Folks,
>
> It seems like https://mesos.apache.org/blog/ doesn't have blog posts
> for the Mesos 0.28.1 or 0.28.2 releases. We generally try to have a
> blog post for each release, right?
>
> Neil
>


Re: Master configuration in the registry

2016-06-10 Thread Neil Conway
Makes sense: arguably you could say that "quota" and "weights" are
part of the master's (mutable) "state", not its "configuration", which
is largely immutable.

Another distinction is that some configuration flags control behavior
that doesn't need to be consistent between master replicas (e.g.,
"--ip", "--port", "--advertise-ip", "--advertise-port", "--hostname",
"--hostname-lookup", "--quiet", "--log_dir", "--modules_dir",
"--work_dir", etc).

Neil

On Fri, Jun 10, 2016 at 3:52 AM, Benjamin Mahler  wrote:
> I'm curious to hear thoughts on the distinction between using flags and
> persisting in the registry for master configuration. This topic had come up
> in a discussion and our current choices are intuitive but the criteria were
> not immediately obvious.
>
> Two cases seem interesting to me:
>
> (1) Quota.
> (2) Weights.
>
> These are configuration, but we persist them in the registry. Why is that?
>
> My intuition is that they reflect the organizational aspects of the
> workloads that are running and so we expect administrators and (most
> importantly!!) tooling to be view and modify these over time.
>
> Timeouts, work directories, etc, on the other hand, are rarely modified and
> require initial values. There are also sane defaults for these that will
> work for most users.
>
> Thought this might be helpful for others that may wonder about this. Let me
> know if there are any other important criteria that I've missed.
>
> Ben


Re: WebUI authentication in 1.0.0-rc1

2016-06-10 Thread Evers Benno
Sure,

it looks like this, not very imaginative. There is currently no
authorization on the agents.

{
"permissive": false,

[...] // Here is the previous ACL with actions "run_tasks" and
"register_frameworks"

"get_endpoints": [
{
"principals": {"type": "ANY"},
"paths": {"type": "ANY"}
}
],

"view_frameworks": [
 {
 "principals": {"type": "ANY"},
 "users": {"type": "ANY"}
 }
],

"view_tasks": [
 {
 "principals": {"type": "ANY"},
 "users": {"type": "ANY"}
 }
],

"view_executors": [
 {
 "principals": {"type": "ANY"},
 "users": {"type": "ANY"}
 }
],

"access_sandboxes": [
 {
 "principals": {"type": "ANY"},
 "users": {"type": "ANY"}
 }
],

"access_mesos_logs": [
 {
 "principals": {"type": "ANY"},
 "logs": {"type": "ANY"}
 }
]
}



On 10.06.2016 00:17, Greg Mann wrote:
> Benno,
> Would you mind providing more information on the ACL definitions that you
> used to gain full access to the web UI? I'm working on some more
> documentation for this. Also, did you have authorization enabled on the
> agents as well?
> 
> Cheers,
> Greg
> 
> On Wed, Jun 8, 2016 at 7:43 AM, Neil Conway  wrote:
> 
>> On Wed, Jun 8, 2016 at 4:27 PM, Alexander Rojas 
>> wrote:
>>> I think we should also think more thoroughly about the expected behaviour
>>> when we introduce new authorizable actions (and we most certainly will).
>>> Since things may break particularly if users set the `permissive` ACL
>> field
>>> to false.
>>>
>>> Perhaps initially, if no ACL is given for the new action we print a
>> warning
>>> message and behave as if the field had an ACL such as
>>>
>>> ```
>>> {
>>>   "principals": {"type": "ANY"}
>>>   "action":{"type": "ANY"}
>>> }
>>> ```
>>
>> An ACL configuration that omits any rules for a particular action is
>> not an invalid way to configure the system. e.g., suppose we added the
>> "/teardown" endpoint in Mesos 1.1, along with the
>> "teardown_frameworks" ACL. A perfectly reasonable way to configure the
>> behavior "no one should be allowed to use the /teardown endpoint" is
>> an ACL configuration that has "permissive: false" and doesn't
>> otherwise mention "teardown_frameworks".
>>
>> The situation here is a little unusual, because we're introducing ACLs
>> for behavior that was previously not covered by the authorization
>> system, rather than new functionality. But overall, I think the
>> situation can be addressed by documenting the new behavior
>> *prominently* in the release notes / upgrade docs -- anyone upgrading
>> to a non-patch release should be reading that document anyway, and the
>> required changes will usually be straightforward.
>>
>> Neil
>>
> 


Re: Welcome Anand and Joseph as new committers!

2016-06-10 Thread Klaus Ma
Congratulations :).


Da (Klaus), Ma (马达), PMP®| Advisory Software Engineer
Platform DCOS Development & Support, STG, IBM GCG
+86-10-8245 4084 | mad...@cn.ibm.com | http://k82.me

From: Guangya Liu 
Sent: Thursday, June 9, 2016 11:55:03 PM
To: dev
Subject: Re: Welcome Anand and Joseph as new committers!

Congrats to both, well deserved!!

On Fri, Jun 10, 2016 at 5:00 AM, Vinod Kone  wrote:

> Hi folks,
>
> I'm happy to announce that the PMC has voted in *Anand Mazumdar *and
> *Joseph
> Wu* as committers and members of PMC for the Apache Mesos project.
>
> A little about the new committers.
>
> Anand has been working on the Apache Mesos project for about an year now
> and has shown incredible commitment to the project and the community. His
> significant contributions include implementing scheduler HTTP API,
> designing and implementing executor HTTP API and helping out with the
> operator HTTP API. His formal committer checklist is here
> <
> https://docs.google.com/document/d/1DGRO-z-0JKS1dIxcrzXD8QznEv-auSqmIP3-aiwhtFI/edit?usp=sharing
> >
> .
>
> Joseph's passion and dedication to the community is phenomenal. His
> significant contributions include Maintenance Primitives and Container
> Logger Modules. He has also been a valuable contributor and reviewer to our
> testing infrastructure and the Windows work. His formal committer checklist
> is here
> <
> https://docs.google.com/document/d/1o7qLQJQ7TZCaf49gSNc6SSl29qAFagYH2STDfhHDDPw/edit?usp=sharing
> >
> .
>
> Please join me in congratulating them on their new roles and especially
> responsibilities :)
>
> On behalf of the PMC,
> Vinod
>


Re: Welcome Anand and Joseph as new committers!

2016-06-10 Thread Justin Ryan
Congrats! Thanks for your contributions!

On 6/9/16, 2:00 PM, "Vinod Kone"  wrote:

>Hi folks,
>
>
>I'm happy to announce that the PMC has voted in Anand Mazumdar and
>Joseph Wu as committers and members of PMC for the Apache Mesos project.
>
>
>A little about the new committers.
>
>
>Anand has been working on the Apache Mesos project for about an year now and 
>has shown incredible commitment to the project and the community. His 
>significant contributions include implementing scheduler HTTP API, designing 
>and implementing executor HTTP
> API and helping out with the operator HTTP API. His formal committer 
> checklist is
>
>here 
>.
>
>
>Joseph's passion and dedication to the community is phenomenal. His 
>significant contributions include Maintenance Primitives and Container
> Logger Modules. He has also been a valuable contributor and reviewer to our 
> testing infrastructure and the Windows work. His formal committer checklist is
>
>here 
>.
>
>
>Please join me in congratulating them on their new roles and especially 
>responsibilities :)
>
>
>On behalf of the PMC,
>Vinod
>



P Please consider the environment before printing this e-mail

The information in this electronic mail message is the sender's confidential 
business and may be legally privileged. It is intended solely for the 
addressee(s). Access to this internet electronic mail message by anyone else is 
unauthorized. If you are not the intended recipient, any disclosure, copying, 
distribution or any action taken or omitted to be taken in reliance on it is 
prohibited and may be unlawful. The sender believes that this E-mail and any 
attachments were free of any virus, worm, Trojan horse, and/or malicious code 
when sent. This message and its attachments could have been infected during 
transmission. By reading the message and opening any attachments, the recipient 
accepts full responsibility for taking protective and remedial action about 
viruses and other defects. The sender's employer is not liable for any loss or 
damage arising in any way.


Re: [Tech-debt] Introduce regex into Mesos

2016-06-10 Thread Kevin Klues
I've run into compiler errors using simple regex stuff from the
standard library on our supported version of gcc.

On Thu, Jun 9, 2016 at 7:30 PM, Klaus Ma  wrote:
> Hi team,
>
>
> We're discussing to introduce regex into Mesos when investigating 
> MESOS-4627; so I'd like to 
> ask whether anyone has experience on regex after C++11? for example, 
> supported compiler, compatibility, performance and so on :).
>
>
> 
>
> Da (Klaus), Ma (??), PMP®| Advisory Software Engineer
> Platform DCOS Development & Support, STG, IBM GCG
> +86-10-8245 4084 | mad...@cn.ibm.com | http://k82.me
>
> 



-- 
~Kevin


Re: [Tech-debt] Introduce regex into Mesos

2016-06-10 Thread Kevin Klues
By compiler errors, I mean "internal compiler errors"

On Fri, Jun 10, 2016 at 11:38 AM, Kevin Klues  wrote:
> I've run into compiler errors using simple regex stuff from the
> standard library on our supported version of gcc.
>
> On Thu, Jun 9, 2016 at 7:30 PM, Klaus Ma  wrote:
>> Hi team,
>>
>>
>> We're discussing to introduce regex into Mesos when investigating 
>> MESOS-4627; so I'd like to 
>> ask whether anyone has experience on regex after C++11? for example, 
>> supported compiler, compatibility, performance and so on :).
>>
>>
>> 
>>
>> Da (Klaus), Ma (??), PMP®| Advisory Software Engineer
>> Platform DCOS Development & Support, STG, IBM GCG
>> +86-10-8245 4084 | mad...@cn.ibm.com | http://k82.me
>>
>> 
>
>
>
> --
> ~Kevin



-- 
~Kevin


failed to start mesos-slave

2016-06-10 Thread Olivier Sallou
Hi,
I upgraded docker on one of my mesos slaves (v0.22)

Now it fails to start with error:

Failed to create a containerizer: Could not create DockerContainerizer:
Insufficient version of Docker! Please upgrade to >= 1.0.0

Though docker is 1.11:

docker -v
Docker version 1.11.2, build b9f10c9

Any idea ?

Thanks

Olivier

-- 
Olivier Sallou
IRISA / University of Rennes 1
Campus de Beaulieu, 35000 RENNES - FRANCE
Tel: 02.99.84.71.95

gpg key id: 4096R/326D8438  (keyring.debian.org)
Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438



Re: Welcome Anand and Joseph as new committers!

2016-06-10 Thread Jörg Schad
Congrats!
Great news!

On Fri, Jun 10, 2016 at 7:47 AM, Abhishek Dasgupta <
a10gu...@linux.vnet.ibm.com> wrote:

> Congrats Anand and Joseph.
>
> Happy to know it.
>
> On শুক্রবার 10 জুন 2016 02:30 পূর্বাহ্ণ, Vinod Kone wrote:
>
>> Hi folks,
>>
>> I'm happy to announce that the PMC has voted in *Anand Mazumdar *and
>> *Joseph
>> Wu* as committers and members of PMC for the Apache Mesos project.
>>
>> A little about the new committers.
>>
>> Anand has been working on the Apache Mesos project for about an year now
>> and has shown incredible commitment to the project and the community. His
>> significant contributions include implementing scheduler HTTP API,
>> designing and implementing executor HTTP API and helping out with the
>> operator HTTP API. His formal committer checklist is here
>> <
>> https://docs.google.com/document/d/1DGRO-z-0JKS1dIxcrzXD8QznEv-auSqmIP3-aiwhtFI/edit?usp=sharing
>> >
>> .
>>
>> Joseph's passion and dedication to the community is phenomenal. His
>> significant contributions include Maintenance Primitives and Container
>> Logger Modules. He has also been a valuable contributor and reviewer to
>> our
>> testing infrastructure and the Windows work. His formal committer
>> checklist
>> is here
>> <
>> https://docs.google.com/document/d/1o7qLQJQ7TZCaf49gSNc6SSl29qAFagYH2STDfhHDDPw/edit?usp=sharing
>> >
>> .
>>
>> Please join me in congratulating them on their new roles and especially
>> responsibilities :)
>>
>> On behalf of the PMC,
>> Vinod
>>
>>
>


Re: failed to start mesos-slave

2016-06-10 Thread Olivier Sallou


On 06/10/2016 11:43 AM, Neil Conway wrote:
> Hi Olivier,
>
> You might be running into
> https://issues.apache.org/jira/browse/MESOS-2986 . Note that Mesos
> 0.22 is quite old and is no longer supported.
certainly but upgrading mesos in production is not a daily task

upgrading to 0.22.2-0.2.62 seems to fix the issue. Thanks

>
> Neil
>
>
> On Fri, Jun 10, 2016 at 11:37 AM, Olivier Sallou
>  wrote:
>> Hi,
>> I upgraded docker on one of my mesos slaves (v0.22)
>>
>> Now it fails to start with error:
>>
>> Failed to create a containerizer: Could not create DockerContainerizer:
>> Insufficient version of Docker! Please upgrade to >= 1.0.0
>>
>> Though docker is 1.11:
>>
>> docker -v
>> Docker version 1.11.2, build b9f10c9
>>
>> Any idea ?
>>
>> Thanks
>>
>> Olivier
>>
>> --
>> Olivier Sallou
>> IRISA / University of Rennes 1
>> Campus de Beaulieu, 35000 RENNES - FRANCE
>> Tel: 02.99.84.71.95
>>
>> gpg key id: 4096R/326D8438  (keyring.debian.org)
>> Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438
>>

-- 
Olivier Sallou
IRISA / University of Rennes 1
Campus de Beaulieu, 35000 RENNES - FRANCE
Tel: 02.99.84.71.95

gpg key id: 4096R/326D8438  (keyring.debian.org)
Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438



Blog posts for 0.28.1, 0.28.2 releases?

2016-06-10 Thread Neil Conway
Folks,

It seems like https://mesos.apache.org/blog/ doesn't have blog posts
for the Mesos 0.28.1 or 0.28.2 releases. We generally try to have a
blog post for each release, right?

Neil


Re: [Tech-debt] Introduce regex into Mesos

2016-06-10 Thread Klaus Ma
As Joseph said, regex works well in llvm 7.3 & gcc 5.3, but does not work in 
gcc 4.8.4; also try "" in gc 4.8.4, but "ld" failed :(.




Da (Klaus), Ma (??), PMP(r)| Advisory Software Engineer
Platform DCOS Development & Support, STG, IBM GCG
+86-10-8245 4084 | mad...@cn.ibm.com | http://k82.me



From: Joseph Wu 
Sent: Friday, June 10, 2016 8:15:51 PM
To: dev
Subject: Re: [Tech-debt] Introduce regex into Mesos

Same here.

Mesos currently requires GCC 4.8.1+.  Regex support was implemented in GCC
4.9.0, see [1].

[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53631

On Fri, Jun 10, 2016 at 11:39 AM, Kevin Klues  wrote:

> By compiler errors, I mean "internal compiler errors"
>
> On Fri, Jun 10, 2016 at 11:38 AM, Kevin Klues  wrote:
> > I've run into compiler errors using simple regex stuff from the
> > standard library on our supported version of gcc.
> >
> > On Thu, Jun 9, 2016 at 7:30 PM, Klaus Ma  wrote:
> >> Hi team,
> >>
> >>
> >> We're discussing to introduce regex into Mesos when investigating
> MESOS-4627; so I'd like
> to ask whether anyone has experience on regex after C++11? for example,
> supported compiler, compatibility, performance and so on :).
> >>
> >>
> >> 
> >>
> >> Da (Klaus), Ma (??), PMP(r)| Advisory Software Engineer
> >> Platform DCOS Development & Support, STG, IBM GCG
> >> +86-10-8245 4084 | mad...@cn.ibm.com | http://k82.me
> >>
> >> 
> >
> >
> >
> > --
> > ~Kevin
>
>
>
> --
> ~Kevin
>


Re: Welcome Anand and Joseph as new committers!

2016-06-10 Thread Benjamin Mahler
Welcome Anand and Joseph, thanks for all of your contributions!

Looking forward to seeing your ongoing positive influences on the community
and the project, let's build great software!

On Thu, Jun 9, 2016 at 2:00 PM, Vinod Kone  wrote:

> Hi folks,
>
> I'm happy to announce that the PMC has voted in *Anand Mazumdar *and
> *Joseph
> Wu* as committers and members of PMC for the Apache Mesos project.
>
> A little about the new committers.
>
> Anand has been working on the Apache Mesos project for about an year now
> and has shown incredible commitment to the project and the community. His
> significant contributions include implementing scheduler HTTP API,
> designing and implementing executor HTTP API and helping out with the
> operator HTTP API. His formal committer checklist is here
> <
> https://docs.google.com/document/d/1DGRO-z-0JKS1dIxcrzXD8QznEv-auSqmIP3-aiwhtFI/edit?usp=sharing
> >
> .
>
> Joseph's passion and dedication to the community is phenomenal. His
> significant contributions include Maintenance Primitives and Container
> Logger Modules. He has also been a valuable contributor and reviewer to our
> testing infrastructure and the Windows work. His formal committer checklist
> is here
> <
> https://docs.google.com/document/d/1o7qLQJQ7TZCaf49gSNc6SSl29qAFagYH2STDfhHDDPw/edit?usp=sharing
> >
> .
>
> Please join me in congratulating them on their new roles and especially
> responsibilities :)
>
> On behalf of the PMC,
> Vinod
>


Re: [Tech-debt] Introduce regex into Mesos

2016-06-10 Thread Joseph Wu
Same here.

Mesos currently requires GCC 4.8.1+.  Regex support was implemented in GCC
4.9.0, see [1].

[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53631

On Fri, Jun 10, 2016 at 11:39 AM, Kevin Klues  wrote:

> By compiler errors, I mean "internal compiler errors"
>
> On Fri, Jun 10, 2016 at 11:38 AM, Kevin Klues  wrote:
> > I've run into compiler errors using simple regex stuff from the
> > standard library on our supported version of gcc.
> >
> > On Thu, Jun 9, 2016 at 7:30 PM, Klaus Ma  wrote:
> >> Hi team,
> >>
> >>
> >> We're discussing to introduce regex into Mesos when investigating
> MESOS-4627; so I'd like
> to ask whether anyone has experience on regex after C++11? for example,
> supported compiler, compatibility, performance and so on :).
> >>
> >>
> >> 
> >>
> >> Da (Klaus), Ma (??), PMP®| Advisory Software Engineer
> >> Platform DCOS Development & Support, STG, IBM GCG
> >> +86-10-8245 4084 | mad...@cn.ibm.com | http://k82.me
> >>
> >> 
> >
> >
> >
> > --
> > ~Kevin
>
>
>
> --
> ~Kevin
>


Re: mesos/kafka issues (reviving old thread)

2016-06-10 Thread Steve Niemitz
Just to chime in, I've been running the 0.9.5.1 scheduler as a task on
another slave without issues. (Aurora runs the kafka-mesos scheduler in my
case).

On Thu, Jun 9, 2016 at 2:50 PM, Vinit Mahedia 
wrote:

> Justin,
>
> When you say "working" - does it mean kafka-scheduler still has to be on
> the same box as mesos-master? or you
> have it working without that constraint?
>
> On Wed, Jun 8, 2016 at 6:07 PM, Justin Ryan  wrote:
>
> > inline
> >
> > On 6/8/16, 4:06 PM, "Justin Ryan"  wrote:
> >
> > >FYI, when I updated to the latest kafka-mesos (0.5.1.0) this problem
> went
> > away.  FWIW, I’m actually using a branch which updates kafka to 0.10.0.0
> as
> > well:
> > >
> >
> > Correction: 0.9.5.1 (current git master)
> >
> > > PR for kafka 0.10.0.0 (tests still fail, someone else did the bulk of
> > porting but didn’t PR it) : https://github.com/mesos/kafka/pull/220
> > >   ( ./gradlew jar –x test gets a successful build )
> > >
> > > Issue for the problem discussed in this thread:
> > https://github.com/mesos/kafka/issues/199
> > >
> > >Cheers!
> > >
> >
> >
> > 
> >
> > P Please consider the environment before printing this e-mail
> >
> > The information in this electronic mail message is the sender's
> > confidential business and may be legally privileged. It is intended
> solely
> > for the addressee(s). Access to this internet electronic mail message by
> > anyone else is unauthorized. If you are not the intended recipient, any
> > disclosure, copying, distribution or any action taken or omitted to be
> > taken in reliance on it is prohibited and may be unlawful. The sender
> > believes that this E-mail and any attachments were free of any virus,
> worm,
> > Trojan horse, and/or malicious code when sent. This message and its
> > attachments could have been infected during transmission. By reading the
> > message and opening any attachments, the recipient accepts full
> > responsibility for taking protective and remedial action about viruses
> and
> > other defects. The sender's employer is not liable for any loss or damage
> > arising in any way.
> >
>
>
>
> --
> ~Vinit
>


Re: Blog posts for 0.28.1, 0.28.2 releases?

2016-06-10 Thread Jie Yu
Apologies! Will write a blog post for 0.28.2.

- Jie

On Fri, Jun 10, 2016 at 2:50 AM, Neil Conway  wrote:

> Folks,
>
> It seems like https://mesos.apache.org/blog/ doesn't have blog posts
> for the Mesos 0.28.1 or 0.28.2 releases. We generally try to have a
> blog post for each release, right?
>
> Neil
>


Re: Welcome Anand and Joseph as new committers!

2016-06-10 Thread Brenden Matthews
Welcome and congrats!

On Fri, Jun 10, 2016 at 2:11 PM, Benjamin Mahler  wrote:

> Welcome Anand and Joseph, thanks for all of your contributions!
>
> Looking forward to seeing your ongoing positive influences on the
> community and the project, let's build great software!
>
> On Thu, Jun 9, 2016 at 2:00 PM, Vinod Kone  wrote:
>
>> Hi folks,
>>
>> I'm happy to announce that the PMC has voted in *Anand Mazumdar *and
>> *Joseph
>> Wu* as committers and members of PMC for the Apache Mesos project.
>>
>> A little about the new committers.
>>
>> Anand has been working on the Apache Mesos project for about an year now
>> and has shown incredible commitment to the project and the community. His
>> significant contributions include implementing scheduler HTTP API,
>> designing and implementing executor HTTP API and helping out with the
>> operator HTTP API. His formal committer checklist is here
>> <
>> https://docs.google.com/document/d/1DGRO-z-0JKS1dIxcrzXD8QznEv-auSqmIP3-aiwhtFI/edit?usp=sharing
>> >
>> .
>>
>> Joseph's passion and dedication to the community is phenomenal. His
>> significant contributions include Maintenance Primitives and Container
>> Logger Modules. He has also been a valuable contributor and reviewer to
>> our
>> testing infrastructure and the Windows work. His formal committer
>> checklist
>> is here
>> <
>> https://docs.google.com/document/d/1o7qLQJQ7TZCaf49gSNc6SSl29qAFagYH2STDfhHDDPw/edit?usp=sharing
>> >
>> .
>>
>> Please join me in congratulating them on their new roles and especially
>> responsibilities :)
>>
>> On behalf of the PMC,
>> Vinod
>>
>
>


[GPU] [Allocation] "Scarce" Resource Allocation

2016-06-10 Thread Benjamin Mahler
I wanted to start a discussion about the allocation of "scarce" resources.
"Scarce" in this context means resources that are not present on every
machine. GPUs are the first example of a scarce resource that we support as
a known resource type.

Consider the behavior when there are the following agents in a cluster:

999 agents with (cpus:4,mem:1024,disk:1024)
1 agent with (gpus:1,cpus:4,mem:1024,disk:1024)

Here there are 1000 machines but only 1 has GPUs. We call GPUs a "scarce"
resource here because they are only present on a small percentage of the
machines.

We end up with some problematic behavior here with our current allocation
model:

(1) If a role wishes to use both GPU and non-GPU resources for tasks,
consuming 1 GPU will lead DRF to consider the role to have a 100% share of
the cluster, since it consumes 100% of the GPUs in the cluster. This
framework will then not receive any other offers.

(2) Because we do not have revocation yet, if a framework decides to
consume the non-GPU resources on a GPU machine, it will prevent the GPU
workloads from running!



I filed an epic [1] to track this. The plan for the short-term is to
introduce two mechanisms to mitigate these issues:

-Introduce a resource fairness exclusion list. This allows the shares
of resources like "gpus" to be excluded from the dominant share.

-Introduce a GPU_AWARE framework capability. This indicates that the
scheduler is aware of GPUs and will schedule tasks accordingly. Old
schedulers will not have the capability and will not receive any offers for
GPU machines. If a scheduler has the capability, we'll advise that they
avoid placing their additional non-GPU workloads on the GPU machines.



Longer term, we'll want a more robust way to manage scarce resources. The
first thought we had was to have sub-pools of resources based on machine
profile and perform fair sharing / quota within each pool. This addresses
(1) cleanly, and for (2) the operator needs to explicitly disallow non-GPU
frameworks from participating in the GPU pool.

Unfortunately, by excluding non-GPU frameworks from the GPU pool we may
have a lower level of utilization. In the even longer term, as we add
revocation it will be possible to allow a scheduler desiring GPUs to revoke
the resources allocated to the non-GPU workloads running on the GPU
machines. There are a number of things we need to put in place to support
revocation ([2], [3], [4], etc), so I'm glossing over the details here.

If anyone has any thoughts or insight in this area, please share!

Ben

[1] https://issues.apache.org/jira/browse/MESOS-5377
[2] https://issues.apache.org/jira/browse/MESOS-5524
[3] https://issues.apache.org/jira/browse/MESOS-5527
[4] https://issues.apache.org/jira/browse/MESOS-4392


Re: Welcome Anand and Joseph as new committers!

2016-06-10 Thread Abhishek Dasgupta

Congrats Anand and Joseph.

Happy to know it.

On শুক্রবার 10 জুন 2016 02:30 পূর্বাহ্ণ, Vinod Kone wrote:

Hi folks,

I'm happy to announce that the PMC has voted in *Anand Mazumdar *and *Joseph
Wu* as committers and members of PMC for the Apache Mesos project.

A little about the new committers.

Anand has been working on the Apache Mesos project for about an year now
and has shown incredible commitment to the project and the community. His
significant contributions include implementing scheduler HTTP API,
designing and implementing executor HTTP API and helping out with the
operator HTTP API. His formal committer checklist is here

.

Joseph's passion and dedication to the community is phenomenal. His
significant contributions include Maintenance Primitives and Container
Logger Modules. He has also been a valuable contributor and reviewer to our
testing infrastructure and the Windows work. His formal committer checklist
is here

.

Please join me in congratulating them on their new roles and especially
responsibilities :)

On behalf of the PMC,
Vinod