Re: 1.0.0 RC2

[Vinod Kone]

There are still 17 un-resolved issues targeted for 1.0. We have only couple
more days left for the RC cut. Whoever is driving & shepherding these
please make sure to land them.



On Mon, Jun 13, 2016 at 1:58 PM, Vinod Kone  wrote:

> Hi folks,
>
> I'm planning to cut 1.0 RC2 later this week (likely friday). So please
> make sure to get any patches targeted for 1.0 (esp. blockers) upstreamed.
>
> The dashboard for the release is here:
> https://issues.apache.org/jira/issues/?filter=12335793
>
> Thanks,
> Vinod
>

Re: [Compatibility] More strict parsing of ranges, e.g. port of resources

[Benjamin Mahler]

Sounds OK to me if there are no objections, since it should not be a
difficult adjustment for users to make and users can use the more
expressive JSON format for resources already. (e.g.
https://github.com/dcos/dcos/blob/1.7-open/gen/dcos-config.yaml#L95)

Also, please document this in the CHANGELOG as a backwards-incompatible
change to the CLI.

On Mon, Jun 13, 2016 at 9:27 AM, Klaus Ma  wrote:

> Hi team,
>
>
> In "r/43561/", we'd like to move
> towards more strict parsing of ranges. If any comments, please let me know.
>
>
> Before the patch, the follow ranges are all valid:
>
> [1-2,3-4]
>
> [1-4]
>
> [[1-4]]
>
> [1-2]\n[3-4]
>
> [1-2],[3-4]
>
>
> After the patch, only the following ranges are valid:
>
> [1-2,3-4]
>
> [1-4]
>
>
> 
>
> Da (Klaus), Ma (??), PMP®| Advisory Software Engineer
> Platform DCOS Development & Support, STG, IBM GCG
> +86-10-8245 4084 | mad...@cn.ibm.com | http://k82.me
>
> 
>

Re: Flat Protobuf hierarchy

[Benjamin Mahler]

Is this the right project?
https://github.com/tomas-abrahamsson/gpb

If so, it seems to support package namespacing, just needs to be enabled:

"Gpb can optionally make use of the package attribute by prepending the
name of the package to every contained message type (if defined), which is
useful to avoid name clashes of message types across packages."

On Wed, Jun 15, 2016 at 1:37 AM, Sargun Dhillon  wrote:

> It turns out that not all protobufs compilers support dealing with
> packages correctly. The specific one I'm hitting an issue with today
> is gpb, the Erlang protobuf compiler. It's also a little bit weird in
> Python, and some of the other protobuf compilers. After doing a little
> bit of poking around, it appears we're doing this in only one place:
>
> Message:
>
> https://github.com/apache/mesos/blob/master/include/mesos/v1/mesos.proto#L1123-L1132
>
> Reference:
>
> https://github.com/apache/mesos/blob/master/include/mesos/v1/scheduler/scheduler.proto#L344-L346
>
> Given the reference is literally just wrapping the other one directly,
> I'm not even sure of what the immediate point of his is. Instead,
> could we move to a flat namespace, so that for us who don't have
> compilers that can handle protobuf packages, we can just treat all of
> the pb files as one giant flat file?
>
> I don't think this would take much change for the existing code, but
> it's more about the future.
>

Re: hellenic mesos user group

[haosdent]

Hi, @Ioannis. I think the PR @vinodkone mean is the pull request like
https://github.com/apache/mesos/pull/95/files
May you mind create a pull request like that in github?

On Wed, Jun 15, 2016 at 3:49 PM, Ioannis Petrousov 
wrote:

> Hello Vinod
>
> Thank you for approving our meetup.
>
> Here is the PR
>
> ### Greece
> * [Athens](http://www.meetup.com/Hellenic-Mesos-User-Group/)
>
> --
> *Ioannis Petrousov*
>
> On 14/6/2016 8:05 μμ, Vinod Kone wrote:
>
> Great to hear. Thanks for starting it!
>
> Mind sending a PR for
> https://github.com/apache/mesos/blob/df29bf0338771c92d1b1d3848181a35429cdcf0f/site/source/community/user-groups.html.md
> ?
>
> On Tue, Jun 14, 2016 at 3:43 AM, Ioannis Petrousov < 
> petrou...@gmail.com> wrote:
>
>> Hello Mesos world
>>
>> I would like to start the first Hellenic Mesos User Group based in
>> Athens, Greece.
>> I have created a meetup group on
>> 
>> http://www.meetup.com/Hellenic-Mesos-User-Group/ and a twitter account
>> on @HellenicMesos and are gathering members for our first meetup.
>> Not long ago I made a presentation about running Docker containers on
>> Mesos cluster on the Athens Docker meetup group
>> http://www.meetup.com/Docker-Athens/events/230640009/
>> There was a lot of interest from the attendants, so I decided to make a
>> group specializing in technologies built around Mesos platform.
>> On my full time job, I manage an in-house mesos cluster and write various
>> utilities, such as our scaling platform.
>>
>> I would appreciate if you add our group to the list of MUGs
>> http://mesos.apache.org/community/user-groups/ and make it official.
>> --
>> *Ioannis Petrousov*
>>
>
>


-- 
Best Regards,
Haosdent Huang

Re: Rack awareness support for Mesos

[james]

@Joris,


OK. Now I understand where you are coming from. As soon as I get some 
time, I'll join that design discussion. Thanks for the clarifications.


James





On 06/15/2016 02:45 AM, Joris Van Remoortere wrote:

Since your interest is in the determination of the values, as
opposed to

their propagation, I would just urge that you keep in mind that
we may

(as a project) not want to support this information as the current

string attributes.


Huh? Why not? If the attributes change, why can't this sub-project
just change with those changing string attributes? Maybe some
elaboration how this might not naturally be able to evolve is a
warranted detail of discussion?


Sorry, I should clarify what I meant by support. By support I mean that
we may not want to promise that those values will be there (support as a
feature), and what schemas are mangled into the random strings that we
currently call attributes. I did not mean that we wouldn't allow users
to inject their own values if they wanted to. We just wouldn't control
the standard or schema as a project and therefore couldn't support it.

Any random collection of strings that has previously had no reserved
keywords is notoriously difficult to build new schemas in.
This is why we may want to instead introduce a typed structure that is
dedicated to fault domain information. This:

  * Prevents us from colliding with current users' attributes.
  * Allows us to have more control over the types (YAY) and ranges of
values.
  * Allows us to introduce explicit structure such as dependency or
hierarchy.

The fact that users have already encoded information in attributes is
not a reason for us to limit ourselves to that scope when better
structures may be available. This is why we shouldn't assume that the
project will *provide support for* (as opposed to allow users to) using
attributes.

As your said, it is their prerogative to join the design discussion to
ensure that any formalized structure or schema we introduce is one that
they are agreeable with.



—
*Joris Van Remoortere*
Mesosphere

On Tue, Jun 14, 2016 at 6:31 PM, james > wrote:

On 06/14/2016 08:14 AM, Joris Van Remoortere wrote:

On the condition of compatible with existing framework which
already rely on parsing attributes for rack information.

There is currently nothing in Mesos that specifies the format or
structure for rack information in attributes.
The fact that operators / frameworks have decided to add this
information out of band is their problem to solve.
We don't need to be backwards compatible with something we never
published to begin with. This is why it's ok for us to consider
adding a
typed form of failure domain information that is separate from the
typeless string attributes.


True. But you have to start somewhere, know that the schema and
codes will morph over time to maintain relevance  and usefulness. In
that vein, if folks have established interesting and useful
parameters for this work, then it is most beneficial that those
methods and codes are considered carefully.  AKA:: speak up now.
Diversity and inclusion are keenly beneficial, where practical.


Since your interest is in the determination of the values, as
opposed to
their propagation, I would just urge that you keep in mind that
we may
(as a project) not want to support this information as the current
string attributes.


Huh? Why not? If the attributes change, why can't this sub-project
just change with those changing string attributes? Maybe some
elaboration how this might not naturally be able to evolve is a
warranted detail of discussion?


I would venture that both 'determination of the values and
propagation (delays)' are inherently important in a cluster of many
things:: hardware, resources, frameworks, security codes, etc etc.
The author
and others seem to be keenly aware that a tight focus is not going
to work, at this stage, so a broad appeal to a multitude of needs is
best.
And in fact, until some idea is proven to be useless or too difficult to
implement, the bigger the tent, the more useful the codes that
define this project/idea become.  Personally, I'm very excited that
someone has stepped up in this area; hoping they keep an open mind
and flexibility geared toward multiplicative usage, in the future.
Most mature hardware folks who build ideas into robust systems do
exactly that, to motivate a multiplicative usage for organizing
hardware, performance and state metrics, and timing signals,
gregariously. All of this is routine semantics from a hardware
perspective.

At some point, folks will realize that kernel configuration, 

Flat Protobuf hierarchy

[Sargun Dhillon]

It turns out that not all protobufs compilers support dealing with
packages correctly. The specific one I'm hitting an issue with today
is gpb, the Erlang protobuf compiler. It's also a little bit weird in
Python, and some of the other protobuf compilers. After doing a little
bit of poking around, it appears we're doing this in only one place:

Message:
https://github.com/apache/mesos/blob/master/include/mesos/v1/mesos.proto#L1123-L1132

Reference:
https://github.com/apache/mesos/blob/master/include/mesos/v1/scheduler/scheduler.proto#L344-L346

Given the reference is literally just wrapping the other one directly,
I'm not even sure of what the immediate point of his is. Instead,
could we move to a flat namespace, so that for us who don't have
compilers that can handle protobuf packages, we can just treat all of
the pb files as one giant flat file?

I don't think this would take much change for the existing code, but
it's more about the future.

Flat Protobuf hierarchy

[Sargun Dhillon]

It turns out that not all protobufs compilers support dealing with packages
correctly. The specific one I'm hitting an issue with today is gpb, the
Erlang protobuf compiler. It's also a little bit weird in Python, and some
of the other protobuf compilers. After doing a little bit of poking around,
it appears we're doing this in only one place:

Re: hellenic mesos user group

[Ioannis Petrousov]

Hello Vinod

Thank you for approving our meetup.

Here is the PR

### Greece
* [Athens](http://www.meetup.com/Hellenic-Mesos-User-Group/)

--
*Ioannis Petrousov*


On 14/6/2016 8:05 μμ, Vinod Kone wrote:

Great to hear. Thanks for starting it!

Mind sending a PR for 
https://github.com/apache/mesos/blob/df29bf0338771c92d1b1d3848181a35429cdcf0f/site/source/community/user-groups.html.md 
?


On Tue, Jun 14, 2016 at 3:43 AM, Ioannis Petrousov 
> wrote:


Hello Mesos world

I would like to start the first Hellenic Mesos User Group based in
Athens, Greece.
I have created a meetup group on
http://www.meetup.com/Hellenic-Mesos-User-Group/ and a twitter
account on @HellenicMesos and are gathering members for our first
meetup.
Not long ago I made a presentation about running Docker containers
on Mesos cluster on the Athens Docker meetup group
http://www.meetup.com/Docker-Athens/events/230640009/
There was a lot of interest from the attendants, so I decided to
make a group specializing in technologies built around Mesos platform.
On my full time job, I manage an in-house mesos cluster and write
various utilities, such as our scaling platform.

I would appreciate if you add our group to the list of MUGs
http://mesos.apache.org/community/user-groups/ and make it official.

-- 
*Ioannis Petrousov*

Re: Rack awareness support for Mesos

[Joris Van Remoortere]

Since your interest is in the determination of the values, as opposed to

their propagation, I would just urge that you keep in mind that we may

(as a project) not want to support this information as the current

string attributes.


Huh? Why not? If the attributes change, why can't this sub-project just
> change with those changing string attributes? Maybe some elaboration how
> this might not naturally be able to evolve is a warranted detail of
> discussion?


Sorry, I should clarify what I meant by support. By support I mean that we
may not want to promise that those values will be there (support as a
feature), and what schemas are mangled into the random strings that we
currently call attributes. I did not mean that we wouldn't allow users to
inject their own values if they wanted to. We just wouldn't control the
standard or schema as a project and therefore couldn't support it.

Any random collection of strings that has previously had no reserved
keywords is notoriously difficult to build new schemas in.
This is why we may want to instead introduce a typed structure that is
dedicated to fault domain information. This:

   - Prevents us from colliding with current users' attributes.
   - Allows us to have more control over the types (YAY) and ranges of
   values.
   - Allows us to introduce explicit structure such as dependency or
   hierarchy.

The fact that users have already encoded information in attributes is not a
reason for us to limit ourselves to that scope when better structures may
be available. This is why we shouldn't assume that the project will
*provide support for* (as opposed to allow users to) using attributes.

As your said, it is their prerogative to join the design discussion to
ensure that any formalized structure or schema we introduce is one that
they are agreeable with.



—
*Joris Van Remoortere*
Mesosphere

On Tue, Jun 14, 2016 at 6:31 PM, james  wrote:

> On 06/14/2016 08:14 AM, Joris Van Remoortere wrote:
>
>> On the condition of compatible with existing framework which already rely
>>> on parsing attributes for rack information.
>>>
>> There is currently nothing in Mesos that specifies the format or
>> structure for rack information in attributes.
>> The fact that operators / frameworks have decided to add this
>> information out of band is their problem to solve.
>> We don't need to be backwards compatible with something we never
>> published to begin with. This is why it's ok for us to consider adding a
>> typed form of failure domain information that is separate from the
>> typeless string attributes.
>>
>
> True. But you have to start somewhere, know that the schema and codes will
> morph over time to maintain relevance  and usefulness. In that vein, if
> folks have established interesting and useful parameters for this work,
> then it is most beneficial that those methods and codes are considered
> carefully.  AKA:: speak up now. Diversity and inclusion are keenly
> beneficial, where practical.
>
>
> Since your interest is in the determination of the values, as opposed to
>> their propagation, I would just urge that you keep in mind that we may
>> (as a project) not want to support this information as the current
>> string attributes.
>>
>
> Huh? Why not? If the attributes change, why can't this sub-project just
> change with those changing string attributes? Maybe some elaboration how
> this might not naturally be able to evolve is a warranted detail of
> discussion?
>
>
> I would venture that both 'determination of the values and propagation
> (delays)' are inherently important in a cluster of many things:: hardware,
> resources, frameworks, security codes, etc etc. The author
> and others seem to be keenly aware that a tight focus is not going to
> work, at this stage, so a broad appeal to a multitude of needs is best.
> And in fact, until some idea is proven to be useless or too difficult to
> implement, the bigger the tent, the more useful the codes that define this
> project/idea become.  Personally, I'm very excited that someone has stepped
> up in this area; hoping they keep an open mind and flexibility geared
> toward multiplicative usage, in the future. Most mature hardware folks who
> build ideas into robust systems do exactly that, to motivate a
> multiplicative usage for organizing hardware, performance and state
> metrics, and timing signals, gregariously. All of this is routine semantics
> from a hardware perspective.
>
> At some point, folks will realize that kernel configuration, testing and
> tweaks are critical to cluster performance, regardless of the codes
> running on top of the cluster. So this project could easily use cgroups
> and such for achieve robustness in many areas of need.
>
>
> Like it or not large amounts of hardware, need to have schema, planning
> and architectural robustness to keep large amounts of hardware, pristinely
> available for software efficiency to be any where near optimal deployment.
> This really