Re: Feature requests for Mesos

[Klaus Ma]

> - willingness of remaining active committers to be active on a regular
> basis in engagements with the community, both on the user and contributor
> side (in PRs, review requests, on mailing lists),
> - transparent and active discussions in the community, among committers
and
> contributors, and among committers, in applicable form, beyond roll calls,
> - timely and consistent process to address user issues, and
> - consistent ownership of the bug and feature backlog.

How can we get more contributors or committers to help on those items?



On Tue, Mar 2, 2021 at 2:13 AM Charles-François Natali 
wrote:

> I couldn't agree more.
>
>
>
> On Mon, 1 Mar 2021, 15:08 Benjamin Bannier,  wrote:
>
> > Hi Charles-François,
> >
> > thanks for your detailed message, you captured important points, and I
> > think I agree with your sentiment here. Mesos might still have a place,
> and
> > before thinking about what new features to add, the project first needs
> to
> > solve more fundamental issues.
> >
> > My previous pessimistic assessment on this list came from a similar angle
> > but I think with wider scope: a healthy project requires a healthy
> > community where users can find help, but also can have some hope that
> > important issues will get fixed. I have not been able to spend much time
> on
> > Mesos in the last year, but was following Slack and the mailing lists
> (the
> > ones with humans and the ones with bots). On the mailing lists I see
> users
> > ask for help with issues they run into or questions, but only rarely will
> > get a response from committers or other community members. Few new JIRA
> > issues were filed in the since fall 2020, but hardly any of them have
> been
> > triaged let alone fixed (this is on top of the existing bug backlog). I
> do
> > not think one needs to be a committer to improve on that situation if one
> > can get help getting patches discussed, reviewed and ultimately merged.
> It
> > looks like Andrei and Qian have committed to help on the latter, but I
> have
> > only rarely seen community members volunteer for the former.
> >
> > When I wrote that I thought starting a new project on top of Apache Mesos
> > today might be not a good idea, I mainly came from that angle. While the
> > software does work for many use cases it seems to be unmaintained with
> > hardly any folks active in taking it further globally, beyond their own
> > immediate needs, and willing to take on the needed work. Being a
> top-level
> > Apache project with a strong history, Apache Mesos still has a brand,
> but I
> > don't think it has lived up to the associated expectations. Similarly,
> big
> > ownership gaps (technical and project-wise) have developed which neither
> > active committers nor community members have filled. Again, one would not
> > need to be a committer to develop expertise and contribute, and actually
> > the natural and historic process was for folks to do exactly that with
> > committership being a thing only after getting involved (see
> > https://community.apache.org/newcommitter.html for Apache's high-level
> > view
> > on that). This is the issue of continued trust Renan mentioned in their
> > message to the user mailing list which I also believe is critical so the
> > project can live up to its promise (this is integral to being an Apache
> > project, see e.g., https://www.apache.org/theapacheway).
> >
> > As a non-user with emotional attachment to the historic Apache Mesos
> brand,
> > my list of areas in need of improvement to resurrect this project would
> be:
> >
> > - willingness of remaining active committers to be active on a regular
> > basis in engagements with the community, both on the user and contributor
> > side (in PRs, review requests, on mailing lists),
> > - transparent and active discussions in the community, among committers
> and
> > contributors, and among committers, in applicable form, beyond roll
> calls,
> > - timely and consistent process to address user issues, and
> > - consistent ownership of the bug and feature backlog.
> >
> > Note that work on new feature requests is absent from my list. That folks
> > want to discuss that here and now seems to me to be another sign that the
> > Mesos community is not in a good place given all its existing
> non-technical
> > issues.
> >
> >
> > Best,
> >
> > Benjamin
> >
>

Re: Feature requests for Mesos

[Charles-François Natali]

I couldn't agree more.



On Mon, 1 Mar 2021, 15:08 Benjamin Bannier,  wrote:

> Hi Charles-François,
>
> thanks for your detailed message, you captured important points, and I
> think I agree with your sentiment here. Mesos might still have a place, and
> before thinking about what new features to add, the project first needs to
> solve more fundamental issues.
>
> My previous pessimistic assessment on this list came from a similar angle
> but I think with wider scope: a healthy project requires a healthy
> community where users can find help, but also can have some hope that
> important issues will get fixed. I have not been able to spend much time on
> Mesos in the last year, but was following Slack and the mailing lists (the
> ones with humans and the ones with bots). On the mailing lists I see users
> ask for help with issues they run into or questions, but only rarely will
> get a response from committers or other community members. Few new JIRA
> issues were filed in the since fall 2020, but hardly any of them have been
> triaged let alone fixed (this is on top of the existing bug backlog). I do
> not think one needs to be a committer to improve on that situation if one
> can get help getting patches discussed, reviewed and ultimately merged. It
> looks like Andrei and Qian have committed to help on the latter, but I have
> only rarely seen community members volunteer for the former.
>
> When I wrote that I thought starting a new project on top of Apache Mesos
> today might be not a good idea, I mainly came from that angle. While the
> software does work for many use cases it seems to be unmaintained with
> hardly any folks active in taking it further globally, beyond their own
> immediate needs, and willing to take on the needed work. Being a top-level
> Apache project with a strong history, Apache Mesos still has a brand, but I
> don't think it has lived up to the associated expectations. Similarly, big
> ownership gaps (technical and project-wise) have developed which neither
> active committers nor community members have filled. Again, one would not
> need to be a committer to develop expertise and contribute, and actually
> the natural and historic process was for folks to do exactly that with
> committership being a thing only after getting involved (see
> https://community.apache.org/newcommitter.html for Apache's high-level
> view
> on that). This is the issue of continued trust Renan mentioned in their
> message to the user mailing list which I also believe is critical so the
> project can live up to its promise (this is integral to being an Apache
> project, see e.g., https://www.apache.org/theapacheway).
>
> As a non-user with emotional attachment to the historic Apache Mesos brand,
> my list of areas in need of improvement to resurrect this project would be:
>
> - willingness of remaining active committers to be active on a regular
> basis in engagements with the community, both on the user and contributor
> side (in PRs, review requests, on mailing lists),
> - transparent and active discussions in the community, among committers and
> contributors, and among committers, in applicable form, beyond roll calls,
> - timely and consistent process to address user issues, and
> - consistent ownership of the bug and feature backlog.
>
> Note that work on new feature requests is absent from my list. That folks
> want to discuss that here and now seems to me to be another sign that the
> Mesos community is not in a good place given all its existing non-technical
> issues.
>
>
> Best,
>
> Benjamin
>

Re: Feature requests for Mesos

[Benjamin Bannier]

Hi Charles-François,

thanks for your detailed message, you captured important points, and I
think I agree with your sentiment here. Mesos might still have a place, and
before thinking about what new features to add, the project first needs to
solve more fundamental issues.

My previous pessimistic assessment on this list came from a similar angle
but I think with wider scope: a healthy project requires a healthy
community where users can find help, but also can have some hope that
important issues will get fixed. I have not been able to spend much time on
Mesos in the last year, but was following Slack and the mailing lists (the
ones with humans and the ones with bots). On the mailing lists I see users
ask for help with issues they run into or questions, but only rarely will
get a response from committers or other community members. Few new JIRA
issues were filed in the since fall 2020, but hardly any of them have been
triaged let alone fixed (this is on top of the existing bug backlog). I do
not think one needs to be a committer to improve on that situation if one
can get help getting patches discussed, reviewed and ultimately merged. It
looks like Andrei and Qian have committed to help on the latter, but I have
only rarely seen community members volunteer for the former.

When I wrote that I thought starting a new project on top of Apache Mesos
today might be not a good idea, I mainly came from that angle. While the
software does work for many use cases it seems to be unmaintained with
hardly any folks active in taking it further globally, beyond their own
immediate needs, and willing to take on the needed work. Being a top-level
Apache project with a strong history, Apache Mesos still has a brand, but I
don't think it has lived up to the associated expectations. Similarly, big
ownership gaps (technical and project-wise) have developed which neither
active committers nor community members have filled. Again, one would not
need to be a committer to develop expertise and contribute, and actually
the natural and historic process was for folks to do exactly that with
committership being a thing only after getting involved (see
https://community.apache.org/newcommitter.html for Apache's high-level view
on that). This is the issue of continued trust Renan mentioned in their
message to the user mailing list which I also believe is critical so the
project can live up to its promise (this is integral to being an Apache
project, see e.g., https://www.apache.org/theapacheway).

As a non-user with emotional attachment to the historic Apache Mesos brand,
my list of areas in need of improvement to resurrect this project would be:

- willingness of remaining active committers to be active on a regular
basis in engagements with the community, both on the user and contributor
side (in PRs, review requests, on mailing lists),
- transparent and active discussions in the community, among committers and
contributors, and among committers, in applicable form, beyond roll calls,
- timely and consistent process to address user issues, and
- consistent ownership of the bug and feature backlog.

Note that work on new feature requests is absent from my list. That folks
want to discuss that here and now seems to me to be another sign that the
Mesos community is not in a good place given all its existing non-technical
issues.


Best,

Benjamin

RE: Feature requests for Mesos

[Chris Holman]

>On 2021-02-28 05:38 PM, Qian Zhang wrote:
>> Hi Folks,
>> 
>> To reboot this awesome project, I'd like to collect feature requests 
>> for Mesos. Please let us know your requirements for Mesos and whether 
>> you or your organization would like to contribute to the 
>> implementation of the requirements. Thanks!
>
>We can already summarize what has been said in previous discussions, mainly 
>making mesos easier to deploy / maintain, and my personal one, with more 
>batteries included by default.
>There are also several ways of improvements, especially regarding GPU support, 
>numa, ZFS integration, volume management...
>
>But first, reducing the effort to have a fully functional mesos cluster would 
>be to me, one of the priorities (and also targeting local development like 
>minimesos was trying to achieve)
>
>
>-- 
>Damien GERARD

Ease of install is definitely something that would make it easier for others to 
use and experiment with Mesos.

I've just been through the process of setting up a Mesos cluster and it's 
definitely a right-of-passage.
Not a particularly fun experience, but primarily because I wanted ssl enabled.
I'm using an Ansible playbook to deploy a Linux/Windows Mesos cluster for a 
Jenkins Build Farm.

I've made a few local mods to:
* Mesos to support Windows and ssl (not complete)
* Jenkins Mesos Plugin (and mesos-usi) to support reusable Agents and Mesos 
attribute operators i.e. match Mesos offers with an attribute version 
[lt|le|gt|eq|ne|ge|gt] '2.2.2'

Regards, Chris

Re: Feature requests for Mesos

[Sargun Dhillon]

On Sun, Feb 28, 2021 at 12:39 AM Qian Zhang  wrote:
>
> Hi Folks,
>
> To reboot this awesome project, I'd like to collect feature requests for
> Mesos. Please let us know your requirements for Mesos and whether you or
> your organization would like to contribute to the implementation of the
> requirements. Thanks!
>
>
> Regards,
> Qian Zhang
IPv6 and unidirectional connections -- or moving off Mesos would be wonderful.

Re: Feature requests for Mesos

[Damien GERARD]

On 2021-03-01 04:14 PM, Klaus Ma wrote:

Mesos is really a great project!

In addition to new features, is it possible to make Mesos focus on
some specific area, e.g. HPC?


I really do think also that HPC should be a target :)



-- Klaus
<http://k82.me/>

From: Damien GERARD 
Sent: Monday, March 1, 2021 1:05 AM
To: u...@mesos.apache.org 
Cc: mesos 
Subject: Re: Feature requests for Mesos

On 2021-02-28 05:38 PM, Qian Zhang wrote:

Hi Folks,

To reboot this awesome project, I'd like to collect feature requests
for Mesos. Please let us know your requirements for Mesos and whether
you or your organization would like to contribute to the
implementation of the requirements. Thanks!


We can already summarize what has been said in previous discussions,
mainly
making mesos easier to deploy / maintain, and my personal one, with 
more

batteries included by default.
There are also several ways of improvements, especially regarding GPU
support,
numa, ZFS integration, volume management...

But first, reducing the effort to have a fully functional mesos cluster
would be to me, one of the priorities (and also targeting local
development
like minimesos was trying to achieve)


--
Damien GERARD


--
Damien GERARD

Re: Feature requests for Mesos

[Klaus Ma]

Mesos is really a great project!

In addition to new features, is it possible to make Mesos focus on some 
specific area, e.g. HPC?

-- Klaus
<http://k82.me/>

From: Damien GERARD 
Sent: Monday, March 1, 2021 1:05 AM
To: u...@mesos.apache.org 
Cc: mesos 
Subject: Re: Feature requests for Mesos

On 2021-02-28 05:38 PM, Qian Zhang wrote:
> Hi Folks,
>
> To reboot this awesome project, I'd like to collect feature requests
> for Mesos. Please let us know your requirements for Mesos and whether
> you or your organization would like to contribute to the
> implementation of the requirements. Thanks!

We can already summarize what has been said in previous discussions,
mainly
making mesos easier to deploy / maintain, and my personal one, with more
batteries included by default.
There are also several ways of improvements, especially regarding GPU
support,
numa, ZFS integration, volume management...

But first, reducing the effort to have a fully functional mesos cluster
would be to me, one of the priorities (and also targeting local
development
like minimesos was trying to achieve)


--
Damien GERARD

Re: Feature requests for Mesos

[Damien GERARD]

On 2021-02-28 05:38 PM, Qian Zhang wrote:

Hi Folks,

To reboot this awesome project, I'd like to collect feature requests
for Mesos. Please let us know your requirements for Mesos and whether
you or your organization would like to contribute to the
implementation of the requirements. Thanks!


We can already summarize what has been said in previous discussions, 
mainly

making mesos easier to deploy / maintain, and my personal one, with more
batteries included by default.
There are also several ways of improvements, especially regarding GPU 
support,

numa, ZFS integration, volume management...

But first, reducing the effort to have a fully functional mesos cluster
would be to me, one of the priorities (and also targeting local 
development

like minimesos was trying to achieve)


--
Damien GERARD

Re: Feature requests for Mesos

[Samuel Marks]

RE: Kubernetes and FoundationDB commentary

The idea with having etcd, Consul, and ZooKeeper is not to limit it to
those, but to push everyone to use our library [
https://github.com/offscale/?q=liboffkv]. Then that library can expand,
with more and more storage engines built in.

On the Kubernetes side, yes you are right, it is super popular. My team
intended to decouple etcd from Kubernetes and enable the choice betwixt
these three also, but were waiting for it to be merged into Mesos first.
This turned out to be a misallocation of resources, I should've tasked my
team with Kubernetes as the primary target, seeing as Mesos didn't end up
being very open to this contribution…

Samuel Marks
Charity  | consultancy 
| open-source  | LinkedIn



On Mon, Mar 1, 2021 at 4:56 AM Charles-François Natali 
wrote:

> I'm not sure that the comparison to Kubernetes is very apt - I don't think
> anyone is under the illusion that Mesos is a contender for it, that ship
> has long sailed.
>
> Also, features availability is only one of the many aspects which should be
> considered when choosing among potential candidate technologies:
> reliability, scalability, operational complexity, extensibility and fitness
> to the actual use case are all aspects which drove us to choose Mesos over
> the many alternative back when we made the decision - which we haven't
> regretted since.
>
> To give a very bad analogy, Firefox is a much better and feature complete
> browser than Curl, however it doesn't mean that the latter doesn't have its
> use cases, even in 2021.
>
> As far as I'm concerned, I'd be quite happy with Mesos continuing as a
> "distributed systems kernel" as described on http://mesos.apache.org/,
> allowing interested individuals and organisations to build on top of it -
> we're certainly happy with this paradigm at my company.
>
> Now obviously it doesn't mean that we shouldn't consider adding new
> features - for example something which was requested by some users a few
> months ago and which we ended up implementing ourselves at the framework,
> agent resource and custom executor level was support for NUMA topology
> awareness, which could probably make sense to add to Mesos.
> For more inspiration for new features, I think Nomad and Slurm might be
> worth looking into as inspiration.
>
> Also, simply continuing to address existing bug reports and MRs would I
> think be a good starting point to try to revive the project, get potential
> new contributors familiar with the code base, etc.
> And merging some changes maintained in third party forks such as the ones
> mentioned by the people from Criteo who also mentioned an interest in
> keeping Mesos going forward.
>
> People like Javi Roman seem to have some idea on potential new features -
> those would be great to hear as well!
>
> However the main barrier to any progress I can see right now - and which
> was discussed in the previous thread - is that none of the current
> maintainers seem to have any time to dedicate to the project, including
> reviewing existing patches/MR. I still haven't seen any suggestion from the
> current project members on ways to address that.
> I know some people objected that electing new committers might not be
> enough to revive the project - I most certainly agree it might very well
> not be sufficient, however I think it is a necessary condition, unless some
> of the current maintainers are willing to dedicate some time to onboarding
> potential new contributors.
>
> Cheers,
>
> Charles
>
>
>
>
>
> On Sun, 28 Feb 2021, 16:42 Jorge Machado,  wrote:
>
> > Hi Samuel,
> >
> > To be honest, I would not invest any more Time on Mesos. The features
> from
> > Kubernetes are just way better. :)
> >
> > > On 28. Feb 2021, at 12:54, Samuel Marks  wrote:
> > >
> > > Decouple Apache ZooKeeper, enabling Apache Mesos to run completely
> > without
> > > ZooKeeper. Specifically enable a choice between ZooKeeper, etcd, and
> > consul.
> > >
> > > My organisation is somewhat interested in contributing this. We tried
> in
> > > the past but came across some hurdles on the Mesos organisation end.
> Open
> > > to trying again, but will need a clear pathway to getting this
> accepted.
> > >
> > > Samuel Marks
> > > Charity  | consultancy <
> > https://offscale.io>
> > > | open-source  | LinkedIn
> > > 
> > >
> > >
> > > On Sun, Feb 28, 2021 at 7:39 PM Qian Zhang 
> wrote:
> > >
> > >> Hi Folks,
> > >>
> > >> To reboot this awesome project, I'd like to collect feature requests
> for
> > >> Mesos. Please let us know your requirements for Mesos and whether you
> or
> > >> your organization would like to contribute to the implementation of
> the
> > >> requirements. Thanks!
> > >>
> > >>
> > >> Regards,
> > >> Qian Zhang
> > >>
> >
> >
>

Re: Feature requests for Mesos

[Charles-François Natali]

I'm not sure that the comparison to Kubernetes is very apt - I don't think
anyone is under the illusion that Mesos is a contender for it, that ship
has long sailed.

Also, features availability is only one of the many aspects which should be
considered when choosing among potential candidate technologies:
reliability, scalability, operational complexity, extensibility and fitness
to the actual use case are all aspects which drove us to choose Mesos over
the many alternative back when we made the decision - which we haven't
regretted since.

To give a very bad analogy, Firefox is a much better and feature complete
browser than Curl, however it doesn't mean that the latter doesn't have its
use cases, even in 2021.

As far as I'm concerned, I'd be quite happy with Mesos continuing as a
"distributed systems kernel" as described on http://mesos.apache.org/,
allowing interested individuals and organisations to build on top of it -
we're certainly happy with this paradigm at my company.

Now obviously it doesn't mean that we shouldn't consider adding new
features - for example something which was requested by some users a few
months ago and which we ended up implementing ourselves at the framework,
agent resource and custom executor level was support for NUMA topology
awareness, which could probably make sense to add to Mesos.
For more inspiration for new features, I think Nomad and Slurm might be
worth looking into as inspiration.

Also, simply continuing to address existing bug reports and MRs would I
think be a good starting point to try to revive the project, get potential
new contributors familiar with the code base, etc.
And merging some changes maintained in third party forks such as the ones
mentioned by the people from Criteo who also mentioned an interest in
keeping Mesos going forward.

People like Javi Roman seem to have some idea on potential new features -
those would be great to hear as well!

However the main barrier to any progress I can see right now - and which
was discussed in the previous thread - is that none of the current
maintainers seem to have any time to dedicate to the project, including
reviewing existing patches/MR. I still haven't seen any suggestion from the
current project members on ways to address that.
I know some people objected that electing new committers might not be
enough to revive the project - I most certainly agree it might very well
not be sufficient, however I think it is a necessary condition, unless some
of the current maintainers are willing to dedicate some time to onboarding
potential new contributors.

Cheers,

Charles





On Sun, 28 Feb 2021, 16:42 Jorge Machado,  wrote:

> Hi Samuel,
>
> To be honest, I would not invest any more Time on Mesos. The features from
> Kubernetes are just way better. :)
>
> > On 28. Feb 2021, at 12:54, Samuel Marks  wrote:
> >
> > Decouple Apache ZooKeeper, enabling Apache Mesos to run completely
> without
> > ZooKeeper. Specifically enable a choice between ZooKeeper, etcd, and
> consul.
> >
> > My organisation is somewhat interested in contributing this. We tried in
> > the past but came across some hurdles on the Mesos organisation end. Open
> > to trying again, but will need a clear pathway to getting this accepted.
> >
> > Samuel Marks
> > Charity  | consultancy <
> https://offscale.io>
> > | open-source  | LinkedIn
> > 
> >
> >
> > On Sun, Feb 28, 2021 at 7:39 PM Qian Zhang  wrote:
> >
> >> Hi Folks,
> >>
> >> To reboot this awesome project, I'd like to collect feature requests for
> >> Mesos. Please let us know your requirements for Mesos and whether you or
> >> your organization would like to contribute to the implementation of the
> >> requirements. Thanks!
> >>
> >>
> >> Regards,
> >> Qian Zhang
> >>
>
>

Re: Feature requests for Mesos

[Jorge Machado]

Hi Samuel, 

To be honest, I would not invest any more Time on Mesos. The features from 
Kubernetes are just way better. :) 

> On 28. Feb 2021, at 12:54, Samuel Marks  wrote:
> 
> Decouple Apache ZooKeeper, enabling Apache Mesos to run completely without
> ZooKeeper. Specifically enable a choice between ZooKeeper, etcd, and consul.
> 
> My organisation is somewhat interested in contributing this. We tried in
> the past but came across some hurdles on the Mesos organisation end. Open
> to trying again, but will need a clear pathway to getting this accepted.
> 
> Samuel Marks
> Charity  | consultancy 
> | open-source  | LinkedIn
> 
> 
> 
> On Sun, Feb 28, 2021 at 7:39 PM Qian Zhang  wrote:
> 
>> Hi Folks,
>> 
>> To reboot this awesome project, I'd like to collect feature requests for
>> Mesos. Please let us know your requirements for Mesos and whether you or
>> your organization would like to contribute to the implementation of the
>> requirements. Thanks!
>> 
>> 
>> Regards,
>> Qian Zhang
>> 

Re: Feature requests for Mesos

[Damien GERARD]

On 2021-02-28 08:54 PM, Samuel Marks wrote:
Decouple Apache ZooKeeper, enabling Apache Mesos to run completely 
without
ZooKeeper. Specifically enable a choice between ZooKeeper, etcd, and 
consul.




To be honest, I would prefer something like foundationdb, and slowly 
moving

to a multi-master mode

My organisation is somewhat interested in contributing this. We tried 
in
the past but came across some hurdles on the Mesos organisation end. 
Open
to trying again, but will need a clear pathway to getting this 
accepted.


Samuel Marks
Charity  | consultancy 


| open-source  | LinkedIn



On Sun, Feb 28, 2021 at 7:39 PM Qian Zhang  wrote:


Hi Folks,

To reboot this awesome project, I'd like to collect feature requests 
for
Mesos. Please let us know your requirements for Mesos and whether you 
or
your organization would like to contribute to the implementation of 
the

requirements. Thanks!


Regards,
Qian Zhang



--
Damien GERARD

Re: Feature requests for Mesos

[Samuel Marks]

Decouple Apache ZooKeeper, enabling Apache Mesos to run completely without
ZooKeeper. Specifically enable a choice between ZooKeeper, etcd, and consul.

My organisation is somewhat interested in contributing this. We tried in
the past but came across some hurdles on the Mesos organisation end. Open
to trying again, but will need a clear pathway to getting this accepted.

Samuel Marks
Charity  | consultancy 
| open-source  | LinkedIn



On Sun, Feb 28, 2021 at 7:39 PM Qian Zhang  wrote:

> Hi Folks,
>
> To reboot this awesome project, I'd like to collect feature requests for
> Mesos. Please let us know your requirements for Mesos and whether you or
> your organization would like to contribute to the implementation of the
> requirements. Thanks!
>
>
> Regards,
> Qian Zhang
>