Re: Myriad Sync 11/29

[Adam Bordelon]

What time (and time zone) is this again? Slipped off my calendar.

On Tue, Nov 28, 2017 at 7:04 AM, mohit soni  wrote:

> Here you go:
> https://hangouts.google.com/hangouts/_/calendar/
> bW9oaXRzb25pMTk4OUBnbWFpbC5jb20.ajan3qfvb8c48egfc8rpc92b10
>
> On Tue, Nov 28, 2017 at 8:51 AM, Darin Johnson 
> wrote:
>
> > Can you put a link to the Hangouts in the thread?
> >
> > On Nov 28, 2017 8:47 AM, "mohit soni"  wrote:
> >
> > > Hi All,
> > >
> > > I'm planning to attend the Myriad sync tomorrow. Please join if you
> would
> > > like to discuss the upcoming changes to the project.
> > >
> > > Best
> > > Mohit
> > >
> >
>

Re: Myriad development status

[Adam Bordelon]

Nicolas,
Myriad is still alive, but hasn't seen much activity lately.
If you don't need YARN, etc. and only need HDFS, you may want to check out
the HDFS package for DC/OS (which is built on Apache Mesos).
https://docs.mesosphere.com/service-docs/hdfs/
There's also a deprecated HDFS package for vanilla Apache Mesos:
https://github.com/mesosphere/hdfs-deprecated

On Tue, Nov 14, 2017 at 12:55 AM, nicolas dutronc <
nicolas.dutr...@soyhuce.fr> wrote:

> Good morning,
>
> I am Nicolas Dutronc and I work in a small company called SoyHuCe. We are
> currently working on setting up a Big Data architecture with Mesos and we
> want to use HDFS to store our master dataset. Myriad seems to be a good
> solution to do it. However, before getting started with Myriad, we want to
> make sure that the development is still in progress. Could you tell me if
> it is the case ?
>
> Thank you very much in advance ! :)
>
> Best regards.
>
> Nicolas Dutronc
>

Re: Report

[Adam Bordelon]

+1, on behalf of myself

>From a Mesosphere perspective, we're not committing any resources to it in
the foreseeable future.

On Tue, Oct 10, 2017 at 11:21 AM, Darin Johnson 
wrote:

> I'll second that sentiment.
>
> On Tue, Oct 10, 2017 at 11:45 AM, yuliya Feldman <
> yufeld...@yahoo.com.invalid> wrote:
>
> >  Feels like project is dead, Wonder if anybody at MapR and/or Mesosphere
> > interested in continuing the project?
> > On Tuesday, October 10, 2017, 4:42:56 AM PDT, Ted Dunning <
> > ted.dunn...@gmail.com> wrote:
> >
> >  The incubator report for myriad seems to be missing.
> >
> > Is anybody up for filing it? (probably too late for this month's report)
> >
> >
>

Re: Podling Report Reminder - July 2017

[Adam Bordelon]

I will be out of town during this period. Can somebody else fill out the
podling report in the wiki, with particular attention to the retirement
vote/discussion?

On Tue, Jun 27, 2017 at 4:54 PM,  wrote:

> Dear podling,
>
> This email was sent by an automated system on behalf of the Apache
> Incubator PMC. It is an initial reminder to give you plenty of time to
> prepare your quarterly board report.
>
> The board meeting is scheduled for Wed, 19 July 2017, 10:30 am PDT.
> The report for your podling will form a part of the Incubator PMC
> report. The Incubator PMC requires your report to be submitted 2 weeks
> before the board meeting, to allow sufficient time for review and
> submission (Wed, July 05).
>
> Please submit your report with sufficient time to allow the Incubator
> PMC, and subsequently board members to review and digest. Again, the
> very latest you should submit your report is 2 weeks prior to the board
> meeting.
>
> Thanks,
>
> The Apache Incubator PMC
>
> Submitting your Report
>
> --
>
> Your report should contain the following:
>
> *   Your project name
> *   A brief description of your project, which assumes no knowledge of
> the project or necessarily of its field
> *   A list of the three most important issues to address in the move
> towards graduation.
> *   Any issues that the Incubator PMC or ASF Board might wish/need to be
> aware of
> *   How has the community developed since the last report
> *   How has the project developed since the last report.
> *   How does the podling rate their own maturity.
>
> This should be appended to the Incubator Wiki page at:
>
> https://wiki.apache.org/incubator/July2017
>
> Note: This is manually populated. You may need to wait a little before
> this page is created from a template.
>
> Mentors
> ---
>
> Mentors should review reports for their project(s) and sign them off on
> the Incubator wiki page. Signing off reports shows that you are
> following the project - projects that are not signed may raise alarms
> for the Incubator PMC.
>
> Incubator PMC
>

Re: Is Apache Myriad dead?

[Adam Bordelon]

I can vet and vote too, but I don't have a lot of time these days to run a
release myself.

On Wed, Jun 7, 2017 at 4:33 AM, Darin Johnson 
wrote:

> Can you name the 3-5 active PMC members who will vet the next release?
>
> I'm willing to vet next release and contribute additional work we did to
> Myriad but only if I get a solid commitment from others.  Otherwise I'm
> happy to retire and let mapr host their fork.
>
> On Jun 6, 2017 2:29 AM, "Ted Dunning"  wrote:
>
> >
> >
> > On Tue, Jun 6, 2017 at 12:56 AM, Swapnil Daingade <
> > swapnil.daing...@gmail.com> wrote:
> >
> >> >> The problem is that there is essentially no real community that is
> >> happening.
> >>
> >> retiring doesn't help that
> >>
> >
> > The core problem here is lack of a viable PMC. A PMC has to have 3 active
> > members at any given point. Typically this requires about 8 live members.
> > Myriad is wildly short of that and thus will have serious problems doing
> > any releases.
> >
> >
> >>
> >> >> None of the engineers previously working on this will be working on
> >> this now. And that sort of situation isn't going to change.
> >>
> >> Events at MapR contributed to this situation. MapR scaled back its
> >> involvement in Myriad and all its committers left.
> >>
> >
> > Well, that is one way to look at it.
> >
> > On the other hand, if you actually were involved in the situations, you
> > would know that none of the committers left because they didn't get to
> work
> > on Myriad as part of their day jobs, nor did any of them feel enough
> > attachment to work after hours (as I do on my projects), nor did any of
> > them continue with the project after leaving for a new startup.
> >
> >
> >> MapR is of course free to take its own decisions. But it sounds like
> >> there is interest in working on Myriad, just not under the ASF umbrella.
> >> I feel without ASF, one company will have too much control on Myriad.
> >>
> >
> > The ASF is moving to retire Myriad because it can't make the cut as a
> > viable project. No company will have control over the Apache version of
> the
> > project at that point because the project is nothing to control.
> >
> > The desire to try to reboot the project outside of Apache has almost
> > everything to do with the fact that Apache processes and the lack of
> active
> > contributors means that nothing can happen. It isn't an end run around
> > Apache constraints for the purpose of control, it is an attempt to keep
> the
> > project alive at all.
> >
> >
> >> Ted, you yourself warned us against this
> >> http://www.zdnet.com/article/hadoop-veteran-ted-dunning-when
> >> -open-source-is-anything-but-open/
> >>
> >
> > Read the article. I warned about projects like Ambari. One company has
> all
> > of the PMC.
> >
> > At this point, the situation with Myriad is almost the opposite.
> >
> >
> >>
> >> >>That means that it will always be a distraction to get committers
> >> qualified as PMC so that they can approve releases and it will never
> really
> >> be possible to exit from incubation.
> >>
> >> I suggest we start with the contributions first.
> >>
> >
> > Can you name the 3-5 active PMC members who will vet the next release?
> >
> >
> >
> >>
> >>
> >> On Mon, Jun 5, 2017 at 2:53 PM, Ted Dunning 
> >> wrote:
> >>
> >>>
> >>> On Mon, Jun 5, 2017 at 10:15 PM, Swapnil Daingade <
> >>> swapnil.daing...@gmail.com> wrote:
> >>>
>  In that case I suggest we not retire
> 
>  >> "Darin - yes we've done more planning internally, and we do plan on
>  having some engineers spend some time on this project, doing some
> (minor)
>  maintenance for our customers."
> 
> >>>
> >>> The problem is that there is essentially no real community that is
> >>> happening.
> >>>
> >>> None of the engineers previously working on this will be working on
> this
> >>> now. And that sort of situation isn't going to change.
> >>>
> >>> That means that it will always be a distraction to get committers
> >>> qualified as PMC so that they can approve releases and it will never
> really
> >>> be possible to exit from incubation.
> >>>
> >>> Outside of the Apache limits, we can have a much more flexible
> structure
> >>> of who can commit. We don't plan to limit who can commit. In fact, we
> will
> >>> probably make it more open than an Apache project normally is.
> >>>
> >>>
> >>
> >
>

Re: Podling Report Reminder - June 2017

[Adam Bordelon]

I agree with the recommendation to retire. I'm happy to review if you write
it, otherwise I'll just fill in some of our recent stats at the last minute
like I usually do.

On Thu, Jun 1, 2017 at 4:58 AM, Darin Johnson 
wrote:

> I'm willing to write this, but I think it'll be with the recommendation to
> retire.  Does anyone else wish to volunteer?
>
> On Jun 1, 2017 7:43 AM,  wrote:
>
> > Dear podling,
> >
> > This email was sent by an automated system on behalf of the Apache
> > Incubator PMC. It is an initial reminder to give you plenty of time to
> > prepare your quarterly board report.
> >
> > The board meeting is scheduled for Wed, 21 June 2017, 10:30 am PDT.
> > The report for your podling will form a part of the Incubator PMC
> > report. The Incubator PMC requires your report to be submitted 2 weeks
> > before the board meeting, to allow sufficient time for review and
> > submission (Wed, June 07).
> >
> > Please submit your report with sufficient time to allow the Incubator
> > PMC, and subsequently board members to review and digest. Again, the
> > very latest you should submit your report is 2 weeks prior to the board
> > meeting.
> >
> > Thanks,
> >
> > The Apache Incubator PMC
> >
> > Submitting your Report
> >
> > --
> >
> > Your report should contain the following:
> >
> > *   Your project name
> > *   A brief description of your project, which assumes no knowledge of
> > the project or necessarily of its field
> > *   A list of the three most important issues to address in the move
> > towards graduation.
> > *   Any issues that the Incubator PMC or ASF Board might wish/need to be
> > aware of
> > *   How has the community developed since the last report
> > *   How has the project developed since the last report.
> > *   How does the podling rate their own maturity.
> >
> > This should be appended to the Incubator Wiki page at:
> >
> > https://wiki.apache.org/incubator/June2017
> >
> > Note: This is manually populated. You may need to wait a little before
> > this page is created from a template.
> >
> > Mentors
> > ---
> >
> > Mentors should review reports for their project(s) and sign them off on
> > the Incubator wiki page. Signing off reports shows that you are
> > following the project - projects that are not signed may raise alarms
> > for the Incubator PMC.
> >
> > Incubator PMC
> >
>

Re: Sync today?

[Adam Bordelon]

Darin and I met briefly, and decided to cancel these regular syncs going
forward. There's been such low attendance that they haven't been very
useful lately. We can continue to discuss things over the mailing lists, or
schedule adhoc meetings to discuss specific topics.
Thanks for participating!

On Wed, Mar 22, 2017 at 9:06 AM, Yuliya  wrote:

> Sorry, traveling
>
> > On Mar 22, 2017, at 8:10 AM, Darin Johnson 
> wrote:
> >
> > Just wondering.
> >
> > Darin
>

Re: Sync today?

[Adam Bordelon]

Sorry, my previous meeting ran late.

On Wed, Mar 8, 2017 at 9:06 AM, Darin Johnson 
wrote:

> Tried joining.
>

Canceling this week's Myriad sync

[Adam Bordelon]

Many people are off for the holidays. We'll pick up again in two weeks, on
1/11.

Cheers,
-Adam-

Re: Unable to join dev sync

[Adam Bordelon]

Sorry, I just joined, and you should be able to now as well.

On Wed, Dec 14, 2016 at 9:08 AM, Darin Johnson 
wrote:

> Eom
>

Re: Hangout today

[Adam Bordelon]

Nothing from me either. See you guys next time.

On Wed, Nov 16, 2016 at 11:06 AM, yuliya Feldman <
yufeld...@yahoo.com.invalid> wrote:

> Sorry - could not attend - was on the road
>
>   From: Darin Johnson 
>  To: Dev 
>  Sent: Wednesday, November 16, 2016 9:33 AM
>  Subject: Hangout today
>
> Missed due to another meeting going long.  Anyone else attend?
>
>
>
>

Re: [DISCUSS] handling roles in Myriad code

[Adam Bordelon]

If you just add both sets of resources as is to the launchTask call with
the proper offerId(s), you should be able to launch a single task that uses
both sets of resources. I wouldn't advise modifying the roles fields on the
Resources, in case Mesos looks for an exact match with known sets of
resources. Just copy the resources from the offer and reduce the cpu/mem
values or select ports from the range if necessary.

On Wed, Nov 2, 2016 at 4:44 AM, Darin Johnson <dbjohnson1...@gmail.com>
wrote:

> Adam to clarify with an example. An offer may have an offer with reserved
> resources (role hadoop) 2gb Mem, 1 CPU and 31000-31005 ports, and *
> resources 2 GB Mem 3 CPU and 31500-31999 ports.
> If one wants to create a task using 4 CPU and 3 GB of memory 31000 and
> 35001 ports, does one have to add resources of 2gb Mem, 1 CPU, 35001 port
> of role hadoop and 3 CPU and 2 GB 31000 of role * (or empty)?
> We had issues with storm not being able to accept both reserved and
> unreserved resources leading to not being able to use all memory.  We
> worked with Brenden to correct the issue and the solution was to add the
> two resources to the task.  Failure to add both resources and simply using
> one resource 4 CPU and 3 GB 31000 and 35001 resulted in TaskFailed.
>
> Darin
>
> On Nov 2, 2016 5:54 AM, "Adam Bordelon" <a...@mesosphere.io> wrote:
>
> > Sorry for the delayed response. If you haven't already, I'd recommend
> > reading https://mesos.apache.org/documentation/latest/roles/
> > Beyond that, let me try to clear up a few things:
> >
> > ==FrameworkInfo.role
> > 1. Every framework registers with Mesos with a role. If you don't specify
> > one, Mesos defaults the framework to '*'. Reservations and quota cannot
> be
> > assigned to "*", but it is in the list (with weight=1) when DRF
> calculates
> > which "role" is furthest below its fair share, to decide which role's
> > framework(s) should get the next offer.
> > 2. At offer time, every resource offered to a framework is allocated (in
> > the Mesos allocator) to the framework's role, regardless of reservations.
> > This is how Mesos determines a role's current usage vs. its "fair share".
> > If the framework declines the offer, then the resource is "recovered" or
> > deallocated from the framework's role. If the framework launches a task
> > with the resource, it is not recovered until the task exits. Mesos
> > considers offered resources when calculating current "usage" so that
> > frameworks cannot hoard offers to take over the cluster.
> >
> > ==Resource.role
> > 3.  Straight from the protobuf: "The role that this resource is reserved
> > for. If "*", this indicates that the resource is unreserved. Otherwise,
> the
> > resource will only be offered to frameworks that belong to this role."
> > 4. Resources offered to a framework may be reserved for that framework's
> > role, so that they are never offered to other frameworks in different
> > roles. These reserved resources will have role="foo" (for some value of
> foo
> > other than "*") set in their offers.
> > 5. Unreserved resources may be offered to any framework, regardless of
> what
> > role the framework registered with. Mesos uses weighted DRF to select the
> > role that is furthest below its fair share and offer the unreserved
> > resource to a framework registered with that role (using DRF between
> > frameworks in the same role).
> > 6.  When unreserved resources are offered, even if Mesos didn't set the
> > role field, the protobuf parser scheduler-side should set it to the
> default
> > "*". Either is equivalent to "unreserved".
> >
> > ==Implications for Myriad
> > 7. If there is code (like Yuliya references) checking if offered
> resources
> > are unreserved, we should be checking if role == "*", not role.isEmpty.
> Or
> > check for both if you like.
> > 8. There shouldn't be any reason to select reserved resources over
> > unreserved resources in the same offer (for the same agent). Mesos
> > allocates all resources to the framework's role when it makes the offer.
> > And Mesos ignores the role field in the resources selected for launching
> a
> > task. Mesos just verifies that the sum of the resources for the
> > executor/task do not exceed the available resources on that node.
> > 9. In fact, you can combine multiple offers (perhaps cached for a second
> or
> > two) from the same agent and launch one or more tasks in the same request
> > with a combination of the offers' resourc

Re: [DISCUSS] handling roles in Myriad code

[Adam Bordelon]

Sorry for the delayed response. If you haven't already, I'd recommend
reading https://mesos.apache.org/documentation/latest/roles/
Beyond that, let me try to clear up a few things:

==FrameworkInfo.role
1. Every framework registers with Mesos with a role. If you don't specify
one, Mesos defaults the framework to '*'. Reservations and quota cannot be
assigned to "*", but it is in the list (with weight=1) when DRF calculates
which "role" is furthest below its fair share, to decide which role's
framework(s) should get the next offer.
2. At offer time, every resource offered to a framework is allocated (in
the Mesos allocator) to the framework's role, regardless of reservations.
This is how Mesos determines a role's current usage vs. its "fair share".
If the framework declines the offer, then the resource is "recovered" or
deallocated from the framework's role. If the framework launches a task
with the resource, it is not recovered until the task exits. Mesos
considers offered resources when calculating current "usage" so that
frameworks cannot hoard offers to take over the cluster.

==Resource.role
3.  Straight from the protobuf: "The role that this resource is reserved
for. If "*", this indicates that the resource is unreserved. Otherwise, the
resource will only be offered to frameworks that belong to this role."
4. Resources offered to a framework may be reserved for that framework's
role, so that they are never offered to other frameworks in different
roles. These reserved resources will have role="foo" (for some value of foo
other than "*") set in their offers.
5. Unreserved resources may be offered to any framework, regardless of what
role the framework registered with. Mesos uses weighted DRF to select the
role that is furthest below its fair share and offer the unreserved
resource to a framework registered with that role (using DRF between
frameworks in the same role).
6.  When unreserved resources are offered, even if Mesos didn't set the
role field, the protobuf parser scheduler-side should set it to the default
"*". Either is equivalent to "unreserved".

==Implications for Myriad
7. If there is code (like Yuliya references) checking if offered resources
are unreserved, we should be checking if role == "*", not role.isEmpty. Or
check for both if you like.
8. There shouldn't be any reason to select reserved resources over
unreserved resources in the same offer (for the same agent). Mesos
allocates all resources to the framework's role when it makes the offer.
And Mesos ignores the role field in the resources selected for launching a
task. Mesos just verifies that the sum of the resources for the
executor/task do not exceed the available resources on that node.
9. In fact, you can combine multiple offers (perhaps cached for a second or
two) from the same agent and launch one or more tasks in the same request
with a combination of the offers' resources. For example, with two
different offers of 2GB on the same agent, you can call launchTask with the
list of two offers to launch a 1GB task and a 3GB task.
10. That said, given an offer of reserved resources on one node and
unreserved resources on another, the Myriad scheduler might want to choose
a) the reserved resources, to be a good citizen and be more likely to
respawn on a reserved node where it already has cached
data/binaries/images; or b) the unreserved resources, so that Myriad can
greedily claim more of the cluster beyond its reserved minimum.

==The Future: Multi-role frameworks
11. In a future Mesos, we plan to allow multi-role frameworks, so that a
framework can register to receive reserved resources from multiple
different roles, and subscribe to their quotas as well. That's currently in
design, but will take months to implement.

Hope this helps. I've numbered my points in case you have questions about
any of it.
Cheers,
-Adam-


On Fri, Oct 28, 2016 at 4:10 PM, yuliya Feldman  wrote:

> We clearly need a word from Adam, Mohit, Ken
> My impression is that Myriad will not get any resources that are not
> specific to the role it has (or entitled to), so we may not need much roles
> manipulation in Myriad code.
> Just my 2c of gut feelings :)
> Thanks,Yuliya
>
>   From: Darin Johnson 
>  To: Dev 
>  Sent: Friday, October 28, 2016 2:54 PM
>  Subject: Re: [DISCUSS] handling roles in Myriad code
>
> Any word from Adam or Mohit?
>
> On Oct 20, 2016 12:36 AM, "Klaus Ma"  wrote:
>
> > I can help on this discussion; I used to be Mesos contributor for a year
> > :).
> >
> > Mesos allocate regular resources based on role by DRF; and role is also
> > used for reservation & quotas. So, the framework (like Myriad), may get
> two
> > kind of resources: "*" or "myriad-s role". When Myriad launch tasks, it
> can
> > not overuse any kind of resources: for example, if Myarid got offers:
> > cpu(*):1;cpu(myriad):1, Myriad can not launch tasks by 

Re: Do we have a call today? I am not able to get it

[Adam Bordelon]

I'm stuck in another meeting, but Miguel is waiting in
https://hangouts.google.com/hangouts/_/mesosphere.io/myriad

On Wed, Sep 21, 2016 at 9:12 AM, yuliya Feldman  wrote:

>
>

Re: Sync tomorrow?

[Adam Bordelon]

We try to meet every other Wednesday at 9am Pacific.
See agenda/notes at
https://docs.google.com/a/mesosphere.io/document/d/1JGmJrgeg98bHw_0_sSRmyX6WiAe13OdErcFlaz6Aa04/edit

Hangout:
https://plus.google.com/hangouts/_/mesosphere.io/myriad?hceid=YWRhbUBtZXNvc3BoZXJlLmlv.f941u4avg2vf4319366dajm73o=0

On Tue, Aug 9, 2016 at 5:41 PM, Sam Chen <rc...@linkernetworks.com> wrote:

> Guys,
> What time? By how?
>
> Regards,
> Sam
>
> Sent from my iPhone
>
> > On Aug 10, 2016, at 8:32 AM, Adam Bordelon <a...@mesosphere.io> wrote:
> >
> > Yeah, I'll be there. Don't have a lot of news though.
> >
> > On Tue, Aug 9, 2016 at 5:21 PM, Darin Johnson <dbjohnson1...@gmail.com>
> > wrote:
> >
> >> Trying to plan my day tomorrow.
> >
>
>

Re: Sync tomorrow?

[Adam Bordelon]

Yeah, I'll be there. Don't have a lot of news though.

On Tue, Aug 9, 2016 at 5:21 PM, Darin Johnson 
wrote:

> Trying to plan my day tomorrow.
>

Re: Sync tomorrow?

[Adam Bordelon]

Yes, I'll be there. Sorry for not making it last time.

On Tue, Jul 26, 2016 at 11:20 AM, Darin Johnson 
wrote:

> Is there going to be a sync tomorrow?
>
> Darin
>

Re: Website is updated, 0.2.0 is official!

[Adam Bordelon]

Great work Darin and everyone! I'm really proud of the progress we've made,
and I look forward to the continued growth of the Myriad project and
community.

On Wed, Jun 29, 2016 at 6:31 PM, Santosh Marella 
wrote:

> Awesome! Congratulations on the release.
>
> Special kudos to Darin! Bravo!!
> On Jun 29, 2016 6:11 PM, "Swapnil Daingade" 
> wrote:
>
> > Congratulations!!! everyone and special thanks to Darin.
> >
> > Also, How did we get https://myriad.apache.org ?
> > I thought it was for TLPs :)
> >
> > Regards
> > Swapnil
> >
> >
> > On Wed, Jun 29, 2016 at 5:24 PM, Sarjeet Singh <
> sarjeetsi...@maprtech.com>
> > wrote:
> >
> > > Great! Good job everyone and special thanks to Darin for the release :)
> > >
> > > -Sarjeet
> > >
> > > On Wed, Jun 29, 2016 at 2:05 PM, Darin Johnson <
> dbjohnson1...@gmail.com>
> > > wrote:
> > >
> > > > http://myriad.apache.org/
> > > >
> > > > Tell your friends!
> > > >
> > >
> >
>

[Update] Myriad Dev Sync

[Adam Bordelon]

Reminder: Myriad Dev Sync tomorrow!

You have been invited to the following event.

Title: Myriad Dev Sync
Biweekly meeting for the Myriad developer community to discuss:
- Announcements and Current Events (conferences, meetups, etc.)
- Incubator Updates
- Pull Requests in Review
- Features/issues in progress

Agenda/Notes:  
https://docs.google.com/a/mesosphere.io/document/d/1JGmJrgeg98bHw_0_sSRmyX6WiAe13OdErcFlaz6Aa04/edit

When: Wed Jun 29, 2016 9am – 9:50am Pacific Time
Video call: https://plus.google.com/hangouts/_/mesosphere.io/myriad  
<https://plus.google.com/hangouts/_/mesosphere.io/myriad?hceid=YWRhbUBtZXNvc3BoZXJlLmlv.f941u4avg2vf4319366dajm73o>

Who:
* Adam Bordelon - organizer
* dev@myriad.incubator.apache.org
* mo...@mesosphere.io
* Ken Sipe
Attachments:
* Myriad Sync Notes -  
https://docs.google.com/a/mesosphere.io/document/d/1JGmJrgeg98bHw_0_sSRmyX6WiAe13OdErcFlaz6Aa04/edit?usp=drive_web

Re: Myriad hangout tomorrow?

[Adam Bordelon]

I'm in.

On Tue, Jun 14, 2016 at 1:31 PM, Darin Johnson 
wrote:

> I'm planning on calling in.
>
> Darin
>
> On Tue, Jun 14, 2016 at 4:25 PM, Swapnil Daingade <
> swapnil.daing...@gmail.com> wrote:
>
> > Hi All,
> >
> > Was wondering if we have a Myriad hangout tomorrow.
> >
> > Regards
> > Swapnil
> >
>

Re: Question on Mesos Reservation

[Adam Bordelon]

The Myriad framework can register with a Mesos role if you set the
'frameworkRole'
config param. Then you can configure Mesos agents to have static
reservations, or use the operator endpoints to grant dynamic reservations
to that role, or configure a minimum resource quota for that role. See
http://mesos.apache.org/documentation/latest/roles/ and
http://mesos.apache.org/documentation/latest/reservation/ and
http://mesos.apache.org/documentation/latest/quota/

There is currently no support yet for Myriad to create dynamic reservations
on its own (I just created MYRIAD-221
 to track it), nor for
Myriad to take advantage of revocable resources (MYRIAD-179
). These would be great
features for Myriad 0.3, but I don't think we have anybody committed to
work on them yet. Your comments/help are appreciated.

On Tue, Jun 14, 2016 at 2:24 AM, Sam Chen  wrote:

> Guys,
> Talking about the latest Myriad distribution, can it support the feature
> of Mesos Reservation? If no, what's the roadmap and plan on this?
> Appreciated.
>
> Regards,
> Sam
>
> Sent from my iPhone

Re: [Vote] Release apache-myriad-0.2.0-incubating (release candidate 4)

[Adam Bordelon]

Looks like we got our 3rd binding vote! Let's announce the result and ask
Incubator PMC to begin their vote. Darin, let me/Santosh know if you need
advice on this part of the process.


On Thu, Jun 9, 2016 at 5:44 PM, Brandon Gulla 
wrote:

> +1
>
> built and tested on a test cluster. great work guys.
>
> On Thu, Jun 9, 2016 at 6:39 PM, mohit soni 
> wrote:
>
> > +1 (Binding)
> >
> > - Verified signature
> > - Verified MD5 and SHA512 hashes
> > - Builds from source tar ball.
> > - Installed Myriad on a Mesos cluster and ran sanity tests.
> >
> > Thanks
> > Mohit
> >
> > On Thu, Jun 2, 2016 at 7:25 AM, John Yost  wrote:
> >
> > > I'm voting +1
> > >
> > > --John
> > >
> > > On Tue, May 24, 2016 at 10:46 PM, Darin Johnson <
> dbjohnson1...@gmail.com
> > >
> > > wrote:
> > >
> > > > I'm voting +1 (Binding)
> > > >
> > > > Verified md5/sha hashes.  Compiled with gradle build, gradle
> > > buildRMDocker
> > > > (on OSX with docker-machine).
> > > >
> > > > Ran remote distribution (with cgroups) on a 4 node cluster (Ubuntu,
> > > > hadoop-2.6.0, hadoop 2.7.0) with one CGS NM and 3 FGS NM.  Ran 8
> > > > simultaneous jobs.  Shut down Framework.  Restarted NodeManager, ran
> an
> > > > additional 3 jobs.
> > > >
> > > > Ran the same with docker (minus cgroups).
> > > >
> > > > Darin
> > > >
> > > > On Tue, May 24, 2016 at 10:40 PM, Darin Johnson <
> > dbjohnson1...@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > I have created a source tar ball for Apache Myriad
> 0.2.0-incubating,
> > > > > release candidate 3 based off the feed back received from release
> > > > > candidate 1,2 & 3.  Thanks Sarjeet for a very thorough review!
> > > > >
> > > > > Here’s the release notes:
> > > > > https://cwiki.apache.org/confluence/display/MYRIAD/Release+Notes
> > > > >
> > > > > The commit to be voted upon is tagged with
> > > "myriad-0.2.0-incubating-rc4"
> > > > > and is available here:
> > > > >
> > > > >
> > > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=incubator-myriad.git;a=shortlog;h=refs/tags/myriad-0.2.0-incubating-rc
> > > > > <
> > > >
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=incubator-myriad.git;a=shortlog;h=refs/tags/myriad-0.2.0-incubating-rc4
> > > > >
> > > > > 4
> > > > >
> > > > > The artifacts to be voted upon are located below. Please note that
> > this
> > > > is
> > > > > a source release:
> > > > >
> > > > >
> > > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/incubator/myriad/myriad-0.2.0-incubating-rc4/
> > > > >
> > > > > Release artifacts are signed with the following key:
> > > > > *https://home.apache.org/~darinj/gpg/2AAE9E3F.asc
> > > > > *
> > > > >
> > > > > **Please note that the release tar ball does not include the
> gradlew
> > > > script
> > > > > to build. You need to install gradle in order to build.**
> > > > >
> > > > > Please try out the release candidate and vote. The vote is open
> for a
> > > > > minimum of 3 business days (Friday May 27) or until the necessary
> > > number
> > > > > of votes (3 binding +1s)
> > > > > is reached.
> > > > >
> > > > > If/when this vote succeeds, I will call for a vote with IPMC
> seeking
> > > > > permission to release RC3 as Apache Myriad 0.2.0 (incubating).
> > > > >
> > > > > [ ] +1 Release this package as Apache Myriad 0.2.0-incubating
> > > > > [ ]  0 I don't feel strongly about it, but I'm okay with the
> release
> > > > > [ ] -1 Do not release this package because...
> > > > >
> > > > > Thanks,
> > > > > Darin
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Brandon
>

Re: request for write privileges for Myriad Confluence

[Adam Bordelon]

Updated. Darin and other committers should have access to add new users
with:
https://cwiki.apache.org/confluence/spaces/spacepermissions.action?key=MYRIAD

On Thu, May 26, 2016 at 6:13 AM, John Yost  wrote:

> Hi Everyone,
>
> I am working with Darin Johnson on Myriad and would like write access to
> our Confluence site to start adding FAQ topics.
>
> Thanks
>
> --John
>

Re: Showcase your project at ApacheCON: consider Podling Shark Tank

[Adam Bordelon]

Sounds great! Mohit and I will sign up.

On Mon, Apr 25, 2016 at 1:44 PM, Swapnil Daingade 
wrote:

> Folks going to ApacheCon might want to consider participating.
>
> Regards
> Swapnil
>
>
> -- Forwarded message --
> From: Roman Shaposhnik 
> Date: Mon, Apr 25, 2016 at 1:27 PM
> Subject: Showcase your project at ApacheCON: consider Podling Shark Tank
> To: r...@apache.org
>
>
> Hi!
>
> I apologize for the wide distribution, but since my email to general@
> didn't seem to have reached all of the podling communities, I took
> the liberty of direct email. Here's the deal:
>
> If you are (or anybody you know who's passionate about your project is)
> going to travel to Vancouver for ApacheCON we've got an awesome
> opportunity for you to showcase your project. Even if you don't have
> talks scheduled in the regular program, consider doing a lighting talk
> at Podling Shark Tank:
>  https://wiki.apache.org/apachecon/ACEU16PodlingSharkTank
>
> You've got nothing to lose (in fact, the opposite: you're likely to get
> a prize!) and you will get a chance to receive feedback that might
> actually help you grow your community and ultimately graduate to the
> TLP status. Given our awesome panel of judges:
>  * Jim Jagielski
>  * Shane Curcuru
>  * Milind Bhandarkar
> We guarantee this to be a fun and useful event for your community!
>
> Please sign up on the wiki ASAP. The time is running out!
>
> Thanks,
> Roman.
>
> P.S. If you have *any* questions whatsoever, but especially if you have
> questions on logistics please email me directly.
>

Re: Myriad Releases

[Adam Bordelon]

+1 to skipping 0.1.1 if 0.2.0 is coming soon enough
I don't think we have any production users eagerly awaiting the 0.1.1 fixes

On Wed, Apr 20, 2016 at 5:52 PM, Darin Johnson 
wrote:

> Hey Zachary! Thanks, for the upvote.
>
> Also if you're looking for projects ping me! I'm going to be adding more
> tickets in the next few days.
>
> On Wed, Apr 20, 2016 at 8:38 PM, Zachary Jaffee  wrote:
>
> > If I recall the original reason for the 0.1.1 release was that it would
> be
> > able to get it out earlier than the 0.2.0 release. Since it looks like
> they
> > will be released at the same time essentially, the reason to release
> 0.1.1
> > over just waiting to release 0.2.0 goes away.
> >
> > On Wed, Apr 20, 2016 at 5:29 PM, Darin Johnson 
> > wrote:
> >
> > > During the dev sync today we discussed the upcoming 0.1.1 and 0.2.0
> > > releases.  Currently, the only out standing issue for 0.1.1 is
> MYRIAD-192
> > > (Cgroups), for Myriad 0.2.0 the outstanding issues are Myriad-36 and
> > Myriad
> > > 192 (Configuration and Docker/Appc support).  Currently I have a WIP PR
> > for
> > > Docker Support which I'd like some feedback on (it's should be super
> easy
> > > to test), I'll probably complete Myriad 192 and part of that PR as
> it's a
> > > natural fit.  I estimate I can get all patches done by early may and
> > > hopefully get a release or release candidate out by May 11 (ApacheCon).
> > >
> > > Due to the Alpha nature of Myriad and the significant value of Docker
> and
> > > Configuration support, I think most people would opt for 0.2.0 over
> 0.1.1
> > > and don't feel it's worth the effort to provide both releases at this
> > > time.  I suggest simply doing a 0.2.0 release.  Are there any
> objections?
> > >
> > > Darin
> > >
> >
> >
> >
> > --
> > Zach Jaffee
> > B.S. Computer Science
> > Case Western Reserve University Class of 2017
> > Operations Director | WRUW FM 91.1 Cleveland
> > (917) 881-0646
> > zjaffee.com
> > linkedin.com/in/zjaffee
> > github.com/ZJaffee
> >
>

Re: Observations on Fine Grained Scaling

[Adam Bordelon]

The Mesos offer allocation cycle should happen once every second by default
(--allocation_interval), but if Myriad declines an offer (or doesn't use
all the resource in an offer), then those resources will be filtered from
being offered to that framework for 5sec (by default). We could reduce that
decline filter if necessary, but reducing it too much and always staying
below your fair share can prevent other frameworks from being offered those
resources.

On Wed, Apr 13, 2016 at 12:23 PM, Darin Johnson 
wrote:

> Santosh, I get a lot of 2-3 containers.  But I can only get 9-12 (Topped of
> the cpu resources at 12 cores) containers if a task runs for more than 30
> secs (preferably 60 secs), that's generally not an issue but I thought it
> was worth putting on the list for general knowledge.  It's also less of a
> deal when more jobs are running.
>
> I have a few ideas on how to improve it and data locality, but think that
> it'll likely involve refactoring YarnNodeCapacityManager and
> OfferLifeCycleManager to interfaces that can be extended to handle
> different strategies which can be configured at on startup.  I'd love to
> start that discussion once we finish getting the basic mechanics working.
> Maybe a 0.3.0 or 0.4.0 release?
>
> Darin
>
>
> On Wed, Apr 13, 2016 at 2:14 PM, Santosh Marella 
> wrote:
>
> > > After the patches it seems stable, I'm able to run multiple terasort/pi
> > > jobs and a few scalding jobs without difficulty.
> > Great work, Darin. Glad to see FGS is now stable.
> >
> > >Noticed with jobs with short map tasks (8-12 secs), I rarely got more
> > > than two containers per node, I'm curious if I'm not consuming
> resources
> > > fast enough.
> > Yes. Perhaps we need to tune the rate at which Mesos sends out resource
> > offers
> > to frameworks. The default that we observe in Myriad is 5 seconds.
> However,
> > if your
> > job has many map tasks and Mesos offer is big enough to accommodate
> several
> > of them,
> > then you should ideally see lot more than 2-3 containers per node.
> >
> > Isn't that happening? How many map tasks does your job have?
> >
> > Thanks,
> > Santosh
> >
> > On Wed, Apr 13, 2016 at 8:34 AM, Darin Johnson 
> > wrote:
> >
> > > I've been running a number of tests on the Fine Grained scaling aspect
> on
> > > Myriad.  Here's a few notes:
> > >
> > > 1. After the patches it seems stable, I'm able to run multiple
> > terasort/pi
> > > jobs and a few scalding jobs without difficulty.
> > > 2. Noticed with jobs with short map tasks (8-12 secs), I rarely got
> more
> > > than two containers per node, I'm curious if I'm not consuming
> resources
> > > fast enough.  The issue goes away on the reduce side (able to get far
> > > better utilization of offers).  The issue can be lessened by increasing
> > > mapred.splits.min.size and mapred.splits.max.size.  This may be an
> issue
> > > for things like Hive.
> > >
> > > Darin
> > >
> >
>

Re: Myriad 0.1.1 Release

[Adam Bordelon]

Thanks for heading this up Darin!

Here's a link to the JIRA Issues page for Myriad 0.1.1:
https://issues.apache.org/jira/browse/MYRIAD/fixforversion/12335455/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-issues-panel

Looking at MYRIAD-194, it seems to be a new feature request, and only
bugfixes should go into a patch release like 0.1.1, so I vote against
its inclusion. MYRIAD-191 could be worth including, depending on the
root-cause.

On Wed, Apr 6, 2016 at 11:06 AM, Darin Johnson  wrote:
> Hi,
>
> I'm the release manager for Myriad 0.1.1, which we're hoping to get out in
> the next couple weeks. Here's the list of PR's and JIRA's that I think
> should go into the release since 0.1.0:
>
> Complete:
> #56  MYRIAD-181 Build failure due to dependency on zookeeper test jar
> 
> #57 MYRIAD-153: tasks not finishing when FGS is enabled
> #60 MYRIAD-186 Clean up the build
> 
> #62 Myriad 188 - NodeManager switch to UNHEALTHY causes NPE
> 
> #63 MYRIAD-171, MYRIAD-190
> , compatibility issues
> with 2.7.1+ and 2.6.2+
>
> Todo:
> Myriad-192
>
> Possibly (Pending determination of issue):
> Myriad-194
> Myriad-191
>
> This is certainly open for discussion so if you think something should be
> added or removed please respond.  This is a fix release so no new features
> are to be added.  However, we plan to release 0.2.0 shortly after so new
> features can be added then.
>
> Darin

Re: Myriad talk link for MesosCon?

[Adam Bordelon]

Based on the count, that looks like all ~150 submissions.
I couldn't find any Myriad-specific, so I guess none of us submitted a
Myriad talk to MesosCon.
Found this though:
Experiences running HPC & Big Data frameworks on Cray Analytics Platform
"... Cray ..  leverage Apache Mesos and create interfaces to allow other
resources managers (Slurm and YARN) to dynamically acquire and release
resources from Mesos."
Not sure if they tried Mesos or used their own thing.


On Wed, Mar 23, 2016 at 10:17 AM, Darin Johnson 
wrote:

> Yeah I didn't see one either.
>
> Darin
>
> On Wed, Mar 23, 2016 at 1:10 PM, Sarjeet Singh 
> wrote:
>
> > I couldn't find any associated link of myriad talk for MesosCon voting.
> > Anyone?
> >
> > Though, I found these proposal doc:
> >
> > Developers: http://bit.ly/1RpZPvj
> > Users: http://bit.ly/1Mspaxp
> >
> >
> > *It seems the deadline for the proposal voting is today, March 23 2016.*
> >
> > -Sarjeet
> >
>

Re: 0.2.0 release

[Adam Bordelon]

+1 on Darin as release manager

I'd like to see 0.2 have:
- Usable FGS
- Dockerized NM (for multitenancy)

On Tue, Mar 15, 2016 at 9:46 AM, Darin Johnson 
wrote:

> We've talked about a 0.2.0 release slated for mid April at the dev sync.
> I'd like to nail down any features people would like and have time to work
> on.
>
> I've been spend some time fixing major bugs to the FGS feature and plan to
> address MYRIAD-136 and MYRIAD-189.
>
> I'd also be willing to be the release manager on this release if necessary.
>
> Darin
>

Re: Mesos and Myriad Kerberos Support

[Adam Bordelon]

We at Mesosphere have a proprietary implementation of Kerberos ticket
forwarding using various Mesos hooks/modules, but this is specific to a
particular customer use case. We're actively working on a way to pass
keytabs/credentials to spark-submit so that it can forward them on to HDFS
or other services. While this is still a specific use case (Spark -> HDFS),
we're exploring how to generalize this approach beyond just Kerberos.

On Wed, Feb 10, 2016 at 5:56 PM,  wrote:

> Hello guys,
>
> I wanted to follow up a little further on today’s Hangouts call about
> Kerberos. For everyone else who may have not been on the call the idea is
> if you have Spark, Myriad and some task running top of Mesos and it needs
> access to some third party service like HDFS that needs kerberos
> credentials how will that work?
>
> Adam has mentioned one solution he’s seen. This was to have credentials
> cached on the master that will then intercept the calls and annotate the
> task with their credentials and wrap the calls with something that unwraps
> the credentials and puts it into place to authenticate. This will require
> update the TGT as they expire.
>
> Adam, you’ve mentioned that is Mesosphere doing in this space as well, do
> you know if that is specific to Kerberos or something else? Any other
> suggestion will be helpful!
>
> Thanks!
>
>
> *Known Jiras regarding this adding kerberos support for Mesos
>
> https://issues.apache.org/jira/browse/MESOS-907
>
> > Miguel Bernadin Accenture Technology Labs – System Engineering
> Contact: W (408) 817-2742 | M (631) 835-6345 |
> miguel.berna...@accenture.com
>
> 
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited. Where allowed
> by local law, electronic communications with Accenture and its affiliates,
> including e-mail and instant messaging (including content), may be scanned
> by our systems for the purposes of information security and assessment of
> internal compliance with Accenture policy.
>
> __
>
> www.accenture.com
>

Re: Next dev sync hangout will be on 1/6/2016

[Adam Bordelon]

Sorry, this slipped off my calendar, so I didn't even try to attend. I
guess we'll pick up again next week?

On Wed, Jan 6, 2016 at 11:22 AM, Paul Curtis  wrote:

> I attempted to join as well  it seems that no one was in the
> hangout. I gave up after about 15 minutes.
>
> paul
>
> On Wed, Jan 6, 2016 at 12:19 PM, Darin Johnson 
> wrote:
> > Can't seem to join...
> > On Jan 6, 2016 12:16 PM, "Darin Johnson" 
> wrote:
> >
> >> Trying to join
> >> On Jan 6, 2016 12:06 PM, "yuliya Feldman" 
> >> wrote:
> >>
> >>> Do we have a sync today?
> >>>
> >>>
> >>>   From: Santosh Marella 
> >>>  To: dev@myriad.incubator.apache.org
> >>>  Sent: Wednesday, December 16, 2015 9:47 AM
> >>>  Subject: Next dev sync hangout will be on 1/6/2016
> >>>
> >>> We have decided to hold the next dev sync on 1/6/2016 (instead of
> >>> 12/30/2015).
> >>>
> >>> Meeting notes from today's hangout:
> >>>
> >>>
> https://docs.google.com/document/d/1JGmJrgeg98bHw_0_sSRmyX6WiAe13OdErcFlaz6Aa04/edit#
> >>>
> >>> Thanks,
> >>> Santosh
> >>>
> >>>
> >>>
> >>
> >>
>
>
>
> --
> Paul Curtis - Senior Product Technologist
> O: +1 203-660-0015 - M: +1 203-539-9705
>
> Now Available - Free Hadoop On-Demand Training
>

Re: Myriad is 0.1.0

[Adam Bordelon]

Hooray! Time for champagne!



On Thu, Dec 10, 2015 at 1:13 AM, Santosh Marella 
wrote:

> Hi All,
>
>   Congratulations on a the first Apache Myriad release..! Kudos to everyone
> involved for making this happen.
>
>   As we now have IPMC's approval, there are a few things that I did to wrap
> up the release:
>   - Make 0.1.0 artifacts available from the release SVN repo [1].
>   - Git tag the voted RC as the "0.1.0" release [2].
>   - Delete the previously marked git RC tags.
>   - Closed the remaining JIRAs marked for 0.1.0 version and marked the
> 0.1.0 version as "released" [3].
>   - Submitted a PR [4] with scripts to prepare a RC and release a RC
> (automates the above git and svn steps)
>   - Updated the release guide [5] with voting links to help future release
> managers.
>
>   Here are a couple of things still remaining:
>   - Update the downloads page on Myriad's website with links to 0.1.0
> artifacts on svn, git tag, release notes.
>   - Do an announcement blog post. Here is a draft [6]. Please suggest any
> changes.
>
>   If I may be missing something for 0.1.0, appreciate if you could bring to
> my notice.
>
>1.
>
> https://dist.apache.org/repos/dist/release/incubator/myriad/myriad-0.1.0-incubating/
>2.
>
> https://github.com/apache/incubator-myriad/releases/tag/myriad-0.1.0-incubating
>3.
>
> https://issues.apache.org/jira/browse/MYRIAD/?selectedTab=com.atlassian.jira.jira-projects-plugin:versions-panel
>4. https://github.com/apache/incubator-myriad/pull/53
>5. https://cwiki.apache.org/confluence/display/MYRIAD/Release+Guide
>6.
>
> https://docs.google.com/document/d/1zCXnDlqzNhj0BL_CqRz5-poCap9QFah7R3tKkHdspYg/edit
>
> Thanks,
> Santosh
>

Re: Oversubscription in Mesos

[Adam Bordelon]

Sounds great! The all-revocable vs. all-guaranteed strategy seems like the
easiest to implement (just add the Capability to FrameworkInfo), but making
Myriad itself smart about high-priority vs. best-effort backfill jobs.

On Wed, Dec 2, 2015 at 6:27 AM, John Omernik  wrote:

> Hey all, just curious if there has been any discussion around supporting
> oversubscription in Myriad.  Based on my reading of things, Myriad would be
> an awesome use case for over subscription, especially when you combine it
> with the FGS.  Based on what I've read on oversubscription, if Myriad was
> aware of oversubscription, we could have Myriad be smart about various Yarn
> containers, and have some jobs that may be production jobs, they could run
> on non-revocable resources, but could we have yarn jobs with certain
> users/flags, especially in FGS mode be submitted using the revocable
> resources?   These are the jobs that would be adhoc in nature, and in
> addition to not using resources when no jobs are running, the node
> managers, when they did run certain jobs would run on the revocable
> resources.
>
> I am speaking now not from a Dev perspective, so this may be a lot harder
> than it seems.
>
> Another approach would be once we have the the multi-tenancy built in, have
> a whole myriad framework dedicate to adhoc type jobs, and have another
> myriad framework dedicated to production jobs.
>
> I see use cases for both, this just seems to add another layer of awesome
> flexibility as it pertains to jobs on the cluster.
>
> I'd be interested in the group's thoughts here.
>
> John
>

Re: Merges to Myriad master branch

[Adam Bordelon]

Sounds like a code freeze/thaw. What are the conditions for a *major* PR?
Even a minor PR could introduce major bugs.
I will point out that you have the option of cherry-picking specific new
patches on top of the 0.1.0-rc3 tag to create a new rc. This ensures that
0.1.0 only includes changes that were tested in the previous rc's plus
specific critical fixes. This is how Mesos handles patch releases (e.g.
0.23.0 -> 0.23.1) or release candidates after the first. Cut the rc0/1 from
HEAD, then cherry-pick on top for all future rcs.

On Mon, Nov 30, 2015 at 12:42 PM, Santosh Marella 
wrote:

> Hello All,
>
>   As we are in the middle of IPMC voting to release myriad-0.1.0, I request
> the committers to hold off merging any *major* PRs to the "master" branch.
>
>  The current RC has been tested well for the features that went into it.
> Just in case we receive more feedback from IPMC, we want the "master"
> branch to stay as close to current RC as possible so that when we cut a new
> RC with IPMC's feedback, we don't need to worry too much about the
> stability of the new RC.
>
> Thanks,
> Santosh
>

Re: [Help needed from Mentors] SVN PubSub

[Adam Bordelon]

Thanks, Luciano. And yes, I am m...@apache.org

On Tue, Nov 17, 2015 at 8:11 AM, Luciano Resende <luckbr1...@gmail.com>
wrote:

> I have pushed a "myriad" branch, and waiting for Infra to review/merge.
>
> On Tue, Nov 17, 2015 at 7:55 AM, Santosh Marella <smare...@maprtech.com>
> wrote:
>
>> More Apache Ids need to be added: me,darinj,sdaingade
>>
>> --
>> Sent from mobile
>> On Nov 17, 2015 7:44 AM, "Luciano Resende" <luckbr1...@gmail.com> wrote:
>>
>> > Are we missing anyone ?
>> >
>> > myriad=benh,danese,kensipe,lresende,mohit,smarella,tdunning
>> >
>> >
>> > On Mon, Nov 16, 2015 at 5:23 AM, Benjamin Hindman <b...@mesosphere.io>
>> > wrote:
>> >
>> > > I've never edited an "svn authorization template" before, is there
>> some
>> > > documentation that one of the more experienced mentors can point us/me
>> > to?
>> > >
>> > > On Fri, Nov 13, 2015 at 7:47 AM, Adam Bordelon <a...@mesosphere.io>
>> > wrote:
>> > >
>> > >> Correct. As mentioned in the Nov 4-5 comments on INFRA-10696
>> > >> <
>> >
>> https://issues.apache.org/jira/browse/INFRA-10696?focusedCommentId=14990668=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14990668
>> > >,
>> > >> we got 403 Forbidden errors when trying to commit. Geoffrey from
>> INFRA
>> > >> tells us, "You need to get your mentor (or someone with enough
>> karma) to
>> > >> edit the svn authorization template and add a list of names from
>> myriad
>> > >> that you want to be able to commit."
>> > >>
>> > >> On Thu, Nov 12, 2015 at 6:37 PM, Jim Klucar <klu...@gmail.com>
>> wrote:
>> > >>
>> > >>> Ted,
>> > >>>
>> > >>> Santosh and Adam both tried to commit to the svn repo but got
>> > permission
>> > >>> denied. Looks like they just need some kind of svn permissions to
>> push
>> > our
>> > >>> site out.
>> > >>>
>> > >>> Jim
>> > >>>
>> > >>> On Thursday, November 12, 2015, Ted Dunning <ted.dunn...@gmail.com>
>> > >>> wrote:
>> > >>>
>> > >>>> OK.
>> > >>>>
>> > >>>> I am really confused.  I hear you need SVN something.  And you need
>> > some
>> > >>>> kind of web action.
>> > >>>>
>> > >>>> What specifically do you need?
>> > >>>>
>> > >>>> (I am heavily loaded right now and in a difficult time zone)
>> > >>>>
>> > >>>>
>> > >>>> On Fri, Nov 13, 2015 at 5:34 AM, Adam Bordelon <a...@mesosphere.io
>> >
>> > >>>> wrote:
>> > >>>>
>> > >>>> > + Mentor email addresses, in case filters are failing us.
>> > >>>> > Any mentor, help! We still need help with this.
>> > >>>> >
>> > >>>> > Don't you want to see this content:
>> > >>>> http://klucar.github.io/myriad-site/
>> > >>>> > at this address: http://myriad.incubator.apache.org/
>> > >>>> > ?
>> > >>>> >
>> > >>>> > On Tue, Nov 10, 2015 at 9:34 AM, Santosh Marella <
>> > >>>> smare...@maprtech.com>
>> > >>>> > wrote:
>> > >>>> >
>> > >>>> >> Gentle reminder... Appreciate your help with this.
>> > >>>> >>
>> > >>>> >> --
>> > >>>> >> Sent from mobile
>> > >>>> >> On Nov 8, 2015 7:02 PM, "Santosh Marella" <
>> smare...@maprtech.com>
>> > >>>> wrote:
>> > >>>> >>
>> > >>>> >> > Hi Ben,Danese,Luciano,Ted,
>> > >>>> >> >
>> > >>>> >> >   In INFRA-10696 [1], we've requested for a SVN directory to
>> > >>>> publish
>> > >>>> >> > Myriad's website. However, it seems to need a svn
>> authorization
>> > >>>> >> template to
>> > >>>> >> > be submitted by one of the Mentors in order to push website
>> > >>>> commits to
>> > >>>> >> SVN.
>> > >>>> >> > Can one of you please help with that?
>> > >>>> >> >
>> > >>>> >> >   [1] https://issues.apache.org/jira/browse/INFRA-10696
>> > >>>> >> >
>> > >>>> >> > Thanks,
>> > >>>> >> > Santosh
>> > >>>> >> >
>> > >>>> >>
>> > >>>> >
>> > >>>> >
>> > >>>>
>> > >>>
>> > >>
>> > >
>> > >
>> > > --
>> > > Benjamin Hindman
>> > > Founder of Mesosphere and Co-Creator of Apache Mesos
>> > > Mesosphere Inc.  <http://www.mesosphere.io/>
>> > >
>> > > Follow us on Twitter: @mesosphere <http://twitter.com/mesosphere>
>> > >
>> >
>> >
>> >
>> > --
>> > Luciano Resende
>> > http://people.apache.org/~lresende
>> > http://twitter.com/lresende1975
>> > http://lresende.blogspot.com/
>> >
>>
>
>
>
> --
> Luciano Resende
> http://people.apache.org/~lresende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [Help needed from Mentors] SVN PubSub

[Adam Bordelon]

Correct. As mentioned in the Nov 4-5 comments on INFRA-10696
<https://issues.apache.org/jira/browse/INFRA-10696?focusedCommentId=14990668=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14990668>,
we got 403 Forbidden errors when trying to commit. Geoffrey from INFRA
tells us, "You need to get your mentor (or someone with enough karma) to
edit the svn authorization template and add a list of names from myriad
that you want to be able to commit."

On Thu, Nov 12, 2015 at 6:37 PM, Jim Klucar <klu...@gmail.com> wrote:

> Ted,
>
> Santosh and Adam both tried to commit to the svn repo but got permission
> denied. Looks like they just need some kind of svn permissions to push our
> site out.
>
> Jim
>
> On Thursday, November 12, 2015, Ted Dunning <ted.dunn...@gmail.com> wrote:
>
>> OK.
>>
>> I am really confused.  I hear you need SVN something.  And you need some
>> kind of web action.
>>
>> What specifically do you need?
>>
>> (I am heavily loaded right now and in a difficult time zone)
>>
>>
>> On Fri, Nov 13, 2015 at 5:34 AM, Adam Bordelon <a...@mesosphere.io>
>> wrote:
>>
>> > + Mentor email addresses, in case filters are failing us.
>> > Any mentor, help! We still need help with this.
>> >
>> > Don't you want to see this content:
>> http://klucar.github.io/myriad-site/
>> > at this address: http://myriad.incubator.apache.org/
>> > ?
>> >
>> > On Tue, Nov 10, 2015 at 9:34 AM, Santosh Marella <smare...@maprtech.com
>> >
>> > wrote:
>> >
>> >> Gentle reminder... Appreciate your help with this.
>> >>
>> >> --
>> >> Sent from mobile
>> >> On Nov 8, 2015 7:02 PM, "Santosh Marella" <smare...@maprtech.com>
>> wrote:
>> >>
>> >> > Hi Ben,Danese,Luciano,Ted,
>> >> >
>> >> >   In INFRA-10696 [1], we've requested for a SVN directory to publish
>> >> > Myriad's website. However, it seems to need a svn authorization
>> >> template to
>> >> > be submitted by one of the Mentors in order to push website commits
>> to
>> >> SVN.
>> >> > Can one of you please help with that?
>> >> >
>> >> >   [1] https://issues.apache.org/jira/browse/INFRA-10696
>> >> >
>> >> > Thanks,
>> >> > Santosh
>> >> >
>> >>
>> >
>> >
>>
>

Re: [VOTE] Release apache-myriad-0.1.0-incubating (release candidate 0)

[Adam Bordelon]

Do we need to rev the version on the executor jars up to 0.1.0?
Is there anywhere else we need to bump the version?

$ grep -rin "myriad.*[^ \[\(0-9]0" . |grep -v Apache
./docker/Dockerfile:34:ADD /libs/myriad-executor-runnable-0.0.1.jar
/usr/local/libexec/mesos/
./docker/ResourceManager.dockerfile:30:ADD
/libs/myriad-executor-runnable-0.0.1.jar /usr/local/libexec/mesos/
./docker/build-myriad.sh:40:cp -rf
../myriad-executor/build/libs/myriad-executor-runnable-0.0.1.jar libs/
./docs/vagrant.md:100
:/vagrant/myriad-scheduler/build/libs/myriad-executor-0.0.1.jar
./docs/vagrant.md:107:cp
/vagrant/myriad-executor/build/libs/myriad-executor-0.0.1.jar
$YARN_HOME/share/hadoop/yarn/lib/
./docs/vagrant.md:150:  path:
file://localhost/usr/local/libexec/mesos/myriad-executor-runnable-0.0.1.jar
./docs/myriad-dev.md:67:cp
myriad-executor/build/libs/myriad-executor-0.0.1.jar
/opt/hadoop-2.7.1/share/hadoop/yarn/lib/
./docs/myriad-remote-distribution-configuration.md:39:cp
myriad-executor/build/libs/myriad-executor-0.0.1.jar
/opt/hadoop-2.7.1/share/hadoop/yarn/lib/
./myriad-scheduler/src/main/resources/myriad-config-default.yml:55:  path:
file:///usr/local/libexec/mesos/myriad-executor-runnable-0.0.1.jar
./myriad-scheduler/src/main/resources/myriad-config-default.yml:58:  #path:
hdfs://namenode:port/dist/myriad-executor-runnable-0.0.1.jar
./myriad-scheduler/src/main/java/org/apache/myriad/configuration/MyriadConfiguration.java:55:
* path:
file://localhost/usr/local/libexec/mesos/myriad-executor-runnable-0.0.1.jar
./myriad-scheduler/src/test/resources/myriad-config-test-default.yml:31:
path: file:///usr/local/libexec/mesos/myriad-executor-runnable-0.0.1.jar
./myriad-scheduler/src/test/resources/myriad-config-test-default.yml:34:
#path: hdfs://namenode:port/dist/myriad-executor-runnable-0.0.1.jar


On Mon, Nov 9, 2015 at 1:20 AM, Santosh Marella <smare...@maprtech.com>
wrote:

> I haven't created a git tag for the rc. I will try creating one in the
> morning.
>
> --
> Sent from mobile
> On Nov 9, 2015 1:10 AM, "Adam Bordelon" <a...@mesosphere.io> wrote:
>
> > Awesome! I'll try to build and run this myself this week.
> > Is there a git tag I can sync to for 0.1.0-rc0?
> >
> > On Mon, Nov 9, 2015 at 1:01 AM, Santosh Marella <smare...@maprtech.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > I have created a build for Apache Myriad 0.1.0-incubating, release
> > > candidate 0.
> > >
> > > Thanks to everyone who has contributed to this release.
> > >
> > > Here’s the release notes:
> > > https://cwiki.apache.org/confluence/display/MYRIAD/Release+Notes
> > >
> > > The commit to be voted upon is:
> > >
> > >
> >
> https://github.com/apache/incubator-myriad/commit/343a11aceb0fc9b5c84f46bba43b2423643eff6f
> > >
> > > The artifacts to be voted on are located here:
> > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/incubator/myriad/myriad-0.1.0-incubating-rc0/
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/smarella.asc
> > >
> > > Please vote on releasing this package as Apache Myriad
> 0.1.0-incubating.
> > >
> > > The vote is open for the next 72 hours and passes if a majority of
> > > at least three +1 PPMC votes are cast.
> > >
> > > [ ] +1 Release this package as Apache Myriad 0.1.0-incubating
> > > [ ]  0 I don't feel strongly about it, but I'm okay with the release
> > > [ ] -1 Do not release this package because...
> > >
> > >
> > > Here is my vote:
> > > +1 (binding)
> > >
> > > Thanks,
> > > Santosh
> > >
> >
>

Re: [DISCUSS] Release prep for Myriad 0.1.0

[Adam Bordelon]

Website sources don't need to be in the release tarball (would bloat it
unnecessarily), and don't need to block the release of the 0.1.0 binaries,
but we'll want the podling site up so we can announce our release on the
website.

On Fri, Nov 6, 2015 at 10:15 AM, Santosh Marella 
wrote:

> Starting this thread to discuss the last few items left before calling out
> for a vote.
>
> * Blockers *
>- MYRIAD-170 [1], reported last night.
>- MYRIAD-144 [2] (Not sure if this is a blocker, please read below)
>
> *Current Status*
> - MYRIAD-170: Swapnil is working on a fix.
> - MYRIAD-144: Podling Website is not up yet.
> - needs "svn authorization template" filled in by a Mentor
> - ** Ben/Danese/Luciano/Ted: We seek your help **
> - A running document of release guide is being prepared at [3]
> - My GPG  public key is available at [4] and [5]
> - A *test* RC candidate was prepared and uploaded to [6].
>   (*This is not the final RC*. I prepared it to get familiar with the
> process)
>
> * Discussion topics *
> - MYRIAD-144
>Not sure if this is a blocker. Should the "website" sources
> necessarily part of the release tarball?
>If not, then the SVN issues can be resolved independent of the
> voting process on a RC.
> - Does anyone have any other topics ?
>
> [1] https://issues.apache.org/jira/browse/MYRIAD-170
> [2] https://issues.apache.org/jira/browse/MYRIAD-144
> [3]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61324959
> [4] http://people.apache.org/keys/committer/smarella.asc
> [5] https://dist.apache.org/repos/dist/dev/incubator/myriad/KEYS
> [6]
>
> https://dist.apache.org/repos/dist/dev/incubator/myriad/myriad-0.1.0-incubating-rc0/
>
> Thanks,
> Santosh
>

New committer: Darin J

[Adam Bordelon]

The Podling Project Management Committee (PPMC) for Apache Myriad has asked
Darin to become a committer and PPMC member and we are pleased to announce
that he has accepted.

Please join me in welcoming Darin as a Myriad committer, and let's thank
him for all his contributions so far. Looking forward to more!

Cheers,
-Adam-

New Committer: Swapnil Daingade

[Adam Bordelon]

The Podling Project Management Committee (PPMC) for Apache Myriad has asked
Swapnil Daingade to become a committer and PPMC member and we are pleased
to announce that he has accepted.

Please join me in welcoming Swapnil as a Myriad committer, and let's thank
him for all his contributions so far. Looking forward to more!

Cheers,
-Adam-

Re: PRs and reviews

[Adam Bordelon]

FYI, I asked the infrastructure-dev mailing list about merging github PRs,
and these are the answers I got:
http://mail-archives.apache.org/mod_mbox/www-infrastructure-dev/201510.mbox/%3CCANyrgveNjTDWOXj6fM4v2hRhJXwB%3DS2ko8rZ_2if6AY2c-Knyw%40mail.gmail.com%3E
and
http://mail-archives.apache.org/mod_mbox/www-infrastructure-dev/201510.mbox/%3CCAPbPdOY%2BYq0VACKbqXx3%2B0zquD3Ro9t7ciECqEEiUtZ6JExrvw%40mail.gmail.com%3E

TL;DR:
- Cloudstack has a git-pr script:
https://github.com/apache/cloudstack/tree/master/tools/git
- jfarrell created a rbt extension: https://github.com/jfarrell/rbt-github
- Including "Fixes #123" in the commit message will tell asfbot to close
the PR/issue.


On Wed, Nov 4, 2015 at 4:18 PM, Jim Klucar  wrote:

> I'd like to discuss how we do PRs and code reviews. Right now we are using
> github most likely because that's how we did it before Apache. Now the code
> lives in the Apache hosted repo and we have Review Board available to us.
> My assumption is that we are individually more familiar with one way or the
> other and we could argue their virtues here, but I have another suggestion.
> Why not after this release goes out, we switch off github and use review
> board for the next release and then discuss?
>

Re: apache rat plugin

[Adam Bordelon]

+1 to automated checking. I don't want to have to remember to tell people
to add licenses to every new file.

On Thu, Oct 15, 2015 at 1:15 PM, Jim Klucar  wrote:

> I assume gradle can use Maven plugins? If so, there's a rat (Release Audit
> Tool) plugin that checks file headers for licenses. What do you think about
> using it?
>
> http://creadur.apache.org/rat/apache-rat-plugin/
>
> Jim
>

Re: Default Mesos version

[Adam Bordelon]

Certainly. Mesos only promises +/-1 version compatibility between the Mesos
master and the scheduler's libmesos, and between the agent process and an
executor's libmesos. So a modern (0.23+) Mesos may not work well with
Myriad. We should be building Myriad with at least Mesos 0.24 now.

On Wed, Oct 14, 2015 at 10:10 PM, yuliya Feldman <
yufeld...@yahoo.com.invalid> wrote:

> Hello here,
> Currently mesos version in build.gradle is 0.21. It is a bit outdated,
> considering Mesos 0.25 is out.
> Does it make sense to change default version from 0.21 to a fresher one?
> Thoughts?
> Thanks,Yuliya

Myriad 0.1.0 Release Planning

[Adam Bordelon]

Santosh has volunteered as the Myriad 0.1.0 release manager, and I will
assist.

As my first contribution, I submit these links:
https://incubator.apache.org/guides/releasemanagement.html
https://incubator.apache.org/incubation/Incubation_Policy.html#Releases
https://incubator.apache.org/guides/release.html
as well as the Mesos release guide as a reference (what I'm familiar with):
http://mesos.apache.org/documentation/latest/release-guide/

We are hoping to cut the first release candidate for 0.1.0 in a couple of
weeks, so please tag any blocker JIRAs with the "Myriad 0.1.0" version in
JIRA: https://issues.apache.org/jira/browse/MYRIAD/fixforversion/12333141

Mentors, we'd love your help with this process, since we'll need to get
this approved by the Incubator PMC before we can release.

Re: Code Migration to Apache Git

[Adam Bordelon]

Yup. I think we just need to remove the phase1 branch now and references to
it in the README.

On Fri, Oct 2, 2015 at 10:08 AM, mohit soni <mohitsoni1...@gmail.com> wrote:

> Hi Jim
>
> Thanks for identifying this, and bringing this up.
>
> As part of code migration, phase1 is already merged into master. See
> MYRIAD-143 (https://issues.apache.org/jira/browse/MYRIAD-143) for more
> details.
>
> Thanks
> Mohit
>
>
> On Mon, Sep 28, 2015 at 1:43 PM, Jim Klucar <klu...@gmail.com> wrote:
>
> > Want to go ahead and make the phase1 branch master to get back to more
> > standard git?
> >
> > doing a merge with --theirs should merge phase1 into master and make
> master
> > phase1.
> >
> > On Sun, Sep 27, 2015 at 9:37 PM, Ken Sipe <k...@mesosphere.io> wrote:
> >
> > > well done sir!
> > > > On Sep 25, 2015, at 5:27 PM, mohit soni <mohitsoni1...@gmail.com>
> > wrote:
> > > >
> > > > Hello Everyone!
> > > >
> > > > I was able to migrate the myriad code-base from
> > > > https://github.com/mesos/myriad to Apache's git. The migration
> > included
> > > all
> > > > the existing commit history, branches, etc.
> > > >
> > > > The code is hosted here:
> > > > * https://git-wip-us.apache.org/repos/asf?p=incubator-myriad.git
> > > > * Apache git mirror: git://git.apache.org/incubator-myriad.git
> > > > * Github mirror: https://github.com/apache/incubator-myriad
> > > >
> > > > Please feel free to start using Apache repository for new development
> > > work
> > > > from this point onwards.
> > > >
> > > > Mohit
> > > >
> > > >
> > > > On Wed, Sep 23, 2015 at 10:20 AM, Jim Scott <jsc...@maprtech.com>
> > wrote:
> > > >
> > > >> This is really great news guys!
> > > >>
> > > >>
> > > >> On Wed, Sep 23, 2015 at 12:08 PM, Adam Bordelon <a...@mesosphere.io
> >
> > > >> wrote:
> > > >>
> > > >>> Hi Mohit,
> > > >>>
> > > >>> We brought this up at today's dev community sync, and we're all
> > excited
> > > >>> about the move.
> > > >>> Once the code is migrated to Apache git, we can use the Apache
> github
> > > >>> mirror for PRs:
> > > >>> https://github.com/apache/incubator-myriad/pulls
> > > >>> I would recommend that we migrate the existing PRs asap and
> shutdown
> > > the
> > > >>> old repo.
> > > >>>
> > > >>> Once the git repo is populated, we can also reopen the ticket for
> > > >>> ReviewBoard: https://issues.apache.org/jira/browse/INFRA-9840
> > > >>>
> > > >>> On Wed, Sep 23, 2015 at 10:00 AM, Mohit Soni <mo...@apache.org>
> > wrote:
> > > >>>
> > > >>>> Hello Everyone!
> > > >>>>
> > > >>>> I'm working towards migrating myriad code-base from:
> > > >>>> https://github.com/mesos/myriad to Apache's git.
> > > >>>>
> > > >>>> The code migration would entail following:
> > > >>>>
> > > >>>> * Mirroring https://github.com/mesos/myriad to Apache git:
> > > >>>> git clone --bare g...@github.com:mesos/myriad.git
> > > >>>> git push --mirror git://git.apache.org/incubator-myriad.git
> > > >>>>
> > > >>>> * Syncing Apache Git from https://github.com/mesos/myriad, until
> > all
> > > >>>> the outstanding
> > > >>>> PRs <https://github.com/mesos/myriad/pulls> are merged. Doing
> this
> > > >> will
> > > >>>> ensure that any current development activity happening on
> > > >>>> https://github.com/mesos/myriad is not disrupted.
> > > >>>> * Shutdown https://github.com/mesos/myriad, and migrate all new
> > > >>>> development
> > > >>>> work to Apache git repo.
> > > >>>>
> > > >>>> I'll start working on code-migration tomorrow, unless I receive a
> > > major
> > > >>>> concern from the community. I'll update the progress of migration
> on
> > > >>>> [MYRIAD-143] <https://issues.apache.org/jira/browse/MYRIAD-143>.
> > Once
> > > >>> the
> > > >>>> code migration is finished, I'll send out an email to the dev
> list,
> > > and
> > > >>>> from that point onward we request community members to start using
> > > >> Apache
> > > >>>> git for new development work.
> > > >>>>
> > > >>>> ~Mohit Soni
> > > >>>>
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> *Jim Scott*
> > > >> Director, Enterprise Strategy & Architecture
> > > >> +1 (347) 746-9281
> > > >> @kingmesal <https://twitter.com/kingmesal>
> > > >>
> > > >> <http://www.mapr.com/>
> > > >> [image: MapR Technologies] <http://www.mapr.com>
> > > >>
> > > >> Now Available - Free Hadoop On-Demand Training
> > > >> <
> > > >>
> > >
> >
> http://www.mapr.com/training?utm_source=Email_medium=Signature_campaign=Free%20available
> > > >>>
> > > >>
> > >
> > >
> >
>

Re: Pull requests

[Adam Bordelon]

incubator-myriad, please

On Wed, Sep 30, 2015 at 2:22 PM, Santosh Marella 
wrote:

> Mohit/Adam,
>
> Against which repo should the new PRs be submitted?
>
> github.com/apache/incubator-myriad or mesos/myriad ?
>

Re: Presentation links

[Adam Bordelon]

http://mesosphere.github.io/presentations-community/ has a few, plus the
youtube video from MesosCon

On Wed, Sep 30, 2015 at 12:39 PM, Jim Klucar  wrote:

> Does anyone have a nice list of Myriad presentation links I can slam into
> the website?
>
> Thanks
>

Re: Incubator PMC/Board report for Oct 2015 ([ppmc])

[Adam Bordelon]

I'm on it. (Unless somebody else wants to give it a go this time)

On Mon, Sep 28, 2015 at 7:15 AM, Marvin  wrote:

>
>
> Dear podling,
>
> This email was sent by an automated system on behalf of the Apache
> Incubator PMC.
> It is an initial reminder to give you plenty of time to prepare your
> quarterly
> board report.
>
> The board meeting is scheduled for Wed, 14 October 2015, 10:30 am PST. The
> report
> for your podling will form a part of the Incubator PMC report. The
> Incubator PMC
> requires your report to be submitted 2 weeks before the board meeting, to
> allow
> sufficient time for review and submission (Wed, Sep 30th).
>
> Please submit your report with sufficient time to allow the incubator PMC,
> and
> subsequently board members to review and digest. Again, the very latest you
> should submit your report is 2 weeks prior to the board meeting.
>
> Thanks,
>
> The Apache Incubator PMC
>
> Submitting your Report
> --
>
> Your report should contain the following:
>
>  * Your project name
>  * A brief description of your project, which assumes no knowledge of the
> project
>or necessarily of its field
>  * A list of the three most important issues to address in the move towards
>graduation.
>  * Any issues that the Incubator PMC or ASF Board might wish/need to be
> aware of
>  * How has the community developed since the last report
>  * How has the project developed since the last report.
>
> This should be appended to the Incubator Wiki page at:
>
>   http://wiki.apache.org/incubator/October2015
>
> Note: This is manually populated. You may need to wait a little before
> this page
>   is created from a template.
>
> Mentors
> ---
> Mentors should review reports for their project(s) and sign them off on the
> Incubator wiki page. Signing off reports shows that you are following the
> project - projects that are not signed may raise alarms for the Incubator
> PMC.
>
> Incubator PMC
>
>

Fwd: [jira] [Closed] (INFRA-9840) ReviewBoard for Myriad podling

[Adam Bordelon]

We now have a 'myriad' project in https://reviews.apache.org/r/new/ which
should be linked to the Apache git repo.
Feel free to start using ReviewBoard for future review requests. We'll
figure out the best way to commit those patches.

-- Forwarded message --no
From: Gavin (JIRA) 
Date: Sat, Sep 26, 2015 at 2:36 AM
Subject: [jira] [Closed] (INFRA-9840) ReviewBoard for Myriad podling
To: a...@mesosphere.io



 [
https://issues.apache.org/jira/browse/INFRA-9840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gavin closed INFRA-9840.

Resolution: Fixed

Done.

> ReviewBoard for Myriad podling
> --
>
> Key: INFRA-9840
> URL: https://issues.apache.org/jira/browse/INFRA-9840
> Project: Infrastructure
>  Issue Type: Wish
>  Components: Podling Setup, ReviewBoard
>Reporter: Adam B
>Assignee: Gavin
>
> Can you please help us set up a ReviewBoard for Myriad?
> ML: dev@myriad.incubator.apache.org
> Repo: https://git-wip-us.apache.org/repos/asf/incubator-myriad.git
> Mirror: git://git.apache.org/incubator-myriad.git
> (We have yet to migrate our code, but the repo exists)
> Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Reg Basic Myriad functionality, referring to MesosCon 2015 talk slides

[Adam Bordelon]

Haripriya, the RM (and Myriad scheduler) run as a Mesos task, and as a part
of that task, the Myriad scheduler sends a RegisterFrameworkMessage to the
Mesos master, asking Mesos to register a new Myriad framework and
communicate with this scheduler process for resource offers and status
updates.

So you ask Marathon to launch an app with 1 instance of the
RM/MyriadScheduler, and then that runs as a Mesos/Marathon task, and it
registers with Mesos as a new framework scheduler, after which point Mesos
starts making resource offers to the MyriadScheduler (as well as Marathon),
and the MyriadScheduler uses its offers to launch NMs.

The slightly tricky/confusing part is that the Myriad scheduler process'
resources are accounted for under the Marathon framework that launched it,
not as a part of the Myriad framework itself. Only tasks launched by the
Myriad scheduler (i.e. NodeManagers) are accounted for under the Myriad
framework's share of resources.

On Thu, Sep 24, 2015 at 3:59 PM, Aashreya Shankar 
wrote:

> Hi,
>
> With Myriad we do not launch Node Manager using Marathon.
> We launch ResourceManager from Marathon.
> Now Myriad takes care of launching more NodeManagers on Mesos slaves as it
> gets flex up requests.
>
> Thank you
> Aashreya
>
>
> On Thu, Sep 24, 2015 at 3:54 PM, Haripriya Ayyalasomayajula <
> aharipriy...@gmail.com> wrote:
>
> > Thanks.
> >
> > So when we use marathon to launch node manager - it starts as a mesos
> slave
> > task. In this case how can it register as a framework with Mesos-Master?
> is
> > there any document that you can point me to - I'm trying to understand
> the
> > workflow better in this case?
> >
> > On Thu, Sep 24, 2015 at 12:56 PM, Swapnil Daingade <
> > swapnil.daing...@gmail.com> wrote:
> >
> > > RM is just another Mesos task if launched using something like
> Marathon.
> > RM
> > > in turn will register with Mesos as another framework.
> > > You will need mesos slave to launch a mesos task. The RM mesos task can
> > get
> > > launched on any node having a Mesos slave.
> > >
> > > You can technically launch a RM on any node which has the right
> binaries
> > > and config files from the cli
> > > but then resources consumed by RM will not be accounted for by Mesos.
> > Also
> > > you will not have access
> > > to features like HA.
> > >
> > > Regards
> > > Swapnil
> > >
> > >
> > >
> > > On Thu, Sep 24, 2015 at 12:21 PM, Haripriya Ayyalasomayajula <
> > > aharipriy...@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > My understanding was that the Yarn Resource manager will run
> alongside
> > > > Mesos Master (In practise I was starting a resource manager on a node
> > > with
> > > > Mesos master running and no mesos slave process) but I see that the
> RM
> > is
> > > > running on a node which is a mesos slave - can anyone please clarify?
> > > >
> > > >
> > > >
> > >
> >
> http://events.linuxfoundation.org/sites/events/files/slides/Apache_Myriad_MesosCon_2015.pdf
> > > > (slide 24)
> > > >
> > > > --
> > > > Regards,
> > > > Haripriya Ayyalasomayajula
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> > Haripriya Ayyalasomayajula
> >
>

Re: Getting Nodes to be "Running" in Mesos

[Adam Bordelon]

You can certainly reduce the 1min delay in mesos-dns, although you might
not want to go much lower than 5-10s, especially on a large cluster, since
it can actually take a while for the master to generate all that
state.json. We're working on the performance of state.json so that more
frequent polling can be less of an impact.

On Wed, Sep 9, 2015 at 8:56 AM, Santosh Marella 
wrote:

> > ready right away (1 minute delay after kicking off Myriad)
> That's true. The reason a medium NM instance is kicked off at RM's startup
> was to allow non-zero capacity in the cluster, without which YARN seems
> to reject app submissions. I'll look at YARN's code base more carefully
> and see if this behavior can be disabled via configuration
> (I didn't notice such option last timeI looked).
>
> The other workaround might be to reduce the 1 min delay in mesos-dns
> to create DNS entries for mesos tasks. Not sure if that's recommended
> in production, but sometimes I felt 1 min is too long to create a DNS
> entry.
> If RM failsover, it means the new RM instance can't be discovered for 1
> min.
> Job's that were previously running have to wait > 1 min to resume.
>
> Santosh
>
> On Tue, Sep 8, 2015 at 1:23 PM, John Omernik  wrote:
>
> > Also a side note:  The Flexing up and now having to have at least one
> node
> > manager specified at startup:
> >
> > nmInstances: # NMs to start with. Requires at least 1 NM with a non-zero
> > profile.
> >
> >   medium: 1 # 
> >
> >
> > Is going to lead to task failures with mesos dns because the name won't
> be
> > ready right away (1 minute delay after kicking off Myriad) do we NEED to
> > have a non-0 profile nodemanager startup with the resource manager?
> >
> > On Tue, Sep 8, 2015 at 3:16 PM, John Omernik  wrote:
> >
> > > Cool.  Question about the yarn-site.xml in general.
> > >
> > > I was struggling with some things in the wiki on this page:
> > >
> >
> https://cwiki.apache.org/confluence/display/MYRIAD/Installing+for+Administrators
> > >
> > > Basically in step 5:
> > > Step 5: Configure YARN to use Myriad
> > >
> > > Modify the */opt/hadoop-2.7.0/etc/hadoop/yarn-site.xml* file as
> > > instructed in Sample: myriad-config-default.yml
> > > <
> >
> https://cwiki.apache.org/confluence/display/MYRIAD/Sample%3A+myriad-config-default.yml
> > >
> > > .
> > >
> > >
> > > (It should not link to the yml, but to the yarn site, side issue) it
> has
> > > us put that information in the yarn-site.xml This makes sense.  The
> > > resource manager needs to be aware of the myriad stuff.
> > >
> > > Then I go to create a tarbal, (which I SHOULD be able to use for both
> > > resource manager and nodemanager... right?) However, the instructions
> > state
> > > to remove the *.xml files.
> > >
> > > Step 6: Create the Tarball
> > >
> > > The tarball has all of the files needed for the Node Managers and
> > > Resource Managers. The following shows how to create the tarball and
> > place
> > > it in HDFS:
> > > cd ~
> > > sudo cp -rp /opt/hadoop-2.7.0 .
> > > sudo rm hadoop-2.7.0/etc/hadoop/*.xml
> > > sudo tar -zcpf ~/hadoop-2.7.0.tar.gz hadoop-2.7.0
> > > hadoop fs -put ~/hadoop-2.7.0.tar.gz /dist
> > >
> > >
> > > What I ended up doing... since I am running the resourcemanager
> (myriad)
> > > in marathon, is I created two tarballs. One is my
> hadoop-2.7.0-RM.tar.gz
> > > which has the all the xml files still in the tar ball for shipping to
> > > marathon. Then other is hadoop-2.7.0-NM.tar.gz which per the
> instructions
> > > removes the *.xml files from the /etc/hadoop/ directory.
> > >
> > >
> > > I guess... my logic is that myriad creates the conf directory for the
> > > nodemanagers... but then I thought, and I overthinking something? Am I
> > > missing something? Could that be factoring into what I am doing here?
> > >
> > >
> > > Obviously my first steps are to add the extra yarn-site.xml entries,
> but
> > > in this current setup, they are only going into the resource manager
> > > yarn-site as the the node-managers don't have a yarn-site in their
> > > directories.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Tue, Sep 8, 2015 at 3:09 PM, yuliya Feldman <
> > > yufeld...@yahoo.com.invalid> wrote:
> > >
> > >> Take a look at :   https://github.com/mesos/myriad/pull/128
> > >> for yarn-site.xml updates
> > >>
> > >>   From: John Omernik 
> > >>  To: dev@myriad.incubator.apache.org
> > >>  Sent: Tuesday, September 8, 2015 12:38 PM
> > >>  Subject: Getting Nodes to be "Running" in Mesos
> > >>
> > >> So I am playing around with a recent build of Myriad, and I am using
> > MapR
> > >> 5.0 (hadoop-2.7.0) I hate to use the dev list as a "help Myriad won't
> > run"
> > >> forum, so please forgive me if I am using the list wrong.
> > >>
> > >> Basically, I seem to be able to get myriad running, and the things up,
> > and
> > >> it tries to start a nodemanager.
> > >>
> > >> In mesos, the status of the nodemanager 

Re: Understanding Myriad workflows

[Adam Bordelon]

Haripriya,

You may want to use something like Marathon to start the RM/scheduler and
keep it up, so that Marathon can restart it somewhere else if the RM
app/process, or even its entire machine fails. You can, of course, start
the RM/scheduler manually on a node, but you'll have to manage its HA on
your own.

On Mon, Sep 21, 2015 at 10:06 AM, Haripriya Ayyalasomayajula <
aharipriy...@gmail.com> wrote:

> Hi all,
>
> https://cwiki.apache.org/confluence/display/MYRIAD/Administration
>
> I see that there is a mention of Marathon in this document - I'm not sure
> where in the Myriad workflow marathon is being used - can anyone help me
> understand this better - or if I'm missing something ?
>
> On Wed, Sep 16, 2015 at 11:07 AM, Ruth Harris 
> wrote:
>
> > hi Haripriya,
> >
> > No problem. Let me know if you see any errors or inconsistencies.
> >
> > Regards,  --ruth
> >
> > Ruth Harris
> > Sr. Tech. Writer, MapR
> >
> > On Wed, Sep 16, 2015 at 9:44 AM, Haripriya Ayyalasomayajula <
> > aharipriy...@gmail.com> wrote:
> >
> > > Thanks Ruth! That was helpful!
> > >
> > > On Mon, Sep 14, 2015 at 1:06 PM, Ruth Harris 
> > wrote:
> > >
> > > > The REST API is documented in the Apache wiki:
> > > >
> https://cwiki.apache.org/confluence/display/MYRIAD/Myriad+Cluster+API
> > > >
> > > > Also, in progress, documentation on fine-grained scaling:
> > > >
> > https://cwiki.apache.org/confluence/display/MYRIAD/Fine-grained+Scaling
> > > >
> > > > --ruth
> > > >
> > > > On Mon, Sep 14, 2015 at 12:24 PM, Haripriya Ayyalasomayajula <
> > > > aharipriy...@gmail.com> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Is there any architecture document describing the basic workflows -
> > on
> > > > flex
> > > > > up the methods called etc? I'm going through the source code on git
> > hub
> > > > and
> > > > > was wondering if there is any such source which would help me
> > > understand
> > > > > the internals more..
> > > > >
> > > > > --
> > > > > Regards,
> > > > > Haripriya Ayyalasomayajula
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Ruth Harris
> > > > Sr. Technical Writer, MapR
> > > >
> > >
> > >
> > >
> > > --
> > > Regards,
> > > Haripriya Ayyalasomayajula
> > >
> >
> >
> >
> > --
> > Ruth Harris
> > Sr. Technical Writer, MapR
> >
>
>
>
> --
> Regards,
> Haripriya Ayyalasomayajula
>

Re: add to wiki

[Adam Bordelon]

Darin, you now have edit permissions on the wiki.
Santosh, you now have edit/admin permissions.

On Wed, Sep 9, 2015 at 12:37 PM, Santosh Marella 
wrote:

> Hi Darin,
>
>   I've added you to JIRA. Please let me know if you face problems using
> MYRIAD JIRA.
>
> Thanks,
> Santosh
>
> On Wed, Sep 9, 2015 at 12:01 PM, Darin Johnson 
> wrote:
>
> > Also, can I be added to JIRA.  Would like to assign some issues to myself
> > and close a few I completed.
> >
> > On Wed, Sep 9, 2015 at 1:53 PM, Santosh Marella 
> > wrote:
> >
> > > Please give me edit perms as well. Username: smarella
> > >
> > > Santosh
> > >
> > > On Wed, Sep 9, 2015 at 11:40 AM, Darin Johnson <
> dbjohnson1...@gmail.com>
> > > wrote:
> > >
> > > > could someone give me edit perm to the wiki?  Thanks, Darin.
> Username:
> > > > DarinJ
> > > >
> > >
> >
>

Re: Question for Mesos gurus

[Adam Bordelon]

Yes, you'll have to recompile your scheduler/executor against the latest
libmesos.
In the upgrade guide, this is mentioned as Upgrade the schedulers by
linking the latest native library / jar / egg (if necessary).
The modules instructions apply to C++ plugins for the Mesos master/slaves
themselves.

On Tue, Aug 25, 2015 at 10:52 AM, yuliya Feldman 
yufeld...@yahoo.com.invalid wrote:

 Thank you Adam
 When you say:but before you upgrade Mesos to 0.23, you should upgrade
 your scheduler (and executor) libmesos to 0.22.x

 Do you mean - recompile?
 Does this sentence from link with upgrade instructions you provided means
 the same? Rebuild and install any modules so that upgraded
 masters/slaves can use them
 Thanks,Yuliya  From: Adam Bordelon a...@mesosphere.io
  To: dev@myriad.incubator.apache.org; yuliya Feldman yufeld...@yahoo.com
  Sent: Tuesday, August 25, 2015 10:06 AM
  Subject: Re: Question for Mesos gurus

 Mesos guarantees forward and backward compatibility by one minor version.
 It is expected that you upgrade the entire cluster to one consecutive
 version before upgrading any component to the next. So, if your scheduler
 jar's libmesos is from 0.21.x, you can upgrade your Mesos master/agents to
 0.22.x safely, but before you upgrade Mesos to 0.23, you should upgrade
 your scheduler (and executor) libmesos to 0.22.x. See
 http://mesos.apache.org/documentation/latest/upgrades/ for other special
 notes and recommended upgrade order.
 Once we reach Mesos 1.0 (when the new HTTP API stabilizes), then we'll have
 stronger guarantees about version compatibility within a major version.



 On Tue, Aug 25, 2015 at 8:33 AM, yuliya Feldman
 yufeld...@yahoo.com.invalid
  wrote:

  Hello guys,
  I wonder about compatibility of Mesos protobuf for Myriad usage.
  If I complied Myriad with Mesos version 0.22.1/0.21.1 but on the cluster
 I
  have Mesos 0.23 - is it suppose to be compatible?
  Yesterday our guys came across an exception(see below).
  When switching jars to mesos-0.21.1 issue went away.
  Thanks,Yuliya
  15/08/24 10:57:40 INFO scheduler.TaskFactory$NMTaskFactoryImpl:
  yarn.resourcemanager.hostname is set to rm.marathon.mesos via
  YARN_RESOURCEMANAGER_OPTS. Passing it into YARN_NODEMANAGER_OPTS.
  Aug 24, 2015 10:57:40 AM com.lmax.disruptor.FatalExceptionHandler
  handleEventException
  SEVERE: Exception processing: 1
  com.ebay.myriad.scheduler.event.ResourceOffersEvent@74a1e0a5
  java.lang.NoSuchMethodError:
 
 
 org.apache.mesos.Protos$TaskInfo$Builder.setData(Lcom/google/protobuf/ByteString;)Lorg/apache/mesos/Protos$TaskInfo$Builder;
   at
 
 
 com.ebay.myriad.scheduler.TaskFactory$NMTaskFactoryImpl.createTask(TaskFactory.java:310)
   at
 
 
 com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:98)
   at
 
 
 com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:55)
   at
  com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
   at
 
 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at
 
 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
 
  15/08/24 10:57:40 ERROR yarn.YarnUncaughtExceptionHandler: Thread
  Thread[pool-2-thread-3,5,main] threw an Exception.
  java.lang.RuntimeException: java.lang.NoSuchMethodError:
 
 
 org.apache.mesos.Protos$TaskInfo$Builder.setData(Lcom/google/protobuf/ByteString;)Lorg/apache/mesos/Protos$TaskInfo$Builder;
   at
 
 
 com.lmax.disruptor.FatalExceptionHandler.handleEventException(FatalExceptionHandler.java:45)
   at
  com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:147)
   at
 
 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at
 
 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
  Caused by: java.lang.NoSuchMethodError:
 
 
 org.apache.mesos.Protos$TaskInfo$Builder.setData(Lcom/google/protobuf/ByteString;)Lorg/apache/mesos/Protos$TaskInfo$Builder;
   at
 
 
 com.ebay.myriad.scheduler.TaskFactory$NMTaskFactoryImpl.createTask(TaskFactory.java:310)
   at
 
 
 com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:98)
   at
 
 
 com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:55)
   at
  com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
   ... 3 more
 

Re: Licensing for new files

[Adam Bordelon]

According to https://github.com/mesos/myriad/pull/126/files the code
already was, and will continue to be licensed under Apache 2.0. The only
question is the Copyright, which was previously eBay Software Foundation,
will soon be PayPal, and finally Apache Software Foundation. I imagine
you'd just use whatever copyright is currently in use, and we'll update
them all at once.

On Mon, Aug 24, 2015 at 11:38 AM, yuliya Feldman 
yufeld...@yahoo.com.invalid wrote:

 Hello here,
 What licensing should we use for newly created files in Myriad source code
 - pure Apache?
 Thanks,Yuliya

Re: making contributor

[Adam Bordelon]

Hmm.. you already are a contributor, Yuliya, but it looks like the
_Default Permission Scheme_ doesn't allow contributors to assign JIRAs to
themselves (or anybody else). Only committers/PMC and administrators can
assign issues to people.
Mesos, on the other hand, uses Default plus Contributor Assign Permission
Scheme, which (as you can imagine) allows contributors to assign JIRAs to
themselves (or any other contributor).
If nobody objects, I will file an INFRA JIRA to change the Myriad
permission scheme to match the Mesos scheme, since that makes a lot more
sense to me, and allows contributors to be a lot more autonomous.

On Wed, Aug 19, 2015 at 4:42 PM, yuliya Feldman yufeld...@yahoo.com.invalid
 wrote:

 Hello guys,
 Could you make me contributor to Myriad, so I could assign JIRAs to myself?
 Thanks,Yuliya

Re: making contributor

[Adam Bordelon]

Filed https://issues.apache.org/jira/browse/INFRA-10166
Leave a comment there ASAP if you object.

On Thu, Aug 20, 2015 at 7:54 AM, Santosh Marella smare...@maprtech.com
wrote:

 +1

 --
 Sent from mobile
 On Aug 20, 2015 7:53 AM, Jim Scott jsc...@maprtech.com wrote:

  +1
 
  On Thu, Aug 20, 2015 at 3:10 AM, Adam Bordelon a...@mesosphere.io
 wrote:
 
   Hmm.. you already are a contributor, Yuliya, but it looks like the
   _Default Permission Scheme_ doesn't allow contributors to assign
 JIRAs
  to
   themselves (or anybody else). Only committers/PMC and administrators
 can
   assign issues to people.
   Mesos, on the other hand, uses Default plus Contributor Assign
  Permission
   Scheme, which (as you can imagine) allows contributors to assign JIRAs
  to
   themselves (or any other contributor).
   If nobody objects, I will file an INFRA JIRA to change the Myriad
   permission scheme to match the Mesos scheme, since that makes a lot
 more
   sense to me, and allows contributors to be a lot more autonomous.
  
   On Wed, Aug 19, 2015 at 4:42 PM, yuliya Feldman
   yufeld...@yahoo.com.invalid
wrote:
  
Hello guys,
Could you make me contributor to Myriad, so I could assign JIRAs to
   myself?
Thanks,Yuliya
  
 
 
 
  --
  *Jim Scott*
  Director, Enterprise Strategy  Architecture
  +1 (347) 746-9281
  @kingmesal https://twitter.com/kingmesal
 
  http://www.mapr.com/
  [image: MapR Technologies] http://www.mapr.com
 
  Now Available - Free Hadoop On-Demand Training
  
 
 http://www.mapr.com/training?utm_source=Emailutm_medium=Signatureutm_campaign=Free%20available
  
 

Re: myriad scheduler startup with HDP2.7

[Adam Bordelon]

Myriad should be using jackson 2.5.1
https://github.com/mesos/myriad/blob/d6d765736ba1c8f59aa967457527331e1dab6743/myriad-scheduler/build.gradle#L13
Double-check your build.gradle, and make sure you don't have a jackson
2.2.3 preinstalled somewhere else on your system

On Wed, Aug 19, 2015 at 8:20 AM, Bill Sparks jspa...@cray.com wrote:

 Odd the class path reported in the yarn log contains jackson-core-2.2.3
 and not 2.5.1. Is there a way to build myriad to match the version
 supported by HDP - that being 2.2.3 ?


 --
 Jonathan (Bill) Sparks
 Software Architecture
 Cray Inc.





 On 8/19/15 10:11 AM, Bill Sparks jspa...@cray.com wrote:

 Thanks I'll check..
 
 --
 Jonathan (Bill) Sparks
 Software Architecture
 Cray Inc.
 
 
 
 
 
 On 8/19/15 10:09 AM, yuliya Feldman yufeld...@yahoo.com.INVALID
 wrote:
 
 This method is part of JsonFactory class which is part of jackson-core
 jar
 See if you have some other jars on the classpath (different versions)
 that precede jackson-core-2.5.1.jar
   From: Bill Sparks jspa...@cray.com
  To: dev@myriad.incubator.apache.org dev@myriad.incubator.apache.org
  Sent: Wednesday, August 19, 2015 7:08 AM
  Subject: myriad scheduler startup with HDP2.7
 
 I'm sure this is been resolved, but I've been triaging why I'm getting
 the following error on resourcemanager startup. Everything on the
 configuration side looks correct, but I must have missed something.
 
 
 
 2015-08-19 08:53:04,718 FATAL
 org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
 starting ResourceManager
 
 java.lang.NoSuchMethodError:
 com.fasterxml.jackson.dataformat.yaml.YAMLFactory._decorate(Ljava/io/Inpu
 t
 Stream;Lcom/fasterxml/jackson/core/io/IOContext;)Ljava/io/InputStream;
 
 at
 com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFactor
 y
 .java:299)
 
 at
 com.fasterxml.jackson.dataformat.yaml.YAMLFactory.createParser(YAMLFactor
 y
 .java:14)
 
 at
 com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2
 0
 11)
 
 at com.ebay.myriad.Main.initialize(Main.java:70)
 
 at
 com.ebay.myriad.scheduler.yarn.interceptor.MyriadInitializationIntercepto
 r
 .init(MyriadInitializationInterceptor.java:32)
 
 at
 com.ebay.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(Comp
 o
 siteInterceptor.java:76)
 
 at
 com.ebay.myriad.scheduler.yarn.MyriadFairScheduler.serviceInit(MyriadFair
 S
 cheduler.java:50)
 
 at
 org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
 
 at
 org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.j
 a
 va:107)
 
 at
 org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveSer
 v
 ices.serviceInit(ResourceManager.java:572)
 
 at
 org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
 
 at
 org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndIn
 i
 tActiveServices(ResourceManager.java:972)
 
 at
 org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit
 (
 ResourceManager.java:259)
 
 at
 org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
 
 at
 org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(Resour
 c
 eManager.java:1202)
 
 
 I have placed all the myriad jar in the hadoop-yarn/lib directory and the
 classpath reflect that.
 
 
 cp /tmp/myriad/myriad-scheduler/build/libs/*
 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib
 
 cp
 /tmp/myriad/myriad-executor/build/libs/myriad-executor-runnable-0.0.1.jar
 /usr/libexec/mesos/
 
 
 [root@nid00037 myriad]# su - yarn
 
 -bash-4.1$ yarn classpath
 
 /usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/
 h
 dp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-2557/hadoop/lib/*:/usr/hdp/2
 .
 3.0.0-2557/hadoop/.//*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/./:/usr/hdp/2.3.
 0
 .0-2557/hadoop-hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-hdfs/.//*:/usr/hdp
 /
 2.3.0.0-2557/hadoop-yarn/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-yarn/.//*:/us
 r
 /hdp/2.3.0.0-2557/hadoop-mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-map
 r
 educe/.//*::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/ja
 v
 a/mysql-connector-java.jar:/usr/hdp/current/hadoop-mapreduce-client/*:/us
 r
 /hdp/2.3.0.0-2557/tez/*:/usr/hdp/2.3.0.0-2557/tez/lib/*:/usr/hdp/2.3.0.0-
 2
 557/tez/conf:/usr/hdp/current/hadoop-yarn-client/.//*:/usr/hdp/current/ha
 d
 oop-yarn-client/lib/*
 
 
 
 ls /usr/hdp/current/hadoop-yarn-client/lib/* has all the libraries
 
 
 -bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad*
 
 -rw-r--r-- 1 root root  3456 Aug 19 08:50
 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-commons-0.0.1.jar
 
 -rw-r--r-- 1 root root 950687 Aug 19 08:50
 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib//myriad-scheduler-0.0.1.jar
 
 and
 
 
 -bash-4.1$ ls -l /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson*
 
 -rw-r--r-- 1 root root  39817 Aug 17 18:32
 /usr/hdp/2.3.0.0-2557/hadoop-yarn/lib/jackson-annotations-2.5.1.jar
 
 -rw-r--r-- 1 root root  192699 Jul 14 08:22
 

Re: Documentation Comments

[Adam Bordelon]

John, thanks a ton for your valuable feedback! We're glad to have your
perspective as a user of the project, and I'm ready+willing to give you
edit access to the wiki if you want to update it with your learnings,
elaborate anything that's unclear, or add a new John's tips page. Just
sign up for a wiki account, send me your accountId, and I'll grant you edit
access.
(I'll let others answer your specific questions)

On Wed, Aug 19, 2015 at 6:28 AM, John Omernik j...@omernik.com wrote:

 Today, I will be playing the role of the fool/jester trying to get Myriad
 running. Basically, since getting Myriad running with Santosh quite a while
 ago, and now trying again with new versions of Hadoop, MapR, and Myriad, I
 wanted to hit up the wiki (
 https://cwiki.apache.org/confluence/display/MYRIAD/Myriad+Home) and
 outline
 points that as a non-dev living the code, are unclear to someone trying to
 utilize myriad or understand it's operation.

 Obviously, some of my points can be answered with look here in the code
 or look at this page, but I will try to outline my thought processes as I
 reviewed the current docs.  Sometimes the way I approached the problem led
 me down a path of to a certain page, missing the answer in a different
 page, and thus some cross linking could be helpful.

 Please do not let my points be taken as anything other than a desire to
 improve how accessible Myriad is to the community, this is not a critique
 of the hard work everyone has done on the project.  I also understand that
 given the work load and other issues, that fixing these issues in
 documentation may not be a priority.  I am listing them out here, so that
 those folks who are SMEs on various points may be able to quickly add stuff
 and we'll organize it later.


 *Remote Distribution: *

 https://cwiki.apache.org/confluence/display/MYRIAD/Myriad+Remote+Distribution

 This whole section could use some work from a standpoint of what runs where
 and where that component gets its files.  For example, I think it would
 help people to understand that the whole tarball created in step 6 has all
 the files for node managers and resource managers.  Basically, everything
 runs from there. Here is a small example I am currently working with:


 Starting Myriad:
 Option 1: Use Marathon (provide example json, here is mine)
 {
 cmd: env  export

 YARN_RESOURCEMANAGER_OPTS=-Dyarn.resourcemanager.hostname=myriad.marathon.mesos
  hadoop-2.7.0/bin/yarn resourcemanager,
 uris: [maprfs:///mesos/myriad/hadoop-2.7.0.tar.gz],
 cpus: 1.0,
 mem: 1024,
 id: myriad,
 instances: 1,
 user: mapr
 }

 In this case, Marathon grabs the hadoop tarball and pulls it down, this
 tarball also has the Myriad yml file. When it executes the resource
 manager, it is brought up in Myriad and ready to run node managers by
 pulling the tarball to the slave nodes and executing the nodemanager.  (I
 would imagine the work with history server etc would also use this
 tarball?).

 From here it will us NMInstances to launch a node manager.  (Note, this is
 different from when I originally set things up... before, I could run the
 resource manager/myriad without a nodemanager, now it seems it's required
 based on the config in the src... could we expound on this in the docs
 somewhere?)


 Option 2:  (Are there other ways to launch the resource manager?)

 Step 6: So something that is unclear to me is  the handling of the
 hadoop/yarn config files.  In Step 6 on this page, there is sudo rm
 hadoop-
 2.5.0/etc/hadoop/*.xml  This doesn't makes sense to me. I actually ignored
 this step.  For me, if I remove these xml files, then there is no place to
 get my files... I think? Since I am running the RM and NM from the same
 tarball, and Myriad config is here, and my goal is to not have anything
 installed on a node, where would I set  yarn settings? This could be much
 clearer to me, and probably others.

 Step 2:  Should we just be copying the Myriad files to
 /share/hadoop/yarn/lib folder? Do we worry about potentially overwrites of
 jars or version conflicts?

 *Configuring Cgroups*
 https://cwiki.apache.org/confluence/display/MYRIAD/Configuring+Cgroups
 At some point a little bit more about why one would want CGroups and issues
 that could occur with them. While many folks using Mesos/Myriad may
 understand this, others may not, and it's a good way to help people think
 positively about our project if we help educate them along the way.

 Minor point on enabling CGroups. This is confusing given my questions in
 remote distribution. in this it says I need to edit my yarn-site.xml, but
 in remote distribution it says delete my hadoop xml files. We need to
 address this conflict cause it can be confusing for a user coming onboard

 Nitpick: Enabling cgroups for mess-slave - should be - Enabling cgroups for
 mesos-slave

 *Myriad Configuration Properties*:

 https://cwiki.apache.org/confluence/display/MYRIAD/Myriad+Configuration+Properties
 Based on the conversation on list 

Re: wiki access

[Adam Bordelon]

Permission granted.

On Mon, Aug 17, 2015 at 8:30 AM, Jim Klucar klu...@gmail.com wrote:

 Please add me to wiki. Found a few bits to edit quick.

 jklucar

 Thanks!

Re: Trying to build/run Myriad

[Adam Bordelon]

To subscribe to the myriad dev list, send an email to:
dev-subscr...@myriad.incubator.apache.org
For other commands, and the full archive, see:
http://mail-archives.apache.org/mod_mbox/incubator-myriad-dev/

On Mon, Aug 17, 2015 at 10:18 AM, Evgeniy Chupriyanov t...@tchu.ru wrote:


 17 авг. 2015 г., в 9:46, yuliya Feldman yufeld...@yahoo.com написал(а):

 I second Adam here.

 Also - is your job failed to run or it is hanging  - AM is not starting?

 It was stuck in SCHEDULED state and no AM was starting. And as Adam said -
 the problem was I had no NMs running in Mesos.
 Now I was able to start job and it finished successfully, though I had to
 start really large NM for small wordcount example.

 Eugene

Re: wiki access

[Adam Bordelon]

Brandon, you should have already had access, since you Liked my first wiki
page. :)

On Mon, Aug 17, 2015 at 9:40 AM, Brandon Gulla gulla.bran...@gmail.com
wrote:

 I would also like access.

 bgulla

 On Mon, Aug 17, 2015 at 12:11 PM, Adam Bordelon a...@mesosphere.io
 wrote:

  Permission granted.
 
  On Mon, Aug 17, 2015 at 8:30 AM, Jim Klucar klu...@gmail.com wrote:
 
   Please add me to wiki. Found a few bits to edit quick.
  
   jklucar
  
   Thanks!
  
 



 --
 Brandon

Re: Trying to build/run Myriad

[Adam Bordelon]

++ dev@myriad mailing list

Evgeniy,

Thanks for trying out Myriad. Sorry for my delayed response. Just got back
from vacation.
Have you flexed up to add any NodeManagers to your YARN cluster?
See
https://github.com/mesos/myriad/blob/phase1/docs/API.md#put-apiclusterflexup

On Fri, Aug 7, 2015 at 10:56 AM, Evgeniy Chupriyanov t...@tchu.ru wrote:

 Hello, Adam!

 Sorry to contact you directly, but I couldn’t find where is the better
 place to ask questions about Myriad.

 I am working for small russian startup Cybertonica (
 https://www.cyberonica.com) and we are useng Mesos as a base for our
 distributed system.
 Currently, I am exploring for better way to run Hadoop MR tasks on Mesos
 and Myriad looks very promising.

 I’ve tried to build Myriad/Mesos in local vagrant VM as described here:
 https://github.com/mesos/myriad/blob/phase1/docs/vagrant.md
 Everything went well and build/setup was successful. I can see Myriad
 framewrok registered in Mesos.

 Having that done, I have tried to small wordcount example like this:
 hduser@vagrant-ubuntu-trusty-64:/vagrant$ cat th.sh
 #!/bin/bash

 echo “I love UCSB”  /tmp/file0
 echo “Do you love UCSB?”  /tmp/file1
 hadoop fs -mkdir -p /tmp/foo/data
 hadoop fs -put /tmp/file? /tmp/foo/data
 yarn jar
 $YARN_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.2.jar
 wordcount /tmp/foo/data /tmp/foo/out
 hadoop fs -ls /tmp/foo/out
 hadoop fs -cat /tmp/foo/out/part*
 hduser@vagrant-ubuntu-trusty-64:/vagrant$

 But job fails to run and I see following message in resourcemanager log:
 2015-08-07 16:51:35,635 INFO
 org.apache.hadoop.yarn.server.resourcemanager.ClientRMService: Allocated
 new applicationId: 1
 2015-08-07 16:51:36,533 INFO
 org.apache.hadoop.yarn.server.resourcemanager.ClientRMService: Application
 with id 1 submitted by user hduser
 2015-08-07 16:51:36,534 INFO
 org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: Storing
 application with id application_1438966241928_0001
 2015-08-07 16:51:36,536 INFO
 org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hduser
 IP=10.0.2.15OPERATION=Submit Application Request
 TARGET=ClientRMService  RESULT=SUCCESS  APPID=application_1438966241928_0001
 2015-08-07 16:51:36,537 INFO
 org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
 application_1438966241928_0001 State change from NEW to NEW_SAVING
 2015-08-07 16:51:36,541 INFO
 org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore:
 Storing info for app: application_1438966241928_0001
 2015-08-07 16:51:36,542 INFO
 org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
 application_1438966241928_0001 State change from NEW_SAVING to SUBMITTED
 2015-08-07 16:51:36,556 INFO
 org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler:
 Accepted application application_1438966241928_0001 from user: hduser, in
 queue: default, currently num of applications: 1
 2015-08-07 16:51:36,582 INFO
 org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
 application_1438966241928_0001 State change from SUBMITTED to ACCEPTED
 2015-08-07 16:51:36,582 INFO
 org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService:
 Registering app attempt : appattempt_1438966241928_0001_01
 2015-08-07 16:51:36,587 INFO
 org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
 appattempt_1438966241928_0001_01 State change from NEW to SUBMITTED
 2015-08-07 16:51:36,612 INFO
 org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler:
 Added Application Attempt appattempt_1438966241928_0001_01 to scheduler
 from user: hduser
 2015-08-07 16:51:36,614 INFO
 org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
 appattempt_1438966241928_0001_01 State change from SUBMITTED to
 SCHEDULED
 2015-08-07 16:51:37,002 INFO
 com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler:
 Received offers 1
 2015-08-07 16:51:37,002 INFO
 com.ebay.myriad.scheduler.event.handlers.ResourceOffersEventHandler: No
 pending tasks, declining all offers

 As I can see application gets status SCHEDULED and stays there. No any
 attempts are done to launch Mesos jobs are done. myriad scheduler reports
 it has no tasks pending.
 What I am doing wrong here? The only difference with the guide I did - I’m
 using Mesos 0.23.0, instead of 0.21 as described in guide (I changed
 version in build.gradle)


 Great thanks for a wonderful project.
 Sorry for any inconvenience

 Eugene

Re: Wiki access please

[Adam Bordelon]

Access granted. Enjoy!

On Thu, Aug 13, 2015 at 3:51 PM, Swapnil Daingade 
swapnil.daing...@gmail.com wrote:

 Hi committers,

 I am looking for access to the Myriad wiki.

 Full Name: Swapnil Daingade
 Email: swapnil.daing...@gmail.com
 Username: sdaingade

 Regards
 Swapnil

Re: Myriad 0.1 release scope

[Adam Bordelon]

We do have a JIRA 0.1.0 fix version field, but none of our issues use it
yet.
I think the goal was just to take what we have and make it work under
Apache infrastructure, then vote on that for 0.1.0.
Although other features like HA or FGS would be great, let's try to get our
first Apache release out ASAP.
We can create 0.1.1 or 0.2.0 fix versions for subsequent releases with
other issues/features. Roadmap would be great.
(I'm just summarizing what we discussed a month or two ago. Feel free to
correct me or disagree with this approach.)

On Tue, Aug 4, 2015 at 4:44 PM, Swapnil Daingade swapnil.daing...@gmail.com
 wrote:

 Hi all,

 Was wondering what would be the scope for the Myriad 0.1 release.
 It would be nice to have a roadmap page somewhere and target
 features to releases (JIRA 'fix version' field perhaps)

 Regards
 Swapnil

Re: Jira ready (mesos/myriad github issues imported)

[Adam Bordelon]

I just disabled our github issues link, and added a link to the new JIRA at
the top of the github page.
Please use JIRA from now on for any new issues or updates to existing
issues.
You probably need to create an account if you don't already have one.
Let us know if you need to be added to the Developers group to assign an
issue to yourself.
All committers have JIRA Admin access.

Thanks!

On Sun, Aug 2, 2015 at 6:23 AM, Gavin McDonald gmcdon...@apache.org wrote:


 Hi All,

 mesos/myriad Github issues imported into the ASF Jira.

 as per https://issues.apache.org/jira/browse/INFRA-9516

 Please now stop using Github and start using Jira.

 https://issues.apache.org/jira/browse/MYRIAD

 Please comment on the INFRA-9516 ticket if you spot anything
 or otherwise please confirm all ok and close the ticket.

 Thanks

 Gav…

Re: Introducing Ruth - documentation

[Adam Bordelon]

Thanks for the link. While I personally prefer ReviewBoard's feature-set
(multi-line select, tracking patch issues) for larger code changes, github
PRs are nice for reviewing (generated) markdown, and fine for
documentation/small changes. But, yeah, well, that's just like, my opinion,
man.

On Fri, Jul 31, 2015 at 8:20 AM, Jim Scott jsc...@maprtech.com wrote:

 Thanks for sharing that. I was looking for more information on it, as the
 Apache Drill team is also moving in this direction, and is a little further
 ahead, but I was not finding the right information to share back with the
 group.

 Most of the projects are tending toward this process now as it is becoming
 much easier to manage, offers a higher level of visibility and makes it
 that much easier on contributors and committers.


 On Fri, Jul 31, 2015 at 10:13 AM, Jim Klucar klu...@gmail.com wrote:

  Adam,
 
  Apache has actually improved github integration. I talked to an Accumulo
  committer because I know they interact with people via github sometimes.
 He
  pointed me here
 
 
 
 https://blogs.apache.org/infra/entry/improved_integration_between_apache_and
 
 
  On Thu, Jul 30, 2015 at 7:52 PM, Adam Bordelon a...@mesosphere.io
 wrote:
 
   We should start with our Confluence wiki:
   https://cwiki.apache.org/confluence/display/MYRIAD/Myriad+Home
   It's also recommended that we create a proper website at
   http://myriad.incubator.apache.org/
  
   I'd rather we didn't rely on gh-pages, especially since the github repo
  is
   just a mirror of Apache git, so we can't actually use github PRs
 without
   downloading the patches, applying them locally, and then pushing to
  Apache
   git.
  
   On Thu, Jul 30, 2015 at 4:42 PM, Will Ochandarena 
   wochandar...@maprtech.com
wrote:
  
Myriad Team, I want to introduce you to Ruth Harris from the MapR
documentation team.  She'll be leading the lead on documentation for
Myriad, hopefully with ample help from each of you.
   
She's been digging into Myriad for a few weeks now, and has a
  preliminary
outline.  I'll let her reply to this and send it to start getting
   feedback.
   
One thing she needs is a place to start posting to.  Looking at other
Apache projects, it seems like the most common practice is to use a
gh-pages branch of the main github
https://github.com/apache/incubator-myriad.
   
Adam - could you create the branch so she has something to send PRs
 to?
Jim - you seem to be our resident web/css expert.  Any chance you
 could
build a simple layout for the doc site?
   
Thanks!
Will
   
  
 



 --
 *Jim Scott*
 Director, Enterprise Strategy  Architecture
 +1 (347) 746-9281

 http://www.mapr.com/
 [image: MapR Technologies] http://www.mapr.com

 Now Available - Free Hadoop On-Demand Training
 
 http://www.mapr.com/training?utm_source=Emailutm_medium=Signatureutm_campaign=Free%20available
 

Re: Introducing Ruth - documentation

[Adam Bordelon]

We should start with our Confluence wiki:
https://cwiki.apache.org/confluence/display/MYRIAD/Myriad+Home
It's also recommended that we create a proper website at
http://myriad.incubator.apache.org/

I'd rather we didn't rely on gh-pages, especially since the github repo is
just a mirror of Apache git, so we can't actually use github PRs without
downloading the patches, applying them locally, and then pushing to Apache
git.

On Thu, Jul 30, 2015 at 4:42 PM, Will Ochandarena wochandar...@maprtech.com
 wrote:

 Myriad Team, I want to introduce you to Ruth Harris from the MapR
 documentation team.  She'll be leading the lead on documentation for
 Myriad, hopefully with ample help from each of you.

 She's been digging into Myriad for a few weeks now, and has a preliminary
 outline.  I'll let her reply to this and send it to start getting feedback.

 One thing she needs is a place to start posting to.  Looking at other
 Apache projects, it seems like the most common practice is to use a
 gh-pages branch of the main github
 https://github.com/apache/incubator-myriad.

 Adam - could you create the branch so she has something to send PRs to?
 Jim - you seem to be our resident web/css expert.  Any chance you could
 build a simple layout for the doc site?

 Thanks!
 Will

Re: Merging MyriadExecutor with NodeManager

[Adam Bordelon]

Thanks for the clear explanation, Swapnil. This sounds awesome. A much
cleaner design, and it seems like exactly the kind of thing YARN
AuxServices were created for.

On Thu, Jul 16, 2015 at 5:24 PM, Swapnil Daingade 
swapnil.daing...@gmail.com wrote:

 Hi All,

 Currently with Fine Grained Scheduling (FGS), the workflow for reporting
 status and relinquishing resources used
 by a YARN container is as following

 1. The NodeManager reports the status/completion of the container to the
 ResourceManager
  as part of container statuses included in the NM to RM heartbeat

 2. This container status is intercepted by the Myriad Scheduler. The
 Scheduler sends a
 frameworkMessage to the MyriadExecutor running on the NodeManager node.
 See NMHeartBeatHandler.handleStatusUpdate here


 https://github.com/mesos/myriad/blob/issue_14/myriad-scheduler/src/main/java/com/ebay/myriad/scheduler/NMHeartBeatHandler.java#L112

 3. This frameworkMessage instructs the MyriadExecutor to report the task
 state corresponding to the YARN container status back to mesos.
  See MyriadExecutor.frameworkMessage here


 https://github.com/mesos/myriad/blob/issue_14/myriad-executor/src/main/java/com/ebay/myriad/executor/MyriadExecutor.java#L252

 There are some disadvantages to this approach

 1. In step 2 we use SchedulerDriver.sendFrameworkMessage() API. According
 to the API documentation, this message is best effort.
   /**
 * Sends a message from the framework to one of its executors. These
 * messages are best effort; do not expect a framework message to be
 * retransmitted in any reliable fashion.

 2. This requires the Scheduler/RM to be up for YARN containers/Mesos Tasks
 to be able to report statuses to Mesos Master.
  If Scheduler/RM goes down, we will not be able to send task statuses
 to Mesos, until the Scheduler/RM is back up.
  This can lead to resource leakages.

 3. There is additional overhead of sending messages back from Scheduler/RM
 back to the Executors for each container on each
  heartbeat. (Number of yarn containers/node * Number of Nodes)
 additional messages.

 In order to avoid the above mentioned issues, we are proposing merging of
 the MyriadExecutor and NodeManager.
 The MyriadExecutor will run as a NM auxiliary service (same process as NM).
 It will be able to intercept YARN container completion locally and inform
 mesos-master irrespective of weather scheduler is running.
 We will no longer have to use the sendFrameworkMessage method.
 There will be less message traffic from scheduler to executor.

 I have posted my proposed changes as part of the pull request here
 https://github.com/mesos/myriad/pull/118

 Request you take a look and let me know your feedback.

 Regards
 Swapnil

Re: Fine Grained Scaling and Hadoop-2.7.

[Adam Bordelon]

Ah, I'm understanding better now. Leaving the 2G,1CPU unused is certainly
flawed and undesirable.
I'm unopposed to the idea of an initial/minimum profile size that grows and
shrinks but never goes below its initial/minimum capacity. As for your
concern, a recently completed task will give up its unnamed resources like
cpu and memory, without knowing/caring where they go. There is no
distinction between the cpu from one task and the cpu from another. First
priority goes to maintaining the minimum capacity. Anything beyond that can
be offered back to Mesos (perhaps after some timeout for promoting reuse).
The only concern might be with named resources like ports or persistent
volumes. Why do you worry that Myriad needs to figure out which container
is associated with which offer/profile? Is it not already tracking the YARN
containers? How else does it know when to release resources?
That said, a zero profile also makes sense, as does mixing profiles of
different sizes (including zero/non-zero) within a cluster. You could
restrict dynamic NM resizing to zero-profile NMs for starters, but I'd
imagine we'd want them all to be resizable in the future.

On Fri, Jul 10, 2015 at 6:47 PM, Santosh Marella smare...@maprtech.com
wrote:

  a) Give the executor at least a minimal 0.01cpu, 1MB RAM

 Myriad does this already. The problem is not with respect to executor's
 capacity.

  b) ... I don't think I understand your zero profile use case

 Let's take an example. Let's say the low profile corresponds to
 (2G,1CPU). When
 Myriad wants to launch a NM with low profile, it waits for a mesos offer
 that can
 hold an executor + a java process for NM + a (2G,1CPU) capacity that NM
 can advertise to RM for launching future YARN containers. With CGS,
 when NM registers with RM, YARN scheduler believes the NM has (2G,1CPU)
 and hence can allocate containers worth (2G,1CPU) when apps require
 containers.

 With FGS, YARN scheduler believes NM has (0G,0CPU). This is because, FGS
 intercepts NM's registration with RM and sets NM's advertised capacity to
 (0G,0CPU),
 although NM has originally started with (2G,1CPU).  At this point, YARN
 scheduler
 cannot allocate containers to this NM. Subsequently, when mesos offers
 resources
 on the same slave node, FGS increases the capacity of the NM and notifies
 RM
 that NM now has capacity available. For e.g. if (5G,4CPU) are offered to
 Myriad,
 then FGS notifies RM that the NM now has (5G,4CPU). RM can now allocate
 containers worth (5G,4CPU) for this NM. If you now count the total
 resources Myriad has consumed from the given slave node, we observe that
 Myriad
 never utilizes the (2G,1CPU) [low profile size] that was obtained at NM's
 launch time.
 The notion of a zero profile tries to eliminate this wastage by allowing
 NM to
 be launched with an advertisable capacity of (0G,0CPU) in the first place.

 Why does FGS change NM's initial capacity from (2G,1CPU) to (0G,0CPU)?
 That's the way it had been until now, but it need not be. FGS can choose to
 not reset
 NM's capacity to (0G,0CPU) and instead allow NM to grow beyond initial
 capacity of
 (2G,1CPU) and shrink back to (2G,1CPU). I tried this approach recently, but
 there
 are other problems if we do that (mentioned under option#1 in my first
 email) that
 seemed more complex than going with a zero profile.

  c)... We should still investigate pushing a disable flag into YARN.
 Absolutely. It totally makes sense to turn off admission restriction
 for auto-scaling YARN clusters.

 FWIW, I will be sending out a PR shortly from my private issue_14 branch
 with the changes I made so far. Comments/suggestions are welcome!

 Thanks,
 Santosh

 On Fri, Jul 10, 2015 at 11:44 AM, Adam Bordelon a...@mesosphere.io
 wrote:

  a) Give the executor at least a minimal 0.01cpu, 1MB RAM, since the
  executor itself will use some resources, and Mesos gets confused when the
  executor claims no resources. See
  https://issues.apache.org/jira/browse/MESOS-1807
  b) I agree 100% with needing a way to enable/disable FGS vs. CGS, but I
  don't think I understand your zero profile use case. I'd recommend
 going
  with a simple enable/disable flag for the MVP, and then we can extend it
  later if/when necessary.
  c) Interesting. Seems like a hacky workaround for the admission control
  problem, but I'm intrigued by its complexities and capabilities for other
  scenarios. We should still investigate pushing a disable flag into YARN.
   YARN-2604, YARN-3079. YARN-2604 seems to have been added because of a
   genuine problem where an app's AM container size exceeds the size of
 the
   largest NM node in the cluster.
  This still needs a way to be disabled, because an auto-scaling Hadoop
  cluster wouldn't worry about insufficient capacity. It would just make
  more.
 
  On Fri, Jul 10, 2015 at 11:13 AM, Santosh Marella smare...@maprtech.com
 
  wrote:
 
   Good point. YARN seems to have added this admission control as part of
   YARN-2604, YARN-3079. YARN

Re: Fine Grained Scaling and Hadoop-2.7.

[Adam Bordelon]

Ok, this makes sense now. With zero profile, tracking these will be much
easier, since each YARN container would have a placeholder task of the same
size.
But with an initial/minimum capacity, you'd need to do extra bookkeeping to
know how many resources belong to each task, what the initial NM capacity
was, and its current size. Then, when a task completes, you'll see how many
resources it was using, and determine whether some/all of those resources
should be freed and given back to Mesos, or whether they just go back to
idle minimum capacity for the NM. However, since Mesos doesn't (yet)
support resizeTask, you'd have to kill the placeholder task that best
matches the size of the completed task (even though that task may have
originally launched in the miminum capacity). Tricky indeed.
So, I like the idea of the zero-profile NM in that case, but it still
doesn't solve the problem of admission control of AMs/containers that are
bigger than the current cluster capacity. If we keep some minimum capacity
NMs that can resize with placeholder tasks, you run into the same problem
as above. The only options I can imagine are to a) use fixed-size NMs that
cannot grow, alongside the elastic zero-profile NMs; or b) disable
admission control in the RM so this isn't a problem. I'd vote for b), but
depending on how long that takes, you may want to implement a) in the
meantime.

On Tue, Jul 14, 2015 at 2:02 PM, Santosh Marella smare...@maprtech.com
wrote:

 Why do you worry that Myriad needs to figure out which container is
 associated with which offer/profile

 The framework needs to figure out the size of the placeholder task that
 it needs to launch corresponding
 to a YARN container. The size of the placeholder is not always 1:1 with
 the size of the YARN
 container (zero profile is trying to make it 1:1).

 Let's take an example flow:

 1. Let's say the NM's initial capacity was (4G,4CPU) and YARN wants to
 launch a container with size (2G, 2CPU). No problem. NM already has
 capacity
 to accommodate it. No need to wait for more offers or to launch placeholder
 mesos tasks.
 Just launch the YARN container via NM's HB.

 2. Let's say the NM's initial capacity was (4G,4CPU) and (2G,2CPU) is under
 use due to previously launched YARN container. If the RM's next request
 requires a container with (3G,3CPU), that container doesn't get allocated
 to
 this NM, since NM doesn't have enough capacity. No problem here too.

 3. Let's say mesos offers a (1G,1CPU) at this point. NM has (2G,2CPU)
 available and Myriad allows adding (1G,1CPU) to it. Thus, RM believes
 NM now has (3G,3CPU) and allocates a (3G,3CPU) container on the NM.
 At this point, since Myriad needs to use the launchTasks() API to
 launch a placeholder task with (1G,1CPU).

 Thanks,
 Santosh

 On Tue, Jul 14, 2015 at 1:12 AM, Adam Bordelon a...@mesosphere.io wrote:

  Ah, I'm understanding better now. Leaving the 2G,1CPU unused is certainly
  flawed and undesirable.
  I'm unopposed to the idea of an initial/minimum profile size that grows
 and
  shrinks but never goes below its initial/minimum capacity. As for your
  concern, a recently completed task will give up its unnamed resources
 like
  cpu and memory, without knowing/caring where they go. There is no
  distinction between the cpu from one task and the cpu from another. First
  priority goes to maintaining the minimum capacity. Anything beyond that
 can
  be offered back to Mesos (perhaps after some timeout for promoting
 reuse).
  The only concern might be with named resources like ports or persistent
  volumes. Why do you worry that Myriad needs to figure out which container
  is associated with which offer/profile? Is it not already tracking the
 YARN
  containers? How else does it know when to release resources?
  That said, a zero profile also makes sense, as does mixing profiles of
  different sizes (including zero/non-zero) within a cluster. You could
  restrict dynamic NM resizing to zero-profile NMs for starters, but I'd
  imagine we'd want them all to be resizable in the future.
 
  On Fri, Jul 10, 2015 at 6:47 PM, Santosh Marella smare...@maprtech.com
  wrote:
 
a) Give the executor at least a minimal 0.01cpu, 1MB RAM
  
   Myriad does this already. The problem is not with respect to executor's
   capacity.
  
b) ... I don't think I understand your zero profile use case
  
   Let's take an example. Let's say the low profile corresponds to
   (2G,1CPU). When
   Myriad wants to launch a NM with low profile, it waits for a mesos
  offer
   that can
   hold an executor + a java process for NM + a (2G,1CPU) capacity that NM
   can advertise to RM for launching future YARN containers. With CGS,
   when NM registers with RM, YARN scheduler believes the NM has (2G,1CPU)
   and hence can allocate containers worth (2G,1CPU) when apps require
   containers.
  
   With FGS, YARN scheduler believes NM has (0G,0CPU). This is because,
 FGS
   intercepts NM's registration with RM and sets NM's

Myriad DCOS Certification Checklist

[Adam Bordelon]

Now, as Mesosphere ramps up our DCOS Service Certification program, I would
like to share this checklist and the linked DCOS Service/CLI Specifications
with the Myriad mailing list. (Ping me with your preferred email if you
want checklist edit access.)

https://docs.google.com/document/d/1gr5hgFHgh2ZBAF4GkZNph8Y1idF_UsyiwGaImXsa6vM/edit

Many of these items are general requirements/recommendations for any
production-ready Mesos framework, and many of them are already satisfied by
Myriad or are currently in work. I know I gave some vague recommendations
before, but now it's all concrete and documented. Dig in! :)

Cheers,
-Adam-

Re: Logo update

[Adam Bordelon]

Sexy. Love the color scheme too.

On Thu, Jul 9, 2015 at 12:19 PM, Jim Klucar klu...@gmail.com wrote:

 I updated the logo PR with the new logo. Screenshot is in the comments.

 https://github.com/mesos/myriad/pull/113

Fwd: [jira] [Resolved] (INFRA-9841) Confluence Wiki for Myriad Podling

[Adam Bordelon]

We have a (confluence) wiki!

If you have problems editing, send me your cwiki id and I'll figure out how
to add you as an editor/admin.

-- Forwarded message --
From: David Nalley (JIRA) j...@apache.org
Date: Mon, Jun 22, 2015 at 6:54 PM
Subject: [jira] [Resolved] (INFRA-9841) Confluence Wiki for Myriad Podling
To: a...@mesosphere.io



 [
https://issues.apache.org/jira/browse/INFRA-9841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

David Nalley resolved INFRA-9841.
-
Resolution: Fixed

https://cwiki.apache.org/confluence/display/MYRIAD/Myriad+Home

Enjoy


 Confluence Wiki for Myriad Podling
 --

 Key: INFRA-9841
 URL: https://issues.apache.org/jira/browse/INFRA-9841
 Project: Infrastructure
  Issue Type: New Confluence Wiki
  Components: Confluence, Podling Setup
Reporter: Adam B





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Doe we have a meeting? - seems I can not connect EOM

[Adam Bordelon]

Please try again.
https://plus.google.com/hangouts/_/mesosphere.io/myriad?authuser=0

On Wed, Jun 3, 2015 at 9:06 AM, yuliya Feldman yufeld...@yahoo.com.invalid
wrote:

Re: Logo!

[Adam Bordelon]

I think I found it on deviantart a while back, but can't seem to find it
again.
Here's another alternative (with a skinnier elephant):
https://s-media-cache-ak0.pinimg.com/236x/78/d6/e1/78d6e1db46945b13da04241f06688ab9.jpg
Maybe we can commission somebody to create a version that we can freely
license/use.

On Thu, May 28, 2015 at 6:05 PM, Will Ochandarena wochandar...@maprtech.com
 wrote:

 It's nice.  Any idea who owns it?

 On Thu, May 28, 2015 at 5:58 PM, Adam Bordelon a...@mesosphere.io wrote:

  FWIW, I'm a fan of the flying elephant imagery
  http://s5.favim.com/orig/71/flying-art-birds-cute-Favim.com-669833.jpg
  Reminds me of Twitter's fail whale which Mesos helped eliminate
  http://media.downloadblog.it/2/290/twitter-fail-whale.jpg
 
  On Thu, May 28, 2015 at 5:52 PM, Will Ochandarena 
  wochandar...@maprtech.com
   wrote:
 
   Thanks everyone for the input.  I'll combine it and toss it out to the
   design community and we'll vote on what comes back.
  
   Will
  
   On Mon, May 25, 2015 at 5:57 AM, John Omernik j...@omernik.com
 wrote:
  
What are the rules in alluding to other project logos? Permission?
  How
   do
other projects feel about that?  Showing the Mesos triangles, and he
   having
a few elephants, perhaps in color distributed around, and he having
  other
triangles, perhaps with allusions to other logos like storm, spark,
   docker,
chronos, marathon, aurora, others?  To show what we are focusing on
 the
other logos or allusions to logos could be in grey while the
 elephants
could be colored (in a traditional blue?).   I wouldn't like showing
  the
elephant as bigger just highlighted as that's what we are doing.
   
Just a thought.
   
On Sunday, May 24, 2015, Jim Klucar klu...@gmail.com wrote:
   
 I like how Flink used the Apache feather colors in their squirrel
  logo.
 https://flink.apache.org/

 On Sat, May 23, 2015 at 11:08 AM, Ken Sipe k...@mesosphere.io
 javascript:; wrote:

  I like that... How about a couple of elephants sitting on top of
 a
mesos
  platform.
 
  Sent from my iPhone
 
   On May 22, 2015, at 9:19 PM, Brandon Gulla 
   gulla.bran...@gmail.com
 javascript:;
  wrote:
  
   I too like the triangle idea.
  
   Another idea: the hadoop elephant wearing a cape displaying the
   Mesos
  logo.
   Curled in its trunk is a ball of yarn dangling.
   On May 22, 2015 6:19 PM, Darin Johnson 
  dbjohnson1...@gmail.com
 javascript:;
  wrote:
  
   I like the idea of several pixelated(with mesos M's) possibly
  interesting.
   On May 22, 2015 7:07 PM, yuliya Feldman
yufeld...@yahoo.com.invalid
 
   wrote:
  
   Yeah - sorry - that is what I meant elephant per triangle,
 but
  to
be
   modest - not in all triangles, since people run other stuff -
besides
   hadoop
From: John Omernik j...@omernik.com javascript:;
   To: dev@myriad.incubator.apache.org javascript:;; yuliya
Feldman 
   yufeld...@yahoo.com javascript:;
   Sent: Friday, May 22, 2015 3:42 PM
   Subject: Re: Logo!
  
   Ooo, elephantS  :) (you can have a herd of elephants now!)
  
   :)
  
  
  
  
   On Fri, May 22, 2015 at 4:59 PM, yuliya Feldman
   yufeld...@yahoo.com.invalid
   wrote:
  
   How about small elephant in some of the little mesos
   triangles.
   From: Will Ochandarena wochandar...@maprtech.com
 javascript:;
   To: dev@myriad.incubator.apache.org javascript:;
   Cc: Nitin Bandugula nbandug...@maprtech.com
 javascript:;
   Sent: Friday, May 22, 2015 2:15 PM
   Subject: Logo!
  
   Dev team - now that the name has been locked down it's time
 we
 started
   building our identity by getting a logo.
  
   The MapR marketing team has agreed to fund and arrange for a
   logo,
 but
   in
   true Apache style we want to make this a community process.
  We
will
   soon
   reach out to an agency who will crowd-source the design,
   allowing
   everyone
   to vote on your favorite.  As a first step we need to come
 up
with a
   general idea of what the logo should be - a creative
 direction
   if
 you
   will.  Below are a couple of ideas, please chime in with
 your
   own.
  
   *Please reply by 5/27.*
  
   1. Represent joining of two worlds/communities
   -- May incorporate elements of Hadoop
   http://hadoop.apache.org/images/hadoop-logo.jpg and Mesos
   

 https://tctechcrunch2011.files.wordpress.com/2013/09/mesos_logo.png
   logos
  
   2. Expand on the word Myriad (countless or great number) -
  maybe
 write
   out
   myriad with a *myriad* of tiny dots
  
  
  
  
  
  
  
  
 

   
   
--
Sent from my iThing
   
  
 

Re: Mesos presentation for first China MUG meet up

[Adam Bordelon]

Looks great, Zhongyue! I had a few minor spelling corrections, but I really
like the content.
I look forward to seeing your demo video!

On Wed, May 6, 2015 at 8:52 AM, Zhongyue Luo zhongyue@gmail.com wrote:

 Strange, here is the link to the deck I've put on google drive.


 https://docs.google.com/presentation/d/1GvuV_CB-ecHV8aON4qrnNAfCSmsceP1dQULuXffmRjs/edit?usp=sharing


 On Wed, May 6, 2015 at 11:40 PM, Adam Bordelon a...@mesosphere.io wrote:

  Zhongyue, I don't see your slides linked/attached, so I can't provide any
  feedback.
  Regardless, here are the slides we used at our recent east-bay meetup:
  http://mesosphere.github.io/presentations-community/myriad-eastbay/#/
 
  On Wed, May 6, 2015 at 6:33 AM, Zhongyue Luo zhongyue@gmail.com
  wrote:
 
   Hi, team
  
   As a follow up of our last sync up meeting I am sending out the deck
 that
   I'll be presenting May 9th at the first China MUG meetup at Shenzhen.
  
   The slides are only in its first draft so, if interested, please let me
   know if anything isn't correct.
  
   Thanks.
  
   --
   *Intel SSG/STO/BDT*
   880 Zixing Road, Zizhu Science Park, Minhang District, 200241,
 Shanghai,
   China
   +862161166500
  
 



 --
 *Intel SSG/STO/BDT*
 880 Zixing Road, Zizhu Science Park, Minhang District, 200241, Shanghai,
 China
 +862161166500

Re: proposal to change time for Myriad biweekly meeting

[Adam Bordelon]

I'm not opposed, but 9-10am Pacific is lunchtime for our east-coasters,
which is why I originally suggested 10am.
We could do before lunch east-coast, meaning 8am Pacific, but that's really
early for this night owl. :(
Thoughts, suggestions?

On Mon, May 4, 2015 at 11:14 AM, yuliya Feldman yufeld...@yahoo.com.invalid
 wrote:

 Hello guys,
 I wanted to propose to change time for Myriad biweekly hangouts.Could we
 do 9 AM to 10 AM instead?
 As far as I remember we have people form China and for them it is very
 late and of course out of selfish reasons - I have recurring meeting I can
 not change from 10 to 11 AM.
 Thanks,Yuliya

Re: Mesos frameworks and roles

[Adam Bordelon]

We've discussed multi-role frameworks before (JIRA:
https://issues.apache.org/jira/browse/MESOS-1142 ), but haven't yet found
the strong validating use case that would drive the feature forward. Many
users will instead just run N instances of the same framework, each with a
different role. What use case are you thinking of?

On Mon, Apr 27, 2015 at 11:32 PM, Ken Sipe k...@mesosphere.io wrote:

 yes… currently a framework can only be associated with 1 role.

 - ken
  On Apr 27, 2015, at 8:22 PM, yuliya Feldman yufeld...@yahoo.com.INVALID
 wrote:
 
  Hello here,
  This question is really for Mesos guys :).
  roles and frameworks in Mesos. As far as I understand today each Mesos
 framework can be associated with a role and subsequently mesos-slave can
 allocate shares of resources per role.
  Is it still the case? Meaning we still have no more then one role
 associated with a framework.
  Thanks

Re: Myriad incubator projects

[Adam Bordelon]

Yup. Luciano has been helping us file the INFRA tickets to get these
created.
So far, the dev@ mailing list works, and the (empty) git repo has been
created.
We're waiting for ebay to officially donate the source before we update the
license and migrate the repo.
We still need to create tickets for the JIRA (INFRA ticket closed due to
wrong type), webpage, and wiki.
And update the status page with committers who have submitted ICLAs.

On Sun, Apr 19, 2015 at 10:07 PM, yuliya Feldman 
yufeld...@yahoo.com.invalid wrote:

 Hello here,
 When I am looking at: Myriad Incubator Status - Apache Incubator  link
 that suppose to be Myriad incubator project none of the links are workable
 there (404).
 Either I am looking at the wrong place, or we need at least to fix those
 links.
 Thanks,Yuliya

Re: Myriad sync next week

[Adam Bordelon]

Yup. I meant 10am Pacific. Sounds like Wednesday is easier for most. Sorry,
Ken.
Sending out an hangout invite for Wed 10am PST and every 2 weeks thereafter.
I have to drop out at 10:30 this week, but feel free to carry on without me
(take notes!).
We can also adjust this earlier/later for future meetings as schedules
dictate.

On Tue, Apr 7, 2015 at 10:34 AM, mohit soni mohitsoni1...@gmail.com wrote:

 Wednesday 10am works for me as well. I have an overlapping meeting on
 Friday until 10:30am PST that I cannot move around, but if it works for
 everyone please go ahead.

 Regards
 Mohit

 On Tue, Apr 7, 2015 at 6:45 AM, Ken Sipe k...@mesosphere.io wrote:

  10am… what timezone are you referring to.   Assuming PST, tomorrow is a
  challenge for me, Friday works.
 
  ken
 
   On Apr 7, 2015, at 2:49 AM, Adam Bordelon a...@mesosphere.io wrote:
  
   How about Wednesday(s) or Friday(s) at 10am?
  
   On Fri, Apr 3, 2015 at 11:21 AM, Santosh Marella 
 smare...@maprtech.com
   wrote:
  
   AM (PDT) on all days next week are free for me.
  
   +1 for recurring meetings every 2 weeks.
  
   Santosh
  
   On Thu, Apr 2, 2015 at 12:03 AM, Adam Bordelon a...@mesosphere.io
  wrote:
  
   Just wanted to start a thread to schedule the next Myriad sync.
   My T/Th are filling up, so I would prefer a M/W/F (but not before
  10:30am
   Mon)
   How about the rest of you? Maybe there's even a day/time for a
  recurring
   meeting?