Re: Regarding Big Data Mesos Frameworks and Myriad

yes, you don't need a new Mesos framework for your YARN apps with Myriad. Regards Swapnil On Tue, Jan 30, 2018 at 10:25 PM, Javi Roman wrote: > Hi there! > > I have seen that this project probably is in the end of its life, it's > a real shame. > > I would like

Re: Podling Report Reminder - July 2017

), Darin Johnson <dbjohnson1...@gmail.com> wrote: Swapnil, generally a request on incubator does it. On Jul 7, 2017 5:23 AM, "Swapnil Daingade" <swapnil.daing...@gmail.com> wrote: I created an account on the wiki but was not able to edit the page. Do I need permission

Re: Podling Report Reminder - July 2017

I created an account on the wiki but was not able to edit the page. Do I need permissions to edit it? I'll try to figure it out but any pointers would be helpful. My account name is sdaingade Regards Swapnil On 07/06/2017 08:20 PM, John D. Ament wrote: Ping. Based on the active dicsussion

Re: [Vote] Retire Myriad

contributors (but not any > explicit votes from committers). > > > On Jun 22, 2017 6:09 PM, "Swapnil Daingade" <swapnil.daing...@gmail.com> > wrote: > > > yes, for more discussion. > > > > Unfortunately my situation too is no different, at least for the n

Re: [Vote] Retire Myriad

1...@gmail.com> > > To: Dev <dev@myriad.incubator.apache.org> > > Sent: Tuesday, June 20, 2017 11:19 AM > > Subject: Re: [Vote] Retire Myriad > > > > Swapnil: I only counted myself and Adam, if there's two additional > > committers who are willi

Re: [Vote] Retire Myriad

I am trying to understand what changed since the last discussion. If I remember correctly, Ted asked for 3-5 committers to vet the next release. 4 committers said they were willing. Did I miss something ? Regards Swapnil On Tue, Jun 20, 2017 at 7:35 AM, John Yost wrote:

Fwd: Is Apache Myriad dead?

I think I ended up replying only to Ted last night. -- Forwarded message -- From: Swapnil Daingade <swapnil.daing...@gmail.com> Date: Wed, Jun 7, 2017 at 1:07 AM Subject: Re: Is Apache Myriad dead? To: Ted Dunning <ted.dunn...@gmail.com> Please see inline On Mon,

Re: Is Apache Myriad dead?

rom incubation. I suggest we start with the contributions first. Regards Swapnil On Mon, Jun 5, 2017 at 2:53 PM, Ted Dunning <ted.dunn...@gmail.com> wrote: > > On Mon, Jun 5, 2017 at 10:15 PM, Swapnil Daingade < > swapnil.daing...@gmail.com> wrote: > >> In that case I s

Re: Is Apache Myriad dead?

In that case I suggest we not retire >> "Darin - yes we've done more planning internally, and we do plan on having some engineers spend some time on this project, doing some (minor) maintenance for our customers." Regards Swapnil On Mon, Jun 5, 2017 at 7:34 AM, Will Ochandarena

Re: Myraid Slack

Sent from my iPhone On Jun 23, 2016, at 10:39 AM, Swapnil Daingade <swapnil.daing...@gmail.com> wrote: MapR supports Myriad 0.1 currently https://www.mapr.com/products/whats-included https://www.mapr.com/products/product-overview/apache-myriad Regards Swapnil On Wed, Jun 22, 2016 a

Myriad hangout tomorrow?

Hi All, Was wondering if we have a Myriad hangout tomorrow. Regards Swapnil

[jira] [Commented] (MYRIAD-134) Support Zookeeper based implementation of RMStateStore for storing Myriad state

[ https://issues.apache.org/jira/browse/MYRIAD-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312461#comment-15312461 ] Swapnil Daingade commented on MYRIAD-134: - You will need to extend the ZKRMStateStore that Hadoop

Re: Questions about Myriad background

Great Questions Dave! Here is what I think 1. The current way an organization would run Yarn and Mesos is to have two separate clusters with dedicated resources (compute, storage, networking). This, I feel, is not the best use of the clusters as resources cannot be shared. You could have one

Fwd: Showcase your project at ApacheCON: consider Podling Shark Tank

Folks going to ApacheCon might want to consider participating. Regards Swapnil -- Forwarded message -- From: Roman Shaposhnik Date: Mon, Apr 25, 2016 at 1:27 PM Subject: Showcase your project at ApacheCON: consider Podling Shark Tank To: r...@apache.org Hi!

Re: Build failure leaves workspace in inconsistent state

with our webui which fails during clean unless I remove that directory by hand so that it can be re-built Todd On Mar 26, 2016, at 10:43 AM, Swapnil Daingade <swapnil.daing...@gmail.com> wrote: Hi All, I have faced this issue a couple of times and wondering if anyone has a way t

[jira] [Assigned] (MYRIAD-193) Umbrella JIRA for security in Myriad

[ https://issues.apache.org/jira/browse/MYRIAD-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Daingade reassigned MYRIAD-193: --- Assignee: Swapnil Daingade > Umbrella JIRA for security in Myr

Re: 0.2.0 release

Hi Darin, I feel docker networking is something we should spent time to think through. A user should be able to use multiple options provided by Mesos, Docker, 3rd party etc It would be great if we can abstract the specific implementation to provide container ip addresses behind interfaces. User

Re: 0.2.0 release

+1 Darin for RM. Would like be involved in the multi-tenency with Dockerized NM work. Perhaps we can start filing JIRA's targeted for 0.2 and start adding design proposals for people to review and comment. Regards Swapnil On Thu, Mar 17, 2016 at 8:13 AM, Darin Johnson

Re: Myriad Vagrant Setup Issue

Hi Matt, Can you please check your yarn-site.xml for the following property yarn.nodemanager.aux-services The value for the property should contain 'myriad _executor' in addition to other entries. The various aux service entries are comma separated and there is no space between entries. The

Re: Myriad Vagrant Setup Issue

Hi Matt, Looks like the Mesos slave now launches the NodeManager mesos task. However, the NodeManager seems to be dying after a while. The log files for the NodeManager Mesos task (that Darin mentioned) below should help figure out why the NodeManager died. Could you please post those files as

Re: Myriad is 0.1.0

Congratulations and thank you everyone for all the hard work!!! Special thanks to Santosh for being an awesome release manager!!! Regards Swapnil On Fri, Dec 11, 2015 at 12:11 AM, Darin Johnson wrote: > Sweet, thanks for all the work on the release guys! > > On Thu,

Re: [PROPOSAL(s)] Use Release Branches, and Delete Obsolete Branches

I did not understand the difference between Proposal A and Alt 1 (the way Adam described it). Seem quite similar to me +1 A or Alt1 +1 B Regards Swapnil On Fri, Dec 4, 2015 at 8:42 AM, Darin Johnson wrote: > My experience with alt 1 is it takes a lot of discipline or

Re: Oversubscription in Mesos

Haven't tried it but this might be possible using label based scheduling (MapR/YARN). I am thinking if all CGS nodes are labeled 'Production' and FGS ones 'Non Production' and there are two queues one labeled 'Production' and the other 'Non Production'. If a job is submitted to the Non Production

Re: [VOTE] Release apache-myriad-0.1.0-incubating (release candidate 3)

Downloaded rc3 tar.gz * Verified md5 and sha512 hashes successfully * Built binaries successfully * Deployed on 3 node MapR cluster * Tested NM flexup/flexdown with HA enabled and disabled. * Tried HA * Tried Framework Shutdown. All operations worked as expected. +1 Regards Swapnil On Fri, Nov

Re: [VOTE] Release apache-myriad-0.1.0-incubating (release candidate 1)

Downloaded rc1 tar.gz * Verified md5 and sha512 hashes successfully * Built binaries successfully * Deployed on 3 node MapR cluster * Tested NM flexup/flexdown with HA enabled and disabled. * Tried Framework Shutdown. All operations worked as expected. +1 Regards Swapnil On Tue, Nov 10, 2015

Re: [VOTE] Release apache-myriad-0.1.0-incubating (release candidate 0)

Downloaded rc0 tar.gz. Built and deployed on 3 node MapR cluster. Tested NM flexup/flexdown with HA enabled and disabled. Tried Framework Shutdown. All operations worked as expected. Will try out rc1 if we plan to cut one. Else +1 Regards Swapnil On Mon, Nov 9, 2015 at 5:54 PM, yuliya Feldman

Re: New Committer: Swapnil Daingade

t; > > Well done > > > Yuliya > > > From: Adam Bordelon <a...@mesosphere.io> > > > To: dev@myriad.incubator.apache.org > > > Sent: Thursday, November 5, 2015 4:38 AM > > > Subject: New Committer: Swapnil Daingade > > > > > > The Podling Pro

[jira] [Resolved] (MYRIAD-154) Restarting RM starts a "default" medium-profile NM on each restart.

[ https://issues.apache.org/jira/browse/MYRIAD-154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Daingade resolved MYRIAD-154. - Resolution: Fixed Fix Version/s: Myriad 0.1.0 Fixed as part of commit https

[jira] [Assigned] (MYRIAD-170) Myriad initialization fails with "parameter 5 of org.apache.myriad.scheduler.MyriadOperations.() is not @Nullable"

[ https://issues.apache.org/jira/browse/MYRIAD-170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Daingade reassigned MYRIAD-170: --- Assignee: Swapnil Daingade > Myriad initialization fails with "par

[jira] [Assigned] (MYRIAD-156) NullPointerException from "Error in handling event type NODE_RESOURCE_UPDATE to the scheduler"

[ https://issues.apache.org/jira/browse/MYRIAD-156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Daingade reassigned MYRIAD-156: --- Assignee: Swapnil Daingade > NullPointerException from "Error in handling ev

[jira] [Assigned] (MYRIAD-164) Myriad fails to start up again after Shutdown is called

[ https://issues.apache.org/jira/browse/MYRIAD-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Daingade reassigned MYRIAD-164: --- Assignee: Swapnil Daingade > Myriad fails to start up again after Shutdown is cal

[jira] [Commented] (MYRIAD-133) Multiple flexed up NMs try to run on same node, altogether.

[ https://issues.apache.org/jira/browse/MYRIAD-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14962082#comment-14962082 ] Swapnil Daingade commented on MYRIAD-133: - Fixed as part of commit https://github.com/apache

[jira] [Commented] (MYRIAD-139) CapacityScheduler doesn't work with Fine Grained Scaling

[ https://issues.apache.org/jira/browse/MYRIAD-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14953839#comment-14953839 ] Swapnil Daingade commented on MYRIAD-139: - Filed yarn JIRA for this here https://issues.apache.org

[jira] [Commented] (MYRIAD-139) CapacityScheduler doesn't work with Fine Grained Scaling

[ https://issues.apache.org/jira/browse/MYRIAD-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936073#comment-14936073 ] Swapnil Daingade commented on MYRIAD-139: - Oh, My bad. I looked at the CapacityScheduler code

Re: Reg Basic Myriad functionality, referring to MesosCon 2015 talk slides

RM is just another Mesos task if launched using something like Marathon. RM in turn will register with Mesos as another framework. You will need mesos slave to launch a mesos task. The RM mesos task can get launched on any node having a Mesos slave. You can technically launch a RM on any node

Re: Getting Nodes to be "Running" in Mesos

Hi John, We recently merged NodeManager with Mesos executor (change got merged on Saturday). If you built and deployed Myriad after that, can you please add the following entries to your yarn-site.xml Also, if you are using the MapR distro. Please use the MapR specific value for the

[jira] [Created] (MYRIAD-132) Support active-standby RM in Myriad HA implementation

Swapnil Daingade created MYRIAD-132: --- Summary: Support active-standby RM in Myriad HA implementation Key: MYRIAD-132 URL: https://issues.apache.org/jira/browse/MYRIAD-132 Project: Myriad

Re: Complete Myriad HA implementation

t; wrote: > > > Swapnil, > > > > Generally no, they don't which makes it confusing and so I asked. > Started > > playing with it, going to be travelling tomorrow but will try out soon. > > > > Darin > > > > > > On Mon, Aug 24, 2015 at

Re: [jira] [Updated] (MYRIAD-128) Issue with Flex down, Pending NMs stuck in staging and don't get to active task.

On Aug 26, 2015 12:02 PM, "Swapnil Daingade" <swapnil.daing...@gmail.com> > wrote: > > > Hi Jim, > > > > I was also working on this as part of the review comments that I received > > for the Myriad HA changes. > > Are you too far along in fix

Re: Not receiving calls to Executor after Scheduler restart

'[ 20150826-222858-2321877514-5050-1026-O645 ]': Offer 20150826-222858-2321877514-5050-1026-O645 is no longer valid. I am adding a fix to this and will update the pull request. Regards Swapnil On Thu, Aug 27, 2015 at 2:39 AM, Swapnil Daingade swapnil.daing...@gmail.com wrote: The Apache email server

Re: Complete Myriad HA implementation

that's a local fs in /var/ if that's we're you're keeping the state how is it regained if the RM is restarted on a different node in marathon? Haven't read through all the code yet but I'm trying to get oriented, sorry. On Mon, Aug 24, 2015 at 9:35 PM, Swapnil Daingade swapnil.daing

[jira] [Created] (MYRIAD-127) Recommend nodes to launch NodeManagers optimized for locality based on input data for Yarn jobs

Swapnil Daingade created MYRIAD-127: --- Summary: Recommend nodes to launch NodeManagers optimized for locality based on input data for Yarn jobs Key: MYRIAD-127 URL: https://issues.apache.org/jira/browse/MYRIAD

Not receiving calls to Executor after Scheduler restart

Hi All, Need help with a Mesos issue. I am currently working on the Myriad HA implementation. Currently we launch NodeManager as a mesos task. With Fine grained scaling, we also launch yarn containers as mesos tasks. I learned from Santosh that in order to launch tasks on the same executor, the

[jira] [Assigned] (MYRIAD-13) High Availability Mode for Myriad

[ https://issues.apache.org/jira/browse/MYRIAD-13?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Daingade reassigned MYRIAD-13: -- Assignee: Swapnil Daingade High Availability Mode for Myriad

[jira] [Assigned] (MYRIAD-118) Changes for merging MyriadExecutor with NodeManager.

[ https://issues.apache.org/jira/browse/MYRIAD-118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swapnil Daingade reassigned MYRIAD-118: --- Assignee: Swapnil Daingade Changes for merging MyriadExecutor with NodeManager

Complete Myriad HA implementation

Hi All, I have updated my pull request with the complete Myriad HA implementation rebased on top of the FGS changes here https://github.com/mesos/myriad/pull/123 I am planning to send out another email with details on how to configure it. Regards Swapnil

Myriad HA state store implementation

Hi all, I have sent a pull request for Myriad HA state store implementation here https://github.com/mesos/myriad/pull/123 * I want to mention that the design for Myriad HA has been strongly influenced by Paul Read's work here https://github.com/pdread100/myriad/tree/issue-13 and here

Myriad HA design doc

Hi All, I am posting a draft of the Myriad HA design doc using Google docs. You should be able to use the link below to add your comments. https://docs.google.com/document/d/1BkcDChhOLU5TDU6ZQEpIh-WBKoCwYPPi9OV-__mQlmQ/edit?usp=sharing Please let me know your thoughts and suggestions. I should

Re: Issues 16 and Issue 12

Hi Darin, The Myriad HA work will involve work related to issue 16. I already have the Myriad HA design doc for review. Your feedback on it would be really helpful. I also plan to send out for review parts of the Myriad HA implementation (although it does not address task reconciliation yet). I

Re: Merging MyriadExecutor with NodeManager

it working. Please let me know if you face any issues or any other feedback you may have. Regards Swapnil On Fri, Jul 17, 2015 at 8:08 AM, Swapnil Daingade swapnil.daing...@gmail.com wrote: Should work with 2.5 as earlier (I tested on single node mapr cluster with hadoop-common 2.5.1). Only

Merging MyriadExecutor with NodeManager

Hi All, Currently with Fine Grained Scheduling (FGS), the workflow for reporting status and relinquishing resources used by a YARN container is as following 1. The NodeManager reports the status/completion of the container to the ResourceManager as part of container statuses included in the

Re: Logo!

Here's another similar one, superhero lifting an elephant http://www.dreamstime.com/stock-illustration-cute-superhero-cartoon-figure-lifting-elephant-vector-illustration-image42572048 Regards Swapnil On Thu, May 28, 2015 at 6:20 PM, Adam Bordelon a...@mesosphere.io wrote: I think I found it