[jira] [Commented] (MYRIAD-171) Hadoop-2.7.1 UpdateFIle

2015-11-06 Thread Santosh Marella (JIRA) 

[ 
https://issues.apache.org/jira/browse/MYRIAD-171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994655#comment-14994655
 ] 

Santosh Marella commented on MYRIAD-171:


Since Myriad has a compile dependency on hadoop 2.7.0, it'd work best with 
hadoop 2.7.0 and may not work well with other hadoop versions with API changes 
like this. 

Are you ok if we do a myriad 0.1.0 release with this restriction (i.e. Myriad 
works just with hadoop 2.7.0), and fix the binary compatibility issues with 
other hadoop versions in Myriad 0.2.0? 

> Hadoop-2.7.1 UpdateFIle
> ---
>
> Key: MYRIAD-171
> URL: https://issues.apache.org/jira/browse/MYRIAD-171
> Project: Myriad
>  Issue Type: Bug
>Affects Versions: Myriad 0.1.0
>Reporter: Brandon Gulla
>Assignee: Brandon Gulla
>Priority: Blocker
>
> Hadoop 2.7.1 introduced an API changed that breaks Myriad when using HA 
> ResourceManager. They added an extra argument to the method that we are 
> currently using. 
> To get around this, switch hadoop version to 2.7.0 or wait for the impending 
> code change. 
> Hadoop 2.7.1
> https://github.com/apache/hadoop/blob/release-2.7.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
> Hadoop 2.7.0
> https://github.com/apache/hadoop/blob/release-2.7.0/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
> I1106 19:26:55.098060  5135 sched.cpp:164] Version: 0.25.0
> 15/11/06 19:26:55 INFO scheduler.MyriadDriver: Driver started with status: 
> DRIVER_RUNNING
> 15/11/06 19:26:55 INFO scheduler.MyriadDriverManager: Driver started with 
> status: DRIVER_RUNNING
> 15/11/06 19:26:55 INFO myriad.Main: started mesosDriver..
> 15/11/06 19:26:55 INFO interceptor.CompositeInterceptor: Registered 
> org.apache.myriad.policy.LeastAMNodesFirstPolicy into the registry.
> I1106 19:26:55.100406  5185 sched.cpp:262] New master detected at 
> master@172.31.45.229:5050
> I1106 19:26:55.100591  5185 sched.cpp:272] No credentials provided. 
> Attempting to register without authentication
> 15/11/06 19:26:55 INFO myriad.Main: Launching 1 NM(s) with profile medium
> I1106 19:26:55.102533  5179 sched.cpp:641] Framework registered with 
> 2f8adca0-12da-4200-867d-2b742955cabd-0004
> 15/11/06 19:26:55 INFO scheduler.MyriadOperations: Adding 1 NM instances to 
> cluster
> 15/11/06 19:26:55 INFO handlers.RegisteredEventHandler: Received event: 
> org.apache.myriad.scheduler.event.RegisteredEvent@6a005de5 with frameworkId: 
> value: "2f8adca0-12da-4200-867d-2b742955cabd-0004"
> Nov 06, 2015 7:26:55 PM com.lmax.disruptor.FatalExceptionHandler 
> handleEventException
> SEVERE: Exception processing: 0 
> org.apache.myriad.scheduler.event.RegisteredEvent@6a005de5
> java.lang.NoSuchMethodError: 
> org.apache.hadoop.yarn.server.resourcemanager.recovery.MyriadFileSystemRMStateStore.updateFile(Lorg/apache/hadoop/fs/Path;[B)V
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.recovery.MyriadFileSystemRMStateStore.storeMyriadState(MyriadFileSystemRMStateStore.java:95)
>   at 
> org.apache.myriad.state.SchedulerState.updateStateStore(SchedulerState.java:374)
>   at 
> org.apache.myriad.state.SchedulerState.setFrameworkId(SchedulerState.java:362)
>   at 
> org.apache.myriad.scheduler.event.handlers.RegisteredEventHandler.onEvent(RegisteredEventHandler.java:42)
>   at 
> org.apache.myriad.scheduler.event.handlers.RegisteredEventHandler.onEvent(RegisteredEventHandler.java:30)
>   at 
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> 15/11/06 19:26:55 INFO handlers.ResourceOffersEventHandler: Received offers 3
> 15/11/06 19:26:55 ERROR yarn.YarnUncaughtExceptionHandler: Thread 
> Thread[pool-6-thread-1,5,main] threw an Exception.
> java.lang.RuntimeException: java.lang.NoSuchMethodError: 
> org.apache.hadoop.yarn.server.resourcemanager.recovery.MyriadFileSystemRMStateStore.updateFile(Lorg/apache/hadoop/fs/Path;[B)V
>   at 
> com.lmax.disruptor.FatalExceptionHandler.handleEventException(FatalExceptionHandler.java:45)
>   at 
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:147)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by:

[jira] [Commented] (MYRIAD-171) Hadoop-2.7.1 UpdateFIle

2015-11-06 Thread Brandon Gulla (JIRA) 

[ 
https://issues.apache.org/jira/browse/MYRIAD-171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994648#comment-14994648
 ] 

Brandon Gulla commented on MYRIAD-171:
--

Bingo.

Looks like I should be able to hit the RM rest-endpoint and pull the version 
from there. Hopefully there is another way that doesn't pose some race 
conditions, but I'll explore that further. 

https://github.com/apache/hadoop/blob/f8adeb712dc834c27cec15c04a986f2f635aba83/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md

> Hadoop-2.7.1 UpdateFIle
> ---
>
> Key: MYRIAD-171
> URL: https://issues.apache.org/jira/browse/MYRIAD-171
> Project: Myriad
>  Issue Type: Bug
>Affects Versions: Myriad 0.1.0
>Reporter: Brandon Gulla
>Assignee: Brandon Gulla
>Priority: Blocker
>
> Hadoop 2.7.1 introduced an API changed that breaks Myriad when using HA 
> ResourceManager. They added an extra argument to the method that we are 
> currently using. 
> To get around this, switch hadoop version to 2.7.0 or wait for the impending 
> code change. 
> Hadoop 2.7.1
> https://github.com/apache/hadoop/blob/release-2.7.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
> Hadoop 2.7.0
> https://github.com/apache/hadoop/blob/release-2.7.0/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
> I1106 19:26:55.098060  5135 sched.cpp:164] Version: 0.25.0
> 15/11/06 19:26:55 INFO scheduler.MyriadDriver: Driver started with status: 
> DRIVER_RUNNING
> 15/11/06 19:26:55 INFO scheduler.MyriadDriverManager: Driver started with 
> status: DRIVER_RUNNING
> 15/11/06 19:26:55 INFO myriad.Main: started mesosDriver..
> 15/11/06 19:26:55 INFO interceptor.CompositeInterceptor: Registered 
> org.apache.myriad.policy.LeastAMNodesFirstPolicy into the registry.
> I1106 19:26:55.100406  5185 sched.cpp:262] New master detected at 
> master@172.31.45.229:5050
> I1106 19:26:55.100591  5185 sched.cpp:272] No credentials provided. 
> Attempting to register without authentication
> 15/11/06 19:26:55 INFO myriad.Main: Launching 1 NM(s) with profile medium
> I1106 19:26:55.102533  5179 sched.cpp:641] Framework registered with 
> 2f8adca0-12da-4200-867d-2b742955cabd-0004
> 15/11/06 19:26:55 INFO scheduler.MyriadOperations: Adding 1 NM instances to 
> cluster
> 15/11/06 19:26:55 INFO handlers.RegisteredEventHandler: Received event: 
> org.apache.myriad.scheduler.event.RegisteredEvent@6a005de5 with frameworkId: 
> value: "2f8adca0-12da-4200-867d-2b742955cabd-0004"
> Nov 06, 2015 7:26:55 PM com.lmax.disruptor.FatalExceptionHandler 
> handleEventException
> SEVERE: Exception processing: 0 
> org.apache.myriad.scheduler.event.RegisteredEvent@6a005de5
> java.lang.NoSuchMethodError: 
> org.apache.hadoop.yarn.server.resourcemanager.recovery.MyriadFileSystemRMStateStore.updateFile(Lorg/apache/hadoop/fs/Path;[B)V
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.recovery.MyriadFileSystemRMStateStore.storeMyriadState(MyriadFileSystemRMStateStore.java:95)
>   at 
> org.apache.myriad.state.SchedulerState.updateStateStore(SchedulerState.java:374)
>   at 
> org.apache.myriad.state.SchedulerState.setFrameworkId(SchedulerState.java:362)
>   at 
> org.apache.myriad.scheduler.event.handlers.RegisteredEventHandler.onEvent(RegisteredEventHandler.java:42)
>   at 
> org.apache.myriad.scheduler.event.handlers.RegisteredEventHandler.onEvent(RegisteredEventHandler.java:30)
>   at 
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> 15/11/06 19:26:55 INFO handlers.ResourceOffersEventHandler: Received offers 3
> 15/11/06 19:26:55 ERROR yarn.YarnUncaughtExceptionHandler: Thread 
> Thread[pool-6-thread-1,5,main] threw an Exception.
> java.lang.RuntimeException: java.lang.NoSuchMethodError: 
> org.apache.hadoop.yarn.server.resourcemanager.recovery.MyriadFileSystemRMStateStore.updateFile(Lorg/apache/hadoop/fs/Path;[B)V
>   at 
> com.lmax.disruptor.FatalExceptionHandler.handleEventException(FatalExceptionHandler.java:45)
>   at 
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:147)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by:

[jira] [Commented] (MYRIAD-171) Hadoop-2.7.1 UpdateFIle

2015-11-06 Thread Brandon Gulla (JIRA) 

[ 
https://issues.apache.org/jira/browse/MYRIAD-171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994663#comment-14994663
 ] 

Brandon Gulla commented on MYRIAD-171:
--

2.7.0, Absolutely. The defined hadoop version in the docker build is 2.7.1 
currently but I can that patch tonight. 

I think taking our time with the reflection and setting up a ShimLoader will 
benefit us greater in the long run. 



> Hadoop-2.7.1 UpdateFIle
> ---
>
> Key: MYRIAD-171
> URL: https://issues.apache.org/jira/browse/MYRIAD-171
> Project: Myriad
>  Issue Type: Bug
>Affects Versions: Myriad 0.1.0
>Reporter: Brandon Gulla
>Assignee: Brandon Gulla
>Priority: Blocker
>
> Hadoop 2.7.1 introduced an API changed that breaks Myriad when using HA 
> ResourceManager. They added an extra argument to the method that we are 
> currently using. 
> To get around this, switch hadoop version to 2.7.0 or wait for the impending 
> code change. 
> Hadoop 2.7.1
> https://github.com/apache/hadoop/blob/release-2.7.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
> Hadoop 2.7.0
> https://github.com/apache/hadoop/blob/release-2.7.0/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/FileSystemRMStateStore.java
> I1106 19:26:55.098060  5135 sched.cpp:164] Version: 0.25.0
> 15/11/06 19:26:55 INFO scheduler.MyriadDriver: Driver started with status: 
> DRIVER_RUNNING
> 15/11/06 19:26:55 INFO scheduler.MyriadDriverManager: Driver started with 
> status: DRIVER_RUNNING
> 15/11/06 19:26:55 INFO myriad.Main: started mesosDriver..
> 15/11/06 19:26:55 INFO interceptor.CompositeInterceptor: Registered 
> org.apache.myriad.policy.LeastAMNodesFirstPolicy into the registry.
> I1106 19:26:55.100406  5185 sched.cpp:262] New master detected at 
> master@172.31.45.229:5050
> I1106 19:26:55.100591  5185 sched.cpp:272] No credentials provided. 
> Attempting to register without authentication
> 15/11/06 19:26:55 INFO myriad.Main: Launching 1 NM(s) with profile medium
> I1106 19:26:55.102533  5179 sched.cpp:641] Framework registered with 
> 2f8adca0-12da-4200-867d-2b742955cabd-0004
> 15/11/06 19:26:55 INFO scheduler.MyriadOperations: Adding 1 NM instances to 
> cluster
> 15/11/06 19:26:55 INFO handlers.RegisteredEventHandler: Received event: 
> org.apache.myriad.scheduler.event.RegisteredEvent@6a005de5 with frameworkId: 
> value: "2f8adca0-12da-4200-867d-2b742955cabd-0004"
> Nov 06, 2015 7:26:55 PM com.lmax.disruptor.FatalExceptionHandler 
> handleEventException
> SEVERE: Exception processing: 0 
> org.apache.myriad.scheduler.event.RegisteredEvent@6a005de5
> java.lang.NoSuchMethodError: 
> org.apache.hadoop.yarn.server.resourcemanager.recovery.MyriadFileSystemRMStateStore.updateFile(Lorg/apache/hadoop/fs/Path;[B)V
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.recovery.MyriadFileSystemRMStateStore.storeMyriadState(MyriadFileSystemRMStateStore.java:95)
>   at 
> org.apache.myriad.state.SchedulerState.updateStateStore(SchedulerState.java:374)
>   at 
> org.apache.myriad.state.SchedulerState.setFrameworkId(SchedulerState.java:362)
>   at 
> org.apache.myriad.scheduler.event.handlers.RegisteredEventHandler.onEvent(RegisteredEventHandler.java:42)
>   at 
> org.apache.myriad.scheduler.event.handlers.RegisteredEventHandler.onEvent(RegisteredEventHandler.java:30)
>   at 
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> 15/11/06 19:26:55 INFO handlers.ResourceOffersEventHandler: Received offers 3
> 15/11/06 19:26:55 ERROR yarn.YarnUncaughtExceptionHandler: Thread 
> Thread[pool-6-thread-1,5,main] threw an Exception.
> java.lang.RuntimeException: java.lang.NoSuchMethodError: 
> org.apache.hadoop.yarn.server.resourcemanager.recovery.MyriadFileSystemRMStateStore.updateFile(Lorg/apache/hadoop/fs/Path;[B)V
>   at 
> com.lmax.disruptor.FatalExceptionHandler.handleEventException(FatalExceptionHandler.java:45)
>   at 
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:147)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NoSuchMethodError: 
>

Re: [DISCUSS] Release prep for Myriad 0.1.0

2015-11-06 Thread Santosh Marella 
Sounds good, Adam.

Santosh

On Fri, Nov 6, 2015 at 10:32 AM, Adam Bordelon  wrote:

> Website sources don't need to be in the release tarball (would bloat it
> unnecessarily), and don't need to block the release of the 0.1.0 binaries,
> but we'll want the podling site up so we can announce our release on the
> website.
>
> On Fri, Nov 6, 2015 at 10:15 AM, Santosh Marella 
> wrote:
>
> > Starting this thread to discuss the last few items left before calling
> out
> > for a vote.
> >
> > * Blockers *
> >- MYRIAD-170 [1], reported last night.
> >- MYRIAD-144 [2] (Not sure if this is a blocker, please read below)
> >
> > *Current Status*
> > - MYRIAD-170: Swapnil is working on a fix.
> > - MYRIAD-144: Podling Website is not up yet.
> > - needs "svn authorization template" filled in by a Mentor
> > - ** Ben/Danese/Luciano/Ted: We seek your help **
> > - A running document of release guide is being prepared at [3]
> > - My GPG  public key is available at [4] and [5]
> > - A *test* RC candidate was prepared and uploaded to [6].
> >   (*This is not the final RC*. I prepared it to get familiar with the
> > process)
> >
> > * Discussion topics *
> > - MYRIAD-144
> >Not sure if this is a blocker. Should the "website" sources
> > necessarily part of the release tarball?
> >If not, then the SVN issues can be resolved independent of the
> > voting process on a RC.
> > - Does anyone have any other topics ?
> >
> > [1] https://issues.apache.org/jira/browse/MYRIAD-170
> > [2] https://issues.apache.org/jira/browse/MYRIAD-144
> > [3]
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61324959
> > [4] http://people.apache.org/keys/committer/smarella.asc
> > [5] https://dist.apache.org/repos/dist/dev/incubator/myriad/KEYS
> > [6]
> >
> >
> https://dist.apache.org/repos/dist/dev/incubator/myriad/myriad-0.1.0-incubating-rc0/
> >
> > Thanks,
> > Santosh
> >
>

[jira] [Updated] (MYRIAD-170) Myriad initialization fails with "parameter 5 of org.apache.myriad.scheduler.MyriadOperations.() is not @Nullable"

2015-11-06 Thread Santosh Marella (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MYRIAD-170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Santosh Marella updated MYRIAD-170:
---
Fix Version/s: Myriad 0.1.0

> Myriad initialization fails with "parameter 5 of 
> org.apache.myriad.scheduler.MyriadOperations.() is not @Nullable"
> 
>
> Key: MYRIAD-170
> URL: https://issues.apache.org/jira/browse/MYRIAD-170
> Project: Myriad
>  Issue Type: Bug
>  Components: Scheduler
>Affects Versions: Myriad 0.2.0
> Environment: CentOS7
>Reporter: Zhongyue Luo
>Assignee: Swapnil Daingade
> Fix For: Myriad 0.1.0
>
>
> After the framework gets registered, scheduler initialization complains that 
> " null returned by binding at 
> org.apache.myriad.MyriadModule.providesMyriadStateStore()"
> I have build Myriad according to the remote binary distribution document.
> Additional steps I did out of the document was setting up the http_proxy 
> environment value.
> I've search if this problem was brought up before but failed to find answers.
> Below is the log output of the scheduler.
> ==
> I1106 10:42:01.039393 11243 sched.cpp:641] Framework registered with 
> c70248ae-62a9-4a02-82b9-46a5e10fd15f-0016
> 15/11/06 10:42:01 INFO service.AbstractService: Service 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler 
> failed in state STARTED; cause: java.lang.RuntimeException: Failed to 
> initialize myriad
> java.lang.RuntimeException: Failed to initialize myriad
>   at 
> org.apache.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:52)
>   at 
> org.apache.myriad.scheduler.yarn.interceptor.CompositeInterceptor.init(CompositeInterceptor.java:92)
>   at 
> org.apache.myriad.scheduler.yarn.MyriadFairScheduler.serviceStart(MyriadFairScheduler.java:75)
>   at 
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
>   at 
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:503)
>   at 
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:898)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:938)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:935)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:935)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:979)
>   at 
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1104)
> Caused by: com.google.inject.ProvisionException: Guice provision errors:
> 1) null returned by binding at 
> org.apache.myriad.MyriadModule.providesMyriadStateStore()
>  but parameter 5 of org.apache.myriad.scheduler.MyriadOperations.() is 
> not @Nullable
>   at 
> org.apache.myriad.MyriadModule.providesMyriadStateStore(MyriadModule.java:154)
>   while locating org.apache.myriad.state.MyriadStateStore
> for parameter 5 at 
> org.apache.myriad.scheduler.MyriadOperations.(MyriadOperations.java:59)
>   while locating org.apache.myriad.scheduler.MyriadOperations
> 1 error
>   at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:987)
>   at 
> com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1013)
>   at org.apache.myriad.Main.startNMInstances(Main.java:202)
>   at org.apache.myriad.Main.run(Main.java:113)
>   at org.apache.myriad.Main.initialize(Main.java:88)
>   at 
> org.apache.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:49)
>   ... 16 more
> 15/11/06 10:42:01 INFO service.AbstractService: Service RMActiveServices 
> failed in state STARTED; cause: java.lang.RuntimeException: Failed to 
> initialize myriad
> java.lang.RuntimeException: Failed to initialize myriad
>   at 
> org.apache.myriad.scheduler.yarn.interceptor.MyriadInitializationInterceptor.init(MyriadInitializationInterceptor.java:52)
>   at 
>

Re: [DISCUSS] Release prep for Myriad 0.1.0

2015-11-06 Thread Adam Bordelon 
Website sources don't need to be in the release tarball (would bloat it
unnecessarily), and don't need to block the release of the 0.1.0 binaries,
but we'll want the podling site up so we can announce our release on the
website.

On Fri, Nov 6, 2015 at 10:15 AM, Santosh Marella 
wrote:

> Starting this thread to discuss the last few items left before calling out
> for a vote.
>
> * Blockers *
>- MYRIAD-170 [1], reported last night.
>- MYRIAD-144 [2] (Not sure if this is a blocker, please read below)
>
> *Current Status*
> - MYRIAD-170: Swapnil is working on a fix.
> - MYRIAD-144: Podling Website is not up yet.
> - needs "svn authorization template" filled in by a Mentor
> - ** Ben/Danese/Luciano/Ted: We seek your help **
> - A running document of release guide is being prepared at [3]
> - My GPG  public key is available at [4] and [5]
> - A *test* RC candidate was prepared and uploaded to [6].
>   (*This is not the final RC*. I prepared it to get familiar with the
> process)
>
> * Discussion topics *
> - MYRIAD-144
>Not sure if this is a blocker. Should the "website" sources
> necessarily part of the release tarball?
>If not, then the SVN issues can be resolved independent of the
> voting process on a RC.
> - Does anyone have any other topics ?
>
> [1] https://issues.apache.org/jira/browse/MYRIAD-170
> [2] https://issues.apache.org/jira/browse/MYRIAD-144
> [3]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61324959
> [4] http://people.apache.org/keys/committer/smarella.asc
> [5] https://dist.apache.org/repos/dist/dev/incubator/myriad/KEYS
> [6]
>
> https://dist.apache.org/repos/dist/dev/incubator/myriad/myriad-0.1.0-incubating-rc0/
>
> Thanks,
> Santosh
>

Re: [DISCUSS] Release prep for Myriad 0.1.0

2015-11-06 Thread Brandon Gulla 
I just found some badness that may be a blocker. A change happened in 2.7.1
that broke our API. See https://issues.apache.org/jira/browse/MYRIAD-171

I am going to try to submit the patch tonight.

On Fri, Nov 6, 2015 at 1:57 PM, Santosh Marella 
wrote:

> Sounds good, Adam.
>
> Santosh
>
> On Fri, Nov 6, 2015 at 10:32 AM, Adam Bordelon  wrote:
>
> > Website sources don't need to be in the release tarball (would bloat it
> > unnecessarily), and don't need to block the release of the 0.1.0
> binaries,
> > but we'll want the podling site up so we can announce our release on the
> > website.
> >
> > On Fri, Nov 6, 2015 at 10:15 AM, Santosh Marella 
> > wrote:
> >
> > > Starting this thread to discuss the last few items left before calling
> > out
> > > for a vote.
> > >
> > > * Blockers *
> > >- MYRIAD-170 [1], reported last night.
> > >- MYRIAD-144 [2] (Not sure if this is a blocker, please read below)
> > >
> > > *Current Status*
> > > - MYRIAD-170: Swapnil is working on a fix.
> > > - MYRIAD-144: Podling Website is not up yet.
> > > - needs "svn authorization template" filled in by a Mentor
> > > - ** Ben/Danese/Luciano/Ted: We seek your help **
> > > - A running document of release guide is being prepared at [3]
> > > - My GPG  public key is available at [4] and [5]
> > > - A *test* RC candidate was prepared and uploaded to [6].
> > >   (*This is not the final RC*. I prepared it to get familiar with
> the
> > > process)
> > >
> > > * Discussion topics *
> > > - MYRIAD-144
> > >Not sure if this is a blocker. Should the "website" sources
> > > necessarily part of the release tarball?
> > >If not, then the SVN issues can be resolved independent of the
> > > voting process on a RC.
> > > - Does anyone have any other topics ?
> > >
> > > [1] https://issues.apache.org/jira/browse/MYRIAD-170
> > > [2] https://issues.apache.org/jira/browse/MYRIAD-144
> > > [3]
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61324959
> > > [4] http://people.apache.org/keys/committer/smarella.asc
> > > [5] https://dist.apache.org/repos/dist/dev/incubator/myriad/KEYS
> > > [6]
> > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/incubator/myriad/myriad-0.1.0-incubating-rc0/
> > >
> > > Thanks,
> > > Santosh
> > >
> >
>



-- 
Brandon

[jira] [Commented] (MYRIAD-18) staging - pending loop

2015-11-06 Thread Adam B (JIRA) 

[ 
https://issues.apache.org/jira/browse/MYRIAD-18?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14993548#comment-14993548
 ] 

Adam B commented on MYRIAD-18:
--

TASK_LOST can occur for many reasons, including a network partition or 
lost/crashed agent. Generally this message implies that restarting the task may 
be successful, as opposed to a TASK_FAILED/TASK_ERROR where a retry is 
likely/guaranteed to fail again.
Other TASK_LOST scenarios:
- The scheduler driver is disconnected from the Mesos master at the time of an 
acceptOffers (e.g. launchTasks) call from the scheduler.
- Accept/Launch call uses invalid/rescinded offers. (Maybe this should be a 
TASK_ERROR?)
- Master asked to launch a task on an agent that has since been removed or 
disconnected.
- Tried to reconcile a task unknown to Mesos.
- When a master discovers that a slave process has exited, it reports TASK_LOST 
for any tasks from non-checkpointing frameworks.
- If an agent is shutdown/removed completely, then all tasks will report a 
TASK_LOST.
- Upon agent reregistration, any tasks known by the master but unknown by the 
agent will report TASK_LOST.
- Agent could not launch the task because it failed to unschedule directories 
for garbage collection.
- If the task/executor uses persistent volumes unknown to the agent.
- If the agent is asked to run a task using an existing executor that is 
terminating/terminated.
- Agent asked to killTask for an unrecognized executor.
- Executor reregistration timeout expired.
- Failed to update resources for executor container (e.g. grow to launch new 
task).
- Container/executor preempted by QoS controller.

> staging - pending loop
> --
>
> Key: MYRIAD-18
> URL: https://issues.apache.org/jira/browse/MYRIAD-18
> Project: Myriad
>  Issue Type: Bug
>Reporter: Maysam Yabandeh
>
> if staging task is lost for any reason it gets stuck in a staging-pending 
> loop.
> case TASK_LOST:
> schedulerState.makeTaskPending(taskId);
> break;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)