[jira] [Assigned] (MYRIAD-238) Convert groovy-based unit tests to JUnit tests
[ https://issues.apache.org/jira/browse/MYRIAD-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost reassigned MYRIAD-238: Assignee: (was: John Yost) > Convert groovy-based unit tests to JUnit tests > -- > > Key: MYRIAD-238 > URL: https://issues.apache.org/jira/browse/MYRIAD-238 > Project: Myriad > Issue Type: Test >Reporter: John Yost >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MYRIAD-229) Put upper limit on FGS
[ https://issues.apache.org/jira/browse/MYRIAD-229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost reassigned MYRIAD-229: Assignee: (was: John Yost) > Put upper limit on FGS > -- > > Key: MYRIAD-229 > URL: https://issues.apache.org/jira/browse/MYRIAD-229 > Project: Myriad > Issue Type: Bug > Components: Scheduler >Affects Versions: Myriad 0.2.0 >Reporter: DarinJ >Priority: Minor > > Currently FGS can utilize as many resources as on the machine this is > sometimes problematic when considering machines with disproportionate core to > mem to disk ratios. One way to fix the issue is to cap the amount of CPU/MEM > that Myriad will utilize with FGS. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MYRIAD-237) SchedulerState getKillableTasks method mislabelled
[ https://issues.apache.org/jira/browse/MYRIAD-237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost reassigned MYRIAD-237: Assignee: (was: John Yost) > SchedulerState getKillableTasks method mislabelled > -- > > Key: MYRIAD-237 > URL: https://issues.apache.org/jira/browse/MYRIAD-237 > Project: Myriad > Issue Type: Bug >Reporter: John Yost >Priority: Minor > Labels: easyfix, newbie > > In the process of writing SchedulerUtils JUnit tests, the tests for > setTaskKillable was failing and I determined that the > SchedulerState.getKillableTasks returns TaskID as opposed to NodeTask > objects. The method name is misleading and should be changed to be consistent > with the other SchedulerState getter methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MYRIAD-225) Loading state in MyriadStateStore leads to inconsistent unit test results
[ https://issues.apache.org/jira/browse/MYRIAD-225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost reassigned MYRIAD-225: Assignee: (was: John Yost) > Loading state in MyriadStateStore leads to inconsistent unit test results > - > > Key: MYRIAD-225 > URL: https://issues.apache.org/jira/browse/MYRIAD-225 > Project: Myriad > Issue Type: Bug > Components: Scheduler >Reporter: John Yost >Priority: Minor > > After fixing bug in loading state in the MyriadStateStore I noticed the > MyriadOperationsTest unit test results were inconsistent. Need to investigate > and, if applicable, add @Before and @After logic to reset state correctly to > fix inconsistencies. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MYRIAD-239) Setting frameworkRole to * causes exception
[ https://issues.apache.org/jira/browse/MYRIAD-239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost reassigned MYRIAD-239: Assignee: (was: John Yost) > Setting frameworkRole to * causes exception > --- > > Key: MYRIAD-239 > URL: https://issues.apache.org/jira/browse/MYRIAD-239 > Project: Myriad > Issue Type: Bug >Reporter: John Yost >Priority: Major > > Setting the frameworkRole in myriad-config-default.yml causes the following > exception to be thrown: > org.apache.myriad.scheduler.event.handlers.ResourceOffersEventHandler: > Exception thrown while trying to create a task for nm > java.lang.IllegalArgumentException: n must be positive > at java.util.Random.nextInt(Random.java:300) > at > org.apache.myriad.scheduler.resource.RangeResource.getRandomValues(RangeResource.java:128) > at > org.apache.myriad.scheduler.resource.RangeResource.consumeResource(RangeResource.java:99) > at > org.apache.myriad.scheduler.resource.ResourceOfferContainer.consumePorts(ResourceOfferContainer.java:171) > at > org.apache.myriad.scheduler.NMTaskFactory.createTask(NMTaskFactory.java:45) > at > org.apache.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:119) > at > org.apache.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:49) > at > com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MYRIAD-251) ZERO size NodeManager fail to obtain resource from Mesos Offer
[
https://issues.apache.org/jira/browse/MYRIAD-251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749693#comment-15749693
]
John Yost commented on MYRIAD-251:
--
[~darinj] Gotcha, OK, I can do that.
> ZERO size NodeManager fail to obtain resource from Mesos Offer
> --
>
> Key: MYRIAD-251
> URL: https://issues.apache.org/jira/browse/MYRIAD-251
> Project: Myriad
> Issue Type: Bug
> Components: Scheduler
>Reporter: Tao Jie
>
> I tried Fine-grained Scaling and flexed up zero size NodeManager, then I run
> a MR job which request for resource.
> However zero size NM did not obtain resource from mesos offer. RM logs like:
> {code}
> 2016-12-14 16:58:23,929 INFO
> org.apache.myriad.scheduler.fgs.NMHeartBeatHandler: Did not update
> bdi13.cmss.com with 10 cores and 5888 memory, over max cpu cores and/or max
> memory
> 2016-12-14 16:58:23,931 WARN
> org.apache.myriad.scheduler.fgs.YarnNodeCapacityManager: Asked to set Node
> bdi13.cmss.com:31905 to a value less than zero! Had ,
> setting to .
> 2016-12-14 16:58:23,931 WARN
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler:
> Update resource on node: bdi13.cmss.com with the same resource: vCores:0>
> {code}
> It seems that mesos offer with memory larger than 2252.8mb would be denied,
> and 2252.8mb is fixed value in code :
> {code}
> private Double generateNodeManagerMemory() {
> return (NodeManagerConfiguration.DEFAULT_JVM_MAX_MEMORY_MB) * (1 +
> NodeManagerConfiguration.JVM_OVERHEAD);
> }
> {code}
> where DEFAULT_JVM_MAX_MEMORY_MB=2048 and
> NodeManagerConfiguration.JVM_OVERHEAD=0.1
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MYRIAD-225) Loading state in MyriadStateStore leads to inconsistent unit test results
[ https://issues.apache.org/jira/browse/MYRIAD-225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15434925#comment-15434925 ] John Yost commented on MYRIAD-225: -- Updated the MYRIAD-229 branch, tested locally, checking in to see if Travis CI build succeeds > Loading state in MyriadStateStore leads to inconsistent unit test results > - > > Key: MYRIAD-225 > URL: https://issues.apache.org/jira/browse/MYRIAD-225 > Project: Myriad > Issue Type: Bug > Components: Scheduler >Reporter: John Yost >Assignee: John Yost >Priority: Minor > > After fixing bug in loading state in the MyriadStateStore I noticed the > MyriadOperationsTest unit test results were inconsistent. Need to investigate > and, if applicable, add @Before and @After logic to reset state correctly to > fix inconsistencies. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (MYRIAD-225) Loading state in MyriadStateStore leads to inconsistent unit test results
[ https://issues.apache.org/jira/browse/MYRIAD-225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost reopened MYRIAD-225: -- Was able to recreate this issue following failed Travis CI build. > Loading state in MyriadStateStore leads to inconsistent unit test results > - > > Key: MYRIAD-225 > URL: https://issues.apache.org/jira/browse/MYRIAD-225 > Project: Myriad > Issue Type: Bug > Components: Scheduler >Reporter: John Yost >Assignee: John Yost >Priority: Minor > > After fixing bug in loading state in the MyriadStateStore I noticed the > MyriadOperationsTest unit test results were inconsistent. Need to investigate > and, if applicable, add @Before and @After logic to reset state correctly to > fix inconsistencies. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MYRIAD-240) Created lower limit on FGS
[ https://issues.apache.org/jira/browse/MYRIAD-240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost updated MYRIAD-240: - Issue Type: New Feature (was: Improvement) > Created lower limit on FGS > -- > > Key: MYRIAD-240 > URL: https://issues.apache.org/jira/browse/MYRIAD-240 > Project: Myriad > Issue Type: New Feature >Reporter: John Yost > > In analogy to MYRIAD-229, create a lower limit in the FGS -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MYRIAD-240) Created lower limit on FGS
[ https://issues.apache.org/jira/browse/MYRIAD-240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost updated MYRIAD-240: - Issue Type: Improvement (was: Bug) > Created lower limit on FGS > -- > > Key: MYRIAD-240 > URL: https://issues.apache.org/jira/browse/MYRIAD-240 > Project: Myriad > Issue Type: Improvement >Reporter: John Yost > > In analogy to MYRIAD-229, create a lower limit in the FGS -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-241) Create Docker container based upon Cloudera Hadoop distribution
John Yost created MYRIAD-241: Summary: Create Docker container based upon Cloudera Hadoop distribution Key: MYRIAD-241 URL: https://issues.apache.org/jira/browse/MYRIAD-241 Project: Myriad Issue Type: Bug Reporter: John Yost -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MYRIAD-241) Create Docker container based upon Cloudera Hadoop distribution
[ https://issues.apache.org/jira/browse/MYRIAD-241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost updated MYRIAD-241: - Issue Type: Improvement (was: Bug) > Create Docker container based upon Cloudera Hadoop distribution > --- > > Key: MYRIAD-241 > URL: https://issues.apache.org/jira/browse/MYRIAD-241 > Project: Myriad > Issue Type: Improvement >Reporter: John Yost > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-240) Created lower limit on FGS
John Yost created MYRIAD-240: Summary: Created lower limit on FGS Key: MYRIAD-240 URL: https://issues.apache.org/jira/browse/MYRIAD-240 Project: Myriad Issue Type: Bug Reporter: John Yost In analogy to MYRIAD-229, create a lower limit in the FGS -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MYRIAD-237) SchedulerState getKillableTasks method mislabelled
[ https://issues.apache.org/jira/browse/MYRIAD-237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431285#comment-15431285 ] John Yost commented on MYRIAD-237: -- In MYRIAD-229 PR > SchedulerState getKillableTasks method mislabelled > -- > > Key: MYRIAD-237 > URL: https://issues.apache.org/jira/browse/MYRIAD-237 > Project: Myriad > Issue Type: Bug >Reporter: John Yost >Assignee: John Yost >Priority: Minor > Labels: easyfix, newbie > > In the process of writing SchedulerUtils JUnit tests, the tests for > setTaskKillable was failing and I determined that the > SchedulerState.getKillableTasks returns TaskID as opposed to NodeTask > objects. The method name is misleading and should be changed to be consistent > with the other SchedulerState getter methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (MYRIAD-234) Increased JUnit Test Coverage
[ https://issues.apache.org/jira/browse/MYRIAD-234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost closed MYRIAD-234. Merged into master by Darin > Increased JUnit Test Coverage > - > > Key: MYRIAD-234 > URL: https://issues.apache.org/jira/browse/MYRIAD-234 > Project: Myriad > Issue Type: Improvement >Reporter: John Yost >Assignee: John Yost >Priority: Minor > > Added more JUnit test coverage for the scheduler, webapp, and api packages, > fixed some other JUnit tests that were throwing checked exceptions that I > observed in debugger mode. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MYRIAD-238) Convert groovy-based unit tests to JUnit tests
[ https://issues.apache.org/jira/browse/MYRIAD-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431286#comment-15431286 ] John Yost commented on MYRIAD-238: -- In MYRIAD-229 PR > Convert groovy-based unit tests to JUnit tests > -- > > Key: MYRIAD-238 > URL: https://issues.apache.org/jira/browse/MYRIAD-238 > Project: Myriad > Issue Type: Test >Reporter: John Yost >Assignee: John Yost >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (MYRIAD-225) Loading state in MyriadStateStore leads to inconsistent unit test results
[ https://issues.apache.org/jira/browse/MYRIAD-225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost closed MYRIAD-225. > Loading state in MyriadStateStore leads to inconsistent unit test results > - > > Key: MYRIAD-225 > URL: https://issues.apache.org/jira/browse/MYRIAD-225 > Project: Myriad > Issue Type: Bug > Components: Scheduler >Reporter: John Yost >Assignee: John Yost >Priority: Minor > > After fixing bug in loading state in the MyriadStateStore I noticed the > MyriadOperationsTest unit test results were inconsistent. Need to investigate > and, if applicable, add @Before and @After logic to reset state correctly to > fix inconsistencies. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MYRIAD-225) Loading state in MyriadStateStore leads to inconsistent unit test results
[ https://issues.apache.org/jira/browse/MYRIAD-225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost resolved MYRIAD-225. -- Resolution: Fixed Assignee: John Yost JUnit tests now pass repeatably > Loading state in MyriadStateStore leads to inconsistent unit test results > - > > Key: MYRIAD-225 > URL: https://issues.apache.org/jira/browse/MYRIAD-225 > Project: Myriad > Issue Type: Bug > Components: Scheduler >Reporter: John Yost >Assignee: John Yost >Priority: Minor > > After fixing bug in loading state in the MyriadStateStore I noticed the > MyriadOperationsTest unit test results were inconsistent. Need to investigate > and, if applicable, add @Before and @After logic to reset state correctly to > fix inconsistencies. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-239) Setting frameworkRole to * causes exception
John Yost created MYRIAD-239: Summary: Setting frameworkRole to * causes exception Key: MYRIAD-239 URL: https://issues.apache.org/jira/browse/MYRIAD-239 Project: Myriad Issue Type: Bug Reporter: John Yost Setting the frameworkRole in myriad-config-default.yml causes the following exception to be thrown: org.apache.myriad.scheduler.event.handlers.ResourceOffersEventHandler: Exception thrown while trying to create a task for nm java.lang.IllegalArgumentException: n must be positive at java.util.Random.nextInt(Random.java:300) at org.apache.myriad.scheduler.resource.RangeResource.getRandomValues(RangeResource.java:128) at org.apache.myriad.scheduler.resource.RangeResource.consumeResource(RangeResource.java:99) at org.apache.myriad.scheduler.resource.ResourceOfferContainer.consumePorts(ResourceOfferContainer.java:171) at org.apache.myriad.scheduler.NMTaskFactory.createTask(NMTaskFactory.java:45) at org.apache.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:119) at org.apache.myriad.scheduler.event.handlers.ResourceOffersEventHandler.onEvent(ResourceOffersEventHandler.java:49) at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-238) Convert groovy-based unit tests to JUnit tests
John Yost created MYRIAD-238: Summary: Convert groovy-based unit tests to JUnit tests Key: MYRIAD-238 URL: https://issues.apache.org/jira/browse/MYRIAD-238 Project: Myriad Issue Type: Test Reporter: John Yost Assignee: John Yost Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-237) SchedulerState getKillableTasks method mislabelled
John Yost created MYRIAD-237: Summary: SchedulerState getKillableTasks method mislabelled Key: MYRIAD-237 URL: https://issues.apache.org/jira/browse/MYRIAD-237 Project: Myriad Issue Type: Bug Reporter: John Yost Assignee: John Yost Priority: Minor In the process of writing SchedulerUtils JUnit tests, the tests for setTaskKillable was failing and I determined that the SchedulerState.getKillableTasks returns TaskID as opposed to NodeTask objects. The method name is misleading and should be changed to be consistent with the other SchedulerState getter methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MYRIAD-236) YarnNodeCapacityManager--enable reset in setNodeCapacity or make setNodeCapacity protected/private
[ https://issues.apache.org/jira/browse/MYRIAD-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost updated MYRIAD-236: - Priority: Minor (was: Major) > YarnNodeCapacityManager--enable reset in setNodeCapacity or make > setNodeCapacity protected/private > -- > > Key: MYRIAD-236 > URL: https://issues.apache.org/jira/browse/MYRIAD-236 > Project: Myriad > Issue Type: Bug >Reporter: John Yost >Priority: Minor > > Currently the setNodeCapacity method does not enable setting of the node > capacity to a specific value--only incrementing or decrementing. Moreover, > the incrementNodeCapacity and decrementNodeCapacity methods both delegate to > setNodeCapacity. I recommend one of two options: (1) adding the capability > to set the node capacity to a specific value and enable this capability > within the GUI or (2) making setNodeCapacity protected or private. > If there are use cases for option (1) where it would be good to set to a > specific capacity--irrespective of the current capacity--I recommend that one. > This issue arose during my work on MYRIAD-229, so I can fix it within that > branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (MYRIAD-236) YarnNodeCapacityManager--enable reset in setNodeCapacity or make setNodeCapacity protected/private
[ https://issues.apache.org/jira/browse/MYRIAD-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost updated MYRIAD-236: - Comment: was deleted (was: Investigating this further, the incrementNodeCapacity and decrementNodeCapacity are not invoked anywhere in the codebase outside of my JUnit tests. The setNodeCapacity methos is only invoked in the NMHeartBeatHandler.handleStatusUpdate method which is only visible for testing. Consequently, I recommend closing this ticket.) > YarnNodeCapacityManager--enable reset in setNodeCapacity or make > setNodeCapacity protected/private > -- > > Key: MYRIAD-236 > URL: https://issues.apache.org/jira/browse/MYRIAD-236 > Project: Myriad > Issue Type: Bug >Reporter: John Yost > > Currently the setNodeCapacity method does not enable setting of the node > capacity to a specific value--only incrementing or decrementing. Moreover, > the incrementNodeCapacity and decrementNodeCapacity methods both delegate to > setNodeCapacity. I recommend one of two options: (1) adding the capability > to set the node capacity to a specific value and enable this capability > within the GUI or (2) making setNodeCapacity protected or private. > If there are use cases for option (1) where it would be good to set to a > specific capacity--irrespective of the current capacity--I recommend that one. > This issue arose during my work on MYRIAD-229, so I can fix it within that > branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MYRIAD-236) YarnNodeCapacityManager--enable reset in setNodeCapacity or make setNodeCapacity protected/private
[ https://issues.apache.org/jira/browse/MYRIAD-236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15407772#comment-15407772 ] John Yost commented on MYRIAD-236: -- Investigating this further, the incrementNodeCapacity and decrementNodeCapacity are not invoked anywhere in the codebase outside of my JUnit tests. The setNodeCapacity methos is only invoked in the NMHeartBeatHandler.handleStatusUpdate method which is only visible for testing. Consequently, I recommend closing this ticket. > YarnNodeCapacityManager--enable reset in setNodeCapacity or make > setNodeCapacity protected/private > -- > > Key: MYRIAD-236 > URL: https://issues.apache.org/jira/browse/MYRIAD-236 > Project: Myriad > Issue Type: Bug >Reporter: John Yost > > Currently the setNodeCapacity method does not enable setting of the node > capacity to a specific value--only incrementing or decrementing. Moreover, > the incrementNodeCapacity and decrementNodeCapacity methods both delegate to > setNodeCapacity. I recommend one of two options: (1) adding the capability > to set the node capacity to a specific value and enable this capability > within the GUI or (2) making setNodeCapacity protected or private. > If there are use cases for option (1) where it would be good to set to a > specific capacity--irrespective of the current capacity--I recommend that one. > This issue arose during my work on MYRIAD-229, so I can fix it within that > branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-236) YarnNodeCapacityManager--enable reset in setNodeCapacity or make setNodeCapacity protected/private
John Yost created MYRIAD-236: Summary: YarnNodeCapacityManager--enable reset in setNodeCapacity or make setNodeCapacity protected/private Key: MYRIAD-236 URL: https://issues.apache.org/jira/browse/MYRIAD-236 Project: Myriad Issue Type: Bug Reporter: John Yost Currently the setNodeCapacity method does not enable setting of the node capacity to a specific value--only incrementing or decrementing. Moreover, the incrementNodeCapacity and decrementNodeCapacity methods both delegate to setNodeCapacity. I recommend one of two options: (1) adding the capability to set the node capacity to a specific value and enable this capability within the GUI or (2) making setNodeCapacity protected or private. If there are use cases for option (1) where it would be good to set to a specific capacity--irrespective of the current capacity--I recommend that one. This issue arose during my work on MYRIAD-229, so I can fix it within that branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MYRIAD-234) Increased JUnit Test Coverage
[ https://issues.apache.org/jira/browse/MYRIAD-234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost resolved MYRIAD-234. -- Resolution: Fixed Code merged in from PR 86, work complete > Increased JUnit Test Coverage > - > > Key: MYRIAD-234 > URL: https://issues.apache.org/jira/browse/MYRIAD-234 > Project: Myriad > Issue Type: Improvement >Reporter: John Yost >Assignee: John Yost >Priority: Minor > > Added more JUnit test coverage for the scheduler, webapp, and api packages, > fixed some other JUnit tests that were throwing checked exceptions that I > observed in debugger mode. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MYRIAD-229) Put upper limit on FGS
[ https://issues.apache.org/jira/browse/MYRIAD-229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost reassigned MYRIAD-229: Assignee: John Yost > Put upper limit on FGS > -- > > Key: MYRIAD-229 > URL: https://issues.apache.org/jira/browse/MYRIAD-229 > Project: Myriad > Issue Type: Bug > Components: Scheduler >Affects Versions: Myriad 0.2.0 >Reporter: DarinJ >Assignee: John Yost >Priority: Minor > > Currently FGS can utilize as many resources as on the machine this is > sometimes problematic when considering machines with disproportionate core to > mem to disk ratios. One way to fix the issue is to cap the amount of CPU/MEM > that Myriad will utilize with FGS. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MYRIAD-220) Improve reliability of kill task messaging
[ https://issues.apache.org/jira/browse/MYRIAD-220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost resolved MYRIAD-220. -- Resolution: Fixed Fix Version/s: Myriad 0.3.0 Development complete, tested on AWS cluster, corresponding PR #84 merged into master branch > Improve reliability of kill task messaging > -- > > Key: MYRIAD-220 > URL: https://issues.apache.org/jira/browse/MYRIAD-220 > Project: Myriad > Issue Type: Improvement > Components: Scheduler >Reporter: John Yost >Assignee: John Yost > Fix For: Myriad 0.3.0 > > > Currently within the YarnNodeCapacityManager there is a two-step process of > killing a YARN task via the following method invocations: > state.makeTaskKillable(taskId); > myriadDriver.kill(taskId); > Need to add logic to ensure all killable tasks are killed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MYRIAD-200) Increase JUnit Test Coverage
[ https://issues.apache.org/jira/browse/MYRIAD-200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost resolved MYRIAD-200. -- Resolution: Fixed Corresponding PR #78 integrated into master, issue resolved. > Increase JUnit Test Coverage > > > Key: MYRIAD-200 > URL: https://issues.apache.org/jira/browse/MYRIAD-200 > Project: Myriad > Issue Type: Bug > Components: Executor, Scheduler >Reporter: DarinJ >Assignee: John Yost > > Currently Unit Test coverage is weak in places also, a good integration test > framework would be helpful. (potentially [minimesos|http://minimesos.org]?) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (MYRIAD-202) minimesos-based integration tests
[ https://issues.apache.org/jira/browse/MYRIAD-202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost closed MYRIAD-202. Resolution: Won't Fix JUnit test coverage obviates the need for this story > minimesos-based integration tests > - > > Key: MYRIAD-202 > URL: https://issues.apache.org/jira/browse/MYRIAD-202 > Project: Myriad > Issue Type: Improvement >Reporter: John Yost >Assignee: John Yost > > Implement integration tests with mini-mesos to bridge the gap between a unit > test harness and deployment to a mesos cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-234) Increased JUnit Test Coverage
John Yost created MYRIAD-234: Summary: Increased JUnit Test Coverage Key: MYRIAD-234 URL: https://issues.apache.org/jira/browse/MYRIAD-234 Project: Myriad Issue Type: Improvement Reporter: John Yost Priority: Minor Added more JUnit test coverage for the scheduler, webapp, and api packages, fixed some other JUnit tests that were throwing checked exceptions that I observed in debugger mode. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-233) Myriad UI should display the entire Myriad configuration
John Yost created MYRIAD-233: Summary: Myriad UI should display the entire Myriad configuration Key: MYRIAD-233 URL: https://issues.apache.org/jira/browse/MYRIAD-233 Project: Myriad Issue Type: Improvement Reporter: John Yost Priority: Minor Currently the Myriad UI only displays the service profile configuration. I recommend display of all Myriad config params in the UI -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-232) Display Flex Down/Up errors to UI
John Yost created MYRIAD-232: Summary: Display Flex Down/Up errors to UI Key: MYRIAD-232 URL: https://issues.apache.org/jira/browse/MYRIAD-232 Project: Myriad Issue Type: Improvement Components: Scheduler Reporter: John Yost Priority: Minor Currently flex up/flex down errors such as requesting flex down instances number > number of corresponding tasks are recorded in logs but not displayed on UI. I recommend updating UI and REST API accordingly to enable display of these error messages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-227) Jersey-based exception prevents refresh of Myriad UI
John Yost created MYRIAD-227: Summary: Jersey-based exception prevents refresh of Myriad UI Key: MYRIAD-227 URL: https://issues.apache.org/jira/browse/MYRIAD-227 Project: Myriad Issue Type: Bug Components: Scheduler Reporter: John Yost Priority: Minor Periodically a Jersey-related exception is thrown, which prevents the Myriad web UI from refreshing, requiring a manual webpage refresh. Stacktrace: SEVERE: null java.lang.IllegalAccessException: Class com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8 can not access a member of class javax.ws.rs.core.Response with modifiers "protected" at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:102) at java.lang.Class.newInstance(Class.java:436) at com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8.resolve(WadlGeneratorJAXBGrammarGenerator.java:467) at com.sun.jersey.server.wadl.WadlGenerator$ExternalGrammarDefinition.resolve(WadlGenerator.java:181) at com.sun.jersey.server.wadl.ApplicationDescription.resolve(ApplicationDescription.java:81) at com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator.attachTypes(WadlGeneratorJAXBGrammarGenerator.java:518) at com.sun.jersey.server.wadl.WadlBuilder.generate(WadlBuilder.java:124) at com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl.getApplication(WadlApplicationContextImpl.java:104) at com.sun.jersey.server.impl.wadl.WadlResource.getWadl(WadlResource.java:89) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263) at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178) at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62) at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118) at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.Htt
[jira] [Created] (MYRIAD-226) Clean up missing generics parameters and unused object references
John Yost created MYRIAD-226: Summary: Clean up missing generics parameters and unused object references Key: MYRIAD-226 URL: https://issues.apache.org/jira/browse/MYRIAD-226 Project: Myriad Issue Type: Improvement Components: Executor, Scheduler Reporter: John Yost Priority: Minor There are a few places (i.e., references to AbstractYarnScheduler) where the generics parameters are not specified. There are also unused object references (i.e., SchedulerState in ReRegisteredEventHandler). Would be a good newbie task to clean these up -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-225) Loading state in MyriadStateStore leads to inconsistent unit test results
John Yost created MYRIAD-225: Summary: Loading state in MyriadStateStore leads to inconsistent unit test results Key: MYRIAD-225 URL: https://issues.apache.org/jira/browse/MYRIAD-225 Project: Myriad Issue Type: Bug Components: Scheduler Reporter: John Yost Priority: Minor After fixing bug in loading state in the MyriadStateStore I noticed the MyriadOperationsTest unit test results were inconsistent. Need to investigate and, if applicable, add @Before and @After logic to reset state correctly to fix inconsistencies. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MYRIAD-220) Improve reliability of kill task messaging
[ https://issues.apache.org/jira/browse/MYRIAD-220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15334515#comment-15334515 ] John Yost commented on MYRIAD-220: -- Myriad interacts with the Mesos scheduler via the Mesos ScheduleDriver class and receives acknowledgement of requests via a callback mechanism defined in the Mesos Scheduler interface. Myriad Tasks are killed via a process that is ideally one-step but can be two-step if there are any system or network issues at the time the task kill request is submitted to Mesos: 1. The YarnNodeCapacityManager invokes MyriadDriver.kill, which delegates to the Mesos SchedulerDriver.killTask method. At this stage the kill request is sent by the ScheduleDriver to Mesos. A Protos.Status object is returned, but that Status object only indicates whether the SchedulerDriver is in a running state. Until a callback is invoked by Mesos upon the Myriad Scheduler implementation, there is no guarantee that the task kill request succeeded. 2. The TaskTerminator is a daemon that periodically checks the SchedulerState killable tasks queue and invokes MyriadDriverManager.kill, which is a wrapper for the MyriadDriver (and, again, the Mesos SchedulerDriver). In the master branch the TaskTerminator also invokes SchedulerState.removeTask to remove the killable task from the SchedulerState. This is potentially an issue because, again, the task kill request is not guaranteed to have worked unless the corresponding Mesos callback method is invoked. The only way to ensure that all Killable Myriad tasks are eventually killed is to invoke SchedulerState.removeTask from within Myriad's Mesos lifecycle callback implementation, MyriadScheduler, either directly or within a local Java object. Specifically, MyriadScheduler implements the org.apache.mesos.Scheduler interface, which is the Mesos callback interface for all Mesos Task lifecycle methods (i.e., register, statusUpdate, disconnected, etc...). When the statusUpdate callback is invoked, one of the status updates is that the Myriad task was killed. When a StatusUpdate is received from Mesos, a Myriad StatusUpdateEvent is created and fired via Disruptor. The message listener is the StatusUpdateEventHandler class. Within it's onEvent method, there is logic to decline new resource offers as well as remove the corresponding task from the Myriad SchedulerState. Consequently, the only code change required is to remove the SchedulerState.removeTask method invocation from TaskTerminator. I am also adding some JUnit tests and Javadoc comments. --John > Improve reliability of kill task messaging > -- > > Key: MYRIAD-220 > URL: https://issues.apache.org/jira/browse/MYRIAD-220 > Project: Myriad > Issue Type: Improvement > Components: Scheduler >Reporter: John Yost >Assignee: John Yost > > Currently within the YarnNodeCapacityManager there is a two-step process of > killing a YARN task via the following method invocations: > state.makeTaskKillable(taskId); > myriadDriver.kill(taskId); > Need to add logic to ensure all killable tasks are killed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-224) Add AOP layer for SchedulerState (and maybe other classes)
John Yost created MYRIAD-224: Summary: Add AOP layer for SchedulerState (and maybe other classes) Key: MYRIAD-224 URL: https://issues.apache.org/jira/browse/MYRIAD-224 Project: Myriad Issue Type: Improvement Components: Scheduler Reporter: John Yost Priority: Minor The SchedulerState has a protected updateStore() method that is invoked on the last line of SchedulerState public methods. There is boolean logic to short-circuit the method if HA is disabled. Configuring this to instead use AOP would prevent invocation where HA is disabled and also deliver cleaner code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-223) ExecutorInfo in both Node and NodeTask classes--remove from Node
John Yost created MYRIAD-223: Summary: ExecutorInfo in both Node and NodeTask classes--remove from Node Key: MYRIAD-223 URL: https://issues.apache.org/jira/browse/MYRIAD-223 Project: Myriad Issue Type: Improvement Reporter: John Yost Priority: Minor As Swapnil mentioned in a TODO comment within the YarnNodeCapacityManager class, ExecutorInfo is an attribute of both the Node and NodeTask classes. It is recommended that the ExecutorInfo be removed from the Node class -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-222) Triage killable tasks to prevent retries on inaccessible nodes
John Yost created MYRIAD-222: Summary: Triage killable tasks to prevent retries on inaccessible nodes Key: MYRIAD-222 URL: https://issues.apache.org/jira/browse/MYRIAD-222 Project: Myriad Issue Type: Improvement Reporter: John Yost Priority: Minor When a Myriad task is marked killable an attempt is made to kill the task via the MyriadDriver, which delegates the call to the underlying MesosSchedulerDriver. In addition, the TaskTerminator daemon periodically attempts to kill any killable tasks that remain in the SchedulerState. Darin suggested adding configurable logic to partition task ids pertaining to tasks for which multiple unsuccessful kill attempts have been made; this feature will prevent repeated attempts to kill tasks on inaccessible nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-220) Improve reliability of kill task messaging
John Yost created MYRIAD-220: Summary: Improve reliability of kill task messaging Key: MYRIAD-220 URL: https://issues.apache.org/jira/browse/MYRIAD-220 Project: Myriad Issue Type: Improvement Components: Scheduler Reporter: John Yost Currently within the YarnNodeCapacityManager there is a two-step process of killing a YARN task via the following method invocations: state.makeTaskKillable(taskId); myriadDriver.kill(taskId); Need to add logic to ensure all killable tasks are killed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-217) Make fine-grained scaling for all NodeManager configurations
John Yost created MYRIAD-217: Summary: Make fine-grained scaling for all NodeManager configurations Key: MYRIAD-217 URL: https://issues.apache.org/jira/browse/MYRIAD-217 Project: Myriad Issue Type: Improvement Components: Scheduler Reporter: John Yost Currently only Node Managers configured with zero CPU cores and memory are eligible for fine-grained scaling. I recommend removing this restriction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MYRIAD-200) Increase JUnit Test Coverage
[ https://issues.apache.org/jira/browse/MYRIAD-200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost updated MYRIAD-200: - Summary: Increase JUnit Test Coverage (was: Increased JUnit Test Coverage) > Increase JUnit Test Coverage > > > Key: MYRIAD-200 > URL: https://issues.apache.org/jira/browse/MYRIAD-200 > Project: Myriad > Issue Type: Bug > Components: Executor, Scheduler >Reporter: DarinJ >Assignee: John Yost > > Currently Unit Test coverage is weak in places also, a good integration test > framework would be helpful. (potentially [minimesos|http://minimesos.org]?) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-216) Make OfferLifecycleManager thread safe
John Yost created MYRIAD-216: Summary: Make OfferLifecycleManager thread safe Key: MYRIAD-216 URL: https://issues.apache.org/jira/browse/MYRIAD-216 Project: Myriad Issue Type: Bug Components: Scheduler Reporter: John Yost Fix For: Myriad 0.3.0 There is a TODO note indicating that parts of OfferLifecycleManager are not thread-safe. We should fix this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MYRIAD-200) Increased JUnit Test Coverage
[ https://issues.apache.org/jira/browse/MYRIAD-200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost updated MYRIAD-200: - Summary: Increased JUnit Test Coverage (was: Increased Unit Test and Integration Testing) > Increased JUnit Test Coverage > - > > Key: MYRIAD-200 > URL: https://issues.apache.org/jira/browse/MYRIAD-200 > Project: Myriad > Issue Type: Bug > Components: Executor, Scheduler >Reporter: DarinJ >Assignee: John Yost > > Currently Unit Test coverage is weak in places also, a good integration test > framework would be helpful. (potentially [minimesos|http://minimesos.org]?) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MYRIAD-202) minimesos-based integration tests
[ https://issues.apache.org/jira/browse/MYRIAD-202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost reassigned MYRIAD-202: Assignee: John Yost > minimesos-based integration tests > - > > Key: MYRIAD-202 > URL: https://issues.apache.org/jira/browse/MYRIAD-202 > Project: Myriad > Issue Type: Improvement >Reporter: John Yost >Assignee: John Yost > > Implement integration tests with mini-mesos to bridge the gap between a unit > test harness and deployment to a mesos cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MYRIAD-198) Remove optionals when sane defaults are available
[
https://issues.apache.org/jira/browse/MYRIAD-198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Yost reassigned MYRIAD-198:
Assignee: John Yost
> Remove optionals when sane defaults are available
> -
>
> Key: MYRIAD-198
> URL: https://issues.apache.org/jira/browse/MYRIAD-198
> Project: Myriad
> Issue Type: Bug
> Components: Executor, Scheduler
>Affects Versions: Myriad 0.2.0
>Reporter: DarinJ
>Assignee: John Yost
>Priority: Minor
> Labels: easyfix, newbie
>
> Currently we overuse Optionals in the config and then use an or method in
> various factories later. In many cases having the configuration return a
> default when the parameter was specified would create cleaner code. For
> instance:
> {quote}
> Optional getCgroups() {
> Optional.fromNullable(cgroups);
> }
> {quote}
> vs
> {quote}
> Boolean getCgroups() {
> return cgroups != null ? cgroups : false;
> }
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MYRIAD-200) Increased Unit Test and Integration Testing
[ https://issues.apache.org/jira/browse/MYRIAD-200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost reassigned MYRIAD-200: Assignee: John Yost > Increased Unit Test and Integration Testing > --- > > Key: MYRIAD-200 > URL: https://issues.apache.org/jira/browse/MYRIAD-200 > Project: Myriad > Issue Type: Bug > Components: Executor, Scheduler >Reporter: DarinJ >Assignee: John Yost > > Currently Unit Test coverage is weak in places also, a good integration test > framework would be helpful. (potentially [minimesos|http://minimesos.org]?) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-211) Refactor MyriadOperations.flexUpAService
John Yost created MYRIAD-211: Summary: Refactor MyriadOperations.flexUpAService Key: MYRIAD-211 URL: https://issues.apache.org/jira/browse/MYRIAD-211 Project: Myriad Issue Type: Improvement Reporter: John Yost The MyriadOperations flexUpAService method currently throws a MyriadBadConfigurationException if the requested flex up request makes the total number of active, staging, pending, and requested instances of a service > max number of service instances. I may be missing something, but this does not appear to me to be a configuration issue. Perhaps we should just return a boolean indicating whether the request succeeded. Also, perhaps we should build in retry logic to re-attempt for a dynamically configurable number of times with a dynamically-configured backoff params -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-210) Fully implement ZookeeperHealthCheck
John Yost created MYRIAD-210: Summary: Fully implement ZookeeperHealthCheck Key: MYRIAD-210 URL: https://issues.apache.org/jira/browse/MYRIAD-210 Project: Myriad Issue Type: Improvement Components: Executor, Scheduler Reporter: John Yost Priority: Minor As Ken Sipe mentioned in his TODO, this class remains partially implemented -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MYRIAD-86) Consider an API for desired # instances, rather than flex up/down
[ https://issues.apache.org/jira/browse/MYRIAD-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310798#comment-15310798 ] John Yost commented on MYRIAD-86: - I think this is a good idea, would recommend an integrated API where a starting number of configs can then be flexed up/down, kinda analogous to Java -Xms (initial heap size) and -Xmx (max heap size). > Consider an API for desired # instances, rather than flex up/down > - > > Key: MYRIAD-86 > URL: https://issues.apache.org/jira/browse/MYRIAD-86 > Project: Myriad > Issue Type: Improvement >Reporter: Adam B > > There is value in having both APIs, so I'm not suggesting we remove flex > up/down, just add a new call. > This could also be valuable for AutoScaling (MYRIAD-12) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-209) Abstract Health API
John Yost created MYRIAD-209: Summary: Abstract Health API Key: MYRIAD-209 URL: https://issues.apache.org/jira/browse/MYRIAD-209 Project: Myriad Issue Type: Improvement Components: Executor, Scheduler Reporter: John Yost Priority: Minor Recommend abstracting the org.apache.myriad.health package to enable integration of Consul and/or Marathon health checks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-208) Make ReconcileService backoff parameters configurable
John Yost created MYRIAD-208: Summary: Make ReconcileService backoff parameters configurable Key: MYRIAD-208 URL: https://issues.apache.org/jira/browse/MYRIAD-208 Project: Myriad Issue Type: Improvement Components: Scheduler Reporter: John Yost Priority: Minor Fix For: Myriad 0.3.0 As Mohit recommended in his TODO, render backoff params configurable in ReconcileService. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MYRIAD-134) Support Zookeeper based implementation of RMStateStore for storing Myriad state
[ https://issues.apache.org/jira/browse/MYRIAD-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310555#comment-15310555 ] John Yost commented on MYRIAD-134: -- Good suggestion, and I recommend taking it a step further by enabling storage of Myriad state within Consul, which could centralize state and health checking within Consul > Support Zookeeper based implementation of RMStateStore for storing Myriad > state > --- > > Key: MYRIAD-134 > URL: https://issues.apache.org/jira/browse/MYRIAD-134 > Project: Myriad > Issue Type: Task > Components: Scheduler >Reporter: Swapnil Daingade >Assignee: Swapnil Daingade > > Currently we support a DFS based implementation of RMStateStore for storing > Myriad State (MyriadFileSystemRMStateStore). We need to similarly support a > Zookeeper base one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-207) Better exception handling for configuration file input values
John Yost created MYRIAD-207: Summary: Better exception handling for configuration file input values Key: MYRIAD-207 URL: https://issues.apache.org/jira/browse/MYRIAD-207 Project: Myriad Issue Type: Improvement Components: Executor, Scheduler Affects Versions: Myriad 0.2.0 Reporter: John Yost Fix For: Myriad 0.3.0 There is alot of great exception handling within the Myriad project. To make it even better, I recommend enhanced checking and exception handling logic for Myriad classes that receive input values from configuration files. For example, in the HealthCheckUtils class, I recommend adding a code block to ensure the URL is properly formatted and, if not, an IllegalArgumentException is thrown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-206) Recommend removal of myriad-commons MyriadExecutorDefaults class
John Yost created MYRIAD-206: Summary: Recommend removal of myriad-commons MyriadExecutorDefaults class Key: MYRIAD-206 URL: https://issues.apache.org/jira/browse/MYRIAD-206 Project: Myriad Issue Type: Improvement Components: Executor Affects Versions: Myriad 0.2.0 Reporter: John Yost Priority: Minor Fix For: Myriad 0.3.0 I recommend replacing a centralized defaults class with putting the defaults with their corresponding classes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-205) Myriad UI should warn user when framework registration fails/framework removed
John Yost created MYRIAD-205: Summary: Myriad UI should warn user when framework registration fails/framework removed Key: MYRIAD-205 URL: https://issues.apache.org/jira/browse/MYRIAD-205 Project: Myriad Issue Type: Bug Components: Scheduler Affects Versions: Myriad 0.2.0 Reporter: John Yost Priority: Minor The Myriad UI should display a warning when Myriad Mesos framework registration fails and/or Myriad is removed from the Mesos Active Frameworks list. Also, flex up/flex down requests should fail gracefully when Myriad is not an active framework. Currently flex up requests remain in pending state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MYRIAD-203) Update NodeTask constructor to ensure uniqueness
[ https://issues.apache.org/jira/browse/MYRIAD-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15287432#comment-15287432 ] John Yost commented on MYRIAD-203: -- This can be accomplished by retaining default equals method (returns false if the two object references are to duplicate instances with same state) as it is now, but it would be safer if the object state of each NodeTask is unique. > Update NodeTask constructor to ensure uniqueness > > > Key: MYRIAD-203 > URL: https://issues.apache.org/jira/browse/MYRIAD-203 > Project: Myriad > Issue Type: Bug >Reporter: John Yost > > The NodeTask constructor does not currently ensure uniqueness in that two > NodeTasks could have the same ServiceResourceProfile and Constraint, even > though they are separate NodeTask objects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MYRIAD-203) Update NodeTask constructor to ensure uniqueness
[ https://issues.apache.org/jira/browse/MYRIAD-203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Yost updated MYRIAD-203: - Priority: Minor (was: Major) > Update NodeTask constructor to ensure uniqueness > > > Key: MYRIAD-203 > URL: https://issues.apache.org/jira/browse/MYRIAD-203 > Project: Myriad > Issue Type: Bug >Reporter: John Yost >Priority: Minor > > The NodeTask constructor does not currently ensure uniqueness in that two > NodeTasks could have the same ServiceResourceProfile and Constraint, even > though they are separate NodeTask objects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-203) Update NodeTask constructor to ensure uniqueness
John Yost created MYRIAD-203: Summary: Update NodeTask constructor to ensure uniqueness Key: MYRIAD-203 URL: https://issues.apache.org/jira/browse/MYRIAD-203 Project: Myriad Issue Type: Bug Reporter: John Yost The NodeTask constructor does not currently ensure uniqueness in that two NodeTasks could have the same ServiceResourceProfile and Constraint, even though they are separate NodeTask objects. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-202) minimesos-based integration tests
John Yost created MYRIAD-202: Summary: minimesos-based integration tests Key: MYRIAD-202 URL: https://issues.apache.org/jira/browse/MYRIAD-202 Project: Myriad Issue Type: Improvement Reporter: John Yost Implement integration tests with mini-mesos to bridge the gap between a unit test harness and deployment to a mesos cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MYRIAD-201) Myriad state objects should be immutable
John Yost created MYRIAD-201: Summary: Myriad state objects should be immutable Key: MYRIAD-201 URL: https://issues.apache.org/jira/browse/MYRIAD-201 Project: Myriad Issue Type: Improvement Components: Scheduler Reporter: John Yost Priority: Minor I recommend that all Myriad state objects that should remain unchanged following startup be refactored to be immutable. For objects that have some state that could change (i.e., Cluster), make the unchanging instance attributes immutable. Rendering as much state as possible to be immutable will hopefully prevent runtime errors and mysterious anomalies. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MYRIAD-198) Remove optionals when sane defaults are available
[
https://issues.apache.org/jira/browse/MYRIAD-198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15287094#comment-15287094
]
John Yost commented on MYRIAD-198:
--
Updated per feedback from Darin, returning empty Map objects and updated
calling code accordingly.
> Remove optionals when sane defaults are available
> -
>
> Key: MYRIAD-198
> URL: https://issues.apache.org/jira/browse/MYRIAD-198
> Project: Myriad
> Issue Type: Bug
> Components: Executor, Scheduler
>Affects Versions: Myriad 0.2.0
>Reporter: DarinJ
>Priority: Minor
> Labels: easyfix, newbie
>
> Currently we overuse Optionals in the config and then use an or method in
> various factories later. In many cases having the configuration return a
> default when the parameter was specified would create cleaner code. For
> instance:
> {quote}
> Optional getCgroups() {
> Optional.fromNullable(cgroups);
> }
> {quote}
> vs
> {quote}
> Boolean getCgroups() {
> return cgroups != null ? cgroups : false;
> }
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MYRIAD-200) Increased Unit Test and Integration Testing
[ https://issues.apache.org/jira/browse/MYRIAD-200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280028#comment-15280028 ] John Yost commented on MYRIAD-200: -- Hi Everyone, I just created a branch off of my fork (https://github.com/hokiegeek2/incubator-myriad) and am currently working this ticket. --John > Increased Unit Test and Integration Testing > --- > > Key: MYRIAD-200 > URL: https://issues.apache.org/jira/browse/MYRIAD-200 > Project: Myriad > Issue Type: Bug > Components: Executor, Scheduler >Reporter: DarinJ > > Currently Unit Test coverage is weak in places also, a good integration test > framework would be helpful. (potentially [minimesos|http://minimesos.org]?) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MYRIAD-198) Remove optionals when sane defaults are available
[
https://issues.apache.org/jira/browse/MYRIAD-198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15279991#comment-15279991
]
John Yost commented on MYRIAD-198:
--
Wrapped up work, submitted pull request
> Remove optionals when sane defaults are available
> -
>
> Key: MYRIAD-198
> URL: https://issues.apache.org/jira/browse/MYRIAD-198
> Project: Myriad
> Issue Type: Bug
> Components: Executor, Scheduler
>Affects Versions: Myriad 0.2.0
>Reporter: DarinJ
>Priority: Minor
> Labels: easyfix, newbie
>
> Currently we overuse Optionals in the config and then use an or method in
> various factories later. In many cases having the configuration return a
> default when the parameter was specified would create cleaner code. For
> instance:
> {quote}
> Optional getCgroups() {
> Optional.fromNullable(cgroups);
> }
> {quote}
> vs
> {quote}
> Boolean getCgroups() {
> return cgroups != null ? cgroups : false;
> }
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MYRIAD-198) Remove optionals when sane defaults are available
[
https://issues.apache.org/jira/browse/MYRIAD-198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15274175#comment-15274175
]
John Yost commented on MYRIAD-198:
--
Hi Everyone,
I forked the project (https://github.com/hokiegeek2/incubator-myriad) and am
working on this ticket.
--John
> Remove optionals when sane defaults are available
> -
>
> Key: MYRIAD-198
> URL: https://issues.apache.org/jira/browse/MYRIAD-198
> Project: Myriad
> Issue Type: Bug
> Components: Executor, Scheduler
>Affects Versions: Myriad 0.2.0
>Reporter: DarinJ
>Priority: Minor
> Labels: easyfix, newbie
>
> Currently we overuse Optionals in the config and then use an or method in
> various factories later. In many cases having the configuration return a
> default when the parameter was specified would create cleaner code. For
> instance:
> {quote}
> Optional getCgroups() {
> Optional.fromNullable(cgroups);
> }
> {quote}
> vs
> {quote}
> Boolean getCgroups() {
> return cgroups != null ? cgroups : false;
> }
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
