[jira] [Commented] (TWILL-211) Retries of failed runnable instances may result in unsatisfiable provisioning requests
[
https://issues.apache.org/jira/browse/TWILL-211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849407#comment-15849407
]
ASF GitHub Bot commented on TWILL-211:
--
Github user poornachandra commented on the issue:
https://github.com/apache/twill/pull/29
LGTM from me too
> Retries of failed runnable instances may result in unsatisfiable provisioning
> requests
> --
>
> Key: TWILL-211
> URL: https://issues.apache.org/jira/browse/TWILL-211
> Project: Apache Twill
> Issue Type: Bug
> Components: core
>Affects Versions: 0.9.0
>Reporter: Martin Serrano
>Assignee: Martin Serrano
>Priority: Critical
> Fix For: 0.10.0
>
>
> In my investigation into the intermittent failures of tests for TWILL-181 I
> discovered this bug. This code (starting on line 703 of
> ApplicationMasterService):
> {code}
> if (expectedContainers.getExpected(runnableName) ==
> runningContainers.count(runnableName) ||
>
> provisioning.peek().getType().equals(AllocationSpecification.Type.ALLOCATE_ONE_INSTANCE_AT_A_TIME))
> {
> provisioning.poll();
> }
> {code}
> There is a case when instances are failing (but not simultaneously) where the
> retries for the instances will be spread over two invocations of
> `ApplicationMasterService.handleCompleted`. This means they will be part of
> separate `RunnableContainerRequests` and thus will be provisioned separately.
> But because the code above does not anticipate this case, the first
> provisionRequest will never appear to be satisfied, never be polled and the
> total can never be met.
> The first provisionRequest does not appear to be satisfied because the
> expected containers will never equal the running containers. The code as-is
> expects every request to be an `ALLOCATE_ONE_INSTANCE_AT_A_TIME` or for all
> instances. In the case of retries, requests may can in all at once or in
> other patterns which result in multiple provision requests.
> When retrying instances, the code should set the type to be
> `ALLOCATE_ONE_INSTANCE_AT_A_TIME` to reflect the situation.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[GitHub] twill pull request #29: (TWILL-211) use ALLOCATE_ONE_INSTANCE_AT_A_TIME for ...
Github user asfgit closed the pull request at: https://github.com/apache/twill/pull/29 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (TWILL-191) Add coding style preferences for Eclipse to site
[ https://issues.apache.org/jira/browse/TWILL-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849243#comment-15849243 ] ASF GitHub Bot commented on TWILL-191: -- Github user hsaputra commented on the issue: https://github.com/apache/twill/pull/30 Oh sorry, Terence already commented about merging to the Site. Pls continue =P > Add coding style preferences for Eclipse to site > > > Key: TWILL-191 > URL: https://issues.apache.org/jira/browse/TWILL-191 > Project: Apache Twill > Issue Type: Task >Reporter: Martin Serrano >Assignee: Martin Serrano >Priority: Minor > Fix For: 0.10.0 > > > As mentioned on the dev list it would be good to have preferences files for > the major IDEs. This ticket is to track the Eclipse version of those > preferences. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] twill issue #30: (TWILL-191), add eclipse settings to site
Github user hsaputra commented on the issue: https://github.com/apache/twill/pull/30 Oh sorry, Terence already commented about merging to the Site. Pls continue =P --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] twill issue #29: (TWILL-211) use ALLOCATE_ONE_INSTANCE_AT_A_TIME for retries...
Github user hsaputra commented on the issue: https://github.com/apache/twill/pull/29 Will merge this if no more comment --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] twill issue #25: (TWILL-205) Add getOwner(), getGroup() and setGroup() to Lo...
Github user hsaputra commented on the issue: https://github.com/apache/twill/pull/25 @anew please close it if already merged --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] twill issue #25: (TWILL-205) Add getOwner(), getGroup() and setGroup() to Lo...
Github user anwar6953 commented on the issue: https://github.com/apache/twill/pull/25 This PR can be closed now? The changes appear to be on master branch already. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
interest in bare-bones cluster/agent solution?
Hey Devs, I've been evangelizing Twill at here at my company. We have a for several years had a basic clustering solution for running our system and its components across multiple hosts. As we've worked to migrate the platform to Hadoop and YARN, we have started to move our cluster control to be Yarn-based. And my goal is to make it Twill-based. However as we do this work we anticipate the need to support customers that do not have and do not want to operate Hadoop infrastructure. I'm working on an architecture which would use Twill for command and control for all of our services. For customers that did not want the full enterprise capabilities that come with Hadoop we could continue to offer our basic clustering support (albeit with reduced capabilities) by plugging our clustering solution into Twill. One of the aspects of Twill that interested me from the start was that the control API was abstracted from YARN. A couple questions: 1) Do any plugins for other clustering backends already exist? Open source? Commercial? 2) If we were to do this, would the Twill community be interested in a donation of this code to the Twill project? I recall that Henry and I had a conversation at the last Apache Big Data that there had been talk of plugins for other clustering backends but I don't know if anything ever came of that. Cheers, Martin Serrano
