[jira] [Created] (TEZ-4009) Zookeeper based AM Registry
Eric Wohlstadter created TEZ-4009: - Summary: Zookeeper based AM Registry Key: TEZ-4009 URL: https://issues.apache.org/jira/browse/TEZ-4009 Project: Apache Tez Issue Type: Sub-task Reporter: Eric Wohlstadter Assignee: Eric Wohlstadter Zookeeper based implementation of the interface defined by TEZ-4008. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (TEZ-4008) Pluggable AM pool registry
Eric Wohlstadter created TEZ-4008:
-
Summary: Pluggable AM pool registry
Key: TEZ-4008
URL: https://issues.apache.org/jira/browse/TEZ-4008
Project: Apache Tez
Issue Type: Sub-task
Reporter: Eric Wohlstadter
Assignee: Eric Wohlstadter
Interface and reflective plugin configuration for a pool of AMs identified by a
namespace.
The registry should allow each {{DAGClientServer}} to register/unregister
themselves from a pool.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (TEZ-4007) Zookeeper based FrameworkClient
Eric Wohlstadter created TEZ-4007: - Summary: Zookeeper based FrameworkClient Key: TEZ-4007 URL: https://issues.apache.org/jira/browse/TEZ-4007 Project: Apache Tez Issue Type: Sub-task Reporter: Eric Wohlstadter Assignee: Eric Wohlstadter A sub-class of FrameworkClient that interacts with application state managed in Zookeeper rather than Yarn. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (TEZ-4006) Make FrameworkClient Pluggable
Eric Wohlstadter created TEZ-4006:
-
Summary: Make FrameworkClient Pluggable
Key: TEZ-4006
URL: https://issues.apache.org/jira/browse/TEZ-4006
Project: Apache Tez
Issue Type: Sub-task
Reporter: Eric Wohlstadter
Assignee: Eric Wohlstadter
Currently {{FrameworkClient.createFrameworkClient}} is hardcoded to use either
LocalClient or YarnClient.
Change it to allow specifying a FrameworkClient impl. class from configuration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] tez issue #33: TEZ-3998: support constructing DAG with concurrent edge
Github user yingdachen commented on the issue: https://github.com/apache/tez/pull/33 > @yingdachen since this is part of a larger end-to-end feature (Tez-3997) would it be better for you to own a feature branch, in which you commit for the subtasks, and later, when the feature is complete, merge into master? > It is easier to provide meaningful feedback when seeing how it all comes together - right now, most of the changes are stubs. @anicoara thanks for the feedback. couple of things 1. The design doc in 3997 was meant to provide the big picture and as the place to collect feedback for the overall design. We believe suitable level of details about proposed changes have been provided, and it would be great hear from community on the design. 2. The feature was broken into 4 tasks carefully that range from basic support and plugin addition(3998), api change in AM component (3999), change in runtime (4000) and more complete scenario support (4001). We believe such break-down is suitable for us as new contributors to fit in, beginning with a simple change discussed here (3998) that is standalone in itself. We are hoping that our changes will take a gradual path to completion, which would not necessitate a separate feature branch. 3. In the change introduced here, the SilentEdgeManager is indeed a stub, that is by design and would not likely to change. However, I would not categorize the other change (such as the new VertexManagerWithConcurrentInput) as so, since it is functional as it is and we have added UT coverage to cover that as well. ---
Re: [DISCUSS] Slack for Apache Tez
+1 > On Oct 16, 2018, at 12:00 PM, Eric Wohlstadter wrote: > > I like the idea of a 2-3 month trial. > > On Tue, Oct 16, 2018 at 9:42 AM Jonathan Eagles wrote: > >> Thanks for the feedback everyone. I can see that on the one hand, 1) >> this tool could help the community feel more connected. At the same >> time, if we move discussions to this tool, 2) they no longer are part >> of the public record, and could to lead to information loss. >> >> If we started a 2-3 month trial, would that be enough time to >> understand if the tool is working? Then we could hold another >> discussion about if it is working. >> >> Regards, >> jeagles >> On Sat, Oct 6, 2018 at 7:21 PM Kuhu Shukla >> wrote: >>> >>> "How does everyone feel about trying slack temporarily to see if it aids >> Tez >>> development?" >>> >>> This would be a great way to get attention on certain JIRAs and ask >>> questions/discuss releases and features. Agree that reviews should be >> JIRA >>> based only. >>> >>> Regards, >>> Kuhu >>> >>> On Tue, Oct 2, 2018 at 2:17 PM Eric Wohlstadter >> wrote: >>> I like the idea for general questions and discussions. Agree that >> reviews shouldn't take place on Slack. On Tue, Oct 2, 2018 at 7:13 AM Eric Badger wrote: > What would the purpose of the slack be? Would it be for hashing out >> big > details about Tez, discussing issues that people are seeing, and >> maybe > identifying patches that need review? My main concern would be that >> patch > reviews would become dominated by slack messaging back and forth and >> so we > would lose the information on JIRA. > > On the surface this sounds like a really nice idea. But, I think we >> need to > be clear about how much reviewing we do in slack vs on the JIRA so >> that we > have a centralized history of patches and reviews. > > Eric > > On Mon, Oct 1, 2018 at 5:44 PM Jonathan Eagles wrote: > >> At our most recent meetup we discussed the possibility of creating >> a > slack >> channel that could be used for developers of Apache Tez. >> >> How does everyone feel about trying slack temporarily to see if it >> aids > Tez >> development? >> Since ASF does not host or support slack channel they are run by > volunteers >> on behalf of projects. I would be happy to volunteer to setup this >> up and >> run the trial. >> >> Jon >> > >>
Re: [DISCUSS] Slack for Apache Tez
I like the idea of a 2-3 month trial. On Tue, Oct 16, 2018 at 9:42 AM Jonathan Eagles wrote: > Thanks for the feedback everyone. I can see that on the one hand, 1) > this tool could help the community feel more connected. At the same > time, if we move discussions to this tool, 2) they no longer are part > of the public record, and could to lead to information loss. > > If we started a 2-3 month trial, would that be enough time to > understand if the tool is working? Then we could hold another > discussion about if it is working. > > Regards, > jeagles > On Sat, Oct 6, 2018 at 7:21 PM Kuhu Shukla > wrote: > > > > "How does everyone feel about trying slack temporarily to see if it aids > Tez > > development?" > > > > This would be a great way to get attention on certain JIRAs and ask > > questions/discuss releases and features. Agree that reviews should be > JIRA > > based only. > > > > Regards, > > Kuhu > > > > On Tue, Oct 2, 2018 at 2:17 PM Eric Wohlstadter > wrote: > > > > > I like the idea for general questions and discussions. Agree that > reviews > > > shouldn't take place on Slack. > > > > > > On Tue, Oct 2, 2018 at 7:13 AM Eric Badger > > > wrote: > > > > > > > What would the purpose of the slack be? Would it be for hashing out > big > > > > details about Tez, discussing issues that people are seeing, and > maybe > > > > identifying patches that need review? My main concern would be that > patch > > > > reviews would become dominated by slack messaging back and forth and > so > > > we > > > > would lose the information on JIRA. > > > > > > > > On the surface this sounds like a really nice idea. But, I think we > need > > > to > > > > be clear about how much reviewing we do in slack vs on the JIRA so > that > > > we > > > > have a centralized history of patches and reviews. > > > > > > > > Eric > > > > > > > > On Mon, Oct 1, 2018 at 5:44 PM Jonathan Eagles > > > wrote: > > > > > > > > > At our most recent meetup we discussed the possibility of creating > a > > > > slack > > > > > channel that could be used for developers of Apache Tez. > > > > > > > > > > How does everyone feel about trying slack temporarily to see if it > aids > > > > Tez > > > > > development? > > > > > Since ASF does not host or support slack channel they are run by > > > > volunteers > > > > > on behalf of projects. I would be happy to volunteer to setup this > up > > > and > > > > > run the trial. > > > > > > > > > > Jon > > > > > > > > > > > > >
[GitHub] tez issue #33: TEZ-3998: support constructing DAG with concurrent edge
Github user anicoara commented on the issue: https://github.com/apache/tez/pull/33 @yingdachen since this is part of a larger end-to-end feature (Tez-3997) would it be better for you to own a feature branch, in which you commit for the subtasks, and later, when the feature is complete, merge into master? It is easier to provide meaningful feedback when seeing how it all comes together - right now, most of the changes are stubs. ---
Re: [DISCUSS] Slack for Apache Tez
Thanks for the feedback everyone. I can see that on the one hand, 1) this tool could help the community feel more connected. At the same time, if we move discussions to this tool, 2) they no longer are part of the public record, and could to lead to information loss. If we started a 2-3 month trial, would that be enough time to understand if the tool is working? Then we could hold another discussion about if it is working. Regards, jeagles On Sat, Oct 6, 2018 at 7:21 PM Kuhu Shukla wrote: > > "How does everyone feel about trying slack temporarily to see if it aids Tez > development?" > > This would be a great way to get attention on certain JIRAs and ask > questions/discuss releases and features. Agree that reviews should be JIRA > based only. > > Regards, > Kuhu > > On Tue, Oct 2, 2018 at 2:17 PM Eric Wohlstadter wrote: > > > I like the idea for general questions and discussions. Agree that reviews > > shouldn't take place on Slack. > > > > On Tue, Oct 2, 2018 at 7:13 AM Eric Badger > > wrote: > > > > > What would the purpose of the slack be? Would it be for hashing out big > > > details about Tez, discussing issues that people are seeing, and maybe > > > identifying patches that need review? My main concern would be that patch > > > reviews would become dominated by slack messaging back and forth and so > > we > > > would lose the information on JIRA. > > > > > > On the surface this sounds like a really nice idea. But, I think we need > > to > > > be clear about how much reviewing we do in slack vs on the JIRA so that > > we > > > have a centralized history of patches and reviews. > > > > > > Eric > > > > > > On Mon, Oct 1, 2018 at 5:44 PM Jonathan Eagles > > wrote: > > > > > > > At our most recent meetup we discussed the possibility of creating a > > > slack > > > > channel that could be used for developers of Apache Tez. > > > > > > > > How does everyone feel about trying slack temporarily to see if it aids > > > Tez > > > > development? > > > > Since ASF does not host or support slack channel they are run by > > > volunteers > > > > on behalf of projects. I would be happy to volunteer to setup this up > > and > > > > run the trial. > > > > > > > > Jon > > > > > > > > >
