[
https://issues.apache.org/jira/browse/BIGTOP-1746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355716#comment-14355716
]
Richard Pelavin edited comment on BIGTOP-1746 at 3/10/15 9:10 PM:
------------------------------------------------------------------
Vishnu,
Good comments. I think useful to divide into two issues:
1) having a simple topology representation that is more intuitive and neutral;
one that resonates with those who dont know hiera, vagrant, etc
2) having such a representation that has the flexibility to capture a wide
variety of topologies
Think if 1 is implemented we could write simple code that "compiled" the
topology into hiera/vagrant, etc and then power by existing bigtop
instrumentation/mechanism.
I think your comments are centered on '2'. The way I would handle this is as
follows, which is predicated on whether there is a desire in bigtop to move in
the direction of providing more sophisticated deployment and orchestration
capabilities:
An approach is to flesh out a rich topology that we would eventually want to
instrument, but we could view this as an internal spec and then just
expose/support a subset of the topologies that are instrumented currently by
bigtop, such as a limited set of topologies that have a single master and set
of slaves. So in a way the rich topology serves as a roadmap to future
extensions and can help capture in advance how potential extensions to treat a
richer set of topologies can be modularly folded in.
Rich
was (Author: rnp):
Vishnu,
Good comments. I think useful to divide into two issues:
1) having a simple topology representation that is more intuitive and neutrall
one that resonates with those who dont know hiera, vagrant, etc
2) having such a representation that has the flexibility to capture a wide
variety of topologies
Think if 1 is implemented we could write simple code that "compiled" the
topology into hiera/vagrant, etc and then power by existing bigtop
instrumentation/mechanism.
I think your comments are centered on '2'. The way I would handle this is as
follows, which is predicated on whether there is a desire in bigtop to move in
the direction of providing more sophisticated deployment and orchestration
capabilities:
An approach is to flesh out a rich topology that we would eventually want to
instrument, but we could view this as an internal spec and then just
expose/support a subset of the topologies that are instrumented currently by
bigtop, such as a limited set of topologies that have a single master and set
of slaves. So in a way the rich topology serves as a roadmap to future
extensions and can help capture in advance how potential extensions to treat a
richer set of topologies can be modularly folded in.
- Rich
> Introduce the concept of roles in bigtop cluster deployment
> -----------------------------------------------------------
>
> Key: BIGTOP-1746
> URL: https://issues.apache.org/jira/browse/BIGTOP-1746
> Project: Bigtop
> Issue Type: New Feature
> Components: deployment
> Reporter: vishnu gajendran
> Labels: features
> Fix For: 0.9.0
>
>
> Currently, during cluster deployment, puppet categorizes nodes as head_node,
> worker_nodes, gateway_nodes, standy_node based on user specified info. This
> functionality gives user control over picking up a particular node as
> head_node, standy_node, gateway_node and rest others as worker_nodes. But, I
> woulld like to have more fine-grained control on which deamons should run on
> which node. For example, I do not want to run namenode, datanode on the same
> node. This functionality can be introduced with the concept of roles. Each
> node can be assigned a set of role. For example, Node A can be assigned
> ["namenode", "resourcemanager"] roles. Node B can be assigned ["datanode",
> "nodemanager"] and Node C can be assigned ["nodemanager", "hadoop-client"].
> Now, each node will only run the specified daemons. Prerequisite for this
> kind of deployment is that each node should be given the necessary
> configurations that it needs to know. For example, each datanode should know
> which is the namenode etc... This functionality will allow users to customize
> the cluster deployment according to their needs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
