Re: [openstack-dev] [sahara] Nominate Trevor McKay for sahara-core
+1 Excellent addition. -John On Tue, May 13, 2014 at 2:26 PM, Andrew Lazarev alaza...@mirantis.comwrote: +1 It will be good addition to the core team. Thanks, Andrew. On Mon, May 12, 2014 at 2:31 PM, Sergey Lukjanov slukja...@mirantis.comwrote: Hey folks, I'd like to nominate Trevor McKay (tmckay) for sahara-core. He is among the top reviewers of Sahara subprojects. Trevor is working on Sahara full time since summer 2013 and is very familiar with current codebase. His code contributions and reviews have demonstrated a good knowledge of Sahara internals. Trevor has a valuable knowledge of EDP part and Hadoop itself. He's working on both bugs and new features implementation. Some links: http://stackalytics.com/report/contribution/sahara-group/30 http://stackalytics.com/report/contribution/sahara-group/90 http://stackalytics.com/report/contribution/sahara-group/180 https://review.openstack.org/#/q/owner:tmckay+sahara+AND+-status:abandoned,n,z https://launchpad.net/~tmckay Sahara cores, please, reply with +1/0/-1 votes. Thanks. -- Sincerely yours, Sergey Lukjanov Sahara Technical Lead (OpenStack Data Processing) Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [savanna] plugin version or hadoop version?
Andrew +1 The HDP plugin also returns the HDP distro version. The version needs to make sense in the context of the plugin. Also, many plugins including the HDP plugin will support deployment of several hadoop versions. -John On Mon, Feb 17, 2014 at 2:36 PM, Andrew Lazarev alaza...@mirantis.comwrote: IDH uses version of IDH distro and there is no direct mapping between distro version and hadoop version. E.g. IDH 2.5.1 works with apache hadoop 1.0.3. I suggest to call the field as just 'version' everywhere and assume this version as plugin specific property. Andrew. On Mon, Feb 17, 2014 at 5:06 AM, Matthew Farrellee m...@redhat.comwrote: $ savanna plugins-list +-+--+---+ | name| versions | title | +-+--+---+ | vanilla | 1.2.1| Vanilla Apache Hadoop | | hdp | 1.3.2| Hortonworks Data Platform | +-+--+---+ above is output from the /plugins endpoint - http://docs.openstack.org/ developer/savanna/userdoc/rest_api_v1.0.html#plugins the question is, should the version be the version of the plugin or the version of hadoop the plugin installs? i ask because it seems like we have version == plugin version for hdp and version == hadoop version for vanilla. the documentation is somewhat vague on the subject, mostly stating version without qualification. however, the json passed to the service references hadoop_version and the arguments in the client are called hadoop_version fyi, this could be complicated by the idh and spark plugins. best, matt ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Savanna] Definition of template
I strongly agree that we should try to keep templates as flexible as possible by allowing some values to be omitted and provided at a later time. But, in this case, we are talking about cluster templates without any node groups being specified. I think that at a minimum a cluster template would contain node groups but could omit the node group counts which could be provided at launch time. This makes a lot of sense. But, in my opinion, without at least specifying the set of node groups in a cluster template, configuration really wouldn't make sense and therefore the template would not be of much/any value. On Wed, Nov 13, 2013 at 10:08 AM, Alexander Ignatov aigna...@mirantis.comwrote: Hi, Andrew Agreed with your opinion. Initially Savanna’s templates approach is the option 1 you are talking about. This was designed at the start of Savanna 0.2 release cycle. It was also documented here: https://wiki.openstack.org/wiki/Savanna/Templates . Maybe some points are outdated but the idea is the same as the option 1: user can create cluster template and don’t need to specify all fields, for example ’node_groups’ field. And these fields, both required and optional, can be overwritten in the cluster object even if it contains ‘cluster_template_id’. I see you raised this question because of patch https://review.openstack.org/#/c/56060/. I think it’s just a bug in the validation level not in api. I also agree that we should change UI part accordingly, at least add an ability for users to override fields set in cluster and node group templates during the cluster creation. Regards, Alexander Ignatov On 12 Nov 2013, at 23:20, Andrey Lazarev alaza...@mirantis.com wrote: Hi all, I want to raise the question What template is. Answer to this question could influence UI, validation and user experience significantly. I see two possible answers: 1. Template is a simplification for object creation. It allows to keep common params in one place and not specify them each time. 2. Template is a full description of object. User should be able to create object from template without specifying any params. As I see the current approach is the option 1, but UI is done mostly for option 2. This leads to situations when user creates incomplete template (backend allows it because of option 1), but can't use it later (UI doesn't allow to work with incomplete templates). Let's define common vision on how will we treat templates and document this somehow. My opinion is that we should proceed with the option 1 and change UI accordingly. Thanks, Andrew ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [savanna] Program name and Mission statement
+1 On 9/11/13 1:13 PM, Matthew Farrellee wrote: That sounds quite good. Best, matt On 09/11/2013 11:42 AM, Andrei Savu wrote: +1 I guess this will also clarify how Savanna relates to other projects like OpenStack Trove. -- Andrei Savu On Wed, Sep 11, 2013 at 5:16 PM, Mike Spreitzer mspre...@us.ibm.com mailto:mspre...@us.ibm.com wrote: To provide a simple, reliable and repeatable mechanism by which to deploy Hadoop and related Big Data projects, including management, monitoring and processing mechanisms driving further adoption of OpenStack. That sounds like it is at about the right level of specificity. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [savanna] Service Relationships and Dependencies
Some services/components are related or have dependencies on other services and components.As an example, in HDP, the Hive service depends on HBase and Zookeeper.In Savanna, there is no way to express this relationship.If a user wanted to deploy Hive, they would need to know to install both HBase and Zookeeper a priori.Also, because the list of service components(node processes) that is provided to a user to be used in node groups is a flat list, only the component name gives any indication as to what service the components belong to.Because of this, it will likely be difficult for the user to understand exactly what components are required to be installed for a given service(s).Currently, the HDP stack consists of approximately 25 service components. A primary reason that it isn't currently possible to express service/component relationships is that topology is defined from the bottom up.This means that a user first selects components and assigns them to a node template.The users first interaction is with components, not services.Currently, the user will not know if a given topology is valid until an attempt is made to deploy a cluster and validate is called on the plugin.At this point, if the topology were invalid, the user would need to go back and create new node and cluster templates. One way to express service relationships would be to define topology top down, with the user first selecting services.After selecting services, the related service components could be listed and the required components could be noted. This approach is a significant change to how Savanna currently works, has not been thoroughly thought through and and is only meant to promote conversation on the matter. After making new services available from the HDP plugin, it is clear that defining a desired (valid) topology will be very difficult and error prone with the current savanna architecture.I look forward to discussing solutions to this matter with the community. -John -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev