Re: [openstack-dev] [heat] [savanna] [trove] Place for software configuration

[Jay Pipes]

On 11/01/2013 10:29 AM, Alexander Kuznetsov wrote:

Jay. Do you have a plan to add a Savanna (type: Heat::Savanna) and Trove
  (type: Heat::Trove)  providers to the HOT DSL?


Hi Alexander,

No, but I'd be interested in working on them, particularly a Savannah 
provider for Heat. I can start on it once the instance group API 
extension is in Nova and wired into Heat.


Best,
-jay


On Thu, Oct 31, 2013 at 10:33 PM, Jay Pipes mailto:jaypi...@gmail.com>> wrote:

On 10/31/2013 01:51 PM, Alexander Kuznetsov wrote:

Hi Heat, Savanna and Trove teams,

All this projects have common part related to software configuration
management.  For creation,  an environment  user should specify a
hardware parameter for vms:  choose flavor, decide use cinder or
not,
configure networks for virtual machines, choose topology for hole
deployment. Next step is linking of software parameters with
hardware
specification. From the end user point of view, existence of three
different places and three different ways (HEAT Hot DSL, Trove
clustering API and Savanna Hadoop templates) for software
configuration
is not convenient, especially if user want to create an environment
simultaneously involving components from Savanna, Heat and Trove.

I can suggest two approaches to overcome this situations:

Common library in oslo. This approach allows a deep domain specific
customization. The user will still have 3 places with same UI
where user
should perform configuration actions.

Heat or some other component for software configuration
management. This
approach is the best for end users. In feature possible will be some
limitation on deep domain specific customization for configuration
management.


I think this would be my preference.

In other words, describe and orchestrate a Hadoop or Database setup
using HOT templates and using Heat as the orchestration engine.

Best,
-jay

Heat, Savanna and Trove teams can you comment these ideas, what
approach
are the best?

Alexander Kuznetsov.


_
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.__org

http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev




_
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.__org

http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev 





___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [heat] [savanna] [trove] Place for software configuration

[Alexander Kuznetsov]

On Fri, Nov 1, 2013 at 12:39 AM, Clint Byrum  wrote:

> Excerpts from Alexander Kuznetsov's message of 2013-10-31 10:51:54 -0700:
> > Hi Heat, Savanna and Trove teams,
> >
> > All this projects have common part related to software configuration
> > management.  For creation,  an environment  user should specify a
> hardware
> > parameter for vms:  choose flavor, decide use cinder or not, configure
> > networks for virtual machines, choose topology for hole deployment. Next
> > step is linking of software parameters with hardware specification. From
> > the end user point of view, existence of three different places and three
> > different ways (HEAT Hot DSL, Trove clustering API and Savanna Hadoop
> > templates) for software configuration is not convenient, especially if
> user
> > want to create an environment simultaneously involving components from
> > Savanna, Heat and Trove.
> >
>
> I'm having a hard time extracting the problem statement. I _think_ that
> the problem is:
>
> As a user I want to tune my software for my available hardware.
>
> So what you're saying is, if you select a flavor that has 4GB of RAM
> for your application, you want to also tell your application that it
> can use 3GB of RAM for an in-memory cache. Likewise, if one has asked
> Trove for an 8GB flavor, they will want to tell it to use 6.5GB of RAM
> for InnoDB buffer cache.
>
> What you'd like to see is one general pattern to express these types
> of things?
>
Exactly.

>
> > I can suggest two approaches to overcome this situations:
> >
> > Common library in oslo. This approach allows a deep domain specific
> > customization. The user will still have 3 places with same UI where user
> > should perform configuration actions.
> >
> > Heat or some other component for software configuration management. This
> > approach is the best for end users. In feature possible will be some
> > limitation on deep domain specific customization for configuration
> > management.
>
> Can you maybe be more concrete with your proposed solutions? The lack
> of a clear problem statement combined with these vague solutions has
> thoroughly confused me.
>
>
> Sure. I suggest creating a some library or component for standardization
of  software and hardware configuration. It will contain a validation logic
and parameters lists.

Now Trove, Savanna and Heat all have part related to hardware
configuration. For end user, VMs description should not depend on component
where it will be used.

Here is an example of VM description which could be common for Savanna and
Trove:

{
   flavor_id: 42,
   image_id: ”test”,
   volumes: [{
   # extra contains a domain specific parameters.
   # For instance aim for Savanna
   # could be hdfs-dir or mapreduce-dir.
   # For trove: journal-dir or db-dir.
   extra: {
   aim: hdfs-dir
   },
   size: 10GB,
   filesystem: ext3
 },{
   extra: {
 aim: mapreduce-dir
   },
   size: 5GB,
   filesystem: ext3
 }]
networks: [{
   private-network: some-private-net-id,
   public-network: some-public-net-id
 }]


Also, it will be great if this library or component will standardize some
software configuration parameters, like a credential for database or LDAP.
This greatly simplify integration between different components. For example
if user want process data on Hadoop from Cassandra, user should provide a
database location and credentials to Hadoop. If we have some standard for
both Trove and Savanna, it can be done the same way in both components. An
example for Cassandra could look like that:


{
  type: cassandra,
  host: example.com,
  port: 1234,
  credentials: {
 user: ”test”,
 password: ”123”
  }
}


This parameters names and schema should be the same for different
components referencing a Cassandra server.
>
> 
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [heat] [savanna] [trove] Place for software configuration

[Alexander Kuznetsov]

Jay. Do you have a plan to add a Savanna (type: Heat::Savanna) and Trove
 (type: Heat::Trove)  providers to the HOT DSL?


On Thu, Oct 31, 2013 at 10:33 PM, Jay Pipes  wrote:

> On 10/31/2013 01:51 PM, Alexander Kuznetsov wrote:
>
>> Hi Heat, Savanna and Trove teams,
>>
>> All this projects have common part related to software configuration
>> management.  For creation,  an environment  user should specify a
>> hardware parameter for vms:  choose flavor, decide use cinder or not,
>> configure networks for virtual machines, choose topology for hole
>> deployment. Next step is linking of software parameters with hardware
>> specification. From the end user point of view, existence of three
>> different places and three different ways (HEAT Hot DSL, Trove
>> clustering API and Savanna Hadoop templates) for software configuration
>> is not convenient, especially if user want to create an environment
>> simultaneously involving components from Savanna, Heat and Trove.
>>
>> I can suggest two approaches to overcome this situations:
>>
>> Common library in oslo. This approach allows a deep domain specific
>> customization. The user will still have 3 places with same UI where user
>> should perform configuration actions.
>>
>> Heat or some other component for software configuration management. This
>> approach is the best for end users. In feature possible will be some
>> limitation on deep domain specific customization for configuration
>> management.
>>
>
> I think this would be my preference.
>
> In other words, describe and orchestrate a Hadoop or Database setup using
> HOT templates and using Heat as the orchestration engine.
>
> Best,
> -jay
>
>  Heat, Savanna and Trove teams can you comment these ideas, what approach
>> are the best?
>>
>> Alexander Kuznetsov.
>>
>>
>> __**_
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.**org 
>> http://lists.openstack.org/**cgi-bin/mailman/listinfo/**openstack-dev
>>
>>
>
> __**_
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.**org 
> http://lists.openstack.org/**cgi-bin/mailman/listinfo/**openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [heat] [savanna] [trove] Place for software configuration

[Clint Byrum]

Excerpts from Alexander Kuznetsov's message of 2013-10-31 10:51:54 -0700:
> Hi Heat, Savanna and Trove teams,
> 
> All this projects have common part related to software configuration
> management.  For creation,  an environment  user should specify a hardware
> parameter for vms:  choose flavor, decide use cinder or not, configure
> networks for virtual machines, choose topology for hole deployment. Next
> step is linking of software parameters with hardware specification. From
> the end user point of view, existence of three different places and three
> different ways (HEAT Hot DSL, Trove clustering API and Savanna Hadoop
> templates) for software configuration is not convenient, especially if user
> want to create an environment simultaneously involving components from
> Savanna, Heat and Trove.
> 

I'm having a hard time extracting the problem statement. I _think_ that
the problem is:

As a user I want to tune my software for my available hardware.

So what you're saying is, if you select a flavor that has 4GB of RAM
for your application, you want to also tell your application that it
can use 3GB of RAM for an in-memory cache. Likewise, if one has asked
Trove for an 8GB flavor, they will want to tell it to use 6.5GB of RAM
for InnoDB buffer cache.

What you'd like to see is one general pattern to express these types
of things?

> I can suggest two approaches to overcome this situations:
> 
> Common library in oslo. This approach allows a deep domain specific
> customization. The user will still have 3 places with same UI where user
> should perform configuration actions.
> 
> Heat or some other component for software configuration management. This
> approach is the best for end users. In feature possible will be some
> limitation on deep domain specific customization for configuration
> management.

Can you maybe be more concrete with your proposed solutions? The lack
of a clear problem statement combined with these vague solutions has
thoroughly confused me.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [heat] [savanna] [trove] Place for software configuration

[Jay Pipes]

On 10/31/2013 01:51 PM, Alexander Kuznetsov wrote:

Hi Heat, Savanna and Trove teams,

All this projects have common part related to software configuration
management.  For creation,  an environment  user should specify a
hardware parameter for vms:  choose flavor, decide use cinder or not,
configure networks for virtual machines, choose topology for hole
deployment. Next step is linking of software parameters with hardware
specification. From the end user point of view, existence of three
different places and three different ways (HEAT Hot DSL, Trove
clustering API and Savanna Hadoop templates) for software configuration
is not convenient, especially if user want to create an environment
simultaneously involving components from Savanna, Heat and Trove.

I can suggest two approaches to overcome this situations:

Common library in oslo. This approach allows a deep domain specific
customization. The user will still have 3 places with same UI where user
should perform configuration actions.

Heat or some other component for software configuration management. This
approach is the best for end users. In feature possible will be some
limitation on deep domain specific customization for configuration
management.


I think this would be my preference.

In other words, describe and orchestrate a Hadoop or Database setup 
using HOT templates and using Heat as the orchestration engine.


Best,
-jay


Heat, Savanna and Trove teams can you comment these ideas, what approach
are the best?

Alexander Kuznetsov.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [heat] [savanna] [trove] Place for software configuration

[Alexander Kuznetsov]

Hi Heat, Savanna and Trove teams,

All this projects have common part related to software configuration
management.  For creation,  an environment  user should specify a hardware
parameter for vms:  choose flavor, decide use cinder or not, configure
networks for virtual machines, choose topology for hole deployment. Next
step is linking of software parameters with hardware specification. From
the end user point of view, existence of three different places and three
different ways (HEAT Hot DSL, Trove clustering API and Savanna Hadoop
templates) for software configuration is not convenient, especially if user
want to create an environment simultaneously involving components from
Savanna, Heat and Trove.

I can suggest two approaches to overcome this situations:

Common library in oslo. This approach allows a deep domain specific
customization. The user will still have 3 places with same UI where user
should perform configuration actions.

Heat or some other component for software configuration management. This
approach is the best for end users. In feature possible will be some
limitation on deep domain specific customization for configuration
management.

Heat, Savanna and Trove teams can you comment these ideas, what approach
are the best?

Alexander Kuznetsov.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev