Re: [Openstack] EC2 API (euca2ools) vs OpenStack API (novaclient)

2012-02-06 Thread Kevin Jackson
Thanks All,
@Jaypipes - I'm quickly coming to a point where I'm looking at
describing tasks using both nova client and euca2ools - its certainly
not a bad thing, the issue would be on simplicity (i.e. showing people
how things can be done vs you could do this or you could do that -
that choice bit leads to confusion).

I've been using nova client to get my environment off the ground, but
equally using the euca2ools to do some basic stuff like launch
instance (as per the hat-tip) has been going great - it was the
all-round integration that was the issue.

The clear thing it seems though is that its good to know that if I am
encountering issues that its not because I'm fighting a losing battle
with some design choices, there's a genuine reason and there will be
some fix, even if that fix gets resolved in the next few months.

I'll start making notes on the bugs and raise them as appropriate.

Cheers,

Kev

-- 
Kevin Jackson
@itarchitectkev

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Nova] Essex dead wood cutting

2012-02-06 Thread Daniel P. Berrange
On Thu, Feb 02, 2012 at 11:52:17AM +, Armando Migliaccio wrote:
 To the best of my knowledge, the ESXi support is up to date. There may be 
 bugs, but which virt driver is perfect ;)?
 
 Sateesh may know more, because he is the main contributor/maintainer from 
 Citrix.
 
 However, as Vish pointed out in a previous email, any driver is doomed to rot 
 if:
 
 a) no one is deploying OpenStack using the specific driver, thus unveiling 
 potential problems;
 b) a pool of developers (not necessarily the first committer) keep the code 
 up to date, increase functionality and test coverage (both unit and 
 functional);
 
 Clearly both xenapi and libvirt are actively developed and deployed. How 
 about vmwareapi? Anyone?

 Let's make sure that vmwareapi is not going to be the next one to bite the 
 dust.

FWIW, libvirt has pretty reasonable abilities to manage VMWare ESX servers,
and some very basic support for Hyper-V. It would be interesting to see if
the OpenStack libvirt driver can be developed to support these targets too.
If the libvirt VMWare/HyperV drivers are not currently good enough for
OpenStack's needs, IMHO, it would be worth putting effort into improving
libvirt. It seems like a needless duplicated effort to have the libvirt
 OpenStack communities both trying to write hypervisor portability
layers.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Nova] Essex dead wood cutting

2012-02-06 Thread Nathanael Burton
+1!!
On Feb 6, 2012 5:38 AM, Daniel P. Berrange berra...@redhat.com wrote:

 On Thu, Feb 02, 2012 at 11:52:17AM +, Armando Migliaccio wrote:
  To the best of my knowledge, the ESXi support is up to date. There may
 be bugs, but which virt driver is perfect ;)?
 
  Sateesh may know more, because he is the main contributor/maintainer
 from Citrix.
 
  However, as Vish pointed out in a previous email, any driver is doomed
 to rot if:
 
  a) no one is deploying OpenStack using the specific driver, thus
 unveiling potential problems;
  b) a pool of developers (not necessarily the first committer) keep the
 code up to date, increase functionality and test coverage (both unit and
 functional);
 
  Clearly both xenapi and libvirt are actively developed and deployed. How
 about vmwareapi? Anyone?
 
  Let's make sure that vmwareapi is not going to be the next one to bite
 the dust.

 FWIW, libvirt has pretty reasonable abilities to manage VMWare ESX servers,
 and some very basic support for Hyper-V. It would be interesting to see if
 the OpenStack libvirt driver can be developed to support these targets too.
 If the libvirt VMWare/HyperV drivers are not currently good enough for
 OpenStack's needs, IMHO, it would be worth putting effort into improving
 libvirt. It seems like a needless duplicated effort to have the libvirt
  OpenStack communities both trying to write hypervisor portability
 layers.

 Regards,
 Daniel
 --
 |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/:|
 |: http://libvirt.org  -o- http://virt-manager.org:|
 |: http://autobuild.org   -o- http://search.cpan.org/~danberr/:|
 |: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc:|

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] how to make merge proposal to openstack-manuals

2012-02-06 Thread Kei.masumoto

Thanks for your help!
Kei

There is additional specific information at :

http://wiki.openstack.org/Documentation/HowTo

mike





___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Cross-Origin Resource Sharing OpenStack APIs

2012-02-06 Thread Adrian_F_Smith
Sorry for bringing up an old thread but since the question was asked I thought 
someone might find this CORS middleware module I worked on useful.

https://github.com/adrian/swift/tree/cors

What it does is examine each request, decide if it's cross origin or not and 
then deal with it appropriately (i.e. is it a pre-flight request or an actual 
request). In theory (and outside a browser) it works but the problem (and why 
I've parked any further work on it for now) is that none of the browsers out 
there today fully support the CORS specification. They can deal with simple 
requests but when the server returns non-standard headers (such as X-Auth-Token 
and X-Storage-Service) the browser hides them from the client Javascript. The 
browser should whitelist all headers listed in Access-Control-Expose-Headers 
and pass them on to the Javascript. Unfortunately this isn't happening. I've 
tested the latest versions of Chrome, Firefox and IE.

There has been some reference to this problem on various projects so for now 
I'm tracking their progress.

http://code.google.com/p/chromium/issues/detail?id=87338
https://bugzilla.mozilla.org/show_bug.cgi?id=608735
https://bugs.webkit.org/show_bug.cgi?id=41210

When I last tested (1-Feb) Access-Control-Expose-Headers was still being 
ignored.

I'm not holding this code out as being complete or working. I just thought it 
was worth sharing where I got to and what my findings were.

Adrian


-Original Message-
From: openstack-bounces+adrian_f_smith=dell@lists.launchpad.net 
[mailto:openstack-bounces+adrian_f_smith=dell@lists.launchpad.net] On 
Behalf Of Brian Waldon
Sent: Sunday, January 08, 2012 11:11 PM
To: Paul Querna
Cc: openstack@lists.launchpad.net
Subject: Re: [Openstack] Cross-Origin Resource Sharing  OpenStack APIs

Thanks for bringing this to our attention, Paul. It is extremely important to 
design our APIs to be as consumable as possible. I am personally more involved 
in Nova and Glance, but I would still love to try to fit these headers into 
each of their APIs. We are in the process of drafting new API specs for both 
right now (Glance v2 might actually be finalized), but I will work with you to 
see what we can do.

Thanks!
Brian Waldon


On Jan 6, 2012, at 3:26 PM, Paul Querna wrote:

 Hello,
 
 I was wondering what people think about adding Cross-Origin Resource 
 Sharing Headers to all OpenStack APIs by default.
 
 Essentially, this would allow people to make pure frontend javascript 
 clients, which would could be hosted on domains different than the
 API:
  http://www.w3.org/TR/cors/
 
 More examples of how to use this are here:
  https://developer.mozilla.org/En/HTTP_access_control
 
 This boils down to a few changes in OpenStack implementations, mostly 
 it is when an OPTIONS request is sent to the APIs, respond with a 200, 
 and send something like following headers:
 
 Access-Control-Allow-Origin: *
 Access-Control-Allow-Methods: GET, OPTIONS,POST,PUT,DELETE
 Access-Control-Allow-Headers: X-Auth-Token
 Access-Control-Max-Age: 1728000
 
 This would especially be helpful when working against Keystone, as it 
 would let you make applications where the user is prompted for their 
 username  password, and then their token is fetched purely from 
 client side javascript.
 
 The only concern of course is potential security implications.
 However, I've not been able to think of a viable threat model for any 
 OpenStack APIs at this time.
 
 Thoughts?
 
 Thanks,
 
 Paul
 
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] EC2 API (euca2ools) vs OpenStack API (novaclient)

2012-02-06 Thread Jay Pipes

On 02/06/2012 04:48 AM, Kevin Jackson wrote:

Thanks All,
@Jaypipes - I'm quickly coming to a point where I'm looking at
describing tasks using both nova client and euca2ools - its certainly
not a bad thing, the issue would be on simplicity (i.e. showing people
how things can be done vs you could do this or you could do that -
that choice bit leads to confusion).


I understand completely.


I've been using nova client to get my environment off the ground, but
equally using the euca2ools to do some basic stuff like launch
instance (as per the hat-tip) has been going great - it was the
all-round integration that was the issue.

The clear thing it seems though is that its good to know that if I am
encountering issues that its not because I'm fighting a losing battle
with some design choices, there's a genuine reason and there will be
some fix, even if that fix gets resolved in the next few months.


Yep.


I'll start making notes on the bugs and raise them as appropriate.


Awesome! :)

Cheers,
-jay

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] glance performance gains via sendfile()

2012-02-06 Thread Eoghan Glynn


Hi Reynolds,

I've been looking into your interesting idea around sendfile()[1]
usage, here are a few initial thoughts:

- There's potentially even more speed-up to be harnessed in serving
  out images from the filesystem store via sendfile(), than from using
  it client-side on the initial upload (based on the assumption that
  images would typically uploaded once but downloaded many times, also
  that the download time is more crucial for perceived responsiveness
  as an instance being spun up by nova may be held up until the image
  is retreived from glance, unless already cached).

- I'd suspect that some of the potential gain on the client side is
  currently thrown away by the syntax of the glance add CLI, specifically
  the use of shell redirection to pass the image content:

glance add name=MyImage  /path/to/image/file

  I'm open to correction, but this seems to needlessly copy the image
  content via user-space, even if the glance client avoids a second
  copy internally via the sendfile() usage. So I'd propose to also add
  a new cmd line arg to allow the file be directly specified, e.g.

glance add name=MyImage path=/path/to/image/file

  This would have different semantics to the location field, e.g 

location=file:///path/to/image/file

  (which would imply that the content is not uploaded to the remote
  store).

- The structure of typical pysendfile usage gets in the way of
  glance's image iterator pattern. On the client side this is more an
  an incovenience, requiring some restructuring of the code. However
  on the service-side, it seems we're a bit a hamstrung by the
  WSGI/webob APIs. For example the webob.response.body_file is
  filelike but doesn't expose a fileno attribute as there's no real
  underlying FD available intially, so the following kind of approach
  isn't workable:

sendfile(response.body_file.fileno(), 
 filestore_path.fileno(), ...)

  Seems a better approach would be to allow glance to be optionally
  deployed on a WSGI container that directly supports the [2]
  wsgi.file_wrapper extension (e.g. mod_wsgi on Apache, or uWSGI) or
  even allow one of the non-standard headers like X-Sendfile or 
  X-Accel-Redirect to be set where supported.

In case, I'll crack on with the basic client-side usage to begin with,
so that we can quantify the performance gain.

Cheers,
Eoghan

[1] http://code.google.com/p/pysendfile/
[2] 
http://www.python.org/dev/peps/pep-0333/#optional-platform-specific-file-handling

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] three things about OpenStack docs

2012-02-06 Thread Jaesuk Ahn
Anne,

This is Jaesuk Ahn from Korea User Group.

Since the opening of OpenStack Korea Community last year, we have been
keeping our (korean) wiki to share the information among the members.
http://wiki.openstack.or.kr
Most of wiki contents is about overview and installation.
Most of wiki contents is created by our community members, not a direct
translation of openstack documentation.

As the number of people participating in the community activity grows up,
there has been more people interested in documentation in two ways;
1) contributing to Korean wiki, and 2) participating in translation of
the openstack documents, something like localization team at openstack
docs.

Although it has been a good thing for us to have our own wikipage to share
information, IMHO, there should be a better way to collaborate with
original openstack docs team to align what we are doing in Korea with
global community.
For example, we can gather the people from our Korea community, and make a
team to work on Korean-version of openstack docs.

If you have any idea for us to collaborate more with openstack.org, please
let us know it.
Thank you.





On Sat, Jan 28, 2012 at 3:31 AM, Anne Gentle a...@openstack.org wrote:

 Hi all -
 I thought you'd want to know these three things about docs going on lately.

 1. The new installation guide for Diablo that includes Compute,
 Identity, Image, and the Dashboard is now published. Find it
 here:http://docs.openstack.org/diablo/openstack-compute/install/content/.

 2. The PDF links on the home page are being replaced with PDF links in
 the top bar of the HTML manual itself. PDFs aren't going missing, but
 the button is moving. Tell your friends and neighbors.

 3. Now that we have feature freeze for many features, I'm going to
 start a list of documentation needs for the Essex release. This list
 will assist with doc priorities.

 4. I lied when I said three things.

 5. We'd like to hold a Doc Day in March to prep for the Essex release,
 similar to the Bug Squash day coming up next week. I'd like to visit
 sunny California for Doc Day but want other locations to feel free to
 hold their own as well.

 6. The stable/diablo branch of openstack-manuals has been blocked from
 publishing for a little while but I'm aware of the problem and so are
 the members of the CI team. We'll get it fixed and will free the
 floodgates for the backported fixes.

 7. We've got a great new manual titled Programming OpenStack Compute
 API with Shell and Python authored by Jacek Artymiak of DevGuide.net.
 I'm working on the backend to automate publishing from Markdown to get
 it on docs.openstack.org, but you can review the Markdown submission
 here: https://review.openstack.org/3515.

 8. I'd like to go to a weekly docs team meeting for the remainder of
 the Essex release. I'd also like to change the time of day, but likely
 keep it on Monday prior to the team meeting. Suggestions welcome!

 As always, feel free to ask questions and rock the docs.

 Warmly,
 Anne

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp




-- 
*Jaesuk Ahn*, Ph.D.
Team Leader | Cloud OS Dev. Team
Cloud Infrastructure Department
KT (Korea Telecom)
*T. +82-10-9888-0328 | F. +82-303-0993-5340*
*Active member on **OpenStack Korea Community* http://www.openstack.or.kr/
* *
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Swift S3 with Keystone anyone?

2012-02-06 Thread Akira Yoshiyama
Hi Pete,

2012/02/06 10:56 Pete Zaitcev zait...@redhat.com:

 On Thu, 2 Feb 2012 19:03:54 +0900
 Akira Yoshiyama akirayoshiy...@gmail.com wrote:

  I modified the wiki:
  http://wiki.openstack.org/Keystone-BP-S3Token
 
  I added authtoken in main pipeline just now.

 I was unable to find the official configuration for authtoken.
 Since it's only a name of convention, I cannot even guess what
 module is meant here.

 Some people seem to use authtoken like this:

 [filter:authtoken]
 paste.filter_factory = keystone.middleware.auth_token:filter_factory
 service_protocol = http
 service_host = 127.0.0.1
 service_port = 5000
 auth_host = 127.0.0.1
 auth_port = 35357
 auth_protocol = http
 auth_uri = http://127.0.0.1:5000/
 admin_token = 999888777666
 ;Uncomment next line and check ip:port to use memcached to cache token
requests
 ;memcache_hosts = 127.0.0.1:11211

 Is it what you meant? If yes, what does it do in case of S3?

Yes.

Swift has one port with two different APIs: AWS S3 compaible one and
OpenStack one.

So it needs two different authentication capabilities;
authtoken is one middleware for OpenStack API and s3token is the other for
S3 API.
If there are multiple authentication middleware, the primary shouldn't
reject requests without legal credentials because the second may accept
them. So s3token (and ec2token) have to come before authtoken and to be
used with authtoken because it doesn't reject non-S3 requests.

In other words, s3token is just a converter from S3 credentials to
OpenStack tokens.

Regards,
Akira Yoshiyama
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] Atlas-LB - what's the project status?

2012-02-06 Thread Oleg Gelbukh
Hello, everyone

What is the status of this LBaaS project for the OpenStack? As far as I
know, the open-source version is compatible with OpenStack. But is it
possible to merge the Java code in the OpenStack ecosystem? Is someone
working on re-implementing Atlas-LB in Python and eventually adding to the
projects incubator, or there are some other lbaas projects out there?

Thanks in advance,

--
Oleg Gelbukh
Mirantis Inc.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] Swift and devstack.

2012-02-06 Thread Chmouel Boudjnah
Hello,

I was meaning to do add the feature of launching the swift services in the 
foreground in a screen/tmux window' instead of in foreground but would like to 
gather ideas before implementing it. Currently for the Swift All in One we 
launch 4 (3 replicas and one hand-off) different services: account, container 
and object server, 

On each service there is other daemons like container-sync *-updater *-auditor 
etc... which make a grand total of 68

I guess that's not very manageable to have it on a screen, I think to only 
bring on foreground the object servers proxy server since IMO they are the most 
useful ones for devs.

Another thing I want to implement is to add Swift as a back-end to Glance.

And one last thing, which make the thing a bit annoying when you want a 
devstack only to test swift is that currently it's very nova dependent, since 
these lines :

https://github.com/cloudbuilders/devstack/blob/master/stack.sh#L1100

would always get parsed no matter if you don't have any n-* services in there, 
which is not a big deal as add_nova_services is only creating file but it fails 
miserably on :

https://github.com/cloudbuilders/devstack/blob/master/stack.sh#L1404

(Swift need keystone and keystone need MySQL)

What do you think if I add a meta n-svc feature which get enabled only if one 
of the n- services has been activated ?

There is as well python-MySQLDB and python-migrate packages need to don't be 
just in nova but as well in keystone or when nova is not installed keystone 
would fail.

Thoughts? 

Thanks,
Chmouel.___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] glance performance gains via sendfile()

2012-02-06 Thread Rick Jones
If one wants to experiment with the performance effects of sendfile(), 
the netperf benchmark http://www.netperf.org/ has a TCP_SENDFILE 
test which complements the TCP_STREAM test.  It can also report CPU 
utilization and service demand to allow a comparison of efficiency.


netperf -H destination -t TCP_SENDFILE -F file -c -C -l 30

will run a 30-second TCP_SENDFILE tests using file as the data source 
(one is created if no -F option is specified) sending to destination 
(assumes that netserver has been launched on destination.  The 
corresponding TCP_STREAM test would be the obvious substitution.


One area of investigation would be the effect of send size on things. 
That can be accomplished with a test-specific (following a -- on the 
command line) -m option:


netperf  ...as above...  -- -m 64K

would cause netperf to send 65536 bytes in each send call.  The manual 
for the current top-of-trunk version of netperf is at:


http://www.netperf.org/svn/netperf2/trunk/doc/netperf.html

and the top-of-trunk bits can be pulled via subversion pointing at 
http://www.netperf.org/svn/netperf2/trunk


happy benchmarking,

rick jones

For example, between a pair of Ubuntu 11.04 systems with Mellanox 10GbE, 
and a pair of X5650 processors each (so 24 CPUs):


~$ ./netperf -p 12866 -H ndestination -c -C -l 30 -- -P 12867 -m 64K
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 12867 AF_INET to 
destination () port 12867 AF_INET : demo
Recv   SendSend  Utilization   Service 
Demand

Socket Socket  Message  Elapsed  Send Recv SendRecv
Size   SizeSize Time Throughput  localremote   local 
remote

bytes  bytes   bytessecs.10^6bits/s  % S  % S  us/KB   us/KB

 87380  16384  6553630.00  9271.36   2.52 2.72 0.535 
0.576


~$ ./netperf -t TCP_SENDFILE -p 12866 -H destination -c -C -l 30 -- -P 
12867 -m 64K
TCP SENDFILE TEST from 0.0.0.0 (0.0.0.0) port 12867 AF_INET to 
destination () port 12867 AF_INET : demo
Recv   SendSend  Utilization   Service 
Demand

Socket Socket  Message  Elapsed  Send Recv SendRecv
Size   SizeSize Time Throughput  localremote   local 
remote

bytes  bytes   bytessecs.10^6bits/s  % S  % S  us/KB   us/KB

 87380  16384  6553630.00  9332.46   0.82 2.71 0.173 
0.572


It would be good to repeat each a couple times, but in this case at 
least, we see a considerable drop in sending side CPU utilization and 
service demand, the latter being a direct measure of efficiency.


(the socket sizes are simply what they were at the onset of the 
connection, not by the end.  for that, use omni output selectors - 
http://www.netperf.org/svn/netperf2/trunk/doc/netperf.html#Omni-Output-Selection 
- the test-specific -P option is to explicitly select port numbers for 
the data connection to deal with firewalls in my test environment - 
similarly for the global -p option selecting the port number on which 
netserver at destination is waiting)


With a smaller send size  the results may be a bit different:

~$ ./netperf -p 12866 -H destination -c -C -l 30 -- -P 12867
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 12867 AF_INET to 
destination () port 12867 AF_INET : demo
Recv   SendSend  Utilization   Service 
Demand

Socket Socket  Message  Elapsed  Send Recv SendRecv
Size   SizeSize Time Throughput  localremote   local 
remote

bytes  bytes   bytessecs.10^6bits/s  % S  % S  us/KB   us/KB

 87380  16384  1638430.00  9332.43   2.64 2.74 0.556 
0.578
~$ ./netperf -t TCP_SENDFILE -p 12866 -H destination -c -C -l 30 -- -P 
12867
TCP SENDFILE TEST from 0.0.0.0 (0.0.0.0) port 12867 AF_INET to 
destination () port 12867 AF_INET : demo
Recv   SendSend  Utilization   Service 
Demand

Socket Socket  Message  Elapsed  Send Recv SendRecv
Size   SizeSize Time Throughput  localremote   local 
remote

bytes  bytes   bytessecs.10^6bits/s  % S  % S  us/KB   us/KB

 87380  16384  1638430.00  9351.32   1.26 2.73 0.264 
0.574


Mileage will vary depending on link-type, CPU's present, etc etc etc...

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Atlas-LB - what's the project status?

2012-02-06 Thread Jesse Andrews
Hi Oleg,

NOTE: this is my opinion - I do not speak for all of OpenStack!

While our focus is on a successful Essex, the RCB team has started
thinking about Folsom.  Our current thoughts is focusing on enabling
an eco-system **around** core.  OpenStack shouldn't try to be IaaS,
PaaS and SaaS - instead a solid base to build these other systems on.
[1]

OpenStack is about Essential Infrastructure Services (currently
compute, storage, network) and supporting tools/apis/docs.
Determining if LB is considered Infrastructure (vs. platform) and if
it is Essential (a fuzzy word - what is essential to one isn't
essential to another)

That said - regardless of whether Atlas land in core [2], my team wants to add:

 * documentation/tutorials/examples about how to add a new (iaas or
paas) services to a cloud
 * simple integration of LB service (for instance an optional devstack
component).
 * an opensource backend for the LB service (haproxy, pound, ...)

The thought is that an entire eco-system of components that plug into
a cloud is more powerful than having OpenStack choose winners that
become core. [3]

I look forward to conversations about LBaaS and the definition of OpenStack.

Jesse Andrews
Rackspace Cloud Builders

[1] the analogy I use is that the Apache Web Server doesn't try to be
Django or Rails, but instead be a great web server to run rails on top
of.
[2] in addition to the question about if lbaas belongs in core, the
incubation process would need to be gone through
http://wiki.openstack.org/Governance/Approved/Incubation
[3] rather than blessing project X to be an official platform
component, enable many projects to run on top and let open source /
market dynamics determine winners.

On Mon, Feb 6, 2012 at 9:48 AM, Oleg Gelbukh ogelb...@mirantis.com wrote:
 Hello, everyone

 What is the status of this LBaaS project for the OpenStack? As far as I
 know, the open-source version is compatible with OpenStack. But is it
 possible to merge the Java code in the OpenStack ecosystem? Is someone
 working on re-implementing Atlas-LB in Python and eventually adding to the
 projects incubator, or there are some other lbaas projects out there?

 Thanks in advance,

 --
 Oleg Gelbukh
 Mirantis Inc.

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to     : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Joshua Harlow
Hi all,

Over the weekend I was thinking (I know a first, haha).

I was wondering if the community could elevate devstack to a official 
openstack project, instead of being a unofficial project. Since it seems like 
pretty much every developer (and even CI) is either depending on the shell 
script or the python script, so the unofficial wording seems incorrect. 
Hopefully we can have that happen and have this official project focus on 
just a developer setup script (imho the python version, since it fits in with 
the whole python model every other component is using and allows for features 
the shell script is not doing, multi-distro support, starting, stopping, 
uninstalling, object oriented design, to name a few...) of the openstack 
components (and not dive into the scripts that are showing up), ie leave that 
to 3rd party websites.

What does everyone think?

Maybe this can happen after essex?

-Josh
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Jesse Andrews
I think having a session on devstack at the summit would be valuable.

I'm still torn on it being python vs. bash because I see
non-developers using it all the time because they can take snippets of
the shell script and use it.

Jesse

On Mon, Feb 6, 2012 at 10:29 AM, Joshua Harlow harlo...@yahoo-inc.com wrote:
 Hi all,

 Over the weekend I was thinking (I know a first, haha).

 I was wondering if the community could elevate devstack to a official
 openstack project, instead of being a unofficial project. Since it seems
 like pretty much every developer (and even CI) is either depending on the
 shell script or the python script, so the unofficial wording seems
 incorrect. Hopefully we can have that happen and have this official
 project focus on just a developer setup script (imho the python version,
 since it fits in with the whole python model every other component is using
 and allows for features the shell script is not doing, multi-distro support,
 starting, stopping, uninstalling, object oriented design, to name a few...)
 of the openstack components (and not dive into the scripts that are showing
 up), ie leave that to 3rd party websites.

 What does everyone think?

 Maybe this can happen after essex?

 -Josh

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to     : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Duncan McGreggor
On 06 Feb 2012 - 10:29, Joshua Harlow wrote:
 Hi all,

 Over the weekend I was thinking (I know a first, haha).

 I was wondering if the community could elevate devstack to a
 official openstack project, instead of being a unofficial project.
 Since it seems like pretty much every developer (and even CI) is
 either depending on the shell script or the python script, so the
 unofficial wording seems incorrect. Hopefully we can have that happen
 and have this official project focus on just a developer setup
 script (imho the python version, since it fits in with the whole
 python model every other component is using and allows for features
 the shell script is not doing, multi-distro support, starting,
 stopping, uninstalling, object oriented design, to name a few...) of
 the openstack components (and not dive into the scripts that are
 showing up), ie leave that to 3rd party websites.

 What does everyone think?

+1

d

 Maybe this can happen after essex?

 -Josh

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Joshua Harlow
Interesting, what is the non-developer use case or the one u see as existing?

Can't they take snippets of python code as well ;)

On 2/6/12 10:37 AM, Jesse Andrews anotherje...@gmail.com wrote:

I think having a session on devstack at the summit would be valuable.

I'm still torn on it being python vs. bash because I see
non-developers using it all the time because they can take snippets of
the shell script and use it.

Jesse

On Mon, Feb 6, 2012 at 10:29 AM, Joshua Harlow harlo...@yahoo-inc.com wrote:
 Hi all,

 Over the weekend I was thinking (I know a first, haha).

 I was wondering if the community could elevate devstack to a official
 openstack project, instead of being a unofficial project. Since it seems
 like pretty much every developer (and even CI) is either depending on the
 shell script or the python script, so the unofficial wording seems
 incorrect. Hopefully we can have that happen and have this official
 project focus on just a developer setup script (imho the python version,
 since it fits in with the whole python model every other component is using
 and allows for features the shell script is not doing, multi-distro support,
 starting, stopping, uninstalling, object oriented design, to name a few...)
 of the openstack components (and not dive into the scripts that are showing
 up), ie leave that to 3rd party websites.

 What does everyone think?

 Maybe this can happen after essex?

 -Josh

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Sean Roberts
Hmm. Sounds like a good idea. I will reach out to the board and ask.

~sean

On Feb 6, 2012, at 10:34 AM, Joshua Harlow 
harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com wrote:

Hi all,

Over the weekend I was thinking (I know a first, haha).

I was wondering if the community could elevate devstack to a official 
openstack project, instead of being a unofficial project. Since it seems like 
pretty much every developer (and even CI) is either depending on the shell 
script or the python script, so the unofficial wording seems incorrect. 
Hopefully we can have that happen and have this official project focus on 
just a developer setup script (imho the python version, since it fits in with 
the whole python model every other component is using and allows for features 
the shell script is not doing, multi-distro support, starting, stopping, 
uninstalling, object oriented design, to name a few...) of the openstack 
components (and not dive into the scripts that are showing up), ie leave that 
to 3rd party websites.

What does everyone think?

Maybe this can happen after essex?

-Josh
___
Mailing list: https://launchpad.net/~openstack
Post to : 
openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] multiple floating ip pools

2012-02-06 Thread Xu (Simon) Chen
Hi all,

I am running devstack and got a dev instance of OpenStack running.

I am happy to see the concept of multiple floating IP pools, and the
per-floating-ip interface in the trunk, which I consider a very good basis
for my blueprint proposal here:
https://blueprints.launchpad.net/nova/+spec/multi-network-without-multi-nic

I have a quick question. Is there a plan (or maybe it's already there) for
access control whether a project is allowed to take floating IPs from a
pool?

Thanks!
-Simon
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Chmouel Boudjnah
On Mon, Feb 6, 2012 at 7:40 PM, Joshua Harlow harlo...@yahoo-inc.comwrote:

  Interesting, what is the non-developer use case or the one u see as
 existing?


This is my own personal opinion and not as commiter of devstack.sh,

I found very valuable to  point sysops to devstack.sh with the argument
it's just a shell script and you'll learn how openstack components are
getting plugged together. The argument being is that no matter they are
using automated deployment  like puppet/chef rpm/deb reading that well
documented shell script would actually make them understand how it works
under the hood and how to investigate problems if needed.

This argument can have less weight with that sort of audience when there is
a ''proper'' programming language involved which can maybe scared them away.

Chnmouel.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Joshua Harlow
Hmmm, I'm not sure I agree with that.
The its just a shell script argument only works when its pretty simple, 
setting up all these components is not simple. I think you are dumbing down 
your sysops to much :-)
I think that having a well documented python script can be just as easy to 
follow, if not easier (since the rest of openstack is in python). In the end 
the complexity of the system setup should really not rely on a documented piece 
of code, but should actually follow what the rest of openstack is doing (with a 
real set of documentation, ie a http://docs.openstack.org/ guide) imho. Having 
a shell script act as a run book seems pretty wrong in the end.

On 2/6/12 11:06 AM, Chmouel Boudjnah chmo...@openstack.org wrote:

On Mon, Feb 6, 2012 at 7:40 PM, Joshua Harlow harlo...@yahoo-inc.com wrote:
Interesting, what is the non-developer use case or the one u see as existing?

This is my own personal opinion and not as commiter of devstack.sh,

I found very valuable to  point sysops to devstack.sh with the argument it's 
just a shell script and you'll learn how openstack components are getting 
plugged together. The argument being is that no matter they are using automated 
deployment  like puppet/chef rpm/deb reading that well documented shell script 
would actually make them understand how it works under the hood and how to 
investigate problems if needed.

This argument can have less weight with that sort of audience when there is a 
''proper'' programming language involved which can maybe scared them away.

Chnmouel.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Swift and devstack.

2012-02-06 Thread Pete Zaitcev
On Mon, 6 Feb 2012 18:53:36 +0100
Chmouel Boudjnah chmo...@chmouel.com wrote:

 Another thing I want to implement is to add Swift as a back-end to Glance.

Wait, what is this then

[zaitcev@lembas glance-tip]$ ls glance/store/swift.py
glance/store/swift.py
[zaitcev@lembas glance-tip]$ cat .git/HEAD
ref: refs/heads/master

-- Pete

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Atlas-LB - what's the project status?

2012-02-06 Thread Gabriel Hurley
I would add that the Horizon team has been hard at work in the Essex release 
cycle to support this ecosystem concept as well. The Essex release has been 
completely re-architected for extensibility; any project can easily add their 
own dashboards, panels, etc. via reusable components. We'll be talking lots 
about that at the next summit.

Maintaining a solid core OpenStack is absolutely important, but it shouldn't 
be the bottleneck to successful projects.

- Gabriel

 -Original Message-
 From: openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net
 [mailto:openstack-
 bounces+gabriel.hurley=nebula@lists.launchpad.net] On Behalf Of
 Jesse Andrews
 Sent: Monday, February 06, 2012 10:13 AM
 To: Oleg Gelbukh
 Cc: openstack@lists.launchpad.net
 Subject: Re: [Openstack] Atlas-LB - what's the project status?
 
 Hi Oleg,
 
 NOTE: this is my opinion - I do not speak for all of OpenStack!
 
 While our focus is on a successful Essex, the RCB team has started thinking
 about Folsom.  Our current thoughts is focusing on enabling an eco-system
 **around** core.  OpenStack shouldn't try to be IaaS, PaaS and SaaS -
 instead a solid base to build these other systems on.
 [1]
 
 OpenStack is about Essential Infrastructure Services (currently compute,
 storage, network) and supporting tools/apis/docs.
 Determining if LB is considered Infrastructure (vs. platform) and if it is
 Essential (a fuzzy word - what is essential to one isn't essential to another)
 
 That said - regardless of whether Atlas land in core [2], my team wants to
 add:
 
  * documentation/tutorials/examples about how to add a new (iaas or
 paas) services to a cloud
  * simple integration of LB service (for instance an optional devstack
 component).
  * an opensource backend for the LB service (haproxy, pound, ...)
 
 The thought is that an entire eco-system of components that plug into a
 cloud is more powerful than having OpenStack choose winners that
 become core. [3]
 
 I look forward to conversations about LBaaS and the definition of OpenStack.
 
 Jesse Andrews
 Rackspace Cloud Builders
 
 [1] the analogy I use is that the Apache Web Server doesn't try to be Django
 or Rails, but instead be a great web server to run rails on top of.
 [2] in addition to the question about if lbaas belongs in core, the incubation
 process would need to be gone through
 http://wiki.openstack.org/Governance/Approved/Incubation
 [3] rather than blessing project X to be an official platform component,
 enable many projects to run on top and let open source / market dynamics
 determine winners.
 
 On Mon, Feb 6, 2012 at 9:48 AM, Oleg Gelbukh ogelb...@mirantis.com
 wrote:
  Hello, everyone
 
  What is the status of this LBaaS project for the OpenStack? As far as
  I know, the open-source version is compatible with OpenStack. But is
  it possible to merge the Java code in the OpenStack ecosystem? Is
  someone working on re-implementing Atlas-LB in Python and eventually
  adding to the projects incubator, or there are some other lbaas projects
 out there?
 
  Thanks in advance,
 
  --
  Oleg Gelbukh
  Mirantis Inc.
 
  ___
  Mailing list: https://launchpad.net/~openstack Post to     :
  openstack@lists.launchpad.net Unsubscribe :
  https://launchpad.net/~openstack More help   :
  https://help.launchpad.net/ListHelp
 
 
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Swift and devstack.

2012-02-06 Thread Chmouel Boudjnah
On 6 Feb 2012, at 19:33, Pete Zaitcev wrote:
 
 Another thing I want to implement is to add Swift as a back-end to Glance.
 Wait, what is this then
 [zaitcev@lembas glance-tip]$ ls glance/store/swift.py

My email may not have been very clear, I am talking about configuring swift and 
glance via devstack. Support is already there indeed.

Cheers,
Chmouel.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] Extensions, code vs. API

2012-02-06 Thread Andrew Bogott
I've just finished re-reading 'OpenStack API Extensions -- an 
overview.'  I understand the distinction between a core API and an API 
extension, and the value of that distinction is clear.


What I don't understand is how that distinction plays out when it 
comes to implementation.  Within our shared implementation, is there any 
actual coding distinction between 'core' and 'extension'?  For example, 
when I implemented the DNS extension I put most of the code in 
api/openstack/compute/contrib (clearly the place for 'extensions'.)  But 
to support the actual functionality I made changes in 
nova/network/manager.py and added a couple of db tables.  Everyone 
seemed fine with that.


Can I expect that any non-test-breaking code will be merged as long 
as it doesn't alter the core API, even if it's for an unpopular or 
obscure extension (e.g. puppet support)?  If not, how have other 
developers handled cases where adding a private API extension requires 
modifications of shared code?


Thanks!

-Andrew


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Atlas-LB - what's the project status?

2012-02-06 Thread Oleg Gelbukh
Jesse,

Thank you for quick answer and interesting information. Personally I like
the idea of multiple projects as ecosystem around OpenStack core.

--
Best regards,
Oleg Gelbukh
Mirantis Inc.

On Mon, Feb 6, 2012 at 10:13 PM, Jesse Andrews anotherje...@gmail.comwrote:

 Hi Oleg,

 NOTE: this is my opinion - I do not speak for all of OpenStack!

 While our focus is on a successful Essex, the RCB team has started
 thinking about Folsom.  Our current thoughts is focusing on enabling
 an eco-system **around** core.  OpenStack shouldn't try to be IaaS,
 PaaS and SaaS - instead a solid base to build these other systems on.
 [1]

 OpenStack is about Essential Infrastructure Services (currently
 compute, storage, network) and supporting tools/apis/docs.
 Determining if LB is considered Infrastructure (vs. platform) and if
 it is Essential (a fuzzy word - what is essential to one isn't
 essential to another)

 That said - regardless of whether Atlas land in core [2], my team wants to
 add:

  * documentation/tutorials/examples about how to add a new (iaas or
 paas) services to a cloud
  * simple integration of LB service (for instance an optional devstack
 component).
  * an opensource backend for the LB service (haproxy, pound, ...)

 The thought is that an entire eco-system of components that plug into
 a cloud is more powerful than having OpenStack choose winners that
 become core. [3]

 I look forward to conversations about LBaaS and the definition of
 OpenStack.

 Jesse Andrews
 Rackspace Cloud Builders

 [1] the analogy I use is that the Apache Web Server doesn't try to be
 Django or Rails, but instead be a great web server to run rails on top
 of.
 [2] in addition to the question about if lbaas belongs in core, the
 incubation process would need to be gone through
 http://wiki.openstack.org/Governance/Approved/Incubation
 [3] rather than blessing project X to be an official platform
 component, enable many projects to run on top and let open source /
 market dynamics determine winners.

 On Mon, Feb 6, 2012 at 9:48 AM, Oleg Gelbukh ogelb...@mirantis.com
 wrote:
  Hello, everyone
 
  What is the status of this LBaaS project for the OpenStack? As far as I
  know, the open-source version is compatible with OpenStack. But is it
  possible to merge the Java code in the OpenStack ecosystem? Is someone
  working on re-implementing Atlas-LB in Python and eventually adding to
 the
  projects incubator, or there are some other lbaas projects out there?
 
  Thanks in advance,
 
  --
  Oleg Gelbukh
  Mirantis Inc.
 
  ___
  Mailing list: https://launchpad.net/~openstack
  Post to : openstack@lists.launchpad.net
  Unsubscribe : https://launchpad.net/~openstack
  More help   : https://help.launchpad.net/ListHelp
 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Extensions, code vs. API

2012-02-06 Thread Vishvananda Ishaya
Yes some extensions modify core code. We are trying to strike a middle ground 
between ease-of-implementation and stability.  Stability of the API is vital, 
so public facing features definitely need to be extensions until we create a 
new version of the api. Internal stability is less important, so modifying core 
components to implement important extensions seems ok.

Eventually we might have to develop db-extensions and manager and driver 
plugins so you could ship all levels of an extension separately.  This would be 
very valuable, but it also significantly complicates the code base, so we have 
been avoiding generalizing the code so far.

Something to think about at the design summit.  I'm going to propose a 
discussion regarding shipping external drivers, and this seems like it fits 
right in with that.

Vish

On Feb 6, 2012, at 11:30 AM, Andrew Bogott wrote:

I've just finished re-reading 'OpenStack API Extensions -- an overview.'  
 I understand the distinction between a core API and an API extension, and the 
 value of that distinction is clear.
 
What I don't understand is how that distinction plays out when it comes to 
 implementation.  Within our shared implementation, is there any actual coding 
 distinction between 'core' and 'extension'?  For example, when I implemented 
 the DNS extension I put most of the code in api/openstack/compute/contrib 
 (clearly the place for 'extensions'.)  But to support the actual 
 functionality I made changes in nova/network/manager.py and added a couple of 
 db tables.  Everyone seemed fine with that.
 
Can I expect that any non-test-breaking code will be merged as long as it 
 doesn't alter the core API, even if it's for an unpopular or obscure 
 extension (e.g. puppet support)?  If not, how have other developers handled 
 cases where adding a private API extension requires modifications of shared 
 code?
 
 Thanks!
 
 -Andrew
 
 
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Christian Berendt
Hello together.

 I was wondering if the community could elevate devstack to a
 official openstack project, instead of being a unofficial
 project.

I think devstack.org is already some kind of official project (provided
by Rackspace Cloud Builders).

Where is the benefit of becoming a core project? At the moment I only
see a lot of overhead (release management, downstream distribution,
technical lead, feature frozen zones, ..) without any benefits.

Also it would take a lot of efforts (see [0] for details) to set up a
new core project.

Devstack is an instrument to help and improve the development. I think
a core component must have the opportunity to be used in a productive
environment and should not only be used to support the development.

Can you please describe in more detail what are the benefits of
becoming a core project?

An other point is that the official CI systems (and I think everybody
else, too) are using devstack.org and and that the script is doing a
well job.

You're starting two discussions in this mail: Should devstack become a
part of the core and should devstack be rewritten to Python. I think
the discussions should be splitted and I don't see any motivation of
the devstack.org developers to join the discussion of a Python rewrite
at the moment (maybe I'm wrong).

I don't find the definition and requirements of a core project at the
moment, but I'm pretty sure that there exist some documents.

Maybe it makes sense to define some kind of requirements about OpenStack
specific tools used by the official CI, but that's an other discussion.

[0] http://wiki.openstack.org/Governance/Approved/NewProjectProcess

Bye, Christian.

-- 
Christian Berendt
Linux / Unix Consultant  Developer
Mail: bere...@b1-systems.de

B1 Systems GmbH
Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de
GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB 3537

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] Nova command line versus Euca2ools

2012-02-06 Thread Lillie Ross-CDSR11
I currently have OpenStack installed (using the ManagedIT PPA) to use Keystone 
for authentication.  However I'm still receiving a number of Malformed request 
URL messages, both in Dashboard as well as when using the Nova command line 
client.  Also, some of the Euca2ools command run OK, others (such as 
euca-describe-images) don't.  I'm aware that not all the Euca commands are 
supported in Diablo, but currently I don't have any commands that let me launch 
instances.

For example:  euca-describe-availability-zones verbose yields

root@nova:~# euca-describe-availability-zones verbose
AVAILABILITYZONE nova available
AVAILABILITYZONE |- nova
AVAILABILITYZONE | |- nova-network enabled :-) 2012-02-06 22:15:16
AVAILABILITYZONE | |- nova-scheduler enabled :-) 2012-02-06 22:15:15
AVAILABILITYZONE | |- nova-vncproxy enabled :-) 2012-02-06 22:15:14
AVAILABILITYZONE | |- nova-compute enabled :-) 2012-02-06 22:15:07
AVAILABILITYZONE |- nova1
AVAILABILITYZONE | |- nova-compute enabled :-) 2012-02-06 22:15:08

however, euca-describe-images yields (with debug enabled)

root@nova:~# euca-describe-images --debug
2012-02-06 16:16:28,289 euca2ools [DEBUG]:Method: POST
2012-02-06 16:16:28,289 euca2ools [DEBUG]:Path: /services/Cloud/
2012-02-06 16:16:28,289 euca2ools [DEBUG]:Data:
2012-02-06 16:16:28,289 euca2ools [DEBUG]:Headers: {}
2012-02-06 16:16:28,290 euca2ools [DEBUG]:Host: 173.23.181.1:8773
2012-02-06 16:16:28,290 euca2ools [DEBUG]:establishing HTTP connection: 
kwargs={}
2012-02-06 16:16:28,290 euca2ools [DEBUG]:using _calc_signature_2
2012-02-06 16:16:28,290 euca2ools [DEBUG]:query string: 
AWSAccessKeyId=admin%3AadminAction=DescribeImagesSignatureMethod=HmacSHA256SignatureVersion=2Timestamp=2012-02-06T22%3A16%3A28ZVersion=2010-08-31
2012-02-06 16:16:28,290 euca2ools [DEBUG]:string_to_sign: POST
173.23.181.1:8773
/services/Cloud/
AWSAccessKeyId=admin%3AadminAction=DescribeImagesSignatureMethod=HmacSHA256SignatureVersion=2Timestamp=2012-02-06T22%3A16%3A28ZVersion=2010-08-31
2012-02-06 16:16:28,290 euca2ools [DEBUG]:len(b64)=44
2012-02-06 16:16:28,290 euca2ools [DEBUG]:base64 encoded digest: 
vSajubq/uXVIsFyMiUjxViprJ1zYHPpIONPcW5cN5yI=
2012-02-06 16:16:28,290 euca2ools [DEBUG]:query_string: 
AWSAccessKeyId=admin%3AadminAction=DescribeImagesSignatureMethod=HmacSHA256SignatureVersion=2Timestamp=2012-02-06T22%3A16%3A28ZVersion=2010-08-31
 Signature: vSajubq/uXVIsFyMiUjxViprJ1zYHPpIONPcW5cN5yI=
send: 'POST /services/Cloud/ HTTP/1.1\r\nHost: 
173.23.181.1:8773\r\nAccept-Encoding: identity\r\nContent-Length: 
207\r\nContent-Type: application/x-www-form-urlencoded; 
charset=UTF-8\r\nUser-Agent: Boto/2.0 
(linux2)\r\n\r\nAWSAccessKeyId=admin%3AadminAction=DescribeImagesSignatureMethod=HmacSHA256SignatureVersion=2Timestamp=2012-02-06T22%3A16%3A28ZVersion=2010-08-31Signature=vSajubq/uXVIsFyMiUjxViprJ1zYHPpIONPcW5cN5yI%3D'
reply: 'HTTP/1.1 400 Bad Request\r\n'
header: Content-Type: text/xml
header: Content-Length: 239
header: Date: Mon, 06 Feb 2012 22:16:28 GMT
2012-02-06 16:16:28,299 euca2ools [DEBUG]:?xml version=1.0?
ResponseErrorsErrorCodeUnknownError/CodeMessageAn unknown error has 
occurred. Please try your request 
again./Message/Error/ErrorsRequestIDb7f94f66-d309-4dcf-bc5f-c6aa5a09a83a/RequestID/Response
2012-02-06 16:16:28,299 euca2ools [ERROR]:400 Bad Request
2012-02-06 16:16:28,300 euca2ools [ERROR]:?xml version=1.0?
ResponseErrorsErrorCodeUnknownError/CodeMessageAn unknown error has 
occurred. Please try your request 
again./Message/Error/ErrorsRequestIDb7f94f66-d309-4dcf-bc5f-c6aa5a09a83a/RequestID/Response
UnknownError: An unknown error has occurred. Please try your request again.

The corresponding nova command yields the following (again with debug enabled)

root@nova:~# nova --debug image-list
connect: (173.23.181.1, 5000)
send: 'POST /v2.0/tokens HTTP/1.1\r\nHost: 173.23.181.1:5000\r\nContent-Length: 
100\r\ncontent-type: application/json\r\naccept-encoding: gzip, 
deflate\r\nuser-agent: python-novaclient\r\n\r\n{auth: {tenantName: 
admin, passwordCredentials: {username: admin, password: admin}}}'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: application/json; charset=UTF-8
header: Content-Length: 1007
header: Date: Mon, 06 Feb 2012 22:19:52 GMT
connect: (173.23.181.1, 8774)
send: u'GET /v1.1/1/images/detail HTTP/1.1\r\nHost: 
173.23.181.1:8774\r\nx-auth-project-id: admin\r\nx-auth-token: 
10111213141516171819\r\naccept-encoding: gzip, deflate\r\nuser-agent: 
python-novaclient\r\n\r\n'
reply: 'HTTP/1.1 400 Bad Request\r\n'
header: Content-Length: 65
header: Content-Type: application/json; charset=UTF-8
header: Date: Mon, 06 Feb 2012 22:19:52 GMT
Traceback (most recent call last):
  File /usr/bin/nova, line 9, in module
load_entry_point('python-novaclient==2012.1', 'console_scripts', 'nova')()
  File /usr/lib/python2.7/dist-packages/novaclient/shell.py, line 353, in main
OpenStackComputeShell().main(sys.argv[1:])
  File /usr/lib/python2.7/dist-packages/novaclient/shell.py, line 304, in main

[Openstack] Propose to make Monsyne Dragon a nova core developer

2012-02-06 Thread Matt Dietz
Hey guys,

Dragon has really stepped up lately on reviewing patches into Nova, and has a 
ton of knowledge around Nova proper, so I propose he be added to Nova core. I 
think he'd be a great addition to the team.

Matt
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Monty Taylor
I think the thing you are discussing already exists.

devstack is currently part of and managed by all of the normal OpenStack
development infrastructure. The canonical repository for it is
https://review.openstack.org/p/openstack-dev/devstack which is mirrored
to https://github.com/openstack-dev/devstack. Every change to OpenStack
is not only gated on devstack properly functioning, every change to
devstack is gated on OpenStack properly functioning.

Additionally, branches match up, so there is a stable/diablo that works
with stable/diablo of all of the OpenStack branches and is a part of
their trunk gating.

In that sense, it's actually the first install OpenStack method that
_is_ fully a part of OpenStack - even though there are also chef recipes
and puppet modules in OpenStack's gerrit as well. (although at some
point I wouldn't mind getting some installation testing and gating on
them as well)

So it's pretty official already.

However, as to becoming an official project - it's a developer tool,
same as git-review or gerrit or the openstack nose-plugin. It's
something that's useful for developers for developing and testing
OpenStack. It is not, nor is it meant to be, part of the software we
ship -- which is the current definition of what it means to be a
core project. i.e. - If I'm a deployer and I want to install
OpenStack - is this one of the things I install? With devstack - the
answer is no.

Is is MASSIVELY helpful and a part of everyday life for all of us?
ABSOLUTELY (this is why we have to be careful with changes to it and run
them through the same process everything else gets)

All of that to say - I agree with you, and it's already done. :)

Monty

On 02/06/2012 01:43 PM, Joshua Harlow wrote:
 So the part that worries me about what u just said is the part about “it
 is already some kind of official project”.
 When you have to question whether a project is official or not, that
 seems to pretty make the whole point for making it official ;)
 
 Overall though I think what u are saying is correct, but the overhead I
 don’t see as being a bad thing.
 
 In my idea release management is good since it allows developers to be
 able to setup a development environment for a given openstack release
 (good for when you need to fix bugs against a given release as well as
 good for providing a stable point for other distributions to know what
 goes in a release and what configs need to be adjusted to make that
 release work for all the different components). So I don’t see that as a
 drawback (even though yes it does add work/overhead in, but I don’t see
 that as a valid point, in any case).
 
 Downstream distribution, I am not exactly sure what you mean here?
 
 A technical lead I think is something good to have, as this
 script/code/documentation is not as simple as you might think (and most
 likely won’t get any simpler).
 
 Maybe the correct wording isn’t that this is a core project, but it
 seems like it is already a widely used project, so I don’t see the
 difference, either way it should become official and follow some of the
 same processes as the rest of openstack. Yes it might be developer
 oriented but if that doesn’t fit a definition of a core project (or
 whatever u want to call it), because of it being developer focused, then
 maybe the core project definition needs to be updated?
 
 As for:
 
 An other point is that the official CI systems (and I think
 everybody else, too) are using devstack.org and and that the script
 is doing a well job.
 
 
 That’s the whole point, a un-official script shouldn’t be doing these
 tasks ;)
 
 -Josh
 
 On 2/6/12 12:36 PM, Christian Berendt bere...@b1-systems.de wrote:
 
 Hello together.
 
  I was wondering if the community could elevate devstack to a
  official openstack project, instead of being a unofficial
  project.
 
 I think devstack.org is already some kind of official project (provided
 by Rackspace Cloud Builders).
 
 Where is the benefit of becoming a core project? At the moment I only
 see a lot of overhead (release management, downstream distribution,
 technical lead, feature frozen zones, ..) without any benefits.
 
 Also it would take a lot of efforts (see [0] for details) to set up a
 new core project.
 
 Devstack is an instrument to help and improve the development. I think
 a core component must have the opportunity to be used in a productive
 environment and should not only be used to support the development.
 
 Can you please describe in more detail what are the benefits of
 becoming a core project?
 
 An other point is that the official CI systems (and I think everybody
 else, too) are using devstack.org and and that the script is doing a
 well job.
 
 You're starting two discussions in this mail: Should devstack become a
 part of the core and should devstack be rewritten to Python. I think
 the discussions should be splitted and I don't 

Re: [Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Monty Taylor


On 02/06/2012 10:37 AM, Jesse Andrews wrote:
 I think having a session on devstack at the summit would be valuable.

++

 I'm still torn on it being python vs. bash because I see
 non-developers using it all the time because they can take snippets of
 the shell script and use it.


I'm similarly torn. One of the things Jesse and I have spoke of several
times is in making sure that we don't let the scope of devstack grow to
the point where we re-invent chef or puppet or juju. It's not really
intended as a tool for doing real deployments - it's intended to make
developer's lives easier, and to serve as an easy to follow living
document of what a base working openstack installation looks like.

My concern is that if we go and get all software developer on it and
make it python and right and better - that it will cease to be an
easy to follow annotated installation and start to become a
custom-written multi-purpose deployment software.

Monty

 On Mon, Feb 6, 2012 at 10:29 AM, Joshua Harlow harlo...@yahoo-inc.com wrote:
 Hi all,

 Over the weekend I was thinking (I know a first, haha).

 I was wondering if the community could elevate devstack to a official
 openstack project, instead of being a unofficial project. Since it seems
 like pretty much every developer (and even CI) is either depending on the
 shell script or the python script, so the unofficial wording seems
 incorrect. Hopefully we can have that happen and have this official
 project focus on just a developer setup script (imho the python version,
 since it fits in with the whole python model every other component is using
 and allows for features the shell script is not doing, multi-distro support,
 starting, stopping, uninstalling, object oriented design, to name a few)
 of the openstack components (and not dive into the scripts that are showing
 up), ie leave that to 3rd party websites.

 What does everyone think?

 Maybe this can happen after essex?

 -Josh

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp

 
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp
 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Jesse Andrews
Our team had discussions on python vs. bash at the beginning of devstack.

If we go with python it would be nice to do something similar to
python-novaclient where setting DEBUG lets users see CURL commands
that you can execute to do the same thing.

That said, we are focused on essex until the design summit...  We look
forward to reviewing ideas about devstack after essex (including
installing atlas/reddwarf/other-service-here  python - our team uses
fabric to do multi-node devstack deploys and so python would be nice
:)

Jesse

On Mon, Feb 6, 2012 at 3:40 PM, Monty Taylor mord...@inaugust.com wrote:


 On 02/06/2012 10:37 AM, Jesse Andrews wrote:
 I think having a session on devstack at the summit would be valuable.

 ++

 I'm still torn on it being python vs. bash because I see
 non-developers using it all the time because they can take snippets of
 the shell script and use it.


 I'm similarly torn. One of the things Jesse and I have spoke of several
 times is in making sure that we don't let the scope of devstack grow to
 the point where we re-invent chef or puppet or juju. It's not really
 intended as a tool for doing real deployments - it's intended to make
 developer's lives easier, and to serve as an easy to follow living
 document of what a base working openstack installation looks like.

 My concern is that if we go and get all software developer on it and
 make it python and right and better - that it will cease to be an
 easy to follow annotated installation and start to become a
 custom-written multi-purpose deployment software.

 Monty

 On Mon, Feb 6, 2012 at 10:29 AM, Joshua Harlow harlo...@yahoo-inc.com 
 wrote:
 Hi all,

 Over the weekend I was thinking (I know a first, haha).

 I was wondering if the community could elevate devstack to a official
 openstack project, instead of being a unofficial project. Since it seems
 like pretty much every developer (and even CI) is either depending on the
 shell script or the python script, so the unofficial wording seems
 incorrect. Hopefully we can have that happen and have this official
 project focus on just a developer setup script (imho the python version,
 since it fits in with the whole python model every other component is using
 and allows for features the shell script is not doing, multi-distro support,
 starting, stopping, uninstalling, object oriented design, to name a few)
 of the openstack components (and not dive into the scripts that are showing
 up), ie leave that to 3rd party websites.

 What does everyone think?

 Maybe this can happen after essex?

 -Josh

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to     : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


 ___
 Mailing list: https://launchpad.net/~openstack
 Post to     : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Joshua Harlow
Sure I think CURL commands are the least of the problems though.

Just getting this devstack python version running on multiple distributions 
is a pain enough (pkg dependencies, conf files are different...)

That's one of the other benefits I see, the current stack.sh would turn pretty 
ugly real quick if more than 1 distro ever got supported.

Unless there is some object oriented bash I am missing :-P

On 2/6/12 3:57 PM, Jesse Andrews anotherje...@gmail.com wrote:

Our team had discussions on python vs. bash at the beginning of devstack.

If we go with python it would be nice to do something similar to
python-novaclient where setting DEBUG lets users see CURL commands
that you can execute to do the same thing.

That said, we are focused on essex until the design summit...  We look
forward to reviewing ideas about devstack after essex (including
installing atlas/reddwarf/other-service-here  python - our team uses
fabric to do multi-node devstack deploys and so python would be nice
:)

Jesse

On Mon, Feb 6, 2012 at 3:40 PM, Monty Taylor mord...@inaugust.com wrote:


 On 02/06/2012 10:37 AM, Jesse Andrews wrote:
 I think having a session on devstack at the summit would be valuable.

 ++

 I'm still torn on it being python vs. bash because I see
 non-developers using it all the time because they can take snippets of
 the shell script and use it.


 I'm similarly torn. One of the things Jesse and I have spoke of several
 times is in making sure that we don't let the scope of devstack grow to
 the point where we re-invent chef or puppet or juju. It's not really
 intended as a tool for doing real deployments - it's intended to make
 developer's lives easier, and to serve as an easy to follow living
 document of what a base working openstack installation looks like.

 My concern is that if we go and get all software developer on it and
 make it python and right and better - that it will cease to be an
 easy to follow annotated installation and start to become a
 custom-written multi-purpose deployment software.

 Monty

 On Mon, Feb 6, 2012 at 10:29 AM, Joshua Harlow harlo...@yahoo-inc.com 
 wrote:
 Hi all,

 Over the weekend I was thinking (I know a first, haha).

 I was wondering if the community could elevate devstack to a official
 openstack project, instead of being a unofficial project. Since it seems
 like pretty much every developer (and even CI) is either depending on the
 shell script or the python script, so the unofficial wording seems
 incorrect. Hopefully we can have that happen and have this official
 project focus on just a developer setup script (imho the python version,
 since it fits in with the whole python model every other component is using
 and allows for features the shell script is not doing, multi-distro support,
 starting, stopping, uninstalling, object oriented design, to name a few)
 of the openstack components (and not dive into the scripts that are showing
 up), ie leave that to 3rd party websites.

 What does everyone think?

 Maybe this can happen after essex?

 -Josh

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Jay Pipes
cc'ing Matt Ray from OpsCode, since he and I discussed related topics 
this past Thursday during the bug squash day...


On 02/06/2012 06:35 PM, Monty Taylor wrote:

I think the thing you are discussing already exists.

devstack is currently part of and managed by all of the normal OpenStack
development infrastructure. The canonical repository for it is
https://review.openstack.org/p/openstack-dev/devstack which is mirrored
to https://github.com/openstack-dev/devstack. Every change to OpenStack
is not only gated on devstack properly functioning, every change to
devstack is gated on OpenStack properly functioning.

Additionally, branches match up, so there is a stable/diablo that works
with stable/diablo of all of the OpenStack branches and is a part of
their trunk gating.


This is a critical piece of the puzzle. If I want a Diablo install for 
testing, all I need to do is:


cd $devstack_dir
git checkout stable/diablo
rm -rf /opt/stack
./stack.sh

And I get a Diablo installation of OpenStack. Likewise, if I want a 
development (Essex currently) version of OpenStack, I just do:


cd $devstack_dir
git checkout master
rm -rf /opt/stack
./stack.sh

And I get a development installation of OpenStack.

Now, I'm not entirely sure I even need to do the rm -rf /opt/stack part, 
but I do that for good measure, even if it does mean it takes a little 
longer... ;)


This is not something I can do currently with the other deployment methods.


In that sense, it's actually the first install OpenStack method that
_is_ fully a part of OpenStack - even though there are also chef recipes
and puppet modules in OpenStack's gerrit as well. (although at some
point I wouldn't mind getting some installation testing and gating on
them as well)


Yes, and getting those projects aligned with the core projects' branch 
layout would be good, too. Followup email on the Chef stuff coming 
shortly, as Matt ray and I discussed this last Thursday at length and I 
think there's a lot we can do to improve things.


-jay


So it's pretty official already.

However, as to becoming an official project - it's a developer tool,
same as git-review or gerrit or the openstack nose-plugin. It's
something that's useful for developers for developing and testing
OpenStack. It is not, nor is it meant to be, part of the software we
ship -- which is the current definition of what it means to be a
core project. i.e. - If I'm a deployer and I want to install
OpenStack - is this one of the things I install? With devstack - the
answer is no.

Is is MASSIVELY helpful and a part of everyday life for all of us?
ABSOLUTELY (this is why we have to be careful with changes to it and run
them through the same process everything else gets)

All of that to say - I agree with you, and it's already done. :)

Monty

On 02/06/2012 01:43 PM, Joshua Harlow wrote:

So the part that worries me about what u just said is the part about “it
is already some kind of official project”.
When you have to question whether a project is official or not, that
seems to pretty make the whole point for making it official ;)

Overall though I think what u are saying is correct, but the overhead I
don’t see as being a bad thing.

In my idea release management is good since it allows developers to be
able to setup a development environment for a given openstack release
(good for when you need to fix bugs against a given release as well as
good for providing a stable point for other distributions to know what
goes in a release and what configs need to be adjusted to make that
release work for all the different components). So I don’t see that as a
drawback (even though yes it does add work/overhead in, but I don’t see
that as a valid point, in any case).

Downstream distribution, I am not exactly sure what you mean here?

A technical lead I think is something good to have, as this
script/code/documentation is not as simple as you might think (and most
likely won’t get any simpler).

Maybe the correct wording isn’t that this is a core project, but it
seems like it is already a widely used project, so I don’t see the
difference, either way it should become official and follow some of the
same processes as the rest of openstack. Yes it might be developer
oriented but if that doesn’t fit a definition of a core project (or
whatever u want to call it), because of it being developer focused, then
maybe the core project definition needs to be updated?

As for:

 An other point is that the official CI systems (and I think
 everybody else, too) are using devstack.org and and that the script
 is doing a well job.


That’s the whole point, a un-official script shouldn’t be doing these
tasks ;)

-Josh

On 2/6/12 12:36 PM, Christian Berendtbere...@b1-systems.de  wrote:

 Hello together.

   I was wondering if the community could elevate devstack to a
   official openstack project, instead of being a unofficial
   project.

 I think devstack.org is already some kind of official 

Re: [Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Joshua Harlow
+ There needs to be a way to install on multiple distributions (without saying 
go figure out the deps yourself).

I know everyone is ubuntu, ubuntu, ubuntu, but this really needs to be fixed 
(process wise as well).

:-/

On 2/6/12 5:12 PM, Jay Pipes jaypi...@gmail.com wrote:

cc'ing Matt Ray from OpsCode, since he and I discussed related topics
this past Thursday during the bug squash day...

On 02/06/2012 06:35 PM, Monty Taylor wrote:
 I think the thing you are discussing already exists.

 devstack is currently part of and managed by all of the normal OpenStack
 development infrastructure. The canonical repository for it is
 https://review.openstack.org/p/openstack-dev/devstack which is mirrored
 to https://github.com/openstack-dev/devstack. Every change to OpenStack
 is not only gated on devstack properly functioning, every change to
 devstack is gated on OpenStack properly functioning.

 Additionally, branches match up, so there is a stable/diablo that works
 with stable/diablo of all of the OpenStack branches and is a part of
 their trunk gating.

This is a critical piece of the puzzle. If I want a Diablo install for
testing, all I need to do is:

cd $devstack_dir
git checkout stable/diablo
rm -rf /opt/stack
./stack.sh

And I get a Diablo installation of OpenStack. Likewise, if I want a
development (Essex currently) version of OpenStack, I just do:

cd $devstack_dir
git checkout master
rm -rf /opt/stack
./stack.sh

And I get a development installation of OpenStack.

Now, I'm not entirely sure I even need to do the rm -rf /opt/stack part,
but I do that for good measure, even if it does mean it takes a little
longer... ;)

This is not something I can do currently with the other deployment methods.

 In that sense, it's actually the first install OpenStack method that
 _is_ fully a part of OpenStack - even though there are also chef recipes
 and puppet modules in OpenStack's gerrit as well. (although at some
 point I wouldn't mind getting some installation testing and gating on
 them as well)

Yes, and getting those projects aligned with the core projects' branch
layout would be good, too. Followup email on the Chef stuff coming
shortly, as Matt ray and I discussed this last Thursday at length and I
think there's a lot we can do to improve things.

-jay

 So it's pretty official already.

 However, as to becoming an official project - it's a developer tool,
 same as git-review or gerrit or the openstack nose-plugin. It's
 something that's useful for developers for developing and testing
 OpenStack. It is not, nor is it meant to be, part of the software we
 ship -- which is the current definition of what it means to be a
 core project. i.e. - If I'm a deployer and I want to install
 OpenStack - is this one of the things I install? With devstack - the
 answer is no.

 Is is MASSIVELY helpful and a part of everyday life for all of us?
 ABSOLUTELY (this is why we have to be careful with changes to it and run
 them through the same process everything else gets)

 All of that to say - I agree with you, and it's already done. :)

 Monty

 On 02/06/2012 01:43 PM, Joshua Harlow wrote:
 So the part that worries me about what u just said is the part about it
 is already some kind of official project.
 When you have to question whether a project is official or not, that
 seems to pretty make the whole point for making it official ;)

 Overall though I think what u are saying is correct, but the overhead I
 don't see as being a bad thing.

 In my idea release management is good since it allows developers to be
 able to setup a development environment for a given openstack release
 (good for when you need to fix bugs against a given release as well as
 good for providing a stable point for other distributions to know what
 goes in a release and what configs need to be adjusted to make that
 release work for all the different components). So I don't see that as a
 drawback (even though yes it does add work/overhead in, but I don't see
 that as a valid point, in any case).

 Downstream distribution, I am not exactly sure what you mean here?

 A technical lead I think is something good to have, as this
 script/code/documentation is not as simple as you might think (and most
 likely won't get any simpler).

 Maybe the correct wording isn't that this is a core project, but it
 seems like it is already a widely used project, so I don't see the
 difference, either way it should become official and follow some of the
 same processes as the rest of openstack. Yes it might be developer
 oriented but if that doesn't fit a definition of a core project (or
 whatever u want to call it), because of it being developer focused, then
 maybe the core project definition needs to be updated?

 As for:

  An other point is that the official CI systems (and I think
  everybody else, too) are using devstack.org and and that the script
  is doing a well job.


 That's the whole point, a un-official script shouldn't be doing 

[Openstack] [CHEF] Aligning Cookbook Efforts

2012-02-06 Thread Jay Pipes

Hi Stackers,

tl;dr
-

There are myriad Chef cookbooks out there in the ecosystem and locked 
up behind various company firewalls. It would be awesome if we could 
agree to:


* Align to a single origin repository for OpenStack cookbooks
* Consolidate OpenStack Chef-based deployment experience into a single 
knowledge base
* Have branches on the origin OpenStack cookbooks repository that align 
with core OpenStack projects
* Automate the validation and testing of these cookbooks on multiple 
supported versions of the OpenStack code base


Details
---

Current State of Forks
==

Matt Ray and I tried to outline the current state of the various 
OpenStack Chef cookbooks this past Thursday, and we came up with the 
following state of affairs:


** The official OpenStack Chef cookbooks **

https://github.com/openstack/openstack-chef

These chef cookbooks are the ones maintained mostly by Dan Prince and 
Brian Lamar and these are the cookbooks used by the SmokeStack project. 
The cookbooks contained in the above repo can install all the core 
OpenStack projects with the exception of Swift and Horizon.


This repo is controlled by the Gerrit instance at review.openstack.org 
just like other core OpenStack projects.


However, these cookbooks DO NOT currently have a stable/diablo branch -- 
they are updated when the development trunks of any OpenStack project 
merges a commit that requires deployment or configuration-related 
changes to their associated cookbook.


Important note: it's easy for Dan and Brian to know when updates to 
these cookbooks are necessary -- SmokeStack will bomb out if a 
deployment-affecting configuration change hits a core project trunk :)


These cookbooks are the ONLY cookbooks that contain stuff for deploying 
with XenServer, AFAICT.


** NTT PF Lab Diablo Chef cookbooks **

https://github.com/ntt-pf-lab/openstack-chef/

So, NTT PF Lab forked the upstream Chef cookbooks back in Nov 11, 2011, 
because they needed a set of Chef cookbooks for OpenStack that 
functioned for the Diablo code base.


While Nov 11, 2011, is not the *exact* date of the Diablo release, these 
cookbooks do in fact work for a Diablo install -- Nati Ueno is using 
them for the FreeCloud deployment so we know they work...


** OpsCode OpenStack Chef Cookbooks **

Matt Ray from OpsCode created a set of cookbooks for OpenStack for the 
Cactus release of OpenStack:


https://github.com/mattray/openstack-cookbooks
http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef

These cookbooks were forked from the Anso Labs' original OpenStack 
cookbooks from the Bexar release and were the basis for the Chef work 
that Dell did for Crowbar. Crowbar was originally based on Cactus, and 
according to Matt, the repositories of OpenStack cookbooks that OpsCode 
houses internally and uses most often are Cactus-based cookbooks. (Matt, 
please correct me if I am wrong here...)


** Rackspace CloudBuilders OpenStack Chef Cookbooks **

The RCB team also has a repository of OpenStack Chef cookbooks:

https://github.com/cloudbuilders/openstack-cookbooks

Now, GitHub *says* that these cookbooks were forked from the official 
upstream cookbooks, but I do not think that is correct. Looking at this 
repo, I believe that this repo was *actually* forked from the Anso Labs 
OpenStack Chef Cookbooks, as the list of cookbooks is virtually identical.


** Anso Labs OpenStack Chef Cookbooks **

These older cookbooks are in this repo:

https://github.com/ansolabs/openstack-cookbooks/tree/master/cookbooks

Interestingly, this repo DOES contain a cookbook for Swift.

Current State of Documentation
==

Documentation for best practices on using Chef for your OpenStack 
deployments is, well, a bit scattered. Matt Ray has some good 
information on the README on his cookbook repo and the OpsCode wiki:


https://github.com/mattray/openstack-cookbooks/blob/cactus/README.md
http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef

But it is unfortunately not going to help people looking to deploy 
Diablo and later versions of OpenStack.


Most of the other repos contain virtually no documentation on using the 
cookbooks or how they are written.


I have a suspicion that one of the reasons that there has been such a 
proliferation of cookbooks has been the lack of documentation pointing 
people to an appropriate repo, how to use the cookbooks properly, and 
what the best practices for deployment are. That, and the fact that 
folks are just trying to stand up complex clouds and Get Things Done, 
and documentation is annoying to write ;)


Proposal for Alignment
==

I think the following steps would be good to get done by the time Essex 
rolls out the door in April:


1) Create a stable/diablo branch of the openstack/openstack-chef 
cookbook repo and maintain it in the same way that we maintain stable 
branches for core OpenStack projects. I propose we use the branch 

Re: [Openstack] [CHEF] Aligning Cookbook Efforts

2012-02-06 Thread Mark Collier
Awesome leadership on this Jay! (And Matt) You should come down to Austin more 
often :)

Definitely seems like an area with a lot of duplicated effort.


Jay Pipes jaypi...@gmail.com wrote:


Hi Stackers,

tl;dr
-

There are myriad Chef cookbooks out there in the ecosystem and locked
up behind various company firewalls. It would be awesome if we could
agree to:

* Align to a single origin repository for OpenStack cookbooks
* Consolidate OpenStack Chef-based deployment experience into a single
knowledge base
* Have branches on the origin OpenStack cookbooks repository that align
with core OpenStack projects
* Automate the validation and testing of these cookbooks on multiple
supported versions of the OpenStack code base

Details
---

Current State of Forks
==

Matt Ray and I tried to outline the current state of the various
OpenStack Chef cookbooks this past Thursday, and we came up with the
following state of affairs:

** The official OpenStack Chef cookbooks **

https://github.com/openstack/openstack-chef

These chef cookbooks are the ones maintained mostly by Dan Prince and
Brian Lamar and these are the cookbooks used by the SmokeStack project.
The cookbooks contained in the above repo can install all the core
OpenStack projects with the exception of Swift and Horizon.

This repo is controlled by the Gerrit instance at review.openstack.org
just like other core OpenStack projects.

However, these cookbooks DO NOT currently have a stable/diablo branch --
they are updated when the development trunks of any OpenStack project
merges a commit that requires deployment or configuration-related
changes to their associated cookbook.

Important note: it's easy for Dan and Brian to know when updates to
these cookbooks are necessary -- SmokeStack will bomb out if a
deployment-affecting configuration change hits a core project trunk :)

These cookbooks are the ONLY cookbooks that contain stuff for deploying
with XenServer, AFAICT.

** NTT PF Lab Diablo Chef cookbooks **

https://github.com/ntt-pf-lab/openstack-chef/

So, NTT PF Lab forked the upstream Chef cookbooks back in Nov 11, 2011,
because they needed a set of Chef cookbooks for OpenStack that
functioned for the Diablo code base.

While Nov 11, 2011, is not the *exact* date of the Diablo release, these
cookbooks do in fact work for a Diablo install -- Nati Ueno is using
them for the FreeCloud deployment so we know they work...

** OpsCode OpenStack Chef Cookbooks **

Matt Ray from OpsCode created a set of cookbooks for OpenStack for the
Cactus release of OpenStack:

https://github.com/mattray/openstack-cookbooks
http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef

These cookbooks were forked from the Anso Labs' original OpenStack
cookbooks from the Bexar release and were the basis for the Chef work
that Dell did for Crowbar. Crowbar was originally based on Cactus, and
according to Matt, the repositories of OpenStack cookbooks that OpsCode
houses internally and uses most often are Cactus-based cookbooks. (Matt,
please correct me if I am wrong here...)

** Rackspace CloudBuilders OpenStack Chef Cookbooks **

The RCB team also has a repository of OpenStack Chef cookbooks:

https://github.com/cloudbuilders/openstack-cookbooks

Now, GitHub *says* that these cookbooks were forked from the official
upstream cookbooks, but I do not think that is correct. Looking at this
repo, I believe that this repo was *actually* forked from the Anso Labs
OpenStack Chef Cookbooks, as the list of cookbooks is virtually identical.

** Anso Labs OpenStack Chef Cookbooks **

These older cookbooks are in this repo:

https://github.com/ansolabs/openstack-cookbooks/tree/master/cookbooks

Interestingly, this repo DOES contain a cookbook for Swift.

Current State of Documentation
==

Documentation for best practices on using Chef for your OpenStack
deployments is, well, a bit scattered. Matt Ray has some good
information on the README on his cookbook repo and the OpsCode wiki:

https://github.com/mattray/openstack-cookbooks/blob/cactus/README.md
http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef

But it is unfortunately not going to help people looking to deploy
Diablo and later versions of OpenStack.

Most of the other repos contain virtually no documentation on using the
cookbooks or how they are written.

I have a suspicion that one of the reasons that there has been such a
proliferation of cookbooks has been the lack of documentation pointing
people to an appropriate repo, how to use the cookbooks properly, and
what the best practices for deployment are. That, and the fact that
folks are just trying to stand up complex clouds and Get Things Done,
and documentation is annoying to write ;)

Proposal for Alignment
==

I think the following steps would be good to get done by the time Essex
rolls out the door in April:

1) Create a stable/diablo branch of the openstack/openstack-chef

Re: [Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Jesse Andrews
Agreed.  Supporting more than just ubuntu is important!

On Mon, Feb 6, 2012 at 5:22 PM, Joshua Harlow harlo...@yahoo-inc.com wrote:
 + There needs to be a way to install on multiple distributions (without
 saying go figure out the deps yourself).

 I know everyone is ubuntu, ubuntu, ubuntu, but this really needs to be fixed
 (process wise as well).

 :-/


 On 2/6/12 5:12 PM, Jay Pipes jaypi...@gmail.com wrote:

 cc'ing Matt Ray from OpsCode, since he and I discussed related topics
 this past Thursday during the bug squash day...

 On 02/06/2012 06:35 PM, Monty Taylor wrote:
 I think the thing you are discussing already exists.

 devstack is currently part of and managed by all of the normal OpenStack
 development infrastructure. The canonical repository for it is
 https://review.openstack.org/p/openstack-dev/devstack which is mirrored
 to https://github.com/openstack-dev/devstack. Every change to OpenStack
 is not only gated on devstack properly functioning, every change to
 devstack is gated on OpenStack properly functioning.

 Additionally, branches match up, so there is a stable/diablo that works
 with stable/diablo of all of the OpenStack branches and is a part of
 their trunk gating.

 This is a critical piece of the puzzle. If I want a Diablo install for
 testing, all I need to do is:

 cd $devstack_dir
 git checkout stable/diablo
 rm -rf /opt/stack
 ./stack.sh

 And I get a Diablo installation of OpenStack. Likewise, if I want a
 development (Essex currently) version of OpenStack, I just do:

 cd $devstack_dir
 git checkout master
 rm -rf /opt/stack
 ./stack.sh

 And I get a development installation of OpenStack.

 Now, I'm not entirely sure I even need to do the rm -rf /opt/stack part,
 but I do that for good measure, even if it does mean it takes a little
 longer... ;)

 This is not something I can do currently with the other deployment methods.

 In that sense, it's actually the first install OpenStack method that
 _is_ fully a part of OpenStack - even though there are also chef recipes
 and puppet modules in OpenStack's gerrit as well. (although at some
 point I wouldn't mind getting some installation testing and gating on
 them as well)

 Yes, and getting those projects aligned with the core projects' branch
 layout would be good, too. Followup email on the Chef stuff coming
 shortly, as Matt ray and I discussed this last Thursday at length and I
 think there's a lot we can do to improve things.

 -jay

 So it's pretty official already.

 However, as to becoming an official project - it's a developer tool,
 same as git-review or gerrit or the openstack nose-plugin. It's
 something that's useful for developers for developing and testing
 OpenStack. It is not, nor is it meant to be, part of the software we
 ship -- which is the current definition of what it means to be a
 core project. i.e. - If I'm a deployer and I want to install
 OpenStack - is this one of the things I install? With devstack - the
 answer is no.

 Is is MASSIVELY helpful and a part of everyday life for all of us?
 ABSOLUTELY (this is why we have to be careful with changes to it and run
 them through the same process everything else gets)

 All of that to say - I agree with you, and it's already done. :)

 Monty

 On 02/06/2012 01:43 PM, Joshua Harlow wrote:
 So the part that worries me about what u just said is the part about “it
 is already some kind of official project”.
 When you have to question whether a project is official or not, that
 seems to pretty make the whole point for making it official ;)

 Overall though I think what u are saying is correct, but the overhead I
 don’t see as being a bad thing.

 In my idea release management is good since it allows developers to be
 able to setup a development environment for a given openstack release
 (good for when you need to fix bugs against a given release as well as
 good for providing a stable point for other distributions to know what
 goes in a release and what configs need to be adjusted to make that
 release work for all the different components). So I don’t see that as a
 drawback (even though yes it does add work/overhead in, but I don’t see
 that as a valid point, in any case).

 Downstream distribution, I am not exactly sure what you mean here?

 A technical lead I think is something good to have, as this
 script/code/documentation is not as simple as you might think (and most
 likely won’t get any simpler).

 Maybe the correct wording isn’t that this is a core project, but it
 seems like it is already a widely used project, so I don’t see the
 difference, either way it should become official and follow some of the
 same processes as the rest of openstack. Yes it might be developer
 oriented but if that doesn’t fit a definition of a core project (or
 whatever u want to call it), because of it being developer focused, then
 maybe the core project definition needs to be updated?

 As for:

  An other point is that the official CI systems (and I think
  

[Openstack] confused about libvirt nwfilter and iptables rules

2012-02-06 Thread heut2008
hi,all:
  I am confued about how security  rules works ,i read the
 /nova/virt/libvirt/firewall.py  and /nova/network/linux_net.py ,
my understanding is when create or change a  security  rule ,the process is
as below.
reuqest to  nova osapi-update db  for the rule-call method
 trigger_security_group_rules_refresh()-rpc.cast to all reletave compute
node.
-call refresh_security_group_rules(),it seems
that refresh_security_group_rules get the rule from the db and use libvirt
to define the rules.
but how  iptables are invoked to create rules  like nova-compute-inst-22.

anther question is  libvirt defines  nova-base-filter which allow any
packets out and drop all packets  in ,but it does not used by the instance
nwfilter.
the instance nwfilter only has no-mac-spoofing
,no-arp-spoofing,no-ip-spoofing ,and allow-dhcp-server filter.

if I misunderstand some thing ,please correct me ,thks .
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [CHEF] Aligning Cookbook Efforts

2012-02-06 Thread Jesse Andrews
I know that the RCB deploy team works with the Crowbar team on chef
recipes for that project.

Regarding the github.com/ansolabs  github.com/rcb recipes - I'll have
to delegate to Vishy who worked on those.

Jesse

On Mon, Feb 6, 2012 at 6:07 PM, Jay Pipes jaypi...@gmail.com wrote:
 Hi Stackers,

 tl;dr
 -

 There are myriad Chef cookbooks out there in the ecosystem and locked up
 behind various company firewalls. It would be awesome if we could agree to:

 * Align to a single origin repository for OpenStack cookbooks
 * Consolidate OpenStack Chef-based deployment experience into a single
 knowledge base
 * Have branches on the origin OpenStack cookbooks repository that align with
 core OpenStack projects
 * Automate the validation and testing of these cookbooks on multiple
 supported versions of the OpenStack code base

 Details
 ---

 Current State of Forks
 ==

 Matt Ray and I tried to outline the current state of the various OpenStack
 Chef cookbooks this past Thursday, and we came up with the following state
 of affairs:

 ** The official OpenStack Chef cookbooks **

 https://github.com/openstack/openstack-chef

 These chef cookbooks are the ones maintained mostly by Dan Prince and Brian
 Lamar and these are the cookbooks used by the SmokeStack project. The
 cookbooks contained in the above repo can install all the core OpenStack
 projects with the exception of Swift and Horizon.

 This repo is controlled by the Gerrit instance at review.openstack.org just
 like other core OpenStack projects.

 However, these cookbooks DO NOT currently have a stable/diablo branch --
 they are updated when the development trunks of any OpenStack project merges
 a commit that requires deployment or configuration-related changes to their
 associated cookbook.

 Important note: it's easy for Dan and Brian to know when updates to these
 cookbooks are necessary -- SmokeStack will bomb out if a
 deployment-affecting configuration change hits a core project trunk :)

 These cookbooks are the ONLY cookbooks that contain stuff for deploying with
 XenServer, AFAICT.

 ** NTT PF Lab Diablo Chef cookbooks **

 https://github.com/ntt-pf-lab/openstack-chef/

 So, NTT PF Lab forked the upstream Chef cookbooks back in Nov 11, 2011,
 because they needed a set of Chef cookbooks for OpenStack that functioned
 for the Diablo code base.

 While Nov 11, 2011, is not the *exact* date of the Diablo release, these
 cookbooks do in fact work for a Diablo install -- Nati Ueno is using them
 for the FreeCloud deployment so we know they work...

 ** OpsCode OpenStack Chef Cookbooks **

 Matt Ray from OpsCode created a set of cookbooks for OpenStack for the
 Cactus release of OpenStack:

 https://github.com/mattray/openstack-cookbooks
 http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef

 These cookbooks were forked from the Anso Labs' original OpenStack cookbooks
 from the Bexar release and were the basis for the Chef work that Dell did
 for Crowbar. Crowbar was originally based on Cactus, and according to Matt,
 the repositories of OpenStack cookbooks that OpsCode houses internally and
 uses most often are Cactus-based cookbooks. (Matt, please correct me if I am
 wrong here...)

 ** Rackspace CloudBuilders OpenStack Chef Cookbooks **

 The RCB team also has a repository of OpenStack Chef cookbooks:

 https://github.com/cloudbuilders/openstack-cookbooks

 Now, GitHub *says* that these cookbooks were forked from the official
 upstream cookbooks, but I do not think that is correct. Looking at this
 repo, I believe that this repo was *actually* forked from the Anso Labs
 OpenStack Chef Cookbooks, as the list of cookbooks is virtually identical.

 ** Anso Labs OpenStack Chef Cookbooks **

 These older cookbooks are in this repo:

 https://github.com/ansolabs/openstack-cookbooks/tree/master/cookbooks

 Interestingly, this repo DOES contain a cookbook for Swift.

 Current State of Documentation
 ==

 Documentation for best practices on using Chef for your OpenStack
 deployments is, well, a bit scattered. Matt Ray has some good information on
 the README on his cookbook repo and the OpsCode wiki:

 https://github.com/mattray/openstack-cookbooks/blob/cactus/README.md
 http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef

 But it is unfortunately not going to help people looking to deploy Diablo
 and later versions of OpenStack.

 Most of the other repos contain virtually no documentation on using the
 cookbooks or how they are written.

 I have a suspicion that one of the reasons that there has been such a
 proliferation of cookbooks has been the lack of documentation pointing
 people to an appropriate repo, how to use the cookbooks properly, and what
 the best practices for deployment are. That, and the fact that folks are
 just trying to stand up complex clouds and Get Things Done, and
 documentation is annoying to write ;)

 Proposal for Alignment
 

Re: [Openstack] [CHEF] Aligning Cookbook Efforts

2012-02-06 Thread Jay Pipes

On 02/06/2012 09:37 PM, Jesse Andrews wrote:

I know that the RCB deploy team works with the Crowbar team on chef
recipes for that project.


OK. Are you in agreement about the proposal in my email?


Regarding the github.com/ansolabs  github.com/rcb recipes - I'll have
to delegate to Vishy who worked on those.


Yup, no prob.

Best,
-jay


Jesse

On Mon, Feb 6, 2012 at 6:07 PM, Jay Pipesjaypi...@gmail.com  wrote:

Hi Stackers,

tl;dr
-

There are myriad Chef cookbooks out there in the ecosystem and locked up
behind various company firewalls. It would be awesome if we could agree to:

* Align to a single origin repository for OpenStack cookbooks
* Consolidate OpenStack Chef-based deployment experience into a single
knowledge base
* Have branches on the origin OpenStack cookbooks repository that align with
core OpenStack projects
* Automate the validation and testing of these cookbooks on multiple
supported versions of the OpenStack code base

Details
---

Current State of Forks
==

Matt Ray and I tried to outline the current state of the various OpenStack
Chef cookbooks this past Thursday, and we came up with the following state
of affairs:

** The official OpenStack Chef cookbooks **

https://github.com/openstack/openstack-chef

These chef cookbooks are the ones maintained mostly by Dan Prince and Brian
Lamar and these are the cookbooks used by the SmokeStack project. The
cookbooks contained in the above repo can install all the core OpenStack
projects with the exception of Swift and Horizon.

This repo is controlled by the Gerrit instance at review.openstack.org just
like other core OpenStack projects.

However, these cookbooks DO NOT currently have a stable/diablo branch --
they are updated when the development trunks of any OpenStack project merges
a commit that requires deployment or configuration-related changes to their
associated cookbook.

Important note: it's easy for Dan and Brian to know when updates to these
cookbooks are necessary -- SmokeStack will bomb out if a
deployment-affecting configuration change hits a core project trunk :)

These cookbooks are the ONLY cookbooks that contain stuff for deploying with
XenServer, AFAICT.

** NTT PF Lab Diablo Chef cookbooks **

https://github.com/ntt-pf-lab/openstack-chef/

So, NTT PF Lab forked the upstream Chef cookbooks back in Nov 11, 2011,
because they needed a set of Chef cookbooks for OpenStack that functioned
for the Diablo code base.

While Nov 11, 2011, is not the *exact* date of the Diablo release, these
cookbooks do in fact work for a Diablo install -- Nati Ueno is using them
for the FreeCloud deployment so we know they work...

** OpsCode OpenStack Chef Cookbooks **

Matt Ray from OpsCode created a set of cookbooks for OpenStack for the
Cactus release of OpenStack:

https://github.com/mattray/openstack-cookbooks
http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef

These cookbooks were forked from the Anso Labs' original OpenStack cookbooks
from the Bexar release and were the basis for the Chef work that Dell did
for Crowbar. Crowbar was originally based on Cactus, and according to Matt,
the repositories of OpenStack cookbooks that OpsCode houses internally and
uses most often are Cactus-based cookbooks. (Matt, please correct me if I am
wrong here...)

** Rackspace CloudBuilders OpenStack Chef Cookbooks **

The RCB team also has a repository of OpenStack Chef cookbooks:

https://github.com/cloudbuilders/openstack-cookbooks

Now, GitHub *says* that these cookbooks were forked from the official
upstream cookbooks, but I do not think that is correct. Looking at this
repo, I believe that this repo was *actually* forked from the Anso Labs
OpenStack Chef Cookbooks, as the list of cookbooks is virtually identical.

** Anso Labs OpenStack Chef Cookbooks **

These older cookbooks are in this repo:

https://github.com/ansolabs/openstack-cookbooks/tree/master/cookbooks

Interestingly, this repo DOES contain a cookbook for Swift.

Current State of Documentation
==

Documentation for best practices on using Chef for your OpenStack
deployments is, well, a bit scattered. Matt Ray has some good information on
the README on his cookbook repo and the OpsCode wiki:

https://github.com/mattray/openstack-cookbooks/blob/cactus/README.md
http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef

But it is unfortunately not going to help people looking to deploy Diablo
and later versions of OpenStack.

Most of the other repos contain virtually no documentation on using the
cookbooks or how they are written.

I have a suspicion that one of the reasons that there has been such a
proliferation of cookbooks has been the lack of documentation pointing
people to an appropriate repo, how to use the cookbooks properly, and what
the best practices for deployment are. That, and the fact that folks are
just trying to stand up complex clouds and Get Things Done, and
documentation is annoying to 

Re: [Openstack] [CHEF] Aligning Cookbook Efforts

2012-02-06 Thread Jason Cannavale
RCB deploy has a set of chef cookbooks we use for diablo all in one testing 
(minus swift) in addition to the work we have been doing with the crowbar team.

Since we seem to be adding to the problem, we'd be happy to collaborate on the 
consolidation..

Jason

On Feb 6, 2012, at 20:56, Jesse Andrews anotherje...@gmail.com wrote:

 I know that the RCB deploy team works with the Crowbar team on chef
 recipes for that project.
 
 Regarding the github.com/ansolabs  github.com/rcb recipes - I'll have
 to delegate to Vishy who worked on those.
 
 Jesse
 
 On Mon, Feb 6, 2012 at 6:07 PM, Jay Pipes jaypi...@gmail.com wrote:
 Hi Stackers,
 
 tl;dr
 -
 
 There are myriad Chef cookbooks out there in the ecosystem and locked up
 behind various company firewalls. It would be awesome if we could agree to:
 
 * Align to a single origin repository for OpenStack cookbooks
 * Consolidate OpenStack Chef-based deployment experience into a single
 knowledge base
 * Have branches on the origin OpenStack cookbooks repository that align with
 core OpenStack projects
 * Automate the validation and testing of these cookbooks on multiple
 supported versions of the OpenStack code base
 
 Details
 ---
 
 Current State of Forks
 ==
 
 Matt Ray and I tried to outline the current state of the various OpenStack
 Chef cookbooks this past Thursday, and we came up with the following state
 of affairs:
 
 ** The official OpenStack Chef cookbooks **
 
 https://github.com/openstack/openstack-chef
 
 These chef cookbooks are the ones maintained mostly by Dan Prince and Brian
 Lamar and these are the cookbooks used by the SmokeStack project. The
 cookbooks contained in the above repo can install all the core OpenStack
 projects with the exception of Swift and Horizon.
 
 This repo is controlled by the Gerrit instance at review.openstack.org just
 like other core OpenStack projects.
 
 However, these cookbooks DO NOT currently have a stable/diablo branch --
 they are updated when the development trunks of any OpenStack project merges
 a commit that requires deployment or configuration-related changes to their
 associated cookbook.
 
 Important note: it's easy for Dan and Brian to know when updates to these
 cookbooks are necessary -- SmokeStack will bomb out if a
 deployment-affecting configuration change hits a core project trunk :)
 
 These cookbooks are the ONLY cookbooks that contain stuff for deploying with
 XenServer, AFAICT.
 
 ** NTT PF Lab Diablo Chef cookbooks **
 
 https://github.com/ntt-pf-lab/openstack-chef/
 
 So, NTT PF Lab forked the upstream Chef cookbooks back in Nov 11, 2011,
 because they needed a set of Chef cookbooks for OpenStack that functioned
 for the Diablo code base.
 
 While Nov 11, 2011, is not the *exact* date of the Diablo release, these
 cookbooks do in fact work for a Diablo install -- Nati Ueno is using them
 for the FreeCloud deployment so we know they work...
 
 ** OpsCode OpenStack Chef Cookbooks **
 
 Matt Ray from OpsCode created a set of cookbooks for OpenStack for the
 Cactus release of OpenStack:
 
 https://github.com/mattray/openstack-cookbooks
 http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef
 
 These cookbooks were forked from the Anso Labs' original OpenStack cookbooks
 from the Bexar release and were the basis for the Chef work that Dell did
 for Crowbar. Crowbar was originally based on Cactus, and according to Matt,
 the repositories of OpenStack cookbooks that OpsCode houses internally and
 uses most often are Cactus-based cookbooks. (Matt, please correct me if I am
 wrong here...)
 
 ** Rackspace CloudBuilders OpenStack Chef Cookbooks **
 
 The RCB team also has a repository of OpenStack Chef cookbooks:
 
 https://github.com/cloudbuilders/openstack-cookbooks
 
 Now, GitHub *says* that these cookbooks were forked from the official
 upstream cookbooks, but I do not think that is correct. Looking at this
 repo, I believe that this repo was *actually* forked from the Anso Labs
 OpenStack Chef Cookbooks, as the list of cookbooks is virtually identical.
 
 ** Anso Labs OpenStack Chef Cookbooks **
 
 These older cookbooks are in this repo:
 
 https://github.com/ansolabs/openstack-cookbooks/tree/master/cookbooks
 
 Interestingly, this repo DOES contain a cookbook for Swift.
 
 Current State of Documentation
 ==
 
 Documentation for best practices on using Chef for your OpenStack
 deployments is, well, a bit scattered. Matt Ray has some good information on
 the README on his cookbook repo and the OpsCode wiki:
 
 https://github.com/mattray/openstack-cookbooks/blob/cactus/README.md
 http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef
 
 But it is unfortunately not going to help people looking to deploy Diablo
 and later versions of OpenStack.
 
 Most of the other repos contain virtually no documentation on using the
 cookbooks or how they are written.
 
 I have a suspicion that one of the reasons that there has been 

Re: [Openstack] [CHEF] Aligning Cookbook Efforts

2012-02-06 Thread Vishvananda Ishaya

On Feb 6, 2012, at 6:37 PM, Jesse Andrews wrote:

 I know that the RCB deploy team works with the Crowbar team on chef
 recipes for that project.
 
 Regarding the github.com/ansolabs  github.com/rcb recipes - I'll have
 to delegate to Vishy who worked on those.

They were the basis of dan and matt's cookbooks, but they are now ancient 
history.  i've been using them as a repository for a few helper devstack 
recipes, but waldon pulled those out into a separate repo so it is fine if we 
torch them.


 
 Jesse
 
 On Mon, Feb 6, 2012 at 6:07 PM, Jay Pipes jaypi...@gmail.com wrote:
 Hi Stackers,
 
 tl;dr
 -
 
 There are myriad Chef cookbooks out there in the ecosystem and locked up
 behind various company firewalls. It would be awesome if we could agree to:
 
 * Align to a single origin repository for OpenStack cookbooks
 * Consolidate OpenStack Chef-based deployment experience into a single
 knowledge base
 * Have branches on the origin OpenStack cookbooks repository that align with
 core OpenStack projects
 * Automate the validation and testing of these cookbooks on multiple
 supported versions of the OpenStack code base
 
 Details
 ---
 
 Current State of Forks
 ==
 
 Matt Ray and I tried to outline the current state of the various OpenStack
 Chef cookbooks this past Thursday, and we came up with the following state
 of affairs:
 
 ** The official OpenStack Chef cookbooks **
 
 https://github.com/openstack/openstack-chef
 
 These chef cookbooks are the ones maintained mostly by Dan Prince and Brian
 Lamar and these are the cookbooks used by the SmokeStack project. The
 cookbooks contained in the above repo can install all the core OpenStack
 projects with the exception of Swift and Horizon.
 
 This repo is controlled by the Gerrit instance at review.openstack.org just
 like other core OpenStack projects.
 
 However, these cookbooks DO NOT currently have a stable/diablo branch --
 they are updated when the development trunks of any OpenStack project merges
 a commit that requires deployment or configuration-related changes to their
 associated cookbook.
 
 Important note: it's easy for Dan and Brian to know when updates to these
 cookbooks are necessary -- SmokeStack will bomb out if a
 deployment-affecting configuration change hits a core project trunk :)
 
 These cookbooks are the ONLY cookbooks that contain stuff for deploying with
 XenServer, AFAICT.
 
 ** NTT PF Lab Diablo Chef cookbooks **
 
 https://github.com/ntt-pf-lab/openstack-chef/
 
 So, NTT PF Lab forked the upstream Chef cookbooks back in Nov 11, 2011,
 because they needed a set of Chef cookbooks for OpenStack that functioned
 for the Diablo code base.
 
 While Nov 11, 2011, is not the *exact* date of the Diablo release, these
 cookbooks do in fact work for a Diablo install -- Nati Ueno is using them
 for the FreeCloud deployment so we know they work...
 
 ** OpsCode OpenStack Chef Cookbooks **
 
 Matt Ray from OpsCode created a set of cookbooks for OpenStack for the
 Cactus release of OpenStack:
 
 https://github.com/mattray/openstack-cookbooks
 http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef
 
 These cookbooks were forked from the Anso Labs' original OpenStack cookbooks
 from the Bexar release and were the basis for the Chef work that Dell did
 for Crowbar. Crowbar was originally based on Cactus, and according to Matt,
 the repositories of OpenStack cookbooks that OpsCode houses internally and
 uses most often are Cactus-based cookbooks. (Matt, please correct me if I am
 wrong here...)
 
 ** Rackspace CloudBuilders OpenStack Chef Cookbooks **
 
 The RCB team also has a repository of OpenStack Chef cookbooks:
 
 https://github.com/cloudbuilders/openstack-cookbooks
 
 Now, GitHub *says* that these cookbooks were forked from the official
 upstream cookbooks, but I do not think that is correct. Looking at this
 repo, I believe that this repo was *actually* forked from the Anso Labs
 OpenStack Chef Cookbooks, as the list of cookbooks is virtually identical.
 
 ** Anso Labs OpenStack Chef Cookbooks **
 
 These older cookbooks are in this repo:
 
 https://github.com/ansolabs/openstack-cookbooks/tree/master/cookbooks
 
 Interestingly, this repo DOES contain a cookbook for Swift.
 
 Current State of Documentation
 ==
 
 Documentation for best practices on using Chef for your OpenStack
 deployments is, well, a bit scattered. Matt Ray has some good information on
 the README on his cookbook repo and the OpsCode wiki:
 
 https://github.com/mattray/openstack-cookbooks/blob/cactus/README.md
 http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef
 
 But it is unfortunately not going to help people looking to deploy Diablo
 and later versions of OpenStack.
 
 Most of the other repos contain virtually no documentation on using the
 cookbooks or how they are written.
 
 I have a suspicion that one of the reasons that there has been such a
 proliferation of cookbooks has 

Re: [Openstack] [CHEF] Aligning Cookbook Efforts

2012-02-06 Thread Andiabes
Jesse, right. The results are here: 
https://github.com/dellcloudedge/crowbar/tree/openstack-os-build/barclamps . 
With separate repos for nova, swift, keystone and horizon ( at this location, 
they're git submodules)

The v1.2 tag deploys diablo/stable. 
Most Cookbooks are written to be useable within and without crowbar, thoug we 
mostly test within crowbar. 

(writing from home, hence the gmail, rather than @dell address)

On Feb 6, 2012, at 9:37 PM, Jesse Andrews anotherje...@gmail.com wrote:

 I know that the RCB deploy team works with the Crowbar team on chef
 recipes for that project.
 
 Regarding the github.com/ansolabs  github.com/rcb recipes - I'll have
 to delegate to Vishy who worked on those.
 
 Jesse
 
 On Mon, Feb 6, 2012 at 6:07 PM, Jay Pipes jaypi...@gmail.com wrote:
 Hi Stackers,
 
 tl;dr
 -
 
 There are myriad Chef cookbooks out there in the ecosystem and locked up
 behind various company firewalls. It would be awesome if we could agree to:
 
 * Align to a single origin repository for OpenStack cookbooks
 * Consolidate OpenStack Chef-based deployment experience into a single
 knowledge base
 * Have branches on the origin OpenStack cookbooks repository that align with
 core OpenStack projects
 * Automate the validation and testing of these cookbooks on multiple
 supported versions of the OpenStack code base
 
 Details
 ---
 
 Current State of Forks
 ==
 
 Matt Ray and I tried to outline the current state of the various OpenStack
 Chef cookbooks this past Thursday, and we came up with the following state
 of affairs:
 
 ** The official OpenStack Chef cookbooks **
 
 https://github.com/openstack/openstack-chef
 
 These chef cookbooks are the ones maintained mostly by Dan Prince and Brian
 Lamar and these are the cookbooks used by the SmokeStack project. The
 cookbooks contained in the above repo can install all the core OpenStack
 projects with the exception of Swift and Horizon.
 
 This repo is controlled by the Gerrit instance at review.openstack.org just
 like other core OpenStack projects.
 
 However, these cookbooks DO NOT currently have a stable/diablo branch --
 they are updated when the development trunks of any OpenStack project merges
 a commit that requires deployment or configuration-related changes to their
 associated cookbook.
 
 Important note: it's easy for Dan and Brian to know when updates to these
 cookbooks are necessary -- SmokeStack will bomb out if a
 deployment-affecting configuration change hits a core project trunk :)
 
 These cookbooks are the ONLY cookbooks that contain stuff for deploying with
 XenServer, AFAICT.
 
 ** NTT PF Lab Diablo Chef cookbooks **
 
 https://github.com/ntt-pf-lab/openstack-chef/
 
 So, NTT PF Lab forked the upstream Chef cookbooks back in Nov 11, 2011,
 because they needed a set of Chef cookbooks for OpenStack that functioned
 for the Diablo code base.
 
 While Nov 11, 2011, is not the *exact* date of the Diablo release, these
 cookbooks do in fact work for a Diablo install -- Nati Ueno is using them
 for the FreeCloud deployment so we know they work...
 
 ** OpsCode OpenStack Chef Cookbooks **
 
 Matt Ray from OpsCode created a set of cookbooks for OpenStack for the
 Cactus release of OpenStack:
 
 https://github.com/mattray/openstack-cookbooks
 http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef
 
 These cookbooks were forked from the Anso Labs' original OpenStack cookbooks
 from the Bexar release and were the basis for the Chef work that Dell did
 for Crowbar. Crowbar was originally based on Cactus, and according to Matt,
 the repositories of OpenStack cookbooks that OpsCode houses internally and
 uses most often are Cactus-based cookbooks. (Matt, please correct me if I am
 wrong here...)
 
 ** Rackspace CloudBuilders OpenStack Chef Cookbooks **
 
 The RCB team also has a repository of OpenStack Chef cookbooks:
 
 https://github.com/cloudbuilders/openstack-cookbooks
 
 Now, GitHub *says* that these cookbooks were forked from the official
 upstream cookbooks, but I do not think that is correct. Looking at this
 repo, I believe that this repo was *actually* forked from the Anso Labs
 OpenStack Chef Cookbooks, as the list of cookbooks is virtually identical.
 
 ** Anso Labs OpenStack Chef Cookbooks **
 
 These older cookbooks are in this repo:
 
 https://github.com/ansolabs/openstack-cookbooks/tree/master/cookbooks
 
 Interestingly, this repo DOES contain a cookbook for Swift.
 
 Current State of Documentation
 ==
 
 Documentation for best practices on using Chef for your OpenStack
 deployments is, well, a bit scattered. Matt Ray has some good information on
 the README on his cookbook repo and the OpsCode wiki:
 
 https://github.com/mattray/openstack-cookbooks/blob/cactus/README.md
 http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef
 
 But it is unfortunately not going to help people looking to deploy Diablo
 and later versions of OpenStack.
 
 Most 

Re: [Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Maru Newby
-1 on multi-distribution devstack.  Being cross-platform is arguably a place 
where chef/puppet/cfengine automation comes into play, and that's not where 
devstack's self-declared mission lies.

+1 to continuing to have Ubuntu be the reference devstack target.  Maintaining 
support for an apt-based distribution is much easier than the alternatives from 
a developer perspective.

Mind you, I don't think anybody would complain if Redhat et al wanted to 
maintain their own targeted version of devstack.

Thanks,


Maru

On 2012-02-06, at 5:22 PM, Joshua Harlow wrote:

 + There needs to be a way to install on multiple distributions (without 
 saying go figure out the deps yourself).
 
 I know everyone is ubuntu, ubuntu, ubuntu, but this really needs to be fixed 
 (process wise as well).
 
 :-/
 
 On 2/6/12 5:12 PM, Jay Pipes jaypi...@gmail.com wrote:
 
 cc'ing Matt Ray from OpsCode, since he and I discussed related topics
 this past Thursday during the bug squash day...
 
 On 02/06/2012 06:35 PM, Monty Taylor wrote:
  I think the thing you are discussing already exists.
 
  devstack is currently part of and managed by all of the normal OpenStack
  development infrastructure. The canonical repository for it is
  https://review.openstack.org/p/openstack-dev/devstack which is mirrored
  to https://github.com/openstack-dev/devstack. Every change to OpenStack
  is not only gated on devstack properly functioning, every change to
  devstack is gated on OpenStack properly functioning.
 
  Additionally, branches match up, so there is a stable/diablo that works
  with stable/diablo of all of the OpenStack branches and is a part of
  their trunk gating.
 
 This is a critical piece of the puzzle. If I want a Diablo install for
 testing, all I need to do is:
 
 cd $devstack_dir
 git checkout stable/diablo
 rm -rf /opt/stack
 ./stack.sh
 
 And I get a Diablo installation of OpenStack. Likewise, if I want a
 development (Essex currently) version of OpenStack, I just do:
 
 cd $devstack_dir
 git checkout master
 rm -rf /opt/stack
 ./stack.sh
 
 And I get a development installation of OpenStack.
 
 Now, I'm not entirely sure I even need to do the rm -rf /opt/stack part,
 but I do that for good measure, even if it does mean it takes a little
 longer... ;)
 
 This is not something I can do currently with the other deployment methods.
 
  In that sense, it's actually the first install OpenStack method that
  _is_ fully a part of OpenStack - even though there are also chef recipes
  and puppet modules in OpenStack's gerrit as well. (although at some
  point I wouldn't mind getting some installation testing and gating on
  them as well)
 
 Yes, and getting those projects aligned with the core projects' branch
 layout would be good, too. Followup email on the Chef stuff coming
 shortly, as Matt ray and I discussed this last Thursday at length and I
 think there's a lot we can do to improve things.
 
 -jay
 
  So it's pretty official already.
 
  However, as to becoming an official project - it's a developer tool,
  same as git-review or gerrit or the openstack nose-plugin. It's
  something that's useful for developers for developing and testing
  OpenStack. It is not, nor is it meant to be, part of the software we
  ship -- which is the current definition of what it means to be a
  core project. i.e. - If I'm a deployer and I want to install
  OpenStack - is this one of the things I install? With devstack - the
  answer is no.
 
  Is is MASSIVELY helpful and a part of everyday life for all of us?
  ABSOLUTELY (this is why we have to be careful with changes to it and run
  them through the same process everything else gets)
 
  All of that to say - I agree with you, and it's already done. :)
 
  Monty
 
  On 02/06/2012 01:43 PM, Joshua Harlow wrote:
  So the part that worries me about what u just said is the part about “it
  is already some kind of official project”.
  When you have to question whether a project is official or not, that
  seems to pretty make the whole point for making it official ;)
 
  Overall though I think what u are saying is correct, but the overhead I
  don’t see as being a bad thing.
 
  In my idea release management is good since it allows developers to be
  able to setup a development environment for a given openstack release
  (good for when you need to fix bugs against a given release as well as
  good for providing a stable point for other distributions to know what
  goes in a release and what configs need to be adjusted to make that
  release work for all the different components). So I don’t see that as a
  drawback (even though yes it does add work/overhead in, but I don’t see
  that as a valid point, in any case).
 
  Downstream distribution, I am not exactly sure what you mean here?
 
  A technical lead I think is something good to have, as this
  script/code/documentation is not as simple as you might think (and most
  likely won’t get any simpler).
 
  Maybe the correct wording isn’t that this is 

Re: [Openstack] [CHEF] Aligning Cookbook Efforts

2012-02-06 Thread Maru Newby
I've submitted a Swift AIO cookbook for review:

https://review.openstack.org/#change,3613

It follows the latest single-node AIO instructions pretty much to the letter, 
so the resulting environment is well-documented.  We use this cookbook as the 
basis for building Swift development environments here at Internap.

Thanks,


Maru

On 2012-02-06, at 6:07 PM, Jay Pipes wrote:

 Hi Stackers,
 
 tl;dr
 -
 
 There are myriad Chef cookbooks out there in the ecosystem and locked up 
 behind various company firewalls. It would be awesome if we could agree to:
 
 * Align to a single origin repository for OpenStack cookbooks
 * Consolidate OpenStack Chef-based deployment experience into a single 
 knowledge base
 * Have branches on the origin OpenStack cookbooks repository that align with 
 core OpenStack projects
 * Automate the validation and testing of these cookbooks on multiple 
 supported versions of the OpenStack code base
 
 Details
 ---
 
 Current State of Forks
 ==
 
 Matt Ray and I tried to outline the current state of the various OpenStack 
 Chef cookbooks this past Thursday, and we came up with the following state of 
 affairs:
 
 ** The official OpenStack Chef cookbooks **
 
 https://github.com/openstack/openstack-chef
 
 These chef cookbooks are the ones maintained mostly by Dan Prince and Brian 
 Lamar and these are the cookbooks used by the SmokeStack project. The 
 cookbooks contained in the above repo can install all the core OpenStack 
 projects with the exception of Swift and Horizon.
 
 This repo is controlled by the Gerrit instance at review.openstack.org just 
 like other core OpenStack projects.
 
 However, these cookbooks DO NOT currently have a stable/diablo branch -- they 
 are updated when the development trunks of any OpenStack project merges a 
 commit that requires deployment or configuration-related changes to their 
 associated cookbook.
 
 Important note: it's easy for Dan and Brian to know when updates to these 
 cookbooks are necessary -- SmokeStack will bomb out if a deployment-affecting 
 configuration change hits a core project trunk :)
 
 These cookbooks are the ONLY cookbooks that contain stuff for deploying with 
 XenServer, AFAICT.
 
 ** NTT PF Lab Diablo Chef cookbooks **
 
 https://github.com/ntt-pf-lab/openstack-chef/
 
 So, NTT PF Lab forked the upstream Chef cookbooks back in Nov 11, 2011, 
 because they needed a set of Chef cookbooks for OpenStack that functioned for 
 the Diablo code base.
 
 While Nov 11, 2011, is not the *exact* date of the Diablo release, these 
 cookbooks do in fact work for a Diablo install -- Nati Ueno is using them for 
 the FreeCloud deployment so we know they work...
 
 ** OpsCode OpenStack Chef Cookbooks **
 
 Matt Ray from OpsCode created a set of cookbooks for OpenStack for the Cactus 
 release of OpenStack:
 
 https://github.com/mattray/openstack-cookbooks
 http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef
 
 These cookbooks were forked from the Anso Labs' original OpenStack cookbooks 
 from the Bexar release and were the basis for the Chef work that Dell did for 
 Crowbar. Crowbar was originally based on Cactus, and according to Matt, the 
 repositories of OpenStack cookbooks that OpsCode houses internally and uses 
 most often are Cactus-based cookbooks. (Matt, please correct me if I am wrong 
 here...)
 
 ** Rackspace CloudBuilders OpenStack Chef Cookbooks **
 
 The RCB team also has a repository of OpenStack Chef cookbooks:
 
 https://github.com/cloudbuilders/openstack-cookbooks
 
 Now, GitHub *says* that these cookbooks were forked from the official 
 upstream cookbooks, but I do not think that is correct. Looking at this repo, 
 I believe that this repo was *actually* forked from the Anso Labs OpenStack 
 Chef Cookbooks, as the list of cookbooks is virtually identical.
 
 ** Anso Labs OpenStack Chef Cookbooks **
 
 These older cookbooks are in this repo:
 
 https://github.com/ansolabs/openstack-cookbooks/tree/master/cookbooks
 
 Interestingly, this repo DOES contain a cookbook for Swift.
 
 Current State of Documentation
 ==
 
 Documentation for best practices on using Chef for your OpenStack deployments 
 is, well, a bit scattered. Matt Ray has some good information on the README 
 on his cookbook repo and the OpsCode wiki:
 
 https://github.com/mattray/openstack-cookbooks/blob/cactus/README.md
 http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef
 
 But it is unfortunately not going to help people looking to deploy Diablo and 
 later versions of OpenStack.
 
 Most of the other repos contain virtually no documentation on using the 
 cookbooks or how they are written.
 
 I have a suspicion that one of the reasons that there has been such a 
 proliferation of cookbooks has been the lack of documentation pointing people 
 to an appropriate repo, how to use the cookbooks properly, and what the best 
 practices for deployment are. That, and the 

Re: [Openstack] [CHEF] Aligning Cookbook Efforts

2012-02-06 Thread Andiabes
The alignment proposal sounds great, and would definitely help reduce 
redundancy.

However, it might be useful to define clear goals of the resulting deployment 
using these cookbooks.
As an example - Looking at the anso recipes for swift - they appear to deploy a 
SAIO swift cluster. The Crowbar cookbook assumes a multi node deployment. 
Similarly for nova - the official cookbooks appear to focus only on flat 
networking (unless I'm missing something) while the Crowbar version supports 
multiple network configs ( e.g. Vlan). OTH, the official recipes support both 
MySQL and Postgres, while crowbar only supports MySQL.

( the above not intended to recommend brands of sliced bread ;)

The above raises a few questions ( and I'm sure there might be more):
- SAIO or multi node ?
- possibly repeat of the above - are the cookbooks to be used beyond unit 
testing, or just serve as an example?
- what coverage for the breadth of options ? Or stated differently - are the 
cookbooks prescriptive and opinionated about deployments, or flexible?
- does the above apply just to openstack components, or 3rd party dependencies ?

A.










On Feb 6, 2012, at 11:53 PM, Vishvananda Ishaya vishvana...@gmail.com wrote:

 
 On Feb 6, 2012, at 6:37 PM, Jesse Andrews wrote:
 
 I know that the RCB deploy team works with the Crowbar team on chef
 recipes for that project.
 
 Regarding the github.com/ansolabs  github.com/rcb recipes - I'll have
 to delegate to Vishy who worked on those.
 
 They were the basis of dan and matt's cookbooks, but they are now ancient 
 history.  i've been using them as a repository for a few helper devstack 
 recipes, but waldon pulled those out into a separate repo so it is fine if we 
 torch them.
 
 
 
 Jesse
 
 On Mon, Feb 6, 2012 at 6:07 PM, Jay Pipes jaypi...@gmail.com wrote:
 Hi Stackers,
 
 tl;dr
 -
 
 There are myriad Chef cookbooks out there in the ecosystem and locked up
 behind various company firewalls. It would be awesome if we could agree to:
 
 * Align to a single origin repository for OpenStack cookbooks
 * Consolidate OpenStack Chef-based deployment experience into a single
 knowledge base
 * Have branches on the origin OpenStack cookbooks repository that align with
 core OpenStack projects
 * Automate the validation and testing of these cookbooks on multiple
 supported versions of the OpenStack code base
 
 Details
 ---
 
 Current State of Forks
 ==
 
 Matt Ray and I tried to outline the current state of the various OpenStack
 Chef cookbooks this past Thursday, and we came up with the following state
 of affairs:
 
 ** The official OpenStack Chef cookbooks **
 
 https://github.com/openstack/openstack-chef
 
 These chef cookbooks are the ones maintained mostly by Dan Prince and Brian
 Lamar and these are the cookbooks used by the SmokeStack project. The
 cookbooks contained in the above repo can install all the core OpenStack
 projects with the exception of Swift and Horizon.
 
 This repo is controlled by the Gerrit instance at review.openstack.org just
 like other core OpenStack projects.
 
 However, these cookbooks DO NOT currently have a stable/diablo branch --
 they are updated when the development trunks of any OpenStack project merges
 a commit that requires deployment or configuration-related changes to their
 associated cookbook.
 
 Important note: it's easy for Dan and Brian to know when updates to these
 cookbooks are necessary -- SmokeStack will bomb out if a
 deployment-affecting configuration change hits a core project trunk :)
 
 These cookbooks are the ONLY cookbooks that contain stuff for deploying with
 XenServer, AFAICT.
 
 ** NTT PF Lab Diablo Chef cookbooks **
 
 https://github.com/ntt-pf-lab/openstack-chef/
 
 So, NTT PF Lab forked the upstream Chef cookbooks back in Nov 11, 2011,
 because they needed a set of Chef cookbooks for OpenStack that functioned
 for the Diablo code base.
 
 While Nov 11, 2011, is not the *exact* date of the Diablo release, these
 cookbooks do in fact work for a Diablo install -- Nati Ueno is using them
 for the FreeCloud deployment so we know they work...
 
 ** OpsCode OpenStack Chef Cookbooks **
 
 Matt Ray from OpsCode created a set of cookbooks for OpenStack for the
 Cactus release of OpenStack:
 
 https://github.com/mattray/openstack-cookbooks
 http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef
 
 These cookbooks were forked from the Anso Labs' original OpenStack cookbooks
 from the Bexar release and were the basis for the Chef work that Dell did
 for Crowbar. Crowbar was originally based on Cactus, and according to Matt,
 the repositories of OpenStack cookbooks that OpsCode houses internally and
 uses most often are Cactus-based cookbooks. (Matt, please correct me if I am
 wrong here...)
 
 ** Rackspace CloudBuilders OpenStack Chef Cookbooks **
 
 The RCB team also has a repository of OpenStack Chef cookbooks:
 
 https://github.com/cloudbuilders/openstack-cookbooks
 
 Now, GitHub *says* 

Re: [Openstack] [CHEF] Aligning Cookbook Efforts

2012-02-06 Thread Monty Taylor


On 02/06/2012 06:07 PM, Jay Pipes wrote:
 Hi Stackers,
 
 tl;dr
 -
 
 There are myriad Chef cookbooks out there in the ecosystem and locked
 up behind various company firewalls. It would be awesome if we could
 agree to:
 
 * Align to a single origin repository for OpenStack cookbooks
 * Consolidate OpenStack Chef-based deployment experience into a single
 knowledge base
 * Have branches on the origin OpenStack cookbooks repository that align
 with core OpenStack projects
 * Automate the validation and testing of these cookbooks on multiple
 supported versions of the OpenStack code base
 
 Details
 ---
 
 Current State of Forks
 ==
 
 Matt Ray and I tried to outline the current state of the various
 OpenStack Chef cookbooks this past Thursday, and we came up with the
 following state of affairs:
 
 ** The official OpenStack Chef cookbooks **
 
 https://github.com/openstack/openstack-chef
 
 These chef cookbooks are the ones maintained mostly by Dan Prince and
 Brian Lamar and these are the cookbooks used by the SmokeStack project.
 The cookbooks contained in the above repo can install all the core
 OpenStack projects with the exception of Swift and Horizon.
 
 This repo is controlled by the Gerrit instance at review.openstack.org
 just like other core OpenStack projects.
 
 However, these cookbooks DO NOT currently have a stable/diablo branch --
 they are updated when the development trunks of any OpenStack project
 merges a commit that requires deployment or configuration-related
 changes to their associated cookbook.
 
 Important note: it's easy for Dan and Brian to know when updates to
 these cookbooks are necessary -- SmokeStack will bomb out if a
 deployment-affecting configuration change hits a core project trunk :)

I would like to get these to have a stable/diablo branch.

 These cookbooks are the ONLY cookbooks that contain stuff for deploying
 with XenServer, AFAICT.

I think Dan and Brian are also deploying kvm as part of smokestack.

 ** NTT PF Lab Diablo Chef cookbooks **
 
 https://github.com/ntt-pf-lab/openstack-chef/
 
 So, NTT PF Lab forked the upstream Chef cookbooks back in Nov 11, 2011,
 because they needed a set of Chef cookbooks for OpenStack that
 functioned for the Diablo code base.
 
 While Nov 11, 2011, is not the *exact* date of the Diablo release, these
 cookbooks do in fact work for a Diablo install -- Nati Ueno is using
 them for the FreeCloud deployment so we know they work...

If these are a fork of openstack/openstack-chef, could we perhaps make
these the basis of a stable/diablo branch in openstack-chef?

 ** OpsCode OpenStack Chef Cookbooks **
 
 Matt Ray from OpsCode created a set of cookbooks for OpenStack for the
 Cactus release of OpenStack:
 
 https://github.com/mattray/openstack-cookbooks
 http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef
 
 These cookbooks were forked from the Anso Labs' original OpenStack
 cookbooks from the Bexar release and were the basis for the Chef work
 that Dell did for Crowbar. Crowbar was originally based on Cactus, and
 according to Matt, the repositories of OpenStack cookbooks that OpsCode
 houses internally and uses most often are Cactus-based cookbooks. (Matt,
 please correct me if I am wrong here...)
 
 ** Rackspace CloudBuilders OpenStack Chef Cookbooks **
 
 The RCB team also has a repository of OpenStack Chef cookbooks:
 
 https://github.com/cloudbuilders/openstack-cookbooks
 
 Now, GitHub *says* that these cookbooks were forked from the official
 upstream cookbooks, but I do not think that is correct. Looking at this
 repo, I believe that this repo was *actually* forked from the Anso Labs
 OpenStack Chef Cookbooks, as the list of cookbooks is virtually identical.
 
 ** Anso Labs OpenStack Chef Cookbooks **
 
 These older cookbooks are in this repo:
 
 https://github.com/ansolabs/openstack-cookbooks/tree/master/cookbooks
 
 Interestingly, this repo DOES contain a cookbook for Swift.
 
 Current State of Documentation
 ==
 
 Documentation for best practices on using Chef for your OpenStack
 deployments is, well, a bit scattered. Matt Ray has some good
 information on the README on his cookbook repo and the OpsCode wiki:
 
 https://github.com/mattray/openstack-cookbooks/blob/cactus/README.md
 http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef
 
 But it is unfortunately not going to help people looking to deploy
 Diablo and later versions of OpenStack.
 
 Most of the other repos contain virtually no documentation on using the
 cookbooks or how they are written.
 
 I have a suspicion that one of the reasons that there has been such a
 proliferation of cookbooks has been the lack of documentation pointing
 people to an appropriate repo, how to use the cookbooks properly, and
 what the best practices for deployment are. That, and the fact that
 folks are just trying to stand up complex clouds and Get Things Done,
 and documentation is annoying to 

Re: [Openstack] [DEVSTACK] officialize it!

2012-02-06 Thread Chmouel Boudjnah
On Tue, Feb 7, 2012 at 2:22 AM, Joshua Harlow harlo...@yahoo-inc.comwrote:

  + There needs to be a way to install on multiple distributions (without
 saying go figure out the deps yourself).
 I know everyone is ubuntu, ubuntu, ubuntu, but this really needs to be
 fixed (process wise as well).

 +1 on supporting more than ubuntu, this is something that devstack-py is
doing nicely.

Chmouel.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp