Re: [Openstack] Image prep. cloud-init user configuration help.
The user should be created automatically afaik (if it doesn't exist). From: Jake G. dj_dark_jungl...@yahoo.commailto:dj_dark_jungl...@yahoo.com Reply-To: Jake G. dj_dark_jungl...@yahoo.commailto:dj_dark_jungl...@yahoo.com Date: Monday, July 22, 2013 11:51 PM To: Jake G. dj_dark_jungl...@yahoo.commailto:dj_dark_jungl...@yahoo.com, openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] Image prep. cloud-init user configuration help. Btw This is for a centos 6.4 image. From: Jake G. dj_dark_jungl...@yahoo.commailto:dj_dark_jungl...@yahoo.com To: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Sent: Tuesday, July 23, 2013 3:49 PM Subject: [Openstack] Image prep. cloud-init user configuration help. Hi! I am following the part on how to set up cloud-init from the below guide, but I was wondering how to change the user? I do not have the default ec2-user on my instance image. Do I just create this user in the instance? What permissions do I give the user? http://docs.openstack.org/trunk/openstack-image/content/centos-image.html#d6e689 ___ Mailing list: https://launchpad.net/~openstack Post to: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] How to Install OpenStack ???????????
Begin the openstack aol CDs :-) Sent from my really tiny device... On Jul 12, 2013, at 1:24 PM, Matt Joyce matt.jo...@cloudscaling.commailto:matt.jo...@cloudscaling.com wrote: You know, I am surprised none of us OpenStack distribution vendors have taken a page from 1999 and started selling OpenStack grizzly/havana books with installation CDs for our distribution in the back. -Matt On Fri, Jul 12, 2013 at 11:15 AM, Joshua McKenty jos...@pistoncloud.commailto:jos...@pistoncloud.com wrote: Tiny product plug for Piston's Enterprise OpenStack distro as well. Neutron support is in our next release, but we can fix you up with a beta if it's critical. -- Joshua McKenty Chief Technology Officer Piston Cloud Computing, Inc. +1 (650) 242-5683 +1 (650) 283-6846 http://www.pistoncloud.com Oh, Westley, we'll never survive! Nonsense. You're only saying that because no one ever has. On Jul 12, 2013, at 11:10 AM, Samuel Winchenbach swinc...@gmail.commailto:swinc...@gmail.com wrote: Wow, Mirantis Fuel looks impressive. Very impressive. Thanks for pointing that out. I wonder if there support for Quantum/Neutron. Hmm I might have to play around with that. On Fri, Jul 12, 2013 at 1:52 PM, Logan McNaughton lo...@bacoosta.commailto:lo...@bacoosta.com wrote: I think these are the 3 best options for an automated OpenStack install: RDO (Packstack), supports RHEL, CentOS, Fedora. MAAS/Juju, supports Ubuntu. Mirantis Fuel, supports RHEL/CentOS for now, they say Ubuntu support is coming. Try all 3 if you can. Fuel was just recently open sourced and has a pretty fancy web GUI. On Jul 12, 2013 10:16 AM, Min Pae sputni...@gmail.commailto:sputni...@gmail.com wrote: I was able to go from nothing to a running Openstack environment using Ubuntu Juju/MAAS inside 2 weeks with no prior knowledge or experience with Juju nor MAAS nor Openstack. The trick seemed to be having enough boxes as some charms didn't seem to like running on the same boxes or whatever, and I ended up with a non-functional dashboard when trying to install all the services to the same box. Currently I have it working well with 7 physical boxes in total with one of those being a nova-compute node. On Fri, Jul 12, 2013 at 7:39 AM, claudio marques mrqss_...@hotmail.commailto:mrqss_...@hotmail.com wrote: Hi You can use this guide to. https://github.com/mseknibilel/OpenStack-Grizzly-Install-Guide Good luck Claudio Marques clau...@onesource.ptmailto:clau...@onesource.pt http://www.onesource.pt/ From: luisguilherme...@gmail.commailto:luisguilherme...@gmail.com Date: Fri, 12 Jul 2013 09:56:18 -0300 To: dj_dark_jungl...@yahoo.commailto:dj_dark_jungl...@yahoo.com CC: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] How to Install OpenStack ??? Hello Jake, I am new at OpenStack too, and I'm running a little environment with three computers, one is the controller + network node and the others are compute node. I've been following the manual at the OpenStack's documentation page but it's attached here, the most mess part is to create the networks, I ran the script attached too. Hope it helps you. Regards. Guilherme. 2013/7/12 Jake G. dj_dark_jungl...@yahoo.commailto:dj_dark_jungl...@yahoo.com Hi Mark, Thanks for your reply. I have about 3 physical rack servers and practically unlimited virtual machines. Right now I only need a test environment. I was thinking one physical server that will house and power openstack instances and virtual for all the other roles. How does that sound? What is your recommended setup? Best, Jake From: Mark Baker mark.ba...@canonical.commailto:mark.ba...@canonical.com To: Jake G. dj_dark_jungl...@yahoo.commailto:dj_dark_jungl...@yahoo.com Sent: Friday, July 12, 2013 6:05 PM Subject: Re: [Openstack] How to Install OpenStack ??? On 12/07/13 07:58, Jake G. wrote: Hi All, I have been struggling with installing Openstack for the past 2 weeks and I am about to rip my own hair out. rant Does anyone have installation instructions that a human being can actually understand and follow? I am usually pretty good at installing new tech but OpenStack is the most convoluted environment (even worse documentation) I have ever come in contact with (Worse than IBM software). The advanced install and config of CloudStack 4.1 is a breeze compare to Openstack. Was this made to purposely line the pockets of Openstack deployment consulting companies? Openstack might be great but no one will know because its impossible to deploy. /rant I`m sure I am not the only one who feels this way. I would appreciate any help anyone can give. Someones blog, other installation methods, anything How many servers do you have? Instructions for using the Ubuntu packaging are at: http://www.ubuntu.com/download/cloud/install-ubuntu-cloud There are
Re: [Openstack] New code name for networks
I vote for calling it spacetime since spacetime binds us all together in the same way networks bind computers together. Doubt it's copyrightable if that's needed, unless Einstein holds the copyright :P Sent from my really tiny device... On May 12, 2013, at 10:43 AM, John Wong gokoproj...@gmail.commailto:gokoproj...@gmail.com wrote: Cool. I think I did a search here: http://tmsearch.uspto.gov/bin/gate.exe?f=searchssstate=4804:3j8874.1.1 for Netpune. Are we only concern with registered trademark? I am sure a lot of names we come up with are either full or partial of some company's name. What's the bottom line? Who's going to come up the list of names? Where do we put our suggestion? For the next OS release we have a Wiki page setup. Do we have a Wiki page setup yet? Best, John On Sun, May 12, 2013 at 12:27 PM, Sean Dague s...@dague.netmailto:s...@dague.net wrote: Quark is also the trademark of a computer software company - http://tess2.uspto.gov/bin/showfield?f=docstate=4808:7k4xqd.2.44 This stuff is trickier than you might realize. Hence why car names are now all made-ity-up words. -Sean On 05/12/2013 12:21 PM, Jacob Godin wrote: I like this, +1 On 2013-05-11 8:08 PM, David Shrewsbury shrewsbury.d...@gmail.commailto:shrewsbury.d...@gmail.com mailto:shrewsbury.d...@gmail.commailto:shrewsbury.d...@gmail.com wrote: quark Keeps the sciency theme going, same first two letters, and represents something fundamental to other sciency stuff (much like networking is fundamental to OpenStack). http://en.wikipedia.org/wiki/Quark -Dave On May 11, 2013, at 4:13 PM, Monty Taylor mord...@inaugust.commailto:mord...@inaugust.com mailto:mord...@inaugust.commailto:mord...@inaugust.com wrote: Jeremy Stanly on IRC just suggested kumquat... but to that I respond: qumkuat Same benefits as qumutna - except it's more pronouncable. On 05/11/2013 04:07 PM, Monty Taylor wrote: I have been arguing for: mutnuaq Granted, it takes a minute to learn how to type, but it's just a little snarky, and it takes up the exact same number of letter. However, it does screw with sorting. SO - what about: qumutna It's a little bit easier to wrap your head around, it's still clearly an homage, and it should be super easy to bulk cut/replace. On 05/11/2013 03:58 PM, Davanum Srinivas wrote: Lattice -- dims On Sat, May 11, 2013 at 3:52 PM, Mark Turner m...@amerine.netmailto:m...@amerine.net mailto:m...@amerine.netmailto:m...@amerine.net wrote: Tubes ;-) On Sat, May 11, 2013 at 12:51 PM, Jason Smith jason.sm...@rackspace.commailto:jason.sm...@rackspace.com mailto:jason.sm...@rackspace.commailto:jason.sm...@rackspace.com wrote: Hello, I understand why we had to give up Quantum code name but rather than just refer to it as networking let's come up with a new code name! Thoughts? Thanks, -js ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net mailto:openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net mailto:openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net mailto:openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net mailto:openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net mailto:openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net
Re: [Openstack] New site for questions http://ask.openstack.org
+2 :-) Sent from my really tiny device... On Mar 27, 2013, at 1:54 AM, Razique Mahroua razique.mahr...@gmail.commailto:razique.mahr...@gmail.com wrote: Wow amazing :D Razique Mahroua - Nuage Co razique.mahr...@gmail.commailto:razique.mahr...@gmail.com Tel : +33 9 72 37 94 15 NUAGECO-LOGO-Fblan_petit.jpg Le 26 mars 2013 à 23:15, Stefano Maffulli stef...@openstack.orgmailto:stef...@openstack.org a écrit : The OpenStack Foundation has launched a new and improved service to help OpenStack users, operators and developers exchange information and find answers to their questions. Go to http://ask.openstack.org and get familiar with it, ask questions there and search for answers. If you want to be a moderator and help others by giving answers please contact me. The new service Ask OpenStack will take the place of the old forums any time now (the dns for forums.openstack.orghttp://forums.openstack.org will point to ask.openstack.orghttp://ask.openstack.org) Unfortunately I have found no way to reach current users of the old forums. If you know somebody that used to use forums.openstack.orghttp://forums.openstack.org please tell them to use http://Ask.OpenStack.org instead. thanks, stef ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack login page
Can u jump on irc, channel anvil, and I will help. I think that bootstrap dependency doesn't need to be pinned anymore. Sent from my really tiny device... On Mar 6, 2013, at 2:52 AM, Ashutosh Narayan aashutoshnara...@gmail.commailto:aashutoshnara...@gmail.com wrote: Sorry for continuous email chain. I went through the logs and learnt that the package itself is not found. Find below few lines of pip.log file. Could not fetch URL http://pypi.python.org/simple/Cheetah/2.4.4: HTTP Error 404: Not Found (Cheetah/2.4.4) Will skip URL http://pypi.python.org/simple/Cheetah/2.4.4 when looking for download links for Cheetah==2.4.4 Even http://pypi.python.org/packages/source/M/Markdown/Markdown-2.2.1.zip#md5=67bd5f47864862708ede8eb85b1618ee package times out. Any suggestions for troubleshooting ? Thank you, On Wed, Mar 6, 2013 at 4:09 PM, Ashutosh Narayan aashutoshnara...@gmail.commailto:aashutoshnara...@gmail.com wrote: Python version : $ python Python 2.6.6 (r266:84292, Sep 11 2012, 08:34:23) [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2 Type help, copyright, credits or license for more information. On Wed, Mar 6, 2013 at 4:06 PM, Ashutosh Narayan aashutoshnara...@gmail.commailto:aashutoshnara...@gmail.com wrote: Here is a snippet of console logs that I get : Cleaning up... Installing pypi requirement 'Cheetah==2.4.4' Downloading/unpacking Cheetah==2.4.4 Downloading Cheetah-2.4.4.tar.gz (190Kb): 190Kb downloaded Running setup.py egg_info for package Cheetah warning: no files found matching 'examples' warning: no files found matching 'docs' warning: no files found matching 'bin' warning: no files found matching '*' under directory 'docs' warning: no files found matching '*' under directory 'examples' warning: no previously-included files matching '*.pyc' found under directory 'cheetah' warning: no previously-included files matching '*~' found under directory 'cheetah' warning: no previously-included files matching '*.aux' found under directory 'cheetah' warning: no previously-included files matching '*~' found under directory 'docs' warning: no previously-included files matching '*.aux' found under directory 'docs' Downloading/unpacking Markdown=2.0.1 (from Cheetah==2.4.4) Error urlopen error timed out while getting http://pypi.python.org/packages/source/M/Markdown/Markdown-2.2.1.zip#md5=67bd5f47864862708ede8eb85b1618ee (from http://pypi.python.org/simple/Markdown/) Exception: Traceback (most recent call last): File /usr/lib/python2.6/site-packages/pip/basecommand.py, line 124, in main self.run(options, args) File /usr/lib/python2.6/site-packages/pip/commands/install.py, line 178, in run requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle) File /usr/lib/python2.6/site-packages/pip/req.py, line 903, in prepare_files self.unpack_url(url, location, self.ishttp://self.is_download) File /usr/lib/python2.6/site-packages/pip/req.py, line 1008, in unpack_url return unpack_http_url(link, location, self.download_cache, only_download) File /usr/lib/python2.6/site-packages/pip/download.py, line 298, in unpack_http_url resp = _get_response_from_url(target_url, link) File /usr/lib/python2.6/site-packages/pip/download.py, line 327, in _get_response_from_url resp = urllib2.urlopen(target_url) File /usr/lib64/python2.6/urllib2.py, line 126, in urlopen return _opener.open(url, data, timeout) File /usr/lib64/python2.6/urllib2.py, line 391, in open response = self._open(req, data) File /usr/lib64/python2.6/urllib2.py, line 409, in _open '_open', req) File /usr/lib64/python2.6/urllib2.py, line 369, in _call_chain result = func(*args) File /usr/lib64/python2.6/urllib2.py, line 1190, in http_open return self.do_open(httplib.HTTPConnection, req) File /usr/lib64/python2.6/urllib2.py, line 1165, in do_open raise URLError(err) URLError: urlopen error timed out Storing complete log in /root/.pip/pip.log Error: Installation of pypi package 'Cheetah==2.4.4' failed! Success! Bootstrapped for CENTOS 6.3 Any suggestions ? On Wed, Mar 6, 2013 at 4:01 PM, Ashutosh Narayan aashutoshnara...@gmail.commailto:aashutoshnara...@gmail.com wrote: Hi Joshua, I followed the instructions mentioned in the link http://anvil.readthedocs.org you sent. When I excute ./smithy -av install, I encounter an error saying your system is not up to date. And it asks me to execute sudo smithy --bootstrap.When I run this command - I get a failure in installing Cheetah=2.4.4 form PyPi. Error: Installation of pypi package 'Cheetah==2.4.4' failed! How should I proceed now ? On Wed, Mar 6, 2013 at 12:17 PM, Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com wrote: http://anvil.readthedocs.org Sent from my really tiny device... On Mar 5, 2013, at 10:29 PM, Ashutosh Narayan aashutoshnara...@gmail.commailto:aashutoshnara...@gmail.com wrote: Can someone point me
Re: [Openstack] How to create a Image which can extend the partition automatically.
+1 I think cloud-init can do this all in a more correct manner and in a manner that works across more distributions and file system types in the long term. On 3/5/13 5:08 PM, Scott Moser smo...@ubuntu.com wrote: On Wed, 6 Mar 2013, Pádraig Brady wrote: On 03/05/2013 02:04 PM, Scott Moser wrote: On Tue, 5 Mar 2013, Sylvain Bauza wrote: If you look at http://www.pixelbeat.org/docs/openstack_libvirt_images/#1361764412, you'll see that resize2fs is performed. But there is a caveat with RHEL6 which is Linux 2.6 (contrary to Ubuntu 12.04 which is Linux 3.0). If you look at man resize2fs : Please don't do this, or rely on this. Having the hypervisor do this for your guest is simply wrong. Hypervisors should know little to nothing about the internals of the instances they're launching. Just to point out the alternative for when the VM doesn't have the smarts to resize itself, is to use something like virt-resize: https://blueprints.launchpad.net/nova/+spec/resize-no-raw Using libguestfs is worlds better than having the host directly do it. But really whats happening here is *still* something with very little information mucking with (and possibly breaking) the inside of a disk image. The more we consider that a bucket of bytes the better. If you are using libguestfs, chances are you're using it from your distro, and patching it to deal with a new filesystem (or otherwise get the update to your hostOS) is still problematic. Just let/expect the guest do it, and we'll avoid a whole silly game of cat and mouse that doesn't even have to be played. When was the last time you updated your bios on your laptop so you could use a new linux filesystem? That sounds silly, doesn't it. Having openstack reach inside the contents of the disk is just as silly. One of the major benefits of having cloud-init direct the partition resize *and* subsequent filesystem resize is that the user can (in user-data) disable this! (currently the initramfs doesn't take any instruction from user-data, so disabling it isn't really a possibility). perhaps they didn't want that extra instance-store space to be part of the root filesystem. If you put that function inside the hypervisor, you either can't do it, of you have to expose some silly api-launch parameter of do_not_modify_disk. It complicates the API and complicates the host. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [nova] python-novaclient 2.11.0 release
Can't this be fixed by configuring the underlying keyring config file to use a different backing storage? http://pypi.python.org/pypi/keyring#customize-your-keyring-by-config-file Sent from my really tiny device... On Feb 13, 2013, at 2:14 AM, Daniel P. Berrange berra...@redhat.commailto:berra...@redhat.com wrote: On Tue, Feb 12, 2013 at 09:41:11PM -0800, Vishvananda Ishaya wrote: Hello Everyone, I just pushed version 2.11.0 of python-novaclient to Pypi. There are a lot of fixes and features in this release. Here is a brief overview: Bug Fixes - simplified keyring support Sigh, this bug fix actually creates worse bugs we didn't appear to get the fix for it merged before you cut the release :-( With this change I have been unable to get nova client to work at all, outside of an X session when the gnome-keyring package installed, which is basically any default Fedora / GNOME install https://bugs.launchpad.net/python-novaclient/+bug/1116302 https://review.openstack.org/#/c/21690/ Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cirros 0.3.1 released
+2 Sent from my really tiny device... On Feb 8, 2013, at 6:35 PM, Scott Moser smo...@ubuntu.com wrote: Hi, I just thought I'd announce cirros 0.3.1 here. The biggest feature is config-drive-v2 support, but a more complete list is below. Download images at http://download.cirros-cloud.net/0.3.1 I'm interested in feedback on how they work. - move to buildroot 2012.05 (busybox 1.20.1) - support https via curl/openssl (LP: #918702) - ssh-import-id uses https directly to landscape.net - support mounting of vfat filesystems (LP: #929841) - support acpi shutdown (LP: #944151) - ec2metadata: remove double '/' in --public-keys requests (LP: #992492) - upgrade kernel to Ubuntu 12.04 LTS kernel (3.2.0-25.40) - mount first ephemeral device to /mnt in openstack instances (LP: #1022619) - ec2 data source has socket timeouts (10 second) when looking for metadata service (LP: #1022600) - wait longer than 3 seconds (60) for dhcp response on eth0 (LP: #918699) - support openstack config-drive-v2 data source (LP: #1097488) including insertion of interfaces(5) - support NoCloud data source, as supported by cloud-init (LP: #918375) Scott Moser ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] workflows in OpenStack?
Not sure about details but a single workflow like mechanism would provide a truly unified user experience which seems like a good goal. Sent from my really tiny device... On Jan 25, 2013, at 8:28 PM, Alex Glikson glik...@il.ibm.commailto:glik...@il.ibm.com wrote: Are there any additional details regarding the usage of workflows in Horizon? http://docs.openstack.org/developer/horizon/ref/workflows.html Is this something that can be also reused outside of Horizon, in the broader context of workflows for OpenStack? Thanks, Alex ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Is python-openstackclient dead?
I want to get around to making it better also, a simpler plugin supporting client is really needed. The individual clients have a ton of commands and are going to be a major point of confusion for users. I would almost like to have a nice client that is based on workflows (similar to horizon) that guides users instead of making them angry at the large amount of actions and options Sent from my iPhone On Dec 23, 2012, at 7:14 AM, Alessio Ababilov aababi...@griddynamics.com wrote: Hi everyone! There is a nice project in OpenStack - python-openstackclient. Its purpose is to provide a convenient command line interface to all OpenStack services, including nova. glance, and keystone, according to http://wiki.openstack.org/UnifiedCLI. I noticed that the last commit that added functionality to this project was on August 20, 2012. In current state, python-openstackclient implements only operations with nova instances and several keystone commands, so, the most of required functionality is not ready. In the same time, python-novaclient and python-keystoneclient still extend their command line interfaces - the latest commits were accepted in December, 2012. So, instead of moving CLI to python-openstackclient, it is developed in separate python-*client projects. Is python-openstackclient considered to be dead? Please, do not kill this project! Now it is really convenient, and all it needs is just more attention from the community that will add lacked functionality! Lets us drop CLI support from miscellaneous python-PROJECTclients and add it to python-openstackclient! -- Alessio Ababilov Software Engineer Grid Dynamics ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] New build dependency on keyring
At some point a clear-text password will show up, but that doesn't require said password to always be in clear-text. Think of a remote system that provides said passwords and authenticates the system asking for said password using some private/public key authentication that can be easily revoked (on machine comprise and such). Then u will get a closer view to why it'd be nice to have keys go through a API so that they can be gotten from other sources (to enable such a system to work). The plain-text case is an API, but it restricts it to the simplest one (only plain-text files), other companies (cough cough, yahoo) have different systems. On 12/12/12 9:26 PM, Sam Morrison sorri...@gmail.com wrote: Hi Ken, Yeah OK I agree it doesn't make it that much more complex as long as the dependancy is packaged in the distos which it is. I'm still a little confused though. If nova needs a clear text password to be able to talk to the DB for example then it's going to be needing to access this keyring somehow without human interaction to obtain the password. How does it do this? Sorry if I'm missing something obvious here. Cheers, Sam On 13/12/2012, at 10:16 AM, Ken Thomas k...@yahoo-inc.com wrote: The short answer is that it gives you extra security... if you wish to use it. If you're fine with relying on the file permission of nova.conf, glance.conf, etc. to keep any baddies from seeing the clear text passwords in there, then you're right, it doesn't give you anything. If, on the other hand, you have a large security group that nearly faints when they see clear text passwords, no matter what the file permission are, this allows you to move your password into an encrypted store of your choosing. Just specify a secure_source that implements KeyringBackend and you can be as secure as you wish. The main point is that you don't have to use it and the default behavior (don't specify a 'secure_source') will be that things behave exactly as before. The only real extra complexity is that we'd add an additional package (keyring) to the dependency list. As I mentioned originally, there's already some optional keyring usage in keystone client. It seems like we could have *less* complexity if it were a hard dependency instead of having the code check if the import worked or not. Ken On 12/12/2012 2:46 PM, Sam Morrison wrote: My question is what does this extra dependancy give us apart from extra complexity? I can't see any enhancement in security with this method? Cheers, Sam On 13/12/2012, at 4:44 AM, Ken Thomas k...@yahoo-inc.com wrote: Greetings all! I'm look into using keyring as a way to (optionally) remove clear text passwords from the various config files. (See https://blueprints.launchpad.net/oslo/+spec/pw-keyrings for details.) One of the comments I got back is that I should have the oslo build dependency on keyring be optional until a consensus is reached that it's okay to add it. I see that keystoneclient is already doing an import keyring and catching the exception if it's not there. I can certainly do something similar, but it seems like it would simplify things if we did just have keyring as a regular hard dependency. You don't have to use it, but it's there if you want it. So, is this the proper forum to bring this up? And if so, can we start the ball rolling to get a decision on getting that dependency approved? Thanks, Ken ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] New build dependency on keyring
+ Openstack-dev On 12/13/12 10:05 AM, Joshua Harlow harlo...@yahoo-inc.com wrote: At some point a clear-text password will show up, but that doesn't require said password to always be in clear-text. Think of a remote system that provides said passwords and authenticates the system asking for said password using some private/public key authentication that can be easily revoked (on machine comprise and such). Then u will get a closer view to why it'd be nice to have keys go through a API so that they can be gotten from other sources (to enable such a system to work). The plain-text case is an API, but it restricts it to the simplest one (only plain-text files), other companies (cough cough, yahoo) have different systems. On 12/12/12 9:26 PM, Sam Morrison sorri...@gmail.com wrote: Hi Ken, Yeah OK I agree it doesn't make it that much more complex as long as the dependancy is packaged in the distos which it is. I'm still a little confused though. If nova needs a clear text password to be able to talk to the DB for example then it's going to be needing to access this keyring somehow without human interaction to obtain the password. How does it do this? Sorry if I'm missing something obvious here. Cheers, Sam On 13/12/2012, at 10:16 AM, Ken Thomas k...@yahoo-inc.com wrote: The short answer is that it gives you extra security... if you wish to use it. If you're fine with relying on the file permission of nova.conf, glance.conf, etc. to keep any baddies from seeing the clear text passwords in there, then you're right, it doesn't give you anything. If, on the other hand, you have a large security group that nearly faints when they see clear text passwords, no matter what the file permission are, this allows you to move your password into an encrypted store of your choosing. Just specify a secure_source that implements KeyringBackend and you can be as secure as you wish. The main point is that you don't have to use it and the default behavior (don't specify a 'secure_source') will be that things behave exactly as before. The only real extra complexity is that we'd add an additional package (keyring) to the dependency list. As I mentioned originally, there's already some optional keyring usage in keystone client. It seems like we could have *less* complexity if it were a hard dependency instead of having the code check if the import worked or not. Ken On 12/12/2012 2:46 PM, Sam Morrison wrote: My question is what does this extra dependancy give us apart from extra complexity? I can't see any enhancement in security with this method? Cheers, Sam On 13/12/2012, at 4:44 AM, Ken Thomas k...@yahoo-inc.com wrote: Greetings all! I'm look into using keyring as a way to (optionally) remove clear text passwords from the various config files. (See https://blueprints.launchpad.net/oslo/+spec/pw-keyrings for details.) One of the comments I got back is that I should have the oslo build dependency on keyring be optional until a consensus is reached that it's okay to add it. I see that keystoneclient is already doing an import keyring and catching the exception if it's not there. I can certainly do something similar, but it seems like it would simplify things if we did just have keyring as a regular hard dependency. You don't have to use it, but it's there if you want it. So, is this the proper forum to bring this up? And if so, can we start the ball rolling to get a decision on getting that dependency approved? Thanks, Ken ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] New build dependency on keyring
+ The right openstack-dev, haha On 12/13/12 10:06 AM, Joshua Harlow harlo...@yahoo-inc.com wrote: + Openstack-dev On 12/13/12 10:05 AM, Joshua Harlow harlo...@yahoo-inc.com wrote: At some point a clear-text password will show up, but that doesn't require said password to always be in clear-text. Think of a remote system that provides said passwords and authenticates the system asking for said password using some private/public key authentication that can be easily revoked (on machine comprise and such). Then u will get a closer view to why it'd be nice to have keys go through a API so that they can be gotten from other sources (to enable such a system to work). The plain-text case is an API, but it restricts it to the simplest one (only plain-text files), other companies (cough cough, yahoo) have different systems. On 12/12/12 9:26 PM, Sam Morrison sorri...@gmail.com wrote: Hi Ken, Yeah OK I agree it doesn't make it that much more complex as long as the dependancy is packaged in the distos which it is. I'm still a little confused though. If nova needs a clear text password to be able to talk to the DB for example then it's going to be needing to access this keyring somehow without human interaction to obtain the password. How does it do this? Sorry if I'm missing something obvious here. Cheers, Sam On 13/12/2012, at 10:16 AM, Ken Thomas k...@yahoo-inc.com wrote: The short answer is that it gives you extra security... if you wish to use it. If you're fine with relying on the file permission of nova.conf, glance.conf, etc. to keep any baddies from seeing the clear text passwords in there, then you're right, it doesn't give you anything. If, on the other hand, you have a large security group that nearly faints when they see clear text passwords, no matter what the file permission are, this allows you to move your password into an encrypted store of your choosing. Just specify a secure_source that implements KeyringBackend and you can be as secure as you wish. The main point is that you don't have to use it and the default behavior (don't specify a 'secure_source') will be that things behave exactly as before. The only real extra complexity is that we'd add an additional package (keyring) to the dependency list. As I mentioned originally, there's already some optional keyring usage in keystone client. It seems like we could have *less* complexity if it were a hard dependency instead of having the code check if the import worked or not. Ken On 12/12/2012 2:46 PM, Sam Morrison wrote: My question is what does this extra dependancy give us apart from extra complexity? I can't see any enhancement in security with this method? Cheers, Sam On 13/12/2012, at 4:44 AM, Ken Thomas k...@yahoo-inc.com wrote: Greetings all! I'm look into using keyring as a way to (optionally) remove clear text passwords from the various config files. (See https://blueprints.launchpad.net/oslo/+spec/pw-keyrings for details.) One of the comments I got back is that I should have the oslo build dependency on keyring be optional until a consensus is reached that it's okay to add it. I see that keystoneclient is already doing an import keyring and catching the exception if it's not there. I can certainly do something similar, but it seems like it would simplify things if we did just have keyring as a regular hard dependency. You don't have to use it, but it's there if you want it. So, is this the proper forum to bring this up? And if so, can we start the ball rolling to get a decision on getting that dependency approved? Thanks, Ken ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Duplication of code in nova and cinder
Related to this, how do we in the future stop such code-copying from happening in the first place? Is it just that there needs to be a place for this (oslo?) that can be updated more quickly, or something similar? I'm always sorta 'weirded out' when people say that they copied some code in the name of 'it was quicker' or we are just 'borrowing it'. From: John Griffith john.griff...@solidfire.commailto:john.griff...@solidfire.com Date: Tuesday, December 11, 2012 3:36 PM To: Sam Morrison sorri...@gmail.commailto:sorri...@gmail.com Cc: OpenStack mailing list openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] Duplication of code in nova and cinder On Tue, Dec 11, 2012 at 4:24 PM, Sam Morrison sorri...@gmail.commailto:sorri...@gmail.com wrote: I attempted to create a volume from an image in cinder and was getting this strange error, turns out it was because I had my glance servers specified as https://glanceserver:9292 In cinder the version of images/glance.py is older than the one in nova and is missing the ssl support additions. https://bugs.launchpad.net/cinder/+bug/1089147 My real question is why is there one version is nova and one version in cinder. I also think there is quite a bit more unnecessary duplication. Should it all go into oslo? Cheers, Sam ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Hi Sam, Short answer is yes. Need to check scoping etc and make sure that it does in fact fit within the parameters of OSLO. It's something I thought of a couple weeks ago but to be honest it's been low on my list personally and nobody else that I know of has shown an interest in picking it up. You'll notice another image related item we're *borrowing* from Nova (cinder.image.image_utils). In both cases there are slight modifications to fit Cinder's use case that given a bit of work could easily be shared. John ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Duplication of code in nova and cinder
Isn't that a lets fix the slowness instead of continue bad behavior. Fix the root problem and don't bypass it in the first place? Then said root problem is solved for everyone and isn't pushed into the future (as is typically done). On 12/11/12 5:31 PM, Huang Zhiteng winsto...@gmail.com wrote: On Wed, Dec 12, 2012 at 8:56 AM, Joshua Harlow harlo...@yahoo-inc.com wrote: Related to this, how do we in the future stop such code-copying from happening in the first place? Is it just that there needs to be a place for this (oslo?) that can be updated more quickly, or something similar? I'm always sorta 'weirded out' when people say that they copied some code in the name of 'it was quicker' or we are just 'borrowing it'. But you have to admit 'it is quicker'. My experience is getting common code into Oslo and then port to project usually take two times longer time to merge, in best case. From: John Griffith john.griff...@solidfire.com Date: Tuesday, December 11, 2012 3:36 PM To: Sam Morrison sorri...@gmail.com Cc: OpenStack mailing list openstack@lists.launchpad.net Subject: Re: [Openstack] Duplication of code in nova and cinder On Tue, Dec 11, 2012 at 4:24 PM, Sam Morrison sorri...@gmail.com wrote: I attempted to create a volume from an image in cinder and was getting this strange error, turns out it was because I had my glance servers specified as https://glanceserver:9292 In cinder the version of images/glance.py is older than the one in nova and is missing the ssl support additions. https://bugs.launchpad.net/cinder/+bug/1089147 My real question is why is there one version is nova and one version in cinder. I also think there is quite a bit more unnecessary duplication. Should it all go into oslo? Cheers, Sam ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Hi Sam, Short answer is yes. Need to check scoping etc and make sure that it does in fact fit within the parameters of OSLO. It's something I thought of a couple weeks ago but to be honest it's been low on my list personally and nobody else that I know of has shown an interest in picking it up. You'll notice another image related item we're *borrowing* from Nova (cinder.image.image_utils). In both cases there are slight modifications to fit Cinder's use case that given a bit of work could easily be shared. John ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Regards Huang Zhiteng ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nova metadata service
Its really just a binary that activates https://github.com/openstack/nova/blob/master/nova/api/metadata/handler.py#L100 Its a way to allow for a VM to get metadata about itself and any userdata (of which users may have provided) on boot. Said feature is not just connected to ec2, but provides a generic mechanism for getting this type of data to an instance. The ec2 folks I believe are just the 'originators' of said concept and that’s how it got named initially. From: JuanFra Rodriguez Cardoso juanfra.rodriguez.card...@gmail.commailto:juanfra.rodriguez.card...@gmail.com Date: Monday, December 10, 2012 5:10 PM To: Openstack openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: [Openstack] Nova metadata service Hi guys! After looking for in the mailing list and docs, I honestly still don't understand what really is nova-api-metadata. it's a mandatory service in a multi-host deployment? it's only related to EC2? Thanks! Regards, JuanFra. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] How to deploy openstack automatically in your env.
For RHEL. Although its not made for such big purposes as puppet/chef there is the anvil project. http://anvil.readthedocs.org/en/latest/ http://anvil.readthedocs.org/en/latest/topics/dev_notes/architecture.html Its been used at yahoo with RHEL 6.2+ for a while now, not as our deployment engine but the thing we use pre-deployment to do dev work, run unit tests, build rpm packages and then handoff to our deployment engine (internal tech). Try it out, complaints/comments welcome ;) On 12/3/12 7:19 AM, Jonathan Proulx j...@csail.mit.edu wrote: On Mon, Dec 03, 2012 at 06:21:54PM +0800, Lei Zhang wrote: :It is a wired thing that the openstack is a python project. But many tools :for it are build on ruby? Puppet (http://puppetlabs.com/) and Chef (http://www.opscode.com/chef), the main players in configuration management are both written in Ruby and manage more than just OpenStack so the code just isn't related I can't be much help on your origina RHEL question as I run a Debian / Ubuntu shop. But while it is true that deployment tools are better tested on Ubuntu, certainly any puppe tor chef based solutions *should* work on RHEL if they don't it would be worth reporting their failures to the developers of those tools. There is a Fedora OpenStack wiki at http://fedoraproject.org/wiki/OpenStack which is linked from http://www.openstack.org/software/start/ which has other RedHat family specific links at the bottom of the page -Jon : : :On Mon, Dec 3, 2012 at 5:44 PM, Joe Breu joseph.b...@rackspace.com wrote: : : Hi Lei, : : We have chef cookbooks to install Openstack located at : http://github.com/rcbops/chef-cookbooks. : : --- : Joseph Breu : Deployment Engineer : Rackspace Private Cloud : 210-312-3508 : : On Dec 3, 2012, at 9:15 AM, Lei Zhang wrote: : : Hi all, : : I search the internet for days and found several automatically tool. : Including : :- devstack :- puppet+pupet-openstack :- stackops :- OneStack : : But It seems that all the scripts are well tested on ubuntu not RHEL. How : could you guys to deploy the openstack automatically, especially on RHEL. : -- : Lei Zhang : : Blog: http://jeffrey4l.github.com : twitter/weibo: @jeffrey4l : : ___ : Mailing list: https://launchpad.net/~openstack : Post to : openstack@lists.launchpad.net : Unsubscribe : https://launchpad.net/~openstack : More help : https://help.launchpad.net/ListHelp : : : : : :-- :Lei Zhang : :Blog: http://jeffrey4l.github.com :twitter/weibo: @jeffrey4l :___ :Mailing list: https://launchpad.net/~openstack :Post to : openstack@lists.launchpad.net :Unsubscribe : https://launchpad.net/~openstack :More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] nova-manage service list weirdo !
This brings up some ideas I think others are having (vish I think mentioned it last night). Would it be better to base when a service is up or down by something not time based? Something maybe connection based, or possibly something not using absolute times (so that NTP sync doesn't matter). Possibly this is getting worked on already, but can't remember, haha. From: JuanFra Rodriguez Cardoso juanfra.rodriguez.card...@gmail.commailto:juanfra.rodriguez.card...@gmail.com Date: Friday, November 30, 2012 2:49 AM To: Skible OpenStack skible.openst...@gmail.commailto:skible.openst...@gmail.com, Openstack openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] nova-manage service list weirdo ! Ok. No problem! Be careful with time synchronization. It's one of most common problem in distributed environments. I guess your scenario is like below: - Controller Node (Synchronizing from external NTP servers and at the same time it acts as NTP server to compute nodes) - Compute Nodes (Synchronizing from controller node) Regards, JuanFra. 2012/11/30 Skible OpenStack skible.openst...@gmail.commailto:skible.openst...@gmail.com Hi Juan, its related to my ntp server, it was not configured properly. Thanks ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Verification of Keystone Installation fails
I think the overall issue is connected to https://bugs.launchpad.net/keystone/+bug/962600 Right? Seems like that is still happening :-( From: Ahmed Al-Mehdi ah...@coraid.commailto:ah...@coraid.com Date: Wednesday, October 31, 2012 1:15 PM To: Dolph Mathews dolph.math...@gmail.commailto:dolph.math...@gmail.com Cc: Joseph Heck joe.h...@nebula.commailto:joe.h...@nebula.com, openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] Verification of Keystone Installation fails Hi Dolph, Awesome, that worked. Thank you very much. Just out of curiosity, what was the exact conflict? Between which environment variable and option passed to the CLI? Regards, Ahmed. From: Dolph Mathews dolph.math...@gmail.commailto:dolph.math...@gmail.com Date: Wednesday, October 31, 2012 10:46 AM To: Ahmed Al-Mehdi ah...@coraid.commailto:ah...@coraid.com Cc: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net, Joseph Heck joe.h...@nebula.commailto:joe.h...@nebula.com Subject: Re: [Openstack] Verification of Keystone Installation fails I was able to reproduce by defining SERVICE_ENDPOINT and SERVICE_TOKEN in my own environment, which appear to be overriding the credentials provided on the CLI -- I don't think that's the intended behavior. If you unset them, you should be able to verify the install. If you skip verifying keystone and something is wrong with it, you'll likely find out pretty quick when another service calls keystone for the first time :) -Dolph On Wed, Oct 31, 2012 at 12:22 PM, Ahmed Al-Mehdi ah...@coraid.commailto:ah...@coraid.com wrote: Hi Dolph, Thank you very much for helping me on this issue. Following is the environment variables related to openstack: root@bodega:~# env | egrep OS_|SERVICE_ SERVICE_ENDPOINT=http://10.176.20.158:35357/v2.0/ SERVICE_TOKEN=012345SECRET99TOKEN012345 root@bodega:~# ifconfig eth0 eth0 Link encap:Ethernet HWaddr 00:1e:67:06:1b:67 inet addr:10.176.20.158 Bcast:10.176.255.255 Mask:255.255.0.0 inet6 addr: fe80::21e:67ff:fe06:1b67/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:12760203 errors:0 dropped:0 overruns:0 frame:0 TX packets:203944 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:1044985224 (1.0 GB) TX bytes:22642912 (22.6 MB) Interrupt:16 Memory:b200-b202 root@bodega:~# I am attaching keystone.conf file. Would you happen to know if there is a high level document document on keystone (more than just a user guide, but a architectural/functional doc, but not a API doc). Something similar to http://docs.openstack.org/trunk/openstack-identity/admin/os-identity-starter-guide-trunk.pdf but updated. Does my current issue prohibit me from progressing forward with the next steps in the install document, setting up glance, nova, etc.? Regards, Ahmed. From: Dolph Mathews dolph.math...@gmail.commailto:dolph.math...@gmail.com Date: Wednesday, October 31, 2012 9:44 AM To: Ahmed Al-Mehdi ah...@coraid.commailto:ah...@coraid.com Cc: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] Verification of Keystone Installation fails The error you're seeing is actually client-side, so there won't be anything in keystone's logs. It indicates that you're not actually authenticating with keystone (and instead bypassing authentication using --token and --endpoint, for example) ... however, that's obviously not the case, as you're explicitly providing --os-username, etc. Unfortunately, I'm not able to reproduce this issue. Can you share your OS_* environment variables? I suspect something there is unexpectedly overriding what you're providing on the CLI... which would be a legitimate bug. Thanks, -Dolph On Wed, Oct 31, 2012 at 2:08 AM, Ahmed Al-Mehdi ah...@coraid.commailto:ah...@coraid.com wrote: Hello, I followed the steps in the OpenStack Install Deploy for Ubuntu manual to install Keystone. However, when I issue the commands in section Verifying the Identity Service Installation ( http://docs.openstack.org/trunk/openstack-compute/install/apt/content/verifying-identity-install.html ), I am getting the following error: # keystone --os-username=admin --os-password=admin --os-auth-url=http://10.176.20.158:35357/v2.0 token-get 'Client' object has no attribute 'service_catalog' I don't see any additional info in keystone.log. Can someone please help me. Thank you, Ahmed. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help :
Re: [Openstack] Handling of adminPass is arguably broken (essex)
Just fyi, the cloud-init format 'spec' has something similar that bypasses the file injection (which is a bad/insecure/incompatible concept that needs to be gotten rid of imho) by having the following syntax it understands: http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc /examples/cloud-config-user-groups.txt Is there anyway a windows version of cloud-init could be done, either ported, or patched, or a service like cloud-init could be added to windows images (using a startup program in the windows image that could just be a call-out to a python interpreter or something different...). In the linux image world it is pretty common to have a version of cloud-init be bootstrapped into the image so it would seem like windows could be similar. On 10/31/12 6:09 PM, Lars Kellogg-Stedman l...@seas.harvard.edu wrote: TL;DR: The way OpenStack handles the adminPass attribute during metadata injection is not useful on operating systems without an /etc/passwd and /etc/shadow. I would like to make the adminPass value available on a Windows instance, and this is my proposal for how to do it. I've been putting together a Windows 2008 server image for deploying in our OpenStack environment. I started by setting it up to act just like our Linux images: - It has sshd running as a service via Cygwin - It runs a script at startup to pull an ssh key from the metadata server for the Administrator account. This works great! But I've had some push-back from folks who argue that this process won't be familiar to a typical Windows administrator, so I started trying to figure how to get an administrator password either (a) into the instance from the person creating it, or (b) back to the person creating the instance after generating the password. (a) is relatively easy to do via the user-data attribute, and I have a prototype of that working. However... One of my colleagues mention that there was some mechanism for injecting passwords into instances -- which sounds perfect. Based on my efforts in #openstack, it appears that very few people take advantage of this feature or even know how it operates, so I went diving through the code and eventually found myself in nova/virt/disk/api.py, where I discovered that even with config_drive=True, nova will attempt to copy /etc/passwd and /etc/shadow (which don't exist) off the config drive to modify them locally. This obviously fails, leaving the admin password inaccessible. I would like to propose that if config_drive=True, that the admin password simply get written into a file, where it could be used by operating systems without an /etc/passwd or /etc/shadow file. If this sounds like a good idea, I'll work up a patch. It seems that for this to work, inject_data needs to know whether or not it's targeting a config_drive or an actual partition...and so does inject_data_into_fs. Maybe something like: In virt/libvirt/connection.py: disk.inject_data(injection_path, key, net, metadata, admin_pass, files, partition=target_partition, use_cow=FLAGS.use_cow_images, config_drive=config_drive) And in virt/disk/api.py: def inject_data(image, key=None, net=None, metadata=None, admin_password=None, files=None, partition=None, use_cow=False, config_drive=False): ... inject_data_into_fs(img.mount_dir, key, net, metadata, admin_password, files, config_drive=False) ... def inject_data_into_fs(fs, key, net, metadata, admin_password, files, config_drive=False): ... if admin_password: if config_drive: _inject_admin_password_as_file_into_fs(admin_password, fs) else: _inject_admin_password_into_fs(admin_password, fs) Thoughts? -- Lars Kellogg-Stedman l...@seas.harvard.edu | Senior Technologist | http://ac.seas.harvard.edu/ Academic Computing| http://code.seas.harvard.edu/ Harvard School of Engineering | and Applied Sciences| ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup
Sure, that would make sense, lets see where the next meeting takes us. Nothing is ever in stone when its software :-P On 10/26/12 3:08 AM, Doug Hellmann doug.hellm...@dreamhost.com wrote: On Oct 25, 2012, at 9:44 PM, Joshua Harlow harlo...@yahoo-inc.com wrote: As for statgen, I think that¹s just a temp repo, it'd be nice to have the end result of this be a library that provides somewhat generic metrics and plugins and such so that stacktech could use the outputs of it, ceilometer could the outputs and other systems could use the outputs (where an output goes would be configurable so that each system can configure its outputs as the operator desires, ie I want my MONITOR metrics to go to MQ in ceilomter and stacktech consumable formats, or to files or to...). I think when this gets going we should have some repo/project name that goes on stackforge so that even while this is being developed it can go through the normal review process and such (and not in someones special github repo). But we have to start somewhere, something we can discuss as to what a good solution is @ the meeting. At the summit, as part of the discussions around expanding the scope of ceilometer to cover measurement for monitoring, we discussed developing the library as part of ceilometer for now and either moving it to Oslo for release as a library or just releasing the library as a separate package from the ceilometer project directly. Doug -Josh On 10/25/12 5:47 PM, Jeffrey Budzinski jeffr...@yahoo-inc.com wrote: Yes, I think support for metrics objects that can be leveraged both by monkey patches and decorators was what we'd been thinking along the lines of. The metrics would be controlled via config both in what scopes are active (e.g. on|off for a package, module, etc.) and also the outlet for the metrics (file, datagram, rpc). Support for instrumentation levels would also be nice so that metric flow could be controlled (i.e. instrumentation point is annotated as METRIC, MONITOR, PROFILE and then level to actually emit can be set in conjunction with a metric outlet/sink). With this approach, folks could control both the volume of metrics and also the outlet for the metrics. Ceilometer would also be an outlet that could be leveraged via RPC flow for metrics -- though I'd expect some would want to send via datagram to statsd or file for offline log aggregation. I'll post a diagram tomorrow for review and comment. Oh btw, I updated the spec with most of what was in the etherpad. We can update the spec to reflect whatever we decide is the preferred approach. -jeff On Oct 25, 2012, at 5:30 PM, Angus Salkeld wrote: On 25/10/12 11:13 +, Sandy Walsh wrote: grizzly-common-instrumentation seems to be the best choice ... hopefully the other groups will use this etherpad too. We need a proper blueprint to nail down the approach. IRC is great, but doesn't retain history for other groups. I think we need to get a plan for translating the etherpad into something concise and nailed down. Agree. statgen should really just be a new notifier in Tach (or Scrutinize) ... vs copy-pasting the code into yet-another repo. Hopefully that's the plan? Tach should remain a generic tool and not pegged to OpenStack. Well that was just an ideas play pen not serious code. I might be coming at this from a slightly different angle... I was looking at a library that can be used to generate trace, monitoring and metering data (kind of like log levels for logging). Currently both Tach and Scrutinize don't have enough fields (of course that could be changed). Also I think we should be able to insert instrumentation into the code as well as have the function level performance metrics monkey patched. Then we could have a config that directed metric data to different notifiers like how you do it in Scrutinize perhaps. Also enforcing data rate limits and possible data aggregation could be neat configurable features. Anyway more at the meeting... -Angus -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Angus Salkeld [asalk...@redhat.com] Sent: Thursday, October 25, 2012 1:00 AM To: openstack@lists.launchpad.net Subject: Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup On 24/10/12 23:35 +, Sandy Walsh wrote: Hey y'all, Great to chat during the summit last week, but it's been a crazy few days of catch-up since then. The main takeaway for me was the urgent need to get some common libraries under these efforts. Yip. So, to that end ... 1. To those that asked, I'm going to get my slides / video presentation made available via the list. Stay tuned. 2. I'm having a hard time following all the links to various efforts going on (seems
Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup
As for statgen, I think that¹s just a temp repo, it'd be nice to have the end result of this be a library that provides somewhat generic metrics and plugins and such so that stacktech could use the outputs of it, ceilometer could the outputs and other systems could use the outputs (where an output goes would be configurable so that each system can configure its outputs as the operator desires, ie I want my MONITOR metrics to go to MQ in ceilomter and stacktech consumable formats, or to files or to...). I think when this gets going we should have some repo/project name that goes on stackforge so that even while this is being developed it can go through the normal review process and such (and not in someones special github repo). But we have to start somewhere, something we can discuss as to what a good solution is @ the meeting. -Josh On 10/25/12 5:47 PM, Jeffrey Budzinski jeffr...@yahoo-inc.com wrote: Yes, I think support for metrics objects that can be leveraged both by monkey patches and decorators was what we'd been thinking along the lines of. The metrics would be controlled via config both in what scopes are active (e.g. on|off for a package, module, etc.) and also the outlet for the metrics (file, datagram, rpc). Support for instrumentation levels would also be nice so that metric flow could be controlled (i.e. instrumentation point is annotated as METRIC, MONITOR, PROFILE and then level to actually emit can be set in conjunction with a metric outlet/sink). With this approach, folks could control both the volume of metrics and also the outlet for the metrics. Ceilometer would also be an outlet that could be leveraged via RPC flow for metrics -- though I'd expect some would want to send via datagram to statsd or file for offline log aggregation. I'll post a diagram tomorrow for review and comment. Oh btw, I updated the spec with most of what was in the etherpad. We can update the spec to reflect whatever we decide is the preferred approach. -jeff On Oct 25, 2012, at 5:30 PM, Angus Salkeld wrote: On 25/10/12 11:13 +, Sandy Walsh wrote: grizzly-common-instrumentation seems to be the best choice ... hopefully the other groups will use this etherpad too. We need a proper blueprint to nail down the approach. IRC is great, but doesn't retain history for other groups. I think we need to get a plan for translating the etherpad into something concise and nailed down. Agree. statgen should really just be a new notifier in Tach (or Scrutinize) ... vs copy-pasting the code into yet-another repo. Hopefully that's the plan? Tach should remain a generic tool and not pegged to OpenStack. Well that was just an ideas play pen not serious code. I might be coming at this from a slightly different angle... I was looking at a library that can be used to generate trace, monitoring and metering data (kind of like log levels for logging). Currently both Tach and Scrutinize don't have enough fields (of course that could be changed). Also I think we should be able to insert instrumentation into the code as well as have the function level performance metrics monkey patched. Then we could have a config that directed metric data to different notifiers like how you do it in Scrutinize perhaps. Also enforcing data rate limits and possible data aggregation could be neat configurable features. Anyway more at the meeting... -Angus -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Angus Salkeld [asalk...@redhat.com] Sent: Thursday, October 25, 2012 1:00 AM To: openstack@lists.launchpad.net Subject: Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup On 24/10/12 23:35 +, Sandy Walsh wrote: Hey y'all, Great to chat during the summit last week, but it's been a crazy few days of catch-up since then. The main takeaway for me was the urgent need to get some common libraries under these efforts. Yip. So, to that end ... 1. To those that asked, I'm going to get my slides / video presentation made available via the list. Stay tuned. 2. I'm having a hard time following all the links to various efforts going on (seems every time I turn around there's a new metric/instrumentation effort, which is good I guess :) Here is some fun I have been having with a bit of tach+ceilometer code. https://github.com/asalkeld/statgen Is there a single location I can place my feedback? If not, should we create one? I've got lots of suggestions/ideas and would hate to have to duplicate the threads or leave other groups out. I'll add some links here that I am aware of: https://bugs.launchpad.net/ceilometer/+bug/1071061 https://etherpad.openstack.org/grizzly-common-instrumentation https://etherpad.openstack.org/grizzly-ceilometer-actions
Re: [Openstack] Nova services not running
Check the DB, also check your NTPD and the log files/screen session that these are running in. I've seen XXX often when time is out of sync (the health is based off a 'last seen' time) From: Johannes Baltimore johannes.b...@gmail.commailto:johannes.b...@gmail.com Date: Tuesday, October 23, 2012 5:00 PM To: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] Nova services not running Hello, anyone? 2012/10/22 Johannes Baltimore johannes.b...@gmail.commailto:johannes.b...@gmail.com Umm sorry for the double mail. I had thought that the earlier one hadn't been sent. 2012/10/22 Johannes Baltimore johannes.b...@gmail.commailto:johannes.b...@gmail.com Hello. I've been facing some trouble while installing OpenStack. After I started following the steps on a tutorial to install the nova modules, I tried to see if the services were running, and the command nova-manage service list shows me the following: # nova-manage service list 2012-10-22 10:44:20 DEBUG nova.utils [req-dc8e24ca-6b15-413e-8966-fcde305d6d82 None None] backend module 'nova.db.sqlalchemy.api' from '/opt/stack/nova/nova/db/sqlalchemy/api.pyc' from (pid=14851) __get_backend /opt/stack/nova/nova/utils.py:502 Binary Host Zone Status State Updated_At nova-consoleauth ubuntu-cloud2nova enabled XXX None nova-certubuntu-cloud2nova enabled XXX None nova-scheduler ubuntu-cloud2nova enabled XXX None Here's my /etc/nova/nova.conf: http://pastebin.com/9FMaMEuV Thanks. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] how to deal with failed compute node
What about temporary failures in the nova-compute actually being working but just not reporting. Something like network separations (oops I disconnected the switch...) where both are still working but just unaware of each other. If nova-api starts cleaning up the running instance that might be slightly bad right? Since it is actually still running and useable. I'd rather almost stick to the more pre-cautious 'state' where either it can be manually deleted (by the user) via something like a 'force remove'. Having auto-remove being triggered by multiple components (somewhat partially it seems) on service 'aliveness' seems risky in times where vm's are still working+running but a service has become disconnected temporarily. Shouldn't whichever VM's are 'alive' continue to be 'alive' even if a nova service dies, I would suspect customers wouldn't want there VM lifetime connected to some external component (in this case nova-computes up/downtime). On 10/17/12 11:23 PM, gtt116 gtt...@126.com wrote: Hi guys, Today, when terminate an instance, nova-api will check whether nova-compute service is alive. If nova-compute is dead, nova-api just delete the instance from the database, but do not release the fixed-ip, floating-ip, volumes, etc. If the failed nova-compute start again, it will found the erroneously running instance, and do cleanup. But before the nova-compute started, the resource that dead vm associated can not be used. like fixed-ip can not be associated to another vm. So I found a method to quickly clean these resource. If nova-api find nova-compute is dead. Then it find another nova-compute that is alive. Although the alive nova-compute is not the real host of vm. It can clean the resource in database, even the network by make rpc call to nova-network. maybe some exception it will raise. But that works. What do you think about this? why do we have a lot of nova-compute, nova-network? I think one reason is when one node failed, another can do some work for it. Best regards, gtt ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Performance
And of course: http://summit.openstack.org/cfp/details/93 That seems pretty much the same as what u are doing :-) On 10/14/12 8:36 AM, Michael Still michael.st...@canonical.com wrote: On 10/14/2012 03:06 PM, Sriram Subramanian wrote: Frans: Does the report you are talking about has any info on OpenStack performance? Is it sharable? One of the sessions during the summit next week aims at brainstorming ways to do performance and scalability testing for OpenStack. http://summit.openstack.org/cfp/details/37 This sounds like an interesting session. I will try and make it. I was chasing a performance problem the other day that turned out to be a locking issue. In the process I had to add a bunch of instrumentation to nova, and I'd like to see some of that land in the code base. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Bottleneck of message queue
I haven't tried it but this might be something 'similar' http://www.rabbitmq.com/management.html From: Hao Wang hao.1.w...@gmail.commailto:hao.1.w...@gmail.com Date: Thursday, October 11, 2012 6:20 AM To: Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com Cc: Sandy Walsh sandy.wa...@rackspace.commailto:sandy.wa...@rackspace.com, ale...@rabbitmq.commailto:ale...@rabbitmq.com ale...@rabbitmq.commailto:ale...@rabbitmq.com, openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] Bottleneck of message queue Hi Sandy, I couldn't access your links. Originally I though it's my laptop problem but it is not. It's not a good news for both of us that it seems your website is now being blocked by China GFW. :) Have you posted the same articles on other sites? Sorry for the inconvenience. Thanks. Hi Josh, You're absolutely right. It's impossible to root cause issues without true data. In the meantime, your point reminds me a question, is there a way for rabbitmq to show online statistics of messages, like the output of command netstat -i to show the statistics of the in and out number of packets for network interfaces? Regards, Howard On Thu, Oct 11, 2012 at 1:53 AM, Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com wrote: Also some real data/graphs/metrics if u have anything would go a long way in helping others see the problem. Without data though its hard to know what is broke, what is the limit, and what needs to be fixed. -Josh From: Hao Wang hao.1.w...@gmail.commailto:hao.1.w...@gmail.com Date: Wednesday, October 10, 2012 6:50 AM To: Sandy Walsh sandy.wa...@rackspace.commailto:sandy.wa...@rackspace.com, ale...@rabbitmq.commailto:ale...@rabbitmq.com ale...@rabbitmq.commailto:ale...@rabbitmq.com Cc: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] Bottleneck of message queue Thanks guys for your insightful replies. I'm studying them. If I've got anything, I'll get back to you as soon as possible. Have a nice day! Cheers, Howard On Wed, Oct 10, 2012 at 7:25 PM, Sandy Walsh sandy.wa...@rackspace.commailto:sandy.wa...@rackspace.com wrote: Hey Howard, Queues are generally in memory, but you may turn on persistent (disk) queues in your environment. So that's your limitation. Having rabbitmq on a different server is a good idea. Also, Queues are only used for control, not user data, so they shouldn't be that big of a burden. Having a queue-based architecture adds some complexity for synchronization, but their benefit of giving us burst-handling capabilities far outweigh that (imho). If your queues are filling up, you may: 1. need beefier machines processing the offending queues (or rabbit server) 2. need to add more worker nodes (more network, more scheduler, though more compute isn't appropriate) 3. think about clustering rabbit Notifications are perhaps the chattiest queues in the system, so make sure you have suitable workers there (if you have notifications turned on) This might help you understand the flow through the queues a little more? http://www.sandywalsh.com/2012/04/openstack-nova-internals-pt1-overview.html http://www.sandywalsh.com/2012/09/openstack-nova-internals-pt2-services.html Cheers, Sandy From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.netmailto:rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.netmailto:rackspace@lists.launchpad.net] on behalf of Hao Wang [hao.1.w...@gmail.commailto:hao.1.w...@gmail.com] Sent: Tuesday, October 09, 2012 11:49 PM To: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: [Openstack] Bottleneck of message queue Hi guys, I am trying to figure out how the internal interaction processes within different modules of OpenStack. Frankly speaking, while I'm reading the source codes I lost myself and have to jump out again to look at OpenStack from out of the box. I don't know if anybody has the similiar feeling with me. Is there any picture I can follow to see the message flows? OpenStack is based on message queue to ensure the expansion easy. Here come my questions. Does anybody know the capacity of message queue? Would the capacity be a bottleneck of the platform? Thanks, Howard ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] A simple guide to install OpenStack Folsom
You guys should also consider the 'anvil' way of doing this (pure python baby, haha). Which is improved from lorin's and has been working for yahoo! for a while now. https://github.com/yahoo/Openstack-Anvil/blob/master/anvil/components/helpe rs/keystone.py#L25 Please feel free to take the code!! Its only 'real' dependency is the keystone client + yaml parsing... On 10/10/12 2:23 AM, Alan Pevec ape...@gmail.com wrote: On Wed, Oct 10, 2012 at 11:10 AM, Skible OpenStack skible.openst...@gmail.com wrote: I am counting on our your feedback to enhance my work and contribute it to the OpenStack Eco System. I wonder about https://github.com/mseknibilel/OpenStack-Folsom-Install-guide/tree/master/ Scripts which say: # Mainly inspired by https://github.com/openstack/keystone/blob/master/tools/sample_data.sh Why not submit that as an improvement to Keystone? I'd like to propose consolidation of all keystone initialization scripts around (Keyston's sample_data.sh, Devstack's keystone_data.sh, scripts like yours) and move to Lorin's YAML config (see https://lists.launchpad.net/openstack/msg17204.html) I'm just not sure yet if additional dependency on YAML is worth it. Cheers, Alan ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Bottleneck of message queue
Also some real data/graphs/metrics if u have anything would go a long way in helping others see the problem. Without data though its hard to know what is broke, what is the limit, and what needs to be fixed. -Josh From: Hao Wang hao.1.w...@gmail.commailto:hao.1.w...@gmail.com Date: Wednesday, October 10, 2012 6:50 AM To: Sandy Walsh sandy.wa...@rackspace.commailto:sandy.wa...@rackspace.com, ale...@rabbitmq.commailto:ale...@rabbitmq.com ale...@rabbitmq.commailto:ale...@rabbitmq.com Cc: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] Bottleneck of message queue Thanks guys for your insightful replies. I'm studying them. If I've got anything, I'll get back to you as soon as possible. Have a nice day! Cheers, Howard On Wed, Oct 10, 2012 at 7:25 PM, Sandy Walsh sandy.wa...@rackspace.commailto:sandy.wa...@rackspace.com wrote: Hey Howard, Queues are generally in memory, but you may turn on persistent (disk) queues in your environment. So that's your limitation. Having rabbitmq on a different server is a good idea. Also, Queues are only used for control, not user data, so they shouldn't be that big of a burden. Having a queue-based architecture adds some complexity for synchronization, but their benefit of giving us burst-handling capabilities far outweigh that (imho). If your queues are filling up, you may: 1. need beefier machines processing the offending queues (or rabbit server) 2. need to add more worker nodes (more network, more scheduler, though more compute isn't appropriate) 3. think about clustering rabbit Notifications are perhaps the chattiest queues in the system, so make sure you have suitable workers there (if you have notifications turned on) This might help you understand the flow through the queues a little more? http://www.sandywalsh.com/2012/04/openstack-nova-internals-pt1-overview.html http://www.sandywalsh.com/2012/09/openstack-nova-internals-pt2-services.html Cheers, Sandy From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.netmailto:rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.netmailto:rackspace@lists.launchpad.net] on behalf of Hao Wang [hao.1.w...@gmail.commailto:hao.1.w...@gmail.com] Sent: Tuesday, October 09, 2012 11:49 PM To: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: [Openstack] Bottleneck of message queue Hi guys, I am trying to figure out how the internal interaction processes within different modules of OpenStack. Frankly speaking, while I'm reading the source codes I lost myself and have to jump out again to look at OpenStack from out of the box. I don't know if anybody has the similiar feeling with me. Is there any picture I can follow to see the message flows? OpenStack is based on message queue to ensure the expansion easy. Here come my questions. Does anybody know the capacity of message queue? Would the capacity be a bottleneck of the platform? Thanks, Howard ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] A simple guide to install OpenStack Folsom
That sounds great to me. I can help out in converting this code into that code. It seems like a trivial kind of thing to do, what format would that command take, a yaml file? Something similar to https://github.com/yahoo/Openstack-Anvil/blob/master/conf/templates/keystone/init_what.yaml maybe, idk. From: Dolph Mathews dolph.math...@gmail.commailto:dolph.math...@gmail.com Date: Wednesday, October 10, 2012 11:13 AM To: Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com Cc: Alan Pevec ape...@gmail.commailto:ape...@gmail.com, Skible OpenStack skible.openst...@gmail.commailto:skible.openst...@gmail.com, openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] A simple guide to install OpenStack Folsom I'd like to simplify the scope of sample_data.sh to the absolute bare minimum (service tenant, admin role, admin user, identity service/endpoints, etc), and integrate it into keystone-manage as a 'bootstrap' command: $ keystone-manage bootstrap -Dolph On Wed, Oct 10, 2012 at 12:34 PM, Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com wrote: You guys should also consider the 'anvil' way of doing this (pure python baby, haha). Which is improved from lorin's and has been working for yahoo! for a while now. https://github.com/yahoo/Openstack-Anvil/blob/master/anvil/components/helpe rs/keystone.py#L25 Please feel free to take the code!! Its only 'real' dependency is the keystone client + yaml parsing... On 10/10/12 2:23 AM, Alan Pevec ape...@gmail.commailto:ape...@gmail.com wrote: On Wed, Oct 10, 2012 at 11:10 AM, Skible OpenStack skible.openst...@gmail.commailto:skible.openst...@gmail.com wrote: I am counting on our your feedback to enhance my work and contribute it to the OpenStack Eco System. I wonder about https://github.com/mseknibilel/OpenStack-Folsom-Install-guide/tree/master/ Scripts which say: # Mainly inspired by https://github.com/openstack/keystone/blob/master/tools/sample_data.sh Why not submit that as an improvement to Keystone? I'd like to propose consolidation of all keystone initialization scripts around (Keyston's sample_data.sh, Devstack's keystone_data.sh, scripts like yours) and move to Lorin's YAML config (see https://lists.launchpad.net/openstack/msg17204.html) I'm just not sure yet if additional dependency on YAML is worth it. Cheers, Alan ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] A simple guide to install OpenStack Folsom
I second this idea, seems like a good way forward. From: Dolph Mathews dolph.math...@gmail.commailto:dolph.math...@gmail.com Date: Wednesday, October 10, 2012 3:33 PM To: Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com Cc: Alan Pevec ape...@gmail.commailto:ape...@gmail.com, Skible OpenStack skible.openst...@gmail.commailto:skible.openst...@gmail.com, openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] A simple guide to install OpenStack Folsom I played around with the idea this afternoon, and settled on something as simple as this in keystoneclient rather than keystone-manage: $ keystone help bootstrap usage: keystone bootstrap [--user-name user-name] --pass password [--role-name role-name] [--tenant-name tenant-name] Grants a new role to a new user on a new tenant, after creating each. Optional arguments: --user-name user-name The name of the user to be created (default=admin). --pass password The password for the new user. --role-name role-name The name of the role to be created and granted to the user (default=admin). --tenant-name tenant-name The name of the tenant to be created (default=admin). Example usage: $ keystone-manage db_sync $ keystone-all $ keystone --token=ADMIN --endpoint=http://localhost:35357/v2.0/bootstrap --pass=secrete $ keystone --os-username=admin --os-password=secrete --os-tenant-name=admin --os-auth-url=http://localhost:35357/v2.0/ token-get +---+--+ | Property | Value | +---+--+ | expires | 2012-10-11T22:25:02Z | | id| 4ae78bd2cd9049888060d07acddf88d1 | | tenant_id | 8fbba4f7f77e4acb80d746c65f20882b | | user_id | d8e31d9a341243a2bb8d575707a273ea | +---+--+ The same shortcut idea could apply to other extremely common usage patterns on the CLI (e.g. registering a service *and* all of it's endpoints in a single CLI command), thus eliminating most of the complexity of basic setup scripts like sample_data.sh and it's variants. I also put this up for review: https://review.openstack.org/#/c/14314 -Dolph On Wed, Oct 10, 2012 at 1:15 PM, Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com wrote: That sounds great to me. I can help out in converting this code into that code. It seems like a trivial kind of thing to do, what format would that command take, a yaml file? Something similar to https://github.com/yahoo/Openstack-Anvil/blob/master/conf/templates/keystone/init_what.yaml maybe, idk. From: Dolph Mathews dolph.math...@gmail.commailto:dolph.math...@gmail.com Date: Wednesday, October 10, 2012 11:13 AM To: Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com Cc: Alan Pevec ape...@gmail.commailto:ape...@gmail.com, Skible OpenStack skible.openst...@gmail.commailto:skible.openst...@gmail.com, openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] A simple guide to install OpenStack Folsom I'd like to simplify the scope of sample_data.sh to the absolute bare minimum (service tenant, admin role, admin user, identity service/endpoints, etc), and integrate it into keystone-manage as a 'bootstrap' command: $ keystone-manage bootstrap -Dolph On Wed, Oct 10, 2012 at 12:34 PM, Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com wrote: You guys should also consider the 'anvil' way of doing this (pure python baby, haha). Which is improved from lorin's and has been working for yahoo! for a while now. https://github.com/yahoo/Openstack-Anvil/blob/master/anvil/components/helpe rs/keystone.py#L25https://github.com/yahoo/Openstack-Anvil/blob/master/anvil/components/helpers/keystone.py#L25 Please feel free to take the code!! Its only 'real' dependency is the keystone client + yaml parsing... On 10/10/12 2:23 AM, Alan Pevec ape...@gmail.commailto:ape...@gmail.com wrote: On Wed, Oct 10, 2012 at 11:10 AM, Skible OpenStack skible.openst...@gmail.commailto:skible.openst...@gmail.com wrote: I am counting on our your feedback to enhance my work and contribute it to the OpenStack Eco System. I wonder about https://github.com/mseknibilel/OpenStack-Folsom-Install-guide/tree/master/ Scripts which say: # Mainly inspired by https://github.com/openstack/keystone/blob/master/tools/sample_data.sh Why not submit that as an improvement to Keystone? I'd like to propose consolidation of all keystone initialization scripts around (Keyston's sample_data.sh, Devstack's keystone_data.sh, scripts like yours) and move to Lorin's YAML config (see https://lists.launchpad.net/openstack/msg17204
Re: [Openstack] Cells Status
Along this type of line, I know we've talked about it before. But is cells the right way that we want to go? Not that it isn't, but possibly at the summit we can talk about it in detail before pushing it into trunk. I still really like the idea of making nova-compute nodes more 'dumb', then having a 'global' scheduler service which controls allocates to those compute nodes. Something like the current nova-scheduler but not hooked into the MQ. U could almost think of this 'global' scheduler service hooking into all 'clusters' message queues and it would make the decisions about what nodes get scheduled where. What or how the 'global' scheduler works could be sharded, it could be hierarchal, but it wouldn't be tied to nova tightly. Which is what cells/zones (the previous attempt)/… seem to be. Perhaps a generic scheduling 'entity' where one of the resources is compute nodes would be the best thing overall to solve this (since volumes also need scheduling, networks probably could also and so on). I just think in the long term that might be the best approach, but other thoughts are welcome… From: Chris Behrens cbehr...@codestud.commailto:cbehr...@codestud.com Date: Wednesday, October 3, 2012 4:57 AM To: Sam Morrison sorri...@gmail.commailto:sorri...@gmail.com Cc: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net, Chris Behrens cbehr...@codestud.commailto:cbehr...@codestud.com Subject: Re: [Openstack] Cells Status Ok. This took a lot longer to resolve than I expected, but here we go: https://github.com/comstud/nova/tree/cells_service This is rebased against trunk and contains a bunch of new things since the last branch: Random fixes for things that trunk broke with cells (deleting instances for one) RPC versioning (Thanks to Brian Elliott!) Split Replies and Bandwidth Updates into their own queues to better deal with them A number of admin API extensions modified to support cells (Thanks to Dragon, Alex Meade, Brian Lamar, Matt Sherborne, et al) Snapshots/backups query glance in API cell (Thanks to Iccha) Handle quotas in API cell (Thanks to Johannes for fixes!) Things are rapidly getting more kludgy because of changes in trunk that don't have any consideration for cells (because cells is not in trunk!). I'm hoping we can get this into an acceptable shape such that we can get it merged ASAP. - Chris On Oct 2, 2012, at 10:13 PM, Sam Morrison sorri...@gmail.commailto:sorri...@gmail.com wrote: OK great, will be good to get this into master. I have some stuff relating to key pairs, security groups that I'd like to contribute. Also we are looking at the ability for you to specify the cell when booting an instance. Cheers, Sam On 02/10/2012, at 1:06 PM, Chris Behrens cbehr...@codestud.commailto:cbehr...@codestud.com wrote: Yup, it's done. I just have to deal with some conflicts with our internal branch and my public one.. On Oct 1, 2012, at 7:47 PM, Sam Morrison sorri...@gmail.commailto:sorri...@gmail.com wrote: On 02/10/2012, at 12:33 PM, Chris Behrens cbehr...@codestud.commailto:cbehr...@codestud.com wrote: Thanks, Tom! I have changes to push up that add rpc versioning, etc. Maybe I can get those up tomorrow. Great! I was going to start looking into it but will hold off if you've already done it. Cheers, Sam ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] File Injection through Horizon
Yes, it is much better in the latest 0.7.0. Instead of the stripped down fedora version (which was the 0.6.3 one) there is now 'true' multi-distro support in cloud-init. And its coded in a way that other distros can be easily added (freebsd for example). This allows more modules (a cloud-init concept) to be useful (in 0.6.3 not many worked across distributions) and applicable for rhel6+ (we've got it at yahoo working on rhel5.6 also) and ubuntu (the main ones we have been testing on, fedora/centos should also work fine). At the summit I think scott and I will do some talks about what is this userdata/metadata thingy and how cloud-init uses it (and what cloud-init can do with it, and how it does it - time dependent). So get ready for that (there isn't an official session so keep checking what the unannounced/lighting/unconference sessions are). On 10/4/12 4:41 AM, Scott Moser smo...@ubuntu.com wrote: On Wed, 3 Oct 2012, Kiall Mac Innes wrote: Ah - I had meant the RHEL version :) Josh Harlow has done no small amount of work, and also had some aid from Garret Holstrom. Josh is using cloud-init 0.7.0 on RHEL/Fedora. He can certainly provide more details. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] TryStack latest status
Sweet, I'm hoping and pushing for getting some Y! help here as well. Hopefully this will happen soon :-) On 9/20/12 10:07 AM, Anne Gentle a...@openstack.org wrote: Hi all - We're taking small steps to get the TryStack experience back on track. Yesterday we moved the trystack.org site to Rackspace hosting rather than on an Apache server on the Cloud Controller and enabled SSL on that site. That move should keep the site up and the content is updated to set expectations on the latest, which is: - The ARM zone, running Essex code, is the only available TryStack installation. Both the Essex and Diablo zones (x86) have been closed for maintenance. - Request access to the ARM zone using this form: http://eepurl.com/nGEzj. The next step is to get an x86 zone back up, running Folsom code. - SwiftStack has offered to help with deploying an Object Storage installation on TryStack. Thanks Joe and John! - Nachi Ueno would like assistance with deploying Compute, Image, and Identity on TryStack. Contact him to help. Great opportunity to get some ops chops! Thanks for your patience while we continue to work through repeatable, managed processes to make TryStack run predictably. Anne and Nachi ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] TC Candidacy
Howdy ya'll!
++ Who am I ++
I'd like to also put myself up for the Technical Committee candidate
position, via one of the seats that are being made available. I believe I
have software at my heart (not literally) and that all systems should be
as elegant and architecturally sound as humanly possible. Helping
OpenStack achieve this type of 'beauty' is my goal and it has been a
learning process in getting there (myself included) and I'd like to
contribute what I can to make this continue to happen. In my spare time I
code more than I should, mountain bike, ski (double diamonds ftw), and
rock climb (5.12+ woot). Qui audet adipiscitur ('he who dares wins').
++ Background and Experience ++
I work at Yahoo! as one of the technical leads on the OpenStack team where
we have been working to get better involved in the OpenStack community and
establishing OpenStack internally (top secret!). We are focused on scale
(tens of thousands of servers), reliability, and making the best software
that is humanly possible (who doesn't want that)!
++ What I believe I can bring ++
I have been on various engineering teams at Yahoo! for the last 5 years. I
have designed/architected and implemented code that runs on
http://www.yahoo.com, the ad systems, the social network backends, and so
on. Each project has required understanding how scale and reliability can
be achieved, so that it¹s possible to maximize uptime (thus getting more
customers and so on). I started with OpenStack around the diablo timeframe
(the progress has been amazing so A+ there). Currently I have been working
on establishing OpenStack in Yahoo! and making sure Yahoo! keeps on being
an active and innovative contributor. I believe I can help out in scale
(how far can eventlet go...), architectural decisions (more services or
less??) and help OpenStack be as reliable and manageable as possible.
I have also in that time helped with various bugs and blueprints and have
been the main creator/pusher/driver of anvil (aka devstackpy, see:
http://anvil.readthedocs.org/) which Yahoo! has been using internally for
various tasks. I have also recently jumped on the cloud-init boat
(https://launchpad.net/cloud-init/) and helped out there (I am now the #2
contributor there) in doing some refactoring and adding in multi-distro
support (where possible) in that project so that it can be used more
widely publicly and internally @ Yahoo!.
++ Technical Skills ++
I have been working in software for as long as I can remember, in various
languages, c/c++, java, ruby, python/jython... Strive to create the best
architectures and code that I can (it¹s always a learning process). Some
of my past work is on github, https://github.com/harlowja, other are not
(unfortunately), or just feel free to ask me on irc or email or launchpad
anytime you want (https://launchpad.net/~harlowja)...
++ Coworker Quotes ++
³I¹ve worked with Josh for several years and the first thing you need to
know is that working with Josh forces you to keep on your technical toes.
He¹s always got his eye on emerging technologies and is very good at not
reinventing the wheel. A ready sense of humor, sense of fun, and
willingness to listen to other¹s opinions make him an easy person to work
with.²
-- Ken Thomas (krt)
++ Not Coworker Quotes ++
http://i.imgur.com/TgpXU
Thanks much for the chance (no matter the result). May the best man/woman
win :-)
-Josh
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cells Status
Eck, lets not 'sneak' stuff in ever, please ;) On 9/14/12 8:48 AM, Thierry Carrez thie...@openstack.org wrote: Joe Topjian wrote: This is very disappointing. I was looking forward to cells as well. When was this decided and was the decision announced somewhere else? I'd like to know so I can monitor for other announcements like this. Actually, Cells was never proposed as a Folsom blueprint, so it never was in the Folsom plan (which you can access at http://wiki.openstack.org/releasestatus/ ). ISTR there was some discussion about sneaking it late in Folsom or land it early in Grizzly, and with the PTL's advice the authors decided the latter. -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cells Status
Ya, is there a session on it at the summit. I'd at least like to talk about it and what it could be in the end. Or maybe we can 'freestyle' that session :-P On 9/14/12 8:36 AM, Russell Bryant rbry...@redhat.com wrote: On 09/14/2012 11:08 AM, Joe Topjian wrote: We didnt find any information related to CELLS [which is planned to replace ZONES] in the latest Folsom pre-release. Can any body give us information on this. Unfortunately, cells was unable to make feature freeze. It should be in Grizzly. Sorry for the delay :/ This is very disappointing. I was looking forward to cells as well. When was this decided and was the decision announced somewhere else? I'd like to know so I can monitor for other announcements like this. The patch is here: https://review.openstack.org/#/c/10707/ It was proposed for inclusion on August 2nd, which was just a couple of weeks before the feature freeze. There were enough significant things that came up in discussion that it just wasn't going to make it in for Folsom. In addition to working through the technical details, it also desperately needs some documentation (ideally both architectural, as well as how to use it). It seems like we should be able to get this all wrapped up for Grizzly, though. -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nova PTL candidacy
What is Spreadi? Is that a new word :-P On 9/4/12 12:58 PM, Vishvananda Ishaya vishvana...@gmail.com wrote: Hello Everyone, I'm writing to announce my candidacy for the Project Technical Lead of Nova for the Grizzly Release cycle. Qualifications -- I was part of the original Anso Labs Team that created Nova at NASA[1]. I have more commits to nova than any other contributor[2] and the second most in the past 12 months[3]. I am the most active reviewer for nova commits[4] I have been the Nova PTL since the position waw created[5]. Folsom Accomplishments -- A lot was achieved in Nova during this cycle. I have to give most of the credit to the active contributors we had. I don't have space to cover everything that we achieved during the release but here are some key points: * Split the volume code into its own project and helped it get under way. * Versioned the rpc apis * Improved api testing and xml support * Moved instances to using exclusively UUIDs * Improved our state management and minimized race conditions * Cleaned up the quota management to make it more robust Grizzly Plan While there is a great deal to be determined at the design summit, I think there are a few key things that we need to focus on over the next cycle. * Spreadi * No DB compute: We did a huge amount of preparation during folsom to allow us to remove database access from the compute nodes. This will dramatically improve the security profile of nova and allow us to scale * Better quantum integration: We are very close to a seamless integration with quantum where a provider could be using quantum on the backend and end-users wouldn't even have to know. * Upgrade consistency: Now that we have rpc versioning, we should be able to take steps to allow for the possibility of live upgrades * Better support for custom backends: we need to solidify the driver interfaces so custom backends can potentially live out-of-tree. This will allow the management Vish [1] https://github.com/openstack/nova/commit/f04c6ab2d082ce8fe48ec58cb5c7cc64e d2a282b [2] http://www.ohloh.net/p/novacc/contributors?query=sort=commits [3] http://www.ohloh.net/p/novacc/contributors [4] http://173.203.107.207/~soren/stats/nova-30days.json [5] https://lists.launchpad.net/openstack/msg01674.html ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Migration
Perhaps we should also have a CHANGELOG file to explain the major features/changes... Perhaps a 'MIGRATION' file as well that explains how to migrate from version - 1? On 8/29/12 10:15 AM, Tim Bell tim.b...@cern.ch wrote: I think a new release should contains details of how to do the upgrade (rather than discovering as we try it) I should aim that the deliverables for each of the projects in a new version includes in the release notes: A. dependencies (i.e. does glance folsom need to talk to horizon folsom or can it also talk to horizon essex) B. migration steps to move an instance to the latest version (i.e. how do I get glance essex to glance horizon) Planning an production upgrade will be very time consuming if it requires the person(s) to understand all the components in depth and derive the steps from the bug fixes. One of the items to review within the user/project feedback loop would be how we validate for a release (I used to call this system test as opposed to integration test, years ago). This would be the steps where we validate that a release complies with a set of deployability criteria (such as migration steps and documentation). Would the upcoming Folsom release meet these criteria (A./B.) for each core project ? Tim It would be fascinating (for me at least :)) to know the upgrade process you use - how many stages you use, do you have multiple regions and use one/some as canaries? Does the downtime required to do an upgrade affect you? Do you run skewed versions (e.g. folsom nova, essex glance) or do you do lock-step upgrades of all the components? For Launchpad we've been moving more and more to a model of permitting temporary skew so that we can do rolling upgrades of the component services. That seems in-principle doable here - and could make it easier to smoothly transition between versions, at the cost of a (small) amount of attention to detail while writing changes to the various apis. -Rob ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Keyring support in openstack
Sweet thx all :-) This is great and a step forward… https://blueprints.launchpad.net/openstack-common/+spec/pw-keyrings Now just to get it into those config files to use something similar (no passwords in those pweeease…) -Josh From: Bhuvaneswaran A bhu...@apache.orgmailto:bhu...@apache.org Date: Wednesday, August 22, 2012 4:15 PM To: Adam Young ayo...@redhat.commailto:ayo...@redhat.com Cc: openstack openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] Keyring support in openstack On Mon, Jul 30, 2012 at 5:48 PM, Adam Young ayo...@redhat.commailto:ayo...@redhat.com wrote: On 07/30/2012 06:00 PM, Doug Hellmann wrote: On Mon, Jul 30, 2012 at 5:30 PM, Adam Young ayo...@redhat.commailto:ayo...@redhat.com wrote: On 07/30/2012 05:17 PM, Kevin L. Mitchell wrote: On Mon, 2012-07-30 at 13:50 -0700, Bhuvaneswaran A wrote: The wiki mentions the password being saved using keyring.backend.UncryptedFileKeyring. Does that mean the password is saved in cleartext? Is the file protected in some way besides filesystem permissions? As mentioned in wiki page, the password is stored in base64 format. Which means it's stored in cleartext. That is Not Good(tm) :) Can Keyring be used to store a token instead? That would A) be better than password and B) avoid a Keystone hit. Don't tokens expire? Yes, they do, but that is no reason not to put them in the keyring, With the PKI tokens, you will be able to query a token's expiry without going across the wire. Adam, can you please file a ticket to use keyring to store tokens for keystone? I'll work on it. -- Regards, Bhuvaneswaran A www.livecipher.comhttp://www.livecipher.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Question on tests
Hi all, I'm getting anvil to run tests and I was just wondering on a couple of tests I see there (the issues might be bugs?) I have had to exclude the following since they error (when say swift isn't there, or memcache/ldap isn't there), should those instead be skipping themselves when there needed 3rd party libraries is not installed? Thoughts? That would seem more appropriate then errorring out (when those things really aren't core keystone…) [ # These 2 seem to require swift, not always installed... 'test_swift_auth_middleware', 'test_s3_token_middleware', # Aren't always installing memcache... 'test_backend_memcache', # Oddness: 'unable to access signing dir /root/keystone-signing' 'test_nomemcache', # Aren't always installing ldap... 'test_backend_ldap', ] This running: http://pastebin.com/3Aaz4iLC To accomplish this new feature if u want to try it on RHEL6+ (getting ubuntu back in shape for the core components that anvil is supporting)… $ sudo ./smithy -a install …. $ sudo ./smithy -p conf/personas/solo/keystone.yaml -a test … That Is It :-P -Josh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Is It Safe to Use The OpenStack Packages Distributed in Ubuntu 12.04 Official Repository?
I'm working on making anvil build packages (at least basic ones). My idea is that since it will somewhat track how to setup/uninstall/configure (along with it knowing how to map pip packages to distribution packages, something being helped by the new more complete pip-requires lists) an openstack release that it should also be able to package (for at least rhel/ubuntu). Its a WIP... Feel free to help ;) http://anvil.readthedocs.org/en/latest/topics/features.html This list might be useful anyway: https://github.com/yahoo/Openstack-Anvil/blob/master/conf/distros/rhel-6.ya ml#L73 (yes I know I killed ubuntu, only so much time in the day, will get that back in shape soon). -Josh On 8/19/12 5:20 AM, Ji Zhang zhangj...@gmail.com wrote: Hi, I'm to deploy OpenStack Essex in production environment. According to the official manual, I should either install it manually or use dodai-deploy. After briefly browsing these methods, it seems that both of them are using the packages distributed in official repository (i.e. apt-get install keystone), not building from source like DevStack does. So my question is, as is said in the title, are these packages production ready? Or should I checkout the stable/essex branch of each project and build it by myself? Thanks. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] RedHAt * OPenSTack
Are signups taking a while?? Anyone else got the email yet, I think they lost mine, sad++ On 8/14/12 11:57 AM, Frans Thamura fr...@meruvian.org wrote: hi all Redhat just post in his wall openstack.. http://www.redhat.com/openstack/?sc_cid=7016000TmB8AAK -- Frans Thamura (曽志胜) Shadow Master and Lead Investor Meruvian. Integrated Hypermedia Java Solution Provider. Mobile: +628557888699 Blog: http://blogs.mervpolis.com/roller/flatburger (id) FB: http://www.facebook.com/meruvian TW: http://www.twitter.com/meruvian / @meruvian Website: http://www.meruvian.org We grow because we share the same belief. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Nova] How common is user_data for instances?
I'm pretty sure its common since its the main way to get data into cloud-init. -Josh On 8/13/12 3:02 PM, Michael Still michael.st...@canonical.com wrote: On 14/08/12 01:24, Jay Pipes wrote: Or just set the column to the LONGTEXT type and both MySQL and PostgreSQL will be just as happy. This is what I was originally aiming at -- will large deployers be angry if I change this column to longtext? Will the migration be a significant problem for them? Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Setting Expectations
So many questions, so hard to reply. Whats the best question to answer here ;) From: Andrew Clay Shafer a...@parvuscaptus.commailto:a...@parvuscaptus.com Date: Fri, 10 Aug 2012 14:41:03 -0700 To: openstack openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: [Openstack] Setting Expectations What is OpenStack? Clearly, OpenStack is many things to many people and organizations. What does it mean to contribute to OpenStack? What does it mean to deploy OpenStack? What does it mean to operate OpenStack? What do we mean when we say compatible? interoperable? community? branded? Is OpenStack a framework? a project? a product? Recent discussions make it clear that we have a lot of different ideas about all of these things. Our collective and individual responsibilities to each other are also a point of tension. There is a marked difference in the perspective of those developing the projects, those operating the projects as services and the final consumers/clients of those services. If OpenStack is going to live up to it's full potential and stated mission, I believe there needs to be a much stronger collective conscience about how decisions are made. I feel we will all benefit by making some things more explicit. If the expectation is that OpenStack is a framework, which is a word I've heard people use many times, then does an upgrade path have to exist? The OpenStack dashboard was essentially rewritten to upgrade to a new version of Django. Was there any expectation that Django should upgrade itself for us? Upgrading an application to use a different versions of Rails, another framework, often borders on impossible, particularly if the application happens have some feature with a dependency of a gem that hasn't been kept in sync with the upstream project. Is OpenStack more or less complicated than those frameworks? What responsibility should OpenStack core development have to consider existing deployments? Frameworks are expected be a foundation to build on. By definition, changing foundations is not easy. Clearly, OpenStack can be deployed and operated, but if OpenStack needs to be easier to deploy, operate and upgrade, and that is a responsibility of OpenStack itself, that can't be something that get's tacked on at the end. There is no 'ease of deployment' powder to sprinkle on at the end. Distributions and tooling can and do obscure the difficultly for the downstream user, but that also leads to a lot of potential fragmentation. And is that the right answer? Who can and should answer that? That OpenStack should be easy to deploy and upgrade is somewhat at odds with OpenStack supporting every possible combination of hypervisor, storage and networking option, let alone what the expectation should be with closed source customizations/integrations. Which brings up questions of compatibility. API compatibility is potentially misleading if the semantics and behaviors vary. I've heard several service provider discuss ideas about how they can be differentiated in the market, and many of those ideas lead differences in APIs to expose. Is that wrong? Maybe, maybe not, but it certainly makes a lot of work if your core business is dependent on maintaining integrations with service providers. Taken to an extreme these decisions complicate and call into question any future of federated OpenStack services. I'm not advocating any choice here. I just want to point out there are compromises that have to be made. There are goals and desires for OpenStack that are at odds with each other. Some of that is a function of the perspective of persona, but a lot is also from fundamental differences in understanding about where OpenStack is, where OpenStack needs to be, and how to get there. If there isn't a core guiding conscience about what we are trying to accomplish that gets applied across the board, then I worry that the future of OpenStack ends up with more fragments optimizing for their perspective and inevitable skirmishes when the perspectives collide. I see there are many conversations we aren't having, which leads to surfacing all the unaddressed issues when someone does try to involve the community in discussions. OpenStack can't be all things, but we get to decide what it will be. The question is will we do that explicitly and consciously, or indirectly and passively. There is no one person who can address this alone. I'm hoping this can start a conversation. Best Regards, Andrew ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Question on hostname generation?
Hi all, I was wondering about the following case (and am not sure if its been addressed in folsom). At yahoo, instead of the default hostname that seems to be automatically established (ie 'server-XYZ.novalocal' was in essex) we were wondering if there is anything in folsom that say lets us override that generation with either a module (or a subclass) or a new function without having to patch nova code. Sometimes we want a default hostname of a different format (as probably do others) and I just am not sure if quantum is directing this default hostname, or if nova still is, or if its some mix of both (the metadata + cloud-init can affect the hostname as well, so perhaps the configdrive work should be the only one establish a 'real' hostname?). I have found the following pieces of hostname related code but am not sure if this is all the places that make hostnames :-) * https://github.com/openstack/nova/blob/master/nova/api/metadata/base.py#L109 Is this the only place nowadays?? Is there something in quantum also? Thx much, just trying to track this down. -Josh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] metadata service problem
I would start to check out iptables and routes that are being setup (in vms and outside). If you are running a flat (no dhcp) network that usually makes it a lot harder also. On 8/9/12 7:31 PM, Xin Zhao xz...@bnl.gov wrote: Hello, In my essex install on RHEL6, there is a problem with the metadata service. The metadata service works for instances running on the controller node, where the nova-api(metadata service) is running. But for the other worker nodes, the metadata service is intermittent, ie. the instances sometimes can get its metadata, sometime it fails with errors like: $ curl -v http://169.254.169.254:8775/ * About to connect() to 169.254.169.254 port 8775 (#0) * Trying 169.254.169.254... Connection timed out * couldn't connect to host * Closing connection #0 curl: (7) couldn't connect to host Any idea where should I start investigating this? Thanks, Xin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Don't bypass code-review.
Slap on wrist bad person who did that, badness++ 'Those responsible have been sacked'??? -Josh On 7/10/12 10:52 AM, Eric Windisch e...@cloudscaling.com wrote: I've had code reviews sitting out for over a week, looking to fix issues with the ZeroMQ driver in openstack-common. I'd love to get it fixed, and nudged a couple of people to get the reviews in, but figured that it would get in eventually - and if not, I'd prod harder, or perhaps someone else that would want to see these problems fixed, would help review. Instead... Today, I had someone ignore my reviews in progress, push his own review, and even APPROVED his own review. What the hell? Review in question: https://review.openstack.org/#/c/9594/ (My) Reviews in progress: https://review.openstack.org/#/q/status:open+project:openstack/openstack-common+branch:master+topic:bug/1021459,n,z -- Eric Windisch ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] best practices for merging common into specific projects
Ya, let me know, I'll jump in and see what I can do also... On 7/4/12 3:18 PM, Gabriel Hurley gabriel.hur...@nebula.com wrote: Having a team/leader in that arena would definitely help. I'd contribute to common more if I knew what needed contributing, who to talk to about it, etc... Same goes for helping in terms of packaging, etc. to make it a proper common library. - Gabriel -Original Message- From: openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net [mailto:openstack- bounces+gabriel.hurley=nebula@lists.launchpad.net] On Behalf Of Thierry Carrez Sent: Wednesday, July 04, 2012 2:57 AM To: openstack@lists.launchpad.net Subject: Re: [Openstack] best practices for merging common into specific projects Monty Taylor wrote: However, with a versioned library model, the projects can consume things pinned to specific versions, and then they can submit a change that updates the version depend and the code which needs to be updated to support the version change, and that change can be atomic. So honestly, I'd say the real key is getting us closer to the point where openstack-common is a proper library, because all of the rest of the complexity is stuff we're inventing to make life harder on ourselves, when the standard library with api-contract and a version model of the world works pretty fine without needing coordinated changes across multiple repositories. Yes, that's the end goal. And IMHO we are not very far away. I think the main reason we are not there yet is that while a lot of people enjoy giving their opinions about how openstack-common should be done and consumed by projects, not so many people follow up and actually do the work. Making our multiple projects converge onto consolidated and well-accepted APIs is a bit painful work, but it is a prerequisite to turning openstack- common into a proper library (or set of libraries). I'd say the whole thing suffers from not having a proper team/leader/coordinator dedicated to it: relying on existing, overstretched PTLs to lead that effort might not be the fastest path. Regards, -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] best practices for merging common into specific projects
I think that's a good little explanation as to why we have openstack-common, but when did it become a good reason to copy code around via an inclusion mechanism? Lots of code is in packages (outside of openstack, in pypi and elsewhere) that is also in 'incubation' (in fact, what code isn't in perpetual incubation), that's why u still have version numbers? I just worry about inclusion of code that isn't versioned into other projects, and I don't see the benefit of that when u can just have a package that has that code as well. On 7/3/12 2:35 AM, Thierry Carrez thie...@openstack.org wrote: Thierry Carrez wrote: Gabriel Hurley wrote: On a more fundamental level, did I miss some tremendous reason why we have this merge from common pattern instead of making OpenStack Common a standard python dependency just like anything else? Especially with the work Monty has recently done on versioning and packaging the client libs from Jenkins, I can't see a reason to keep following this update common and merge to everything else pattern at all... This discussion should probably wait for markmc to come back, since he set up most of this framework in the first place. He would certainly produce a more compelling rationale than I can :) Actually http://wiki.openstack.org/CommonLibrary explains it quite well. In particular: openstack-common also provides a process for incubating APIs which, while they are shared between multiple OpenStack projects, have not yet matured to meet the [library inclusion] criteria described above. Incubation shouldn't be seen as a long term option for any API - it is merely a stepping stone to inclusion into the openstack-common library proper. -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Anyone using instance metadata?
+1 for getting over screams earlier rather than later On 7/3/12 11:51 AM, Scott Moser smo...@ubuntu.com wrote: On Tue, 3 Jul 2012, Vishvananda Ishaya wrote: Metadata is supposed to be user tags that are associated with a guest that are available via the api. We discussed displaying these tags inside the guest as well. Am I reading it wrong? It seems like it *is* available inside the guest. At very least with config drive on i know it is. The main difference between user-data and metadata is that metadata is available to the api, whereas user-data is only available in the guest. So to avoid confusion, if the intent was tags, I think we should disable the 'meta.js' file injection, and get over the screams now. half and half is just confusing. Thoughts? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] best practices for merging common into specific projects
I 150% agree that is a red-herring, that's why I wonder what it really offers besides a 'façade' and/or the feeling that what u are using isn't a package, when in concept it really is, except now u have lost all the benefits of using version numbers, having dependency versions (with history) and all that, so the 'façade' seems pretty weak imho. +1 for library that follows the normal packaging methodology On 7/3/12 11:59 AM, Gabriel Hurley gabriel.hur...@nebula.com wrote: The notion that copying code is any protection against APIs that may change is a red herring. It's the exact same effect as pegging a version of a dependency (whether it's a commit hash or a real version number), except now you have code duplication. An unstable upgrade path is an unstable upgrade path, and copying the code into the project doesn't alleviate the pain for the project if the upstream library decides to change its APIs. Also, we're really calling something used by more or less all the core projects incubated? ;-) Seems like it's past the proof-of-concept phase now, at least for many parts of common. Questions of API stability are an issue unto themselves. Anyhow, I'm +1 on turning it into a real library of its own, as a couple people suggested already. - Gabriel -Original Message- From: openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net [mailto:openstack- bounces+gabriel.hurley=nebula@lists.launchpad.net] On Behalf Of James E. Blair Sent: Tuesday, July 03, 2012 6:56 AM To: andrewbog...@gmail.com Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] best practices for merging common into specific projects Andrew Bogott abog...@wikimedia.org writes: I. As soon as a patch drops into common, the patch author should submit merge-from-common patches to all affected projects. A. (This should really be done by a bot, but that's not going to happen overnight) Actually, I think with our current level of tooling, we can have Jenkins do this (run by Zuul as a post-merge job on openstack-common). I very much believe that the long-term goal should be to make openstack- common a library -- so nothing I say here should be construed against that. But as long as it's in an incubation phase, if doing something like this would help move things along, we can certainly implement it, and fairly easily. Note that a naive implementation might generate quite a bit of review spam if several small changes land to openstack-common (there would then be changes*projects number of reviews in gerrit). We have some code laying around which might be useful here that looks for an existing open change and amends it; at least that would let us have at most only one open merge- from-common-change per-project. Okay, that's all on that; I don't want to derail the main conversation, and I'd much rather it just be a library if we're close to being ready for that. -Jim ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack and Google Compute Engine
I'd be interested in hearing any comparisons, but it seems like it just came out so it might take a while... Knowing how google is very secretive about there internal 'architecture' it might be really hard to make any in-depth comparisons. On 7/2/12 7:25 AM, Simon G. semy...@gmail.com wrote: Noone tested or noone is interested in Google Compute Engine and Openstack? On Sat, Jun 30, 2012 at 9:59 PM, Simon G. semy...@gmail.com wrote: Hello, I've heard about Google's cloud recently. What do you think about it? Will it be compatible with openstack? Or will openstack be compatible with them? Anyone knows something about their solution? it's purely their technology? Or maybe they were inspired by openstack or something else. What about scalability? Their test app is really impressive - 600.000 cores. Is it even achievable in openstack? How can I create environment with so many cores, in openstack? I'm waiting for my invitation to google compute, but maybe someone has already tested it. Cheers, ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] best practices for merging common into specific projects
What about using openstack-common as a library instead of a preprocessor 'inclusion' system/copy code around system?? Maybe its time for that to happen? It always seemed sort of silly to me that files are being copied around to different projects like this, instead of referring to code in a common library package. On 7/2/12 12:16 PM, Andrew Bogott abog...@wikimedia.org wrote: Having spent some time last week writing code for openstack-common, and having spent yet more time trying to get those changes into Nova, I think it would be useful to define some best practices when crossing the boundary between common and other openstack projects. Background: The openstack-common project is subject to a standard code-review process (and, soon, will also have Jenkins testing gates.) Sadly, patches that are merged into openstack-common are essentially orphans. Bringing those changes into actual use requires yet another step, a 'merge from common' patch where the code changes in common are copied into a specific project (e.g. nova.) Merge-from-common patches are generated via an automated process. Specific projects express dependencies on specific common components via a config file, e.g. 'nova/openstack-common.conf'. The actual file copy is performed by 'openstack-common/update.py,' and its behavior is governed by the appropriate openstack-common.conf file. Questions: When should changes from common be merged into other projects? What should a 'merge-from-common' patch look like? What code-review standards should core committers observe when reviewing merge-from-common patches? Proposals: I. As soon as a patch drops into common, the patch author should submit merge-from-common patches to all affected projects. A. (This should really be done by a bot, but that's not going to happen overnight) II. In the event that I. is not observed, merge-from-common patches will contain bits from multiple precursor patches. That is not only OK, but encouraged. A. Keeping projects in sync with common is important! B. Asking producers of merge-from-common patches to understand the full diff will discourage the generation of such merges. III.Merge-from-common patches should be the product of a single unedited run of update.py. A. If a merge-from-common patch breaks pep8 or a test in nova, don't fix the patch; fix the code in common. IV.Merge-from-common patches are 'presumed justified'. That means: A. Reviewers of merge-from-common patches should consider test failures and pep8 breakages, and obvious functional problems. B. Reviewers of merge-from-common patches should not consider the usefulness, design, etc. of merge-from-common patches. Such concerns need to be handled within the common review process. C. Merges from common should get reviewed early and approved readily in order to avoid project divergence V. Changes to openstack-common.conf are a special case. A. Patches with openstack-common.conf changes should include the relevant merge-from-common changes. B. Such patches should /not/ include any other merge-from-common changes. C. Such patches should not only include the common merges, but should also include whatever code changes make use of the new merge. D. Such patches require the same justification and scrutiny as any other standard project patch. Please discuss! If we're able to reach general agreement about this process, I will document the process in the openstack-common HACKING guidelines. -Andrew On 7/2/12 11:38 AM, Russell Bryant (Code Review) wrote: I did that before seeing this patch. However, I think I prefer what I pushed since it's individual updates explaining what they update instead of a blanket update everything commit. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] best practices for merging common into specific projects
Maybe its time to break out of that incubation?? Seems like if most projects are depending on these config files for code copying that that break out has really already occurred, but it just hasn't been written down or made official? I don't quite understand how a project can be in incubation, but has components that are required for other projects to work? Isn't this the same as having library versions where a project would pull in X.Y.Z openstack-common library version (with the features it wants) while the openstack-common people work on X.Y.Z + 1? Then when X.Y.Z + 1 is approved 'stable' they can move to X.Y.Z+1. On 7/2/12 12:26 PM, Russell Bryant rbry...@redhat.com wrote: On 07/02/2012 03:16 PM, Andrew Bogott wrote: Background: The openstack-common project is subject to a standard code-review process (and, soon, will also have Jenkins testing gates.) Sadly, patches that are merged into openstack-common are essentially orphans. Bringing those changes into actual use requires yet another step, a 'merge from common' patch where the code changes in common are copied into a specific project (e.g. nova.) Merge-from-common patches are generated via an automated process. Specific projects express dependencies on specific common components via a config file, e.g. 'nova/openstack-common.conf'. The actual file copy is performed by 'openstack-common/update.py,' and its behavior is governed by the appropriate openstack-common.conf file. More background: http://wiki.openstack.org/CommonLibrary Questions: When should changes from common be merged into other projects? What should a 'merge-from-common' patch look like? What code-review standards should core committers observe when reviewing merge-from-common patches? Proposals: I. As soon as a patch drops into common, the patch author should submit merge-from-common patches to all affected projects. A. (This should really be done by a bot, but that's not going to happen overnight) All of the APIs in openstack-common right now are considered to be in incubation, meaning that breaking changes could be made. I don't think automated merges are good for anything in incubation. Automation would be suitable for stable APIs. Once an API is no longer in incubation, we should be looking at how to make releases and treat it as a proper library. The copy/paste madness should be limited to APIs still in incubation. There are multiple APIs close or at the point where I think we should be able to commit to them. I'll leave the specifics for a separate discussion, but I think moving on this front is key to reducing the pain we are seeing with code copying. II. In the event that I. is not observed, merge-from-common patches will contain bits from multiple precursor patches. That is not only OK, but encouraged. A. Keeping projects in sync with common is important! B. Asking producers of merge-from-common patches to understand the full diff will discourage the generation of such merges. I don't see this as much different as any other patches to nova (or whatever project is using common). It should be a proper patch series. If the person pulling it in doesn't understand the merge well enough to produce the patch series with proper commit messages, then they are the wrong person to be doing the merge in the first place. III.Merge-from-common patches should be the product of a single unedited run of update.py. Disagree, see above. A. If a merge-from-common patch breaks pep8 or a test in nova, don't fix the patch; fix the code in common. Agreed. IV.Merge-from-common patches are 'presumed justified'. That means: A. Reviewers of merge-from-common patches should consider test failures and pep8 breakages, and obvious functional problems. B. Reviewers of merge-from-common patches should not consider the usefulness, design, etc. of merge-from-common patches. Such concerns need to be handled within the common review process. C. Merges from common should get reviewed early and approved readily in order to avoid project divergence I think core reviewers of projects using openstack-common are justified in doing critical review of new code being used by their project. A core reviewer of nova for examp[le may have a very good reason why the code is not suitable for consumption in nova that openstack-common reviewers didn't think of. I don't think blind approvals are ever a good idea. V. Changes to openstack-common.conf are a special case. A. Patches with openstack-common.conf changes should include the relevant merge-from-common changes. B. Such patches should /not/ include any other merge-from-common changes. C. Such patches should not only include the common merges, but should also include whatever code changes make use of the new merge. D. Such patches require the same justification and scrutiny as any other standard
Re: [Openstack] setuptools-git
Should there be a separation of build-time setup.py and run-time setup.py?? I'm not sure if something like that is possible (maybe with a setuptools variant, distribute or something similar??) On 6/29/12 4:06 AM, Alan Pevec ape...@gmail.com wrote: On Thu, Jun 28, 2012 at 9:56 PM, Monty Taylor mord...@inaugust.com wrote: https://review.openstack.org/9109 Why is setuptools_git added in pip-requires, I thought that's for run-time, not build-time dependencies? Cheers, Alan ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Openstack special sauce checking tool??
Hi all, I remember hearing once that someone had a openstack import/hacking style checking tool. I was wondering if such a thing existed to verify same the openstack way of doing imports and other special checks to match the openstack style. I know a lot of us run pep8/pylint, but those don't catch these special sauce checks And it would be nice to not have to manually check these (visually...) but be able to run a tool that would just do the basic checks. Does such a thing exist?? Thx, Josh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack special sauce checking tool??
Sweet, didn't know about that :-P Maybe that should be in openstack-common?? On 6/28/12 10:48 AM, Timothy Daly ti...@yahoo-inc.com wrote: nova has tools/hacking.py, which looks like it does check some import stuff, among other things. -tim On Jun 28, 2012, at 10:15 AM, Joshua Harlow wrote: Hi all, I remember hearing once that someone had a openstack import/hacking style checking tool. I was wondering if such a thing existed to verify same the openstack way of doing imports and other special checks to match the openstack style. I know a lot of us run pep8/pylint, but those don't catch these special sauce checks And it would be nice to not have to manually check these (visually...) but be able to run a tool that would just do the basic checks. Does such a thing exist?? Thx, Josh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [devstack] Easing maintenance of list of distro packages to install
Everyone should really check out... https://github.com/yahoo/Openstack-Anvil/tree/master/conf/distros It is nice to have a standard yaml format that isn't a new micro-custom-format that we have to figure out how to parse. In fact I think there is an open work-item to centralize this and make it better for all... https://bugs.launchpad.net/openstack-common/+bug/995607 I think that bug is a good start, but not far enough, a lot of openstack dependences on system level libraries as well, so we need to track those if we can, and not just the pypi/python ones, or at least have a list of known working system level libraries to reference. On 6/20/12 4:05 AM, Ghe Rivero ghe.riv...@stackops.com wrote: On Wed, Jun 20, 2012 at 12:24 PM, Daniel P. Berrange berra...@redhat.com wrote: On Wed, Jun 20, 2012 at 12:06:46PM +0200, Vincent Untz wrote: Hi, In devstack, we currently have two separate lists of packages to install: one for Ubuntu (in files/apts/) and one for Fedora (in files/rpms/). This has two issues: - this leads to incomplete updates for dependencies. It happens that someone updates the apts files but not the rpms ones. (shameless plug: https://review.openstack.org/#/c/8475/ needs some review love) - this just doesn't scale when adding support for another distro, especially as rpm-based distros don't all share the same package names (hence files/rpms/ cannot really be shared). I'd like us to move to a new scheme where we have one list of packages (say the Ubuntu one, for instance) and instead of adding another one Fedora, openSUSE, etc., we have translation tables to map package names from Ubuntu to other distros. Supporting a new distro is then a matter of adding a translation table (+ hacking the code to change the right config files, obviously), and we can easily add tests to make sure the translation tables contain a mapping for each package (and therefore fix the first issue). I already have some working code for that, but I want to check if people are fine with the idea before submitting it for review. I've nothing against your proposal certainly the motivation is good, but thought I'd throw out an alternative idea just in case Instead of having one sub-dir per package/distro, just have a single (CSV/JSON/XML/whatever) file listing all distros in the same place eg a CSV file where the first column is the generic name, and other columns are the distro-specific names (if required). The distro specific column would be empty if the generic name applied without change, or '-' if the package was not applicable to the distro at all. # cat nova.csv Package,Ubuntu,Fedora,RHEL,SUSE python-devel,, libvirt,libvirt-bin, dnsmasq-base,,-,-,, dnsmasq-utils,,, Hmm, using JSON would actually be a bit more readable. The core idea is to have all the data, both the master list and distro mappings in one clear place. We should keep in mind also that not all the packages have an equivalent in all the distros: sometimes one single package is splitted in several in other distros, or it doesn't exist at all. CSV would be not truly manageable, but having everything in one single place would be easier if any update is necessary. JSON can be a good candidate but, in the case of nova (46 packages right now), would have to see if we don't get spaghetti code difficult to deal with. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Intermittent devstack-gate failures
Hmmm that makes it hard to develop on RHEL again (or I guess fedora 16?) Durn. Any reasoning behind the 0.9.7 change? Just out of curiosity... On 6/12/12 6:11 PM, Dan Prince dpri...@redhat.com wrote: Hey Jim, I actually turned off SmokeStack earlier today for potentially the same reason. I was out for a couple days last week and haven't quite put my finger on all the things that are wrong. I'm seeing about half of the functional test runs fail. This issue seems to be the cause of most of my failures: https://bugs.launchpad.net/nova/+bug/1010291 I'm also dealing with the fact that Nova now requires libvirt 0.9.7 or later (I think) due to some of the refactoring. This is fine but it does mean I can't fully test things on Fedora 16 like I used to (reboot is out for example). That is what I've seen. Dan - Original Message - From: James E. Blair cor...@inaugust.com To: OpenStack Mailing List openstack@lists.launchpad.net Sent: Tuesday, June 12, 2012 7:25:01 PM Subject: [Openstack] Intermittent devstack-gate failures Hi, It looks like there are intermittent, but frequent, failures in the devstack-gate. This suggests a non-deterministic bug has crept into some piece of OpenStack software. In this kind of situation, certainly could keep re-approving changes in the hope that they will pass the test and merge, but it would be better to fix the underlying problem. Simply re-approving is mostly just going to make the queue longer. Note that the output from Jenkins has changed recently. I've seen some people misconstrue some normal parts of the test process as errors. In particular, this message from Jenkins is not an error: Looks like the node went offline during the build. Check the slave log for the details. That's a normal part of the way the devstack-gate tests run, where we add a machine to Jenkins as a slave, run the tests, and remove it from the list of slaves before it's done. This is to accommodate the one-shot nature of devstack based testing. It doesn't interfere with the results. To find out why a test failed, you should scroll up a bit to the devstack exercise output, which normally looks like this: * SUCCESS: End DevStack Exercise: ./exercises/volumes.sh * = SKIP boot_from_volume SKIP client-env SKIP quantum SKIP swift PASS aggregates PASS bundle PASS client-args PASS euca PASS floating_ips PASS sec_groups PASS volumes = Everything after that point is test running boilerplate. I'll add some echo statements to that effect in the future. Finally, it may be a little difficult to pinpoint when this started. A number of devstack-gate tests have passed recently without actually running any tests, due to an issue with one of our OpenStack based node providers. We are eating our own dogfood. -Jim ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Libguestfs weirdness??
Ok, so part of the findings here is that when that flag is not enabled, it seems like the resize2fs and commands that run on the image file beforehand might actually screw up the image file (if it is qcow2). Has anyone tried this flag with a false value and used qcow2 images. Is there a patch to ubuntus resize2fs that makes it work with qcow2 images. I don't see how this could have worked without something like that previously (if ever). It seems like the right way forward is to use libguestfs resize (which requires disk images to have a partition). On 6/4/12 10:12 AM, Joshua Harlow harlo...@yahoo-inc.com wrote: Hi all, I was wondering if there has been anyone else who has used this flag with libguestfs (on RH 6.2) that has noticed file sync issues. force_raw_images=true I have been turning that to false so that images need not be expanded for actual usage (it seems this only affects the base images in _base) but when turning that to false and when file injection occurs, it seems that the files get written (I've added os.listdir and such for debugging) but when unmounted and later that image is started it seems like those files either do not get synced back to the image (I even added a manual 'sync' call) or there is some other corruption that happens that causes those images to not contain those files when started (for reference I am using qcow2 files). Has anyone else seen this type of issue? It seems when it is set to true it is not a problem, but when it isn't then some weirdness happens -Josh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] depend discrepancies
+100 for a list. Please feel free to use something like what anvil has... https://github.com/yahoo/Openstack-Anvil/blob/master/conf/distros/ubuntu-oneiric.yaml#L64 Or a subset... On 6/5/12 8:52 AM, Monty Taylor mord...@inaugust.com wrote: Hey guys! One of the things that came out of ODS is the idea of having a single global dependency list. There are two bits to that - the mechanics of managing the list (which I think we may have sorted) and then, you know, making the list. In compiling the list of what the current global list is, most of the things have clear and obvious answers. However, there are three problematic deps: kombu, PasteDeploy and xattr. Here's what we have for them: ./melange/tools/pip-requires:kombu==1.5.1 ./glance/tools/pip-requires:kombu ./nova/tools/pip-requires:kombu==1.0.4 ./cinder/tools/pip-requires:kombu==1.0.4 ./keystone/tools/pip-requires:PasteDeploy ./melange/tools/pip-requires:PasteDeploy ./quantum/tools/pip-requires:PasteDeploy==1.5.0 ./glance/tools/pip-requires:PasteDeploy ./nova/tools/pip-requires:PasteDeploy==1.5.0 ./swift/tools/pip-requires:pastedeploy==1.3.3 ./cinder/tools/pip-requires:PasteDeploy==1.5.0 ./glance/tools/pip-requires:xattr=0.6.0 ./swift/tools/pip-requires:xattr==0.4 I have no personal emotions towards any of these versions ... but if folks who matter could make some call on what they _should_ be, we can move forward with the first step. I should point out that at the moment we're looking at using update.py from openstack-common to copy in the relevant depends from the master list - so a project will only get the ones it needs. It also means that a decision on a version here does not mean that everyone needs to move to that version immediately ... just that we should be moving towards supporting those versions before folsom, really. Anywhoo... thoughts? Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Question on nova disk injection...
Hi all, Just some questions that I had about how nova is doing disk injection and such. I was noticing that it the main disk/api.py does a lot of tee, cat and similar commands. Is there any reason it couldn't just use the standard python open and write data and such. Is it because of sudo access (which is connected to rootwrap?), just wondering since it seems sort of odd that to write a file there a tee call has to be done with piped input, when python already has file operators and such... Thx! ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Question on nova disk injection...
Why couldn't nova just escalate pythons privileges to the super user when writing a file (thus allowing it to use python file writing functions and such). Then after it writes it could drop it back to down to some other user? That might make sense, idk, instead of having the disk injection act like a shell script which basically just emits a bunch of [tee, mv, touch, mkdir, cp] commands. I've done something like this for anvil, not sure if its useful here but who knows: https://github.com/yahoo/Openstack-Anvil/blob/master/anvil/shell.py#L70 On 6/5/12 2:50 PM, Russell Bryant rbry...@redhat.com wrote: On 06/05/2012 05:42 PM, Joshua Harlow wrote: Hi all, Just some questions that I had about how nova is doing disk injection and such. I was noticing that it the main disk/api.py does a lot of tee, cat and similar commands. Is there any reason it couldn't just use the standard python open and write data and such. Is it because of sudo access (which is connected to rootwrap?), just wondering since it seems sort of odd that to write a file there a tee call has to be done with piped input, when python already has file operators and such... Yes, if it is using run_as_root=True, then it has to be run with nova-rootwrap. -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Question on nova disk injection...
Interesting, darn, that sorta makes it harder than it seems like it should be. Is there any pattern that we can follow for this that other programs use, do most other programs shell out as root, and expect there command sets to be restricted? Do other similar programs just assume that they are running as a user that won't need to be restricted? Java seems like it would have the same issue, but of course its threaded, I there any similar concept there of temporarily escalating privileges for a thread, performing some action, then reducing privileges? I wonder if eventlet could support something like this (or be modified to?). Anyone else know other ways of doing this that might be useful? The suggestions that involve RPC being one way. On 6/5/12 5:35 PM, Eric Windisch e...@cloudscaling.com wrote: On Tuesday, June 5, 2012 at 19:18 PM, Joshua Harlow wrote: Re: [Openstack] Question on nova disk injection... Why couldn't nova just escalate pythons privileges to the super user when writing a file (thus allowing it to use python file writing functions and such). Because we use Eventlet. os.setuid applies to the entire process. Coroutine switching during this would be dangerous. Three options seem to exist: 1. We can fork, but then we'll need use IPC, which in our case would be implemented via the RPC abstraction. We would need to make changes to services.py and/or the binaries and possibly the RPC abstraction itself. This would work well with ZeroMQ as it would be actual IPC, but the brokered RPC solutions would be less efficient. Overall, this reeks of complexity and danger, but the end result should be a clear net positive. 2. One less elegant, but easy, solution might just be to extend the rootwrap functionality. Have it support calling commands on the system *and* executing trusted Python methods with trusted arguments. We'd still be shelling out to rootwrap, but rootwrap could internally provide 'mkdir' and 'chmod' style commands around the os stdlib, rather than shelling out a second time, as it does currently. 3. rootwrap itself could actually be implemented as a Nova service, if we could trust the RPC mechanism direct access to the rootwrap methods -- which we is not all too safe, currently. This would effectively be a mix of options 1/2. I'm inclined to suggest option #2 as it is a relatively simple improvement that would give us short-term gains without much friction. This wouldn't exclude the other options from being worked on and seems to be a requirement for #3, anyway. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Libguestfs weirdness??
Hi all, I was wondering if there has been anyone else who has used this flag with libguestfs (on RH 6.2) that has noticed file sync issues. force_raw_images=true I have been turning that to false so that images need not be expanded for actual usage (it seems this only affects the base images in _base) but when turning that to false and when file injection occurs, it seems that the files get written (I've added os.listdir and such for debugging) but when unmounted and later that image is started it seems like those files either do not get synced back to the image (I even added a manual 'sync' call) or there is some other corruption that happens that causes those images to not contain those files when started (for reference I am using qcow2 files). Has anyone else seen this type of issue? It seems when it is set to true it is not a problem, but when it isn't then some weirdness happens -Josh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack with CentOS 6.2
Try anvil. The y! peps run on rhel6.2 so it def should work (minus swift). http://anvil.readthedocs.org/en/latest/index.html On 5/31/12 12:39 AM, Vogel Nicolas nicolas.vo...@heig-vd.ch wrote: Hi everybody, I'm trying to install Openstack (Essex release) with CentOS 6.2 and I have a lot of problems because of dependencies and a deficiency of installation documents. I think I get all the actual packages and I try to make the install with the official doc, which is based on Ubuntu. So I have to search and modify at every step, and that's very difficult for me. For example, beginning with keystone, the syntax for creating tenant, users and roles is different. The keystone.conf file must be changed too. And then I tried to continue with glance, but both .ini files are missing and I don't know if I have to create them or if I have to adapt the .conf files. If someone could give me documentation or a link for the installation of Essex release on CentOS/RHEL 6.2, I would be really gratefull. Thanks for the help, Best regards, Nicolas Vogel Collaborateur scientifique Institut ICT Route de Cheseaux 1 1400 Yverdon-les-Bains Tél. : (+41) 24 557 64 74 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] centos 6 images
Thanks much, feel free to submit some patches, I'll work on the wiki in my off-time, since we have people here making these images for the openstack deployment we are doing and I guess they want me working on other stuff at work, haha. On 5/25/12 7:24 PM, Jason Ford ja...@chatinara.com wrote: Joshua, First off, thanks for getting something together. I think you have a bug in your spec file at line 1. After that I got it to build after renaming some directories. I am in the process of testing this out now in a devstack install. I will give you feedback when I get it. jason - Original Message - From: Joshua Harlow harlo...@yahoo-inc.com To: Jason Ford ja...@chatinara.com Cc: Fedora Cloud SIG cl...@lists.fedoraproject.org, Andy Grimm agr...@gmail.com, openstack openstack@lists.launchpad.net, Pádraig Brady p...@draigbrady.com Sent: Thursday, May 24, 2012 9:18:06 PM Subject: Re: [Openstack] centos 6 images Starting this @ https://github.com/yahoo/Openstack-Condense/wiki/How-To-Use-This I'll try to finish it up soon :-P On 5/22/12 6:33 PM, Joshua Harlow harlo...@yahoo-inc.com wrote: Let me write something up that should explain this. Its not that hard. On 5/22/12 6:31 PM, Jason Ford ja...@chatinara.com wrote: Joshua, Do you have some basic instructions on how to push this into an image and configure it? Any information about what you have here would be great! jason - Original Message - From: Joshua Harlow harlo...@yahoo-inc.com To: Jason ja...@chatinara.com , Pádraig Brady p...@draigbrady.com Cc: Fedora Cloud SIG cl...@lists.fedoraproject.org , Andy Grimm agr...@gmail.com , openstack openstack@lists.launchpad.net Sent: Tuesday, May 22, 2012 1:49:06 PM Subject: Re: [Openstack] centos 6 images U might want to check out, https://github.com/yahoo/Openstack-Condense Its a stripped down/cleaned up/... version of cloud-init that I know works on RHEL6. I tried to improve the following: 1. Code cleanliness (constants being uppercase, paths using os.path.join and so-on) 2. Stripping out some of the odd handlers (byobu, right-scale and such) 3. Improving logging by a lot (so that u can debug this thing) 4. Making what handlers I left work on RH and ubuntu... Might be useful if u want to try it. I know just from doing the above work that the cloud-init for ubuntu, requires some work to get it to work on RH, but not tons, eventually I hope that I can merge this back, but for now its forked so that I could focus on getting it working and cleaned up, rather than pushing code through some review process via launchpad and such (ie the slow as molasses approach). On 5/22/12 10:05 AM, Jason ja...@chatinara.com wrote: I will give these a shot later today and reply with feedback. Thanks for looking into this! Jason On May 22, 2012, at 11:44 AM, Pádraig Brady p...@draigbrady.com wrote: On 05/22/2012 03:39 PM, Andy Grimm wrote: On Tue, May 22, 2012 at 9:38 AM, Pádraig Brady p...@draigbrady.com wrote: On 05/22/2012 04:07 AM, Jason Ford wrote: I am trying to put together an image for centos 6 that works like cloud-init on ubuntu does. Currently I have ssh keys getting imported but having some problems getting the disk to dynamically resize to the flavor template as well as the hostname set in horizon to be pushed into the image. Does anyone have any howtos or suggestions on how to get this done? Is there cloud-init for centos just like ubuntu? I would also be interested in how to do this with debian as well. Well I notice there is no cloud-init package for EPEL. I took a quick stab at it here: http://pbrady.fedorapeople.org/cloud-init-el6/ I've already responded in IRC, but it wouldn't hurt to have a response in the mail archive. In short, the reason there isn't already a cloud-init for EL6 (or EL5, for that matter) is that upstream has been using python 2.7-only calls for a while now. In particular, a couple of calls to subprocess.check_output need to be replaced, and I think there are a few other issues as well. I don't think it's a huge amount of work to make it functional, but it hasn't been high on anyone's list. It would be cool if you have time to fix / test it, though. Ok I've fixed the check_output calls at the above URL. cheers, Pádraig. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Uploading images to glance
U can check out the following: https://github.com/yahoo/Openstack-Anvil/blob/master/anvil/helpers/glance.py#L292 It should be pretty easy to follow, it uses the glance and keystone python clients to upload, given an archive it will try to find the kernel//root/ramdisk images and then upload them with the corresponding metadata and so on... Should be relatively easy to follow. On 5/25/12 10:13 AM, Guilherme Birk guib...@hotmail.com wrote: I'm using cloud-publish to upload images to glance. All my infraestructure is working fine using essex. Now I'm creating a small API using Python and I don't know how to upload an image with a kernel file and a ram-disk file using cURL. I just found a command to upload a single file at the Glance API documentation. Any suggestion of how can I do this upload ? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Nodejs in horizon
Hi all, I was seeing that node.js is now being used in horizon. Is there any details on why that was needed, the reasoning, the technical docs on where it is used. Are there packages available in fedora/ubuntu for this? Such a change seems like it should have a little more reasoning/explanation that what I found @ https://github.com/openstack-dev/devstack/commit/0c2891558122aa9d030811109536caf5c81cfb75 or https://blueprints.launchpad.net/horizon/+spec/transition-to-lesscss Do we really need to have that ?? :-/ -Josh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nodejs in horizon
So was there thought about the fedora and other distributions when adding this as a dependency. I thought we were going to try to support both, but if a package is currently only in a single distribution, that makes it hard to develop on both. Not sure if this is valid, and it might be helpful: http://nodejs.tchol.org/ It seems like this hairy wart is coming back up again, ie, not including stuff on only one distro, or at least planning ahead accordingly to how the other distros can get it... This is especially important in CI/integration tests, where now without a special package, how can this stuff be tested in the other distros... -Josh On 5/24/12 12:27 PM, Chuck Short chuck.sh...@canonical.com wrote: Hi On Thu, 24 May 2012 10:33:32 -0700 Joshua Harlow harlo...@yahoo-inc.com wrote: Hi all, I was seeing that node.js is now being used in horizon. Is there any details on why that was needed, the reasoning, the technical docs on where it is used. Are there packages available in fedora/ubuntu for this? Yes in ubuntu Such a change seems like it should have a little more reasoning/explanation that what I found @ https://github.com/openstack-dev/devstack/commit/0c2891558122aa9d030811109536caf5c81cfb75 or https://blueprints.launchpad.net/horizon/+spec/transition-to-lesscss Do we really need to have that ?? :-/ -Josh Regards chuck ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nodejs in horizon
That's fine, I just want to make sure there is a vetting process for new packages. With documented reasons, documentation on how the distros will get to having that package (if they don't) and so on. Seeing that this is a multiple distribution project, it seems pretty relevant to make sure that everyone that is affected by this, is ready for it, especially for devs. Maybe that's just me missing some IRC log somewhere, but I just wanted to raise this concern on behalf of others (and myself). It'd be nice to have this vetting process be something more formal imho, but I'll let the community decide... On 5/24/12 12:43 PM, Gabriel Hurley gabriel.hur...@nebula.com wrote: Hi Josh, As for packages, I can't speak to Fedora offhand, but Ubuntu has the nodejs package which is what we've used internally for development and for the devstack gate going forward. The LESS binary itself is being bundled with Horizon to alleviate versioning incompatibilities and package differences across OS's. As for why, having been involved in the discussion and decision let me present a few components of the reasoning as they relate to Horizon's core values (expressed here: http://horizon.openstack.org/intro.html#values): 1. Manageable: The CSS stylesheet for the OpenStack Dashboard had become unmaintainable. It was and (until we split it up after LESS lands) is a complete mess, over 1000 lines long. We needed a solution that continues to allow us to build in a more reasonable fashion. There are plenty of modern tools for this (LESS, SASS, etc.). We can split apart the stylesheet into logical modules, define variables, mixins and transformations so we update code in one place rather than twenty... so on and so forth. 2. Extensible: LESS allows third-party developers to import and build from the core stylesheet(s) instead of copying and pasting the stylesheet wholesale, causing the pain of staying up-to-date with changes, bugfixes, etc. 3. Consistent: As mentioned above, we can define variables and mixins which can be used everywhere, preventing the I wrote the same styles ten places and forgot to update them all problems that every frontend developer knows all too well. 4. Stable: At this point node.js and LESS are well-proven technologies, being leveraged by large companies and large projects all over the globe. 5. Usable: From a developer standpoint, LESS and SASS are light-years ahead of static CSS in terms of writing code. The end product to the consumer is more-or-less identical. We did evaluate a number of other implementation options for meeting these needs (develop with LESS and commit both the .less and final .css files, using SASS which is built on Ruby, etc.) but came to this as the best long-term solution. All the best, - Gabriel From: openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net [mailto:openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net] On Behalf Of Joshua Harlow Sent: Thursday, May 24, 2012 10:34 AM To: openstack Cc: Yahoo Openstack Developers Subject: [Openstack] Nodejs in horizon Hi all, I was seeing that node.js is now being used in horizon. Is there any details on why that was needed, the reasoning, the technical docs on where it is used. Are there packages available in fedora/ubuntu for this? Such a change seems like it should have a little more reasoning/explanation that what I found @ https://github.com/openstack-dev/devstack/commit/0c2891558122aa9d030811109536caf5c81cfb75 or https://blueprints.launchpad.net/horizon/+spec/transition-to-lesscss Do we really need to have that ?? :-/ -Josh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nodejs in horizon
Good to know, I thought that fedora was being ran as well in the CI env. If not, I will try my best to get somebody or something here @ y! to make this happen (with fedora or rhel6)... -Josh On 5/24/12 1:42 PM, Gabriel Hurley gabriel.hur...@nebula.com wrote: I agree with the larger problem of being mindful about our supported distros. That said, most of us are (at best) knowledgeable about a single distro's bundled packages. This is amplified by the fact that our CI infrastructure is only gated on a single distro (Ubuntu) currently. As you noted, the nodejs.tchol.org site seems to offer useful resources for running node on Fedora. If someone would like to work up a quick writeup of best-practices for installing/configuring node.js on Fedora (I'm not a Fedora user, sorry) I'd be happy to help get it included in the docs. - Gabriel From: openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net [mailto:openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net] On Behalf Of Joshua Harlow Sent: Thursday, May 24, 2012 12:32 PM To: Chuck Short Cc: Yahoo Openstack Developers; openstack Subject: Re: [Openstack] Nodejs in horizon So was there thought about the fedora and other distributions when adding this as a dependency. I thought we were going to try to support both, but if a package is currently only in a single distribution, that makes it hard to develop on both. Not sure if this is valid, and it might be helpful: http://nodejs.tchol.org/ It seems like this hairy wart is coming back up again, ie, not including stuff on only one distro, or at least planning ahead accordingly to how the other distros can get it... This is especially important in CI/integration tests, where now without a special package, how can this stuff be tested in the other distros... -Josh On 5/24/12 12:27 PM, Chuck Short chuck.sh...@canonical.com wrote: Hi On Thu, 24 May 2012 10:33:32 -0700 Joshua Harlow harlo...@yahoo-inc.com wrote: Hi all, I was seeing that node.js is now being used in horizon. Is there any details on why that was needed, the reasoning, the technical docs on where it is used. Are there packages available in fedora/ubuntu for this? Yes in ubuntu Such a change seems like it should have a little more reasoning/explanation that what I found @ https://github.com/openstack-dev/devstack/commit/0c2891558122aa9d030811109536caf5c81cfb75 or https://blueprints.launchpad.net/horizon/+spec/transition-to-lesscss Do we really need to have that ?? :-/ -Josh Regards chuck ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nodejs in horizon
That's what I was worried about, and why I (just my opinion) think that we need to be a lot stricter about vetting new dependencies. There needs to be time given to say, ensuring that its really needed, if it really is, documenting why it has to be there in depth, getting various PTL's to agree on that, then comes the process of informing and/or discussing with the distributions about how to get that supported (if it isn't). That second stage might influence the first and so on. Since problems like this, although yes, it slows down development overall, seem necessary in a larger project with multiple distributions being enabled (for development usage and for actual prod. usage). On 5/24/12 3:38 PM, Russell Bryant rbry...@redhat.com wrote: On 05/24/2012 03:40 PM, Devin Carlen wrote: Hi Joshua, Node.js is in the standard repos for most modern distros. It's not an issue for Ubuntu/Fedora. It actually is a problem for Fedora. node.js is not in Fedora. Once Horizon requires node.js, it will be broken for Fedora (and EPEL for use with RHEL and its derivatives). https://bugzilla.redhat.com/show_bug.cgi?id=815018 -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nodejs in horizon
Sure I agree with what u said, its a balance... But it worries me when a commit pops up that I had to basically find, and then I get a reply from russell who works at RH that says nope fedora doesn't have it. It seems like that reach u mentioned, wasn't occurring, idk, maybe that is the process, just more of notifying people/companies... Something seems broken if that can happen. I agree its early, but why was it problem in the first place? On 5/24/12 4:22 PM, Devin Carlen de...@openstack.org wrote: -1 to introducing formal processes around this. This will happen from time to time. Development may be briefly impacted on other platforms but hindering innovation and telling developers that they are responsible for package availability across every distro is not healthy. You are concerned about Fedora which is a perfectly reasonable stance. But what about CentOS, RHEL, Debian, etc. I for once don't want to have my technical options limited by whether or not a package is currently maintained by a distro. We have a lot of reach into these communities and we are still in the earliest stages of Folsom. There is plenty of time to deal with this. That said, this is why we made this change in the *first* milestone, and not a week before release. ;) Devin On May 24, 2012, at 4:09 PM, Joshua Harlow wrote: Re: [Openstack] Nodejs in horizon That's what I was worried about, and why I (just my opinion) think that we need to be a lot stricter about vetting new dependencies. There needs to be time given to say, ensuring that its really needed, if it really is, documenting why it has to be there in depth, getting various PTL's to agree on that, then comes the process of informing and/or discussing with the distributions about how to get that supported (if it isn't). That second stage might influence the first and so on. Since problems like this, although yes, it slows down development overall, seem necessary in a larger project with multiple distributions being enabled (for development usage and for actual prod. usage). On 5/24/12 3:38 PM, Russell Bryant rbry...@redhat.com x-msg://602/rbry...@redhat.com wrote: On 05/24/2012 03:40 PM, Devin Carlen wrote: Hi Joshua, Node.js is in the standard repos for most modern distros. It's not an issue for Ubuntu/Fedora. It actually is a problem for Fedora. node.js is not in Fedora. Once Horizon requires node.js, it will be broken for Fedora (and EPEL for use with RHEL and its derivatives). https://bugzilla.redhat.com/show_bug.cgi?id=815018 -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] centos 6 images
Starting this @ https://github.com/yahoo/Openstack-Condense/wiki/How-To-Use-This I'll try to finish it up soon :-P On 5/22/12 6:33 PM, Joshua Harlow harlo...@yahoo-inc.com wrote: Let me write something up that should explain this. Its not that hard. On 5/22/12 6:31 PM, Jason Ford ja...@chatinara.com wrote: Joshua, Do you have some basic instructions on how to push this into an image and configure it? Any information about what you have here would be great! jason - Original Message - From: Joshua Harlow harlo...@yahoo-inc.com To: Jason ja...@chatinara.com, Pádraig Brady p...@draigbrady.com Cc: Fedora Cloud SIG cl...@lists.fedoraproject.org, Andy Grimm agr...@gmail.com, openstack openstack@lists.launchpad.net Sent: Tuesday, May 22, 2012 1:49:06 PM Subject: Re: [Openstack] centos 6 images U might want to check out, https://github.com/yahoo/Openstack-Condense Its a stripped down/cleaned up/... version of cloud-init that I know works on RHEL6. I tried to improve the following: 1. Code cleanliness (constants being uppercase, paths using os.path.join and so-on) 2. Stripping out some of the odd handlers (byobu, right-scale and such) 3. Improving logging by a lot (so that u can debug this thing) 4. Making what handlers I left work on RH and ubuntu... Might be useful if u want to try it. I know just from doing the above work that the cloud-init for ubuntu, requires some work to get it to work on RH, but not tons, eventually I hope that I can merge this back, but for now its forked so that I could focus on getting it working and cleaned up, rather than pushing code through some review process via launchpad and such (ie the slow as molasses approach). On 5/22/12 10:05 AM, Jason ja...@chatinara.com wrote: I will give these a shot later today and reply with feedback. Thanks for looking into this! Jason On May 22, 2012, at 11:44 AM, Pádraig Brady p...@draigbrady.com wrote: On 05/22/2012 03:39 PM, Andy Grimm wrote: On Tue, May 22, 2012 at 9:38 AM, Pádraig Brady p...@draigbrady.com wrote: On 05/22/2012 04:07 AM, Jason Ford wrote: I am trying to put together an image for centos 6 that works like cloud-init on ubuntu does. Currently I have ssh keys getting imported but having some problems getting the disk to dynamically resize to the flavor template as well as the hostname set in horizon to be pushed into the image. Does anyone have any howtos or suggestions on how to get this done? Is there cloud-init for centos just like ubuntu? I would also be interested in how to do this with debian as well. Well I notice there is no cloud-init package for EPEL. I took a quick stab at it here: http://pbrady.fedorapeople.org/cloud-init-el6/ I've already responded in IRC, but it wouldn't hurt to have a response in the mail archive. In short, the reason there isn't already a cloud-init for EL6 (or EL5, for that matter) is that upstream has been using python 2.7-only calls for a while now. In particular, a couple of calls to subprocess.check_output need to be replaced, and I think there are a few other issues as well. I don't think it's a huge amount of work to make it functional, but it hasn't been high on anyone's list. It would be cool if you have time to fix / test it, though. Ok I've fixed the check_output calls at the above URL. cheers, Pádraig. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] centos 6 images
U might want to check out, https://github.com/yahoo/Openstack-Condense Its a stripped down/cleaned up/... version of cloud-init that I know works on RHEL6. I tried to improve the following: 1. Code cleanliness (constants being uppercase, paths using os.path.join and so-on) 2. Stripping out some of the odd handlers (byobu, right-scale and such) 3. Improving logging by a lot (so that u can debug this thing) 4. Making what handlers I left work on RH and ubuntu... Might be useful if u want to try it. I know just from doing the above work that the cloud-init for ubuntu, requires some work to get it to work on RH, but not tons, eventually I hope that I can merge this back, but for now its forked so that I could focus on getting it working and cleaned up, rather than pushing code through some review process via launchpad and such (ie the slow as molasses approach). On 5/22/12 10:05 AM, Jason ja...@chatinara.com wrote: I will give these a shot later today and reply with feedback. Thanks for looking into this! Jason On May 22, 2012, at 11:44 AM, Pádraig Brady p...@draigbrady.com wrote: On 05/22/2012 03:39 PM, Andy Grimm wrote: On Tue, May 22, 2012 at 9:38 AM, Pádraig Brady p...@draigbrady.com wrote: On 05/22/2012 04:07 AM, Jason Ford wrote: I am trying to put together an image for centos 6 that works like cloud-init on ubuntu does. Currently I have ssh keys getting imported but having some problems getting the disk to dynamically resize to the flavor template as well as the hostname set in horizon to be pushed into the image. Does anyone have any howtos or suggestions on how to get this done? Is there cloud-init for centos just like ubuntu? I would also be interested in how to do this with debian as well. Well I notice there is no cloud-init package for EPEL. I took a quick stab at it here: http://pbrady.fedorapeople.org/cloud-init-el6/ I've already responded in IRC, but it wouldn't hurt to have a response in the mail archive. In short, the reason there isn't already a cloud-init for EL6 (or EL5, for that matter) is that upstream has been using python 2.7-only calls for a while now. In particular, a couple of calls to subprocess.check_output need to be replaced, and I think there are a few other issues as well. I don't think it's a huge amount of work to make it functional, but it hasn't been high on anyone's list. It would be cool if you have time to fix / test it, though. Ok I've fixed the check_output calls at the above URL. cheers, Pádraig. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] centos 6 images
Let me write something up that should explain this. Its not that hard. On 5/22/12 6:31 PM, Jason Ford ja...@chatinara.com wrote: Joshua, Do you have some basic instructions on how to push this into an image and configure it? Any information about what you have here would be great! jason - Original Message - From: Joshua Harlow harlo...@yahoo-inc.com To: Jason ja...@chatinara.com, Pádraig Brady p...@draigbrady.com Cc: Fedora Cloud SIG cl...@lists.fedoraproject.org, Andy Grimm agr...@gmail.com, openstack openstack@lists.launchpad.net Sent: Tuesday, May 22, 2012 1:49:06 PM Subject: Re: [Openstack] centos 6 images U might want to check out, https://github.com/yahoo/Openstack-Condense Its a stripped down/cleaned up/... version of cloud-init that I know works on RHEL6. I tried to improve the following: 1. Code cleanliness (constants being uppercase, paths using os.path.join and so-on) 2. Stripping out some of the odd handlers (byobu, right-scale and such) 3. Improving logging by a lot (so that u can debug this thing) 4. Making what handlers I left work on RH and ubuntu... Might be useful if u want to try it. I know just from doing the above work that the cloud-init for ubuntu, requires some work to get it to work on RH, but not tons, eventually I hope that I can merge this back, but for now its forked so that I could focus on getting it working and cleaned up, rather than pushing code through some review process via launchpad and such (ie the slow as molasses approach). On 5/22/12 10:05 AM, Jason ja...@chatinara.com wrote: I will give these a shot later today and reply with feedback. Thanks for looking into this! Jason On May 22, 2012, at 11:44 AM, Pádraig Brady p...@draigbrady.com wrote: On 05/22/2012 03:39 PM, Andy Grimm wrote: On Tue, May 22, 2012 at 9:38 AM, Pádraig Brady p...@draigbrady.com wrote: On 05/22/2012 04:07 AM, Jason Ford wrote: I am trying to put together an image for centos 6 that works like cloud-init on ubuntu does. Currently I have ssh keys getting imported but having some problems getting the disk to dynamically resize to the flavor template as well as the hostname set in horizon to be pushed into the image. Does anyone have any howtos or suggestions on how to get this done? Is there cloud-init for centos just like ubuntu? I would also be interested in how to do this with debian as well. Well I notice there is no cloud-init package for EPEL. I took a quick stab at it here: http://pbrady.fedorapeople.org/cloud-init-el6/ I've already responded in IRC, but it wouldn't hurt to have a response in the mail archive. In short, the reason there isn't already a cloud-init for EL6 (or EL5, for that matter) is that upstream has been using python 2.7-only calls for a while now. In particular, a couple of calls to subprocess.check_output need to be replaced, and I think there are a few other issues as well. I don't think it's a huge amount of work to make it functional, but it hasn't been high on anyone's list. It would be cool if you have time to fix / test it, though. Ok I've fixed the check_output calls at the above URL. cheers, Pádraig. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] 'admin' role hard-coded in keystone and nova, and policy.json
Cool, I'm glad that is the ultimate goal. It seems like nova should be asking keystone for an initial policy template of some kind, which nova then fills in its specifics with or policies can be fully defined in keystone, either or. Just people should be aware that making custom roles might not mean much if policy.json files are not also updated. On 5/11/12 10:51 AM, Vishvananda Ishaya vishvana...@gmail.com wrote: Most of nova is configurable via policy.json, but there is the issue with context.is_admin checks that still exist in a few places. We definitely need to modify that. Joshua, the idea is that policy.json will ultimately be managed in keystone as well. Currently the policy.json is checked for modifications, so it would be possible to throw it on shared storage and modify it for every node at once without having to restart the nodes. This is an interim solution until we allow for creating and retrieving policies inside of keystone. Vish On Thu, May 10, 2012 at 7:13 PM, Joshua Harlow harlo...@yahoo-inc.com wrote: I was also wondering about this, it seems there are lots of policy.json files with hard coded roles in them, which is weird since keystone supports the creation of roles and such, but if u create a role which isn't in a policy.json then u have just caused yourself a problem, which isn't very apparent... On 5/10/12 2:32 PM, Salman A Baset saba...@us.ibm.com http://saba...@us.ibm.com wrote: It seems that 'admin' role is hard-coded cross nova and horizon. As a result if I want to define 'myadmin' role, and grant it all the admin privileges, it does not seem possible. Is this a recognized limitation? Further, is there some good documentation on policy.json for nova, keystone, and glance? Thanks. Best Regards, Salman A. Baset Research Staff Member, IBM T. J. Watson Research Center Tel: +1-914-784-6248 tel:%2B1-914-784-6248 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] 'admin' role hard-coded in keystone and nova, and policy.json
I was also wondering about this, it seems there are lots of policy.json files with hard coded roles in them, which is weird since keystone supports the creation of roles and such, but if u create a role which isn't in a policy.json then u have just caused yourself a problem, which isn't very apparent... On 5/10/12 2:32 PM, Salman A Baset saba...@us.ibm.com wrote: It seems that 'admin' role is hard-coded cross nova and horizon. As a result if I want to define 'myadmin' role, and grant it all the admin privileges, it does not seem possible. Is this a recognized limitation? Further, is there some good documentation on policy.json for nova, keystone, and glance? Thanks. Best Regards, Salman A. Baset Research Staff Member, IBM T. J. Watson Research Center Tel: +1-914-784-6248 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Keystone client, user belongs to many tenants?
A question,
I am using anvil to setup the keystone roles/users/tenants.
It seems like the python keystone client has the following command:
client.users.create
Which seems to take in the following:
create(self, name, password, email, tenant_id=None, enabled=True):
I would assume a user name can be used in multiple tenants but when I am trying
to create a user that spans tenants and it seems like it borks.
ClientException: Conflict occurred attempting to store user. (IntegrityError)
(1062, Duplicate entry 'admin' for key 'name') 'INSERT INTO user (id, name,
extra) VALUES (%s, %s, %s)' ('3e14a9c1fd404c7e81c0dba8bd640575', 'admin',
'{password:
$6$rounds=4$yX5fL51OyGKjuPjr$8yv.S3GpqsKeaHv4GjNY4YW2vvykWzrEV7RX.qJpyy3CjmyXrZMRRJifEzfa7xv1l.NzoggQBXUAESn3Oqm0x/,
enabled: true, email: ad...@example.com, tenantId:
d1506184877a449a91fc6adcb553ad97}') (HTTP 409)
Is this supposed to happen? Is the client supposed to send back this much info
also (hashed password??) :-P
Any ideas?
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Nova] EC2 api testing
So the tempest approach I think starts to pull this EC2 validation suite into openstack to much. A goal that I think is useful is to have this not that tightly integrated with openstack, so that say cloudstack, and others can use this type of suite as well. That way everyone benefits and everyone can contribute (not just the openstack pep's). I'd personally like to see the following kinds of validations/compliance checks: Request compliance: 1. Send in X request, expect Y response (or at least Z sub-fields of Y response) to match known valid response Y2 * This means we have a need for a whole bunch of requests, a whole bunch of valid responses and a way to determine differences (xmlunit in java provides some of this, I don't know of anything in python). I have created something like this (a framework) that was used for the yahoo-bing ad provider xml transtiion project stuff here, that we might be able to use. Schema compliance: 1. Something along the lines of validating XSDs? Amazon has about 10 versions of there api's, which ones are u complaint with (or that kind of question is to be answered here). * Questions as to how strict and such need to be tackled here (all the python XSD validators seem to blow up on the first error, not so useful when trying to see multiple errors) * https://github.com/yahoo/Openstack-EC2/tree/master/data/xsds Functional compliance: 1. This is where we start to say use boto (which itself uses a very tolerant SAX parser to create python objects) and test anything the above 2 types of tests could not accomplish (say u need a if statement to check some conditions, the above 2 ways probably won't get that). Euca2ools runs on boto, so I would consider that the same case, but using a library might not be the best approach because then u start to run into the issue of the library is not compliant either (ie testing the library instead of the apis). Documentation compliance: 1. This is I guess where we or others document how compliant a suite target is and what the known issues with it are. That was my thoughts anyway :-) - Josh On 5/8/12 8:32 AM, John Garbutt john.garb...@citrix.com wrote: I am certainly up for helping with this effort. I wondered about this approach: * Starting with Tempest (mostly for its reporting, and configuration) * Creating a new category EC2_Compat or something like that * Trying to add in all the tests from those two repos into the new thing * Looking at what the gaps are I am not sure it makes sense for these tests to leave in tempest, but it does seem silly to create our own set of configuration and test runner code, if we don't have to. Seems too early to push this shared code into something like openstack-common-tests, but maybe that is what we need long term? A totally different approach, that sounds attractive, is testing with some of the tools people actually want to use: boto, eucatools. However, this does then mean you end up writing N times as many tests, and it means you have to somehow pick which tools you want to test. Maybe the tests are easier to write, so the fact there is more doesn't matter. While those tests don't really test if we comply with ec2, maybe they do test what people care about. Cheers, John From: openstack-bounces+john.garbutt=eu.citrix@lists.launchpad.net [mailto:openstack-bounces+john.garbutt=eu.citrix@lists.launchpad.net] On Behalf Of Joshua Harlow Sent: 07 May 2012 18:17 To: Doug Hellmann; Martin Packman Cc: openstack Subject: Re: [Openstack] [Nova] EC2 api testing TBD afaik. I think it would be nice if we could have one tool to rule them all, but I need to get my hands on this enstrsatus thingy to see what is there :-) I've started documenting some EC2 stuff that I see @ https://github.com/yahoo/Openstack-EC2/issues If others want to put stuff on there (or a better location, that's cool with me). On 5/4/12 3:04 PM, Doug Hellmann doug.hellm...@dreamhost.com wrote: On Fri, May 4, 2012 at 1:09 PM, Martin Packman martin.pack...@canonical.com wrote: At the Folsom Design Summit we discussed[1] trying to collaborate on a test suite for EC2 api support. Currently nova supports the common stuff pretty well has differing behaviour in a lot of edge cases. Having a separate suite, along the lines of tempest, that could be run against other existing clouds as well as OpenStack would let us test the tests as well, and would be useful for other projects. Various parties have done work in this direction in the past, the trick is going to be combining it into something we can all use. The existing code I know about includes aws-compat[2], Openstack-EC2[3], the tests in nova itself, some experimental code in awsome, and an Enstratus test suite. I'm hoping to find out more about the Enstratus code, James Urquhart suggested opening the remaining parts would be a reasonable step. Is there anything else out
Re: [Openstack] [Nova] EC2 api testing
Sure, borrowing might be a good start. I think u are right in that we need different levels of test compliance, the more tests the better, just as long as we don't start testing client libs, although I guess that wouldn't be a bad thing either, but I just would rather get the EC2 100% right, and then report client bugs on a different schedule. On 5/8/12 11:39 AM, John Garbutt john.garb...@citrix.com wrote: I was really just thinking of borrowing the test framework around tempest, rather than push the tests back to tempest. I agree we don't want to be too tied to OpenStack. I certainly would like to have this running against CloudStack too. I hoped we could look at doing testing from a user perspective. Maybe having tests like: test_instatance_s3_style_success, test_instance_s3_style_failure_bad_image, etc. Each can check the request, functional and scheme compliance. Could the documentation start a the list of passed and failed tests? Possibly the failure should either be non-compliant or not implemented. Maybe non-compliant could specify which versions it is not compliant with. Cheers, John From: Joshua Harlow [mailto:harlo...@yahoo-inc.com] Sent: 08 May 2012 18:49 To: John Garbutt; Doug Hellmann; Martin Packman; Joe Gordon; jaypi...@gmail.com Cc: openstack Subject: Re: [Openstack] [Nova] EC2 api testing So the tempest approach I think starts to pull this EC2 validation suite into openstack to much. A goal that I think is useful is to have this not that tightly integrated with openstack, so that say cloudstack, and others can use this type of suite as well. That way everyone benefits and everyone can contribute (not just the openstack pep's). I'd personally like to see the following kinds of validations/compliance checks: Request compliance: 1. Send in X request, expect Y response (or at least Z sub-fields of Y response) to match known valid response Y2 * This means we have a need for a whole bunch of requests, a whole bunch of valid responses and a way to determine differences (xmlunit in java provides some of this, I don't know of anything in python). I have created something like this (a framework) that was used for the yahoo-bing ad provider xml transtiion project stuff here, that we might be able to use. Schema compliance: 1. Something along the lines of validating XSDs? Amazon has about 10 versions of there api's, which ones are u complaint with (or that kind of question is to be answered here). * Questions as to how strict and such need to be tackled here (all the python XSD validators seem to blow up on the first error, not so useful when trying to see multiple errors) * https://github.com/yahoo/Openstack-EC2/tree/master/data/xsds Functional compliance: 1. This is where we start to say use boto (which itself uses a very tolerant SAX parser to create python objects) and test anything the above 2 types of tests could not accomplish (say u need a if statement to check some conditions, the above 2 ways probably won't get that). Euca2ools runs on boto, so I would consider that the same case, but using a library might not be the best approach because then u start to run into the issue of the library is not compliant either (ie testing the library instead of the apis). Documentation compliance: 1. This is I guess where we or others document how compliant a suite target is and what the known issues with it are. That was my thoughts anyway :-) - Josh On 5/8/12 8:32 AM, John Garbutt john.garb...@citrix.com wrote: I am certainly up for helping with this effort. I wondered about this approach: *Starting with Tempest (mostly for its reporting, and configuration) *Creating a new category EC2_Compat or something like that *Trying to add in all the tests from those two repos into the new thing *Looking at what the gaps are I am not sure it makes sense for these tests to leave in tempest, but it does seem silly to create our own set of configuration and test runner code, if we don't have to. Seems too early to push this shared code into something like openstack-common-tests, but maybe that is what we need long term? A totally different approach, that sounds attractive, is testing with some of the tools people actually want to use: boto, eucatools. However, this does then mean you end up writing N times as many tests, and it means you have to somehow pick which tools you want to test. Maybe the tests are easier to write, so the fact there is more doesn't matter. While those tests don't really test if we comply with ec2, maybe they do test what people care about. Cheers, John From: openstack-bounces+john.garbutt=eu.citrix@lists.launchpad.net [mailto:openstack-bounces+john.garbutt=eu.citrix@lists.launchpad.net] On Behalf Of Joshua Harlow Sent: 07 May 2012 18:17 To: Doug Hellmann; Martin Packman Cc: openstack Subject: Re: [Openstack] [Nova] EC2 api testing TBD afaik. I think
Re: [Openstack] [Nova] EC2 api testing
TBD afaik. I think it would be nice if we could have one tool to rule them all, but I need to get my hands on this enstrsatus thingy to see what is there :-) I've started documenting some EC2 stuff that I see @ https://github.com/yahoo/Openstack-EC2/issues If others want to put stuff on there (or a better location, that's cool with me). On 5/4/12 3:04 PM, Doug Hellmann doug.hellm...@dreamhost.com wrote: On Fri, May 4, 2012 at 1:09 PM, Martin Packman martin.pack...@canonical.com wrote: At the Folsom Design Summit we discussed[1] trying to collaborate on a test suite for EC2 api support. Currently nova supports the common stuff pretty well has differing behaviour in a lot of edge cases. Having a separate suite, along the lines of tempest, that could be run against other existing clouds as well as OpenStack would let us test the tests as well, and would be useful for other projects. Various parties have done work in this direction in the past, the trick is going to be combining it into something we can all use. The existing code I know about includes aws-compat[2], Openstack-EC2[3], the tests in nova itself, some experimental code in awsome, and an Enstratus test suite. I'm hoping to find out more about the Enstratus code, James Urquhart suggested opening the remaining parts would be a reasonable step. Is there anything else out there we should look at as well? Are there any strong opinions over the right way of getting started on this? Are you going to try to get all of the code for those projects into one package, or build a meta-tool that downloads the others and uses them? I don't have an opinion one way or the other, I'm just curious. Doug Martin [1] Nova EC2 compatibility sesson etherpad http://etherpad.openstack.org/FolsomEC2Compatibility [2] https://github.com/cloudscaling/aws-compat [3] https://github.com/yahoo/Openstack-EC2 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] i18n of log message
I think option 1 is needed, for obvious reasons. API facing messages, not so sure about that, I would say english for those, since they are meant for people interacting with an API and not front-end users. I would think this would be pretty easily solvable by basically following what other open source projects already do. It would be useful possibly to have a survey of other projects and just follow the same pattern?? On 5/7/12 7:53 AM, Andrew Clay Shafer a...@parvuscaptus.com wrote: I would vote for 0 or 1 on the cost versus benefit. Option 0 is the least overhead, but option 1 would be nice for a lot of reasons. The downside to i18n of the logs and errors in the dilution of information available to find solutions can be higher than the benefit of providing messages in a native language. The level of effort is certainly much much higher to provide option 3. I'd vote for effort to go to improving the OpenStack core technology and features over something that adds a lot of overhead and also some downside. On Mon, May 7, 2012 at 3:40 AM, Ying Chun Guo guoyi...@cn.ibm.com wrote: I will vote option 3, because I think API-user-facing messages is as important as user interface messages. Since the workload of option 3 is not much more than option 2, option 3 will be a better choice. btw, I see documentation, e.g. OpenStack manuals, is excluded in these four options. Does that mean there is no comments against the globalization of documentation? Regards Daisy ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Glance store question??
Opened this blueprint, I've started doing part of it, with the simple way of doing it (add a new config list that specifies the stores and import them at startup...) Something like: --- a/etc/glance-api.conf +++ b/etc/glance-api.conf @@ -5,10 +5,17 @@ verbose = True # Show debugging output in logs (sets DEBUG log level output) debug = False -# Which backend store should Glance use by default is not specified +# Which backend scheme should Glance use by default is not specified # in a request to add a new image to Glance? Default: 'file' -# Available choices are 'file', 'swift', and 's3' -default_store = file +default_scheme = file + +# List of which stores (and schemes) are currently known to glance at +# startup and the module+class used to construct that store. +# Default: glance.store.filesystem.Store(filesystem; file) +known_stores = glance.store.filesystem.Store(filesystem; file) + glance.store.http.Store(http; https), + glance.store.rbd.Store(rbd), + glance.store.s3.Store(s3; s3+http; s3+https) And then the necessary changes in glance to read those and import those. https://blueprints.launchpad.net/glance/+spec/import-dynamic-stores On 5/3/12 11:06 AM, Joshua Harlow harlo...@yahoo-inc.com wrote: Ok, although I wonder if a plug-in framework could benefit from just being generic enough to be used in either place? I think he's focusing on the notification system right now (being more pluggable) there, so that be his scope for now. I'm willing to work together to make that more generic so that it isn't limited to just that though, this seems like one place that could be fixed easily either way (either via simple fix or via plugins). On 5/3/12 11:02 AM, Brian Waldon brian.wal...@rackspace.com wrote: He definitely is, but its scope is limited to Nova (for now?). The idea discussed below is something we could use without having to worry about a Glance plugin implementation. We can talk about plugins for Glance later. Brian On May 3, 2012, at 10:58 AM, Doug Hellmann wrote: Andrew is doing some work on a plugin architecture. This sounds like another good application of that. On Wed, May 2, 2012 at 7:01 PM, Joshua Harlow harlo...@yahoo-inc.com wrote: Right, if there isn't that existing, then I think I might just make a blueprint out of that. I just wanted to check beforehand that I am doing this right, or if it already exists and I did it wrong... Thx :-) On 5/2/12 3:57 PM, Brian Waldon brian.wal...@rackspace.com http://brian.wal...@rackspace.com/ wrote: Jay might have a better answer, but as far as I know, yes. You could probably make the images stores truly pluggable (i.e. not needing to explicitly list them out) without much work. Brian On May 2, 2012, at 12:23 PM, Joshua Harlow wrote: Glance store question?? Hi all, I am making a y! specific backing store for glance and I was wondering if its really necessary to modify the following file to ensure that the code for that new store gets pulled in (or maybe I'm just doing it wrong). diff --git a/glance/api/v1/images.py b/glance/api/v1/images.py index 9839d2e..7c4f565 100644 --- a/glance/api/v1/images.py +++ b/glance/api/v1/images.py @@ -47,6 +47,7 @@ import glance.store.http import glance.store.rbd import glance.store.s3 import glance.store.swift +import glance.store.SUPERAWESOMEYSTORE from glance.store import (get_from_backend, get_size_from_backend, schedule_delete_from_backend, Is there another way to make that happen so that it gets registered with glance besides either modifying that file or having a patch which will modify it upon installation (ie an RPM patch)? Thx! -Josh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net http://openstack@lists.launchpad.net/ Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Glance store question??
Ok, although I wonder if a plug-in framework could benefit from just being generic enough to be used in either place? I think he's focusing on the notification system right now (being more pluggable) there, so that be his scope for now. I'm willing to work together to make that more generic so that it isn't limited to just that though, this seems like one place that could be fixed easily either way (either via simple fix or via plugins). On 5/3/12 11:02 AM, Brian Waldon brian.wal...@rackspace.com wrote: He definitely is, but its scope is limited to Nova (for now?). The idea discussed below is something we could use without having to worry about a Glance plugin implementation. We can talk about plugins for Glance later. Brian On May 3, 2012, at 10:58 AM, Doug Hellmann wrote: Andrew is doing some work on a plugin architecture. This sounds like another good application of that. On Wed, May 2, 2012 at 7:01 PM, Joshua Harlow harlo...@yahoo-inc.com wrote: Right, if there isn't that existing, then I think I might just make a blueprint out of that. I just wanted to check beforehand that I am doing this right, or if it already exists and I did it wrong... Thx :-) On 5/2/12 3:57 PM, Brian Waldon brian.wal...@rackspace.com http://brian.wal...@rackspace.com/ wrote: Jay might have a better answer, but as far as I know, yes. You could probably make the images stores truly pluggable (i.e. not needing to explicitly list them out) without much work. Brian On May 2, 2012, at 12:23 PM, Joshua Harlow wrote: Glance store question?? Hi all, I am making a y! specific backing store for glance and I was wondering if its really necessary to modify the following file to ensure that the code for that new store gets pulled in (or maybe I'm just doing it wrong). diff --git a/glance/api/v1/images.py b/glance/api/v1/images.py index 9839d2e..7c4f565 100644 --- a/glance/api/v1/images.py +++ b/glance/api/v1/images.py @@ -47,6 +47,7 @@ import glance.store.http import glance.store.rbd import glance.store.s3 import glance.store.swift +import glance.store.SUPERAWESOMEYSTORE from glance.store import (get_from_backend, get_size_from_backend, schedule_delete_from_backend, Is there another way to make that happen so that it gets registered with glance besides either modifying that file or having a patch which will modify it upon installation (ie an RPM patch)? Thx! -Josh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net http://openstack@lists.launchpad.net/ Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Metering question??
Hi all, I was just looking over the efficient metering stuff yesterday. Just a couple of questions, that might be dups (sorry if they are). I am noticing that there seems to be a mix of billing specifics there and metering specifics there. If say metering can just provide as much raw data as possible then is that not sufficient? Shouldn't that be the limit of openstack's job? Then there can be sets of business units or other people that figure out how to carve up that raw data into billing equivalents. This is especially since every business will have there own idears around billing. In certain situations, I can even think where one would want to use hadoop to calculate some advanced model of all that raw-data and use that for billing, but the key part that keeps on coming up in my brain is just get as much raw data as possible. Thoughts?? -Josh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Glance store question??
Hi all, I am making a y! specific backing store for glance and I was wondering if its really necessary to modify the following file to ensure that the code for that new store gets pulled in (or maybe I'm just doing it wrong). diff --git a/glance/api/v1/images.py b/glance/api/v1/images.py index 9839d2e..7c4f565 100644 --- a/glance/api/v1/images.py +++ b/glance/api/v1/images.py @@ -47,6 +47,7 @@ import glance.store.http import glance.store.rbd import glance.store.s3 import glance.store.swift +import glance.store.SUPERAWESOMEYSTORE from glance.store import (get_from_backend, get_size_from_backend, schedule_delete_from_backend, Is there another way to make that happen so that it gets registered with glance besides either modifying that file or having a patch which will modify it upon installation (ie an RPM patch)? Thx! -Josh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Glance store question??
Right, if there isn't that existing, then I think I might just make a blueprint out of that. I just wanted to check beforehand that I am doing this right, or if it already exists and I did it wrong... Thx :-) On 5/2/12 3:57 PM, Brian Waldon brian.wal...@rackspace.com wrote: Jay might have a better answer, but as far as I know, yes. You could probably make the images stores truly pluggable (i.e. not needing to explicitly list them out) without much work. Brian On May 2, 2012, at 12:23 PM, Joshua Harlow wrote: Glance store question?? Hi all, I am making a y! specific backing store for glance and I was wondering if its really necessary to modify the following file to ensure that the code for that new store gets pulled in (or maybe I'm just doing it wrong). diff --git a/glance/api/v1/images.py b/glance/api/v1/images.py index 9839d2e..7c4f565 100644 --- a/glance/api/v1/images.py +++ b/glance/api/v1/images.py @@ -47,6 +47,7 @@ import glance.store.http import glance.store.rbd import glance.store.s3 import glance.store.swift +import glance.store.SUPERAWESOMEYSTORE from glance.store import (get_from_backend, get_size_from_backend, schedule_delete_from_backend, Is there another way to make that happen so that it gets registered with glance besides either modifying that file or having a patch which will modify it upon installation (ie an RPM patch)? Thx! -Josh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Nova idear, thoughts wanted.
Hi all, I was thinking today about how nova-compute could become more pluggable. I was wondering if there had been any thought into how say each method, say in the compute-manager could almost become a set of stages in a pipeline. For example the run instance method is really doing the following steps: run_instance: steps: - check_instance_not_already_created - check_image_size - notify_about_instance_usage - instance_update(BUILDING) - allocate_network - prep_block_device - spawn: - instance_update(BUILD) - driver_spawn - instance_update(ACTIVE) on_failure: - deallocate_network This reminds me slightly of what devstackpy (to be renamed soon) does but instead of via code, actions are partially defined via config/persona/ Now say if the above steps are plugins (similar to say a paste pipeline) then it becomes easy for company Y to add special sauce Z before or after stage W. I was just wondering what people thought about this. It sort of makes nova more of a orchestrator that loads plugins that perform various pipelines, where in nova's case those pipelines are VM related. Comments welcome. This might already have been thought of, but if so, that's ok also :-) -Josh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] schema and counter definitions
Agreed, I would get as much low-level data as possible and let other systems combine that as they want to form whatever billing model they choose. On 4/30/12 6:49 AM, Doug Hellmann doug.hellm...@dreamhost.com wrote: On Mon, Apr 30, 2012 at 6:46 AM, Loic Dachary l...@enovance.com wrote: On 04/30/2012 12:15 PM, Loic Dachary wrote: We could start a discussion from the content of the following sections: http://wiki.openstack.org/EfficientMetering#Counters I think the rationale of the counter aggregation needs to be explained. My understanding is that the metering system will be able to deliver the following information: 10 floating IPv4 addresses were allocated to the tenant during three months and were leased from provider NNN. From this, the billing system could add a line to the invoice : 10 IPv4, $N each = $10xN because it has been configured to invoice each IPv4 leased from provider NNN for $N. It is not the purpose of the metering system to display each IPv4 used, therefore it only exposes the aggregated information. The counters define how the information should be aggregated. If the idea was to expose each resource usage individually, defining counters would be meaningless as they would duplicate the activity log from each OpenStack component. What do you think ? At DreamHost we are going to want to show each individual resource (the IPv4 address, the instance, etc.) along with the charge information. Having the metering system aggregate that data will make it difficult/impossible to present the bill summary and detail views that we want. It would be much more useful for us if it tracked the usage details for each resource, and let us aggregate the data ourselves. If other vendors want to show the data differently, perhaps we should provide separate APIs for retrieving the detailed and aggregate data. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Client branches?
Hi all, I was wondering if the clients (ie novaclient, keystoneclient) are supposed to have branches for stable/essex or not, they currently just have master and milestone proposed? Thx! ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Question on notifications
': 1.0,
u'swap': 512,
u'updated_at': None,
u'vcpu_weight': 10,
u'vcpus': 4},
u'num_instances': 1,
u'security_group': [u'default']},
u'weighted_host': {u'host': u'compute-xx-yy-zz-20',
u'weight': 4945.0}},
u'priority': u'INFO',
u'publisher_id': u'scheduler.nova-sched...ee.com',
u'timestamp': u'2012-04-25 20:32:44.506474'}]
On 04/25/2012 06:44 PM, Joshua Harlow wrote:
Hi all,
I was looking at the notification outputs, which are very useful and I
was wondering if the way to say figure out which hypervisor a VM is
being built on.
There seems to be the following key: publisher_id:
compute.buildingbuild (event is compute.instance.create.end)
It would seem the stuff after compute is the hostname, would that be
correct, or should the scheduler messages be intercepted, which as
example has the following:
weighted_host: {
host: buildingbuild,
weight: -1488.0
}
I would think the first practice would be right, since its from the
compute node instead of the scheduler, but would like some feedback :-)
Thx!
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Canonical AWSOME
So if what u are talking about is anything RPC/MQ based, then I would say those are not internal API's. Once a RPC/MQ mechanism is introduced they don't really become internal API's anymore (if we are talking about the same API's, haha). Since other stuff can be reading those messages on the MQ its very useful to have a schema to know exactly what to read :-) On 4/24/12 11:46 AM, Russell Bryant rbry...@redhat.com wrote: On 04/24/2012 01:25 PM, Joshua Harlow wrote: I'm more in favor of just having a schema, I don't care if that compiles to protocol buffers, json, NEWAWESOMEhipsterMSGFORMAT. That schema will force people to think a little more when they add messages, and it will automatically document the messages that are being sent around. That's a big win I think and is a step to getting those schemas versioned... I'm not sure a schema is really necessary aside from the Python classes themselves. They're internal APIs, so they shouldn't be used from outside of Nova. A schema would be useful if we had to define the interfaces in some language neutral-format, but I don't think that really matters here. I'm going to work on a proposal / prototype for how we can handle versioning, though. The big goal here is making sure that we can maintain compatibility with the previous release. -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] raw or qcow2
Could be a misunderstanding there. My fault. Lets see what happens with volumes. I just hope that the new volume service doesn't require X whizbang feature which will require everyone to reformat with newfilesystem-alpha v.1 (questionably stable), or a similar other untested requirement that isn't really needed if its done in a much simpler manner. That will make it hard for companies that don't want to risk using .1 to actually have this feature, which is a loss for all. On 4/24/12 10:55 PM, Vishvananda Ishaya vishvana...@gmail.com wrote: ? Did you mistype your comment or misread mine? Raw does NOT work for snapshots. snapshots only work for qcow2. Implementing snapshotting with raw would be possible. Logic just needs to be added to skip the internal snapshot step and just use the entire file when uploading to glance. This would be pretty darn slow for large images though. If you are asking about differencing images in glance that is a different question and one that we haven't addressed. It has a lot of implications and needs changes in both nova and glance to be useful. Logic needs to be added around dependency chains and coalescing. Plus it has implications when trying to migrate and resize instances, so there is a lot to consider. As caitlin mentioned, something will be implemented in the volume service anyway, so it might be better to wait and see what happens there. Vish On Apr 24, 2012, at 4:30 PM, Joshua Harlow wrote: Re: [Openstack] raw or qcow2 What changes would be needed to make qcow2 files work as snapshots? Some type of image dependency management in glance (and failure cases) and the corresponding dependency fetching in nova (and failure cases)? Might be something pretty useful to have, instead of forcing raw for snapshots? On 4/24/12 3:51 PM, Vishvananda Ishaya vishvana...@gmail.com x-msg://474/vishvana...@gmail.com wrote: On Apr 17, 2012, at 2:04 AM, William Herry wrote: so, what changes should I make if I want use raw in openstack, I didn't find some configure option in nova.conf.sample I also try to modify the source code in nova/virt/libvirt/utils.py, and didn't succeed I noticed that the type of snapshot is same as the instance's image by default, does this right, and what about the type of model image that uploaded to glance, does it affect the disk type I use? Thanks snapshots will not work with raw images. To make openstack use raw images, you simply have to set: use_cow_images=false you can upload to glance in qcow or raw, it will be decoded to raw when the image is downloaded to the compute host. Vish ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net x-msg://474/openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Question on notifications
Hi all,
I was looking at the notification outputs, which are very useful and I was
wondering if the way to say figure out which hypervisor a VM is being built on.
There seems to be the following key: publisher_id: compute.buildingbuild
(event is compute.instance.create.end)
It would seem the stuff after compute is the hostname, would that be correct,
or should the scheduler messages be intercepted, which as example has the
following:
weighted_host: {
host: buildingbuild,
weight: -1488.0
}
I would think the first practice would be right, since its from the compute
node instead of the scheduler, but would like some feedback :-)
Thx!
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] db notification support for API extension?
Has there been any investigation into using already existing plugin frameworks and just use those (and/or make those better)? Just from a quick google search: http://wehart.blogspot.com/2009/01/python-plugin-frameworks.html It might be useful to see if we can find one that is generic and well-supported and already works good enough?? On 4/25/12 3:04 PM, Andrew Bogott abog...@wikimedia.org wrote: On 4/25/12 4:48 PM, Nathanael Burton wrote: On Thu, Mar 8, 2012 at 11:53 AM, Andrew Bogottabog...@wikimedia.org wrote: I'm working on an API and implementation to support the creation of filesystems that are shared among Nova instances. http://wiki.openstack.org/SharedFS My hope is to keep this API isolated from core Nova code, partly to avoid stepping on toes and partly because I hope to be able to drop it into an existing essex install. There are two things I need which I know how to do within Nova but am not clear on how to do without modding Nova code: 1) DB support I need a database table to keep track of some filesystem metadata. My current implementation adds the table via nova/db/sqlalchemy/migrate_repo... but is it really necessary to coordinate this table with the rest of Nova? Would it be reasonable to maintain the table independently within the extension code? And, if so, are there any existing extensions that do something like this? Have you determined a cleaner way of doing this? I have the same issues as you when writing API extensions. Nate -- The short answer is: I'm sure that it's straightforward to create a 'private' table which doesn't collide with existing nova tables, but I have yet to do so. The longer answer is: Everything about that thread is now rolled into the topic of 'the plugin framework' which we discussed at the design summit and which I'm currently devoted to. Please consider adding your use cases to the wiki page at http://wiki.openstack.org/novaplugin, and let me know if you would like me to add you to the list of people I cc: when looking for opinions and/or reporting progress. -Andrew ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Canonical AWSOME
I'm more in favor of just having a schema, I don't care if that compiles to protocol buffers, json, NEWAWESOMEhipsterMSGFORMAT. That schema will force people to think a little more when they add messages, and it will automatically document the messages that are being sent around. That's a big win I think and is a step to getting those schemas versioned... Just don't do pickling, pleasee (for other language users). On 4/24/12 8:39 AM, Eric Windisch e...@cloudscaling.com wrote: Actually, I think JSON schema for our message-bus messages might be a really good idea (tm). Violations could just be warnings until we get things locked down... maybe I should propose a blueprint? (Although I have enough of a blueprint backlog as it is...) This was discussed at the summit in (I believe) the API versioning talk. There is a strong bias toward JSON inside RPC messages. However, this is actually transparent to Nova as the RPC implementations do the decoding and encoding. It is also hard to test and trigger such warnings as, as far as Nova knows, all the interfaces pass python data types, not JSON. Some RPC mechanisms could transparently serialize. As long as there is an abstraction layer, it should be oblivious to the serialization and we should not care too strongly. There was a patch a few weeks ago that has enforced using a common RPC exception serializer using JSON, which I'm not sure is, or is not, a good thing. I haven't yet updated the ZeroMQ driver to use this, but am in the process of making these changes as I update for Folsom. All that said, I do intend that the ZeroMQ driver will use JSON when it lands in Folsom. (it currently Pickles, but only because there was a bug in Essex at one point, breaking JSON serialization) ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] raw or qcow2
What changes would be needed to make qcow2 files work as snapshots? Some type of image dependency management in glance (and failure cases) and the corresponding dependency fetching in nova (and failure cases)? Might be something pretty useful to have, instead of forcing raw for snapshots? On 4/24/12 3:51 PM, Vishvananda Ishaya vishvana...@gmail.com wrote: On Apr 17, 2012, at 2:04 AM, William Herry wrote: so, what changes should I make if I want use raw in openstack, I didn't find some configure option in nova.conf.sample I also try to modify the source code in nova/virt/libvirt/utils.py, and didn't succeed I noticed that the type of snapshot is same as the instance's image by default, does this right, and what about the type of model image that uploaded to glance, does it affect the disk type I use? Thanks snapshots will not work with raw images. To make openstack use raw images, you simply have to set: use_cow_images=false you can upload to glance in qcow or raw, it will be decoded to raw when the image is downloaded to the compute host. Vish ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] raw or qcow2
Ok, but forcing a new block storage project to have this feature, is that the right way? Maybe it will, just wondering. Glance seems to be the registry of images, so a natural progression is that it can maintain the image dependencies? This would allow the qcow2 image to pull the needed dependencies down to nova. On 4/24/12 4:44 PM, Caitlin Bestler caitlin.best...@nexenta.com wrote: Joshua Harlow wrote: What changes would be needed to make qcow2 files work as snapshots? Some type of image dependency management in glance (and failure cases) and the corresponding dependency fetching in nova (and failure cases)? Might be something pretty useful to have, instead of forcing raw for snapshots? I think this is something that should be addressed in the new block storage project, and only incidentally in glance. Basically, glance can record the intent of X being based on snapshot Y, but building a volume X that is based on some copy of snapshot Y is best left to the block layer. Any solution expressed at the Glance layer is going to be biased towards doing the cloning too early. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
