Re: [Openstack] Using Nova APIs from Javascript: possible?
Hi all, I'm glad to hear that there's a lot of interest in the implementation of Openstack JavaScript clients. Actually, in my group we're developing a single page application developed entirely in JavaScript, that widely supports Nova and Keystone APIs. This work is part of a European Project called FI-Ware (http://www.fi-ware.eu/), in which we are currently using Openstack APIs. We've modified Nova and Keystone installations by adding CORS support. We did it by implementing a kind of filter on their APIs. For doing this we used Adam's implementation (https://github.com/adrian/swift/tree/cors), and we adapted it to Nova and Keystone components. We also developed a JS library (http://ging.github.com/jstack/) that can be used by both web and Node.js applications, for example. This library aims to provide same functionalities as python-novaclient, adding support for Keystone API. And finally we are copying Openstack horizon functionality, using JS library and other frameworks such as jQuery and Backbone.js to implement the web application. This web application is an early-stage work, but we will probably publish it by the end of this week. I will let you know the github link. We didn't find much problems with CORS implementation and support in browsers. For the time being, according to our experiments, the only web browser that is not usable at all with this technology is Internet Explorer, but we have tried it in Google Chrome, Safari and Firefox as well and we didn't have any problems. Cheers, Javier Cerviño. On 26 April 2012 06:28, Nick Lothian nick.loth...@gmail.com wrote: On Thu, Apr 26, 2012 at 5:49 AM, Adam Young ayo...@redhat.com wrote: Let me try to summarize: 1. If you are running from a web browser, post requests to hosts or ports other than the origin are allowed, but the headers cannot be modified. This prevents the addition of the token from Keystone to provide single sign on. 2. There are various browser side technologies (JSONP, CORS) that get around this limitation, but they are typically not enabled, and can be considered security issues. While implementing these might require support from teh Openstack server, they are fundamentally browser decisions. This is inaccurate. JSONP is supported by all browsers since ~Netscape 4.0. CORS is supported by all modern browsers: IE 8, Firefox 3.5, Chrome 3, Safari 4 (See http://en.wikipedia.org/wiki/Cross-origin_resource_sharing#Browser_support). Additionally, CORS support is not a browser decision - the server has to EXPLICITLY opt-in to support it. Obviously CORS support *can* be a security issue - that is why it is disabled unless the server enables it. I do not believe that CORS support adds any additional security issues above what the OpenStack APIs already face. Specially, the most common problem (CSRF) is not an issue here because the APIs are not authorised on a session basis. [snip] I've been working on Single Sign on Issues for another project for the past year and a half. Here's a couple things I've learned. Kerberos is designed to solve this problem. It has the benefit of being integrated into the browser. Where Kerberos fails is that: typically it only allows a single authentication provider (KDC in Kerberso speak) and it does not work well with Firewalls. The only crytographically secure way to authenticate on the web that can get around the firewall issue is Client side X509 certificates. This is the foundation for https://blueprints.launchpad.net/keystone/+spec/pki. This could, in theory, work in with OAuth, OpenID, or some other distributed authorization service, or we could embed the authorization information right into the Certitificate, which is what I suggest we do. To be clear, identity/authorisation is NOT the problem here. The OpenStack APIs work well for my use cases, once I work around the cross domain POST problem. However, I've also worked with SSO solutions. The simple truth is that client side certificates do not play well with the web - browser support ranges from non-existent (on some mobile platforms - see http://mobilitydojo.net/2010/12/28/client-certificate-support-across-mobile-platforms-a-summary/) to abysmal (there is a reason why many websites that use certificates end up using a Java applet), and their interaction with cross domain Javascript is unknown. Even if certificates did work for identification, CORS would still be needed - many OpenStack APIs require a POST request which is impossible without it. Nick ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe :
Re: [Openstack] Nova and external NFS
Hi Jorge: These are the permissions when instances are created locally in the Nova server (not using NFS). User nova is the owner of instance directory. The instance is created using the web interface (Horizon dashboard). In the NFS server there is no user named nova. Any file or folder created from any user not root, is changed to nobody:nogroup (root user is keeped). So, when nova user creates a file, appears nobody:nogroup. # ll /var/lib/nova/instances/instance-000b total 150124 drwxrwxr-x 2 nova nova 4096 abr 23 15:17 ./ drwxr-xr-x 4 nova nova 4096 abr 23 15:17 ../ -rw-rw 1 root root 0 abr 24 17:57 console.log -rw-r--r-- 1 root root 140509184 abr 25 18:20 disk -rw-r--r-- 1 root root 16777216 abr 24 17:56 disk.local -rw-rw-r-- 1 root root 4790624 abr 23 15:17 kernel -rw-rw-r-- 1 nova nova 1812 abr 23 15:17 libvirt.xml Thanks... Regards... Sergio Ariel de la Campa Saiz GMV-SES Infraestructura / GMV-SES Infrastructure GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com De: Jorge de la Cruz [jorge.delac...@stackops.com] Enviado el: miércoles, 25 de abril de 2012 19:06 Para: Sergio Ariel de la Campa Saiz CC: openstack@lists.launchpad.net Asunto: Re: [Openstack] Nova and external NFS Hi Sergio, Dont worry about the questions, this a list for help. I see that nobody:nogroup is the owner of the top folder, i think this is wrong, change with a chown -R to root:root for the top folder, be sure of all the files have root:root, and try again. drwxrwxr-x 2 nobody nogroup 4096 abr 25 2012 ./ drwxrwxrwx 4 root root4096 abr 25 2012 ../ -rw-rw 1 root root 0 abr 25 2012 console.log -rw-r--r-- 1 root root25165824 abr 25 2012 disk -rw-r--r-- 1 root root 6291968 abr 25 2012 disk.local -rw-rw-r-- 1 root root 4790624 abr 25 2012 kernel -rw-rw-r-- 1 nobody nogroup 1856 abr 25 2012 libvirt.xml Regards De: Sergio Ariel de la Campa Saiz saca...@gmv.com Para: Jorge de la Cruz jorge.delac...@stackops.com CC: openstack@lists.launchpad.net Enviados: Miércoles, 25 de Abril 2012 18:50:47 Asunto: RE: [Openstack] Nova and external NFS Hi: Thanks for your respond. This is my /etc/export file in the NFS server: /export 192.168.111.0/24(rw,sync,no_root_squash,fsid=0) and my /etc/fstab file in my host is: ip nfs server://nfs-directory nfs4 defaults00 Directories /export and /nfs-directory have 777 permissions Sorry if I bother you... but It is driving me crazy Sergio Ariel de la Campa Saiz GMV-SES Infraestructura / GMV-SES Infrastructure GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com De: Jorge de la Cruz [jorge.delac...@stackops.com] Enviado el: miércoles, 25 de abril de 2012 17:52 Para: Sergio Ariel de la Campa Saiz CC: openstack@lists.launchpad.net Asunto: Re: [Openstack] Nova and external NFS Hi Sergio, We have environment with external NFS Server, NetAPP, Nexenta, EMC, etc and we haven´t this problem, sounds like a problem with privileges, i can see the libvirt.xml is nobody:nogroup, it is wrong, must be root:root. Try to change manually, but maybe you have a wrong parameter in a configuration that generate this file with this permissions. Cheers PS: Nosotros también estamos en Madrid, podemos conocernos un día si os apetece. -- Jorge de la Cruz http://www.stackops.com/Cloud Architect www.stackops.comhttp://www.stackops.com/ | jorge.delac...@stackops.comhttps://mail.gmv.com/owa/UrlBlockedError.aspx | +34 91 309 48 77 | skype:jorge.delacruz.stackops http://www.stackops.com/ [http://stackops.s3-external-3.amazonaws.com/STACKOPSLOGO-ICON.png] ADVERTENCIA LEGAL Le informamos, como destinatario de este mensaje, que el correo electrónico y las comunicaciones por medio de Internet no permiten asegurar ni garantizar la confidencialidad de los mensajes transmitidos, así como tampoco su integridad o su correcta recepción, por lo que STACKOPS TECHNOLOGIES S.L. no asume responsabilidad alguna por tales circunstancias. Si no consintiese en la utilización del correo electrónico o de las comunicaciones vía Internet le rogamos nos lo comunique y ponga en nuestro conocimiento de manera inmediata. Este mensaje va dirigido, de manera exclusiva, a su destinatario y contiene información confidencial y sujeta al secreto profesional, cuya divulgación no está permitida por la ley. En caso de haber recibido este mensaje por error, le rogamos que, de forma inmediata, nos lo comunique mediante correo electrónico remitido a nuestra atención y proceda a su eliminación, así como a la de cualquier documento adjunto al mismo. Asimismo, le comunicamos que la distribución, copia o utilización de este mensaje, o de
Re: [Openstack] How does everyone build OpenStack disk images?
On Wed, Apr 25, 2012 at 06:14:22PM -0700, Justin Santa Barbara wrote: How does everyone build OpenStack disk images? The official documentation describes a manual process (boot VM with ISO), which is sub-optimal in terms of repeatability / automation / etc. I'm hoping we can do better! I posted how I do it on my blog, here: http://blog.justinsb.com/blog/2012/04/25/creating-an-openstack-image/ Please let me know the many ways in which I'm doing it wrong :-) I'm thinking we can have a discussion here, and then I can then compile the responses into a wiki page and/or a nice script... If you have a KVM enabled machine, then 'Oz' has the ability to create JeOS images for all the common distros you'll find. It is a very simple command line tool that just focuses on image building image customization (adding more packages to an existing JeOS image). http://aeolusproject.org/oz.html Yes, it is on the Aeolus project website, but it has no external dependancies on the rest of Aeolus - it just wants kvm, libvirt a few commonly available python modules. I've often thought that it would be desirable to have Oz integrated into OpenStack to provide an native image building capability. Given their common Python heritage I think it would work quite well. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Pending reviews
On 26/04/12 02:28, Mandar Vaze / मंदार वझे wrote: It would be great if someone can spare some time to have a look at these : https://review.openstack.org/#/c/6451/ : I've addressed comments from first review cycle - Second patch set needs to be reviewed and approved https://review.openstack.org/#/c/6452/ : Brad Hall reviewed - But more review and/or approval needed. https://review.openstack.org/#/c/6076/ : Chuck Short and Kevin Mitchell said Looks good - But approval still needed. I'm in the same state, with many reviews which need eyeballs. Mikal ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] questions about IP addressing and network config
Hi everyone, I'm running with Essex 2012.1, and have some questions about the nova network operation, 1. Is it possible manually assigned IP address to a launched instance, my situation is : after instance boot up (OS: CentOS 6.2), I changed the /etc/sysconfig/network-scripts/ifcfg-eth0 setting from dhcp to static (the same subnet as created by command : nova-manage create network), and restart the network service, And then I couldn't ssh or ping the instance from other server with the same subnet. What is the problem ? I checked the iptables policies on the compute host, and find nothing about the DROP packets. I also tried to change the DB record from nova.fixed_ips table and libvirt.xml of the instance directory, then reboot the instance, I can see the instance get new IP from DHCP, but still not worked(can't ping ssh). I used FlatDHCP as my network manager. 2. According to the first question, I have another requirement to set up a loopback IP address (lo:0) on the running instance, after setting completed,I couldn't ping or ssh the loopback IP from the same subnet, and I tried to set a alias IP address with eth0:0, but still not get worked. Any ideas with this ? 3. Is there any way to use 2 NICs with different subnets on instances? I want to separate the network traffic. Now I'm running with one bridged interface (br100), and it works well. In order to backup the large log files, I'm planing to use 2 NICs for the compute hosts, I want use 2 vNICs on instance, one for web service and the other for log backup, I think I should create a new network for the second bridged interface, but I can't find any document to guild me. List my nova.conf below == --dhcpbridge_flagfile=/etc/nova/nova.conf --dhcpbridge=/usr/bin/nova-dhcpbridge --logdir=/var/log/nova --state_path=/var/lib/nova --lock_path=/var/lock/nova --allow_admin_api=true --use_deprecated_auth=false --auth_strategy=keystone --scheduler_driver=nova.scheduler.simple.SimpleScheduler --s3_host=172.19.7.1 --ec2_host=172.19.7.1 --rabbit_host=172.19.7.1 --cc_host=172.19.7.1 --nova_url=http://172.19.7.1:8774/v1.1/ #--routing_source_ip=172.19.7.1 --glance_api_servers=172.19.7.1:9292 --image_service=nova.image.glance.GlanceImageService #--iscsi_ip_prefix=192.168.22 --sql_connection=mysql://nova:nova@172.19.7.1/nova --ec2_url=http://172.19.7.1:8773/services/Cloud --keystone_ec2_url=http://172.19.7.1:5000/v2.0/ec2tokens --api_paste_config=/etc/nova/api-paste.ini --libvirt_type=kvm --libvirt_use_virtio_for_bridges=true --start_guests_on_host_boot=true --resume_guests_state_on_host_boot=true #--vnc_enabled=true --novnc_enabled=true #--vncproxy_url=http://172.19.7.1:6080/vnc_auto.html #--vnc_console_proxy_url=http://172.19.7.1:6080 --novncproxy_base_url=http://172.19.7.1:6080/vnc_auto.html --xvpvncproxy_base_url=http://172.19.7.1:6081/console #--vncserver_listen=172.19.7.1 #--vncserver_proxyclient_address=172.19.7.1 --vncserver_listen=0.0.0.0 --vncserver_proxyclient_address=0.0.0.0 # network specific settings --network_manager=nova.network.manager.FlatDHCPManager #--network_manager=nova.network.quantum.manager.QuantumManager #--quantum_connection_host=172.19.7.1 #--quantum_connection_port=9696 --public_interface=eth0 --flat_interface=eth0 --flat_network_bridge=br100 --fixed_range=172.19.7.0/24 --network_size=254 --flat_network_dhcp_start=172.19.7.41 --multi_host --flat_injected=False --force_dhcp_release --iscsi_helper=tgtadm --connection_type=libvirt --root_helper=sudo nova-rootwrap #--verbose #--verbose=true --verbose=false --libvirt_xml_template=/usr/share/pyshared/nova/virt/libvirt.xml.template --allow_resize_to_same_host=true --max_cores=30 = if I misunderstand something, please correct me, thanks. -Jimmy ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Quantum Integration Part 3
Hi Dan, Le mercredi 25 avril 2012 à 10:54 -0700, Dan Wendlandt a écrit : Are you able to access the VM via VNC? If you statically configure the IP does it work? What happens if you run dhcpc manually? With a Debian image, I connect to the VM with VNC from dashboard, configure /etc/network/interfaces manually, ifup eth0, but I can't ping anything. Maybe the problem come from my OVS configuration ? I've created a bridge br-int, and attached eth1 on each server to this bridge (is it an error from me ?). Here you can find my nova-network.log : http://paste.openstack.org/show/13951/ My ovs-vswitchd.log of my nova-compute : http://paste.openstack.org/show/13952/ My nova-compute.log : http://paste.openstack.org/show/13954/ My Quantum-server.log : http://paste.openstack.org/show/13961/ And finally, the log file of the instance : http://paste.openstack.org/show/13960/ The errors you can read is in ovs-vwitchd.log with eth1, and on the quantum-server.log, it's talking about no route for the network. I continue to investigate today, and please contact me if anyone has an idea. Best regards I would look in the nova-network logs and see if you see an errors related to DHCP. - Horizon with Quantum : I can't integrate Quantum in the dashboard even with http://docs.openstack.org/trunk/openstack-compute/admin/content/build-and-configure-openstack-dashboard.html Essex Horizon does not support Quantum. Its actually not just a Horizon thing, it has to do with how the Nova + Quantum integration works. Each Quantum network has to be associated with appropriate IPAM subnet, meaning the existing . For now, you need to create networks using nova-manage. If you want VMs to get the default set of NICs, you can then boot them with Horizon. If you want to specify what networks vNICs are plugged into, this is only supported via the nova boot command with the --nic option. I'll flush this out in more detail in the Quantum Admin Guide. Dan -- ~~~ Dan Wendlandt Nicira, Inc: www.nicira.com twitter: danwendlandt ~~~ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Using Foreign Keys
On 25/04/12 19:02, Doug Hellmann wrote: From a MySQL prospective that is probably more of an argument to use transactions, not foreign keys. Transactions and referential integrity are related, but not equivalent. Without referential integrity it's quite easy to commit a transaction that leaves the database in a logically inconsistent state (it sounds like that's what was happening in the case described by the OP). From the way I read it the example given wasn't a referential integrity check but a delete across multiple tables. Is there a technical reason to disable strict referential integrity checking with MySQL? Technically it can make upgrades/downgrades harder, no engines other than InnoDB don't support them whereas many engines support transactions, MySQL doesn't actually support them (they are passed down to the InnoDB engine even at the parser layer). There are several other reasons (bugs and performance) why I don't like the MySQL implementation I won't go into here. Kind Regards -- Andrew Hutchings - LinuxJedi - http://www.linuxjedi.co.uk/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nova and external NFS [SOLVED]
Hi: First of all thanks to Jorge :-) I have solved my problem. I have a NFS server external to my Nova servers, so in NFS server there were not defined the same users that are defined in Nova servers. When I created nova group, nova user, libvirt-qemu user and kvm group with same UID and GID of ones defined in Nova servers, my problem disappeared. drwxrwxr-x 2 nova nova 4096 abr 26 2012 ./ drwxrwxrwx 4 root root 4096 abr 26 2012 ../ -rw-rw 1 libvirt-qemu kvm 20454 abr 26 2012 console.log -rw-r--r-- 1 libvirt-qemu kvm 60817408 abr 26 2012 disk -rw-r--r-- 1 libvirt-qemu kvm 12582912 abr 26 2012 disk.local -rw-rw-r-- 1 libvirt-qemu kvm 4790624 abr 26 2012 kernel -rw-rw-r-- 1 nova nova 1820 abr 26 2012 libvirt.xml Sergio Ariel de la Campa Saiz GMV-SES Infraestructura / GMV-SES Infrastructure GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com De: Jorge de la Cruz [jorge.delac...@stackops.com] Enviado el: miércoles, 25 de abril de 2012 19:06 Para: Sergio Ariel de la Campa Saiz CC: openstack@lists.launchpad.net Asunto: Re: [Openstack] Nova and external NFS Hi Sergio, Dont worry about the questions, this a list for help. I see that nobody:nogroup is the owner of the top folder, i think this is wrong, change with a chown -R to root:root for the top folder, be sure of all the files have root:root, and try again. drwxrwxr-x 2 nobody nogroup 4096 abr 25 2012 ./ drwxrwxrwx 4 root root4096 abr 25 2012 ../ -rw-rw 1 root root 0 abr 25 2012 console.log -rw-r--r-- 1 root root25165824 abr 25 2012 disk -rw-r--r-- 1 root root 6291968 abr 25 2012 disk.local -rw-rw-r-- 1 root root 4790624 abr 25 2012 kernel -rw-rw-r-- 1 nobody nogroup 1856 abr 25 2012 libvirt.xml Regards De: Sergio Ariel de la Campa Saiz saca...@gmv.com Para: Jorge de la Cruz jorge.delac...@stackops.com CC: openstack@lists.launchpad.net Enviados: Miércoles, 25 de Abril 2012 18:50:47 Asunto: RE: [Openstack] Nova and external NFS Hi: Thanks for your respond. This is my /etc/export file in the NFS server: /export 192.168.111.0/24(rw,sync,no_root_squash,fsid=0) and my /etc/fstab file in my host is: ip nfs server://nfs-directory nfs4 defaults00 Directories /export and /nfs-directory have 777 permissions Sorry if I bother you... but It is driving me crazy Sergio Ariel de la Campa Saiz GMV-SES Infraestructura / GMV-SES Infrastructure GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com De: Jorge de la Cruz [jorge.delac...@stackops.com] Enviado el: miércoles, 25 de abril de 2012 17:52 Para: Sergio Ariel de la Campa Saiz CC: openstack@lists.launchpad.net Asunto: Re: [Openstack] Nova and external NFS Hi Sergio, We have environment with external NFS Server, NetAPP, Nexenta, EMC, etc and we haven´t this problem, sounds like a problem with privileges, i can see the libvirt.xml is nobody:nogroup, it is wrong, must be root:root. Try to change manually, but maybe you have a wrong parameter in a configuration that generate this file with this permissions. Cheers PS: Nosotros también estamos en Madrid, podemos conocernos un día si os apetece. -- Jorge de la Cruz http://www.stackops.com/Cloud Architect www.stackops.comhttp://www.stackops.com/ | jorge.delac...@stackops.comhttps://mail.gmv.com/owa/UrlBlockedError.aspx | +34 91 309 48 77 | skype:jorge.delacruz.stackops http://www.stackops.com/ ADVERTENCIA LEGAL Le informamos, como destinatario de este mensaje, que el correo electrónico y las comunicaciones por medio de Internet no permiten asegurar ni garantizar la confidencialidad de los mensajes transmitidos, así como tampoco su integridad o su correcta recepción, por lo que STACKOPS TECHNOLOGIES S.L. no asume responsabilidad alguna por tales circunstancias. Si no consintiese en la utilización del correo electrónico o de las comunicaciones vía Internet le rogamos nos lo comunique y ponga en nuestro conocimiento de manera inmediata. Este mensaje va dirigido, de manera exclusiva, a su destinatario y contiene información confidencial y sujeta al secreto profesional, cuya divulgación no está permitida por la ley. En caso de haber recibido este mensaje por error, le rogamos que, de forma inmediata, nos lo comunique mediante correo electrónico remitido a nuestra atención y proceda a su eliminación, así como a la de cualquier documento adjunto al mismo. Asimismo, le comunicamos que la distribución, copia o utilización de este mensaje, o de cualquier documento adjunto al mismo, cualquiera que fuera su finalidad, están prohibidas por la ley. * PRIVILEGED AND CONFIDENTIAL We hereby inform
Re: [Openstack] Using Nova APIs from Javascript: possible?
Excellent, thanks for that information Javier. It's good to know I'm not the only person doing this. On Apr 26, 2012 5:20 PM, javier cerviño jcerv...@dit.upm.es wrote: Hi all, I'm glad to hear that there's a lot of interest in the implementation of Openstack JavaScript clients. Actually, in my group we're developing a single page application developed entirely in JavaScript, that widely supports Nova and Keystone APIs. This work is part of a European Project called FI-Ware (http://www.fi-ware.eu/), in which we are currently using Openstack APIs. We've modified Nova and Keystone installations by adding CORS support. We did it by implementing a kind of filter on their APIs. For doing this we used Adam's implementation (https://github.com/adrian/swift/tree/cors), and we adapted it to Nova and Keystone components. We also developed a JS library (http://ging.github.com/jstack/) that can be used by both web and Node.js applications, for example. This library aims to provide same functionalities as python-novaclient, adding support for Keystone API. And finally we are copying Openstack horizon functionality, using JS library and other frameworks such as jQuery and Backbone.js to implement the web application. This web application is an early-stage work, but we will probably publish it by the end of this week. I will let you know the github link. We didn't find much problems with CORS implementation and support in browsers. For the time being, according to our experiments, the only web browser that is not usable at all with this technology is Internet Explorer, but we have tried it in Google Chrome, Safari and Firefox as well and we didn't have any problems. Cheers, Javier Cerviño. On 26 April 2012 06:28, Nick Lothian nick.loth...@gmail.com wrote: On Thu, Apr 26, 2012 at 5:49 AM, Adam Young ayo...@redhat.com wrote: Let me try to summarize: 1. If you are running from a web browser, post requests to hosts or ports other than the origin are allowed, but the headers cannot be modified. This prevents the addition of the token from Keystone to provide single sign on. 2. There are various browser side technologies (JSONP, CORS) that get around this limitation, but they are typically not enabled, and can be considered security issues. While implementing these might require support from teh Openstack server, they are fundamentally browser decisions. This is inaccurate. JSONP is supported by all browsers since ~Netscape 4.0. CORS is supported by all modern browsers: IE 8, Firefox 3.5, Chrome 3, Safari 4 (See http://en.wikipedia.org/wiki/Cross-origin_resource_sharing#Browser_support ). Additionally, CORS support is not a browser decision - the server has to EXPLICITLY opt-in to support it. Obviously CORS support *can* be a security issue - that is why it is disabled unless the server enables it. I do not believe that CORS support adds any additional security issues above what the OpenStack APIs already face. Specially, the most common problem (CSRF) is not an issue here because the APIs are not authorised on a session basis. [snip] I've been working on Single Sign on Issues for another project for the past year and a half. Here's a couple things I've learned. Kerberos is designed to solve this problem. It has the benefit of being integrated into the browser. Where Kerberos fails is that: typically it only allows a single authentication provider (KDC in Kerberso speak) and it does not work well with Firewalls. The only crytographically secure way to authenticate on the web that can get around the firewall issue is Client side X509 certificates. This is the foundation for https://blueprints.launchpad.net/keystone/+spec/pki. This could, in theory, work in with OAuth, OpenID, or some other distributed authorization service, or we could embed the authorization information right into the Certitificate, which is what I suggest we do. To be clear, identity/authorisation is NOT the problem here. The OpenStack APIs work well for my use cases, once I work around the cross domain POST problem. However, I've also worked with SSO solutions. The simple truth is that client side certificates do not play well with the web - browser support ranges from non-existent (on some mobile platforms - see http://mobilitydojo.net/2010/12/28/client-certificate-support-across-mobile-platforms-a-summary/ ) to abysmal (there is a reason why many websites that use certificates end up using a Java applet), and their interaction with cross domain Javascript is unknown. Even if certificates did work for identification, CORS would still be needed - many OpenStack APIs require a POST request which is impossible without it. Nick ___ Mailing list: https://launchpad.net/~openstack Post to :
Re: [Openstack] Quantum Integration Part 3
Hi again, After inverstigation, it finally works. I have the network on my instances in using Quantum. The fact was I've follow official OVS documentation, and you should not follow this direction : ovs-vsctl add-port br-int eth1 If you see on the logs file of OVS, you can see that the bridge will not work properly... that's why my instances did not get the network. I've tried ovs-vsctl del-port br-int eth1, restart all the services and now, all is working. I have now a dual-node with : Nova-*, Glance, Quantum (without authentification), Keystone, and Horizon (without Quantum UI yet). Next steps : QoS, UI, Isolation testing, and documentation :-) I will continue to post about Quantum when I will advance in the project. Best regards Le mercredi 25 avril 2012 à 10:54 -0700, Dan Wendlandt a écrit : Hi Emilien, On Wed, Apr 25, 2012 at 9:13 AM, Emilien Macchi emilien.openst...@gmail.com wrote: For example, Quantum does not work with Keystone : When nova-network send an API call to Quantum-server, the token is missing in the HTTP HEADER. I've disabled Keystone at this time. That is correct. In the Quantum Admin Guide, it explains some work that was done to support Keystone Authn, but notes that this is all experimental, as the Quantum API does not support Authz. This is because the only person that calls Quantum in Essex is Nova itself. Troy Toman's team is working on Quantum Authz for Folsom, so the API can be exposed directly to tenants. - What is working at this time : - Quantum-Server with MySQL - Quantum-Agent (with OVS) on each nova-compute node - OVS bridge with eth1 - nova-network seems working with Quantum - What is not working (yet !) : - When I create an instance (from dashboard or with Nova CLI), the network is created with Quantum, an private IP is attributed to the instance, but when I check the instance log file, the network interface doesn't get an IP adress. Here the log : http://paste.openstack.org/show/13821 Are you able to access the VM via VNC? If you statically configure the IP does it work? What happens if you run dhcpc manually? I would look in the nova-network logs and see if you see an errors related to DHCP. - Horizon with Quantum : I can't integrate Quantum in the dashboard even with http://docs.openstack.org/trunk/openstack-compute/admin/content/build-and-configure-openstack-dashboard.html Essex Horizon does not support Quantum. Its actually not just a Horizon thing, it has to do with how the Nova + Quantum integration works. Each Quantum network has to be associated with appropriate IPAM subnet, meaning the existing . For now, you need to create networks using nova-manage. If you want VMs to get the default set of NICs, you can then boot them with Horizon. If you want to specify what networks vNICs are plugged into, this is only supported via the nova boot command with the --nic option. I'll flush this out in more detail in the Quantum Admin Guide. Dan -- ~~~ Dan Wendlandt Nicira, Inc: www.nicira.com twitter: danwendlandt ~~~ -- Emilien Macchi Phone : +33 685 117 748 Skype : memilien69 Twitter : EmilienMacchi Website : http://my1.fr ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] How does everyone build OpenStack disk images?
Broader question: what's the best place to capture this information? Right now, there's a chapter in the OpenStack Compute Admin Guide about images, but I'm tempted to break out a new document dedicated to creating and obtaining appropriate images that are compatible with OpenStack (or, more specifically, with OpenStack-supported hypervisors), and have that be accessible on docs.openstack.org. We could use the wiki, but I'm afraid most users wouldn't find it there, and this is a pretty critical issue for new users. Unless Take care, Lorin -- Lorin Hochstein Lead Architect - Cloud Services Nimbis Services, Inc. www.nimbisservices.com On Apr 26, 2012, at 7:04 AM, Michael Basnight wrote: In reddwarf for development we use Ubuntu-vm-builder. It works like a charm for creating qcow2 images. Sent from my digital shackles. On Apr 25, 2012, at 8:14 PM, Justin Santa Barbara jus...@fathomdb.com wrote: How does everyone build OpenStack disk images? The official documentation describes a manual process (boot VM with ISO), which is sub-optimal in terms of repeatability / automation / etc. I'm hoping we can do better! I posted how I do it on my blog, here: http://blog.justinsb.com/blog/2012/04/25/creating-an-openstack-image/ Please let me know the many ways in which I'm doing it wrong :-) I'm thinking we can have a discussion here, and then I can then compile the responses into a wiki page and/or a nice script... Justin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp smime.p7s Description: S/MIME cryptographic signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] questions about IP addressing and network config
Hi everyone, I'm running with Essex 2012.1, and have some questions about the nova network operation, 1. Is it possible manually assigned IP address to a launched instance, my situation is : after instance boot up (OS: CentOS 6.2), I changed the /etc/sysconfig/network-scripts/ifcfg-eth0 setting from dhcp to static (the same subnet as created by command : nova-manage create network), and restart the network service, And then I couldn't ssh or ping the instance from other server with the same subnet. What is the problem ? I checked the iptables policies on the compute host, and find nothing about the DROP packets. I also tried to changed the record from nova.fixed_ips table and libvirt.xml of the instance, then reboot the instance, still not worked. I used FlatDHCP as my network manager. 2. According to the first question, I have another requirement to set up a loopback IP address (lo:0) on the running instance, after setting was completed,I couldn't ping or ssh the loopback IP from the same subnet, and I tried to set a alias IP address with eth0:0, but still not get worked. Any ideas with this ? 3. Is there any way to use 2 NICs with different subnets on instances? I want to separate the network traffic. Now I'm running with one bridged interface (br100), and it works well. In order to backup the large log files, I'm planing to use 2 NICs for the compute hosts, I want use 2 vNICs on instance, one for web service and the other for log backup, I think I should create a new network for the second bridged interface, but I can't find any document to guild me. List my nova.conf below == --dhcpbridge_flagfile=/etc/nova/nova.conf --dhcpbridge=/usr/bin/nova-dhcpbridge --logdir=/var/log/nova --state_path=/var/lib/nova --lock_path=/var/lock/nova --allow_admin_api=true --use_deprecated_auth=false --auth_strategy=keystone --scheduler_driver=nova.scheduler.simple.SimpleScheduler --s3_host=172.19.7.1 --ec2_host=172.19.7.1 --rabbit_host=172.19.7.1 --cc_host=172.19.7.1 --nova_url=http://172.19.7.1:8774/v1.1/ #--routing_source_ip=172.19.7.1 --glance_api_servers=172.19.7.1:9292 --image_service=nova.image.glance.GlanceImageService #--iscsi_ip_prefix=192.168.22 --sql_connection=mysql://nova:nova@172.19.7.1/nova --ec2_url=http://172.19.7.1:8773/services/Cloud --keystone_ec2_url=http://172.19.7.1:5000/v2.0/ec2tokens --api_paste_config=/etc/nova/api-paste.ini --libvirt_type=kvm --libvirt_use_virtio_for_bridges=true --start_guests_on_host_boot=true --resume_guests_state_on_host_boot=true #--vnc_enabled=true --novnc_enabled=true #--vncproxy_url=http://172.19.7.1:6080/vnc_auto.html #--vnc_console_proxy_url=http://172.19.7.1:6080 --novncproxy_base_url=http://172.19.7.1:6080/vnc_auto.html --xvpvncproxy_base_url=http://172.19.7.1:6081/console #--vncserver_listen=172.19.7.1 #--vncserver_proxyclient_address=172.19.7.1 --vncserver_listen=0.0.0.0 --vncserver_proxyclient_address=0.0.0.0 # network specific settings --network_manager=nova.network.manager.FlatDHCPManager #--network_manager=nova.network.quantum.manager.QuantumManager #--quantum_connection_host=172.19.7.1 #--quantum_connection_port=9696 --public_interface=eth0 --flat_interface=eth0 --flat_network_bridge=br100 --fixed_range=172.19.7.0/24 --network_size=254 --flat_network_dhcp_start=172.19.7.41 --multi_host --flat_injected=False --force_dhcp_release --iscsi_helper=tgtadm --connection_type=libvirt --root_helper=sudo nova-rootwrap #--verbose #--verbose=true --verbose=false --libvirt_xml_template=/usr/share/pyshared/nova/virt/libvirt.xml.template --allow_resize_to_same_host=true --max_cores=30 = if I misunderstand something, please correct me, thanks. -Jimmy ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] How does everyone build OpenStack disk images?
Hey Lorin,maybe the starter guide ? since it already contains pretty much everything to discover OPS ? Nuage Co - Razique Mahrouarazique.mahr...@gmail.com Le 26 avr. 2012 à 15:10, Lorin Hochstein a écrit :Broader question: what's the best place to capture this information?Right now, there's a chapter in the OpenStack Compute Admin Guide about images, but I'm tempted to break out a new document dedicated to creating and obtaining appropriate images that are compatible with OpenStack (or, more specifically, with OpenStack-supported hypervisors), and have that be accessible on docs.openstack.org. We could use the wiki, but I'm afraid most users wouldn't find it there, and this is a pretty critical issue for new users.Unless Take care,Lorin--Lorin HochsteinLead Architect - Cloud ServicesNimbis Services, Inc.www.nimbisservices.com On Apr 26, 2012, at 7:04 AM, Michael Basnight wrote:In reddwarf for development we use Ubuntu-vm-builder. It works like a charm for creating qcow2 images.Sent from my digital shackles.On Apr 25, 2012, at 8:14 PM, Justin Santa Barbara jus...@fathomdb.com wrote:How does everyone build OpenStack disk images? The official documentation describes a manual process (boot VM with ISO), which is sub-optimal in terms of repeatability / automation / etc. I'm hoping we can do better! I posted how I do it on my blog, here:http://blog.justinsb.com/blog/2012/04/25/creating-an-openstack-image/ Please let me know the many ways in which I'm doing it wrong :-)I'm thinking we can have a discussion here, and then I can then compile the responses into a wiki page and/or a nice script... Justin ___Mailing list: https://launchpad.net/~openstackPost to : openstack@lists.launchpad.netUnsubscribe : https://launchpad.net/~openstackMore help : https://help.launchpad.net/ListHelp___Mailing list: https://launchpad.net/~openstackPost to : openstack@lists.launchpad.netUnsubscribe : https://launchpad.net/~openstackMore help : https://help.launchpad.net/ListHelp___Mailing list: https://launchpad.net/~openstackPost to : openstack@lists.launchpad.netUnsubscribe : https://launchpad.net/~openstackMore help : https://help.launchpad.net/ListHelp___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] db notification support for API extension?
On Wed, Apr 25, 2012 at 6:04 PM, Andrew Bogott abog...@wikimedia.orgwrote: On 4/25/12 4:48 PM, Nathanael Burton wrote: On Thu, Mar 8, 2012 at 11:53 AM, Andrew Bogottabog...@wikimedia.org wrote: I'm working on an API and implementation to support the creation of filesystems that are shared among Nova instances. http://wiki.openstack.org/**SharedFShttp://wiki.openstack.org/SharedFS My hope is to keep this API isolated from core Nova code, partly to avoid stepping on toes and partly because I hope to be able to drop it into an existing essex install. There are two things I need which I know how to do within Nova but am not clear on how to do without modding Nova code: 1) DB support I need a database table to keep track of some filesystem metadata. My current implementation adds the table via nova/db/sqlalchemy/migrate_** repo... but is it really necessary to coordinate this table with the rest of Nova? Would it be reasonable to maintain the table independently within the extension code? And, if so, are there any existing extensions that do something like this? Have you determined a cleaner way of doing this? I have the same issues as you when writing API extensions. Nate -- The short answer is: I'm sure that it's straightforward to create a 'private' table which doesn't collide with existing nova tables, but I have yet to do so. The longer answer is: Everything about that thread is now rolled into the topic of 'the plugin framework' which we discussed at the design summit and which I'm currently devoted to. Please consider adding your use cases to the wiki page at http://wiki.openstack.org/**novapluginhttp://wiki.openstack.org/novaplugin, and let me know if you would like me to add you to the list of people I cc: when looking for opinions and/or reporting progress. The wiki page says that a plugin may want to Access the Nova database but that phrasing is a little vague. Does it mean Read and write data to its own tables in the Nova database (as mentioned later on the page) or Read data from the Nova tables or even Write data to the Nova tables? I assume if we're talking about Nova tables, access would be through the existing core classes in Nova that manage those tables, rather than manipulating them directly. Should that be stated explicitly? -Andrew __**_ Mailing list: https://launchpad.net/~**openstackhttps://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~**openstackhttps://launchpad.net/~openstack More help : https://help.launchpad.net/**ListHelphttps://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] questions about IP addressing and network config
Hi , This link should help you on the multinic feature . http://docs.openstack.org/trunk/openstack-compute/admin/content/using-multi-nics.html#using-multiple-nics-usage Thanks Meena Raja From: openstack-bounces+raja.meena=wipro@lists.launchpad.net [mailto:openstack-bounces+raja.meena=wipro@lists.launchpad.net] On Behalf Of Jimmy Tsai Sent: Thursday, April 26, 2012 2:52 PM To: openstack@lists.launchpad.net; Jimmy Subject: [Openstack] questions about IP addressing and network config Hi everyone, I'm running with Essex 2012.1, and have some questions about the nova network operation, 1. Is it possible manually assigned IP address to a launched instance, my situation is : after instance boot up (OS: CentOS 6.2), I changed the /etc/sysconfig/network-scripts/ifcfg-eth0 setting from dhcp to static (the same subnet as created by command : nova-manage create network), and restart the network service, And then I couldn't ssh or ping the instance from other server with the same subnet. What is the problem ? I checked the iptables policies on the compute host, and find nothing about the DROP packets. I also tried to change the DB record from nova.fixed_ips table and libvirt.xml of the instance directory, then reboot the instance, I can see the instance get new IP from DHCP, but still not worked(can't ping ssh). I used FlatDHCP as my network manager. 2. According to the first question, I have another requirement to set up a loopback IP address (lo:0) on the running instance, after setting completed,I couldn't ping or ssh the loopback IP from the same subnet, and I tried to set a alias IP address with eth0:0, but still not get worked. Any ideas with this ? 3. Is there any way to use 2 NICs with different subnets on instances? I want to separate the network traffic. Now I'm running with one bridged interface (br100), and it works well. In order to backup the large log files, I'm planing to use 2 NICs for the compute hosts, I want use 2 vNICs on instance, one for web service and the other for log backup, I think I should create a new network for the second bridged interface, but I can't find any document to guild me. List my nova.conf below == --dhcpbridge_flagfile=/etc/nova/nova.conf --dhcpbridge=/usr/bin/nova-dhcpbridge --logdir=/var/log/nova --state_path=/var/lib/nova --lock_path=/var/lock/nova --allow_admin_api=true --use_deprecated_auth=false --auth_strategy=keystone --scheduler_driver=nova.scheduler.simple.SimpleScheduler --s3_host=172.19.7.1 --ec2_host=172.19.7.1 --rabbit_host=172.19.7.1 --cc_host=172.19.7.1 --nova_url=http://172.19.7.1:8774/v1.1/ #--routing_source_ip=172.19.7.1 --glance_api_servers=172.19.7.1:9292http://172.19.7.1:9292/ --image_service=nova.image.glance.GlanceImageService #--iscsi_ip_prefix=192.168.22 --sql_connection=mysql://nova:nova@172.19.7.1/novahttp://nova:nova@172.19.7.1/nova --ec2_url=http://172.19.7.1:8773/services/Cloud --keystone_ec2_url=http://172.19.7.1:5000/v2.0/ec2tokens --api_paste_config=/etc/nova/api-paste.ini --libvirt_type=kvm --libvirt_use_virtio_for_bridges=true --start_guests_on_host_boot=true --resume_guests_state_on_host_boot=true #--vnc_enabled=true --novnc_enabled=true #--vncproxy_url=http://172.19.7.1:6080/vnc_auto.html #--vnc_console_proxy_url=http://172.19.7.1:6080http://172.19.7.1:6080/ --novncproxy_base_url=http://172.19.7.1:6080/vnc_auto.html --xvpvncproxy_base_url=http://172.19.7.1:6081/console #--vncserver_listen=172.19.7.1 #--vncserver_proxyclient_address=172.19.7.1 --vncserver_listen=0.0.0.0 --vncserver_proxyclient_address=0.0.0.0 # network specific settings --network_manager=nova.network.manager.FlatDHCPManager #--network_manager=nova.network.quantum.manager.QuantumManager #--quantum_connection_host=172.19.7.1 #--quantum_connection_port=9696 --public_interface=eth0 --flat_interface=eth0 --flat_network_bridge=br100 --fixed_range=172.19.7.0/24http://172.19.7.0/24 --network_size=254 --flat_network_dhcp_start=172.19.7.41 --multi_host --flat_injected=False --force_dhcp_release --iscsi_helper=tgtadm --connection_type=libvirt --root_helper=sudo nova-rootwrap #--verbose #--verbose=true --verbose=false --libvirt_xml_template=/usr/share/pyshared/nova/virt/libvirt.xml.template --allow_resize_to_same_host=true --max_cores=30 = if I misunderstand something, please correct me, thanks. -Jimmy Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The
[Openstack] Migration of Single, Dual and Multi node
Dear OpenStack Community, is it possible to install a single-node environment and then later on upgrade it to a dual-node environment and even continue to a multi node environment ? I know that it is possible to install all the openstack components (Nova, Glance, Swift) on all nodesyou are using in your network but I'm a bit unsure. Does anyone know there I can find the answer to my question ? Thanks for your help and have a nice day, Nicolas ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Migration of Single, Dual and Multi node
Yes , possible. You need to have the respective components installed as when you progress to dual/multi node have the references(location of Glance/Swift store) corrected in nova.conf respectively. Thanks Meena Raja From: openstack-bounces+raja.meena=wipro@lists.launchpad.net [mailto:openstack-bounces+raja.meena=wipro@lists.launchpad.net] On Behalf Of Nicolas Odermatt Sent: Thursday, April 26, 2012 8:01 PM To: openstack@lists.launchpad.net Subject: [Openstack] Migration of Single, Dual and Multi node Dear OpenStack Community, is it possible to install a single-node environment and then later on upgrade it to a dual-node environment and even continue to a multi node environment ? I know that it is possible to install all the openstack components (Nova, Glance, Swift) on all nodesyou are using in your network but I'm a bit unsure. Does anyone know there I can find the answer to my question ? Thanks for your help and have a nice day, Nicolas Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Using VMWare ESXi with openstack
On Apr 25, 2012, at 7:44 PM, Michael March wrote: I just curious. Is anyone using the VMware functionality in OpenStack? I'm getting the feeling that it is more of a 'check box' thing of yeah, we have that hypervisor covered than something that's seriously being used. If my feeling is wrong, I'd like to know. I am hoping some commercial interests take over the esx hypervisor. It is definitely behind kvm and xen, and I think this is only because no one is committing development resources to improve it. The other hypervisors seem to get more support. Hyper-V will be coming back because Microsoft has made a big commitment to it. Hopefully vmware or some other commercial entity cares enough to start really putting development effort into it. Vish ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] openstack.common setup code
Hey guys, Quick follow up from the summit on things that should happen in projects from the setup module of openstack common as I understand it. (to make sure we're all on the same page) There are currently 5 essential things in openstack.common.setup: parse_requirements parse_dependency_links write_requirements write_git_changelog generate_authors that are being used to varying levels in the various projects. What should happen at this point is this: parse_requirements parse_dependency_links Should be in all of the client libraries and should be removed from all the not-client libraries. These are essential for pip installation of client libs (which is important) as they allow pip to follow the depends. The make things hard for non-client libs, as setuptools doesn't understand git urls, which we use in non-client lib pip-requires files. write_requirements Should die everywhere. It was an attempt to record in our tarballs the state of what was actually tested ... but did not actually provide benefit to anyone - and the distros hate it. write_git_changelog generate_authors Should be added/used everywhere. When generate_authors is added, unit tests testing authors content should be removed. Is this how everyone else understood the outcome of conversations at the summit too? Thanks! Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Using Foreign Keys
On 04/25/2012 05:17 PM, Vishvananda Ishaya wrote: The main issue is when the relevant tables are moved into a separate service a la quantum or cinder. We can't keep referential integrity across multiple databases, so the foreign keys in this case need to be removed. It leads to an odd situation when there is still an internal implementation in addition to the external implementation because the internal implementation no longer has foreign keys. As an example, we used to have foreign key relationships between instances and networks. We can no longer have these because we support networks declared externally. The internal network management now has no referential integrity, but this is the price we pay for separation of concerns. We are going through a similar set of relationship-breaking with the volume code. There are definitely the practical aspects of where this can't be done because the services have split out, and I think that's fine. But enforcing the ref constraints where possible just provides another level of safety in the data. A policy where we break FK relationships if the preferred core model is 2 services (i.e. Nova / Quantum), but we add FK constraints within a service might be a good idea. -Sean -- Sean Dague IBM Linux Technology Center email: slda...@us.ibm.com alt-email: sda...@linux.vnet.ibm.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Using Foreign Keys
On 04/26/2012 10:14 AM, Sean Dague wrote: On 04/25/2012 05:17 PM, Vishvananda Ishaya wrote: The main issue is when the relevant tables are moved into a separate service a la quantum or cinder. We can't keep referential integrity across multiple databases, so the foreign keys in this case need to be removed. It leads to an odd situation when there is still an internal implementation in addition to the external implementation because the internal implementation no longer has foreign keys. As an example, we used to have foreign key relationships between instances and networks. We can no longer have these because we support networks declared externally. The internal network management now has no referential integrity, but this is the price we pay for separation of concerns. We are going through a similar set of relationship-breaking with the volume code. There are definitely the practical aspects of where this can't be done because the services have split out, and I think that's fine. But enforcing the ref constraints where possible just provides another level of safety in the data. A policy where we break FK relationships if the preferred core model is 2 services (i.e. Nova / Quantum), but we add FK constraints within a service might be a good idea. SO ... in a production MySQL service in this situation, under no circumstances should foreign keys actually be applied to the database. Specifying them as part of the SqlAlchemy model is fine, and I believe conveys the informational relationships that are important. But it turns out that in practice, especially with an ORM running things, the performance hit of adding them is pretty bad (generates tons of unneeded index scans, for one thing) If all of your db access is via the ORM layer, there is absolutely zero actual benefit. I think the real key is to have a config option to tell sqlalchemy to not, even if we're running innodb, add the foreign keys to the DDL sent to the database. If sqlalchemy doesn't have that ability, we should write it and contribute it, because anyone using MySQL at scale via sqlalchemy actually wants the feature, whether they recognize it yet or not. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Encrypted virtual machines
Hi Michael,I dunno how the integration is going regarding the encrypted images, but you can if you can use encrypted images with qemu/ qemu-kvm.If your disk is an encrypted qcow2 image, by typing "cont" in the qemu/ qemu-kvm monitor, you would see something like this :QEMU 0.11.0 monitor - type 'help' for more information(qemu) contide0-hd0 (encrypted.qcow2) is encrypted.Password: (qemu)By providing your password, the instance should boot normally. I haven't noticed any perf. issues, since once the image is decrypted, it acts like a normal image. Maybe you weren't thinking to that encryption ? Nuage Co - Razique Mahrouarazique.mahr...@gmail.com Le 26 avr. 2012 à 17:53, Michael Grosser a écrit :Hey,I'm following the openstack development for some time now and I was wondering if there was a solution to spin up encrypted virtual machines by default and if it would be a huge performance blow.Any ideas? Cheers Michael ___Mailing list: https://launchpad.net/~openstackPost to : openstack@lists.launchpad.netUnsubscribe : https://launchpad.net/~openstackMore help : https://help.launchpad.net/ListHelp___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Encrypted virtual machines
Data left on broken disks would be unreadable. -- You don't have to worry about data destruction before selling/throwing out your disks. (That could be realized via encrypting the whole compute-node disk, but that's not quite what I want.) Another benefit would be, that you as a cloud user wouldn't have to worry about the provider accessing your data. (Encrypting every vms disk for additional security.) Or am I seeing this too worry-some? On Thu, Apr 26, 2012 at 6:05 PM, Matt Joyce m...@nycresistor.com wrote: From a security stand point I am curious what you see the benefit as? On Thu, Apr 26, 2012 at 8:53 AM, Michael Grosser d...@seetheprogress.net wrote: Hey, I'm following the openstack development for some time now and I was wondering if there was a solution to spin up encrypted virtual machines by default and if it would be a huge performance blow. Any ideas? Cheers Michael ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Encrypted virtual machines
I'm looking into it, but I'm not sure if that's really how I want it to be. ;) Thanks for the hint. On Thu, Apr 26, 2012 at 6:08 PM, Razique Mahroua razique.mahr...@gmail.comwrote: Hi Michael, I dunno how the integration is going regarding the encrypted images, but you can if you can use encrypted images with qemu/ qemu-kvm. If your disk is an encrypted qcow2 image, by typing cont in the qemu/ qemu-kvm monitor, you would see something like this : QEMU 0.11.0 monitor - type 'help' for more information (qemu) cont ide0-hd0 (encrypted.qcow2) is encrypted. Password: (qemu) By providing your password, the instance should boot normally. I haven't noticed any perf. issues, since once the image is decrypted, it acts like a normal image. Maybe you weren't thinking to that encryption ? *Nuage Co - Razique Mahroua** * razique.mahr...@gmail.com Le 26 avr. 2012 à 17:53, Michael Grosser a écrit : Hey, I'm following the openstack development for some time now and I was wondering if there was a solution to spin up encrypted virtual machines by default and if it would be a huge performance blow. Any ideas? Cheers Michael ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp NUAGECO-LOGO-Fblan_petit.jpg___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] How does everyone build OpenStack disk images?
The oz tool that Daniel mentions makes image creation and customization pretty simple. I've created some templates for building images at github.com/rackerjoe/oz-image-buildhttp://github.com/rackerjoe/oz-image-build if anyone is interested. --- Joseph Breu Deployment Engineer Rackspace Cloud Builders 210-312-3508 On Apr 26, 2012, at 3:19 AM, Daniel P. Berrange wrote: On Wed, Apr 25, 2012 at 06:14:22PM -0700, Justin Santa Barbara wrote: How does everyone build OpenStack disk images? The official documentation describes a manual process (boot VM with ISO), which is sub-optimal in terms of repeatability / automation / etc. I'm hoping we can do better! I posted how I do it on my blog, here: http://blog.justinsb.com/blog/2012/04/25/creating-an-openstack-image/ Please let me know the many ways in which I'm doing it wrong :-) I'm thinking we can have a discussion here, and then I can then compile the responses into a wiki page and/or a nice script... If you have a KVM enabled machine, then 'Oz' has the ability to create JeOS images for all the common distros you'll find. It is a very simple command line tool that just focuses on image building image customization (adding more packages to an existing JeOS image). http://aeolusproject.org/oz.html Yes, it is on the Aeolus project website, but it has no external dependancies on the rest of Aeolus - it just wants kvm, libvirt a few commonly available python modules. I've often thought that it would be desirable to have Oz integrated into OpenStack to provide an native image building capability. Given their common Python heritage I think it would work quite well. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack.common setup code
Hi, write_changelogs is really important to a distro point of view because we ship snapshots of trunk during our development cycle so informing the users whats in the snapshot is really important. chuck On Thu, 26 Apr 2012 10:12:05 -0500 Monty Taylor mord...@inaugust.com wrote: Hey guys, Quick follow up from the summit on things that should happen in projects from the setup module of openstack common as I understand it. (to make sure we're all on the same page) There are currently 5 essential things in openstack.common.setup: parse_requirements parse_dependency_links write_requirements write_git_changelog generate_authors that are being used to varying levels in the various projects. What should happen at this point is this: parse_requirements parse_dependency_links Should be in all of the client libraries and should be removed from all the not-client libraries. These are essential for pip installation of client libs (which is important) as they allow pip to follow the depends. The make things hard for non-client libs, as setuptools doesn't understand git urls, which we use in non-client lib pip-requires files. write_requirements Should die everywhere. It was an attempt to record in our tarballs the state of what was actually tested ... but did not actually provide benefit to anyone - and the distros hate it. write_git_changelog generate_authors Should be added/used everywhere. When generate_authors is added, unit tests testing authors content should be removed. Is this how everyone else understood the outcome of conversations at the summit too? Thanks! Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Encrypted virtual machines
+1 From a security stand point I am curious what you see the benefit as? On Thu, Apr 26, 2012 at 8:53 AM, Michael Grosser d...@seetheprogress.net wrote: Hey, I'm following the openstack development for some time now and I was wondering if there was a solution to spin up encrypted virtual machines by default and if it would be a huge performance blow. Any ideas? Cheers Michael ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Unable to download images using Glance+Keystone+Swift
On 04/26/2012 11:54 AM, Lillie Ross-CDSR11 wrote: 4. However, when I try to download the same image, I receive the following error: curl -v -H 'X-Auth-Token: 45d01460a0e04bff967eb954e7f4fee8' -H 'Content-type: application/json' http://essex3:9292/v1/images/6720c572-12b7-4cc8-a8c5-95b92998671a | python -mjson.tool You need to remove the | python -mjson.tool :) Don't really want to be piping an image file into that module... Best, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] questions about IP addressing and network config
Hi Meena, Thanks for your reply, this solve my third question. :D Wait for someone who could help with the First 2 questions. how to change the IP address after instance has launched and get an address from dnsmasq ? and how to bind a loopback IP address (lo:0) or an alias IP address (eth0:0) ? I just started to test Quantum + Melange, don't know if it helps with my questions. Thanks, -Jimmy 2012/4/26 raja.me...@wipro.com Hi , ** ** This link should help you on the multinic feature . ** ** http://docs.openstack.org/trunk/openstack-compute/admin/content/using-multi-nics.html#using-multiple-nics-usage ** ** ** ** ** ** Thanks Meena Raja ** ** ** ** *From:* openstack-bounces+raja.meena=wipro@lists.launchpad.net[mailto: openstack-bounces+raja.meena=wipro@lists.launchpad.net] *On Behalf Of *Jimmy Tsai *Sent:* Thursday, April 26, 2012 2:52 PM *To:* openstack@lists.launchpad.net; Jimmy *Subject:* [Openstack] questions about IP addressing and network config*** * ** ** Hi everyone, I'm running with Essex 2012.1, and have some questions about the nova network operation, 1. Is it possible manually assigned IP address to a launched instance, my situation is : after instance boot up (OS: CentOS 6.2), I changed the /etc/sysconfig/network-scripts/ifcfg-eth0 setting from dhcp to static (the same subnet as created by command : nova-manage create network), and restart the network service, And then I couldn't ssh or ping the instance from other server with the same subnet. What is the problem ? I checked the iptables policies on the compute host, and find nothing about the DROP packets. I also tried to change the DB record from nova.fixed_ips table and libvirt.xml of the instance directory, then reboot the instance, I can see the instance get new IP from DHCP, but still not worked(can't ping ssh). I used FlatDHCP as my network manager. 2. According to the first question, I have another requirement to set up a loopback IP address (lo:0) on the running instance, after setting completed,I couldn't ping or ssh the loopback IP from the same subnet, and I tried to set a alias IP address with eth0:0, but still not get worked.** ** Any ideas with this ? 3. Is there any way to use 2 NICs with different subnets on instances? I want to separate the network traffic. Now I'm running with one bridged interface (br100), and it works well. In order to backup the large log files, I'm planing to use 2 NICs for the compute hosts, I want use 2 vNICs on instance, one for web service and the other for log backup, I think I should create a new network for the second bridged interface, but I can't find any document to guild me. List my nova.conf below == --dhcpbridge_flagfile=/etc/nova/nova.conf --dhcpbridge=/usr/bin/nova-dhcpbridge --logdir=/var/log/nova --state_path=/var/lib/nova --lock_path=/var/lock/nova --allow_admin_api=true --use_deprecated_auth=false --auth_strategy=keystone --scheduler_driver=nova.scheduler.simple.SimpleScheduler --s3_host=172.19.7.1 --ec2_host=172.19.7.1 --rabbit_host=172.19.7.1 --cc_host=172.19.7.1 --nova_url=http://172.19.7.1:8774/v1.1/ #--routing_source_ip=172.19.7.1 --glance_api_servers=172.19.7.1:9292 --image_service=nova.image.glance.GlanceImageService #--iscsi_ip_prefix=192.168.22 --sql_connection=mysql://nova:nova@172.19.7.1/nova --ec2_url=http://172.19.7.1:8773/services/Cloud --keystone_ec2_url=http://172.19.7.1:5000/v2.0/ec2tokens --api_paste_config=/etc/nova/api-paste.ini --libvirt_type=kvm --libvirt_use_virtio_for_bridges=true --start_guests_on_host_boot=true --resume_guests_state_on_host_boot=true #--vnc_enabled=true --novnc_enabled=true #--vncproxy_url=http://172.19.7.1:6080/vnc_auto.html #--vnc_console_proxy_url=http://172.19.7.1:6080 --novncproxy_base_url=http://172.19.7.1:6080/vnc_auto.html --xvpvncproxy_base_url=http://172.19.7.1:6081/console #--vncserver_listen=172.19.7.1 #--vncserver_proxyclient_address=172.19.7.1 --vncserver_listen=0.0.0.0 --vncserver_proxyclient_address=0.0.0.0 # network specific settings --network_manager=nova.network.manager.FlatDHCPManager #--network_manager=nova.network.quantum.manager.QuantumManager #--quantum_connection_host=172.19.7.1 #--quantum_connection_port=9696 --public_interface=eth0 --flat_interface=eth0 --flat_network_bridge=br100 --fixed_range=172.19.7.0/24 --network_size=254 --flat_network_dhcp_start=172.19.7.41 --multi_host --flat_injected=False --force_dhcp_release
Re: [Openstack] Pending reviews
There's something like 7 pages of open reviews on gerrit. The project has a good kind of problem with so many people trying to contribute. The question now is how to scale the development processes to handle that growth. It was nice to see a number of discussions at the summit in this area. The biggest backlog is nova, and there are discussions of both splitting parts out to make nova smaller, as well as adopting feature branches and merge windows. The feature branches could have more reviewers that are experts in that area, but not necessarily nova-core. Hopefully these things will help in the Folsom cycle. Thanks to all of the core reviewers who regularly invest time into reviewing submissions! :-) Some simple processes that I've seen improve matters on seemingly unmanagable backlogs: 1. An initial short concerted queue draining exercise (e.g. a review-busting day where all core team members agree to dedicate a significant portion of their openstack time to reviews). The intended outcome is a much leaner queue as a starting point (at the cost of potential instability with many more patches landing on master than would normally be the case, so it makes sense to do this early in the release cycle). 2. Prominent visibility to a number of simple stats that capture the trend on responsiveness: - age of the oldest unreviewed patch - average turnaround time from submission to merge or -2 - number of open unreviewed patches - number of reviewed patches needing approval There would an implicit goal not to leave the stats in worse shape than yesterday at the end of each core-team members' rostered review day. 3. A loose SLA indicating the level of responsiveness that patch submitters can expect, e.g. we strive to respond within X working days, average turnaround time is currently Y days. 4. If things get out of hand again GOTO #1. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Encrypted virtual machines
Data left on broken disks would be unreadable. -- You don't have to worry about data destruction before selling/throwing out your disks. I can certainly see the goal here. But this may be harder than you think. For example, if you encrypt the disk image, then launch the VM, are you sure that any unencrypted data is NOT being written back to the drive (e.g., through the host OS swap)? -bryan ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Encrypted virtual machines
On 04/26/2012 12:11 PM, Michael Grosser wrote: Data left on broken disks would be unreadable. -- You don't have to worry about data destruction before selling/throwing out your disks. (That could be realized via encrypting the whole compute-node disk, but that's not quite what I want.) Another benefit would be, that you as a cloud user wouldn't have to worry about the provider accessing your data. (Encrypting every vms disk for additional security.) Or am I seeing this too worry-some? No, I think that's the right level of worry-some - http://www.contextis.co.uk/research/blog/dirtydisks/ -Sean -- Sean Dague IBM Linux Technology Center email: slda...@us.ibm.com alt-email: sda...@linux.vnet.ibm.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Encrypted virtual machines
On Thu, Apr 26, 2012 at 9:05 AM, Matt Joyce m...@nycresistor.com wrote: From a security stand point I am curious what you see the benefit as? I think that long-term there is the potential to have a cloud where you don't have to trust the cloud provider (e.g. Intel Trusted Compute). However, there are a huge number of steps that need to happen first, so I don't know that encrypting the qcow disk image would get you very much today. However, you could encrypt your filesystem (inside the disk image), and have it prompt for a password on boot. Then you could go in via VNC (today) and unlock your disk image. Your cloud provider can still grab memory etc. But I think that's the best you can do today. One day we may be able to automate something similar, yet still have it be secure. Virtualized I/O performance is poor compared to CPU performance, so I guess you wouldn't even notice the hit! But this is pure speculation, A little plug - one of the pieces of the big picture is figuring out how to store secrets; at the design summit I proposed storing them securely in Keystone; I just wrote up the (first draft?) of the blueprint: https://blueprints.launchpad.net/nova/+spec/secure-secret-storage Justin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] questions about IP addressing and network config
On Apr 25, 2012, at 7:31 PM, Jimmy Tsai wrote: Hi everyone, I'm running with Essex 2012.1, and have some questions about the nova network operation, 1. Is it possible manually assigned IP address to a launched instance, my situation is : after instance boot up (OS: CentOS 6.2), I changed the /etc/sysconfig/network-scripts/ifcfg-eth0 setting from dhcp to static (the same subnet as created by command : nova-manage create network), and restart the network service, And then I couldn't ssh or ping the instance from other server with the same subnet. What is the problem ? I checked the iptables policies on the compute host, and find nothing about the DROP packets. I also tried to changed the record from nova.fixed_ips table and libvirt.xml of the instance, then reboot the instance, still not worked. I used FlatDHCP as my network manager. You can't do this. Libvirt sets up no mac spoofing and no ip spoofing so the ip address needs to match the dhcp'd one. You should be able to switch to a static and use the same info that you get from dhcp though. 2. According to the first question, I have another requirement to set up a loopback IP address (lo:0) on the running instance, after setting was completed,I couldn't ping or ssh the loopback IP from the same subnet, and I tried to set a alias IP address with eth0:0, but still not get worked. Any ideas with this ? Not sure 3. Is there any way to use 2 NICs with different subnets on instances? I want to separate the network traffic. Now I'm running with one bridged interface (br100), and it works well. In order to backup the large log files, I'm planing to use 2 NICs for the compute hosts, I want use 2 vNICs on instance, one for web service and the other for log backup, I think I should create a new network for the second bridged interface, but I can't find any document to guild me. This is definitely possible with FlatManager (You could use cloud_config drive and some version of contrib/openstack-config converted to work with centos to set up the interfaces) It was possible at one point with FlatDHCPManager as well by creating multiple networks and using a specific combination of config options like use_single_default_gateway. I don' t know if anyone has tried this for a while so there may be issues with it. You might try creating a second network and setting use_single_default_gateway and see what happens. There are plans underway to support this by only dhcping the first interface and allowing a guest agent to set up the other interfaces, but it isn't in place yet. Vish ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Fwd: Unable to download images using Glance+Keystone+Swift
Begin forwarded message: From: Ross Lillie ross.lil...@motorolasolutions.commailto:ross.lil...@motorolasolutions.com Subject: Re: [Openstack] Unable to download images using Glance+Keystone+Swift Date: April 26, 2012 1:37:45 PM CDT To: Jay Pipes jaypi...@gmail.commailto:jaypi...@gmail.com Cc: Ross Lillie ross.lil...@motorolasolutions.commailto:ross.lil...@motorolasolutions.com Hi Jay, Cut and paste error. It still doesn't work. If I issue the simple command (without the pipe or content-type header) I get the following root@essex1:/etc/keystone# curl -v -H 'X-Auth-Token: 45d01460a0e04bff967eb954e7f4fee8' http://essex3:9292/v1/images/423b0ecc-5ca1-44d8-8e85-5a245ce620e2 * About to connect() to essex3 port 9292 (#0) * Trying 172.16.1.5... connected GET /v1/images/423b0ecc-5ca1-44d8-8e85-5a245ce620e2 HTTP/1.1 User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3 Host: essex3:9292 Accept: */* X-Auth-Token: 45d01460a0e04bff967eb954e7f4fee8 HTTP/1.1 404 Not Found Content-Length: 315 Content-Type: text/html; charset=UTF-8 Date: Thu, 26 Apr 2012 18:35:21 GMT html head title404 Not Found/title /head body h1404 Not Found/h1 An object with the specified identifier was not found. Details: Swift could not find image at uri swift+http://service:glance:glance@essex1:5000/v2.0/glance/423b0ecc-5ca1-44d8-8e85-5a245ce620e2br /br / /body * Connection #0 to host essex3 left intact * Closing connection #0 /html root@essex1:/etc/keystone# Now, I can access the image directly via the Swift CLI using my glance tenant, username, and password. However, the Glance REST call fails. All other REST calls work fine. I'm stumped. Ross On Apr 26, 2012, at 11:55 AM, Jay Pipes wrote: On 04/26/2012 11:54 AM, Lillie Ross-CDSR11 wrote: 4. However, when I try to download the same image, I receive the following error: curl -v -H 'X-Auth-Token: 45d01460a0e04bff967eb954e7f4fee8' -H 'Content-type: application/json' http://essex3:9292/v1/images/6720c572-12b7-4cc8-a8c5-95b92998671a | python -mjson.tool You need to remove the | python -mjson.tool :) Don't really want to be piping an image file into that module... Best, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] Minimum required code coverage per file
It would nice to initially see the code coverage delta per merge proposal as a comment in gerrit (similar to SmokeStack), and not as a gating factor. Kevin, should we start copying openstack-common tests to client projects? Or just make sure to not count openstack-common code in the code coverage numbers for client projects? best, Joe On Wed, Apr 25, 2012 at 7:30 PM, Tim Simpson tim.simp...@rackspace.comwrote: Great point Justin. I've worked on projects where this has happened repeatedly and it's a drag. -- *From:* openstack-bounces+tim.simpson=rackspace@lists.launchpad.net[openstack-bounces+tim.simpson= rackspace@lists.launchpad.net] on behalf of Justin Santa Barbara [ jus...@fathomdb.com] *Sent:* Wednesday, April 25, 2012 5:20 PM *To:* Monty Taylor *Cc:* openstack@lists.launchpad.net *Subject:* Re: [Openstack] [OpenStack][Nova] Minimum required code coverage per file One concern I have is this: suppose we find that a code block is unnecessary, or can be refactored more compactly, but it has test coverage. Then removing it would make the % coverage fall. We want to remove the code, but we'd have to add unrelated tests to the same merge because otherwise the test coverage % would fall? I think we can certainly enhance the metrics, but I do have concerns over strict gating (particularly per file, where the problem is more likely to occur than per-project) Maybe the gate could be that line count of uncovered lines must not increase, unless the new % coverage 80%. Or we could simply have a gate bypass. Justin On Wed, Apr 25, 2012 at 2:45 PM, Monty Taylor mord...@inaugust.comwrote: Hey - funny story - in responding to Justin I re-read the original email and realized it was asking for a static low number, which we _can_ do - at least project-wide. We can't do per-file yet, nor can we fail on a downward inflection... and I've emailed Justin about that. If we have consensus on gating on project-wide threshold, I can certainly add adding that to the gate to the todo list. (If we decide to do that, I'd really like to make that be openstack-wide rather than just nova... although I imagine it might take a few weeks to come to consensus on what the project-wide low number should be. Current numbers on project-wide lines numbers: nova: 79% glance: 75% keystone: 81% swift: 80% horizon: 91% Perhaps we get nova and glance up to 80 and then set the threshold for 80? Also, turns out we're not running this on the client libs... Monty On 04/25/2012 03:53 PM, Justin Santa Barbara wrote: If you let me know in a bit more detail what you're looking for, I can probably whip something up. Email me direct? Justin On Wed, Apr 25, 2012 at 6:59 AM, Monty Taylor mord...@inaugust.com mailto:mord...@inaugust.com wrote: On 04/24/2012 10:08 PM, Lorin Hochstein wrote: On Apr 24, 2012, at 4:11 PM, Joe Gordon wrote: Hi All, I would like to propose a minimum required code coverage level per file in Nova. Say 80%. This would mean that any new feature/file should only be accepted if it has over 80% code coverage. Exceptions to this rule would be allowed for code that is covered by skipped tests (as long as 80% is reached when the tests are not skipped). I like the idea of looking at code coverage numbers. For any particular merge proposal, I'd also like to know whether it increases or decreases the overall code coverage of the project. I don't think we should gate on this, but it would be helpful for a reviewer to see that, especially for larger proposals. Yup... Nati requested this a couple of summits ago - main issue is that while we run code coverage and use the jenkins code coverage plugin to track the coverage numbers, the plugin doesn't fully support this particular kind of report. HOWEVER - if any of our fine java friends out there want to chat with me about adding support to the jenkins code coverage plugin to track and report this, I will be thrilled to put it in as a piece of reported information. With 193 python files in nova/tests, Nova unit tests produce 85% overall code coverage (calculated with ./run_test.sh -c [1]). But 23% of files (125 files) have lower then 80% code coverage (30 tests skipped on my machine). Getting all files to hit the 80% code coverage mark should be one of the goals for Folsom. I would really like to see a visualization of the code coverage distribution, in order to help spot the outliers. Along these lines, there's been a lot of work in the software engineering research community about predicting which parts of the code are most likely to contain bugs
Re: [Openstack] How does everyone build OpenStack disk images?
On 04/26/2012 04:19 AM, Daniel P. Berrange wrote: If you have a KVM enabled machine, then 'Oz' has the ability to create JeOS images for all the common distros you'll find. It is a very simple command line tool that just focuses on image building image customization (adding more packages to an existing JeOS image). http://aeolusproject.org/oz.html Interesting, I'll check it out. Yes, it is on the Aeolus project website, but it has no external dependancies on the rest of Aeolus - it just wants kvm, libvirt a few commonly available python modules. I've often thought that it would be desirable to have Oz integrated into OpenStack to provide an native image building capability. Given their common Python heritage I think it would work quite well. Would be interesting to get that integrated into Glance somehow... Best, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] database migration cleanup
The OpenStack Essex release had 82 database migrations. As these grow in number it seems reasonable to clean house from time to time. Now seems as good a time as any. I came up with a first go at it here: https://review.openstack.org/#/c/6847/ The idea is that we would: * Do this early in the release cycle to minimize risk. * Compact all pre-Folsom migrations into a single migration. This migration would be used for new installations. * New migrations during the Folsom release cycle would proceed as normal. * Migrations added during Folsom release cycle could be compacted during E release cycle. TBD if/when we do the next compaction. * Users upgrading from pre-Essex would need to upgrade to Essex first. Then Folsom. -- I think this scheme would support users who follow stable releases as well as users who follow trunk very closely. We talked about this at the conference but I thought this issue might be near and dear to some of our end users so it was worth discussing on the list. What are general thoughts on this approach? Dan (dprince) ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] OpenStack Quantum plugins
Hi All, I am trying to learn the functionality of Quantum plugins used in OpenStack. I have read through the Quantum Admin Guide and had few basic/quick question about quantum and OVS interaction with it: 1) OVS can have ports in which vNICS can be plugged, so why does it need to use an integration bridge for connecting all VMs on the same node to a network? 2) The OVS quantum plugin seems to implement the core API functions and (viewing the code) I concluded that it just makes maintains the logical mappings e.g. b/w net IDs and VLAN IDs in a database. So how is this mapping implemented on the actual ports of OVS? Is it the OVS quantum agent responsible for directing the packets to correct input/output ports based on the updates that it gets from the database? 3) The quantum admin guide says that the nova client will be the main user of quantum and will interact with it via REST API, so it would be nice if someone can point me to the code (file path name etc.) where this happens. Thanks, Salman PS: What is the purpose of Quantum Manager in this architecture and where should I look for its code? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Using Nova APIs from Javascript: possible?
Interesting! Nice job on jstack! On Apr 26, 2012, at 12:50 AM, javier cerviño wrote: Hi all, I'm glad to hear that there's a lot of interest in the implementation of Openstack JavaScript clients. Actually, in my group we're developing a single page application developed entirely in JavaScript, that widely supports Nova and Keystone APIs. This work is part of a European Project called FI-Ware (http://www.fi-ware.eu/), in which we are currently using Openstack APIs. We've modified Nova and Keystone installations by adding CORS support. We did it by implementing a kind of filter on their APIs. For doing this we used Adam's implementation (https://github.com/adrian/swift/tree/cors), and we adapted it to Nova and Keystone components. We also developed a JS library (http://ging.github.com/jstack/) that can be used by both web and Node.js applications, for example. This library aims to provide same functionalities as python-novaclient, adding support for Keystone API. And finally we are copying Openstack horizon functionality, using JS library and other frameworks such as jQuery and Backbone.js to implement the web application. This web application is an early-stage work, but we will probably publish it by the end of this week. I will let you know the github link. We didn't find much problems with CORS implementation and support in browsers. For the time being, according to our experiments, the only web browser that is not usable at all with this technology is Internet Explorer, but we have tried it in Google Chrome, Safari and Firefox as well and we didn't have any problems. Cheers, Javier Cerviño. On 26 April 2012 06:28, Nick Lothian nick.loth...@gmail.com wrote: On Thu, Apr 26, 2012 at 5:49 AM, Adam Young ayo...@redhat.com wrote: Let me try to summarize: 1. If you are running from a web browser, post requests to hosts or ports other than the origin are allowed, but the headers cannot be modified. This prevents the addition of the token from Keystone to provide single sign on. 2. There are various browser side technologies (JSONP, CORS) that get around this limitation, but they are typically not enabled, and can be considered security issues. While implementing these might require support from teh Openstack server, they are fundamentally browser decisions. This is inaccurate. JSONP is supported by all browsers since ~Netscape 4.0. CORS is supported by all modern browsers: IE 8, Firefox 3.5, Chrome 3, Safari 4 (See http://en.wikipedia.org/wiki/Cross-origin_resource_sharing#Browser_support). Additionally, CORS support is not a browser decision - the server has to EXPLICITLY opt-in to support it. Obviously CORS support *can* be a security issue - that is why it is disabled unless the server enables it. I do not believe that CORS support adds any additional security issues above what the OpenStack APIs already face. Specially, the most common problem (CSRF) is not an issue here because the APIs are not authorised on a session basis. [snip] I've been working on Single Sign on Issues for another project for the past year and a half. Here's a couple things I've learned. Kerberos is designed to solve this problem. It has the benefit of being integrated into the browser. Where Kerberos fails is that: typically it only allows a single authentication provider (KDC in Kerberso speak) and it does not work well with Firewalls. The only crytographically secure way to authenticate on the web that can get around the firewall issue is Client side X509 certificates. This is the foundation for https://blueprints.launchpad.net/keystone/+spec/pki. This could, in theory, work in with OAuth, OpenID, or some other distributed authorization service, or we could embed the authorization information right into the Certitificate, which is what I suggest we do. To be clear, identity/authorisation is NOT the problem here. The OpenStack APIs work well for my use cases, once I work around the cross domain POST problem. However, I've also worked with SSO solutions. The simple truth is that client side certificates do not play well with the web - browser support ranges from non-existent (on some mobile platforms - see http://mobilitydojo.net/2010/12/28/client-certificate-support-across-mobile-platforms-a-summary/) to abysmal (there is a reason why many websites that use certificates end up using a Java applet), and their interaction with cross domain Javascript is unknown. Even if certificates did work for identification, CORS would still be needed - many OpenStack APIs require a POST request which is impossible without it. Nick ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Encrypted virtual machines
On Thu, Apr 26, 2012 at 09:05:41AM -0700, Matt Joyce wrote: From a security stand point I am curious what you see the benefit as? Consider that you might have separate people in your data center managing the virtualization hosts, vs the storage hosts vs the network. As it standards today any of those groups of people can compromise data stored in a VM disk image (assuming a network based filesystem). First you encrypt the disk image, so that a person with access to the storage hosts, or network sniffing can't read any data. Then you have a central key server that only gives out the decryption key to Nova compute nodes when they have been explicitly authorized to run an instance of that VM. So now people with access to the storage hosts cannot compromise any data. People with access to the virtualization hosts can only compromise data if the host has been authorized to use that disk image You would need to compromise the precise host the VM disk is being used on, or compromise the key server or the management service that schedules VMs (thus authorizing key usage on a node). NB this is better than relying on the guest OS to do encryption, since you can do stricter decryption key management from the host side. On Thu, Apr 26, 2012 at 8:53 AM, Michael Grosser d...@seetheprogress.net wrote: Hey, I'm following the openstack development for some time now and I was wondering if there was a solution to spin up encrypted virtual machines by default and if it would be a huge performance blow. Any ideas? I would like to extend the libvirt driver in Nova to make use of the qcow2 encryption capabilities between libvirt QEMU which I describe here: http://berrange.com/posts/2009/12/02/using-qcow2-disk-encryption-with-libvirt-in-fedora-12/ Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Question on notifications
Thx.
With these messages, instead of the compute.instance.create.end it can't be
guaranteed that the instance actually got created right?
If I listen for the compute.instance.create.end and use the hostname (which
is part of the publisher id) then I can know that it actually got created?
Is the weighted_host also dependent on which type of scheduler is used? (I
would assume that not all schedulers do weighting?)
On 4/25/12 5:29 PM, Sandy Walsh sandy.wa...@rackspace.com wrote:
You want these events:
scheduler.run_instance.start (generated when scheduling begins)
scheduler.run_instance.scheduled (when a host is selected. one per instance)
scheduler.run_instance.end (all instances placed)
The .scheduled event will have the target hostname in it in the
weighted_host key ...
For example ...
[u'monitor.info',
{u'_context_auth_token': None,
u'_context_is_admin': True,
u'_context_project_id': None,
u'_context_quota_class': None,
u'_context_read_deleted': u'no',
u'_context_remote_address': None,
u'_context_request_id': u'req-...ac',
u'_context_roles': [u'admin', u'identity:admin'],
u'_context_timestamp': u'2012-04-25T20:32:44.506538',
u'_context_user_id': None,
u'event_type': u'scheduler.run_instance.scheduled',
u'message_id': u'2df8...fc',
u'payload': {u'instance_id': u'7c21...960',
u'request_spec': {u'block_device_mapping': [],
u'image': {u'checksum': u'ee0e...cfcc',
u'container_format': u'ovf',
u'created_at': u'2012-02-29 23:12:16',
u'deleted': False,
u'deleted_at': None,
u'disk_format': u'vhd',
u'id': u'079...b5fb',
u'is_public': True,
u'min_disk': u'10',
u'min_ram': u'256',
u'name': u'CentOS 6.0',
u'properties': {u'arch': u'x86-64',
u'auto_disk_config': u'True',
u'os_distro': u'centos',
u'os_type': u'linux',
u'os_version': u'6.0',
u'rax_managed': u'false',
u'rax_options': u'0'},
u'size': 390243020,
u'status': u'active',
u'updated_at': u'2012-02-29 23:12:32'},
u'instance_properties': {u'access_ip_v4': None,
u'access_ip_v6': None,
u'architecture': u'x86-64',
u'auto_disk_config': True,
u'availability_zone': None,
u'config_drive': u'',
.u'config_drive_id': u'',
u'display_description': u'testserver...9870',
u'display_name': u'testserver...9870',
u'ephemeral_gb': 0,
u'image_ref': u'0790...b5fb',
u'instance_type_id': 1,
u'kernel_id': u'',
u'key_data': None,
u'key_name': None,
u'launch_index': 0,
u'launch_time': u'2012-04-25T20:32:10Z',
u'locked': False,
u'memory_mb': 256,
u'metadata': {},
u'os_type': u'linux',
u'power_state': 0,
u'progress': 0,
u'project_id': u'5820792',
u'ramdisk_id': u'',
u'reservation_id': u'r-j...mm',
u'root_device_name': None,
u'root_gb': 10,
u'user_data': u'',
u'user_id': u'162201',
u'uuid': u'7c210...ed8960',
u'vcpus': 4,
u'vm_mode': None,
u'vm_state': u'building'},
u'instance_type': {u'created_at': None,
u'deleted': False,
u'deleted_at': None,
u'ephemeral_gb': 0,
u'extra_specs': {},
u'flavorid': u'1',
u'id': 1,
u'memory_mb': 256,
u'name': u'256MB instance',
u'root_gb': 10,
Re: [Openstack] database migration cleanup
+1. Might be nice to have some kind of test to verify that the new migration leaves the tables in exactly the same state as the old migrations. Vish On Apr 26, 2012, at 12:24 PM, Dan Prince wrote: The OpenStack Essex release had 82 database migrations. As these grow in number it seems reasonable to clean house from time to time. Now seems as good a time as any. I came up with a first go at it here: https://review.openstack.org/#/c/6847/ The idea is that we would: * Do this early in the release cycle to minimize risk. * Compact all pre-Folsom migrations into a single migration. This migration would be used for new installations. * New migrations during the Folsom release cycle would proceed as normal. * Migrations added during Folsom release cycle could be compacted during E release cycle. TBD if/when we do the next compaction. * Users upgrading from pre-Essex would need to upgrade to Essex first. Then Folsom. -- I think this scheme would support users who follow stable releases as well as users who follow trunk very closely. We talked about this at the conference but I thought this issue might be near and dear to some of our end users so it was worth discussing on the list. What are general thoughts on this approach? Dan (dprince) ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Question on notifications
Yes, correct, I thought you wanted the info as soon as the scheduler decided on
a host. create.end will only fire when the instance has been created.
And you're correct about the scheduler, but all schedulers will likely be a
derivation of FilterScheduler or simply have custom filters/weights. Simple and
Change will turn to filters/weights soon. Depends on your installation.
-Sandy
From: Joshua Harlow [harlo...@yahoo-inc.com]
Sent: Thursday, April 26, 2012 5:07 PM
To: Sandy Walsh; openstack
Subject: Re: [Openstack] Question on notifications
Thx.
With these messages, instead of the “compute.instance.create.end” it can’t be
guaranteed that the instance actually got created right?
If I listen for the “compute.instance.create.end” and use the hostname (which
is part of the publisher id) then I can know that it actually got created?
Is the “weighted_host” also dependent on which type of scheduler is used? (I
would assume that not all schedulers do weighting?)
On 4/25/12 5:29 PM, Sandy Walsh sandy.wa...@rackspace.com wrote:
You want these events:
scheduler.run_instance.start (generated when scheduling begins)
scheduler.run_instance.scheduled (when a host is selected. one per instance)
scheduler.run_instance.end (all instances placed)
The .scheduled event will have the target hostname in it in the
weighted_host key ...
For example ...
[u'monitor.info',
{u'_context_auth_token': None,
u'_context_is_admin': True,
u'_context_project_id': None,
u'_context_quota_class': None,
u'_context_read_deleted': u'no',
u'_context_remote_address': None,
u'_context_request_id': u'req-...ac',
u'_context_roles': [u'admin', u'identity:admin'],
u'_context_timestamp': u'2012-04-25T20:32:44.506538',
u'_context_user_id': None,
u'event_type': u'scheduler.run_instance.scheduled',
u'message_id': u'2df8...fc',
u'payload': {u'instance_id': u'7c21...960',
u'request_spec': {u'block_device_mapping': [],
u'image': {u'checksum': u'ee0e...cfcc',
u'container_format': u'ovf',
u'created_at': u'2012-02-29 23:12:16',
u'deleted': False,
u'deleted_at': None,
u'disk_format': u'vhd',
u'id': u'079...b5fb',
u'is_public': True,
u'min_disk': u'10',
u'min_ram': u'256',
u'name': u'CentOS 6.0',
u'properties': {u'arch': u'x86-64',
u'auto_disk_config': u'True',
u'os_distro': u'centos',
u'os_type': u'linux',
u'os_version': u'6.0',
u'rax_managed': u'false',
u'rax_options': u'0'},
u'size': 390243020,
u'status': u'active',
u'updated_at': u'2012-02-29 23:12:32'},
u'instance_properties': {u'access_ip_v4': None,
u'access_ip_v6': None,
u'architecture': u'x86-64',
u'auto_disk_config': True,
u'availability_zone': None,
u'config_drive': u'',
.u'config_drive_id': u'',
u'display_description': u'testserver...9870',
u'display_name': u'testserver...9870',
u'ephemeral_gb': 0,
u'image_ref': u'0790...b5fb',
u'instance_type_id': 1,
u'kernel_id': u'',
u'key_data': None,
u'key_name': None,
u'launch_index': 0,
u'launch_time': u'2012-04-25T20:32:10Z',
u'locked': False,
u'memory_mb': 256,
u'metadata': {},
u'os_type': u'linux',
u'power_state': 0,
u'progress': 0,
u'project_id': u'5820792',
u'ramdisk_id': u'',
u'reservation_id': u'r-j...mm',
u'root_device_name': None,
u'root_gb': 10,
u'user_data': u'',
u'user_id': u'162201',
u'uuid': u'7c210...ed8960',
u'vcpus': 4,
Re: [Openstack] Encrypted virtual machines
As far as storage is concerned, certainly a cloud storage environment could be leveraged to store pre-encrypted data in such a way that would make it difficult bordering on impossible to seize or access without the consent of the owner. As far as compute hosts are concerned, it is a whole different matter. For the foreseeable future ( barring the invention of new widely distributed in CPU technology ) . Anyone with ring 0 execution access on a system ( ie root / sudo ) will be able to pull data from a running instance pretty much no matter what you do. You can certainly raise the bar on difficulty there, but the fundamental path of sniffing schedulers / paging memory / etc will be there for a fairly long time. Even trusted computing wouldn't be applicable to protecting a vm's scheduler from the hypervisors owner. So, I think functionally it should be assumed that a provider will be able to access anything that you access on a hosted VM. As far as a trust relationship goes in elastic computing, there must be an implicit trust of the cloud provider. And as with any trust relationship there is always going to be an element of risk. -Matt On Thu, Apr 26, 2012 at 9:53 AM, Daniel P. Berrange berra...@redhat.com wrote: On Thu, Apr 26, 2012 at 09:05:41AM -0700, Matt Joyce wrote: From a security stand point I am curious what you see the benefit as? Consider that you might have separate people in your data center managing the virtualization hosts, vs the storage hosts vs the network. As it standards today any of those groups of people can compromise data stored in a VM disk image (assuming a network based filesystem). First you encrypt the disk image, so that a person with access to the storage hosts, or network sniffing can't read any data. Then you have a central key server that only gives out the decryption key to Nova compute nodes when they have been explicitly authorized to run an instance of that VM. So now people with access to the storage hosts cannot compromise any data. People with access to the virtualization hosts can only compromise data if the host has been authorized to use that disk image You would need to compromise the precise host the VM disk is being used on, or compromise the key server or the management service that schedules VMs (thus authorizing key usage on a node). NB this is better than relying on the guest OS to do encryption, since you can do stricter decryption key management from the host side. On Thu, Apr 26, 2012 at 8:53 AM, Michael Grosser d...@seetheprogress.net wrote: Hey, I'm following the openstack development for some time now and I was wondering if there was a solution to spin up encrypted virtual machines by default and if it would be a huge performance blow. Any ideas? I would like to extend the libvirt driver in Nova to make use of the qcow2 encryption capabilities between libvirt QEMU which I describe here: http://berrange.com/posts/2009/12/02/using-qcow2-disk-encryption-with-libvirt-in-fedora-12/ Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] New OpenStack Releases in Ubuntu 12.04LTS
+1! I just about fell out of my chair when Mark Shuttleworth mentioned this on stage in front of everyone. This should have a BIG impact on OpenStack adoption. Cheers, Everett On Tue, Apr 24, 2012 at 3:48 AM, Thierry Carrez thie...@openstack.orgwrote: Robbie Williamson wrote: For those of you who may have missed this announcement. Canonical has created the Ubuntu Cloud archive. Starting with the Folsum release, Folsom :) users will be able to elect to enable this archive, and install newer releases of OpenStack (and the dependencies) as they become available up through the next Ubuntu LTS release (presumably 14.04). There was a need for this: people kept asking the OpenStack PPA maintainers to provide a production-grade latest OpenStack on LTS repo. Great to see that the work has been picked up, as an officially-supported option, by the best team for the job ! -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] How does everyone build OpenStack disk images?
On 04/26/2012 04:42 PM, Scott Moser wrote: On Thu, 26 Apr 2012, Jay Pipes wrote: On 04/26/2012 04:19 AM, Daniel P. Berrange wrote: If you have a KVM enabled machine, then 'Oz' has the ability to create JeOS images for all the common distros you'll find. It is a very simple command line tool that just focuses on image building image customization (adding more packages to an existing JeOS image). http://aeolusproject.org/oz.html Interesting, I'll check it out. Yes, it is on the Aeolus project website, but it has no external dependancies on the rest of Aeolus - it just wants kvm, libvirt a few commonly available python modules. I've often thought that it would be desirable to have Oz integrated into OpenStack to provide an native image building capability. Given their common Python heritage I think it would work quite well. Would be interesting to get that integrated into Glance somehow... really? As an extension :) Would be cool to have something that constructs images from some list of inputs and packages. But it would be an extension, not core functionality to Glance... Best, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] How does everyone build OpenStack disk images?
I use vagrant Sent fom my iPone On Apr 25, 2012, at 6:14 PM, Justin Santa Barbara jus...@fathomdb.com wrote: How does everyone build OpenStack disk images? The official documentation describes a manual process (boot VM with ISO), which is sub-optimal in terms of repeatability / automation / etc. I'm hoping we can do better! I posted how I do it on my blog, here: http://blog.justinsb.com/blog/2012/04/25/creating-an-openstack-image/ Please let me know the many ways in which I'm doing it wrong :-) I'm thinking we can have a discussion here, and then I can then compile the responses into a wiki page and/or a nice script... Justin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Encrypted virtual machines
I think that Intel's trusted cloud work is trying to solve that exact compute host problem. It may already have the framework to do so even if the software hasn't caught up (i.e. if we still have some work to do!) It relies on a TPM chip, all code is measured before being run, and then there's a protocol to prove that a system is running that code (remote attestation). If you change the software stack by introducing a sniffer, you change the hash. So we'd need a stack with no root-access / back-doors. Once a back-door becomes known, the hash should no longer be trusted. I'm by no means an expert (I'm still learning about it), but I believe it is possible, having read this paper: http://www.research.ibm.com/trl/projects/watc/FredericStumpfPaper.pdf I'm sure there are still exploits (hardware RAM taps?), and we rely on a total code audit, but we can raise the bar a long way. Anyone from Intel / familiar with Intel's trusted cloud work want to explain better than I can? Justin On Thu, Apr 26, 2012 at 1:44 PM, Matt Joyce m...@nycresistor.com wrote: As far as storage is concerned, certainly a cloud storage environment could be leveraged to store pre-encrypted data in such a way that would make it difficult bordering on impossible to seize or access without the consent of the owner. As far as compute hosts are concerned, it is a whole different matter. For the foreseeable future ( barring the invention of new widely distributed in CPU technology ) . Anyone with ring 0 execution access on a system ( ie root / sudo ) will be able to pull data from a running instance pretty much no matter what you do. You can certainly raise the bar on difficulty there, but the fundamental path of sniffing schedulers / paging memory / etc will be there for a fairly long time. Even trusted computing wouldn't be applicable to protecting a vm's scheduler from the hypervisors owner. So, I think functionally it should be assumed that a provider will be able to access anything that you access on a hosted VM. As far as a trust relationship goes in elastic computing, there must be an implicit trust of the cloud provider. And as with any trust relationship there is always going to be an element of risk. -Matt On Thu, Apr 26, 2012 at 9:53 AM, Daniel P. Berrange berra...@redhat.com wrote: On Thu, Apr 26, 2012 at 09:05:41AM -0700, Matt Joyce wrote: From a security stand point I am curious what you see the benefit as? Consider that you might have separate people in your data center managing the virtualization hosts, vs the storage hosts vs the network. As it standards today any of those groups of people can compromise data stored in a VM disk image (assuming a network based filesystem). First you encrypt the disk image, so that a person with access to the storage hosts, or network sniffing can't read any data. Then you have a central key server that only gives out the decryption key to Nova compute nodes when they have been explicitly authorized to run an instance of that VM. So now people with access to the storage hosts cannot compromise any data. People with access to the virtualization hosts can only compromise data if the host has been authorized to use that disk image You would need to compromise the precise host the VM disk is being used on, or compromise the key server or the management service that schedules VMs (thus authorizing key usage on a node). NB this is better than relying on the guest OS to do encryption, since you can do stricter decryption key management from the host side. On Thu, Apr 26, 2012 at 8:53 AM, Michael Grosser d...@seetheprogress.net wrote: Hey, I'm following the openstack development for some time now and I was wondering if there was a solution to spin up encrypted virtual machines by default and if it would be a huge performance blow. Any ideas? I would like to extend the libvirt driver in Nova to make use of the qcow2 encryption capabilities between libvirt QEMU which I describe here: http://berrange.com/posts/2009/12/02/using-qcow2-disk-encryption-with-libvirt-in-fedora-12/ Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Encrypted virtual machines
Functionally if the scheduler doesn't know what it's passing to the CPU or into paging memory a lot of optimization possibilities go out the window. If it does know one can infer a great deal about your datasets protected or not. -Matt On Thu, Apr 26, 2012 at 3:08 PM, Justin Santa Barbara jus...@fathomdb.com wrote: I think that Intel's trusted cloud work is trying to solve that exact compute host problem. It may already have the framework to do so even if the software hasn't caught up (i.e. if we still have some work to do!) It relies on a TPM chip, all code is measured before being run, and then there's a protocol to prove that a system is running that code (remote attestation). If you change the software stack by introducing a sniffer, you change the hash. So we'd need a stack with no root-access / back-doors. Once a back-door becomes known, the hash should no longer be trusted. I'm by no means an expert (I'm still learning about it), but I believe it is possible, having read this paper: http://www.research.ibm.com/trl/projects/watc/FredericStumpfPaper.pdf I'm sure there are still exploits (hardware RAM taps?), and we rely on a total code audit, but we can raise the bar a long way. Anyone from Intel / familiar with Intel's trusted cloud work want to explain better than I can? Justin On Thu, Apr 26, 2012 at 1:44 PM, Matt Joyce m...@nycresistor.com wrote: As far as storage is concerned, certainly a cloud storage environment could be leveraged to store pre-encrypted data in such a way that would make it difficult bordering on impossible to seize or access without the consent of the owner. As far as compute hosts are concerned, it is a whole different matter. For the foreseeable future ( barring the invention of new widely distributed in CPU technology ) . Anyone with ring 0 execution access on a system ( ie root / sudo ) will be able to pull data from a running instance pretty much no matter what you do. You can certainly raise the bar on difficulty there, but the fundamental path of sniffing schedulers / paging memory / etc will be there for a fairly long time. Even trusted computing wouldn't be applicable to protecting a vm's scheduler from the hypervisors owner. So, I think functionally it should be assumed that a provider will be able to access anything that you access on a hosted VM. As far as a trust relationship goes in elastic computing, there must be an implicit trust of the cloud provider. And as with any trust relationship there is always going to be an element of risk. -Matt On Thu, Apr 26, 2012 at 9:53 AM, Daniel P. Berrange berra...@redhat.com wrote: On Thu, Apr 26, 2012 at 09:05:41AM -0700, Matt Joyce wrote: From a security stand point I am curious what you see the benefit as? Consider that you might have separate people in your data center managing the virtualization hosts, vs the storage hosts vs the network. As it standards today any of those groups of people can compromise data stored in a VM disk image (assuming a network based filesystem). First you encrypt the disk image, so that a person with access to the storage hosts, or network sniffing can't read any data. Then you have a central key server that only gives out the decryption key to Nova compute nodes when they have been explicitly authorized to run an instance of that VM. So now people with access to the storage hosts cannot compromise any data. People with access to the virtualization hosts can only compromise data if the host has been authorized to use that disk image You would need to compromise the precise host the VM disk is being used on, or compromise the key server or the management service that schedules VMs (thus authorizing key usage on a node). NB this is better than relying on the guest OS to do encryption, since you can do stricter decryption key management from the host side. On Thu, Apr 26, 2012 at 8:53 AM, Michael Grosser d...@seetheprogress.net wrote: Hey, I'm following the openstack development for some time now and I was wondering if there was a solution to spin up encrypted virtual machines by default and if it would be a huge performance blow. Any ideas? I would like to extend the libvirt driver in Nova to make use of the qcow2 encryption capabilities between libvirt QEMU which I describe here: http://berrange.com/posts/2009/12/02/using-qcow2-disk-encryption-with-libvirt-in-fedora-12/ Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net
Re: [Openstack] Encrypted virtual machines
I think one of us is misunderstanding the model. My understanding is that we produce software that we trust, and then prove to the caller that we're running that software. All optimizations remain possible. Check out section 6.1 of the paper! On Thu, Apr 26, 2012 at 3:24 PM, Matt Joyce m...@nycresistor.com wrote: Functionally if the scheduler doesn't know what it's passing to the CPU or into paging memory a lot of optimization possibilities go out the window. If it does know one can infer a great deal about your datasets protected or not. -Matt On Thu, Apr 26, 2012 at 3:08 PM, Justin Santa Barbara jus...@fathomdb.com wrote: I think that Intel's trusted cloud work is trying to solve that exact compute host problem. It may already have the framework to do so even if the software hasn't caught up (i.e. if we still have some work to do!) It relies on a TPM chip, all code is measured before being run, and then there's a protocol to prove that a system is running that code (remote attestation). If you change the software stack by introducing a sniffer, you change the hash. So we'd need a stack with no root-access / back-doors. Once a back-door becomes known, the hash should no longer be trusted. I'm by no means an expert (I'm still learning about it), but I believe it is possible, having read this paper: http://www.research.ibm.com/trl/projects/watc/FredericStumpfPaper.pdf I'm sure there are still exploits (hardware RAM taps?), and we rely on a total code audit, but we can raise the bar a long way. Anyone from Intel / familiar with Intel's trusted cloud work want to explain better than I can? Justin On Thu, Apr 26, 2012 at 1:44 PM, Matt Joyce m...@nycresistor.com wrote: As far as storage is concerned, certainly a cloud storage environment could be leveraged to store pre-encrypted data in such a way that would make it difficult bordering on impossible to seize or access without the consent of the owner. As far as compute hosts are concerned, it is a whole different matter. For the foreseeable future ( barring the invention of new widely distributed in CPU technology ) . Anyone with ring 0 execution access on a system ( ie root / sudo ) will be able to pull data from a running instance pretty much no matter what you do. You can certainly raise the bar on difficulty there, but the fundamental path of sniffing schedulers / paging memory / etc will be there for a fairly long time. Even trusted computing wouldn't be applicable to protecting a vm's scheduler from the hypervisors owner. So, I think functionally it should be assumed that a provider will be able to access anything that you access on a hosted VM. As far as a trust relationship goes in elastic computing, there must be an implicit trust of the cloud provider. And as with any trust relationship there is always going to be an element of risk. -Matt On Thu, Apr 26, 2012 at 9:53 AM, Daniel P. Berrange berra...@redhat.com wrote: On Thu, Apr 26, 2012 at 09:05:41AM -0700, Matt Joyce wrote: From a security stand point I am curious what you see the benefit as? Consider that you might have separate people in your data center managing the virtualization hosts, vs the storage hosts vs the network. As it standards today any of those groups of people can compromise data stored in a VM disk image (assuming a network based filesystem). First you encrypt the disk image, so that a person with access to the storage hosts, or network sniffing can't read any data. Then you have a central key server that only gives out the decryption key to Nova compute nodes when they have been explicitly authorized to run an instance of that VM. So now people with access to the storage hosts cannot compromise any data. People with access to the virtualization hosts can only compromise data if the host has been authorized to use that disk image You would need to compromise the precise host the VM disk is being used on, or compromise the key server or the management service that schedules VMs (thus authorizing key usage on a node). NB this is better than relying on the guest OS to do encryption, since you can do stricter decryption key management from the host side. On Thu, Apr 26, 2012 at 8:53 AM, Michael Grosser d...@seetheprogress.net wrote: Hey, I'm following the openstack development for some time now and I was wondering if there was a solution to spin up encrypted virtual machines by default and if it would be a huge performance blow. Any ideas? I would like to extend the libvirt driver in Nova to make use of the qcow2 encryption capabilities between libvirt QEMU which I describe here: http://berrange.com/posts/2009/12/02/using-qcow2-disk-encryption-with-libvirt-in-fedora-12/ Regards, Daniel --
Re: [Openstack] Encrypted virtual machines
Michael, IMO there are several encryption and key management things to consider so it really depends on your needs. If you are looking to allow VM owners to meet data at rest compliance or policies then allow them to manage their own encryption keys and rotation policies then a solution like Justin described encrypting inside the disk image does work and the performance impact is low. You can do some experimentation with ecryptfs and layer that on your existing storage. You can checkout the Ubuntu encrypted home directories as a reference. Now if you are a service provider and would like to disassociate yourself from any subpoenable content that may be stored on your servers, then you may want to do encrypt entire storage by default, then store your encryption keys in keystone maybe. For compute, as Matt mentioned protecting your in-memory data from root or from the hypervisor is not that easy you can make it harder, but there isn't a really good solution today. Longer term trust models that go from metal to hypervisor to tenants using technologies TPM, remote attestation will provide the extra security layers. -Eddie On Apr 26, 2012, at 5:34 PM, Justin Santa Barbara wrote: I think one of us is misunderstanding the model. My understanding is that we produce software that we trust, and then prove to the caller that we're running that software. All optimizations remain possible. Check out section 6.1 of the paper! On Thu, Apr 26, 2012 at 3:24 PM, Matt Joyce m...@nycresistor.com wrote: Functionally if the scheduler doesn't know what it's passing to the CPU or into paging memory a lot of optimization possibilities go out the window. If it does know one can infer a great deal about your datasets protected or not. -Matt On Thu, Apr 26, 2012 at 3:08 PM, Justin Santa Barbara jus...@fathomdb.com wrote: I think that Intel's trusted cloud work is trying to solve that exact compute host problem. It may already have the framework to do so even if the software hasn't caught up (i.e. if we still have some work to do!) It relies on a TPM chip, all code is measured before being run, and then there's a protocol to prove that a system is running that code (remote attestation). If you change the software stack by introducing a sniffer, you change the hash. So we'd need a stack with no root-access / back-doors. Once a back-door becomes known, the hash should no longer be trusted. I'm by no means an expert (I'm still learning about it), but I believe it is possible, having read this paper: http://www.research.ibm.com/trl/projects/watc/FredericStumpfPaper.pdf I'm sure there are still exploits (hardware RAM taps?), and we rely on a total code audit, but we can raise the bar a long way. Anyone from Intel / familiar with Intel's trusted cloud work want to explain better than I can? Justin On Thu, Apr 26, 2012 at 1:44 PM, Matt Joyce m...@nycresistor.com wrote: As far as storage is concerned, certainly a cloud storage environment could be leveraged to store pre-encrypted data in such a way that would make it difficult bordering on impossible to seize or access without the consent of the owner. As far as compute hosts are concerned, it is a whole different matter. For the foreseeable future ( barring the invention of new widely distributed in CPU technology ) . Anyone with ring 0 execution access on a system ( ie root / sudo ) will be able to pull data from a running instance pretty much no matter what you do. You can certainly raise the bar on difficulty there, but the fundamental path of sniffing schedulers / paging memory / etc will be there for a fairly long time. Even trusted computing wouldn't be applicable to protecting a vm's scheduler from the hypervisors owner. So, I think functionally it should be assumed that a provider will be able to access anything that you access on a hosted VM. As far as a trust relationship goes in elastic computing, there must be an implicit trust of the cloud provider. And as with any trust relationship there is always going to be an element of risk. -Matt On Thu, Apr 26, 2012 at 9:53 AM, Daniel P. Berrange berra...@redhat.com wrote: On Thu, Apr 26, 2012 at 09:05:41AM -0700, Matt Joyce wrote: From a security stand point I am curious what you see the benefit as? Consider that you might have separate people in your data center managing the virtualization hosts, vs the storage hosts vs the network. As it standards today any of those groups of people can compromise data stored in a VM disk image (assuming a network based filesystem). First you encrypt the disk image, so that a person with access to the storage hosts, or network sniffing can't read any data. Then you have a central key server that only gives out the decryption key to Nova compute nodes when they have been explicitly authorized to run an
Re: [Openstack] New OpenStack Releases in Ubuntu 12.04LTS
Awsome news! On Thu, Apr 26, 2012 at 10:45 PM, Everett Toews everett.to...@cybera.cawrote: +1! I just about fell out of my chair when Mark Shuttleworth mentioned this on stage in front of everyone. This should have a BIG impact on OpenStack adoption. Cheers, Everett On Tue, Apr 24, 2012 at 3:48 AM, Thierry Carrez thie...@openstack.orgwrote: Robbie Williamson wrote: For those of you who may have missed this announcement. Canonical has created the Ubuntu Cloud archive. Starting with the Folsum release, Folsom :) users will be able to elect to enable this archive, and install newer releases of OpenStack (and the dependencies) as they become available up through the next Ubuntu LTS release (presumably 14.04). There was a need for this: people kept asking the OpenStack PPA maintainers to provide a production-grade latest OpenStack on LTS repo. Great to see that the work has been picked up, as an officially-supported option, by the best team for the job ! -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- --- Luis Alberto Gervaso Martin Woorea Solutions, S.L CEO CTO mobile: (+34) 627983344 luis@ luis.gerv...@gmail.comwoorea.es ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] post-summit editing of etherpads
Hi all, Thanks for an awesome design summit! I've been reviewing the etherpads: http://wiki.openstack.org/FolsomSummitEtherpads and have noticed a few instances of accidental post-summit corruption of etherpad contents, which is not surprising considering there is no difference between view mode and edit mode, and any changes are automatically recorded. Unfortunately there doesn't seem to be a mechanism for freezing the contents to protect against this, although it's possible to click the Save icon at the right of the blue formatting bar, and it will mark the current state as a saved revision retrievable via the 'Saved revisions' or 'Time Slider' tabs at the top. Perhaps it would be worth the session leaders doing this after (optionally ;-) sanity-checking the content? Also, part of the etherpads list was accidentally copy'n'pasted into itself - I just removed the duplicate chunk, taking care not to remove anything which wasn't duplicated. Cheers, Adam ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] post-summit editing of etherpads
A side note: On the DevOpsTeam session etherpad, I've added a new section for post-event additions, comments, etc., hopefully encouraging responsible after-the-fact contributions :-) d On Thu, Apr 26, 2012 at 7:43 PM, Adam Spiers aspi...@suse.com wrote: Hi all, Thanks for an awesome design summit! I've been reviewing the etherpads: http://wiki.openstack.org/FolsomSummitEtherpads and have noticed a few instances of accidental post-summit corruption of etherpad contents, which is not surprising considering there is no difference between view mode and edit mode, and any changes are automatically recorded. Unfortunately there doesn't seem to be a mechanism for freezing the contents to protect against this, although it's possible to click the Save icon at the right of the blue formatting bar, and it will mark the current state as a saved revision retrievable via the 'Saved revisions' or 'Time Slider' tabs at the top. Perhaps it would be worth the session leaders doing this after (optionally ;-) sanity-checking the content? Also, part of the etherpads list was accidentally copy'n'pasted into itself - I just removed the duplicate chunk, taking care not to remove anything which wasn't duplicated. Cheers, Adam ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Unable to download images using Glance+Keystone+Swift
Jay, These are the Ubuntu 12.04 packages from the beta with all known updates. I'm configuring another set of instances with the Ubuntu Precise final packages just to make sure I didn't miss a patch. However, this error seems fundamental to me. I don't see how a glance POST can work but the corresponding GET fails. All calls that just hit the backend DB work fine. Also I can access the bucket and objects directly via swift w no problem. I'll post my results with the final Ubuntu release sometime tomorrow hopefully. (finger tapped on my iPhone) On Apr 26, 2012, at 1:40 PM, Jay Pipes jaypi...@gmail.com wrote: On 04/26/2012 02:37 PM, Lillie Ross-CDSR11 wrote: Hi Jay, Cut and paste error. It still doesn't work. If I issue the simple command (without the pipe or content-type header) I get the following root@essex1:/etc/keystone# curl -v -H 'X-Auth-Token: 45d01460a0e04bff967eb954e7f4fee8' http://essex3:9292/v1/images/423b0ecc-5ca1-44d8-8e85-5a245ce620e2 * About to connect() to essex3 port 9292 (#0) * Trying 172.16.1.5... connected GET /v1/images/423b0ecc-5ca1-44d8-8e85-5a245ce620e2 HTTP/1.1 User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3 Host: essex3:9292 Accept: */* X-Auth-Token: 45d01460a0e04bff967eb954e7f4fee8 HTTP/1.1 404 Not Found Content-Length: 315 Content-Type: text/html; charset=UTF-8 Date: Thu, 26 Apr 2012 18:35:21 GMT html head title404 Not Found/title /head body h1404 Not Found/h1 An object with the specified identifier was not found. Details: Swift could not find image at uri swift+http://service:glance:glance@essex1:5000/v2.0/glance/423b0ecc-5ca1-44d8-8e85-5a245ce620e2br /br / /body * Connection #0 to host essex3 left intact * Closing connection #0 /html root@essex1:/etc/keystone# Now, I can access the image directly via the Swift CLI using my glance tenant, username, and password. However, the Glance REST call fails. All other REST calls work fine. I'm stumped. Ross, what version of Glance and Swift are you using? Best, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Running openstack on top of a non controlled network
Hey, I'm looking into setting up openstack in a minimal production environment with 2 all in one nodes for swift and nova. First I wanted to ask if you would advise against it or if it should be fine? The plan is to have 2 zones with one all in one node for swift and nova each as a starting point and then work from there on (later scale out to more nodes). As an immediate failover solution, I wanna use rackspace. (failover handled via scalr [but not sure yet]) The problems I see are: First useful to have a minimal setup like this? (both nodes have 16gb ram and 8 cores and 4tb diskspace). Third node available with minimal network connectivity/speed. Second, which is probably the essence of this mail. I'm not controlling the network layer. There are no floating IPs, cause IPs are bound to each server via mac address. I can't trust the network traffic. The provider is hetzner.de, if someone is familiar. Is there any solution to that? Perhaps I just missed it in the docs. I thought about using a vpn mesh network to setup a virtual network and running openstack on top. Still the entry points of the vpn would be the bottleneck and the performance would go down a lot I reckon. How would one run an openstack cloud/cluster across more than one datacentre? Is there a way without bridging the datacentres via vpn or direct link? Thanks for any help/hint/idea. Cheers Michael ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Unable to download images using Glance+Keystone+Swift
Hi Ross, I had the same issue. Could upload images to swift but not download them getting a 404. I needed to apply the patch outlined in this bug to fix it: https://bugs.launchpad.net/glance/+bug/979745 Cheers, Sam On Fri, Apr 27, 2012 at 9:53 AM, Lillie Ross-CDSR11 ross.lil...@motorolasolutions.com wrote: Jay, These are the Ubuntu 12.04 packages from the beta with all known updates. I'm configuring another set of instances with the Ubuntu Precise final packages just to make sure I didn't miss a patch. However, this error seems fundamental to me. I don't see how a glance POST can work but the corresponding GET fails. All calls that just hit the backend DB work fine. Also I can access the bucket and objects directly via swift w no problem. I'll post my results with the final Ubuntu release sometime tomorrow hopefully. (finger tapped on my iPhone) On Apr 26, 2012, at 1:40 PM, Jay Pipes jaypi...@gmail.com wrote: On 04/26/2012 02:37 PM, Lillie Ross-CDSR11 wrote: Hi Jay, Cut and paste error. It still doesn't work. If I issue the simple command (without the pipe or content-type header) I get the following root@essex1:/etc/keystone# curl -v -H 'X-Auth-Token: 45d01460a0e04bff967eb954e7f4fee8' http://essex3:9292/v1/images/423b0ecc-5ca1-44d8-8e85-5a245ce620e2 * About to connect() to essex3 port 9292 (#0) * Trying 172.16.1.5... connected GET /v1/images/423b0ecc-5ca1-44d8-8e85-5a245ce620e2 HTTP/1.1 User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3 Host: essex3:9292 Accept: */* X-Auth-Token: 45d01460a0e04bff967eb954e7f4fee8 HTTP/1.1 404 Not Found Content-Length: 315 Content-Type: text/html; charset=UTF-8 Date: Thu, 26 Apr 2012 18:35:21 GMT html head title404 Not Found/title /head body h1404 Not Found/h1 An object with the specified identifier was not found. Details: Swift could not find image at uri swift+http://service:glance:glance@essex1:5000/v2.0/glance/423b0ecc-5ca1-44d8-8e85-5a245ce620e2br /br / /body * Connection #0 to host essex3 left intact * Closing connection #0 /html root@essex1:/etc/keystone# Now, I can access the image directly via the Swift CLI using my glance tenant, username, and password. However, the Glance REST call fails. All other REST calls work fine. I'm stumped. Ross, what version of Glance and Swift are you using? Best, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Error in scheduler when create a new instance.
Hello, Im getting this error when launching a new instance. Failed to schedule_run_instance: No valid host was found. Is the appropriate service running? http://paste.openstack.org/show/14020/ SO Ubuntu 12.04 libvirt-bin start/running, process 4153 nova-network start/running, process 4174 nova-compute start/running, process 4187 nova-api start/running, process 4198 nova-objectstore start/running, process 4210 nova-scheduler start/running, process 4222 nova-volume start/running, process 4235 nova-vncproxy start/running, process 4246 rabbitmq is up too Thanks Rogério Gonçalves ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Integrating keystone for a public cloud panel
Hi everyone, I'm currently building a customer panel to offer public cloud services based on openstack. I'd like to share my plans to validate my approach is good and hear suggestions/feedback from others working on the same kind of project. I'm trying to get the following behaviour: Users register and get access to the panel, I'm keeping user/pass and permissions/groups in my own mysql DB. I wish that the API is not active by default, the user can go to his/her profile and tick a checkbox to get access to the API. For this, I came up with this plan: 1. The user registers, I keep his username/pass in my DB, generate a random hashed keystoneuser/keystonepass and call keystone to create the user/tenant (using keystoneuser as tenant-name). I store this keystone user/pass/tenant info in my DB (which may be a security hole if someone is able to access this DB as the pass is saved as plain text) 2. On user login with his panel credentials, I'll get his keystoneuser/keystonepass to create a token and use this token during his session on the panel. 3. If the user wish to activate access to the API, he'll go to his profile/api page, where he'll see his keystoneuser/tenant name. 3.1. If he/she ticks activate, I'll show him his current keystonepass (from my own DB). 3.2. If he/she ticks deactivate, I'll generate a new random keystonepass, and call keystone to change the password in Openstack. I don't show this password to the user, so he can't use the API anymore, but the panel can get new tokens to continue working. Does this makes sense? Do you guys have any recommendation/suggestion to this implementation? Keep in mind I'm not a python guy, I tried to understand how to write a keystone driver for identity and policy but got lost in the docs/code. Also, is it currently possible to implement a panel like VPS.net where you buy nodes (1 node = 256MB/10GB) and then you launch instances/services based on the number of nodes you have purchased? (And thus get a fixed bill amount each month) Could anybody point me in the right direction to achieve this? Thanks for your help! Adrian Moya ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] Minimum required code coverage per file
On Thu, 2012-04-26 at 11:53 -0700, Joe Gordon wrote: It would nice to initially see the code coverage delta per merge proposal as a comment in gerrit (similar to SmokeStack), and not as a gating factor. +1 Kevin, should we start copying openstack-common tests to client projects? Or just make sure to not count openstack-common code in the code coverage numbers for client projects? That's a tough one. If we copy in the tests, they end up being somewhat redundant, but slow down the project unit tests, but on the other hand, we'd be able to easily demonstrate that that code works properly. I think I'd prefer if we just try to not count openstack-common code for code coverage numbers… (Personally, I would prefer if openstack-common was a library, rather than copying its code into the client project, but I am not familiar with the arguments for why it was decided to do the copy, and I'm not really involved in openstack-common development at the moment…) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Error in scheduler when create a new instance.
be sure nova-compute is running and is enabled by using nova-manage service list,also confirm that the compute node have enough resources(cpu,mem,disk) . 2012/4/27 Rogerio Goncalves roge...@gmail.com: Hello, Im getting this error when launching a new instance. Failed to schedule_run_instance: No valid host was found. Is the appropriate service running? http://paste.openstack.org/show/14020/ SO Ubuntu 12.04 libvirt-bin start/running, process 4153 nova-network start/running, process 4174 nova-compute start/running, process 4187 nova-api start/running, process 4198 nova-objectstore start/running, process 4210 nova-scheduler start/running, process 4222 nova-volume start/running, process 4235 nova-vncproxy start/running, process 4246 rabbitmq is up too Thanks Rogério Gonçalves ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Integrating keystone for a public cloud panel
Adrian, Rather than managing the user's passwords externally to keystone, you can allow your users to define their own passwords, and instead create your users and/or tenants in a disabled state, by toggling their enabled/disabled state within keystone (an attribute available for both through the API). -Dolph Mathews On Apr 26, 2012, at 7:58 PM, Adrian Moya adrianm...@gmail.com wrote: Hi everyone, I'm currently building a customer panel to offer public cloud services based on openstack. I'd like to share my plans to validate my approach is good and hear suggestions/feedback from others working on the same kind of project. I'm trying to get the following behaviour: Users register and get access to the panel, I'm keeping user/pass and permissions/groups in my own mysql DB. I wish that the API is not active by default, the user can go to his/her profile and tick a checkbox to get access to the API. For this, I came up with this plan: 1. The user registers, I keep his username/pass in my DB, generate a random hashed keystoneuser/keystonepass and call keystone to create the user/tenant (using keystoneuser as tenant-name). I store this keystone user/pass/tenant info in my DB (which may be a security hole if someone is able to access this DB as the pass is saved as plain text) 2. On user login with his panel credentials, I'll get his keystoneuser/keystonepass to create a token and use this token during his session on the panel. 3. If the user wish to activate access to the API, he'll go to his profile/api page, where he'll see his keystoneuser/tenant name. 3.1. If he/she ticks activate, I'll show him his current keystonepass (from my own DB). 3.2. If he/she ticks deactivate, I'll generate a new random keystonepass, and call keystone to change the password in Openstack. I don't show this password to the user, so he can't use the API anymore, but the panel can get new tokens to continue working. Does this makes sense? Do you guys have any recommendation/suggestion to this implementation? Keep in mind I'm not a python guy, I tried to understand how to write a keystone driver for identity and policy but got lost in the docs/code. Also, is it currently possible to implement a panel like VPS.net where you buy nodes (1 node = 256MB/10GB) and then you launch instances/services based on the number of nodes you have purchased? (And thus get a fixed bill amount each month) Could anybody point me in the right direction to achieve this? Thanks for your help! Adrian Moya ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] How to let Flat Networking wok in Essex?
HI Now I can config FlatHHCP mode and working. but the Flat netwok can not working . who can share the the config 1: /etc/network/interface 2: /etc/nova/nova.conf 3: how to create private network fro vm -- Shake Chen ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Quantum plugins
Dan, Salavatore and others, your input is sought here. Can any one provide little explanation please? From: salma...@live.com To: openstack@lists.launchpad.net Subject: OpenStack Quantum plugins Date: Thu, 26 Apr 2012 19:55:20 + Hi All, I am trying to learn the functionality of Quantum plugins used in OpenStack. I have read through the Quantum Admin Guide and had few basic/quick question about quantum and OVS interaction with it: 1) OVS can have ports in which vNICS can be plugged, so why does it need to use an integration bridge for connecting all VMs on the same node to a network? 2) The OVS quantum plugin seems to implement the core API functions and (viewing the code) I concluded that it just makes maintains the logical mappings e.g. b/w net IDs and VLAN IDs in a database. So how is this mapping implemented on the actual ports of OVS? Is it the OVS quantum agent responsible for directing the packets to correct input/output ports based on the updates that it gets from the database? 3) The quantum admin guide says that the nova client will be the main user of quantum and will interact with it via REST API, so it would be nice if someone can point me to the code (file path name etc.) where this happens. Thanks, Salman PS: What is the purpose of Quantum Manager in this architecture and where should I look for its code? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack-poc] [Bug 983734] Re: Keystone fails badly if you miss one option
So we have to require either developer or user to specify every option. My change follows exactly this logic. The separate issue I see with this '/etc' defaults. But even if we keep them, we should enforce developers to provide default values or require user to do it. -- You received this bug notification because you are a member of OpenStack Common Drivers, which is the registrant for openstack-common. https://bugs.launchpad.net/bugs/983734 Title: Keystone fails badly if you miss one option Status in OpenStack Identity (Keystone): Confirmed Status in openstack-common: Invalid Bug description: If you misspell or forget one option in keystone.conf (like template_file for TemplatedCatalog backend), Keystone will fail with misguiding critical failure (in my case, TypeError: coercing to Unicode: need string or buffer, NoneType found). To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/983734/+subscriptions ___ Mailing list: https://launchpad.net/~openstack-poc Post to : openstack-poc@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack-poc More help : https://help.launchpad.net/ListHelp
