Re: [Openstack-poc] [OpenStack Foundation] Technical Committee: reserved seats for PTLs (or not)

2012-06-22 Thread Justin Santa Barbara
that the two be linked. In particular, we shouldn't exclude someone from being a PTL because they wouldn't be a good CTO. There are also other issues with PTL+5 IMHO (e.g incentivizing the Balkanization of OpenStack), but at core I see the two roles as being very different. Justin --- Justin Santa

Re: [Openstack-poc] [OpenStack Foundation] Technical Committee: reserved seats for PTLs (or not)

2012-06-22 Thread Justin Santa Barbara
comprised merely of yes-men for the existing groups. Justin --- Justin Santa Barbara Founder, FathomDB On Fri, Jun 22, 2012 at 12:12 PM, Joe Heck joe.h...@nebula.com wrote: On Jun 22, 2012, at 11:53 AM, Justin Santa Barbara wrote: In my mind, the PTL is responsible for moving

Re: [Openstack] ZFS/ZVol + iscsi for volume

2012-06-12 Thread Justin Santa Barbara
As Diego pointed out, this should all work already. You just point your nova-volume at your Solaris-like box, and it runs all the commands for you (over SSH). I wrote the original way-back-when as a stepping-stone to support for HP SANs (as I had much easier access to Solaris than real SANs),

Re: [Openstack] Encrypted virtual machines

2012-04-26 Thread Justin Santa Barbara
On Thu, Apr 26, 2012 at 9:05 AM, Matt Joyce m...@nycresistor.com wrote: From a security stand point I am curious what you see the benefit as? I think that long-term there is the potential to have a cloud where you don't have to trust the cloud provider (e.g. Intel Trusted Compute). However,

Re: [Openstack] Encrypted virtual machines

2012-04-26 Thread Justin Santa Barbara
I think that Intel's trusted cloud work is trying to solve that exact compute host problem. It may already have the framework to do so even if the software hasn't caught up (i.e. if we still have some work to do!) It relies on a TPM chip, all code is measured before being run, and then there's a

Re: [Openstack] Encrypted virtual machines

2012-04-26 Thread Justin Santa Barbara
, Justin Santa Barbara jus...@fathomdb.com wrote: I think that Intel's trusted cloud work is trying to solve that exact compute host problem. It may already have the framework to do so even if the software hasn't caught up (i.e. if we still have some work to do!) It relies on a TPM chip

[Openstack] Public image repository via a synchronization script

2012-04-25 Thread Justin Santa Barbara
http://images.platformlayer.org/md5sum ... Justin --- Justin Santa Barbara Founder, FathomDB ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help

Re: [Openstack] File injection and disk resize not working

2012-04-25 Thread Justin Santa Barbara
I believe config-drive is the way to fix injection problems. I use it and it works great. (I just wish there was a way to use it from Horizon...) We'll never be able to inject into every image; also some people don't want us messing with their images. Config drive is effectively injection into

Re: [Openstack] [OpenStack][Nova] Minimum required code coverage per file

2012-04-25 Thread Justin Santa Barbara
If you let me know in a bit more detail what you're looking for, I can probably whip something up. Email me direct? Justin On Wed, Apr 25, 2012 at 6:59 AM, Monty Taylor mord...@inaugust.com wrote: On 04/24/2012 10:08 PM, Lorin Hochstein wrote: On Apr 24, 2012, at 4:11 PM, Joe Gordon

Re: [Openstack] [OpenStack][Nova] Minimum required code coverage per file

2012-04-25 Thread Justin Santa Barbara
to 80 and then set the threshold for 80? Also, turns out we're not running this on the client libs... Monty On 04/25/2012 03:53 PM, Justin Santa Barbara wrote: If you let me know in a bit more detail what you're looking for, I can probably whip something up. Email me direct? Justin

[Openstack] How does everyone build OpenStack disk images?

2012-04-25 Thread Justin Santa Barbara
How does everyone build OpenStack disk images? The official documentation describes a manual process (boot VM with ISO), which is sub-optimal in terms of repeatability / automation / etc. I'm hoping we can do better! I posted how I do it on my blog, here:

Re: [Openstack] Canonical AWSOME

2012-04-24 Thread Justin Santa Barbara
If EC2 API is limiting what we can do, that's not going to change just because you move the EC2 API implementation into a proxy in front of the OpenStack API. The only difference is that it's suddenly the AWSOME development team's problem. I think it's actually the EC2 API caller's problem.

Re: [Openstack] Canonical AWSOME

2012-04-24 Thread Justin Santa Barbara
Thanks for the pointer. I found the etherpad: http://etherpad.openstack.org/VersioningNovaRPCAPIs Is there a blueprint that came / is coming out of that? I think the data representation is orthogonal e.g. in theory, we could even use XML schemas: PyDict -- XML -- XML Schema Validation -- Warn /

Re: [Openstack] Canonical AWSOME

2012-04-23 Thread Justin Santa Barbara
What's the advantage of replacing the native EC2 compatibility layer with AWSOME from a user / operator point of view? Although I wasn't able to attend the design summit session, right now we have two native APIs, which means we have two paths into the system. That is poor software

Re: [Openstack] Just JSON, and extensibility

2012-04-14 Thread Justin Santa Barbara
- Each (known) extension has its own strongly-typed model object. Does that mean that an extension cannot add properties to an existing object (such as adding a new attribute an Image), or just that all of those properties will be an a nested object (such as

Re: [Openstack] Just JSON, and extensibility

2012-04-13 Thread Justin Santa Barbara
It's easy when each new version is defined by a central body. The problem we face is that we want to allow HP, Rackspace, Nexenta etc to define their own extensions, without serializing through a central body. Some extensions may only apply to private clouds and never be shared publicly. This

Re: [Openstack] profiling nova-api

2012-04-12 Thread Justin Santa Barbara
Both the profiling the multi-process work are great - good stuff! Jay: Won't going multi-process just make it harder to profile? I think it's actually a good thing to profile without the multi-process patch, find fix and the bottlenecks in single-request performance, and then use multi-process

Re: [Openstack] minimal IaaS openstack installation FROM SOURCE on CentOS

2012-04-12 Thread Justin Santa Barbara
I have it running from source in a more-production-environment than devstack, though on Debian... https://github.com/justinsb/openstack-simple-config Would you like to collaborate on fixing this up for CentOS the Essex release? It was working a week or two before feature freeze, and post

Re: [Openstack] [SWIFT] Looking for an approach to attach an account or a container in operating system . Like a NAS of SAN driver.

2012-04-12 Thread Justin Santa Barbara
I made some patches to cloudfuse to get it to support Keystone auth: http://blog.justinsb.com/blog/2012/03/29/openstack-storage-with-fuse/ I'd like to get this merged upstream, but haven't heard anything on my pull request... Redbo? Bueller? :-) I also saw an SFTP gateway floating about

Re: [Openstack] Image API v2 Draft 4

2012-04-12 Thread Justin Santa Barbara
I may disagree with a couple of the points along the way; but I agree with the conclusion for OpenStack. Thanks for writing it! Justin PS vim or emacs? On Thu, Apr 12, 2012 at 12:58 PM, Mark Nottingham m...@mnot.net wrote: A little fuel for the fire / entertainment before the summit:

Re: [Openstack] Image API v2 Draft 4

2012-04-10 Thread Justin Santa Barbara
it to openstack-common before bringing the code into Glance). Best, -jay On 04/10/2012 06:51 AM, Doug Hellmann wrote: On Mon, Apr 9, 2012 at 5:14 PM, Justin Santa Barbara jus...@fathomdb.com mailto:jus...@fathomdb.com wrote: When you're designing JSON considering only JSON

Re: [Openstack] Metadata and File Injection (code summit session?)

2012-04-10 Thread Justin Santa Barbara
I think config drive is injection done right; it doesn't need to figure out a partition because it creates its own. Config drive + the init script I've contributed is rock-solid for me. No metadata service to go slow or need special configuration, no DHCP problems etc. If there's a

Re: [Openstack] Announcing project Heat

2012-04-10 Thread Justin Santa Barbara
Congratulations on your project! Just to clarify, PlatformLayer can run anything as a service, not just databases. I added support for running memcache as a service last night, and Zookeeper clusters on Sunday... I think RedDwarf is also moving away from just supporting DBs. A crowded space

Re: [Openstack] Agreeing a common set of Image Properties

2012-04-10 Thread Justin Santa Barbara
Signing would definitely be a great v2 feature. For v1, we just need some way to know that an image is provided by the cloud provider, and some idea of what that image is. I believe every cloud is stuck respinning their own images, because we haven't been able to agree a golden image standard.

Re: [Openstack] Image API v2 Draft 4

2012-04-10 Thread Justin Santa Barbara
It definitely has improved - thank you for all your work; I didn't mean to put down anyone's work here. It's simply a Sisyphean task. Either way, though, if I had the choice, I'd rip all of nova's XML support out tomorrow… As a strong supporter of XML, who thinks JSON is for kids that

Re: [Openstack] Metadata and File Injection (code summit session?)

2012-04-10 Thread Justin Santa Barbara
Config drive can support all EC2 functionality, I believe. Images would need to be respun for OpenStack with config-drive, unless we populated the config drive in a way that worked with cloud-init. (Scott?) Personally, I'd rather our effort went into producing great images for OpenStack, than

Re: [Openstack] Metadata and File Injection (code summit session?)

2012-04-10 Thread Justin Santa Barbara
I presume you're looking for more than SSH offers. What are the missing features you need? On Tue, Apr 10, 2012 at 12:36 PM, Andrew Bogott abog...@wikimedia.orgwrote: What I crave is a communication channel between nova and running instances. There was discussion at some point about

Re: [Openstack] Metadata and File Injection (code summit session?)

2012-04-10 Thread Justin Santa Barbara
The immediate use case I have in mind is to support this: http://wiki.openstack.org/**PackageConfigForNovahttp://wiki.openstack.org/PackageConfigForNova. That design requires periodic checkins between an instance agent and a nova driver. It certainly /could/ be implemented using ssh, but I

Re: [Openstack] Metadata and File Injection (code summit session?)

2012-04-10 Thread Justin Santa Barbara
One advantage of a network metadata channel is it allows for communication with cloud provider services without having to put a key into the vm. In other words, the vm can be authenticated via its ipv6 address. Did you have a use case in mind here? It seems that Keystone could use the IPV6

[Openstack] Metadata and File Injection (code summit session?)

2012-04-10 Thread Justin Santa Barbara
...@gmail.comwrote: On Apr 10, 2012, at 4:24 PM, Justin Santa Barbara wrote: One advantage of a network metadata channel is it allows for communication with cloud provider services without having to put a key into the vm. In other words, the vm can be authenticated via its ipv6 address. Did

Re: [Openstack] Agreeing a common set of Image Properties

2012-04-09 Thread Justin Santa Barbara
On Mon, Apr 9, 2012 at 8:18 AM, Doug Hellmann doug.hellm...@dreamhost.comwrote: I'm thinking of the os prefix as standing for OpenStack, not Operating System. I would never have guessed that from the context. Why OpenStack instead of Operating System? We reserve this limited region of the

Re: [Openstack] Image API v2 Draft 4

2012-04-09 Thread Justin Santa Barbara
APPENDIX B: Outstanding issues 4) Need to write xsds :( This is easy if you design a model which works with XML. If you have an XML compatible model, you can generate an XSD and a JSON model from that. Also, it means you can just use common middleware to map XML to JSON, rather than coding

Re: [Openstack] Image API v2 Draft 4

2012-04-09 Thread Justin Santa Barbara
Justin, what does design a model which works with XML mean? Simply avoiding the handful of things that are specific to JSON (or specific to XML). Nothing too onerous (no angle brackets)! I think this is only done in the image properties. No, the image properties have been removed in the

Re: [Openstack] Agreeing a common set of Image Properties

2012-04-09 Thread Justin Santa Barbara
I should probably clarify my terminology a little here, as I may have mangled it: - I'm talking about additional/extension properties, not properties that are well-known to Glance - However, we agree to use a common set of properties to mean certain things. In other words, we all

Re: [Openstack] Image API v2 Draft 4

2012-04-09 Thread Justin Santa Barbara
When you're designing JSON considering only JSON, you'd probably use { key1: value1 } - as you have done. If you're designing generically, you'd probably use { key: key1, value: value1 }. You mean we'd have to do dumb crap because XML doesn't have the native concept of a list? ;) XML

Re: [Openstack] Image API v2 Draft 4

2012-04-09 Thread Justin Santa Barbara
How about we discuss this further at the summit :-) I think that's a sensible proposal. We're not likely to reach a good conclusion here. I think my viewpoint is that even json-dressed-as-xml is fine; no end-user gives two hoots what our JSON/XML/HPSTR looks like. I'd wager most users of

Re: [Openstack] Image API v2 Draft 4

2012-04-09 Thread Justin Santa Barbara
Extensible lists are pointless. Lists have no attributes other than their length. I made this point a couple design summits ago... but whatever :) Looks like the Sapir-Whorf hypothesis might be true after all ;-) Let's dust off the pugil-sticks for the design summit..

Re: [Openstack] Jcloud and openstack relation

2012-04-08 Thread Justin Santa Barbara
There is also a (friendly rival) OpenStack Java binding being developed: https://github.com/platformlayer/openstack-java https://github.com/woorea/openstack-java-sdk/ That library supports a direct-to-OpenStack Jenkins plugin which I confidently predict will rapidly surpass any

[Openstack] Agreeing a common set of Image Properties

2012-04-07 Thread Justin Santa Barbara
and install updates myself, in case I don't want a particular update. Providing a fast apt cache is much better than providing respun images, for my use-case. I would be great if old images stuck around, therefore! Justin --- Justin Santa Barbara Founder, FathomDB

Re: [Openstack] Agreeing a common set of Image Properties

2012-04-07 Thread Justin Santa Barbara
like managed service level for an image that might be the same base distro as an unmanaged image. Sent from my iPhone On Apr 7, 2012, at 5:14 PM, Justin Santa Barbara jus...@fathomdb.com wrote: Is there a (de-facto) standard for image metadata/properties? I'd like to be able to able

Re: [Openstack] Agreeing a common set of Image Properties

2012-04-07 Thread Justin Santa Barbara
! Justin 2012/4/7 Nathanael Burton nathanael.i.bur...@gmail.com Looks like Pádraig and I were thinking alike. On Apr 7, 2012 8:49 PM, Pádraig Brady p...@draigbrady.com wrote: On 04/07/2012 11:13 PM, Justin Santa Barbara wrote: Is there a (de-facto) standard for image metadata/properties? I'd

[Openstack] Introducing PlatformLayer: Everything-as-a-service, for OpenStack

2012-04-06 Thread Justin Santa Barbara
-as-a-service?) Read more: http://blog.justinsb.com/blog/2012/04/06/introducing-platformlayer/ Discussion on hacker news: http://news.ycombinator.com/item?id=3809221 Please contact me if you're interested in using it or helping out! Thanks, Justin --- Justin Santa Barbara Founder, FathomDB

Re: [Openstack] Introducing PlatformLayer: Everything-as-a-service, for OpenStack

2012-04-06 Thread Justin Santa Barbara
show you the head-start PlatformLayer gives you; in return I'll get to see more of the areas where it is lacking! Justin On Fri, Apr 6, 2012 at 4:04 PM, Mark Collier mark.coll...@rackspace.com wrote: Justin-as-a-Service? Justin Santa Barbara jus...@fathomdb.com wrote: FathomDB has just

Re: [Openstack] OpenStack Plugin for Jenkins

2012-04-05 Thread Justin Santa Barbara
I've got Compute functionality working with the OpenStack Jenkins plugin, so it can launch nova instances as on-demand slaves now, run builds on them, and archive the results into swift. I'd like to open GitHub issues to track your requirements, but I have a few questions. We need disposable

[Openstack] Java 7 File Provider for OpenStack Storage

2012-04-05 Thread Justin Santa Barbara
bugs you find! There's almost no documentation out there on how to do this, so I don't think any of the legacy clouds support this. Another small step forward for OpenStack! Justin --- Justin Santa Barbara Founder, FathomDB ___ Mailing list: https

[Openstack] OpenStack Plugin for Jenkins

2012-04-04 Thread Justin Santa Barbara
copying artifacts, that just seems wrong to me, so I'd like to just fix it. Justin --- Justin Santa Barbara Founder, FathomDB ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net

Re: [Openstack] OpenStack Plugin for Jenkins

2012-04-04 Thread Justin Santa Barbara
, Justin Santa Barbara wrote: I've created a quick OpenStack plugin for Jenkins, using the Java bindings that Luis I created. It's available on github here: https://github.com/platformlayer/openstack-jenkins (no binaries - yet!) Whilst it is cool that there is effort going

Re: [Openstack] OpenStack Plugin for Jenkins

2012-04-04 Thread Justin Santa Barbara
Thanks - there are some good feature requests here. Have you proposed a design summit session where you are going to talk about OpenStack features that you would like to see? Things like supporting cloud federation, instance pooling, or fixing that networking bug? The Jenkins features (e.g.

Re: [Openstack] Performance diagnosis of metadata query

2012-03-29 Thread Justin Santa Barbara
Is there a good way to map back where in the code these calls are coming from? There's not a great way currently. I'm trying to get a patch in for Essex which will let deployments easily turn on SQL debugging (though this is proving contentious); it will have a configurable log level to

Re: [Openstack] Performance diagnosis of metadata query

2012-03-29 Thread Justin Santa Barbara
, 2012 at 10:22 AM, David Kranz david.kr...@qrclab.comwrote: On 3/29/2012 12:46 PM, Justin Santa Barbara wrote: Is there a good way to map back where in the code these calls are coming from? There's not a great way currently. I'm trying to get a patch in for Essex which will let deployments

Re: [Openstack] Performance diagnosis of metadata query

2012-03-29 Thread Justin Santa Barbara
for critical bugs in a week or so from release. I am not saying this is great, but if release dates are fixed and features, performance the things that are allowed to vary, then what else is there to do? Just my opinion. -David On 3/29/2012 1:55 PM, Justin Santa Barbara wrote: I'm

Re: [Openstack] KVM crash.

2012-03-28 Thread Justin Santa Barbara
The libvirt hang was a threading deadlock within libvirt, whereas this is a kernel crash. I'd be surprised if they are the same issue. The current theory on the kernel bugzilla is that it might be related to bridge + netfilter. That does tally with the observation of it happening under a

[Openstack] Performance diagnosis of metadata query

2012-03-25 Thread Justin Santa Barbara
The performance of the metadata query with cloud-init has been causing some people problems (it's so slow cloud-init times out!), and has led to the suggestion that we need lots of caching. (My hypothesis is that we don't...) By turning on SQL debugging in SQL Alchemy (for which I've proposed a

Re: [Openstack] Caching strategies in Nova ...

2012-03-23 Thread Justin Santa Barbara
This is great: hard numbers are exactly what we need. I would love to see a statement-by-statement SQL log with timings from someone that has a performance issue. I'm happy to look into any DB problems that demonstrates. The nova database is small enough that it should always be in-memory (if

Re: [Openstack] Please stop the devstack non-sense!

2012-03-20 Thread Justin Santa Barbara
Hi Thomas, I think devstack has done a lot for the developer's use-case, but I believe we should also have a official / semi-official project that does some sort of packaging to help the production use-case. I've proposed a summit discussion: http://summit.openstack.org/sessions/view/26 The

Re: [Openstack] Essex keystone with remote glance endpoint

2012-03-19 Thread Justin Santa Barbara
# glance -I adminUser -K ... -S keystone -N http://192.168.131.141:5000/v2.0' index Failed to show index. Got error: Response from Keystone does not contain a Glance endpoint. I think that means that the glance client can't find a suitable glance endpoint in the response from Keystone.

Re: [Openstack] [NOVA] Snapshotting may require significant disk space (in /tmp). How to properly solve disk space issues?

2012-03-16 Thread Justin Santa Barbara
We're creating a (huge) temp file, uploading it, and then deleting it. So really we should be streaming the snapshot direct to the destination (glance?) Checking the code, we are writing it sequentially (particularly if we're writing in raw): https://github.com/qemu/QEMU/blob/master/qemu-img.c

[Openstack] Networking guru needed: problem with FlatManager ARP when guest and bridge MACs the same

2012-03-14 Thread Justin Santa Barbara
We recently changed the MAC address assigned to guests so that they started with 0xfe, in the hope of avoiding (theoretical?) issues with MAC addresses changing on the bridge device as machines are shut down (because supposedly the bridge grabs the lowest MAC address numerically):

Re: [Openstack] Random libvirt hangs

2012-03-13 Thread Justin Santa Barbara
Santa Barbara wrote: Which operating system(s) are we aiming to support for Essex?  Is the plan to backport the latest libvirt to Oneric, or are we going to wait for Precise? The question is the other way around: which operating systems aim to support Essex ? We try to set the dependencies

Re: [Openstack] Random libvirt hangs

2012-03-12 Thread Justin Santa Barbara
I just today was able to diagnose a libvirt hang. It appears to be (similar to) a known bug in libvirt, likely fixed in the latest Fedora, but it does not appear to be fixed in Ubuntu Oneirc; I think the fix is in Precise: https://bugs.launchpad.net/nova/+bug/953656 I believe this is the

Re: [Openstack] Libvirt Snapshots

2012-03-09 Thread Justin Santa Barbara
Thanks for the background. My thoughts: * Telling a user to build from source isn't a great option for them - it's painful, they don't get updates automatically etc. Are we going to start distributing packages again? * I can't imagine any open source project removing functionality like the

Re: [Openstack] Basic networking/configuration woes

2012-02-24 Thread Justin Santa Barbara
The instance firewall should be configured to only allow DHCP responses from the IP it believes to be the correct DHCP server. Perhaps it has the wrong idea? Ah yes - it probably does because of my unusual network subrange config. It seems I'm not missing something, so I've proposed a init

[Openstack] Basic networking/configuration woes

2012-02-23 Thread Justin Santa Barbara
I'm trying to use OpenStack in what I think to be the typical non-public-cloud deployment, and my experience is not what it could/should be.  I'm hoping someone can point me to the right way, or we can figure out what needs to change. My wishlist: * I want my instances to be on my network e.g.

Re: [Openstack] Basic networking/configuration woes

2012-02-23 Thread Justin Santa Barbara
which is in the 169.254.* range. Not sure if that helped much :) - Chris On Feb 23, 2012, at 3:12 PM, Justin Santa Barbara wrote: I'm trying to use OpenStack in what I think to be the typical non-public-cloud deployment, and my experience is not what it could/should be.  I'm hoping

Re: [Openstack] Basic networking/configuration woes

2012-02-23 Thread Justin Santa Barbara
If you're going to go the cloud-init route... you wouldn't need DHCP, right?   There should be iptables rules to allow you to talk to the metadata service over 169.254.*  (And linux should give you a default link-local address that allows you to talk to the MD service magically) Do you

Re: [Openstack] DeltaCloud

2012-02-21 Thread Justin Santa Barbara
In my experience, the API is the most visible yet smallest problem of working with different clouds. For example, EC2 and Rackspace Cloud have completely different approaches to volumes, such that the way you backup your VMs has to be completely different (disk snapshot vs application level).

Re: [Openstack] Getting pagination right

2011-05-25 Thread Justin Santa Barbara
I definitely agree that the paging logic is questionable; we definitely should be specifying the sort order if we're relying on it, as I believe a database is otherwise free to return results however it sees fit. I'm not convinced that the proposed logic around the queries and update timestamps

Re: [Openstack] Proposal for Justin Santa Barbara to join Nova-Core

2011-04-15 Thread Justin Santa Barbara
Great news - thank you all! Justin On Fri, Apr 15, 2011 at 1:24 AM, Soren Hansen so...@openstack.org wrote: 2011/4/12 Soren Hansen so...@openstack.org: +1 from me, too. As per the process, if no-one objects within 5 business (my interpretation) days, i.e. before Thursday, I'll get Justin

Re: [Openstack] Summit Talk: Information session on Zones? Any interest?

2011-04-14 Thread Justin Santa Barbara
I'm not sure what you mean by an 'informational session'? That sounds rather didactic - is everything sufficiently agreed for a session of this type? I was under the impression that a lot of the details of zones were not yet agreed, and that a lot of the use cases we would like to see in future

[Openstack] Proposal to defer existing OS API v1.1 to Diablo, for greater stability in Cactus

2011-03-30 Thread Justin Santa Barbara
I have been doing some work on getting XML support in our APIs in better shape in the OS API. However we feel about XML, the big benefit is that it gives us really strict validation for free. I am only at the point of adding namespace declarations, but this has already uncovered a lot of issues

Re: [Openstack] Feature Freeze status

2011-03-28 Thread Justin Santa Barbara
No objection to a discussion during the summit, but I've been able to watch all of these branches and others evolve here: https://code.launchpad.net/nova For example, when I wanted to add VNC support because my system wouldn't boot, it was easy for me to look at the vnc_console branch and get the

Re: [Openstack] Feature Freeze status

2011-03-28 Thread Justin Santa Barbara
Rubbish. Open development means knowing the general directions and specifications that people are working on by open discussions, open blueprints/specs, and active communication between teams. I can go to github and see how many people forked this (ugh.). That doesn't give me any clue as to

Re: [Openstack] Instance IDs and Multiple Zones

2011-03-23 Thread Justin Santa Barbara
So I'm going to implement a partition of 1 billion integers per zone, which should allow for approximately 1 billion zones, given a 64 bit integer for the PK. This should be workable for now, and after the design summit, when we've come to a consensus on changing the API to accept something

Re: [Openstack] Instance IDs and Multiple Zones

2011-03-22 Thread Justin Santa Barbara
I think _if_ we want to stick with straight numbers, the following are the 'traditional' choices: 1) Skipping - so zone1 would allocate numbers 1,3,5, zone2 numbers 2,4,6. Requires that you know in advance how many zones there are. 2) Prefixing - so zone0 would get 0xxx, zone1 1xx. 3)

Re: [Openstack] Instance IDs and Multiple Zones

2011-03-22 Thread Justin Santa Barbara
Totally agree with Eric. Two questions that I think can help us move forward: 1. Is the decision to stick with integers still valid? Can someone that was there give us the reason for the decision? Is it documented anywhere? 2. If we must have integers means that we get 128 bit

Re: [Openstack] Instance IDs and Multiple Zones

2011-03-22 Thread Justin Santa Barbara
for ec2 style ids to be assigned to instances, perhaps through a central authority that hands out unique ids. Vish On Mar 22, 2011, at 12:30 PM, Justin Santa Barbara wrote: The API spec doesn't seem to preclude us from doing a fully-synchronous method if we want to (it just reserves the option

Re: [Openstack] Openstack API - Volumes?

2011-03-22 Thread Justin Santa Barbara
...@gmail.com wrote: On Tue, Mar 22, 2011 at 3:21 PM, Justin Santa Barbara jus...@fathomdb.com wrote: So: When can we expect volume support in nova? If I repackaged my volumes API as an extension, can we get it merged into Cactus? I would personally support this. Wasn't one of the ideas

Re: [Openstack] Instance IDs and Multiple Zones

2011-03-22 Thread Justin Santa Barbara
Pure numeric ids will not work in a federated model at scale. Agreed Maybe I'm missing something, but I don't see how you could inject a collision ID downstream - you can just shoot yourself in your own foot. I think that you can get away with it only in simple hierarchical structures.

Re: [Openstack] GLANCE: How to deal with SQLAlchemy-migrate problems

2011-03-21 Thread Justin Santa Barbara
Do we have patches for approach #3? I'm thinking that if we have a few specific issues, that maybe we could maintain our own branch and/or monkey-patch SA-migrate, and get a better handle on the problem, and the upstream's project receptiveness to patches (rather than bug reports). It's a bit

Re: [Openstack] A single cross-zone database?

2011-03-16 Thread Justin Santa Barbara
Thanks for raising this Sandy: +1 on keeping separate DBs until a problem arises. I don't see a performance problem with recursively querying child zones. I guess this will partially depend on our zone topology: if the intent is to have child zones that are geographically distributed where the

Re: [Openstack] A single cross-zone database?

2011-03-16 Thread Justin Santa Barbara
Good point that pagination makes this harder. However, thankfully the limit is implemented using a token (the last ID seen), not an absolute offset, so I believe we can still do pagination even in loosely coordinated DBs. Good job whoever dodged that bullet (Jorge?) (Aside #1: Sorting by uptime

Re: [Openstack] A single cross-zone database?

2011-03-16 Thread Justin Santa Barbara
--- Justin Santa Barbara Founder, FathomDB On Wed, Mar 16, 2011 at 11:14 AM, Andrew Shafer and...@cloudscaling.comwrote: Global temporal consistency is a myth. If you decide not to cache and support pagination then querying every zone for every page is potentially as misleading

Re: [Openstack] A single cross-zone database?

2011-03-16 Thread Justin Santa Barbara
Inline... Can someone explain _why_ we need caching? We don't *need* caching - it is simply the most direct way to avoid multiple expensive calls. So if we don't need it...? You cite avoiding expensive calls, but I think it's entirely unproven that those call are too expensive. If

Re: [Openstack] A single cross-zone database?

2011-03-16 Thread Justin Santa Barbara
I agree that we could have a better marker, but I'm just going off the spec at the moment. I've checked the agreed blueprint, and caching in zones is out of scope for Cactus. Please propose a discussion topic for the Design Summit. Justin On Wed, Mar 16, 2011 at 12:21 PM, Eric Day

Re: [Openstack] A single cross-zone database?

2011-03-16 Thread Justin Santa Barbara
On Wed, Mar 16, 2011 at 1:13 PM, Ed Leafe ed.le...@rackspace.com wrote: On Mar 16, 2011, at 3:39 PM, Justin Santa Barbara wrote: I agree that we could have a better marker, but I'm just going off the spec at the moment. I've checked the agreed blueprint, and caching in zones is out

Re: [Openstack] [Nova] Review and bug priorities

2011-03-14 Thread Justin Santa Barbara
I've been moaning about the fact that Nova is like a happy oblivious little child for way too long, so I'll take on the two bugs that teach it that things can go wrong in the real world: When a node dies, its instances should be marked !running https://bugs.launchpad.net/nova/+bug/661214

Re: [Openstack] [Merge] lp:~mdragon/nova/multi-tenant-accounting into lp:nova

2011-03-04 Thread Justin Santa Barbara
I think, really, we are getting off on a tangent here. The purpose of multitenant is to have a label ('account' or 'project' or whatever ) that we tag resources (instances, etc) in nova with so that we can group together usage reports, etc, that go to some system outside of openstack

Re: [Openstack] server affinity

2011-03-02 Thread Justin Santa Barbara
Thanks Eric: +1, and I appreciate the peace-brokering :-) I'm not wedded to the idea of putting service metadata into the same collection as user metadata; it just seems like a reasonable approach to me. But it's more important to me that we agree on something, than that we agree on the best

Re: [Openstack] OpenStack Compute 1.1

2011-03-02 Thread Justin Santa Barbara
AM, Justin Santa Barbara wrote: Looks like some good changes. Where is this in launchpad, so the community can help develop it? For example, I'm willing to document the reservation of the aws: prefix. Justin On Wed, Mar 2, 2011 at 8:29 AM, Jorge Williams jorge.willi...@rackspace.com

Re: [Openstack] OS API server password generation

2011-03-02 Thread Justin Santa Barbara
I think we need the option _not_ to inject a password (e.g. if I'm on Linux and am going to use SSH private keys, or if I have another higher-security means of accessing my server) Does the API support this (yet)? Also, I know security through obscurity isn't really security, but if we're open

Re: [Openstack] OS API server password generation

2011-03-02 Thread Justin Santa Barbara
On Wed, Mar 2, 2011 at 8:41 PM, Mark Washenberger mark.washenber...@rackspace.com wrote: Yikes, I'm sorry. I didn't mean to give the impression of promoting bad code. I was coming at it from a simplicity perspective because I mistakenly thought my approach was cryptographically equivalent,

Re: [Openstack] server affinity

2011-03-01 Thread Justin Santa Barbara
We decided in the merge to call it Metadata, despite being fully aware of the semantic issues, because that's what the CloudServers / OpenStack API uses. There are many better terms, but for the sake of avoiding a Tower of Babel, let's just call it Metadata. On Tue, Mar 1, 2011 at 6:56 AM,

Re: [Openstack] Entities in OpenStack Auth

2011-03-01 Thread Justin Santa Barbara
Won't putting this in the URL both: 1) Break CloudServers API compatibility (a total no-no)? and 2) Preclude us from having e.g. multi-project queries (show me all my servers in projects A and B)? The options I see open to us are: a) A cookie / header b) A query parameter c) Something in the

Re: [Openstack] Entities in OpenStack Auth

2011-03-01 Thread Justin Santa Barbara
and a crypto signature) Justin On Tue, Mar 1, 2011 at 5:51 PM, Eric Day e...@oddments.org wrote: Hi Justin, On Tue, Mar 01, 2011 at 05:14:42PM -0800, Justin Santa Barbara wrote: However, what I don't understand is how I can query my servers in project1 and project2 (but not those

[Openstack] How to deal with 'tangential' bugs?

2011-02-28 Thread Justin Santa Barbara
Jay and I have been having an interesting discussion about how to deal with bugs that mean that unit tests _should_ fail. So, if I find a bug, I should write a failing unit test first, and fix it (in one merge). However, if I can't fix it, I can't get a failing unit test merged into the trunk

Re: [Openstack] server affinity

2011-02-28 Thread Justin Santa Barbara
Yes - the use case I'm working towards is to use metadata to specify openstack:near=volume-01 when creating a machine, and I will provide a scheduler that will take that information and will assign you a machine e.g. in the same rack as the volume storage. It's unclear right now whether this

Re: [Openstack] server affinity

2011-02-28 Thread Justin Santa Barbara
aren't analogous because if openstack:near changes the instance would migrate to another location? Or if volume-01 was moved, does the instance move too? -Brian -Original Message- From: Justin Santa Barbara jus...@fathomdb.com Sent: Monday, February 28, 2011 6:28pm

Re: [Openstack] server affinity

2011-02-28 Thread Justin Santa Barbara
in our API, probably openstack shouldn't be the only API without a prefix, otherwise it would be unhAPI. Justin -Original Message- From: Justin Santa Barbara jus...@fathomdb.com Sent: Monday, February 28, 2011 6:59pm To: Brian Lamar brian.la...@rackspace.com Subject: Re: [Openstack

Re: [Openstack] Should the OpenStack API re-use the EC2 credentials?

2011-02-24 Thread Justin Santa Barbara
confident we can figure it out. If I missed a conversation, my apologies. pvo From: Vishvananda Ishaya vishvana...@gmail.com Date: Wed, 23 Feb 2011 18:19:41 -0800 To: Justin Santa Barbara jus...@fathomdb.com Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Should the OpenStack API

[Openstack] Should the OpenStack API re-use the EC2 credentials?

2011-02-23 Thread Justin Santa Barbara
I previously fixed OpenStack authentication so it would use the same credentials as EC2. This bugfix was just reverted, because it caused OpenStack API users to have to enter in different credentials (sorry!), but primarily because it hadn't been discussed on the mailing list. So here goes!

  1   2   >