[Openstack] i'm easily confused about time and timezones
hi, Looking at nova/utils.py I *think* that utcnow() is right but utcnow_ts() is incorrect. The former does give UTC, the latter unfortunately changes it to localtime. I haven't written a test in nova, but I extracted the code and wrote a little demo that I *think* shows the right way to do it, using calendar.timegm(). I'd appreciate help understanding / verifying / cluing me in on if I'm barking up the right tree. :-) #!/usr/bin/env python import time, datetime, calendar def utcnow(): return datetime.datetime.utcnow() def utcnow_ts( ts ): return time.mktime( ts.timetuple() ) def dt_to_seconds( dt ): return int( calendar.timegm( dt.timetuple() ) ) def seconds_to_dt( seconds ): return datetime.datetime.utcfromtimestamp( seconds ) now_dt = utcnow() now_ts = utcnow_ts( now_dt ) now_s = dt_to_seconds( now_dt ) nova_dt2 = seconds_to_dt( now_ts ) cal_dt2 = seconds_to_dt( now_s ) print now_dt print now_ts print nova_dt2 print now_s print cal_dt2 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] swift-stats-populate ring expansion
hi, I'm experimenting with expanding a cluster's set of rings. The report is 100% ok before I expand, but when I run swift-stats-report after the expansion it fails to get all the stuff. Anybody have insight/thoughts/guesses re: this? Hrm... Thanks. -Jon. # swift-stats-report -d Queried 24400 containers for dispersion reporting, 34s, 0 retries There were 33 overlapping partitions There were 27 partitions missing one copy. ! There were 227 partitions missing two copies. !!! There were 6830 partitions missing all copies. 71.31% of container copies found (52130 of 73101) Sample represents 1.16% of the container partition space Queried 41942 objects for dispersion reporting, 57s, 0 retries There were 205 overlapping partitions There were 830 partitions missing one copy. ! There were 1366 partitions missing two copies. !!! There were 39260 partitions missing all copies. 3.09% of object copies found (3869 of 125211) Sample represents 1.99% of the object partition space ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] swift-stats-populate ring expansion
hi, thanks for your note. On Thu, Jun 9, 2011 at 12:03 PM, Florian Hines fhi...@rackspace.com wrote: When you updated the ring did you modify more than 1 Zone at a time ? 1) yes. added one node per zone, each having like 20 devices. 2) also, i've learned (had already forgotten :-() that i should re-use the same builder file; that the tools don't know how to assign things the same way every time from scratch. -Jon. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] null ring gz files?
hi, I haven't found a way to do this, I suspect it isn't supported -- is there a way to have a null/zero/empty ring? (I realize it probably isn't The Right Way to do things, but it would be possibly awfully expediently helpful just now. :-) thanks. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] null ring gz files?
hi, solution: don't get confused/mislead into trying rebalance. just do write_ring. thanks. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] bexar swift socket issue?
hi,
On Fri, May 27, 2011 at 1:18 PM, Jon Slenk jsl...@internap.com wrote:
has anybody else (i've been searching but haven't yet precisely hit
pay-dirt) seen socket hangups with wsgi-related socket/worker code?
this is bexar swift.
In particular, has anybody seen issues / experimented with
swift/common/wsgi.py line 130 eventlet.hubs.use_hub('poll')? We
think we're seeing something akin to Java's notifyAll() bad behaviour,
increasing as more connections come in. Wondering if a different hub
like 'select' is known to help.
thanks.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] testing and deploying swift?
hi Jay, Many thanks for the thoughts. The dispersion-report is a great tool for production monitoring but it can be very useful for testing too. ah! i missed finding out about that tool while so far reading docs, thanks for mentioning it. -Jon. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] testing and deploying swift?
On Wed, May 18, 2011 at 8:14 AM, Clay Gerrard clay.gerr...@rackspace.com wrote: Specially regarding expanding the ring - one of the Cloud Files ops wrote some tips in a lp answer awhile back: https://answers.launchpad.net/swift/+question/152024 Cool, thank you for the pointer. -Jon. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] testing and deploying swift?
hi, So what are people's processes for tracking Swift releases, on production systems? I'm guessing Rackspace is probably the most serious deployment to date. If anybody there could comment on what release of Swift is being run and how you expect to deploy newer versions, that would be fun and educational to hear about and mull over. If anybody working on core Swift could comment on which parts of the system are more vs. less dangerous to muck with, that would be great, too. For one example, we're still trying to grok the implications of significantly changing the Rings (expanding them, usually). Like, what even qualifies as significant vs. not. thanks for sharing any experiences, -Jon. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Swift large data ingress thoughts?
hi, Any experience or recommendations or thoughts about getting big data into Swift in the first place? I am figuring there are options like: a) just do it over the public network: slowest but most easy for everybody. b) go sneakernet, and ship a physical device like a single fat disk, or raid, or lots of ssds, and hook those up on the internal Swift network and send them through the proxy: faster but requires some sort of real human service org inside Swift. c) [crazy idea, i have no idea how to get this to work, even if it isn't a dumb idea] directly attach the disk as an object store so that the files are immediately serviceable without any replication in place, and let the automatic gradual replication spread it out to the full cluster? thanks for any insights. sincerely, -Jon. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] ec2 vs. openstack api question
On Mon, Apr 25, 2011 at 5:25 PM, Masanori ITOH masanori.i...@gmail.com wrote: The following two examples are because of lack of Authentication tokens as your guess. Thanks for the sanity check :-) BTW, do you really need to use curl instead of euca2ools, boto,...? Probably I really do need something other than euca2ools, boto. Not curl in the long-run, that's just for slow-me trying to get up to speed and understand what I'd have to write in e.g. C# for custom apps. thanks all! ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] ec2 vs. openstack api question
hi, My local euca2ools works fine for e.g. describe images. They are pointing at port 8774 so they are doing OpenStack-speak I assume. When I try to hit 8773 to do EC2 using curl, I get a stack barf from Python ending in No floating ip for address 127.0.0.1. Might anybody know what the underlying root cause difference is in these 2 ways to hit Nova? thanks. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] ec2 vs. openstack api question
On Mon, Apr 25, 2011 at 4:18 PM, Jon Slenk jsl...@internap.com wrote: ending in No floating ip for address 127.0.0.1. Might anybody know turns out that is a red herring -- thanks to folks in IRC -- seems like that is an error case that should just be a 40something instead of a python barf :-) ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] ec2 vs. openstack api question
On Mon, Apr 25, 2011 at 4:34 PM, Joshua McKenty j...@pistoncc.com wrote: Can you give us the actual command you're running? curl localhost:8773/ works, lists the versions. curl localhost:8773/1.0 python barf with No floating ip curl localhost:8773/services/Cloud 400 Bad Request curl localhost:8773/services/Cloud?Action=DescribeImages 400 Bad Request (guessing i have to do some auth headers or signature for the last one to succeed? i'm trying to read boto to learn.) thanks, -Jon. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] javascript for client-side direct file uploads?
hi, Anybody know of any purely web-browser-client-side-javascript to upload a file to a Swift install? (From what I've googled up on it so far there isn't, and even if there were it would need to deal with security issues. I've seen a Flash tool for uploading directly to S3 but I suspect the authn would be different, and besides it seems to require some un-user-friendly steps to get it working.) thanks. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] dotnet cloud files access?
hi, I've been trying to get https://github.com/rackspace/csharp-cloudfiles to work with a local Swift install, but no dice yet. Has anybody else succeeded with that? (I happen to be running it all on Mono on Mac OS X oh brother. I hacked up a demo C# app myself that does successful REST AuthN and can Download (but for some reason fails to upload) so I know I /can/ get things to work, so I don't grok why the Rackspace code won't work. I'm getting 503's.) thanks for any ideas. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Federated Identity Management (bursting and zones)
On Wed, Mar 30, 2011 at 11:57 AM, Sandy Walsh sandy.wa...@rackspace.com wrote: http://wiki.openstack.org/ZonesOauth At this point the user will be asked to confirm the request (assuming this is the first interaction the user has had with this Zone). So, here's an authenticated user getting this seemingly random prompt ServiceProvider.DataCenter.Tower2 would like access to your Permissions. Will you grant this? Huh? That would make no sense. Even less given the fact that the user had to authenticate to get to Zone A in the first place (her credentials have already been given to the client tool). I think that if the system used capabilities/ZBAC then there would be no such weird prompting. sincerely. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Federated Identity Management (bursting and zones)
On Wed, Mar 30, 2011 at 12:44 PM, Sandy Walsh sandy.wa...@rackspace.com wrote: This flow is saying The AuthZ resource lives on your side of the fence and I'd like to access it, but to do so Alice needs to grant permission and that interaction seems confusing to me. Ja wohl, I don't disagree that it could well be confusing. The classic / bog standard approaches to security very much often suck pardon my lingo with respect to usability. And if we're stuck with whatever system the users want to have themselves rather than being able to impose something actually usable, there's probably not much room to improve? (But I'm a pessimist. :-) I can think of a hack attempt to bandage over the usability horribleness, but it is all just adding complexity and papering over the underlying lameness. I.e. when the Parent realizes that it has to offload to a Child, it could at least inform the user that it is doing that, and that the incoming security request is (probably?) about that. sincerely. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Instance IDs and Multiple Zones
the IDs must be strictly numericalish numbers, with nothing smelling of something like a string in there, i take it? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Instance IDs and Multiple Zones
On Tue, Mar 22, 2011 at 10:41 AM, Ed Leafe e...@leafe.com wrote: Well, since they are defined as: `id` int(11) NOT NULL AUTO_INCREMENT, I would say the chance of a stringish thing slipping in is pretty small. :) if the schema cannot be changed (which might be worth reconsidering since it seems to be a bit of a root cause of trouble) then maybe you have to reserve the last 4 or 5 digits of the id to be the zone id, and then autoincrement on top of that? on the assumption that there would be a limit of or 9 zones ever. but really i'd hazard to suggest that it should somehow be 2 parts, neither of which are super constrained: a zone part and an in-zone-id part. it could even be that the id is left as-is and is semantically required to be joined with the zone name as a prefix before it is a valid interzone id. sincerely. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] api wrappers?
hi, IIUC, it looks like the Nova Dashboard uses EC2 via boto. Questions along those lines: (a) will the community be wanting to move people off of EC2 and towards OpenStack API? (b) if yes, when will the OpenStack tools do that? (c) if yes, when will there be libraries for other languages (we're into C# for whatever reasons right now). thanks for any thoughts. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] api wrappers?
On Fri, Mar 18, 2011 at 12:02 PM, Michael Mayo m...@openstack.org wrote: On Mar 18, 2011, at 11:29 AM, Jon Slenk wrote: (c) if yes, when will there be libraries for other languages (we're into C# for whatever reasons right now). As long as the OpenStack API mirrors the Rackspace Cloud API (as I understand it will), you're covered! There are plenty of language bindings (including C#) available at http://github.com/rackspace Looks like the only Nova binding so far is for Python? The Swift side of things has C# support, but I didn't see anything for compute. sincerely. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] api wrappers?
On Fri, Mar 18, 2011 at 1:48 PM, Michael Mayo m...@openstack.org wrote: Sorry I wasn't more clear about that. Anything you see that's a cloud servers binding would in theory cover nova. Nothing for C# still :-) (hey, I didn't get to pick what language we're using.) I assume nobody is working on such a thing? If we make one I'll try to submit it. sincerely. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
