Re: [Openstack] [Swift] Use JSON in object xattr
On Sat, Apr 6, 2013 at 5:24 PM, Samuel Merritt s...@swiftstack.com wrote: Perhaps one of the original developers will chime in here. All I can really say is that when that decision was made, there was no inkling that portability would ever be a concern. Pickle was fast and effective. Switching it to json should work fine, if someone wants to put in the work. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [swift] how to configure NWR policy
Does any of our documentation say NRW are directly configurable? It's not really quite that simple. N is a property of the cluster, chosen when you build the ring. W is always a simple majority of N. R is generally 1 (eventually consisten reads), but the client can request it be bumped to N ( highly consistent reads) using X-Newest. But then there's stuff like we provide RYW consistency on default reads, which can't really be modeled by NRW notation. - Mike On Mon, Dec 10, 2012 at 9:47 PM, Hua ZZ Zhang zhu...@cn.ibm.com wrote: Alex, that make sense to me. thanks a lot. :-) *Best Regards, * -- *Edward Zhang(张华)* [image: Inactive hide details for Alex Yang ---12/11/2012 11:37:24 AM---Alex Yang alex890...@gmail.com]Alex Yang ---12/11/2012 11:37:24 AM---Alex Yang alex890...@gmail.com *Alex Yang alex890...@gmail.com* 12/11/2012 11:37 AM To Hua ZZ Zhang/China/IBM@IBMCN, cc openstack@lists.launchpad.net openstack@lists.launchpad.net, Hai HJ Ji/China/IBM@IBMCN Subject Re: [Openstack] [swift] how to configure NWR policy Hi, Zhang, Just add 'X-Newest: Ture' in the HTTP GET request. 2012/12/11 Hua ZZ Zhang *zhu...@cn.ibm.com* zhu...@cn.ibm.com Hi all, I have a question about swift configuration for NWR policy. According to some documents of Swift, NWR is configurable. The general configuration is: N=3, W=2, R=1 or 2. Swift can provide both models of consistency: strong and eventual. But I can't find where to configure this option. Any suggestion is appreciated! * Best Regards, * -- *Edward Zhang(张华)* IBM China Software Development Lab ___ Mailing list: *https://launchpad.net/~openstack*https://launchpad.net/~openstack Post to : *openstack@lists.launchpad.net*openstack@lists.launchpad.net Unsubscribe : *https://launchpad.net/~openstack*https://launchpad.net/~openstack More help : *https://help.launchpad.net/ListHelp*https://help.launchpad.net/ListHelp -- 杨雨 Email: *alex890...@gmail.com* alex890...@gmail.com GitHub: *https://github.com/AlexYangYu*https://github.com/AlexYangYu Blog:*http://alexyang.sinaapp.com*http://alexyang.sinaapp.com/ Weibo: *http://www.weibo.com/alexyangyu*http://www.weibo.com/alexyangyu ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] ask for comments - Light weight Erasure code framework for swift
Overall I've never been super enthusiastic about erasure codes for swift. Figuring out which blocks are missing then re-assembling them is a lot more difficult and expensive than what we do now. But if you can come up with a good scheme for identifying missing blocks and it doesn't double the amount of code in Swift, I'm sure we all have use cases where we'd trade latency for disk usage. - Michael On Mon, Oct 15, 2012 at 7:36 PM, Duan, Jiangang jiangang.d...@intel.com wrote: Some of our customers are interested in Erasure code than tri-replicate to save disk space. We propose a BP Light weight Erasure code framework for swift, which can be found here https://blueprints.launchpad.net/swift/+spec/swift-ec The general idea is to have some daemon on storage node to do offline scan - select code object with big enough size to do EC. Will glad to hear any feedback on this. -jiangang ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] General questions regarding zones and replicas
On Mon, Oct 8, 2012 at 8:12 AM, Moritz Krinke mkri...@fotocommunity.net wrote: - I cannot change the number of replicas after creating the ring (e.g. changing it from 1 to 2) Not really. Logically it's not a difficult operation, there's just no code to do it. There was a patch a while back, but I guess it wasn't ever approved: https://review.openstack.org/#/c/5484/ - reading about the unique-as-possible feature in folsom im asking myself how would swift know if nodes are in different places, is there a feature like regions i am not aware of? There's no concept of regions (yet). The commit message is pretty descriptive: https://github.com/openstack/swift/commit/bb509dd863dc99c06a232d1d8e0f950a7e73dcc8 - after migration of data and having two replicas spread over those two nodes, how can i make sure that in the event of a node beeing down swift will not start copying files to second drives on the healty nodes? Swift won't over-replicate when a node's down, only when an individual drive is unmounted. There's no way to disable over-replication in that case, though. - Michael ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] make swift.common.utils.streq_const_time more efficient
That function's purpose is to compare strings without short-circuiting, to foil timing attacks against token comparisons or similar. On Thu, Sep 13, 2012 at 1:28 AM, Mike Green iasy...@gmail.com wrote: def streq_const_time(s1, s2): if len(s1) != len(s2): return False result = 0 for (a, b) in zip(s1, s2): result |= ord(a) ^ ord(b) return result == 0 + If s1 and s2 are of the same length, then the function will compare every characters in them. I think it may be more efficient as follow: def streq_const_time(s1, s2): if len(s1) != len(s2): return False result = 0 for (a, b) in zip(s1, s2): if ord(a) ^ ord(b): return False return True ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Openstack swift][1.6.0] Regarding to swift api for Controlling a Large List of Containers
On Tue, Aug 28, 2012 at 5:46 AM, Irene.Peng-彭怡欣-研究發展部 API like below: Curl –H ‘X-Auth-Token: Token_ID’ http://Proxy_website/Account?marker=bananalimit=2 I think you just need to put quotation marks around that URL -- the is causing the curl command to be backgrounded by the shell, and cutting off the url being sent. - Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] shuffle(nodes) in Swift
On Thu, Jul 5, 2012 at 11:21 AM, Anatoly Legkodymov anat...@nexenta.com wrote: I propose removing shuffle(nodes) from proxy-server will make memory caching 3 times more efficient, without loosing in anything else. I don't know, this kind of stuff is pretty use case dependent. If you're not using page cache for object data (the default), you might want shuffling so you can support some concurrency. If you use page cache but have enough traffic to an object to bottleneck on some other resource, you might want shuffling so you can balance the load. -- Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift] Lost of an object data file is not detected
On Sun, Jun 24, 2012 at 9:05 PM, 山縣陽 bi.yamag...@gmail.com wrote: But if I removed the data file, no process detected it and the data files was not recovered. Is this a bug? or is there any way to detect a data file lost case? Nope, there's nothing that'll detect missing data files once the system's in a steady state. Eventually, we'd like the auditor process that looks for corrupt files to also rebuild indexes as it goes, so we can catch those types of problems. But that guy can take months to traverse a production storage node, so it'll probably always be a bad idea to just go and delete data files. - Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [keystone] v3 API draft (update and questions to the community)
On Tue, Jun 12, 2012 at 3:24 AM, Gabriel Hurley gabriel.hur...@nebula.com wrote: To speak on the specific feature of pagination, the problem of 'corruption' by simultaneous writers is no excuse for not implementing it. You think Google, Facebook, Flickr, etc. etc. etc. don't have this problem? If you consume their feeds you'll notice you can fetch offset-based pagination with ease. You'd never expect to see a navigation element at the bottom of Google search results that said take me to results starting with the letter m. Maybe OT, but the reason Swift doesn't support offset-based pagination is because it doesn't scale well enough. That probably doesn't apply to everyone, though. - Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] gerrit account help
Can someone help me? I think my gerrit account is boned. redbo@ubuntu:~/swift$ git review fatal: A Contributor Agreement must be completed before uploading: http://wiki.openstack.org/HowToContribute -- Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift Consistency Guarantees?
On Tue, Jan 17, 2012 at 4:55 PM, Nikolaus Rath nikol...@rath.org wrote: Amazon S3 and Google Storage make very explicit (non-) consistency guarantees for stored objects. I'm looking for a similar documentation about OpenStack's Swift, but haven't had much success. I don't think there's any documentation on this, but it would probably be good to write up. Consistency in Swift is very similar to S3. That is, there aren't many non-eventual consistency guarantees. Listing updates can happen asynchronously (especially under load), and older versions of files can show up in requests (deletes are just a new deleted version of the file). Swift can generally be relied on for read-after-write consistency, like S3's regions other than the the US Standard region. The reason S3 in US Standard doesn't have this guarantee is because it's more geographically widespread - something Swift isn't good at yet. I can imagine we'll have the same limitation when we get there. Also like S3, Swift can't make any strong guarantees about read-after-update or read-after-delete consistency. We do have an X-Newest header that can be added to GETs and HEADs to make the proxy do a quorum of backend servers and return the newest available version, which greatly improves these, at the cost of latency. - If I receive a (non-error) response to a PUT request, am I guaranteed that the object will be immediately included in all object listings in every possible situation? Nope. - If I receive a (non-error) response to a PUT request, am I guaranteed that a subsequent GET will return the new data in every possible situation (e.g. even if the server accepting the PUT crashes, and another server with an older version of the object is still online)? Nope. - If I receive a (non-error) response to a DELETE request, am I guaranteed that the object will immediately be no longer included in all object listings in every possible situation? Nope. - If the swift server looses an object, will the object name still be returned in object listings? Will attempts to retrieve it result in 404 errors (as if it never existed) or a different error? It will show up in listings, but give a 404 when you attempt to retrieve it. I'm not sure how we can improve that with Swift's general model, but feel free to make suggestions. - Michael ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] swift enforcing ssl?
On Tue, Dec 27, 2011 at 2:11 PM, andi abes andi.a...@gmail.com wrote: Does the swift proxy enforce SSL connections if it's configured with a cert/key file? Or is it assumed that there's an external entity performing that? The Swift proxy's SSL support is probably only useful for light testing - SSL in python (and especially with eventlet) has historically been slow and subtly broken. But basically the way it works, it's either in SSL mode or non-SSL mode. If you configure cert and key files, it switches to SSL mode. In a production environment, I'd suggest putting a reverse proxy like Pound in front of Swift to terminate SSL. Depending on your environment, it may also be a good idea to run that on separate hardware. That can get SSL termination CPU usage off the proxies, and provide all the usual benefits of load balancing like being able to remove proxy servers from rotation without downtime. -Michael ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift slow write performance
On Mon, Dec 19, 2011 at 6:21 AM, Rustam Aliyev rus...@code.az wrote: The only thing which looks suspicious to me are these errors: Dec 18 04:01:28 ec01 object-server ERROR container update failed with 10.0.1.3:6001/d01 (saving for async update later): Timeout (3s) (txn: txdf95ad5a10844ee0b74d70d8a7638082) Dec 18 04:01:28 ec01 object-server ERROR container update failed with 10.0.1.2:6001/d01 (saving for async update later): Timeout (3s) (txn: txee2545ba4610430fa3a6a166ca50c574) Dec 18 04:01:28 ec01 object-server ERROR container update failed with 10.0.1.8:6001/d01 (saving for async update later): Timeout (3s) (txn: tx2546b29b15c643ec90a122a753dfddd3) Yeah, that is likely to be the culprit. Each write is taking at least 3 seconds because it's timing out trying to update the container servers. So you need to debug connectivity from this object server to those IP addresses on port 6001 -- that the IP addresses and port are correct, everything's on the same network, there aren't any firewall rules blocking those connections, that the container servers are running and accepting connections, etc. I'll read through your paste in a bit and see if I notice anything. -- Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Writes are faster than reads in Swift
I can't explain it off the top of my head. I don't have a swift installation to play with at the moment, but it's conceivable that posix_fadvise is slower than we expect (drop_cache is called more frequently during reads than writes, iirc). That could be tested by making drop_cache a no-op in the object server. Or running the object server under a profiler during both operations might shed some light on what is taking so much time. --Mike On Mon, Dec 12, 2011 at 8:44 AM, Zhenhua (Gerald) Guo jen...@gmail.com wrote: Hi, folks Recently, I have run some read/write tests for large files (400GB) in Swift. I found that writes were always faster than reads, which is kinda counter-intuitive. What may be the cause? Has anyone else seen the same problem? Gerald ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Writes are faster than reads in Swift
On Tue, Dec 13, 2011 at 9:21 PM, Huang Zhiteng winsto...@gmail.com wrote: Can anyone explain why Swift doesn't want to utilize page cache _at all_? It's an artifact of the use case swift was built for - heavy on writes, and repeat reads (where a cache would help) are very rare. Having that memory available to cache dirents and inodes has a positive impact on performance, since a swift object server has so many files. The object server used to not drop caches if the file was small and the user wasn't authenticated, but I guess that's been factored out at some point. It'd be nice to have that logic pluggable or configurable somehow, since it does make swift kind of useless for things it'd otherwise be good at, like serving static files directly to browsers. - Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] why swift proxy server use poll not epoll??
On Sun, Dec 11, 2011 at 4:08 AM, pf shineyear shin...@gmail.com wrote: hi all , can any one tell my the reson of swift proxy server http service use python eventlet poll not epoll? Sure. We had a problem where epoll failed to report a socket close event roughly one out of every bazillion times. Then eventlet would freak out because it didn't know that some socket was closed but the OS re-used its file descriptor. I spent some time trying to debug it, but never figured it out. It might have been a problem with the specific kernel we were using, or python not checking an error condition, or some of our aggressive TCP tuning causing trouble. Poll didn't have the same problem and didn't measurably impact performance for the number of connections we do, so I just decided to go with it. i think epoll is more effcient than poll and if i just use one process, many upload action at same time, poll will not work well i think. Epoll is definitely more efficient, but poll does work just fine. If you want to write a bug report or submit a patch to make the eventlet hub configurable, I think that'd be reasonable. -- Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] question about X-Timestamp header
On Mon, Nov 28, 2011 at 9:04 PM, pf shineyear shin...@gmail.com wrote: hi all i think X-Timestamp header is come from the proxy servery to object storage node, the value is proxy server current time. if i have 2 or more proxy server run in one cluster, should i comfirm same account/container/filename use same proxy server? because if i upload one file 2 times use different proxy server, the server time is different, and i think there maybe have some consistent problem if i do so. am i right?? Yeah, Swift's last write wins logic is only as good as proxy server times are synchronized. The idea is you'd use NTP or similar to keep them synced. And NTP generally does a really good job, with clock skews an order of magnitude smaller than the time it takes to PUT an object into Swift (which is about the best conflict resolution level you could hope for anyways). We talked a lot about using logical clocks (e.g. vector clocks) when designing Swift, but realistically they'd probably usually just have to fall back on timestamps to resolve conflicts. Or version objects when there's a conflict and let the client decide which is right, and that's a whole mess for both the clients and the backend. We've also talked about tie breaker ideas, because there's only so much resolution in those timestamps. But in reality, it's a pretty low priority because it's really difficult to exploit and only screws up the user's own data if they manage it. - Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Issues with Packaging and a Proposal
On Wed, Aug 24, 2011 at 3:11 PM, Soren Hansen so...@openstack.org wrote: Rackspace isn't doing their own packaging because of (lack of) Debian support. If they did, they'd have realised almost immediately that the packages actually build on Debian. They're doing it because there'll supposedly be differences in the behaviour Ubuntu (or any other distro for that matter) will want and what Rackspace will want from the packages. I've becried this divergence countless times, but to no avail. That cloudbuilders aren't using the packages either.. I don't really know what to say about that. Nobody's maintaining their own packaging because it's lots of fun, but we can't close shop every time someone decides to go and nerf the openstack packaging. Like right now, the openstack Swift packages are broken for the OS cloudfiles deploys on and they're missing useful features cloudfiles uses. -- Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Getting pagination right
On Wed, May 25, 2011 at 2:40 PM, Jay Pipes jaypi...@gmail.com wrote: The pagination in Swift is not consistent. Inserts into the Swift databases in between the time of the initial query and the requesting the next page can result in rows from the original first page getting on the new second page. No, you only get records not on the first page, because you're sending a marker of the last item from the first page. Though even if that were the case, I wouldn't do very much work to try and provide some sort of point-in-time consistent view of the database for pagination. On Wed, May 25, 2011 at 3:32 PM, Greg Holt gh...@rackspace.com wrote: select w from x where y marker order by y limit z LIMIT X OFFSET Y clause. Your query above would return ALL the rows that match WHERE y marker. That's not what we want. We want a segment of those rows. He had a limit clause in there. The reason we usually shy from offsets is they don't scale. I don't know what cardinality you're expecting on these tables, but if you're querying for an offset of a million, offset's gotta go count a million records before it can return any results. For a marker query, it can just do an index lookup. -- Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] lunr reference iSCSI target driver
What I've been playing with is having a manifest that contains hashes of (4mb) chunks for the volume's backups. When a user initiates a new backup, dm-snapshot does its thing and gives me a block device. I read and hash chunks from that block device and compare them to the manifest, uploading any that differ to Swift, then update the manifest with the new backup. The restore uses fuse with some basic bitmap logic to lazy load chunks from Swift on demand, plus a background thread that fills them in autonomously. I've been pretty happy with fuse's performance and stability (python-fuse that is; fusepy is really slow). The NBD solution isn't really any different logic-wise from the fuse version, but requires a lot more wrangling of server and client processes. And actually we weren't too impressed with the performance of a basic NBD server in some (non-scientific) tests. All of this is sort of at the proof of concept stage at the moment. -- Michael Barton ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] lunr reference iSCSI target driver
On Mon, May 2, 2011 at 9:12 PM, Michael Barton mike-launch...@weirdlooking.com wrote: What I've been playing with is having a manifest that contains hashes of (4mb) chunks for the volume's backups. When a user initiates a new backup, dm-snapshot does its thing and gives me a block device. I read and hash chunks from that block device and compare them to the manifest, uploading any that differ to Swift, then update the manifest with the new backup. Oh, and I don't know if keeping track of dirty chunks so backups are less work is worth putting an indirection layer on top of volumes. It's probably something we can discuss more fully and do some testing around later. -- Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Proposing an Identity Service in OpenStack (a.k.a. Auth)
On Mon, Apr 18, 2011 at 12:15 PM, Eric Day e...@oddments.org wrote: We'll also want to decide if we need a default mechanism for OpenStack deployments, and if so, what should it be. We had a discussion previously and I think it was somewhere between token and HTTP basic w/ SSL. The reason for this is we need to make sure different deployments are compatible. I'm still gonna argue for key signing to be a first-class auth scheme. It enables things that can't be done with token or basic auth, like signed URLs and unencrypted requests. Both of these are desirable for Swift, at the least. It kind of sucks that key signing (as least as implemented by the EC2/S3 API) requires a key to be available to both sides in plaintext. Public key crypto is one way to fix that, but I don't really know how practical that is. -- Mike Barton ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Logging, 500, Debugging
On Mon, Apr 4, 2011 at 4:50 PM, Zed A. Shaw zeds...@zedshaw.com wrote: I'm currently trying to debug some changes to the Swift proxy server and finding the logging facilities a little obtuse. Reading through: Most tracebacks should go to the logs, but yeah, there just isn't a lot of debug-level logging in the Swift proxy server. You might need to add logs for whatever it is you want to see. -- Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Feature Freeze status
I'm gonna +1 Todd. Actually, apache server has a great dev process. They have goals for releases, but people are welcome to submit patches to their mailing list any time, get comments on them, then they're merged if and when people vote them as ready. -- Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Entities in OpenStack Auth
Swift Swift has the concept of accounts, users, and groups. An account contains users, and a user can belong to groups. Accounts names have an abstraction layer, so while you may login with account example.com, the account name used within swift is a UUID with a prefix. By default, a user belongs to a group for the user user:account and a group for the account account. The other group names can be arbitrary strings, so they may be other account names, users, or some application-specific term. All operations are done in the context of a user and account. A user may not be a member of the account it's acting on since resources can specify ACLs, this is especially true for public resources (where user is undefined or anonymous). To be clear, users in swift are entirely a function of the auth middleware. Once you get past middleware, swift only has a concept of accounts, which are designated in the URL. The middleware decides whether or not you have access to that account based on info in the request (or combined with metadata stored in swift, which is how ACLs are implemented). The Cloud Files installation, for example, has no concept of multiple users in an account, because its authentication system doesn't. -- Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] State of OpenStack Auth
On Wed, Mar 2, 2011 at 2:38 PM, Soren Hansen so...@ubuntu.com wrote: I'd like to just mention this blog post: http://www.imperialviolet.org/2010/06/25/overclocking-ssl.html tl;dr quote: If you stop reading now you only need to remember one thing: SSL/TLS is not computationally expensive any more. Oh, thank goodness. I had this dream where it took a non-trivial amount of infrastructure to get many tens of gbps of SSL throughput. - Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Queue Service, next steps
On Fri, Feb 18, 2011 at 5:22 PM, Eric Day e...@oddments.org wrote: The main question right now is where to land on the spectrum of service efficiency vs ease of development (C/C++ on one end and Python on the other). It seems we're landing in the middle with Erlang. :) Maybe I'm describing a separate project, but a fault tolerant and scalable queue would be more interesting to me than something like RabbitMQ with a REST interface. There don't seem to be any reasonable open-source implementations of distributed queues, but they're available and widely used in $closed_source_clouds. So.. I don't know, I guess I'd rather have seen a discussion on what sort of queue service should be built before stuff like implementation language were sussed out. :) And that probably has to be informed by what people need, which I don't really know how we're supposed to analyze. -- Mike Barton ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Queue Service, next steps
On Fri, Feb 18, 2011 at 6:23 PM, Eric Day e...@oddments.org wrote: Perhaps I've been assuming some things, but I thought everyone understood that is what we are looking to build (fault-tolerant, horizontally scalable, ...). We're certainly not looking to build a clustered queue (like RabbitMQ) with a REST API. I know the wiki page is still brief, but I hope that's one of the key take-aways: Sorry, I guess I was confused by some of the discussion, and then I didn't really see the sort of distributed architecture I expected on the wiki. Like, how or whether a single queue will scale or how the system is highly available, how ordered delivery is reconciled with those two, etc. Maybe this is all worked out offline, or it's TBD and the current spec is more of an ideal. But it seems like people want to start banging out code before any of that's on paper. -- Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Queue Service
On Mon, Feb 14, 2011 at 7:57 PM, Paul Voccio paul.voc...@rackspace.com wrote: Looking at the swift docs, they reference a container like so: METHOD /v1/account/container HTTP/1.1 Yeah, this has worked out well for us. Delegated access, authentication methods that don't provide the account name otherwise, anonymous access... And someday we'll likely have a v2 api. So we'll be happy that's in there. -- Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Use of IANA-registered ports
On Sun, Jan 2, 2011 at 7:57 PM, Ewan Mellor ewan.mel...@eu.citrix.com wrote: We've got some complications though: http://swift.openstack.org/howto_installmultinode.html says Auth node: ... This can be on the same node as a Proxy node and Storage nodes: Runs the swift-account-server, swift-container-server, and swift-object-server. This implies that we need at least two ports for a storage proxy, and three ports for a storage node. I think that some people plan to run the Glance API and registry on the same machine too. We could run these things on 80, 81, and 82 in the case of a storage node, but I don't see that that's any better than using arbitrary ports as we are at the moment. 8080 is a possibility too of course, but some people may want to run web UIs on these nodes too, in which case it would be nice to keep 8080 available. All said, I think if people are serious about running storage nodes with account, container, and object servers together, then it's reasonable for us to ask for new ports to be assigned. The argument is weaker (but still reasonable I think) for storage API nodes with auth and proxy together (proxy will use port 80, but we still need one for auth). I don't see a lot of utility in trying to get IANA assigned ports for services that are completely internal to swift. They could change in the future, and vary greatly between different configurations/deployments anyway. I do recommend that in a production environment, public HTTP-based services live on port 80/443. I also recommend that the swift auth server is for entertainment purposes only. -- Mike ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
