Re: [Openstack] swift
On Mon, 8 Jul 2013 23:59:40 +0800 Peter Cheung mcheun...@hotmail.com wrote: Hi AllHow you guy admin swift? by command line or by www.swiftstack.com admin portal? Sounds like a great question for the ops list http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [SWIFT] raising network traffic o the storage node
On Mon, 8 Jul 2013 13:50:38 + Klaus Schürmann klaus.schuerm...@mediabeam.com wrote: I use a swift storage as a mail-store. Now I have about 1.000.000 objects stored in the cluster. Traffic Storagenode: http://www.schuermann.net/temp/storagenode.png Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver.png Can someone explain such behavior? At a guess, raising number of objects makes number of partitions to increase, and that increases the replication traffic, specifically the number of sent MD5s for partitions. It would be interesting to corellate the number of objects and number of non-empty and empty partitions with the amounts of traffic. If the increasing transfer of hashes is the reason, you could also verify by graphing the traffic to port 873 separately. Swift never replicates object bodies through its own HTTP interface, so this splits control traffic from data traffic for you. Data traffic should be driven by customer and node failures, not consistency checking. Be prepared to split up storage nodes, however. Even if there's no bug with replication, its aggregate traffic increases with the increases in object counts. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] curl with swift
On Mon, 10 Jun 2013 11:28:01 -0700
Remo Mattei r...@mattei.org wrote:
I am using this command now but I get not auth to get this object.
curl -X GET \
-H X-Auth-Token: 813c6eef9f474e7f860ef42dcaeeb53b \
http://192.168.235.113:8080/v1/AUTH_9ffeae726f33436b9e0796d31f85f730/remo.pen
Remo.pem
The command looks ok.
A naked token usually means Keystone. You're using Keystone, don't you?
If so, make sure that you're using the token ID and not user ID, which
Keystone also returns in the same reply.
Example:
[root@node_k1 ~]# curl -X POST http://127.0.0.1:5000/v2.0/tokens -d '{auth:
{tenantName:tsa17, passwordCredentials:{username:spare17,
password:secret}}}' -H 'Content-Type: application/json'
{access: {token: {expires: 2013-01-30T18:50:37Z, id:
b2eeb6503bb94a1f8c792c9968947172}, serviceCatalog: {}, user: {username:
spare17, roles_links: [], id: 87c6174170ad471ca141630200f90b9b,
roles: [], name: spare17}}}
Other than that, check the logs. I presume you didn't try to snip the
uuid part of a prefixed token that other authentication systems return.
-- Pete
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Load Balancers for Swift with nginx and pound
On Fri, 14 Jun 2013 18:43:21 +0800 王娜 wangna...@gmail.com wrote: like this,before I stop the proxy service ,there are five containers. I would double-check that both proxies had same ring files. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift 1.7.4 stable - logging middleware adding lag to final responses
On Thu, 10 Jan 2013 12:37:57 -0300 Leandro Reox leandro.r...@gmail.com wrote: When we turn off the proxy-logging middleware, everything starts to response like a charm 8ms or so vs 800-1400ms !!! At a guess, maybe the proxy-logging middle accidentially tries to resolve an IP address somewhere? And i cant find the way to log locally on debug mode without activating the proxy-logging middleware Indeed, in the past I had to resort to custom logging code in instances like that. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] python-swiftclient is missing from epel repo
On Sun, 18 Nov 2012 23:01:05 +0200 George Lekatsas glekats...@gmail.com wrote: after a yum update in centos 6.3 i have the following error Error: Package: python-glance-2012.2-3.el6.noarch (epel) Requires: python-swiftclient Just retry it later. EPEL works fine to find python-swiftclient here, I just checked. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift] account-level and container-level usage information
On Mon, 12 Nov 2012 20:48:35 -0800 Ning Zhang n...@zmanda.com wrote: Is there any Swift (GUI or command line) tool that can retrieve the account-level and container-level usage information (e.g. how large space has been used under an account, how large space has been used under a tenant) and also works with keystone? If you're content with 1st party view, Alex Yang's mail gives the answer. But if you want a 3rd part view (authenticate as administrator or bypassing the authentication), then I don't think there is a tool for that. I tried to find one before I started on swift-report, but found none. Swiftly can be used to ease the problem of formulating the correct URLs when accessing back-ends instead of the proxy, but it's not a complete turnkey solution. Actually I was thinking about pilfering Greg's code from Swiftly and grafting it onto swift-report, but that hasn't happened. So the only way you can do it today is to extract the URLs by authenticating with Keystone as 1st party (with curl), then issue curl -X HEAD to account, container, and object servers. Basically you still authenticate in the end as Alex suggested. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Troubleshooting Swift 1.7.4 on mini servers
On Tue, 30 Oct 2012 11:07:55 -0700 Nathan Trueblood nat...@truebloodllc.com wrote: The file size seems to have no bearing on the issue, although I haven't tried really tiny files. Bigfile3 is only 200K. Okay. BTW, do not forget to use curl and issue the same PUT that proxy does, see if it throws 507 repeateably. That could shortcut some of the testing. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Troubleshooting Swift 1.7.4 on mini servers
On Mon, 29 Oct 2012 18:16:52 -0700 Nathan Trueblood nat...@truebloodllc.com wrote: Definitely NOT a problem with the filesystem, but something is causing the object-server to think there is a problem with the filesystem. If you are willing to go all-out, you can probably catch the error with strace, if it works on ARM. Failing that, find all places where 507 is generated and see if any exceptions are caught, by modifying the source, I'm afraid to say. I suspect a bug in one of the underlying libraries. That's a possibility. Or, it could be a kernel bug. You are using XFS, right? If it were something other than XFS or ext4, I would suspect ARM blowing over the 2GB barrier somewhere, since your object is called bigfile3. As it is, you have little option than to divide the layers until you identify the one that's broken. BTW, make sure to disable the fallocate, since we're at it. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Troubleshooting Swift 1.7.4 on mini servers
On Fri, 26 Oct 2012 17:26:07 -0700 Nathan Trueblood nat...@truebloodllc.com wrote: I'm trying to figure out what's going wrong with my Swift deployment on a small cluster of mini servers. I have a small test cluster (5 storage nodes, 1 proxy) of mini-servers that are ARM-based. The proxy is a regular, Intel-based server with plenty of RAM. The object/account/container servers are relatively small, with 2GB of RAM per node. And the disk is how big? Oct 26 17:07:46 data05 object-server 192.168.1.111 - - [27/Oct/2012:00:07:46 +] PUT /sda6/150861/AUTH_system/myfiles5/home/lab/bigfile1 507 - - tx8dc917a4a8c84c40a4429b7bab0323c6 - 0.0031 Well, what does df say? The Object-servers do give a 507 error, which might indicate a disk problem, but there is nothing wrong with the storage drive. And also if there was a fundamental drive problem then I wouldn't be able to upload objects in the first place. You could upload them to a reduced number of nodes, and then the replication would inflate the space used by the replication ratio. Finally, it's possible that tombstones are not properly expired for some reason. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift] Unresolved reference in swift: test/unit/proxy/test_server.py
On Fri, 12 Oct 2012 15:02:24 +0400 Victor Rodionov victor.rodio...@nexenta.com wrote: I found unresolved reference for exception MemcacheLockError (test/unit/proxy/test_server.py line 3580) in Swift unit test for proxy server. I try to find this exception class on swift code, but can't find it. https://github.com/openstack/swift/blob/master/test/unit/proxy/test_server.py#L3580 Good catch. Just needs a subclassing of Exception with pass, IMHO. -- P ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [folsom] Swift-backed glance fails x-glance-api-copy-from
On Fri, 05 Oct 2012 13:27:17 +0200
Koert van der Veer ko...@cloudvps.com wrote:
ERROR glance.api.v1.images [-] Traceback (most recent call last):
File /usr/lib/python2.7/dist-ackages/glance/api/v1/images.py, line
437, in _upload
image_meta['size'])
File /usr/lib/python2.7/dist-packages/glance/store/swift.py, line
464, in add
content_length=image_size)
File /usr/lib/python2.7/dist-packages/swiftclient/client.py, line 1079,
in put_object
headers=headers)
File /usr/lib/python2.7/dist-packages/swiftclient/client.py, line 981, in
_retry
rv = func(self.url, self.token, *args, **kwargs)
File /usr/lib/python2.7/dist-packages/swiftclient/client.py, line 828, in
put_object
conn.request('PUT', path, contents, headers)
File /usr/lib/python2.7/httplib.py, line 958, in request
self._send_request(method, url, body, headers)
File /usr/lib/python2.7/httplib.py, line 992, in _send_request
self.endheaders(body)
File /usr/lib/python2.7/httplib.py, line 954, in endheaders
self._send_output(message_body)
File /usr/lib/python2.7/httplib.py, line 818, in _send_output
self.send(message_body)
File /usr/lib/python2.7/httplib.py, line 790, in send
self.sock.sendall(data)
File /usr/lib/python2.7/dist-packages/eventlet/greenio.py, line 283,
in sendall
tail = self.send(data, flags)
File /usr/lib/python2.7/dist-packages/eventlet/greenio.py, line 265,
in send
len_data = len(data)
TypeError: object of type 'CooperativeReader' has no len()
Log that traceback using Launchpad bug tracking. Even if Swift
provoked it, Glance should not be tracebacking.
-- Pete
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift account listing
On Thu, 19 Jul 2012 16:10:06 +0100 Juan J. Martinez j...@memset.com wrote: I guess you can use the list of current accounts from Keystone and translate that into the account ring hash. swift-get-nodes /etc/swift/account.ring.gz myKeyStoneAcct | grep Hash | cut -f2 5819de5a52d5813f5ce95c9121b97652 Then you can look for hashes that are not in that list of hashes. Per storage node you should check: /srv/node/$0/accounts/$1/*/$2/ The point is to use Swift itself _and_ Keystone, in order to find discrepancies or orphan accounts. I ended using listdir for now, since our installation is very small, so directories fit in memory. Code is here: https://github.com/zaitcev/swift-report Output looks like this: 15051/4a7/3acbbe2ab55b81269ff88490a1b574a7 SK zaitcev 60690/f22/ed125debcbadbac11ef93c40dede0f22 SK glance 5497/6ee/1579e4404e54e5edb53c00f1206696ee SK shared 52389/69e/cca50f1c92b3b7f2a15d6b8e2aaee69e S- - 3066/787/0bfa11e194ee8889ff1c797a718cf787 SK admin 56328/3d8/dc088209ed71d08a00493c95888583d8 SK testuser S is for accounts found in Swift, K is for accounts found in Keystone. I have a feeling though that I must be reinventing the bicycle here. Surely someone, somewhere, have written a Swift consistency checker before. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [glance] legacy client removal and python-glanceclient
On Wed, 01 Aug 2012 01:06:10 -0400 Jay Pipes jaypi...@gmail.com wrote: I don't disagree with you. At the same time, I think Brian has a good point when he compares having two versions of SQLAlchemy installed on a system: it just doesn't make much sense. But having glance(1) and super-duper-incompatible-glance-ng(1) installed together makes a lot of sense. Count systems in the field that have both ifconfig and ip installed. Millions and millions of them. Brian's analogy with SQUalchemy is pretty amusing but fails to hide the fact that the new binary is incompatible. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Swift account listing
Guys, a simple question hopefuly: How do I list all Swift accounts? Specifically, I have a test installation that I used to experiemnt with various upgrades and migrations. I probably lost a few testing accounts in it. Now it sits there and uses up space. How do I find and eliminate orphan accounts? Once I have a list of accounts, I can match them up with the output of keystone user-list, and I can run swift-account-audit on them. I'm thinking ls /srv/node/partitions/accounts/ on all storage nodes and then building a union set. Seems kind of too ad-hoc though. Greetings, -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift] LFS patch (Ia32c9c34)
On Mon, 16 Jul 2012 22:45:48 + Victor Rodionov victor.rodio...@nexenta.com wrote: Most of patch code was restructured, most of logic was moved to middleware level and use hooks in Swift code. I create separate project (LFS middleware https://github.com/nexenta/lfs) for now there are only 2 supported file system types (XFS and ZFS) there. Also this middleware provide API for getting file system status information (for example, for ZFS it's current pool status, etc). It sounds strange to make Swift aware of the specific LFS. One day I come up with ZaitcevFS and what happens then? Patching Swift again I presume. I wish Gluster people would stop delaying their competing LFS proposal and finally submit it. Their code was out for a year, but they apparently are content to carry a fork forever: http://review.gluster.com/805 for 1.4.5 http://review.gluster.com/3118 for 1.4.8 -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Problem integrating swift with keystone
On Tue, 10 Jul 2012 09:29:12 -0800 (GMT-08:00) Jyothsna Padavala jyothsna.padav...@strongauth.com wrote: $: swift -V 2 -A http://nova.strongauth.com:5000/v2.0 -U admin:adminuser -K adminuserpswd stat There is no object-store endpoint on this auth server. $: keystone --endpoint http://nova.strongauth.com:35357/v2.0/ --token mytoken endpoint-list and could see the list of endpoints which has an object-store endpoint as well. No, please don't describe verbally. Copy-paste the command and its output. It must be some minute typo in publicUrl field that you cannot see because you're desensitized to it. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift]Is the cloudfiles compatible with keystone v2.0?
On Thu, 5 Jul 2012 18:21:48 +0800 Alex Yang alex890...@gmail.com wrote: If i want use the cloudfiles with the keystone(the newest version), I need change the code of cloudfiles by myself? Indeed, clients have to be adapted in cases like yours. However, please look at python-swiftclient. It may be sufficiently similar so you don't need to change much in your application that uses python-cloudfiles. By the way, it was my understanding that Keystone people encourage everyone to migrate from port 5000 to port 35357. The latter is assigned with IANA. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift on WebOb 1.2
I just saw your change uploaded https://review.openstack.org/8195/ Thanks a lot! I'll have a look and review ASAP. -- Pete On Mon, 4 Jun 2012 14:40:21 +0900 iryoung jeong iryo...@gmail.com wrote: Anyway, current status is that I make your PatchSet 3 passed all unit/functests with webob 1.1 and only 4 tests failed with webob 1.2b3. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Single Server Test
On Mon, 04 Jun 2012 10:00:37 +1000 Chandana De Silva chand...@desilva.id.au wrote: I have not worked with OpenStack before this, and want to set up a minimal test instance of Open Stack on a single server. Is this possible, and if so is there a document which goes through the steps ? I used Fedora Test Day wiki as an outline with success in the past. It is quite rudimentary, but if you follow the steps exactly, the result is a somewhat useable Nova, which can be gradually enhanced (for example, I moved mine from a file image volume group to an actual LVM volume group). http://fedoraproject.org/wiki/Test_Day:2012-03-08_OpenStack_Test_Day Probably not the best way to do it, but just as a suggestion. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] openstack.common inside swiftclient and RPM
Hi, Monty:
The python-swiftclient has something that I believe you added:
[zaitcev@lembas python-swiftclient-tip]$ git log
swiftclient/openstack/__init__.py
commit 7df012329f0b22e19f878cee2602407cb23042ef
Author: Monty Taylor mord...@inaugust.com
Date: Wed May 16 17:30:46 2012 -0400
Add openstack project infrastructure.
Now, I am wondering, why is it necessary to keep this inside the
swiftclient/ directory?
My problem is that if I do nothing too crazy, the following RPM
.spec packages the aforementioned directory, too:
%{python_sitelib}/swiftclient
For now I resorted to running this after python setup.py install:
rm -r %{buildroot}%{python_sitelib}/swiftclient/openstack
Still... As an experiement, I moved the openstack/ one level up,
changed swiftclient.openstack.common to openstack.common in setup.py,
and everything seems to be working.
-- Pete
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift3 Github pages
On Mon, 21 May 2012 18:01:04 +0200 Chmouel Boudjnah chmo...@chmouel.com wrote: I have sent already the gerrit review to remove swift3 from swift : https://review.openstack.org/#/c/7628/ and reference your repository http://github.com/fujita/swift3 in associated project. If swift3 goes out of the tree, I am going to drop my -1 on Mike Burton's proxy_logging change. It will move the onus on Tomo to accomodate, although personally I think Mike was wrong and middleware should complete by the time __call__ returns. IMHO it is a deficiency of proxy_logger, which does not get exposed by any in-tree middleware except swift3. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift3 Github pages
On Mon, 21 May 2012 16:06:42 -0500 Gregory Holt gh...@rackspace.com wrote: There are examples of middleware returning generators out there, such as in PEP 333 WSGI, specifically the class example at http://www.python.org/dev/peps/pep-0333/#the-application-framework-side. Obviously I didn't read the PEP closely enough, so I assumed the opposite. Sorry. Here's my message about it: https://lists.launchpad.net/openstack/msg11261.html That is something that swift.common.wsgi.WSGIContext's _app_call works around, ensuring the generator is activated before returning. I see, I missed that. Thanks, -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift on Webob-1.2 anyone?
On Wed, 09 May 2012 18:18:52 +0200 Ionuț Arțăriși iartar...@suse.cz wrote: It would be great to have it on 1.2b3, though. I think I had a pretty good start, here: https://review.openstack.org/7569 Not sure if I am doing the right thing, however. I split it like this: - Proxy server does a lot of processing and checking of the keys, so it needs to operate on byte strings, not the unicode (it may be fixed up, but I found it too hard). So, I just force unicode strings that are actually UTF-8 with each UTF-8 byte taking 1 unicode symbol. This is totally wrong, but it works. If someone can cook up an alternative patch, I'm all for it. Note that this permits Proxy to work on keys that are binary garbage, in case of e.g. LFS. - Object, Container, and Account are fixed up to work on Unicode in Pythonic way. One side effect of this is that they ONLY accept UTF-8 encoded keys now. It appears that they verify such, so I am not breaking any compatibility, I think. Would love to see any fixes, objections, additions, etc. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift on Webob-1.2 anyone?
On Wed, 9 May 2012 16:14:48 +0100 Chmouel Boudjnah chmo...@chmouel.com wrote: The challenge if you are taking a stab at it would probably be to support webob 1.1* and 1.2 at the same time in the code. I noticed that too. 1.1.1 made some intermediate choices that are difficult to reconcile, namely throwing warnings that break tests. The None values that they sprinkled over 1.2 ironically help. Fedora is likely to ship 1.1 in F17, so I guess I have to deal with it somehow. What a pain. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Swift on Webob-1.2 anyone?
I ran .unittests on a box with python-webob-1.2b3 and it throws left and right: errors=72, failures=6. I'm wondering if anyone is working on adapting Swift for WebOb 1.2 and if a patch is available somewhere. I see Ionut fixed lp:984042, but clearly it wasn't enough. If nobody's done it yet, I suppose I could take a swing at it. New webob is required to address the S3-with-colon problem lp:936998. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] swift stat returns Auth GET failed: http://127.0.0.1:5000/v2.0/tokens 401 Not Authorized
On Mon, 7 May 2012 23:34:02 +0530
Shashi Kanth Boddula shashi@gmail.com wrote:
root@OpenStack:~# swift -V 2 -A http://OpenStack:5000/v2.0 -U
adminUser:admin -K 012345SECRET99TOKEN012345 stat
Auth GET failed: http://OpenStack:5000/v2.0/tokens 401 Not Authorized
endpoints: [
{
adminURL: http://OpenStack:35357/v2.0;,
internalURL: http://OpenStack:5000/v2.0;,
publicURL: http://OpenStack:5000/v2.0;,
The above is supposed to be the endpoint of Swift, but you put the
endpoint of Keystone in there. When you pass the -A http://OpenStack:5000/v2.0,
the CLI tool accesses the Keystone to discover where the Swift is, using
the endpoint record.
So:
# With SSL proxy at kvm-san:
$keystone endpoint-create --region RegionOne --service_id $SSID \
--adminurl 'http://kvm-rei.zaitcev.lan/v1.0/' \
--publicurl 'https://kvm-san.zaitcev.lan/v1/AUTH_$(tenant_id)s' \
--internalurl 'http://kvm-rei.zaitcev.lan/v1/AUTH_$(tenant_id)s'
-- P
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
[Openstack] Swift's proxy_logging vs. swift3
Hi, Mike: I applied the proxy_logging patch (change 5284) to another Swift, which happens to have the swift3 loaded, and S3 accesses started to fail. Since swift3 is an existing part of Swift, I put a -1 on the review for now, and I hope we can resolve this somehow and re-spin a patch #7. The pipeline looks like this: [zaitcev@kvm-san ~]$ grep pipeline /etc/swift/proxy-server.conf [pipeline:main] pipeline = healthcheck cache swift3 swauth proxy-logging proxy-server [zaitcev@kvm-san ~]$ As much as I can tell, the scenario is this: - swift3's app.__call__ identifies the request as S3. If it's native Swift, it is passed along unmolested and works fine. - __call__ find the controller, say ServiceController - controller defines an intercept start_response(), do_start_response, invokes the downstream app with it - proxy_logger's app.__call__, however, is a generator (has yield). As I guess, a generator is used as a tool to postpone logging like a hook was before - as downstream app returns to swift3, it assumes that start_response was called by the app, but in fact, only the generator prologue was called and returned an iterable. The fields that do_start_response fills are not filled, and swift3 tracebacks with IndexError. I threw together a patch like this (against proxy log patch #6): commit 66b08d63c41c8aeebe745b00ba942dc564595e58 Author: Pete Zaitcev zait...@kotori.zaitcev.us Date: Fri May 4 17:34:32 2012 -0600 Fixed swift3 but now proxy_logging does not log anything for PUT. diff --git a/swift/common/middleware/proxy_logging.py b/swift/common/middleware/proxy_logging.py index b5f25df..c3f4467 100644 --- a/swift/common/middleware/proxy_logging.py +++ b/swift/common/middleware/proxy_logging.py @@ -101,6 +101,10 @@ class ProxyLoggingMiddleware(object): self.access_logger = get_logger(access_log_conf, log_route='proxy-access') +self.client_disconnect = False +self.input_proxy = None +self.start_time = None + def log_request(self, env, status_int, bytes_received, bytes_sent, request_time, client_disconnect): @@ -144,31 +148,34 @@ class ProxyLoggingMiddleware(object): ))) self.access_logger.txn_id = None -def __call__(self, env, start_response): -status_int = [500] +def log_generator(self, env, status_int, body_resp): bytes_sent = 0 -client_disconnect = False -input_proxy = InputProxy(env['wsgi.input']) -env['wsgi.input'] = input_proxy -start_time = time.time() - -def my_start_response(status, headers, exc_info=None): -status_int[0] = int(status.split()[0]) -return start_response(status, headers, exc_info) - try: -for chunk in self.app(env, my_start_response): +for chunk in body_resp: bytes_sent += len(chunk) yield chunk except GeneratorExit: # generator was closed before we finished -client_disconnect = True +self.client_disconnect = True raise finally: -request_time = time.time() - start_time -self.log_request(env, status_int[0], input_proxy.bytes_received, -bytes_sent, request_time, -client_disconnect or input_proxy.client_disconnect) +request_time = time.time() - sef.start_time +self.log_request(env, status_int[0], +self.input_proxy.bytes_received, bytes_sent, request_time, +self.client_disconnect or self.input_proxy.client_disconnect) + +def __call__(self, env, start_response): +status_int = [500] +self.client_disconnect = False +self.input_proxy = InputProxy(env['wsgi.input']) +env['wsgi.input'] = self.input_proxy +self.start_time = time.time() + +def my_start_response(status, headers, exc_info=None): +status_int[0] = int(status.split()[0]) +return start_response(status, headers, exc_info) +body_resp = self.app(env, my_start_response) +return self.log_generator(env, status_int, body_resp) def filter_factory(global_conf, **local_conf): conf = global_conf.copy() With the above, the server works once again, but I did something wrong and PUT requests are not logged by the proxy_logging anymore. It seems like generator's finally clause is not invoked somehow. I thought it was supposed to be invoked when the iterator is deleted, but most likely I'm confused. I need you to take over and apply your Python-fu, because mine is lacking. Cheers, -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Endpoints problems
On Fri, 13 Apr 2012 04:24:56 + Justin Shepherd jshep...@rackspace.com wrote: And if i am reading the code correctly they support both of the following variable declarations: %(tennant_id)s $(tennant_id)s since it appears the code replaces $( with %( Exactly why documentation is important, since it codifies the intent and not artefacts of implementation. I only wish we had an implementation that left 's' for Python-like %(key)s, and supported a Make-like $(key) without 's'. However, putting it in post-Essex is going to break compatibility, so it's too late. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Endpoints problems
On Fri, 13 Apr 2012 13:28:36 +
Guilherme Birk guib...@hotmail.com wrote:
keystone --token ADMIN
--publicurl=http://192.168.100.142:8774/v2/$(tenant_id)s
But I'm getting a tenant_id: command not found.
Shell expands variables in the double quote () strings, so you
have to use single quote (').
-- Pete
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [SWIFT] Looking for an approach to attach an account or a container in operating system . Like a NAS of SAN driver.
On Thu, 12 Apr 2012 23:57:59 +0800 Kuo Hugo tonyt...@gmail.com wrote: What if user can attach their own account from swift-proxy (or something else) directly via NFS or CIFS or iscsi target will be much better. Would it not be easier to use an actual filesystem for that, like GlusterFS? I don't quite see the point of layering a filesystem API adaptor on top of Swift. Since applications that import a filesystem API expect filesystem characteristics, performance is going to be fragile at best. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Endpoints problems
On Thu, 12 Apr 2012 15:28:21 -0500 Anne Gentle a...@openstack.org wrote: keystone --token 012345SECRET99TOKEN012345 --endpoint http://192.168.206.130:35357/v2.0 endpoint-create \ [] --internalurl http://192.168.206.130:8774/v2/$(tenant_id)s I haven't fixed this yet because I'm not sure if the $(tenant_id)s is literal or which tenant_id specifically to use (the Service tenant for the adminurl possibly)? The expression $(tenant_id)s is really contained inside the pattern in the database. It is substituded with a specific tennant ID when an application makes its request. The weird syntax is inherited from Python, where one can use constructs like %s or %(key)s. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Endpoints problems
On Thu, 12 Apr 2012 22:33:38 -0400 Lorin Hochstein lo...@nimbisservices.com wrote: My understanding (based on somebody's previous answer in the mailing list, or possibly one of the IRC channels) was that you could only do this with the template backend, and when you were using the database backend you had to explicitly add endpoints for each tenant that needed to talk to the API. Can you confirm that you can use the $(tenant_id)s syntax when the backend is set using the following keystone.conf entry: [catalog] driver = keystone.catalog.backends.sql.Catalog The thought that the substitution may not work never occured to me. I have the following in keystone.conf: [catalog] # dynamic, sql-based backend (supports API/CLI-based management commands) driver = keystone.catalog.backends.sql.Catalog template_file = /etc/keystone/default_catalog.templates # static, file-based backend (does *NOT* support any management commands) #driver = keystone.catalog.backends.templated.TemplatedCatalog #template_file = ./etc/default_catalog.templates And I run the following command as a part of re-population script: ## Without proxy: #$keystone endpoint-create --region RegionOne --service_id $SSID \ #--publicurl 'http://kvm-rei.zaitcev.lan/v1/AUTH_$(tenant_id)s' \ #--adminurl 'http://kvm-rei.zaitcev.lan/v1.0/' \ #--internalurl 'http://kvm-rei.zaitcev.lan/v1/AUTH_$(tenant_id)s' # With SSL proxy at kvm-san: $keystone endpoint-create --region RegionOne --service_id $SSID \ --adminurl 'http://kvm-rei.zaitcev.lan/v1.0/' \ --publicurl 'https://kvm-san.zaitcev.lan/v1/AUTH_$(tenant_id)s' \ --internalurl 'http://kvm-rei.zaitcev.lan/v1/AUTH_$(tenant_id)s' Everything seems to work. Also see Yong-Sheng's message about the source code. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Endpoints problems
On Fri, 13 Apr 2012 08:11:28 +0800
Yong Sheng Gong gong...@cn.ibm.com wrote:
Maybe, here some codes are worth thouhands of words:
I quite agree. On the other hand, does Anne care to read it? She needs
a concise answer as to what command to document precisely, if I understad
her query right.
d = dict(CONF.iteritems())
d.update({'tenant_id': tenant_id,
'user_id': user_id})
catalog = {}
It would be great if we kept track of available keys in a document
somewhere. Of course, the source is open. That's how I found my
forgotten 's'. But still, it's a hurdle to clear for some.
-- Pete
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift nee st code weirdness
On Fri, 6 Apr 2012 00:17:49 -0400
Andrew Clay Shafer a...@parvuscaptus.com wrote:
What were you expecting/trying to do with the code change?
Here's a patch that hopefuly answers the question, although be warned
that it does not actually work even if the scope of parser is fixed
(it measures wrong thread):
--- a/bin/swift
+++ b/bin/swift
@@ -1889,7 +1889,7 @@ overridden with -A, -U, or -K.'''.strip('\n'))
return options, args
-if __name__ == '__main__':
+def main():
parser = OptionParser(version='%prog 1.0', usage='''
Usage: %%prog command [options] [args]
@@ -1986,3 +1986,8 @@ Example:
for thread in threading_enumerate():
thread.abort = True
raise
+
+if __name__ == '__main__':
+# P3
+import cProfile
+cProfile.run('main()', stprof)
-- Pete
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
[Openstack] Swift nee st code weirdness
Hi, All:
In the process of tinkering for lp:959221, I made a benign modification
to make it possible to invoke swift as a module, when everything broke
loose with errors like NameError: global name 'parser' is not defined.
Looking at the code it seems like a thinko, with the following fix:
diff --git a/bin/swift b/bin/swift
index 0cac5d6..a14c646 100755
--- a/bin/swift
+++ b/bin/swift
@@ -1202,7 +1203,7 @@ download --all OR download container [options] [object]
[object] ...
stdout.'''.strip('\n')
-def st_download(options, args, print_queue, error_queue):
+def st_download(parser, args, print_queue, error_queue):
parser.add_option('-a', '--all', action='store_true', dest='yes_all',
default=False, help='Indicates that you really want to download '
'everything in the account')
@@ -1378,7 +1379,7 @@ list [options] [container]
'''.strip('\n')
-def st_list(options, args, print_queue, error_queue):
+def st_list(parser, args, print_queue, error_queue):
parser.add_option('-p', '--prefix', dest='prefix', help='Will only list '
'items beginning with the prefix')
parser.add_option('-d', '--delimiter', dest='delimiter', help='Will roll '
@@ -1423,7 +1424,7 @@ stat [container] [object]
args given (if any).'''.strip('\n')
-def st_stat(options, args, print_queue, error_queue):
+def st_stat(parser, args, print_queue, error_queue):
(options, args) = parse_args(parser, args)
args = args[1:]
conn = get_conn(options)
@@ -1548,7 +1549,7 @@ post [options] [container] [object]
post -m Color:Blue -m Size:Large'''.strip('\n')
-def st_post(options, args, print_queue, error_queue):
+def st_post(parser, args, print_queue, error_queue):
parser.add_option('-r', '--read-acl', dest='read_acl', help='Sets the '
'Read ACL for containers. Quick summary of ACL syntax: .r:*, '
'.r:-.example.com, .r:www.example.com, account1, account2:user2')
@@ -1619,7 +1620,7 @@ upload [options] container file_or_directory
[file_or_directory] [...]
'''.strip('\n')
-def st_upload(options, args, print_queue, error_queue):
+def st_upload(parser, args, print_queue, error_queue):
parser.add_option('-c', '--changed', action='store_true', dest='changed',
default=False, help='Will only upload files that have changed since '
'the last upload')
Seems obvious if I look at this:
globals()['st_%s' % args[0]](parser, argv[1:], print_queue, error_queue)
The parser is always the first argument, so... Someone please tell me
if I am missing something here. Or should I just file it in Gerrit?
Weird part is, unit tests pass both on the current code and the one
with my patch above. The problem seems too gross to let it work, but
everything appears in order.
-- Pete
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift and keystone: asking for an auth token.
On Tue, 03 Apr 2012 16:53:05 +0200 Pierre Amadio pierre.ama...@canonical.com wrote: [filter:tokenauth] paste.filter_factory = keystone.middleware.auth_token:filter_factory service_port = 5000 service_host = 192.168.122.102 auth_port = 35357 auth_host = 192.168.122.102 auth_protocol = http I'm surprised this works at all, because if service_protocol is not set, it defaults to https:. However, your -A argument points to http:, like so ubuntu@swift-a:~$ swift -V 2 -U ubuntu:ubuntu -K openstack -A http://192.168.122.102:5000/v2.0 list Strange that nothing shows up in logs. Account GET failed: https://192.168.122.105:8080/v2/AUTH_ed0a2ebb66054096869605fb53c049d7?format=json 403 Forbidden Just in case, i try to change the keystone objectstore endpoint to use v1 instead of v2 in the 3 catalog urls, but i had the same result. This is the most obvious problem. Please try to use /v1 once again, look at swift.log. If that fails, we'll need to start probing with curl step by step. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Code 500 when authenticating with password
Greetings:
It seems that I am unable to authenticate against the current Keystone,
and I am not sure if it is a bug or a local misconfiguration. So, before
I file in Launchpad, I'd like someone to verify that I am doing the
right thing.
The problem looks like this:
[zaitcev@kvm-rei zaitcev]$ keystone --os_auth_url=http://localhost:35357/v2.0
--os_username=admin --os_password=admpass --os_tenant_name=admten user-list
No handlers could be found for logger keystoneclient.v2_0.client
Authorization Failed: An unexpected error prevented the server from fulfilling
your request. incomplete format (HTTP 500)
The server logs this:
2012-03-30 21:35:41ERROR [root] incomplete format
Traceback (most recent call last):
File /usr/lib/python2.7/site-packages/keystone/common/wsgi.py, line 184, in
__call__
result = method(context, **params)
File /usr/lib/python2.7/site-packages/keystone/service.py, line 292, in
authenticate
metadata=metadata_ref)
File /usr/lib/python2.7/site-packages/keystone/common/manager.py, line 48,
in _wrapper
return f(*args, **kw)
File /usr/lib/python2.7/site-packages/keystone/catalog/backends/sql.py,
line 167, in get_catalog
catalog[region][srv_type]['publicURL'] = public_url % d
ValueError: incomplete format
2012-03-30 21:35:41DEBUG [keystone.common.wsgi]
RESPONSE HEADERS
2012-03-30 21:35:41DEBUG [keystone.common.wsgi] Content-Type =
application/json
2012-03-30 21:35:41DEBUG [keystone.common.wsgi] Vary = X-Auth-Token
2012-03-30 21:35:41DEBUG [keystone.common.wsgi] Content-Length = 161
2012-03-30 21:35:41DEBUG [keystone.common.wsgi]
2012-03-30 21:35:41DEBUG [keystone.common.wsgi]
RESPONSE BODY
2012-03-30 21:35:41DEBUG [keystone.common.wsgi] {error: {message: An
unexpected error prevented the server from fulfilling your request. incomplete
format, code: 500, title: Internal Server Error}}
2012-03-30 21:35:41DEBUG [eventlet.wsgi.server] 127.0.0.1 - - [30/Mar/2012
21:35:41] POST /v2.0/tokens HTTP/1.1 500 327 0.260567
Yay, a traceback. However, it may be hinting that some kind of publicURL
field is not set somewhere. Does this looks familiar to anyone?
BTW, authentication with token works, and produces this:
[zaitcev@kvm-rei zaitcev]$ keystone --endpoint http://kvm-rei:35357/v2.0
--token=758ce883df47 user-list
+--+-+---+-+
|id| enabled | email | name |
+--+-+---+-+
| 5c3c279d18f541aa859178869584ca40 | True| None | zaitcev |
| ed6aa912c28849c0913cb1106bb26b8c | True| None | admin |
+--+-+---+-+
So, the Keystone server kinda-sorta works, just not really.
-- Pete
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Code 500 when authenticating with password
On Sat, 31 Mar 2012 10:26:10 +0800
Yong Sheng Gong gong...@cn.ibm.com wrote:
also you can see the request body from server log, just like this:
2012-03-31 10:16:44DEBUG [keystone.common.wsgi]
REQUEST BODY
2012-03-31 10:16:44DEBUG [keystone.common.wsgi] {auth: {tenantName:
admin, passwordCredentials: {username: admin, password:
password}}}
2012-03-31 10:16:44DEBUG [keystone.common.wsgi]
Indeed it's there in the log:
2012-03-30 21:35:41DEBUG [keystone.common.wsgi]
REQUEST BODY
2012-03-30 21:35:41DEBUG [keystone.common.wsgi] {auth: {tenantName:
admten, passwordCredentials: {username: admin, password: admpass}}}
But how does this help?
-- Pete
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Code 500 when authenticating with password
On Sat, 31 Mar 2012 10:26:10 +0800 Yong Sheng Gong gong...@cn.ibm.com wrote: Second: you can use keystone --debug user-list to show what http message sent to server As it turned out, I omitted 's' after the format, because I redone my old Keystone population script, which did not have any of those. E.g. public_url was: http://kvm-rei.zaitcev.lan/v1/AUTH_$(tenant_id) but it should've been: http://kvm-rei.zaitcev.lan/v1/AUTH_$(tenant_id)s -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Object expirer in Swift SAIO
Hello: Is running swift-object-expirer necessary in SAIO? Currently, there is no instruction to set one up, and so swift-init bypasses it, only reports that it's missing. Things seem to work fine without. I found a bunch of tombstones, but they are all zero length. Curious, -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [SWIFT] Skipping disk as it is not mounted
On Thu, 22 Mar 2012 14:09:52 + Leander Bessa leande...@gmail.com wrote: I'm installing swift on ubuntu 11.10 on a physical machine with and the partition mounted on /mnt/disk. I have a symbolic link on /srv/node/disk pointing to /mnt/disk and /srv/node belongs to swift:swift. The disk is mounted, but i keep getting the same error message. Perhaps the symbolic link confuses the check (done with os.path.ismount()). Either mount to /srv/node/disk or slap mount_check = false into the *.conf files. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [SWIFT] Skipping disk as it is not mounted
On Thu, 22 Mar 2012 17:56:01 + Leander Bessa leande...@gmail.com wrote: I've tried mounting it directly but it still didn't work. The only thing that got it running was disabling the mount_check. You don't happen to sub-partition like SAIO does? This works for me (2 volumes on 1 node for a moment): [root@kvm-rei swift]# head -7 account-server/1.conf [DEFAULT] #devices = /srv/node #mount_check = false bind_port = 6012 user = swift log_facility = LOG_LOCAL0 [root@kvm-rei swift]# grep vd /proc/mounts /dev/vda2 / ext4 rw,seclabel,relatime,user_xattr,acl,barrier=1,data=ordered 0 0 /dev/vda1 /boot ext4 rw,seclabel,relatime,user_xattr,acl,barrier=1,data=ordered 0 0 /dev/vdb /srv/node/vdb xfs rw,seclabel,noatime,nodiratime,attr2,delaylog,noquota 0 0 /dev/vdc /srv/node/vdc xfs rw,seclabel,noatime,nodiratime,attr2,delaylog,noquota 0 0 [root@kvm-rei swift]# -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] swift-get-nodes lies to me?
On Sat, 10 Mar 2012 12:42:02 + Souza, Bob bob.so...@hp.com wrote: The first argument to swift-get-nodes is the swift hash of the user name, typically some like AUTH_mumble, not the user name itself. Indeed so, thanks. I eventually fetched that hash from logs. However, I imagine it may get taxing on a server with a bunch of active users, so I'm wondering if there's a value in enhancing swift-get-nodes to poke at authenticator to find the hash (enabled with -A)? -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] swift-get-nodes lies to me?
Dear All: A strange thing seems to happen. It looks like this: - I upload a key named testdata [zaitcev@simbelmyne tests]$ /q/zaitcev/hail/swift-tip/bin/swift -A http://kvm-rei:5000/v1.0 -U zaitcev -K pass list test-1235163301 testdata [zaitcev@simbelmyne tests]$ I have a script that also finds size (104857600) and MD5 (49289935f687585f5e95e94c7c1ec29d). - Then, I run swift-get-node, on the node where proxy resides: [root@kvm-rei zaitcev]# swift-get-nodes /etc/swift/object.ring.gz zaitcev test-1235163301 testdata Account zaitcev Container test-1235163301 Object testdata Partition 147444 Hash8ffd13698cf92ef170db5db21f5efd9f Server:Port Device 192.168.129.20:6010 vdb Server:Port Device 192.168.129.18:6020 vdc Server:Port Device 192.168.129.18:6010 vdb Server:Port Device 192.168.129.20:6020 vdc [Handoff] curl -I -XHEAD http://192.168.129.20:6010/vdb/147444/zaitcev/test-1235163301/testdata; curl -I -XHEAD http://192.168.129.18:6020/vdc/147444/zaitcev/test-1235163301/testdata; curl -I -XHEAD http://192.168.129.18:6010/vdb/147444/zaitcev/test-1235163301/testdata; curl -I -XHEAD http://192.168.129.20:6020/vdc/147444/zaitcev/test-1235163301/testdata; # [Handoff] ssh 192.168.129.20 ls -lah /srv/node/vdb/objects/147444/d9f/8ffd13698cf92ef170db5db21f5efd9f/ ssh 192.168.129.18 ls -lah /srv/node/vdc/objects/147444/d9f/8ffd13698cf92ef170db5db21f5efd9f/ ssh 192.168.129.18 ls -lah /srv/node/vdb/objects/147444/d9f/8ffd13698cf92ef170db5db21f5efd9f/ ssh 192.168.129.20 ls -lah /srv/node/vdc/objects/147444/d9f/8ffd13698cf92ef170db5db21f5efd9f/ # [Handoff] [root@kvm-rei zaitcev]# - Finally, I actually find the file and verify it: [root@kvm-rei zaitcev]# find /srv/node/vdc -type f -a -size 104857600c /srv/node/vdc/objects/221984/51c/d8c8047f0c10449d1a264371874f851c/1331343894.44919.data [root@kvm-rei zaitcev]# md5sum /srv/node/vdc/objects/221984/51c/d8c8047f0c10449d1a264371874f851c/1331343894.44919.data 49289935f687585f5e95e94c7c1ec29d /srv/node/vdc/objects/221984/51c/d8c8047f0c10449d1a264371874f851c/1331343894.44919.data [root@kvm-rei zaitcev]# The question is, why the discrepancy? Before I dive into the construction of the ring, is this a known issue? Or do I do something obviously wrong with arguments of swift-get-nodes? Curious, -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Object Storage Swift on rhel6.0
On Thu, 23 Feb 2012 10:09:07 +0530 Sudhakar Maiya sma...@gmail.com wrote: please check and let me know what needs to be done You have to decide if you run with SSL or without. These two clauses obviously contradict to each other: [DEFAULT] #cert_file = /etc/swift/cert.crt #key_file = /etc/swift/cert.key bind_port = 8080 bind_ip = 10.30.91.80 [filter:swauth] use = egg:swauth#swauth set log_name = swauth super_admin_key = swauthkey default_swift_cluster = local#https://10.30.91.80:8080/v1 Note that swauth loops right back to Swift in order to serve account information. Make sure you've got the right key, too. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] swift keystone help
On Tue, 21 Feb 2012 11:24:06 -0600 Paras pradhan pradhanpa...@gmail.com wrote: Yeah that was a type when I copy pasted. You mean, the commented section header #[]? keystone-manage tenant add adminTenant keystone-manage user add adminUser secretword keystone-manage role add Admin keystone-manage role grant Admin adminUser keystone-manage role grant Admin adminUser adminTenant Did I miss something? You need an endpoint, like this: keystone-manage service add swift storage Swift Object Storage Service keystone-manage endpointTemplates add RegionOne swift \ http://kvm-rei.zaitcev.lan/v1/AUTH_%tenant_id% \ http://kvm-rei.zaitcev.lan/v1.0/ \ http://kvm-rei.zaitcev.lan/v1/AUTH_%tenant_id% \ 1 1 -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] DeltaCloud
On Tue, 21 Feb 2012 23:04:22 +0700 Frans Thamura fr...@meruvian.org wrote: i think the foundation need to be hurry and work closely with Apache ;0 What do you propose this work closely with actually mean? What do you want Foundation actually do in this regard? Issue a press release? -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] swift keystone help
On Mon, 20 Feb 2012 14:34:22 -0600 Paras pradhan pradhanpa...@gmail.com wrote: http://paste.openstack.org/show/4912/ Did you notice this: #[filter:keystone] use = egg:keystone#tokenauth Seems commented :-) The only undocumented trap that I hit was that you MUST create users and grant roles on a tenant, e.g. keystone-manage role grant KeystoneAdmin zaitcev admten The old configuringservices.rst omited tenant there. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift S3 with Keystone anyone?
On Thu, 2 Feb 2012 19:03:54 +0900 Akira Yoshiyama akirayoshiy...@gmail.com wrote: I modified the wiki: http://wiki.openstack.org/Keystone-BP-S3Token I added authtoken in main pipeline just now. I am still unable to figure out how to add the right password. So far, the only thing I figured out is that you do not use the Swift password that I expected. Instead, something like this happens: keystone/logic/service.py: def authenticate_s3(self, credentials): creds = self.credential_manager.get_by_access(credentials.access) # c.access is zaitcev above keystone/backends/sqlalchemy/api/credentials.py: def get_by_access(self, access, session=None): result = session.query(models.Credentials).\ filter_by(type=EC2, key=access).first() So, clearly keystone-manage user add zaitcev passw0rd admten is not sufficient. Would you share the proper incarnation for keystone-manage to add a EC2 type password? Yours, -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] git-review change 3988
Guys, Am I going mad? Can't see what's going on. The resulting code after commit looks is this (copy-pasted precisely): # Workaround bug in urlparse on OSX if parsed_url.scheme == ssh and hostname[:2] == //: hostname = hostname[2:].split(/)[0] # Workaround bug in urlparse on OSX elif parsed_url.scheme == ssh and hostname[:2] == //: hostname = hostname[2:].split(/)[0] How is this supposed to work? The conditions looks absolutely identical! Therefore if A: foo elif A: foo How could elif evaluate to true and the second foo run? -- P ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift S3 with Keystone anyone?
On Thu, 9 Feb 2012 21:23:06 -0500 Andiabes andi.a...@gmail.com wrote: Take a peek here ( this is for diablo/stable) https://github.com/dellcloudedge/barclamp-keystone/blob/master/chef/cookbooks/keystone/ recipes/server.rb#L160 Thanks a lot, this worked: keystone-manage credentials add zaitcev EC2 zaitcev passw0rd admten Or, actually, it worked so I do not get 403 anymore. Still buckets do not get listed (I presume containers are buckets, right?). -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift and devstack.
On Mon, 6 Feb 2012 18:53:36 +0100 Chmouel Boudjnah chmo...@chmouel.com wrote: Another thing I want to implement is to add Swift as a back-end to Glance. Wait, what is this then [zaitcev@lembas glance-tip]$ ls glance/store/swift.py glance/store/swift.py [zaitcev@lembas glance-tip]$ cat .git/HEAD ref: refs/heads/master -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Swift S3 with Keystone anyone?
Hello: Does anyone happen to have Swift running with S3 and Keystone? If yes, send me the proxy-server.conf, please. Also, I'd like to ask a few questions, if I may. I tried to piece it together from the code, but failed. The authentication is done with a special hook into Keystone. It supplies middleware, keystone/keystone/middleware/s3_token.py, which invokes a POST to v2 Keysone with OS-KSS3:s3Credentials, then sets a req. header X-Auth-Token. So far so good. However, how does it fit in with Swift? The actual S3 operations are implemented by swift/common/middleware/swift3.py, which rolls up the canonical string, then stuffs it into env['HTTP_X_AUTH_TOKEN']. The intent is, as I understand, to invoke the special purpose code in tempauth and thus is useless for Keystone. So, how is this supposed to work? I imagine the pipeline should look something like this: [pipeline:main] pipeline = healthcheck cache s3auth swift3 proxy-server [filter:s3auth] use = egg:keystone#swiftauth service_protocol = http service_host = 192.168.129.18 service_port = 5000 [filter:swift3] use = egg:swift#swift3 Except... There is no entry point for s3_auth in keystone egg. Documentation seems to be absent. I suppose I could put it together, if I got it all working at least once. Confused, -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift Consistency Guarantees?
On Fri, 20 Jan 2012 15:17:32 -0500 Nikolaus Rath nikol...@rath.org wrote: Thanks! So there is no way to reliably get the most-recent version of an object under all conditions. If you bend the conditions hard enough to hit the CAP theorem, you do. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Listing non-public images in Glance
Hello: This clearly seems like I am missing something obvious, but is it possible to list non-public images in Glance? It came up because I have a Glance setup without Keystone or other authentication for now, like this: [pipeline:glance-api] pipeline = versionnegotiation context apiv1app Images that have X-Image-Meta-Is_public: False do not get listed with glance index. I am not saying that it is wrong per se, all the documentation implies that a GET to /v1/images only produces a listing of public images, and it looks like all functional and unit tests in ./glance/tests set the public flag as necessary. But I'm wondering: a) If authentication is in effect, can users list their own images? It is easy to forget what you have. The Image Warehouse service in Aeolus permits to list images regardless, as long as bucket is accessible. b) If authentication is not in effect, should we chage to listing everything, public and not? I can file a bug and see it implemented. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Listing non-public images in Glance
On Thu, 19 Jan 2012 16:48:55 -0500 Jay Pipes jaypi...@gmail.com wrote: a) If authentication is in effect, can users list their own images? Yes. If authentication is enabled and a user calls GET /images, they see a list of non-deleted, non-killed-status *public* images (is_public=True) AND any images where the owner_id is the user's Tenant or User ID AND any images that have manually been shared with the Tenant or User ID via the image-memberships functionality. Thanks, sounds logical. Now I just need to ensure it works, and likely update the doc/source/glanceapi.rst a bit. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] High I/O on idle Swift devices
On Tue, 13 Dec 2011 02:30:14 + Rustam Aliyev rus...@code.az wrote: [...] During this period, I can see constant read I/O of 3MB/s on each swift device. When I shutdown object-server I/O goes to zero. Killing swift-object-server is tantamount to unplugging a disk: the I/O goes to zero, but it was not the entity that requested the I/O. I'd look in the logs to see if rsync was flopping around failing or something of that nature. You need to go up the chain and see what activity was triggering the I/O in the first place. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] swfit / keystone auth
On Tue, 13 Dec 2011 11:27:28 -0500
andi abes andi.a...@gmail.com wrote:
A few questions (about keystone 2012.1)
Just to facilitate the tip-sharing, here's what I know - not being
an expert in either Swift of Keystone.
a) does the swift middleware work with v1.0 or 2.0 auth?
I heard that v2.0 is not ready to be used with Swift. BTW, I am at
swift 1.4.3, maybe it wasn't back then. Your 2012.1 should be Essex,
which, AFAIK, is not even branched yet. Anyhow, I configured /v1
(no JSON, just good old X-Storage-Url etc.).
b) are folks using swift-keystone2 or the middleware bundled with keystone
(auth_token + swift_auth).
I am not quite sure what to make out of this question, but here's
what I have, middleware-related
/etc/swift/proxy-server.conf:
[pipeline:main]
pipeline = healthcheck cache keystone proxy-server
[filter:keystone]
use = egg:keystone#tokenauth
auth_protocol = http
#auth_host = 127.0.0.1
auth_host = 192.168.129.18
auth_port = 35357
admin_token = 758ce883df47
delay_auth_decision = 0
service_protocol = http
service_host = 192.168.129.18
service_port = 5000
service_pass = dTpw
I know that keystone ignores a bunch of these
[DEFAULT]
service-header-mappings = {
'nova' : 'X-Server-Management-Url',
'swift' : 'X-Storage-Url',
'cdn' : 'X-CDN-Management-Url'}
[pipeline:admin]
pipeline = urlrewritefilter admin_api
[pipeline:keystone-legacy-auth]
pipeline = urlrewritefilter legacy_auth RAX-KEY-extension service_api
[filter:legacy_auth]
paste.filter_factory = keystone.frontends.legacy_token_auth:filter_factory
c) when trying to use auth_token and swift_auth, I see the keystone log
below trying to stat an account. []
Curious...
This was triggered with:
swift -A http://192.168.124.82:5000/v2.0/ -V 2.0 -U openstack:user -K
password stat
My tests run like so:
swift -A http://kvm-rei:5000/v1.0 -U admin -K adminpass stat -v
-- Pete
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
[Openstack] 404 for account on nodes
Greetings: This is most likely to be an administration problem, but I am trying to use it as a hook to gain understanding into workings of Swift. I have a test cluster with 2 VMs and 4 nodes. At some point, I reinstalled one of the nodes, and now this 404 happens from time to time: Dec 6 19:08:19 kvm-ni account-server 192.168.129.18 - - [07/Dec/2011:02:08:19 +] HEAD /vdb/18239/AUTH_1 404 - - - - 0.0010 The root cause, apparently, is that this account does not exist. On the system which shows no such symptom, we have: [root@kvm-rei log]# ls -l /srv/node/vdb/accounts/ total 12 drwxrwxrwx. 3 swift swift 4096 Nov 19 00:22 124217 drwxrwxrwx. 3 swift swift 4096 Nov 30 15:36 18239 drwxrwxrwx. 3 swift swift 4096 Nov 18 18:44 236930 On the problem node, we have: [root@kvm-ni log]# ls -l /srv/node/vdb/accounts/ total 8 drwxrwxrwx. 3 swift swift 4096 Nov 18 22:22 124217 drwxrwxrwx. 3 swift swift 4096 Nov 18 16:50 236930 All I know at present is this blurb in the Administration Guide: Replication updates are push based. For object replication, updating is just a matter of rsyncing files to the peer. Account and container replication push missing records over HTTP or rsync whole database files. So, the suspicion is that the account updater is not running. Why is it that the accounts are not synchronized? Or in a more interesting way, what part of code, if any, is supposed to be responsible for keeping the cluster consistent? -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Keystone + Swift integration
On Wed, 23 Nov 2011 09:28:01 -0300 Leandro Reox leandro.r...@gmail.com wrote: keystone-manage endpointTemplates add RegionOne swift http://172.16.0.88:8080/v1/AUTH_%tenant_id% http://172.16.0.88:8080/ http://172.16.0.88:8080/v1/AUTH_%tenant_id% 1 1 I'm curious, did you actually put the '%' (percent) into those URLs, or you replaced it with the appropriate tennant ID? The documentation (doc/configuration.rst) tells to use square brackets, not percents, which is probably another hint to substitute the actual tenant ID. But what to do if we have 2 tenants? -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Some insight into the number of instances Nova needs to spin up...
On Wed, 29 Dec 2010 19:27:09 + Erik Carlin erik.car...@rackspace.com wrote: The 1M host limit still seems reasonable to me. [] In my opinion, such numbers are completely out of whack. Google's Chubby article says that the busiest Chubby has 90,000 clients (not hosts!) and the biggest datacenter has 10,000 systems. They found such numbers pushing the border of unmanageable. Granted they did not use virtualization, but we're talking the number of boxes in both cases. So to reach 1M hosts in a Nova instance you have to have it manage 100 datacenters. There are going to be calls for federation of Novas long before this number is reached. Sustaining a high flap rate is a worthy goal and will have an important practical impact. And having realistic sizing ideas is going to help it. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
