Re: [Openstack] Swift tempurl
I had a similar problem. See [1]. The fix was to add delay_auth_decision =
1 in the proxy-server.conf
[filter:authtoken]
...
delay_auth_decision = 1
-Shri
[1] https://answers.launchpad.net/swift/+question/225614
On Wed, Jul 10, 2013 at 4:43 AM, Morten Møller Riis m...@gigahost.dk wrote:
I've been looking at tempurl.py and found that the problem occurs here:
def _get_key(self, env, account):
Returns the X-Account-Meta-Temp-URL-Key header value for the
account, or None if none is set.
:param env: The WSGI environment for the request.
:param account: Account str.
:returns: X-Account-Meta-Temp-URL-Key str value, or None.
key = None
memcache = env.get('swift.cache')
if memcache:
key = memcache.get('temp-url-key/%s' % account)
if not key:
newenv = make_pre_authed_env(env, 'HEAD', '/v1/' + account,
self.agent)
newenv['CONTENT_LENGTH'] = '0'
newenv['wsgi.input'] = StringIO('')
key = [None]
def _start_response(status, response_headers, exc_info=None):
for h, v in response_headers:
if h.lower() == 'x-account-meta-temp-url-key':
key[0] = v
i = iter(self.app(newenv, _start_response))
self.logger.info()
try:
i.next()
except StopIteration:
pass
key = key[0]
if key and memcache:
memcache.set('temp-url-key/%s' % account, key, timeout=60)
return key
The request get 403 forbidden and thus never gets the key in the first
place. I'm looking at the github repo and the implementation there seems to
have changed.
Mvh / Best regards
Morten Møller Riis
Gigahost ApS
m...@gigahost.dk
On Jul 10, 2013, at 6:24 PM, Morten Møller Riis m...@gigahost.dk wrote:
I'm having trouble getting tempurl to work.
I set the X-Account-Meta-Temp-Url-Key metadata on the account. And a GET
request shows it's set correctly.
I've enabled it on the proxy server in /etc/swift/proxy-server.conf:
[DEFAULT]
bind_port = 8080
workers = 8
user = swift
[pipeline:main]
pipeline = healthcheck cache tempurl swauth proxy-server
[app:proxy-server]
use = egg:swift#proxy
allow_account_management = true
account_autocreate = true
[filter:tempurl]
use = egg:swift#tempurl
[filter:swauth]
use = egg:swauth#swauth
set log_name = swauth
super_admin_key = removed
default_swift_cluster = gigahost#https://url/v1#http://localhost:8080/v1
[filter:healthcheck]
use = egg:swift#healthcheck
[filter:cache]
use = egg:swift#memcache
memcache_servers = 127.0.0.1:11211
When creating the tempurl's, even using the swift-temp-url program I get:
~ $ curl -i https://
url/v1/AUTH_224b1001-2c75-444c-aaef-30af13b9154c/000/206.pdf?temp_url_sig=387d79120a591e1cf6f4d4356f5c0a96fb49d202temp_url_expires=1373438360
HTTP/1.1 401 Unauthorized
Server: nginx
Date: Wed, 10 Jul 2013 08:21:32 GMT
Content-Type: text/plain
Content-Length: 35
Connection: keep-alive
401 Unauthorized: Temp URL invalid
I've even tried running it locally on the proxy-server to localhost:8080,
so I'm sure it's not nginx doing any funny stuff.
Does anybody have any suggestions? I'm pretty lost :(
Mvh / Best regards
Morten Møller Riis
Gigahost ApS
m...@gigahost.dk
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift tempurl
I use swauth, and thus authtoken is not present in my pipeline. This might be
why I'm seeing the problem actually.
Mvh / Best regards
Morten Møller Riis
Gigahost ApS
m...@gigahost.dk
On Jul 11, 2013, at 4:04 AM, Shrinand Javadekar shrin...@maginatics.com wrote:
I had a similar problem. See [1]. The fix was to add delay_auth_decision = 1
in the proxy-server.conf
[filter:authtoken]
...
delay_auth_decision = 1
-Shri
[1] https://answers.launchpad.net/swift/+question/225614
On Wed, Jul 10, 2013 at 4:43 AM, Morten Møller Riis m...@gigahost.dk wrote:
I've been looking at tempurl.py and found that the problem occurs here:
def _get_key(self, env, account):
Returns the X-Account-Meta-Temp-URL-Key header value for the
account, or None if none is set.
:param env: The WSGI environment for the request.
:param account: Account str.
:returns: X-Account-Meta-Temp-URL-Key str value, or None.
key = None
memcache = env.get('swift.cache')
if memcache:
key = memcache.get('temp-url-key/%s' % account)
if not key:
newenv = make_pre_authed_env(env, 'HEAD', '/v1/' + account,
self.agent)
newenv['CONTENT_LENGTH'] = '0'
newenv['wsgi.input'] = StringIO('')
key = [None]
def _start_response(status, response_headers, exc_info=None):
for h, v in response_headers:
if h.lower() == 'x-account-meta-temp-url-key':
key[0] = v
i = iter(self.app(newenv, _start_response))
self.logger.info()
try:
i.next()
except StopIteration:
pass
key = key[0]
if key and memcache:
memcache.set('temp-url-key/%s' % account, key, timeout=60)
return key
The request get 403 forbidden and thus never gets the key in the first place.
I'm looking at the github repo and the implementation there seems to have
changed.
Mvh / Best regards
Morten Møller Riis
Gigahost ApS
m...@gigahost.dk
On Jul 10, 2013, at 6:24 PM, Morten Møller Riis m...@gigahost.dk wrote:
I'm having trouble getting tempurl to work.
I set the X-Account-Meta-Temp-Url-Key metadata on the account. And a GET
request shows it's set correctly.
I've enabled it on the proxy server in /etc/swift/proxy-server.conf:
[DEFAULT]
bind_port = 8080
workers = 8
user = swift
[pipeline:main]
pipeline = healthcheck cache tempurl swauth proxy-server
[app:proxy-server]
use = egg:swift#proxy
allow_account_management = true
account_autocreate = true
[filter:tempurl]
use = egg:swift#tempurl
[filter:swauth]
use = egg:swauth#swauth
set log_name = swauth
super_admin_key = removed
default_swift_cluster = gigahost#https://url/v1#http://localhost:8080/v1
[filter:healthcheck]
use = egg:swift#healthcheck
[filter:cache]
use = egg:swift#memcache
memcache_servers = 127.0.0.1:11211
When creating the tempurl's, even using the swift-temp-url program I get:
~ $ curl -i
https://url/v1/AUTH_224b1001-2c75-444c-aaef-30af13b9154c/000/206.pdf?temp_url_sig=387d79120a591e1cf6f4d4356f5c0a96fb49d202temp_url_expires=1373438360
HTTP/1.1 401 Unauthorized
Server: nginx
Date: Wed, 10 Jul 2013 08:21:32 GMT
Content-Type: text/plain
Content-Length: 35
Connection: keep-alive
401 Unauthorized: Temp URL invalid
I've even tried running it locally on the proxy-server to localhost:8080, so
I'm sure it's not nginx doing any funny stuff.
Does anybody have any suggestions? I'm pretty lost :(
Mvh / Best regards
Morten Møller Riis
Gigahost ApS
m...@gigahost.dk
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift tempurl
This turned out to be a known issue with swath.
I upgraded to a newer version and the problem was resolved :)
Mvh / Best regards
Morten Møller Riis
Gigahost ApS
m...@gigahost.dk
On Jul 11, 2013, at 10:52 AM, Morten Møller Riis m...@gigahost.dk wrote:
I use swauth, and thus authtoken is not present in my pipeline. This might be
why I'm seeing the problem actually.
Mvh / Best regards
Morten Møller Riis
Gigahost ApS
m...@gigahost.dk
On Jul 11, 2013, at 4:04 AM, Shrinand Javadekar shrin...@maginatics.com
wrote:
I had a similar problem. See [1]. The fix was to add delay_auth_decision = 1
in the proxy-server.conf
[filter:authtoken]
...
delay_auth_decision = 1
-Shri
[1] https://answers.launchpad.net/swift/+question/225614
On Wed, Jul 10, 2013 at 4:43 AM, Morten Møller Riis m...@gigahost.dk wrote:
I've been looking at tempurl.py and found that the problem occurs here:
def _get_key(self, env, account):
Returns the X-Account-Meta-Temp-URL-Key header value for the
account, or None if none is set.
:param env: The WSGI environment for the request.
:param account: Account str.
:returns: X-Account-Meta-Temp-URL-Key str value, or None.
key = None
memcache = env.get('swift.cache')
if memcache:
key = memcache.get('temp-url-key/%s' % account)
if not key:
newenv = make_pre_authed_env(env, 'HEAD', '/v1/' + account,
self.agent)
newenv['CONTENT_LENGTH'] = '0'
newenv['wsgi.input'] = StringIO('')
key = [None]
def _start_response(status, response_headers, exc_info=None):
for h, v in response_headers:
if h.lower() == 'x-account-meta-temp-url-key':
key[0] = v
i = iter(self.app(newenv, _start_response))
self.logger.info()
try:
i.next()
except StopIteration:
pass
key = key[0]
if key and memcache:
memcache.set('temp-url-key/%s' % account, key, timeout=60)
return key
The request get 403 forbidden and thus never gets the key in the first
place. I'm looking at the github repo and the implementation there seems to
have changed.
Mvh / Best regards
Morten Møller Riis
Gigahost ApS
m...@gigahost.dk
On Jul 10, 2013, at 6:24 PM, Morten Møller Riis m...@gigahost.dk wrote:
I'm having trouble getting tempurl to work.
I set the X-Account-Meta-Temp-Url-Key metadata on the account. And a GET
request shows it's set correctly.
I've enabled it on the proxy server in /etc/swift/proxy-server.conf:
[DEFAULT]
bind_port = 8080
workers = 8
user = swift
[pipeline:main]
pipeline = healthcheck cache tempurl swauth proxy-server
[app:proxy-server]
use = egg:swift#proxy
allow_account_management = true
account_autocreate = true
[filter:tempurl]
use = egg:swift#tempurl
[filter:swauth]
use = egg:swauth#swauth
set log_name = swauth
super_admin_key = removed
default_swift_cluster = gigahost#https://url/v1#http://localhost:8080/v1
[filter:healthcheck]
use = egg:swift#healthcheck
[filter:cache]
use = egg:swift#memcache
memcache_servers = 127.0.0.1:11211
When creating the tempurl's, even using the swift-temp-url program I get:
~ $ curl -i
https://url/v1/AUTH_224b1001-2c75-444c-aaef-30af13b9154c/000/206.pdf?temp_url_sig=387d79120a591e1cf6f4d4356f5c0a96fb49d202temp_url_expires=1373438360
HTTP/1.1 401 Unauthorized
Server: nginx
Date: Wed, 10 Jul 2013 08:21:32 GMT
Content-Type: text/plain
Content-Length: 35
Connection: keep-alive
401 Unauthorized: Temp URL invalid
I've even tried running it locally on the proxy-server to localhost:8080,
so I'm sure it's not nginx doing any funny stuff.
Does anybody have any suggestions? I'm pretty lost :(
Mvh / Best regards
Morten Møller Riis
Gigahost ApS
m...@gigahost.dk
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] swift tempURL requests yield 401 Unauthorized
I upgraded our test cluster to 1.7.4, and still have the same issue. I also bumped the expires to time() + 600 and made sure the clocks on client and servers are in sync to the second (client was 2 minutes off earlier) but so that didn't change anything. expires is def. higher than the current time on the server so.. any help appreciated. thanks, Dieter On Fri, 19 Oct 2012 13:17:39 -0400 Dieter Plaetinck die...@plaetinck.be wrote: Hi, using swift 1.4.8 on Centos machines. (latest packages for centos. note that i'm assuming tempurl works with this version merely because all the code seems to be there, i couldn't find clear docs on whether it should work or not?) I want to use the swift tempURL feature as per http://failverse.com/using-temporary-urls-on-rackspace-cloud-files/ http://docs.rackspace.com/files/api/v1/cf-devguide/content/TempURL-d1a4450.html http://docs.rackspace.com/files/api/v1/cf-devguide/content/Set_Account_Metadata-d1a4460.html TLDR: set up metadata correctly, but tempurl requests yield http 401, can't figure it out, _get_hmac() doesn't seem to be called. First, I set the key metadata (this works fine) (tried both the swift CLI program as well as curl), and I tried setting it both on container level (container uploads) as well as account level (though i would prefer container level) alias vimeoswift=swift -A http://$ip:8080/auth/v1.0 -U system:root -K testpass' vimeoswift post -m Temp-Url-Key:key uploads vimeoswift post -m Temp-Url-Key:key curl -i -X POST -H X-Auth-Token:$t -H X-Account-Meta-Temp-URL-Key:key http://$ip:8080/v1/AUTH_system this seems to work, because when I stat the account and the container, they show up: [root@dfvimeodfsproxy1 ~]# vimeoswift stat uploads Account: AUTH_system Container: uploads Objects: 1 Bytes: 1253 Read ACL: Write ACL: Sync To: Sync Key: Meta Temp-Url-Key: key -- Accept-Ranges: bytes [root@dfvimeodfsproxy1 ~]# vimeoswift stat Account: AUTH_system Containers: 1 Objects: 1 Bytes: 1253 Meta Temp-Url-Key: key -- Accept-Ranges: bytes [root@dfvimeodfsproxy1 ~]# I have already put a file in container uploads (which I can retrieve just fine using an auth token): [root@dfvimeodfsproxy1 ~]# vimeoswift stat uploads mylogfile.log | grep 'Content Length' Content Length: 1253 now however, if i want to retrieve this file using the tempURL feature, it doesn't work: using this script #!/usr/bin/python2 import hmac from hashlib import sha1 from time import time method = 'GET' expires = int(time() + 60) base = 'http://10.90.151.5:8080' path = '/v1/AUTH_system/uploads/mylogfile.log' key = 'key' hmac_body = '%s\n%s\n%s' % (method, expires, path) sig = hmac.new(key, hmac_body, sha1).hexdigest() print '%s%s?temp_url_sig=%stemp_url_expires=%s' % (base, path, sig, expires) ~ ❯ openstack-signed-url2.py http://10.90.151.5:8080/v1/AUTH_system/uploads/mylogfile.log?temp_url_sig=e700f568cd099a432890db00e263b29b999d3604temp_url_expires=1350666309 ~ ❯ wget 'http://10.90.151.5:8080/v1/AUTH_system/uploads/mylogfile.log?temp_url_sig=e700f568cd099a432890db00e263b29b999d3604temp_url_expires=1350666309' --2012-10-19 13:04:14-- http://10.90.151.5:8080/v1/AUTH_system/uploads/mylogfile.log?temp_url_sig=e700f568cd099a432890db00e263b29b999d3604temp_url_expires=1350666309 Connecting to 10.90.151.5:8080... connected. HTTP request sent, awaiting response... 401 Unauthorized Authorization failed. I thought I could easily debug this myself by changing the _get_hmac() function in /usr/lib/python2.6/site-packages/swift/common/middleware/tempurl.py like so: def _get_hmac(self, env, expires, key, request_method=None): (...) if not request_method: request_method = env['REQUEST_METHOD'] self.logger(getting HMAC for method %s, expires %s, path %s % (request_method, expires, env['PATH_INFO'])) hmac = hmac.new(key, '%s\n%s\n%s' % (request_method, expires, env['PATH_INFO']), sha1).hexdigest() self.logger(hmac is + hmac) return hmac however, after restarting the proxy, I don't see my messages showing up anywhere (logging works otherwise, because proxy-server messages are showing up in /var/log/message, showing all incoming http requests and their responses any help is appreciated, thanks! Dieter ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] swift tempURL requests yield 401 Unauthorized
thanks for the help. along with your, and other people's in #openstack-swift on irc, we fixed it. i had not added tempurl to the pipeline in proxy-server.conf. once that was fixed, it worked immediately on 1.7, but not on 1.4 which started saying 500 Internal Server Error. after some more tinkering, still couldn't get it to work. ultimately i just upgraded this cluster to 1.7 too, and it worked straight away. Dieter On Wed, 24 Oct 2012 11:24:38 -0700 Orion Auld or...@swiftstack.com wrote: First, is that the exact logging code that you have? Because AFAIK, self.logger(Message) won't work. self.logger is just the logger object. You'd need to say something like: self.logger.info(Message) to see the message. So you might try that, and then you can see what the issue is more clearly. For me, it's usually one of the following, in order of likelihood: 1. I bungled the TempUrlKey header name setting it with the swift utility. 2. I have a mismatched TempUrlKey. 3. I forgot to set the TempUrlKey. 4. Clock skew. -- Orion Hi, using swift 1.4.8 on Centos machines. (latest packages for centos. note that i'm assuming tempurl works with this version merely because all the code seems to be there, i couldn't find clear docs on whether it should work or not?) I want to use the swift tempURL feature as per http://failverse.com/using-temporary-urls-on-rackspace-cloud-files/ http://docs.rackspace.com/files/api/v1/cf-devguide/content/TempURL-d1a4450.html http://docs.rackspace.com/files/api/v1/cf-devguide/content/Set_Account_Metadata-d1a4460.html TLDR: set up metadata correctly, but tempurl requests yield http 401, can't figure it out, _get_hmac() doesn't seem to be called. First, I set the key metadata (this works fine) (tried both the swift CLI program as well as curl), and I tried setting it both on container level (container uploads) as well as account level (though i would prefer container level) alias vimeoswift=swift -A http://$ip:8080/auth/v1.0 -U system:root -K testpass' vimeoswift post -m Temp-Url-Key:key uploads vimeoswift post -m Temp-Url-Key:key curl -i -X POST -H X-Auth-Token:$t -H X-Account-Meta-Temp-URL-Key:key http://$ip:8080/v1/AUTH_system this seems to work, because when I stat the account and the container, they show up: [root@dfvimeodfsproxy1 ~]# vimeoswift stat uploads Account: AUTH_system Container: uploads Objects: 1 Bytes: 1253 Read ACL: Write ACL: Sync To: Sync Key: Meta Temp-Url-Key: key -- Accept-Ranges: bytes [root@dfvimeodfsproxy1 ~]# vimeoswift stat Account: AUTH_system Containers: 1 Objects: 1 Bytes: 1253 Meta Temp-Url-Key: key -- Accept-Ranges: bytes [root@dfvimeodfsproxy1 ~]# I have already put a file in container uploads (which I can retrieve just fine using an auth token): [root@dfvimeodfsproxy1 ~]# vimeoswift stat uploads mylogfile.log | grep 'Content Length' Content Length: 1253 now however, if i want to retrieve this file using the tempURL feature, it doesn't work: using this script #!/usr/bin/python2 import hmac from hashlib import sha1 from time import time method = 'GET' expires = int(time() + 60) base = 'http://10.90.151.5:8080' path = '/v1/AUTH_system/uploads/mylogfile.log' key = 'key' hmac_body = '%s\n%s\n%s' % (method, expires, path) sig = hmac.new(key, hmac_body, sha1).hexdigest() print '%s%s?temp_url_sig=%stemp_url_expires=%s' % (base, path, sig, expires) ~ ❯ openstack-signed-url2.py http://10.90.151.5:8080/v1/AUTH_system/uploads/mylogfile.log?temp_url_sig=e700f568cd099a432890db00e263b29b999d3604temp_url_expires=1350666309 ~ ❯ wget ' http://10.90.151.5:8080/v1/AUTH_system/uploads/mylogfile.log?temp_url_sig=e700f568cd099a432890db00e263b29b999d3604temp_url_expires=1350666309 ' --2012-10-19 13:04:14-- http://10.90.151.5:8080/v1/AUTH_system/uploads/mylogfile.log?temp_url_sig=e700f568cd099a432890db00e263b29b999d3604temp_url_expires=1350666309 Connecting to 10.90.151.5:8080... connected. HTTP request sent, awaiting response... 401 Unauthorized Authorization failed. I thought I could easily debug this myself by changing the _get_hmac() function in /usr/lib/python2.6/site-packages/swift/common/middleware/tempurl.py like so: def _get_hmac(self, env, expires, key, request_method=None): (...) if not request_method: request_method = env['REQUEST_METHOD'] self.logger(getting HMAC for method %s, expires %s, path %s % (request_method, expires, env['PATH_INFO'])) hmac = hmac.new(key, '%s\n%s\n%s' % (request_method, expires, env['PATH_INFO']), sha1).hexdigest() self.logger(hmac is + hmac) return hmac however, after restarting the proxy, I don't see my messages showing up anywhere
Re: [Openstack] Swift: tempURL
There is a blueprint for this work in Keystone Folsom From: openstack-bounces+jason.rouault=hp@lists.launchpad.net [mailto:openstack-bounces+jason.rouault=hp@lists.launchpad.net] On Behalf Of Suchi Sinha (susinha) Sent: Monday, May 14, 2012 11:29 AM To: openstack@lists.launchpad.net Subject: [Openstack] Swift: tempURL I am trying to run swift temp url feature. We have keystone as identity service. Does this feature works with keystone? I am always getting no such file or directory. I am following all the steps generate the tempURL. I will appreciate any help. Thanks. ~Suchi smime.p7s Description: S/MIME cryptographic signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift: tempURL
tempurl dose not work with keystone. you must follow the instruction of tempurl to use it. the instruction is locate in the comment of swift/common/middleware/tempurl.py 在 2012-5-15 上午2:06，Suchi Sinha (susinha) susi...@cisco.com写道： I am trying to run swift temp url feature. We have keystone as identity service. Does this feature works with keystone? ** ** I am always getting “no such file or directory”. I am following all the steps generate the tempURL. ** ** I will appreciate any help. ** ** Thanks. ~Suchi ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
