Re: [ceph-users] OpenStack Keystone with RadosGW

radosgw supports keystone v3 in Jewel. Can you give more details about the error? what is the exact command are you trying? radosgw log with debug_rgw=20 and debug_ms=5 will be most helpfull On Tue, Nov 22, 2016 at 10:24 AM, 한승진 wrote: > I've figured out the main reason is. > > When swift client

Re: [ceph-users] OpenStack Keystone with RadosGW

I've figured out the main reason is. When swift client request through keystone user like 'admin', keystone returned with X-Auth-Token header. After that, the swift client requests with X-Auth-Token to radosgw, but radosgw returned 'AccessDenied' Some people says radosgw doesn't support keystone

[ceph-users] OpenStack Keystone with RadosGW

Hi All, I am trying to implement radosgw with Openstack as an object storage service. I think there are 2 cases for using radosgw as an object storage First, Keystone <-> Ceph connect directly. like below guide.. http://docs.ceph.com/docs/master/radosgw/keystone/ Second, use ceph as a back-en

Re: [ceph-users] Openstack keystone with Radosgw

Hello Mark - I setup a new Ceph cluster like before. But this time it is talking to Icehouse. Same set of problems like before. That is keystone flags are not being honored if they are under [client.radosgw.gateway]. It seems like the issue is with my radosgw setup. Let me create a new thread

Re: [ceph-users] Openstack keystone with Radosgw

Thanks Mark for looking into this further. As I mentioned earlier, I have following nodes in my ceph cluster - 1 admin node 3 OSD (One of them is a monitor too) 1 gateway node This should have worked technically. But I am not sure where I am going wrong. I will continue to look into this and k

Re: [ceph-users] Openstack keystone with Radosgw

Because this is an interesting problem, I added an additional host to my 4 node ceph setup that is a purely radosgw host. So I have - ceph1 (mon + osd) - ceph2-4 (osd) - ceph5 (radosgw) My ceph.conf on ceph5 included below. Obviously I changed my keystone endpoints to use this host (ceph5). Aft

Re: [ceph-users] Openstack keystone with Radosgw

Right, So you have 3 osds, one of whom is a mon. Your rgw is on another host (called gateway it seems). I'm wondering if is this the issue. In my case I'm using one of my osds as a rgw as well. This *should* not matter... but it might be worth trying out a rgw on one of your osds instead. I'm

Re: [ceph-users] Openstack keystone with Radosgw

Hello Mark - with rgw_keystone_url under radosgw section, I do NOT see keystone handshake. If I move it under global section, I see initial keystone handshake as explained earlier. Below is the output of osd dump and osd tree. I have 3 nodes (node1, node2, node3) acting as OSDs. One of them (nod

Re: [ceph-users] Openstack keystone with Radosgw

Was that with you moving just rgw_keystone_url into [global]? If so then yeah, that won't work as it will be missing your auth token etc (so will fail to authorize always). You need to chase up why it is not seeing some/all settings in the [client.radosgw.gateway] section. I have a suspicion t

Re: [ceph-users] Openstack keystone with Radosgw

I did restart the ceph cluster only to see the ceph health to be NOT OK. I did the purge operation and re-installed ceph packages on all nodes. This time, ceph admin node has 0.80.6 and all other cluster nodes including Openstack client node have 0.80.5 version. Same error logs like before - 2

Re: [ceph-users] Openstack keystone with Radosgw

That's the same version that I'm using. Did you check the other points I mentioned: - check *all* ceph host are running the same version - restart 'em all to be sure I did think that your 'auth list' output looked strange, but I guessed that you have cut out the osd and mon info before placing

Re: [ceph-users] Openstack keystone with Radosgw

I have Ceph 0.85 version. I can still talk to this gateway node like below using swift v1.0. Note that this user was created using radosgw-admin.. swift -V 1.0 -A http://gateway.ex.com/auth/v1.0 -U s3User:swiftUser -K CRV8PeotaW204nE9IyutoVTcnr+2Uw8M8DQuRP7i list my-Test I am at total loss now

Re: [ceph-users] Openstack keystone with Radosgw

Well that certainly looks ok. So entries in [client.radosgw.gateway] *should* work. If they are not then that points to something else not setup right on the ceph or radosgw side. What version of ceph is this? I'd do the following: - check all ceph hosts have the same ceph version running - re

Re: [ceph-users] Openstack keystone with Radosgw

ceph auth list on gateway node has the following. I think I am using the correct name in ceph.conf. gateway@gateway:~$ ceph auth list installed auth entries: client.admin key: AQBL3SxUiMplMxAAjrL6oT+0Q5JtdrD90toXqg== caps: [mds] allow caps: [mon] allow * caps:

Re: [ceph-users] Openstack keystone with Radosgw

Ah, yes. So your gateway is called something other than: [client.radosgw.gateway] So take a look at what $ ceph auth list says (run from your rgw), it should pick up the correct name. Then correct your ceph.conf, restart and see what the rgw log looks like as you edge ever so closer to havin

Re: [ceph-users] Openstack keystone with Radosgw

Your rgw log strongly suggests that it is not even attempting to use keystone auth - did you restart it after changing the settings? FWIW, this is what my rgw log looks like for the same thing: 2014-10-12 17:34:34.907558 7f0477fe7700 20 SERVER_SOFTWARE=Apache/2.4.7 (Ubuntu) 2014-10-12 17:34:3

Re: [ceph-users] Openstack keystone with Radosgw

Right, well I suggest changing it back, and adding debug rgw = 20 in the [client.radosgw...] section of ceph.conf and capture the resulting log when you try 'swift stat'. It might reveal the next thing to check. Regards Mark On 11/10/14 16:02, lakshmi k s wrote: Hello Mark - I tried that

Re: [ceph-users] Openstack keystone with Radosgw

Hello Mark - I tried that as well, but in vain. In fact, that is how I created the endpoint to begin with. Since, that didn't work, I followed Openstack standard which was to include %tenant-id. -Lakshmi. On Friday, October 10, 2014 6:49 PM, Mark Kirkwood wrote: Hi, I think your swift

Re: [ceph-users] Openstack keystone with Radosgw

Hi, I think your swift endpoint: | 2ccd8523954c4491b08b648cfd42ae6c | regionOne | http://gateway.ex.com/swift/v1/AUTH_%(tenant_id)s | http://gateway.ex.com/swift/v1/AUTH_%(tenant_id)s | http://gateway.ex.com/swift/v1 | 77434bc194a3495793b5b4c943248e16 | is the issue. It should be: | 2c

Re: [ceph-users] Openstack keystone with Radosgw

With latest HA build, I found keystone_modwsgi.conf in /etc/apache2/sites-available and added the chunking like below. We have many controller nodes, but single virtual IP - 192.0.2.21 for which keystone is configured. I have verified keystone setup by executing other services like nova list, c

Re: [ceph-users] Openstack keystone with Radosgw

Mark, I am going no where with this. I am going to try with latest OpenStack build (build internal to my company) that has HA support. I will keep you posted. On Thursday, October 9, 2014 10:46 PM, Mark Kirkwood wrote: Oh, I see. That complicates it a wee bit (looks back at your messages)

Re: [ceph-users] Openstack keystone with Radosgw

Given your setup appears to be non standard, it might be useful to see the output of the 2 commands below: $ keystone service-list $ keystone endpoint-list So we can avoid advising you incorrectly. Regards Mark On 10/10/14 18:46, Mark Kirkwood wrote: Also just to double check - 192.0.8.2 *i

Re: [ceph-users] Openstack keystone with Radosgw

Oh, I see. That complicates it a wee bit (looks back at your messages). I see you have: rgw_keystone_url = http://192.0.8.2:5000 So you'll need to amend/create etc a and put it in there. I suspect you might be better off changing your rgw kesytone url to use port 35357 (the public one). How

Re: [ceph-users] Openstack keystone with Radosgw

Yes Mark, but there is no keystone.conf in this modified Openstack code. There is only horizon.conf under /etc/apache2/sites-available folder. And that has virtual host 80 only. Should I simply add :35357? root@overcloud-controller0-fjvtpqjip2hl:/etc/apache2/sites-available# ls 000-default.conf

Re: [ceph-users] Openstack keystone with Radosgw

Hmm - It looks to me like you added the chunked request into Horizon instead of Keystone. You want virtual host *:35357 On 10/10/14 12:32, lakshmi k s wrote: Have done this too, but in vain. I made changes to Horizon.conf as shown below. I had only I do not see the user being validated in rado

Re: [ceph-users] Openstack keystone with Radosgw

Have done this too, but in vain. I made changes to Horizon.conf as shown below. I had only I do not see the user being validated in radosgw log at all. root@overcloud-controller0-fjvtpqjip2hl:/etc/apache2/sites-available# ls 000-default.conf default-ssl.conf horizon.conf -

Re: [ceph-users] Openstack keystone with Radosgw

No, I don't have any explicit ssl enabled in the rgw site. Now you might be running into http://tracker.ceph.com/issues/7796 . So check if you have enabled WSGIChunkedRequest On In your keystone virtualhost setup (explained in the issue). Cheers Mark On 10/10/14 11:03, lakshmi k s wrote:

Re: [ceph-users] Openstack keystone with Radosgw

Right, I have these certs on both nodes - keystone node and rgw gateway node. Not sure where I am going wrong. And what about SSL? Should the following be in rgw.conf in gateway node? I am not using this as it was optional. SSLEngine on SSLCertificateFile /etc/apache2/ssl/apache.crt SSLCertifica

Re: [ceph-users] Openstack keystone with Radosgw

Almost - the converted certs need to be saved on your *rgw* host in nss_db_path (default is /var/ceph/nss but wherever you have it configured should be ok). Then restart the gateway. What is happening is the the rgw needs these certs to speak with encryption to the keystone server (the latter

Re: [ceph-users] Openstack keystone with Radosgw

Thanks Mark. I got past this error being root. So essentially, I copied the certs from openstack controller node to gateway node. Did the conversion using certutil and copied the files back to controller node under /var/lib/ceph/nss directory. Is this the correct directory? Ceph doc says /var/ce

Re: [ceph-users] Openstack keystone with Radosgw

I ran into this - needed to actually be root via sudo -i or similar, *then* it worked. Unhelpful error message is I think referring to no intialized db. On 09/10/14 16:36, lakshmi k s wrote: Good workaround. But it did not work. Not sure what this error is all about now. gateway@gateway:~$ op

Re: [ceph-users] Openstack keystone with Radosgw

Good workaround. But it did not work. Not sure what this error is all about now. gateway@gateway:~$ openssl x509 -in /home/gateway/ca.pem -pubkey | certutil -d /var/lib/ceph/nss -A -n ca -t "TCu,Cu,Tuw" certutil: function failed: SEC_ERROR_LEGACY_DATABASE: The certificate/key database is in an

Re: [ceph-users] Openstack keystone with Radosgw

As a workaround check if your rgw host has openssl and certutil installed, if so you can copy the relevant unconverted certs over to it and convert 'em there. On 09/10/14 15:07, lakshmi k s wrote: Tried aptitude as well, but no luck. Ceph users, have you tried to install libnss3-tools or cert

Re: [ceph-users] Openstack keystone with Radosgw

Tried aptitude as well, but no luck. Ceph users, have you tried to install libnss3-tools or certutil tool on debian/ubuntu? If so, how did you go about this problem. On Wednesday, October 8, 2014 7:01 PM, Mark Kirkwood wrote: Ok, so that is the thing to get sorted. I'd suggest posting

Re: [ceph-users] Openstack keystone with Radosgw

Ok, so that is the thing to get sorted. I'd suggest posting the error(s) you are getting perhaps here (someone else might know), but definitely to one of the Debian specific lists. In the meantime perhaps try installing the packages with aptitude rather than apt-get - if there is some fancy fo

Re: [ceph-users] Openstack keystone with Radosgw

Thanks Mark. I have been trying to install this on controller node. But for some reason, I am unable to install certutil or libnss3-tools on debian. I am not sure how to proceed. On Wednesday, October 8, 2014 6:26 PM, Mark Kirkwood wrote: If you are using ceph + radosgw packages they sho

Re: [ceph-users] Openstack keystone with Radosgw

If you are using ceph + radosgw packages they should be built with the nss option (--with-nss), so nothing to do there. For the server running keystone you need to do: (root) $ mkdir /var/ceph/nss (root) $ openssl x509 -in /etc/keystone/ssl/certs/ca.pem -pubkey | \ certutil -d /var/ceph

Re: [ceph-users] Openstack keystone with Radosgw

Hello Mark, Thanks for your reply. Where should I be installing NSS package? On Gateway or Openstack Controller node? On both, I could not execute the following command as it resulted in bunch of errors. openssl x509 -in /etc/keystone/ssl/certs/ca.pem -pubkey | certutil -d /var/ceph/nss -A -n

Re: [ceph-users] Openstack keystone with Radosgw

On 08/10/14 11:02, lakshmi k s wrote: I am trying to integrate OpenStack Keystone with Ceph Object Store using the link - http://ceph.com/docs/master/radosgw/keystone. Swift V1.0 (without keystone) works quite fine. But for some reason, Swift v2.0 ke

[ceph-users] Openstack keystone with Radosgw

I am trying to integrate OpenStack Keystone with Ceph Object Store using the link - http://ceph.com/docs/master/radosgw/keystone. Swift V1.0 (without keystone) works quite fine. But for some reason, Swift v2.0 keystone calls to Ceph Object Store always results in 401 - Unauthorized message. I ha