UPDATE:
I have installed Tengine (nginx fork) and configured both HTTP and HTTPS to
use radosgw socket.
I can login with S3, create buckets and upload objects.
It's still not possible to use Swift credentials, can you help me on this
part? What do I use when I login (url, username, password) ?
Here is the info for the user:
radosgw-admin user info --uid=mgarces
{ "user_id": "mgarces",
"display_name": "Marco Garces",
"email": "[email protected]",
"suspended": 0,
"max_buckets": 1000,
"auid": 0,
"subusers": [
{ "id": "mgarces:swift",
"permissions": "full-control"}],
"keys": [
{ "user": "mgarces:swift",
"access_key": "AJW2BCBXHFJ1DPXT112O",
"secret_key": ""},
{ "user": "mgarces",
"access_key": "S88Y6ZJRACZG49JFPY83",
"secret_key": "PlubMMjfQecJ5Py46e2kZz5VuUgHgsjLmYZDRdFg"}],
"swift_keys": [
{ "user": "mgarces:swift",
"secret_key": "TtKWhY67ujhjn36\/nhv44A2BVPw5wDi3Sp13YrMM"}],
"caps": [],
"op_mask": "read, write, delete",
"default_placement": "",
"placement_tags": [],
"bucket_quota": { "enabled": false,
"max_size_kb": -1,
"max_objects": -1},
"user_quota": { "enabled": false,
"max_size_kb": -1,
"max_objects": -1},
"temp_url_keys": []}
Thank you in advance.
Marco Garcês
On Tue, Aug 19, 2014 at 10:59 AM, Marco Garcês <[email protected]> wrote:
> I have added the options as suggested, but no success yet!
>
> Im also running radosgw manually (radosgw -c /etc/ceph/ceph.conf -n
> client.radosgw.gw --rgw-frontends "civetweb port=80") using civetweb, and
> I still cant login with Swift, and S3 uploads are broken.
> Somenone on #ceph mention that ceph-radosgw was broken on 0.80.5, can
> someone confirm this?
>
> Tanks you once again, hope to solve this soon.
> Marco Garcês
>
> On Mon, Aug 18, 2014 at 3:23 PM, Marco Garcês <[email protected]> wrote:
>
>> Hi Kurt,
>>
>> I have pointed my DNS '*.gateway.testes.local' and 'gateway.testes.local,
>> to the same IP (the radosgw server).
>>
>> I have added rgw_dns_name has you suggested to the config (it was comment
>> out). I will try everything and give feedback.
>>
>> By the way, when I restart ceph-radosgw service, I get this in the logs
>> (which previous I did not see anything):
>>
>> 2014-08-18 15:19:44.812039 7fbf417fa700 1 handle_sigterm
>> 2014-08-18 15:19:44.812104 7fbf417fa700 1 handle_sigterm set alarm for
>> 120
>> 2014-08-18 15:19:44.812235 7fbf5c495880 -1 shutting down
>> 2014-08-18 15:19:44.812305 7fbf40ff9700 0 ERROR: FCGX_Accept_r returned
>> -4
>> 2014-08-18 15:19:44.812432 7fbf417fa700 1 handle_sigterm
>> 2014-08-18 15:19:44.857506 7fbf5c495880 1 final shutdown
>> 2014-08-18 15:19:45.010597 7fb318b96880 0 ceph version 0.80.5
>> (38b73c67d375a2552d8ed67843c8a65c2c0feba6), process radosgw, pid 3242
>> 2014-08-18 15:19:45.219582 7fb318b96880 0 framework: fastcgi
>> 2014-08-18 15:19:45.219599 7fb318b96880 0 starting handler: fastcgi
>> 2014-08-18 15:19:45.692248 7fb2fe6fb700 0 ERROR: can't read user header:
>> ret=-2
>> 2014-08-18 15:19:45.692273 7fb2fe6fb700 0 ERROR: sync_user() failed,
>> user=teste ret=-2
>>
>> The last 2 lines look suspicious...
>>
>>
>>
>>
>> On Mon, Aug 18, 2014 at 2:58 PM, Bachelder, Kurt <
>> [email protected]> wrote:
>>
>>> Hi Marco,
>>>
>>> Is your DNS setup to use the wildcard (*.gateway.testes.local)?
>>>
>>> I noticed that you're using it in the server alias, but that you don't
>>> have an "rgw_dns_name" configured in your ceph.conf. The rgw_dns_name
>>> should be set to "gateway.testes.local" if your dns is configured to use
>>> the wildcard naming with that subdomain.
>>>
>>> Also see that you're using SSL... which domain have you signed?
>>> *.gateway.testes.local?
>>>
>>> Since you can create a bucket, but not write to it, I'm wondering if
>>> there's an issue with the way your client is attempting to access the
>>> bucket... can you resolve <bucket>.gateway.testes.local from your client?
>>>
>>> Kurt
>>>
>>>
>>> -------- Original message --------
>>> From: Marco Garcês
>>> Date:08/18/2014 6:33 AM (GMT-05:00)
>>> To: Linux Chips
>>> Cc: "Bachelder, Kurt" , [email protected]
>>> Subject: Re: [ceph-users] RadosGW problems
>>>
>>> Hi there,
>>>
>>> I have FastCgiWrapper Off in fastcgi.conf file; I also have SELinux in
>>> permissive state; 'ps aux | grep rados' shows me radosgw is running;
>>>
>>> The problems stays the same... I can login with S3 credentials, create
>>> buckets, but uploads write this in the logs:
>>> [Mon Aug 18 12:00:28.636378 2014] [:error] [pid 11251] [client
>>> 10.5.1.1:49680] FastCGI: comm with server "/var/www/cgi-bin/s3gw.fcgi"
>>> aborted: idle timeout (3
>>> 0 sec)
>>> [Mon Aug 18 12:00:28.676825 2014] [:error] [pid 11251] [client
>>> 10.5.1.1:49680] FastCGI: incomplete headers (0 bytes) received from
>>> server "/var/www/cgi-bin/s3
>>> gw.fcgi"
>>>
>>> When I try Swift credentials, I cannot login at all.. I have tested
>>> both Cyberduck and Swift client on the command line, and I always get this
>>> on the logs:
>>> "GET /v1.0 HTTP/1.1" 404 78 "-" "Cyberduck/4.5 (Mac OS X/10.9.3)
>>> (x86_64)"
>>> "GET /v1.0 HTTP/1.1" 404 78 "-" "python-swiftclient-2.2.0"
>>>
>>> In S3 login, when I upload a file, I can see it almost at 100%
>>> complete, but then it fails with the above errors.
>>>
>>> A strange thing is... the /var/log/ceph/client.radosgw.gateway.log is
>>> not getting updated, I don't see any new logs in there.
>>>
>>> Thank you once again for your help, Marco Garcês
>>>
>>>
>>>
>>>
>>> On Mon, Aug 18, 2014 at 12:08 AM, Linux Chips <[email protected]>
>>> wrote:
>>>
>>>> On Mon 18 Aug 2014 12:45:33 AM AST, Bachelder, Kurt wrote:
>>>>
>>>>> Hi Marco –
>>>>>
>>>>> In CentOS 6, you also had to edit /etc/httpd/conf.d/fastcgi.conf to
>>>>> turn OFF the fastcgi wrapper. I haven’t tested in v7 yet, but I’d
>>>>> guess it’s required there too:
>>>>>
>>>>> # wrap all fastcgi script calls in suexec
>>>>>
>>>>> FastCgiWrapper Off
>>>>>
>>>>> Give that a try, if you haven’t already – restart httpd and
>>>>> ceph-radosgw afterward.
>>>>>
>>>>> Kurt
>>>>>
>>>>> *From:*ceph-users [mailto:[email protected]] *On
>>>>> Behalf Of *Marco Garcês
>>>>> *Sent:* Friday, August 15, 2014 12:46 PM
>>>>> *To:* [email protected]
>>>>> *Subject:* [ceph-users] RadosGW problems
>>>>>
>>>>>
>>>>> Hi there,
>>>>>
>>>>> I am using CentOS 7 with Ceph version 0.80.5
>>>>> (38b73c67d375a2552d8ed67843c8a65c2c0feba6), 3 OSD, 3 MON, 1 RadosGW
>>>>> (which also serves as ceph-deploy node)
>>>>>
>>>>> I followed all the instructions in the docs, regarding setting up a
>>>>> basic Ceph cluster, and then followed the one to setup RadosGW.
>>>>>
>>>>> I can't seem to use the Swift interface, and the S3 interface, times
>>>>> out after 30 seconds.
>>>>>
>>>>> [Fri Aug 15 18:25:33.290877 2014] [:error] [pid 6197] [client
>>>>> 10.5.5.222:58051 <http://10.5.5.222:58051>] FastCGI: comm with
>>>>> server
>>>>>
>>>>> "/var/www/cgi-bin/s3gw.fcgi" aborted: idle timeout (30 sec)
>>>>>
>>>>> [Fri Aug 15 18:25:33.291781 2014] [:error] [pid 6197] [client
>>>>> 10.5.5.222:58051 <http://10.5.5.222:58051>] FastCGI: incomplete
>>>>>
>>>>> headers (0 bytes) received from server "/var/www/cgi-bin/s3gw.fcgi"
>>>>>
>>>>> *My ceph.conf:*
>>>>>
>>>>>
>>>>> [global]
>>>>>
>>>>> fsid = 581bcd61-8760-4756-a7c8-e8275c0957ad
>>>>>
>>>>> mon_initial_members = CEPH01, CEPH02, CEPH03
>>>>>
>>>>> mon_host = 10.2.27.81,10.2.27.82,10.2.27.83
>>>>>
>>>>> public network = 10.2.27.0/25 <http://10.2.27.0/25>
>>>>>
>>>>>
>>>>> auth_cluster_required = cephx
>>>>>
>>>>> auth_service_required = cephx
>>>>>
>>>>> auth_client_required = cephx
>>>>>
>>>>> filestore_xattr_use_omap = true
>>>>>
>>>>> osd pool default size = 2
>>>>>
>>>>> osd pool default pg num = 333
>>>>>
>>>>> osd pool default pgp num = 333
>>>>>
>>>>> osd journal size = 1024
>>>>>
>>>>> [client.radosgw.gateway]
>>>>>
>>>>> host = GATEWAY
>>>>>
>>>>> keyring = /etc/ceph/ceph.client.radosgw.keyring
>>>>>
>>>>> rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
>>>>>
>>>>> log file = /var/log/ceph/client.radosgw.gateway.log
>>>>>
>>>>> rgw print continue = false
>>>>>
>>>>> rgw enable ops log = true
>>>>>
>>>>> *My apache rgw.conf:*
>>>>>
>>>>>
>>>>> FastCgiExternalServer /var/www/cgi-bin/s3gw.fcgi -socket
>>>>> /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
>>>>>
>>>>> <VirtualHost *:443>
>>>>>
>>>>> SSLEngine on
>>>>>
>>>>> SSLCertificateFile /etc/pki/tls/certs/ca_rgw.crt
>>>>>
>>>>> SSLCertificateKeyFile /etc/pki/tls/private/ca_rgw.key
>>>>>
>>>>> SetEnv SERVER_PORT_SECURE 443
>>>>>
>>>>> ServerName gateway.testes.local
>>>>>
>>>>> ServerAlias *.gateway.testes.local
>>>>>
>>>>> ServerAdmin [email protected]
>>>>> <mailto:[email protected]>
>>>>>
>>>>>
>>>>> DocumentRoot /var/www/cgi-bin
>>>>>
>>>>> RewriteEngine On
>>>>>
>>>>> #RewriteRule ^/(.*) /s3gw.fcgi?%{QUERY_STRING}
>>>>> [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]
>>>>>
>>>>> RewriteRule ^/([a-zA-Z0-9-_.]*)([/]?.*)
>>>>> /s3gw.fcgi?page=$1¶ms=$2&%{QUERY_STRING}
>>>>> [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]
>>>>>
>>>>> <IfModule mod_fastcgi.c>
>>>>>
>>>>> <Directory /var/www>
>>>>>
>>>>> Options +ExecCGI
>>>>>
>>>>> AllowOverride All
>>>>>
>>>>> SetHandler fastcgi-script
>>>>>
>>>>> Order allow,deny
>>>>>
>>>>> Allow from all
>>>>>
>>>>> AuthBasicAuthoritative Off
>>>>>
>>>>> </Directory>
>>>>>
>>>>> </IfModule>
>>>>>
>>>>> AllowEncodedSlashes On
>>>>>
>>>>> ErrorLog /var/log/httpd/error_rgw_ssl.log
>>>>>
>>>>> CustomLog /var/log/httpd/access_rgw_ssl.log combined
>>>>>
>>>>> ServerSignature Off
>>>>>
>>>>> </VirtualHost>
>>>>>
>>>>> *My /var/www/cgi-bin/s3gw.fcgi *
>>>>>
>>>>>
>>>>> #!/bin/sh
>>>>>
>>>>> exec /usr/bin/radosgw -c /etc/ceph/ceph.conf -n client.radosgw.gateway
>>>>>
>>>>> *My Rados user:*
>>>>>
>>>>>
>>>>> radosgw-admin user info --uid=johndoe
>>>>>
>>>>> { "user_id": "johndoe",
>>>>>
>>>>> "display_name": "John Doe",
>>>>>
>>>>> "email": "[email protected] <mailto:[email protected]>",
>>>>>
>>>>>
>>>>> "suspended": 0,
>>>>>
>>>>> "max_buckets": 1000,
>>>>>
>>>>> "auid": 0,
>>>>>
>>>>> "subusers": [
>>>>>
>>>>> { "id": "johndoe:swift",
>>>>>
>>>>> "permissions": "full-control"}],
>>>>>
>>>>> "keys": [
>>>>>
>>>>> { "user": "johndoe:swift",
>>>>>
>>>>> "access_key": "265DJESOJGSK953EE4LE",
>>>>>
>>>>> "secret_key": ""},
>>>>>
>>>>> { "user": "johndoe",
>>>>>
>>>>> "access_key": "U4AR5757MCON3AZYAB97",
>>>>>
>>>>> "secret_key": "05rg47Oa+njo8uxTeX+urBPF0ZRPWvVq8nfrC5cN"}],
>>>>>
>>>>> "swift_keys": [
>>>>>
>>>>> { "user": "johndoe:swift",
>>>>>
>>>>> "secret_key": "Lags5xwX5aiPgkG\/QqA8HygKs6AQYO46dBXS0ZGS"}],
>>>>>
>>>>> "caps": [],
>>>>>
>>>>> "op_mask": "read, write, delete",
>>>>>
>>>>> "default_placement": "",
>>>>>
>>>>> "placement_tags": [],
>>>>>
>>>>> "bucket_quota": { "enabled": false,
>>>>>
>>>>> "max_size_kb": -1,
>>>>>
>>>>> "max_objects": -1},
>>>>>
>>>>> "user_quota": { "enabled": false,
>>>>>
>>>>> "max_size_kb": -1,
>>>>>
>>>>> "max_objects": -1},
>>>>>
>>>>> "temp_url_keys": []}
>>>>>
>>>>> I can reach https://gateway.testes.local, and I can login with S3, but
>>>>> cant login with Swift (using Cyberduck). Also, I can create buckets
>>>>> with S3, but if I upload a file, it times out with the error above.
>>>>> There is a necessity to use both the S3 and Swift API.
>>>>>
>>>>> Can you help me? Thank you in advance, regards, Marco Garcês
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> ceph-users mailing list
>>>>> [email protected]
>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>>>
>>>>
>>>> this error appears when httpd can not talk to radosgw process. dose it
>>>> happen with all the requests or some of them? generally if it worked with
>>>> one request it would with all of them, except when it is under load. so if
>>>> you can create a bucket you should be able to upload files.
>>>> i did not try centos7, but it may be similar, as we use 6. may be it is
>>>> a permission issue (if it happens all the time). try turning selinux to
>>>> permissive and see (setenforce 0), make sure the file is readable/writable
>>>> by both the radosgw and httpd. also confirm that radosgw is running (ps aux
>>>> | grep rados) as it tends to silently die if some thing is not correct, it
>>>> drove me nuts until i figured it was the log file that it can not write to.
>>>> if you have high number of connection you would also need to patch fcgi
>>>> to use poll instead of select. that was the only way we could push it to
>>>> serve more than 900 users.
>>>> we have all our radosgws running both swift and s3 with no special
>>>> config, it just worked.
>>>>
>>>
>>>
>>
>
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
