Orit, true. yeah, all my servers are running 10.2.3-1xenial or 10.2.3-1trusty. I have a small cluster and I always update all servers at once.
I don't have any Hammer releases of ceph anywhere on the network. Is 10.2.4 out already? I didn't see an update package to that. Thanks Andrei ----- Original Message ----- > From: "Orit Wasserman" <[email protected]> > To: "Andrei Mikhailovsky" <[email protected]> > Cc: "Yoann Moulin" <[email protected]>, "ceph-users" > <[email protected]> > Sent: Thursday, 10 November, 2016 13:58:32 > Subject: Re: [ceph-users] radosgw - http status 400 while creating a bucket > On Thu, Nov 10, 2016 at 2:55 PM, Andrei Mikhailovsky <[email protected]> > wrote: >> Orit, >> >> Here is what i've done just now: >> >> root@arh-ibstorage1-ib:~# service [email protected] stop >> >> (the above command was ran on both radosgw servers). Checked with ps and no >> radosgw services were running. After that I've done: >> >> >> >> root@arh-ibstorage1-ib:~# ./ceph-zones-fix.sh >> + RADOSGW_ADMIN=radosgw-admin >> + echo Exercise initialization code >> Exercise initialization code >> + radosgw-admin user info --uid=foo >> could not fetch user info: no user info saved >> + echo Get default zonegroup >> Get default zonegroup >> + radosgw-admin zonegroup get --rgw-zonegroup=default >> + sed s/"id":.*/"id": "default",/g >> + sed s/"master_zone.*/"master_zone": "default",/g >> + echo Get default zone >> Get default zone >> + radosgw-admin zone get --zone-id=default >> + echo Creating realm >> Creating realm >> + radosgw-admin realm create --rgw-realm=london-ldex >> ERROR: couldn't create realm london-ldex: (17) File exists >> 2016-11-10 13:44:48.872839 7f87a13d9a00 0 ERROR creating new realm object >> london-ldex: (17) File exists >> + echo Creating default zonegroup >> Creating default zonegroup >> + radosgw-admin zonegroup set --rgw-zonegroup=default >> { >> "id": "default", >> "name": "default", >> "api_name": "", >> "is_master": "true", >> "endpoints": [], >> "hostnames": [], >> "hostnames_s3website": [], >> "master_zone": "default", >> "zones": [ >> { >> "id": "default", >> "name": "default", >> "endpoints": [], >> "log_meta": "false", >> "log_data": "false", >> "bucket_index_max_shards": 0, >> "read_only": "false" >> } >> ], >> "placement_targets": [ >> { >> "name": "default-placement", >> "tags": [] >> } >> ], >> "default_placement": "default-placement", >> "realm_id": "5b41b1b2-0f92-463d-b582-07552f83e66c" >> } >> + echo Creating default zone >> Creating default zone >> + radosgw-admin zone set --rgw-zone=default >> zone id default{ >> "id": "default", >> "name": "default", >> "domain_root": ".rgw", >> "control_pool": ".rgw.control", >> "gc_pool": ".rgw.gc", >> "log_pool": ".log", >> "intent_log_pool": ".intent-log", >> "usage_log_pool": ".usage", >> "user_keys_pool": ".users", >> "user_email_pool": ".users.email", >> "user_swift_pool": ".users.swift", >> "user_uid_pool": ".users.uid", >> "system_key": { >> "access_key": "", >> "secret_key": "" >> }, >> "placement_pools": [ >> { >> "key": "default-placement", >> "val": { >> "index_pool": ".rgw.buckets.index", >> "data_pool": ".rgw.buckets", >> "data_extra_pool": "default.rgw.buckets.non-ec", >> "index_type": 0 >> } >> } >> ], >> "metadata_heap": ".rgw.meta", >> "realm_id": "5b41b1b2-0f92-463d-b582-07552f83e66c" >> } >> + echo Setting default zonegroup to 'default' >> Setting default zonegroup to 'default' >> + radosgw-admin zonegroup default --rgw-zonegroup=default >> + echo Setting default zone to 'default' >> Setting default zone to 'default' >> + radosgw-admin zone default --rgw-zone=default >> >> >> >> >> >> >> After that i've ran the following to make sure the details have been updated: >> >> >> >> >> root@arh-ibstorage1-ib:~# radosgw-admin zone get --rgw-zone=default >> { >> "id": "default", >> "name": "default", >> "domain_root": ".rgw", >> "control_pool": ".rgw.control", >> "gc_pool": ".rgw.gc", >> "log_pool": ".log", >> "intent_log_pool": ".intent-log", >> "usage_log_pool": ".usage", >> "user_keys_pool": ".users", >> "user_email_pool": ".users.email", >> "user_swift_pool": ".users.swift", >> "user_uid_pool": ".users.uid", >> "system_key": { >> "access_key": "", >> "secret_key": "" >> }, >> "placement_pools": [ >> { >> "key": "default-placement", >> "val": { >> "index_pool": ".rgw.buckets.index", >> "data_pool": ".rgw.buckets", >> "data_extra_pool": "default.rgw.buckets.non-ec", >> "index_type": 0 >> } >> } >> ], >> "metadata_heap": ".rgw.meta", >> "realm_id": "5b41b1b2-0f92-463d-b582-07552f83e66c" >> } >> >> >> >> root@arh-ibstorage1-ib:~# radosgw-admin zonegroup get --rgw-zonegroup=default >> { >> "id": "default", >> "name": "default", >> "api_name": "", >> "is_master": "true", >> "endpoints": [], >> "hostnames": [], >> "hostnames_s3website": [], >> "master_zone": "default", >> "zones": [ >> { >> "id": "default", >> "name": "default", >> "endpoints": [], >> "log_meta": "false", >> "log_data": "false", >> "bucket_index_max_shards": 0, >> "read_only": "false" >> } >> ], >> "placement_targets": [ >> { >> "name": "default-placement", >> "tags": [] >> } >> ], >> "default_placement": "default-placement", >> "realm_id": "5b41b1b2-0f92-463d-b582-07552f83e66c" >> } >> >> >> >> >> As far as I can see, the master_zone is now set to default. >> >> Now I start the radosgw service: >> >> >> root@arh-ibstorage1-ib:~# service [email protected] start >> root@arh-ibstorage1-ib:~# >> root@arh-ibstorage1-ib:~# >> root@arh-ibstorage1-ib:~# >> root@arh-ibstorage1-ib:~# >> root@arh-ibstorage1-ib:~# >> root@arh-ibstorage1-ib:~# radosgw-admin zone get --rgw-zone=default >> { >> "id": "default", >> "name": "default", >> "domain_root": ".rgw", >> "control_pool": ".rgw.control", >> "gc_pool": ".rgw.gc", >> "log_pool": ".log", >> "intent_log_pool": ".intent-log", >> "usage_log_pool": ".usage", >> "user_keys_pool": ".users", >> "user_email_pool": ".users.email", >> "user_swift_pool": ".users.swift", >> "user_uid_pool": ".users.uid", >> "system_key": { >> "access_key": "", >> "secret_key": "" >> }, >> "placement_pools": [ >> { >> "key": "default-placement", >> "val": { >> "index_pool": ".rgw.buckets.index", >> "data_pool": ".rgw.buckets", >> "data_extra_pool": "default.rgw.buckets.non-ec", >> "index_type": 0 >> } >> } >> ], >> "metadata_heap": ".rgw.meta", >> "realm_id": "5b41b1b2-0f92-463d-b582-07552f83e66c" >> >> >> root@arh-ibstorage1-ib:~# radosgw-admin zonegroup get --rgw-zonegroup=default >> { >> "id": "default", >> "name": "default", >> "api_name": "", >> "is_master": "true", >> "endpoints": [], >> "hostnames": [], >> "hostnames_s3website": [], >> "master_zone": "", >> "zones": [ >> { >> "id": "default", >> "name": "default", >> "endpoints": [], >> "log_meta": "false", >> "log_data": "false", >> "bucket_index_max_shards": 0, >> "read_only": "false" >> } >> ], >> "placement_targets": [ >> { >> "name": "default-placement", >> "tags": [] >> } >> ], >> "default_placement": "default-placement", >> "realm_id": "" >> } >> >> >> >> >> >> >> As you can see from above, the realm_id and the master_zone have reverted >> back >> to being blank! Crazy!!! >> > > Sad not crazy :( > Are both radosgw and all the radosgw-admin are from jewel? > I suspect you are hitting http://tracker.ceph.com/issues/17371 (it is > also true for infernalis) > It was fixed in 10.2.4 ... > >> >> Andrei >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> ----- Original Message ----- >>> From: "Orit Wasserman" <[email protected]> >>> To: "Andrei Mikhailovsky" <[email protected]> >>> Cc: "Yoann Moulin" <[email protected]>, "ceph-users" >>> <[email protected]> >>> Sent: Thursday, 10 November, 2016 13:36:01 >>> Subject: Re: [ceph-users] radosgw - http status 400 while creating a bucket >> >>> On Thu, Nov 10, 2016 at 2:24 PM, Andrei Mikhailovsky <[email protected]> >>> wrote: >>>> >>>> Hi Orit, >>>> >>>> Thanks for the links. >>>> >>>> I've had a look at the link that you've sent >>>> http://lists.ceph.com/pipermail/ceph-users-ceph.com/2016-July/011157.html >>>> and >>>> followed the instructions. Created the script as depicted in the email. >>>> Changed >>>> the realm name to something relevant. The script ran without any errors. >>>> I've >>>> restarted radosgw services on both servers, but I am still unable to create >>>> buckets. I am getting exactly the same error from the client: >>>> >>> >>> So you have two radosgw running on different servers, on the same zone >>> (default) and on the same rados cluster? >>> Are both gateways from the same version? >>> Did you shutdown both gateway when you run the script? >>> >>>> >>>> S3ResponseError: 400 Bad Request >>>> <?xml version="1.0" >>>> encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>my-new-bucket-31337</BucketName><RequestId>tx000000000000000000003-00582472a4-995ee8c-default</RequestId><HostId>995ee8c-default-default</HostId></Error> >>>> >>>> >>>> I can delete a bucket, but I can't create a new one. >>>> >>>> >>>> What I did notice is that after running the script and getting the >>>> zonegroup >>>> info, I do see both the master_zone and the realm_id fields are set: >>>> >>>> >>>> >>>> "master_zone": "default", >>>> "realm_id": "5b41b1b2-0f92-463d-b582-07552f83e66c" >>>> >>>> >>>> However, after I restart radosgw service, they go back to being blank: >>>> >>>> >>>> "master_zone": "", >>>> "realm_id": "" >>>> >>>> >>>> In any case, creation of buckets doesn't work either way. >>>> >>>> Cheers >>>> >>>> >>>> Andrei >>>> >>>> >>>> ----- Original Message ----- >>>>> From: "Orit Wasserman" <[email protected]> >>>>> To: "Yoann Moulin" <[email protected]> >>>>> Cc: "ceph-users" <[email protected]> >>>>> Sent: Thursday, 10 November, 2016 10:04:46 >>>>> Subject: Re: [ceph-users] radosgw - http status 400 while creating a >>>>> bucket >>>> >>>>> On Wed, Nov 9, 2016 at 10:20 PM, Yoann Moulin <[email protected]> >>>>> wrote: >>>>>> Hello, >>>>>> >>>>>>> many thanks for your help. I've tried setting the zone to master, >>>>>>> followed by >>>>>>> the period update --commit command. This is what i've had: >>>>>> >>>>>> maybe it's related to this issue : >>>>>> >>>>>> http://tracker.ceph.com/issues/16839 (fixe in Jewel 10.2.3) >>>>>> >>>>>> or this one : >>>>>> >>>>>> http://tracker.ceph.com/issues/17239 >>>>>> >>>>>> the "id" of the zonegroup shouldn't be "default" but an uuid afaik >>>>>> >>>>>> Best regards >>>>>> >>>>>> Yoann Moulin >>>>>> >>>>>>> root@arh-ibstorage1-ib:~# radosgw-admin zonegroup get >>>>>>> --rgw-zonegroup=default >>>>>>> { >>>>>>> "id": "default", >>>>>>> "name": "default", >>>>>>> "api_name": "", >>>>>>> "is_master": "true", >>>>>>> "endpoints": [], >>>>>>> "hostnames": [], >>>>>>> "hostnames_s3website": [], >>>>>>> "master_zone": "default", >>>>>>> "zones": [ >>>>>>> { >>>>>>> "id": "default", >>>>>>> "name": "default", >>>>>>> "endpoints": [], >>>>>>> "log_meta": "false", >>>>>>> "log_data": "false", >>>>>>> "bucket_index_max_shards": 0, >>>>>>> "read_only": "false" >>>>>>> } >>>>>>> ], >>>>>>> "placement_targets": [ >>>>>>> { >>>>>>> "name": "default-placement", >>>>>>> "tags": [] >>>>>>> } >>>>>>> ], >>>>>>> "default_placement": "default-placement", >>>>>>> "realm_id": "5b41b1b2-0f92-463d-b582-07552f83e66c" >>>>>>> } >>>>>>> >>>>>>> >>>>>>> root@arh-ibstorage1-ib:~# radosgw-admin period update --commit >>>>>>> cannot commit period: period does not have a master zone of a master >>>>>>> zonegroup >>>>>>> failed to commit period: (22) Invalid argument >>>>>>> >>>>>>> >>>>> >>>>> This is issue http://tracker.ceph.com/issues/17364 (I am working on it >>>>> at he moment). >>>>> >>>>> Do the same procedure as before without the period update --commit >>>>> command, >>>>> it should fix the master zone problem. >>>>> see also >>>>> http://lists.ceph.com/pipermail/ceph-users-ceph.com/2016-July/011157.html >>>>> >>>>> This looks like an upgraded system (the id equals the name after an >>>>> upgrade). >>>>> >>>>> Orit >>>>> >>>>>>> root@arh-ibstorage1-ib:~# radosgw-admin zonegroup get >>>>>>> --rgw-zonegroup=default >>>>>>> { >>>>>>> "id": "default", >>>>>>> "name": "default", >>>>>>> "api_name": "", >>>>>>> "is_master": "true", >>>>>>> "endpoints": [], >>>>>>> "hostnames": [], >>>>>>> "hostnames_s3website": [], >>>>>>> "master_zone": "", >>>>>>> "zones": [ >>>>>>> { >>>>>>> "id": "default", >>>>>>> "name": "default", >>>>>>> "endpoints": [], >>>>>>> "log_meta": "false", >>>>>>> "log_data": "false", >>>>>>> "bucket_index_max_shards": 0, >>>>>>> "read_only": "false" >>>>>>> } >>>>>>> ], >>>>>>> "placement_targets": [ >>>>>>> { >>>>>>> "name": "default-placement", >>>>>>> "tags": [] >>>>>>> } >>>>>>> ], >>>>>>> "default_placement": "default-placement", >>>>>>> "realm_id": "" >>>>>>> } >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> The strange thing as you can see, following the "radosgw-admin period >>>>>>> update >>>>>>> --commit" command, the master_zone and the realm_id values reset to >>>>>>> blank. What >>>>>>> could be causing this? >>>>>>> >>>>>>> Here is my ceph infrastructure setup, perhaps it will help with finding >>>>>>> the >>>>>>> issue?: >>>>>>> >>>>>>> ceph osd and mon servers: >>>>>>> arh-ibstorage1-ib (also radosgw server) >>>>>>> arh-ibstorage2-ib (also radosgw server) >>>>>>> arh-ibstorage3-ib >>>>>>> >>>>>>> ceph mon server: >>>>>>> arh-cloud13-ib >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thus, overall, i have 4 mon servers, 3 osd servers and 2 radosgw servers >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> >>>>>>> >>>>>>> ----- Original Message ----- >>>>>>>> From: "Yehuda Sadeh-Weinraub" <[email protected]> >>>>>>>> To: "Andrei Mikhailovsky" <[email protected]> >>>>>>>> Cc: "ceph-users" <[email protected]> >>>>>>>> Sent: Wednesday, 9 November, 2016 17:12:30 >>>>>>>> Subject: Re: [ceph-users] radosgw - http status 400 while creating a >>>>>>>> bucket >>>>>>> >>>>>>>> On Wed, Nov 9, 2016 at 1:30 AM, Andrei Mikhailovsky >>>>>>>> <[email protected]> wrote: >>>>>>>>> Hi Yehuda, >>>>>>>>> >>>>>>>>> just tried to run the command to set the master_zone to default >>>>>>>>> followed by the >>>>>>>>> bucket create without doing the restart and I still have the same >>>>>>>>> error on the >>>>>>>>> client: >>>>>>>>> >>>>>>>>> <?xml version="1.0" >>>>>>>>> encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>my-new-bucket-31337</BucketName><RequestId>tx000000000000000000010-005822ebbd-9951ad8-default</RequestId><HostId>9951ad8-default-default</HostId></Error> >>>>>>>>> >>>>>>>> >>>>>>>> After setting the master zone, try running: >>>>>>>> >>>>>>>> $ radosgw-admin period update --commit >>>>>>>> >>>>>>>> Yehuda >>>>>>>> >>>>>>>>> >>>>>>>>> Andrei >>>>>>>>> >>>>>>>>> ----- Original Message ----- >>>>>>>>>> From: "Yehuda Sadeh-Weinraub" <[email protected]> >>>>>>>>>> To: "Andrei Mikhailovsky" <[email protected]> >>>>>>>>>> Cc: "ceph-users" <[email protected]> >>>>>>>>>> Sent: Wednesday, 9 November, 2016 01:13:48 >>>>>>>>>> Subject: Re: [ceph-users] radosgw - http status 400 while creating a >>>>>>>>>> bucket >>>>>>>>> >>>>>>>>>> On Tue, Nov 8, 2016 at 5:05 PM, Andrei Mikhailovsky >>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>> Hi Yehuda, >>>>>>>>>>> >>>>>>>>>>> I don't have a multizone setup. The radosgw service was configured >>>>>>>>>>> about two >>>>>>>>>>> years ago according to the documentation on ceph.com and haven't >>>>>>>>>>> changed with >>>>>>>>>>> numerous version updates. All was working okay until i've upgraded >>>>>>>>>>> to version >>>>>>>>>>> 10.2.x. >>>>>>>>>>> >>>>>>>>>>> Could you please point me in the right direction what exactly needs >>>>>>>>>>> to be done? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> # radosgw-admin zonegroup get --rgw-zonegroup=default >>>>>>>>>>> { >>>>>>>>>>> "id": "default", >>>>>>>>>>> "name": "default", >>>>>>>>>>> "api_name": "", >>>>>>>>>>> "is_master": "true", >>>>>>>>>>> "endpoints": [], >>>>>>>>>>> "hostnames": [], >>>>>>>>>>> "hostnames_s3website": [], >>>>>>>>>>> "master_zone": "", >>>>>>>>>>> "zones": [ >>>>>>>>>>> { >>>>>>>>>>> "id": "default", >>>>>>>>>>> "name": "default", >>>>>>>>>>> "endpoints": [], >>>>>>>>>>> "log_meta": "false", >>>>>>>>>>> "log_data": "false", >>>>>>>>>>> "bucket_index_max_shards": 0, >>>>>>>>>>> "read_only": "false" >>>>>>>>>>> } >>>>>>>>>>> ], >>>>>>>>>>> "placement_targets": [ >>>>>>>>>>> { >>>>>>>>>>> "name": "default-placement", >>>>>>>>>>> "tags": [] >>>>>>>>>>> } >>>>>>>>>>> ], >>>>>>>>>>> "default_placement": "default-placement", >>>>>>>>>>> "realm_id": "" >>>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> Try: >>>>>>>>>> >>>>>>>>>> $ radosgw-admin zonegroup get --rgw-zonegroup=default > >>>>>>>>>> zonegroup.json >>>>>>>>>> >>>>>>>>>> ... modify the master_zone to be "default" >>>>>>>>>> >>>>>>>>>> $ radosgw-admin zonegroup set --rgw-zonegroup=default < >>>>>>>>>> zonegroup.json >>>>>>>>>> >>>>>>>>>> (restart radosgw) >>>>>>>>>> >>>>>>>>>> Yehuda >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> # radosgw-admin zone get --rgw-zone=default >>>>>>>>>>> { >>>>>>>>>>> "id": "default", >>>>>>>>>>> "name": "default", >>>>>>>>>>> "domain_root": ".rgw", >>>>>>>>>>> "control_pool": ".rgw.control", >>>>>>>>>>> "gc_pool": ".rgw.gc", >>>>>>>>>>> "log_pool": ".log", >>>>>>>>>>> "intent_log_pool": ".intent-log", >>>>>>>>>>> "usage_log_pool": ".usage", >>>>>>>>>>> "user_keys_pool": ".users", >>>>>>>>>>> "user_email_pool": ".users.email", >>>>>>>>>>> "user_swift_pool": ".users.swift", >>>>>>>>>>> "user_uid_pool": ".users.uid", >>>>>>>>>>> "system_key": { >>>>>>>>>>> "access_key": "", >>>>>>>>>>> "secret_key": "" >>>>>>>>>>> }, >>>>>>>>>>> "placement_pools": [ >>>>>>>>>>> { >>>>>>>>>>> "key": "default-placement", >>>>>>>>>>> "val": { >>>>>>>>>>> "index_pool": ".rgw.buckets.index", >>>>>>>>>>> "data_pool": ".rgw.buckets", >>>>>>>>>>> "data_extra_pool": "default.rgw.buckets.non-ec", >>>>>>>>>>> "index_type": 0 >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> ], >>>>>>>>>>> "metadata_heap": ".rgw.meta", >>>>>>>>>>> "realm_id": "5b41b1b2-0f92-463d-b582-07552f83e66c" >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ----- Original Message ----- >>>>>>>>>>>> From: "Yehuda Sadeh-Weinraub" <[email protected]> >>>>>>>>>>>> To: "Andrei Mikhailovsky" <[email protected]> >>>>>>>>>>>> Cc: "ceph-users" <[email protected]> >>>>>>>>>>>> Sent: Wednesday, 9 November, 2016 00:48:50 >>>>>>>>>>>> Subject: Re: [ceph-users] radosgw - http status 400 while creating >>>>>>>>>>>> a bucket >>>>>>>>>>> >>>>>>>>>>>> On Tue, Nov 8, 2016 at 3:36 PM, Andrei Mikhailovsky >>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>> Hello >>>>>>>>>>>>> >>>>>>>>>>>>> I am having issues with creating buckets in radosgw. It started >>>>>>>>>>>>> with an >>>>>>>>>>>>> upgrade to version 10.2.x >>>>>>>>>>>>> >>>>>>>>>>>>> When I am creating a bucket I get the following error on the >>>>>>>>>>>>> client side: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> boto.exception.S3ResponseError: S3ResponseError: 400 Bad Request >>>>>>>>>>>>> <?xml version="1.0" >>>>>>>>>>>>> encoding="UTF-8"?><Error><Code>InvalidArgument</Code><BucketName>my-new-bucket-31337</BucketName><RequestId>tx000000000000000000002-0058225bae-994d148-default</RequestId><HostId>994d148-default-default</HostId></Error> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> The radosgw logs are (redacted): >>>>>>>>>>>>> >>>>>>>>>>>>> ############################### >>>>>>>>>>>>> >>>>>>>>>>>>> 2016-11-08 23:11:42.876862 7f026d953700 20 enqueued request >>>>>>>>>>>>> req=0x7f02ba07b0e0 >>>>>>>>>>>>> 2016-11-08 23:11:42.876892 7f026d953700 20 RGWWQ: >>>>>>>>>>>>> 2016-11-08 23:11:42.876897 7f026d953700 20 req: 0x7f02ba07b0e0 >>>>>>>>>>>>> 2016-11-08 23:11:42.876912 7f026d953700 10 allocated request >>>>>>>>>>>>> req=0x7f02ba07b140 >>>>>>>>>>>>> 2016-11-08 23:11:42.876975 7f026b94f700 20 dequeued request >>>>>>>>>>>>> req=0x7f02ba07b0e0 >>>>>>>>>>>>> 2016-11-08 23:11:42.876987 7f026b94f700 20 RGWWQ: empty >>>>>>>>>>>>> 2016-11-08 23:11:42.877050 7f026b94f700 20 CONTENT_LENGTH=0 >>>>>>>>>>>>> 2016-11-08 23:11:42.877060 7f026b94f700 20 >>>>>>>>>>>>> CONTEXT_DOCUMENT_ROOT=/var/www >>>>>>>>>>>>> 2016-11-08 23:11:42.877062 7f026b94f700 20 CONTEXT_PREFIX= >>>>>>>>>>>>> 2016-11-08 23:11:42.877063 7f026b94f700 20 DOCUMENT_ROOT=/var/www >>>>>>>>>>>>> 2016-11-08 23:11:42.877081 7f026b94f700 20 FCGI_ROLE=RESPONDER >>>>>>>>>>>>> 2016-11-08 23:11:42.877083 7f026b94f700 20 >>>>>>>>>>>>> GATEWAY_INTERFACE=CGI/1.1 >>>>>>>>>>>>> 2016-11-08 23:11:42.877084 7f026b94f700 20 >>>>>>>>>>>>> HTTP_ACCEPT_ENCODING=identity >>>>>>>>>>>>> 2016-11-08 23:11:42.877086 7f026b94f700 20 HTTP_AUTHORIZATION=AWS >>>>>>>>>>>>> XXXXXXXEDITEDXXXXXX:EDITEDXXXXXXXeWyiacaN26GcME >>>>>>>>>>>>> 2016-11-08 23:11:42.877087 7f026b94f700 20 HTTP_DATE=Tue, 08 Nov >>>>>>>>>>>>> 2016 >>>>>>>>>>>>> 23:11:37 GMT >>>>>>>>>>>>> 2016-11-08 23:11:42.877088 7f026b94f700 20 >>>>>>>>>>>>> HTTP_HOST=s3service.editedname.com >>>>>>>>>>>>> 2016-11-08 23:11:42.877089 7f026b94f700 20 >>>>>>>>>>>>> HTTP_USER_AGENT=Boto/2.38.0 >>>>>>>>>>>>> Python/2.7.12 Linux/4.8.4-040804-generic >>>>>>>>>>>>> 2016-11-08 23:11:42.877090 7f026b94f700 20 HTTPS=on >>>>>>>>>>>>> 2016-11-08 23:11:42.877092 7f026b94f700 20 >>>>>>>>>>>>> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin >>>>>>>>>>>>> 2016-11-08 23:11:42.877093 7f026b94f700 20 proxy-nokeepalive=1 >>>>>>>>>>>>> 2016-11-08 23:11:42.877094 7f026b94f700 20 QUERY_STRING= >>>>>>>>>>>>> 2016-11-08 23:11:42.877095 7f026b94f700 20 >>>>>>>>>>>>> REMOTE_ADDR=192.168.169.91 >>>>>>>>>>>>> 2016-11-08 23:11:42.877096 7f026b94f700 20 REMOTE_PORT=45404 >>>>>>>>>>>>> 2016-11-08 23:11:42.877097 7f026b94f700 20 REQUEST_METHOD=PUT >>>>>>>>>>>>> 2016-11-08 23:11:42.877098 7f026b94f700 20 REQUEST_SCHEME=https >>>>>>>>>>>>> 2016-11-08 23:11:42.877099 7f026b94f700 20 >>>>>>>>>>>>> REQUEST_URI=/my-new-bucket-31337/ >>>>>>>>>>>>> 2016-11-08 23:11:42.877104 7f026b94f700 20 >>>>>>>>>>>>> SCRIPT_FILENAME=proxy:fcgi://localhost:9000/my-new-bucket-31337/ >>>>>>>>>>>>> 2016-11-08 23:11:42.877105 7f026b94f700 20 >>>>>>>>>>>>> SCRIPT_NAME=/my-new-bucket-31337/ >>>>>>>>>>>>> 2016-11-08 23:11:42.877107 7f026b94f700 20 >>>>>>>>>>>>> SCRIPT_URI=https://s3service.editedname.com/my-new-bucket-31337/ >>>>>>>>>>>>> 2016-11-08 23:11:42.877108 7f026b94f700 20 >>>>>>>>>>>>> SCRIPT_URL=/my-new-bucket-31337/ >>>>>>>>>>>>> 2016-11-08 23:11:42.877109 7f026b94f700 20 >>>>>>>>>>>>> SERVER_ADDR=192.168.169.201 >>>>>>>>>>>>> 2016-11-08 23:11:42.877110 7f026b94f700 20 >>>>>>>>>>>>> [email protected] >>>>>>>>>>>>> 2016-11-08 23:11:42.877111 7f026b94f700 20 >>>>>>>>>>>>> SERVER_NAME=s3service.editedname.com >>>>>>>>>>>>> 2016-11-08 23:11:42.877112 7f026b94f700 20 SERVER_PORT=443 >>>>>>>>>>>>> 2016-11-08 23:11:42.877113 7f026b94f700 20 >>>>>>>>>>>>> SERVER_PROTOCOL=HTTP/1.1 >>>>>>>>>>>>> 2016-11-08 23:11:42.877114 7f026b94f700 20 SERVER_SIGNATURE= >>>>>>>>>>>>> 2016-11-08 23:11:42.877115 7f026b94f700 20 >>>>>>>>>>>>> SERVER_SOFTWARE=Apache/2.4.18 >>>>>>>>>>>>> (Ubuntu) >>>>>>>>>>>>> 2016-11-08 23:11:42.877116 7f026b94f700 20 >>>>>>>>>>>>> SSL_TLS_SNI=s3service.editedname.com >>>>>>>>>>>>> 2016-11-08 23:11:42.877119 7f026b94f700 1 ====== starting new >>>>>>>>>>>>> request >>>>>>>>>>>>> req=0x7f02ba07b0e0 ===== >>>>>>>>>>>>> 2016-11-08 23:11:42.877155 7f026b94f700 2 req 2:0.000035::PUT >>>>>>>>>>>>> /my-new-bucket-31337/::initializing for trans_id = >>>>>>>>>>>>> tx000000000000000000002-0058225bae-994d148-default >>>>>>>>>>>>> 2016-11-08 23:11:42.877175 7f026b94f700 10 rgw api priority: s3=5 >>>>>>>>>>>>> s3website=4 >>>>>>>>>>>>> 2016-11-08 23:11:42.877179 7f026b94f700 10 >>>>>>>>>>>>> host=s3service.editedname.com >>>>>>>>>>>>> 2016-11-08 23:11:42.877199 7f026b94f700 20 subdomain= >>>>>>>>>>>>> domain=s3service.editedname.com in_hosted_domain=1 >>>>>>>>>>>>> in_hosted_domain_s3website=0 >>>>>>>>>>>>> 2016-11-08 23:11:42.877203 7f026b94f700 20 final domain/bucket >>>>>>>>>>>>> subdomain= >>>>>>>>>>>>> domain=s3service.editedname.com in_hosted_domain=1 >>>>>>>>>>>>> in_hosted_domain_s3website=0 >>>>>>>>>>>>> s->info.domain=s3service.editedname.com >>>>>>>>>>>>> s->info.request_uri=/my-new-bucket-31337/ >>>>>>>>>>>>> 2016-11-08 23:11:42.877277 7f026b94f700 20 get_handler >>>>>>>>>>>>> handler=25RGWHandler_REST_Bucket_S3 >>>>>>>>>>>>> 2016-11-08 23:11:42.877286 7f026b94f700 10 >>>>>>>>>>>>> handler=25RGWHandler_REST_Bucket_S3 >>>>>>>>>>>>> 2016-11-08 23:11:42.877291 7f026b94f700 2 req 2:0.000172:s3:PUT >>>>>>>>>>>>> /my-new-bucket-31337/::getting op 1 >>>>>>>>>>>>> 2016-11-08 23:11:42.877326 7f026b94f700 10 >>>>>>>>>>>>> op=27RGWCreateBucket_ObjStore_S3 >>>>>>>>>>>>> 2016-11-08 23:11:42.877334 7f026b94f700 2 req 2:0.000215:s3:PUT >>>>>>>>>>>>> /my-new-bucket-31337/:create_bucket:authorizing >>>>>>>>>>>>> 2016-11-08 23:11:42.877386 7f026b94f700 20 get_system_obj_state: >>>>>>>>>>>>> rctx=0x7f026b94b7c0 obj=.users:XXXXXXXEDITEDXXXXXX >>>>>>>>>>>>> state=0x7f02b70cdfe8 >>>>>>>>>>>>> s->prefetch_data=0 >>>>>>>>>>>>> 2016-11-08 23:11:42.877403 7f026b94f700 10 cache get: >>>>>>>>>>>>> name=.users+XXXXXXXEDITEDXXXXXX : miss >>>>>>>>>>>>> 2016-11-08 23:11:42.877516 7f026b94f700 1 -- >>>>>>>>>>>>> 192.168.168.201:0/3703211654 >>>>>>>>>>>>> --> 192.168.168.201:6829/893 -- osd_op(client.160747848.0:1019 >>>>>>>>>>>>> 16.e7d6c03f >>>>>>>>>>>>> XXXXXXXEDITEDXXXXXX [getxattrs,stat] snapc 0=[] >>>>>>>>>>>>> ack+read+known_if_red >>>>>>>>>>>>> irected e97148) v7 -- ?+0 0x7f02b70ab600 con 0x7f02c581a800 >>>>>>>>>>>>> 2016-11-08 23:11:42.879265 7f02e9405700 1 -- >>>>>>>>>>>>> 192.168.168.201:0/3703211654 >>>>>>>>>>>>> <== osd.3 192.168.168.201:6829/893 52 ==== osd_op_reply(1019 >>>>>>>>>>>>> XXXXXXXEDITEDXXXXXX [getxattrs,stat] v0'0 uv7 ondisk = 0) v7 ==== >>>>>>>>>>>>> 182+0+20 >>>>>>>>>>>>> (2521936 >>>>>>>>>>>>> 738 0 3070622072) 0x7f02c9897280 con 0x7f02c581a800 >>>>>>>>>>>>> 2016-11-08 23:11:42.879391 7f026b94f700 10 cache put: >>>>>>>>>>>>> name=.users+XXXXXXXEDITEDXXXXXX info.flags=6 >>>>>>>>>>>>> 2016-11-08 23:11:42.879421 7f026b94f700 10 adding >>>>>>>>>>>>> .users+XXXXXXXEDITEDXXXXXX >>>>>>>>>>>>> to cache LRU end >>>>>>>>>>>>> 2016-11-08 23:11:42.879442 7f026b94f700 20 get_system_obj_state: >>>>>>>>>>>>> s->obj_tag >>>>>>>>>>>>> was set empty >>>>>>>>>>>>> 2016-11-08 23:11:42.879452 7f026b94f700 10 cache get: >>>>>>>>>>>>> name=.users+XXXXXXXEDITEDXXXXXX : type miss (requested=1, >>>>>>>>>>>>> cached=6) >>>>>>>>>>>>> 2016-11-08 23:11:42.879461 7f026b94f700 20 get_system_obj_state: >>>>>>>>>>>>> rctx=0x7f026b94b7c0 obj=.users:XXXXXXXEDITEDXXXXXX >>>>>>>>>>>>> state=0x7f02b70cdfe8 >>>>>>>>>>>>> s->prefetch_data=0 >>>>>>>>>>>>> 2016-11-08 23:11:42.879480 7f026b94f700 20 rados->read ofs=0 >>>>>>>>>>>>> len=524288 >>>>>>>>>>>>> 2016-11-08 23:11:42.879533 7f026b94f700 1 -- >>>>>>>>>>>>> 192.168.168.201:0/3703211654 >>>>>>>>>>>>> --> 192.168.168.201:6829/893 -- osd_op(client.160747848.0:1020 >>>>>>>>>>>>> 16.e7d6c03f >>>>>>>>>>>>> XXXXXXXEDITEDXXXXXX [read 0~524288] snapc 0=[] >>>>>>>>>>>>> ack+read+known_if_redi >>>>>>>>>>>>> rected e97148) v7 -- ?+0 0x7f02b70ab980 con 0x7f02c581a800 >>>>>>>>>>>>> 2016-11-08 23:11:42.880343 7f02e9405700 1 -- >>>>>>>>>>>>> 192.168.168.201:0/3703211654 >>>>>>>>>>>>> <== osd.3 192.168.168.201:6829/893 53 ==== osd_op_reply(1020 >>>>>>>>>>>>> XXXXXXXEDITEDXXXXXX [read 0~37] v0'0 uv7 ondisk = 0) v7 ==== >>>>>>>>>>>>> 140+0+37 >>>>>>>>>>>>> (3791206703 0 >>>>>>>>>>>>> 1897156770) 0x7f02c9897280 con 0x7f02c581a800 >>>>>>>>>>>>> 2016-11-08 23:11:42.880433 7f026b94f700 20 rados->read r=0 >>>>>>>>>>>>> bl.length=37 >>>>>>>>>>>>> 2016-11-08 23:11:42.880464 7f026b94f700 10 cache put: >>>>>>>>>>>>> name=.users+XXXXXXXEDITEDXXXXXX info.flags=1 >>>>>>>>>>>>> 2016-11-08 23:11:42.880476 7f026b94f700 10 moving >>>>>>>>>>>>> .users+XXXXXXXEDITEDXXXXXX >>>>>>>>>>>>> to cache LRU end >>>>>>>>>>>>> 2016-11-08 23:11:42.880509 7f026b94f700 20 get_system_obj_state: >>>>>>>>>>>>> rctx=0x7f026b94b3f0 obj=.users.uid:EDITED - client name >>>>>>>>>>>>> state=0x7f02b70cede8 >>>>>>>>>>>>> s->prefetch_data=0 >>>>>>>>>>>>> 2016-11-08 23:11:42.880519 7f026b94f700 10 cache get: >>>>>>>>>>>>> name=.users.uid+EDITED >>>>>>>>>>>>> - client name : miss >>>>>>>>>>>>> 2016-11-08 23:11:42.880587 7f026b94f700 1 -- >>>>>>>>>>>>> 192.168.168.201:0/3703211654 >>>>>>>>>>>>> --> 192.168.168.203:6825/25725 -- osd_op(client.160747848.0:1021 >>>>>>>>>>>>> 15.d14b4318 >>>>>>>>>>>>> EDITED - client name [call version.read,getxattrs,stat] >>>>>>>>>>>>> snapc 0=[] ack+read+known_if_redirected e97148) v7 -- ?+0 >>>>>>>>>>>>> 0x7f02b70abd00 >>>>>>>>>>>>> con 0x7f02b9057180 >>>>>>>>>>>>> 2016-11-08 23:11:42.882156 7f02bbefe700 1 -- >>>>>>>>>>>>> 192.168.168.201:0/3703211654 >>>>>>>>>>>>> <== osd.27 192.168.168.203:6825/25725 33 ==== osd_op_reply(1021 >>>>>>>>>>>>> EDITED - >>>>>>>>>>>>> client name [call,getxattrs,stat] v0'0 uv14016 ondisk = 0) >>>>>>>>>>>>> v7 ==== 237+0+139 (1797997075 0 3786209825) 0x7f02bc847600 con >>>>>>>>>>>>> 0x7f02b9057180 >>>>>>>>>>>>> 2016-11-08 23:11:42.882322 7f026b94f700 10 cache put: >>>>>>>>>>>>> name=.users.uid+EDITED >>>>>>>>>>>>> - client name info.flags=22 >>>>>>>>>>>>> 2016-11-08 23:11:42.882353 7f026b94f700 10 adding >>>>>>>>>>>>> .users.uid+EDITED - client >>>>>>>>>>>>> name to cache LRU end >>>>>>>>>>>>> 2016-11-08 23:11:42.882363 7f026b94f700 20 get_system_obj_state: >>>>>>>>>>>>> s->obj_tag >>>>>>>>>>>>> was set empty >>>>>>>>>>>>> 2016-11-08 23:11:42.882374 7f026b94f700 10 cache get: >>>>>>>>>>>>> name=.users.uid+EDITED >>>>>>>>>>>>> - client name : type miss (requested=17, cached=22) >>>>>>>>>>>>> 2016-11-08 23:11:42.882383 7f026b94f700 20 get_system_obj_state: >>>>>>>>>>>>> rctx=0x7f026b94b3f0 obj=.users.uid:EDITED - client name >>>>>>>>>>>>> state=0x7f02b70cede8 >>>>>>>>>>>>> s->prefetch_data=0 >>>>>>>>>>>>> 2016-11-08 23:11:42.882427 7f026b94f700 20 rados->read ofs=0 >>>>>>>>>>>>> len=524288 >>>>>>>>>>>>> 2016-11-08 23:11:42.882492 7f026b94f700 1 -- >>>>>>>>>>>>> 192.168.168.201:0/3703211654 >>>>>>>>>>>>> --> 192.168.168.203:6825/25725 -- osd_op(client.160747848.0:1022 >>>>>>>>>>>>> 15.d14b4318 >>>>>>>>>>>>> EDITED - client name [call version.check_conds,call ver >>>>>>>>>>>>> sion.read,read 0~524288] snapc 0=[] ack+read+known_if_redirected >>>>>>>>>>>>> e97148) v7 >>>>>>>>>>>>> -- ?+0 0x7f02b70ac400 con 0x7f02b9057180 >>>>>>>>>>>>> 2016-11-08 23:11:42.883550 7f02bbefe700 1 -- >>>>>>>>>>>>> 192.168.168.201:0/3703211654 >>>>>>>>>>>>> <== osd.27 192.168.168.203:6825/25725 34 ==== osd_op_reply(1022 >>>>>>>>>>>>> EDITED - >>>>>>>>>>>>> client name [call,call,read 0~401] v0'0 uv14016 ondisk = 0) >>>>>>>>>>>>> v7 ==== 237+0+449 (131137144 0 372490823) 0x7f02bc847980 con >>>>>>>>>>>>> 0x7f02b9057180 >>>>>>>>>>>>> 2016-11-08 23:11:42.883653 7f026b94f700 20 rados->read r=0 >>>>>>>>>>>>> bl.length=401 >>>>>>>>>>>>> 2016-11-08 23:11:42.883695 7f026b94f700 10 cache put: >>>>>>>>>>>>> name=.users.uid+EDITED >>>>>>>>>>>>> - client name info.flags=17 >>>>>>>>>>>>> 2016-11-08 23:11:42.883698 7f026b94f700 10 moving >>>>>>>>>>>>> .users.uid+EDITED - client >>>>>>>>>>>>> name to cache LRU end >>>>>>>>>>>>> 2016-11-08 23:11:42.883726 7f026b94f700 10 chain_cache_entry: >>>>>>>>>>>>> cache_locator=.users.uid+EDITED - client name >>>>>>>>>>>>> 2016-11-08 23:11:42.883777 7f026b94f700 10 get_canon_resource(): >>>>>>>>>>>>> dest=/my-new-bucket-31337/ >>>>>>>>>>>>> 2016-11-08 23:11:42.883782 7f026b94f700 10 auth_hdr: >>>>>>>>>>>>> PUT >>>>>>>>>>>>> >>>>>>>>>>>>> Tue, 08 Nov 2016 23:11:37 GMT >>>>>>>>>>>>> /my-new-bucket-31337/ >>>>>>>>>>>>> 2016-11-08 23:11:42.883849 7f026b94f700 15 calculated >>>>>>>>>>>>> digest=EDITEDXXXXXXXeWyiacaN26GcME >>>>>>>>>>>>> 2016-11-08 23:11:42.883852 7f026b94f700 15 >>>>>>>>>>>>> auth_sign=EDITEDXXXXXXXeWyiacaN26GcME >>>>>>>>>>>>> 2016-11-08 23:11:42.883853 7f026b94f700 15 compare=0 >>>>>>>>>>>>> 2016-11-08 23:11:42.883858 7f026b94f700 2 req 2:0.006739:s3:PUT >>>>>>>>>>>>> /my-new-bucket-31337/:create_bucket:normalizing buckets and >>>>>>>>>>>>> tenants >>>>>>>>>>>>> 2016-11-08 23:11:42.883864 7f026b94f700 10 s->object=<NULL> >>>>>>>>>>>>> s->bucket=my-new-bucket-31337 >>>>>>>>>>>>> 2016-11-08 23:11:42.883869 7f026b94f700 2 req 2:0.006750:s3:PUT >>>>>>>>>>>>> /my-new-bucket-31337/:create_bucket:init permissions >>>>>>>>>>>>> 2016-11-08 23:11:42.883872 7f026b94f700 2 req 2:0.006753:s3:PUT >>>>>>>>>>>>> /my-new-bucket-31337/:create_bucket:recalculating target >>>>>>>>>>>>> 2016-11-08 23:11:42.883875 7f026b94f700 2 req 2:0.006756:s3:PUT >>>>>>>>>>>>> /my-new-bucket-31337/:create_bucket:reading permissions >>>>>>>>>>>>> 2016-11-08 23:11:42.883879 7f026b94f700 2 req 2:0.006760:s3:PUT >>>>>>>>>>>>> /my-new-bucket-31337/:create_bucket:init op >>>>>>>>>>>>> 2016-11-08 23:11:42.883884 7f026b94f700 2 req 2:0.006765:s3:PUT >>>>>>>>>>>>> /my-new-bucket-31337/:create_bucket:verifying op mask >>>>>>>>>>>>> 2016-11-08 23:11:42.883886 7f026b94f700 20 required_mask= 2 >>>>>>>>>>>>> user.op_mask=7 >>>>>>>>>>>>> 2016-11-08 23:11:42.883887 7f026b94f700 2 req 2:0.006769:s3:PUT >>>>>>>>>>>>> /my-new-bucket-31337/:create_bucket:verifying op permissions >>>>>>>>>>>>> 2016-11-08 23:11:42.883939 7f026b94f700 1 -- >>>>>>>>>>>>> 192.168.168.201:0/3703211654 >>>>>>>>>>>>> --> 192.168.168.203:6840/36134 -- osd_op(client.160747848.0:1023 >>>>>>>>>>>>> 15.efcbc969 >>>>>>>>>>>>> EDITED - client name.buckets [call user.list_buckets] snapc 0=[] >>>>>>>>>>>>> ack+read+known_if_redirected e97148) v7 -- ?+0 0x7f02b70acb00 con >>>>>>>>>>>>> 0x7f02cd43a900 >>>>>>>>>>>>> 2016-11-08 23:11:42.885157 7f02bbfff700 1 -- >>>>>>>>>>>>> 192.168.168.201:0/3703211654 >>>>>>>>>>>>> <== osd.25 192.168.168.203:6840/36134 37 ==== osd_op_reply(1023 >>>>>>>>>>>>> EDITED - >>>>>>>>>>>>> client name.buckets [call] v0'0 uv0 ack = -2 ((2) No such file or >>>>>>>>>>>>> directory)) v7 ==== 161+0+0 (2433927993 0 0) 0x7f02b886d280 con >>>>>>>>>>>>> 0x7f02cd43a900 >>>>>>>>>>>>> 2016-11-08 23:11:42.885307 7f026b94f700 2 req 2:0.008187:s3:PUT >>>>>>>>>>>>> /my-new-bucket-31337/:create_bucket:verifying op params >>>>>>>>>>>>> 2016-11-08 23:11:42.885338 7f026b94f700 2 req 2:0.008219:s3:PUT >>>>>>>>>>>>> /my-new-bucket-31337/:create_bucket:pre-executing >>>>>>>>>>>>> 2016-11-08 23:11:42.885342 7f026b94f700 2 req 2:0.008223:s3:PUT >>>>>>>>>>>>> /my-new-bucket-31337/:create_bucket:executing >>>>>>>>>>>>> 2016-11-08 23:11:42.885377 7f026b94f700 20 get_system_obj_state: >>>>>>>>>>>>> rctx=0x7f026b94c6d0 obj=.rgw:my-new-bucket-31337 >>>>>>>>>>>>> state=0x7f02b70cdfe8 >>>>>>>>>>>>> s->prefetch_data=0 >>>>>>>>>>>>> 2016-11-08 23:11:42.885390 7f026b94f700 10 cache get: >>>>>>>>>>>>> name=.rgw+my-new-bucket-31337 : miss >>>>>>>>>>>>> 2016-11-08 23:11:42.885444 7f026b94f700 1 -- >>>>>>>>>>>>> 192.168.168.201:0/3703211654 >>>>>>>>>>>>> --> 192.168.168.202:6821/14483 -- osd_op(client.160747848.0:1024 >>>>>>>>>>>>> 13.501be1c3 >>>>>>>>>>>>> my-new-bucket-31337 [call version.read,getxattrs,stat] snapc 0=[] >>>>>>>>>>>>> ack+read+known_if_redirected e97148) v7 -- ?+0 0x7f02b70ace80 con >>>>>>>>>>>>> 0x7f02cd43b980 >>>>>>>>>>>>> 2016-11-08 23:11:42.886940 7f02bfefe700 1 -- >>>>>>>>>>>>> 192.168.168.201:0/3703211654 >>>>>>>>>>>>> <== osd.11 192.168.168.202:6821/14483 11 ==== osd_op_reply(1024 >>>>>>>>>>>>> my-new-bucket-31337 [call,getxattrs,stat] v0'0 uv0 ack = -2 ((2) >>>>>>>>>>>>> No such >>>>>>>>>>>>> file or directory)) v7 ==== 223+0+0 (4048274385 0 0) >>>>>>>>>>>>> 0x7f02bf480280 con >>>>>>>>>>>>> 0x7f02cd43b980 >>>>>>>>>>>>> 2016-11-08 23:11:42.887120 7f026b94f700 10 cache put: >>>>>>>>>>>>> name=.rgw+my-new-bucket-31337 info.flags=0 >>>>>>>>>>>>> 2016-11-08 23:11:42.887135 7f026b94f700 10 adding >>>>>>>>>>>>> .rgw+my-new-bucket-31337 >>>>>>>>>>>>> to cache LRU end >>>>>>>>>>>>> 2016-11-08 23:11:42.887144 7f026b94f700 0 rest connection is >>>>>>>>>>>>> invalid >>>>>>>>>>>> >>>>>>>>>>>> Did you configure multi-site? Seems to me that it tries to connect >>>>>>>>>>>> to >>>>>>>>>>>> a master zone, so if you have a single zonegroup, single zone >>>>>>>>>>>> configuration you need to make sure that the zone (and zonegroup) >>>>>>>>>>>> are >>>>>>>>>>>> set to be the master. >>>>>>>>>>>> >>>>>>>>>>>> Yehuda >>>>>>>>>>>> >>>>>>>>>>>>> 2016-11-08 23:11:42.887149 7f026b94f700 2 req 2:0.010030:s3:PUT >>>>>>>>>>>>> /my-new-bucket-31337/:create_bucket:completing >>>>>>>>>>>>> 2016-11-08 23:11:42.887217 7f026b94f700 2 req 2:0.010098:s3:PUT >>>>>>>>>>>>> /my-new-bucket-31337/:create_bucket:op status=-22 >>>>>>>>>>>>> 2016-11-08 23:11:42.887221 7f026b94f700 2 req 2:0.010102:s3:PUT >>>>>>>>>>>>> /my-new-bucket-31337/:create_bucket:http status=400 >>>>>>>>>>>>> 2016-11-08 23:11:42.887244 7f026b94f700 1 ====== req done >>>>>>>>>>>>> req=0x7f02ba07b0e0 op status=-22 http_status=400 ====== >>>>>>>>>>>>> 2016-11-08 23:11:42.887254 7f026b94f700 20 process_request() >>>>>>>>>>>>> returned -22 >>>>>>>>>>>>> >>>>>>>>>>>>> ############################### >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Apache proxy logs show the following: >>>>>>>>>>>>> >>>>>>>>>>>>> 192.168.169.91 - - [08/Nov/2016:23:11:42 +0000] "PUT >>>>>>>>>>>>> /my-new-bucket-31337/ >>>>>>>>>>>>> HTTP/1.1" 400 4379 "-" "Boto/2.38.0 Python/2.7.12 >>>>>>>>>>>>> Linux/4.8.4-040804-generic" >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> The existing buckets work perfectly well, i can list and put >>>>>>>>>>>>> objects. It's >>>>>>>>>>>>> the creation of new buckets that i am having issues with. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Could someone please help me to figure out what the issue is? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks >>>>>>>>>>>>> >>>>>>>>>>>>> Andrei >>>>>>>>>>>>> >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> ceph-users mailing list >>>>>>>>>>>>> [email protected] >>>>>>>>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>>>>>> _______________________________________________ >>>>>>> ceph-users mailing list >>>>>>> [email protected] >>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Yoann Moulin >>>>>> EPFL IC-IT >>>>> _______________________________________________ >>>>> ceph-users mailing list >>>>> [email protected] > >> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
