Re: [ceph-users] Multipart Upload - POST fails

2018-03-07 Thread Ingo Reimann
No-one?


-Ursprüngliche Nachricht-
Von: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] Im Auftrag von
Ingo Reimann
Gesendet: Freitag, 2. März 2018 14:15
An: ceph-users
Betreff: [ceph-users] Multipart Upload - POST fails

Hi,

we discovered some problem with our installation - Multipart upload is not
working.

What we did:
* tried upload with cyberduck as well as with script from
http://tracker.ceph.com/issues/12790
* tried against jewel gateways and luminous gateways from old cluster
* tried against 12.2.4 gateway with jewel-era cluster

Surprisingly this is no signature problem as in the issue above, instead I
get the following in the logs:

2018-03-02 13:59:04.927353 7fe2053ca700  1 == starting new request
req=0x7fe2053c42c0 =
2018-03-02 13:59:04.927383 7fe2053ca700  2 req 61:0.30::POST
/luminous-12-2-4/Data128MB::initializing for trans_id =
tx0003d-005a994a98-10c84997-default
2018-03-02 13:59:04.927396 7fe2053ca700 10 rgw api priority: s3=5
s3website=4
2018-03-02 13:59:04.927399 7fe2053ca700 10 host=cephrgw01.dunkel.de
2018-03-02 13:59:04.927422 7fe2053ca700 20 subdomain=
domain=cephrgw01.dunkel.de in_hosted_domain=1 in_hosted_domain_s3website=0
2018-03-02 13:59:04.927427 7fe2053ca700 20 final domain/bucket subdomain=
domain=cephrgw01.dunkel.de in_hosted_domain=1 in_hosted_domain_s3website=0
s->info.domain=cephrgw01.dunkel.de
s->info.request_uri=/luminous-12-2-4/Data128MB
2018-03-02 13:59:04.927447 7fe2053ca700 10 meta>>
HTTP_X_AMZ_CONTENT_SHA256
2018-03-02 13:59:04.927454 7fe2053ca700 10 meta>> HTTP_X_AMZ_DATE
2018-03-02 13:59:04.927459 7fe2053ca700 10 x>>
x-amz-content-sha256:254bcc3fc4f27172636df4bf32de9f107f620d559b20d760197e4
52b97453917
2018-03-02 13:59:04.927464 7fe2053ca700 10 x>> x-amz-date:20180302T125904Z
2018-03-02 13:59:04.927493 7fe2053ca700 20 get_handler
handler=22RGWHandler_REST_Obj_S3
2018-03-02 13:59:04.927500 7fe2053ca700 10
handler=22RGWHandler_REST_Obj_S3
2018-03-02 13:59:04.927505 7fe2053ca700  2 req 61:0.000152:s3:POST
/luminous-12-2-4/Data128MB::getting op 4
2018-03-02 13:59:04.927512 7fe2053ca700 10
op=28RGWInitMultipart_ObjStore_S3
2018-03-02 13:59:04.927514 7fe2053ca700  2 req 61:0.000161:s3:POST
/luminous-12-2-4/Data128MB:init_multipart:verifying requester
2018-03-02 13:59:04.927519 7fe2053ca700 20
rgw::auth::StrategyRegistry::s3_main_strategy_t: trying
rgw::auth::s3::AWSAuthStrategy
2018-03-02 13:59:04.927524 7fe2053ca700 20 rgw::auth::s3::AWSAuthStrategy:
trying rgw::auth::s3::S3AnonymousEngine
2018-03-02 13:59:04.927531 7fe2053ca700 20
rgw::auth::s3::S3AnonymousEngine denied with reason=-1
2018-03-02 13:59:04.927533 7fe2053ca700 20 rgw::auth::s3::AWSAuthStrategy:
trying rgw::auth::s3::LocalEngine
2018-03-02 13:59:04.927569 7fe2053ca700 10 v4 signature format =
48cc8c61a70dde17932d925f65f843116199c1ca10094db83e7de05bfbd57dc4
2018-03-02 13:59:04.927584 7fe2053ca700 10 v4 credential format =
8DGDGA57XL9YPM8DGEQQ/20180302/us-east-1/s3/aws4_request
2018-03-02 13:59:04.927587 7fe2053ca700 10 access key id =
8DGDGA57XL9YPM8DGEQQ
2018-03-02 13:59:04.927589 7fe2053ca700 10 credential scope =
20180302/us-east-1/s3/aws4_request
2018-03-02 13:59:04.927620 7fe2053ca700 10 canonical headers format =
content-type:application/octet-stream
date:Fri, 02 Mar 2018 12:59:04 GMT
host:cephrgw01.dunkel.de
x-amz-content-sha256:254bcc3fc4f27172636df4bf32de9f107f620d559b20d760197e4
52b97453917
x-amz-date:20180302T125904Z

2018-03-02 13:59:04.927634 7fe2053ca700 10 payload request hash =
254bcc3fc4f27172636df4bf32de9f107f620d559b20d760197e452b97453917
2018-03-02 13:59:04.927690 7fe2053ca700 10 canonical request = POST
/luminous-12-2-4/Data128MB uploads= content-type:application/octet-stream
date:Fri, 02 Mar 2018 12:59:04 GMT
host:cephrgw01.dunkel.de
x-amz-content-sha256:254bcc3fc4f27172636df4bf32de9f107f620d559b20d760197e4
52b97453917
x-amz-date:20180302T125904Z

content-type;date;host;x-amz-content-sha256;x-amz-date
254bcc3fc4f27172636df4bf32de9f107f620d559b20d760197e452b97453917
2018-03-02 13:59:04.927696 7fe2053ca700 10 canonical request hash =
54e9858263535b46a3c4e51b2ae5c1d0bf5e7a7690c5bba722eea749e7b936c4
2018-03-02 13:59:04.927716 7fe2053ca700 10 string to sign =
AWS4-HMAC-SHA256
20180302T125904Z
20180302/us-east-1/s3/aws4_request
54e9858263535b46a3c4e51b2ae5c1d0bf5e7a7690c5bba722eea749e7b936c4
2018-03-02 13:59:04.927920 7fe2053ca700 10 date_k=
dcef1f3be70873f1cb3240f7a56320e3c6763e7cf4bfae0e3182d2f9525292cd
2018-03-02 13:59:04.927954 7fe2053ca700 10 region_k  =
3d83dd9161cf7ba15e6c8c28d264f6cfce9b848e927359f34364a6c8c98209b7
2018-03-02 13:59:04.927963 7fe2053ca700 10 service_k =
e0708e00dc6b52aa1d889f45cd1dcced2bb1b2eee1b62e94ad9813c555e8eda9
2018-03-02 13:59:04.927972 7fe2053ca700 10 signing_k =
1ae362c4b2f1666786404fdb56c62d4f393635b2ce76d46ba325097fd3aa645e
2018-03-02 13:59:04.928021 7fe2053ca700 10 generated signature =
48cc8c61a70dde17932d925f65f843116199c1ca10094db83e7de05bfbd57dc4
2018-03-02 1

[ceph-users] Multipart Upload - POST fails

2018-03-02 Thread Ingo Reimann
Hi,

we discovered some problem with our installation - Multipart upload is not
working. 

What we did:
* tried upload with cyberduck as well as with script from
http://tracker.ceph.com/issues/12790
* tried against jewel gateways and luminous gateways from old cluster
* tried against 12.2.4 gateway with jewel-era cluster

Surprisingly this is no signature problem as in the issue above, instead I
get the following in the logs:

2018-03-02 13:59:04.927353 7fe2053ca700  1 == starting new request
req=0x7fe2053c42c0 =
2018-03-02 13:59:04.927383 7fe2053ca700  2 req 61:0.30::POST
/luminous-12-2-4/Data128MB::initializing for trans_id =
tx0003d-005a994a98-10c84997-default
2018-03-02 13:59:04.927396 7fe2053ca700 10 rgw api priority: s3=5
s3website=4
2018-03-02 13:59:04.927399 7fe2053ca700 10 host=cephrgw01.dunkel.de
2018-03-02 13:59:04.927422 7fe2053ca700 20 subdomain=
domain=cephrgw01.dunkel.de in_hosted_domain=1 in_hosted_domain_s3website=0
2018-03-02 13:59:04.927427 7fe2053ca700 20 final domain/bucket subdomain=
domain=cephrgw01.dunkel.de in_hosted_domain=1 in_hosted_domain_s3website=0
s->info.domain=cephrgw01.dunkel.de
s->info.request_uri=/luminous-12-2-4/Data128MB
2018-03-02 13:59:04.927447 7fe2053ca700 10 meta>>
HTTP_X_AMZ_CONTENT_SHA256
2018-03-02 13:59:04.927454 7fe2053ca700 10 meta>> HTTP_X_AMZ_DATE
2018-03-02 13:59:04.927459 7fe2053ca700 10 x>>
x-amz-content-sha256:254bcc3fc4f27172636df4bf32de9f107f620d559b20d760197e4
52b97453917
2018-03-02 13:59:04.927464 7fe2053ca700 10 x>> x-amz-date:20180302T125904Z
2018-03-02 13:59:04.927493 7fe2053ca700 20 get_handler
handler=22RGWHandler_REST_Obj_S3
2018-03-02 13:59:04.927500 7fe2053ca700 10
handler=22RGWHandler_REST_Obj_S3
2018-03-02 13:59:04.927505 7fe2053ca700  2 req 61:0.000152:s3:POST
/luminous-12-2-4/Data128MB::getting op 4
2018-03-02 13:59:04.927512 7fe2053ca700 10
op=28RGWInitMultipart_ObjStore_S3
2018-03-02 13:59:04.927514 7fe2053ca700  2 req 61:0.000161:s3:POST
/luminous-12-2-4/Data128MB:init_multipart:verifying requester
2018-03-02 13:59:04.927519 7fe2053ca700 20
rgw::auth::StrategyRegistry::s3_main_strategy_t: trying
rgw::auth::s3::AWSAuthStrategy
2018-03-02 13:59:04.927524 7fe2053ca700 20 rgw::auth::s3::AWSAuthStrategy:
trying rgw::auth::s3::S3AnonymousEngine
2018-03-02 13:59:04.927531 7fe2053ca700 20
rgw::auth::s3::S3AnonymousEngine denied with reason=-1
2018-03-02 13:59:04.927533 7fe2053ca700 20 rgw::auth::s3::AWSAuthStrategy:
trying rgw::auth::s3::LocalEngine
2018-03-02 13:59:04.927569 7fe2053ca700 10 v4 signature format =
48cc8c61a70dde17932d925f65f843116199c1ca10094db83e7de05bfbd57dc4
2018-03-02 13:59:04.927584 7fe2053ca700 10 v4 credential format =
8DGDGA57XL9YPM8DGEQQ/20180302/us-east-1/s3/aws4_request
2018-03-02 13:59:04.927587 7fe2053ca700 10 access key id =
8DGDGA57XL9YPM8DGEQQ
2018-03-02 13:59:04.927589 7fe2053ca700 10 credential scope =
20180302/us-east-1/s3/aws4_request
2018-03-02 13:59:04.927620 7fe2053ca700 10 canonical headers format =
content-type:application/octet-stream
date:Fri, 02 Mar 2018 12:59:04 GMT
host:cephrgw01.dunkel.de
x-amz-content-sha256:254bcc3fc4f27172636df4bf32de9f107f620d559b20d760197e4
52b97453917
x-amz-date:20180302T125904Z

2018-03-02 13:59:04.927634 7fe2053ca700 10 payload request hash =
254bcc3fc4f27172636df4bf32de9f107f620d559b20d760197e452b97453917
2018-03-02 13:59:04.927690 7fe2053ca700 10 canonical request = POST
/luminous-12-2-4/Data128MB
uploads=
content-type:application/octet-stream
date:Fri, 02 Mar 2018 12:59:04 GMT
host:cephrgw01.dunkel.de
x-amz-content-sha256:254bcc3fc4f27172636df4bf32de9f107f620d559b20d760197e4
52b97453917
x-amz-date:20180302T125904Z

content-type;date;host;x-amz-content-sha256;x-amz-date
254bcc3fc4f27172636df4bf32de9f107f620d559b20d760197e452b97453917
2018-03-02 13:59:04.927696 7fe2053ca700 10 canonical request hash =
54e9858263535b46a3c4e51b2ae5c1d0bf5e7a7690c5bba722eea749e7b936c4
2018-03-02 13:59:04.927716 7fe2053ca700 10 string to sign =
AWS4-HMAC-SHA256
20180302T125904Z
20180302/us-east-1/s3/aws4_request
54e9858263535b46a3c4e51b2ae5c1d0bf5e7a7690c5bba722eea749e7b936c4
2018-03-02 13:59:04.927920 7fe2053ca700 10 date_k=
dcef1f3be70873f1cb3240f7a56320e3c6763e7cf4bfae0e3182d2f9525292cd
2018-03-02 13:59:04.927954 7fe2053ca700 10 region_k  =
3d83dd9161cf7ba15e6c8c28d264f6cfce9b848e927359f34364a6c8c98209b7
2018-03-02 13:59:04.927963 7fe2053ca700 10 service_k =
e0708e00dc6b52aa1d889f45cd1dcced2bb1b2eee1b62e94ad9813c555e8eda9
2018-03-02 13:59:04.927972 7fe2053ca700 10 signing_k =
1ae362c4b2f1666786404fdb56c62d4f393635b2ce76d46ba325097fd3aa645e
2018-03-02 13:59:04.928021 7fe2053ca700 10 generated signature =
48cc8c61a70dde17932d925f65f843116199c1ca10094db83e7de05bfbd57dc4
2018-03-02 13:59:04.928031 7fe2053ca700 15 string_to_sign=AWS4-HMAC-SHA256
20180302T125904Z
20180302/us-east-1/s3/aws4_request
54e9858263535b46a3c4e51b2ae5c1d0bf5e7a7690c5bba722eea749e7b936c4
2018-03-02 13:59:04.928056 7fe2053ca700 15 server