[ceph-users] Re: RGW: Cannot write to bucket anymore
Hello Robin, thank you. The object-stat did not show anything suspicious. And the logs do show s3:get_obj decode_policy Read AccessControlPolicyxmlns="http://s3.amazonaws.com/doc/2006-03-01/";>XY xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:type="CanonicalUser">XY FULL_CONTROL and than it fails with s3:put_obj http status=403 So we do not see any errors or something. Everything looks the same to the other working buckets. No versioning. But there has to be something. Where can I have a look? I tried almost anything with the aws cli to find something. But there is nothing. Are there any rados or other commands to debug this? Best, Malte On 22.03.24 02:35, Robin H. Johnson wrote: On Thu, Mar 21, 2024 at 11:20:44AM +0100, Malte Stroem wrote: Hello Robin, thanks a lot. Yes, I set debug to debug_rgw=20 & debug_ms=1. It's that 403 I always get. There is no versioning enabled. There is a lifecycle policy for removing the files after one day. Did the object stat call return anything? Can you show more of the debug output (redact the keys/hostname/filename)? ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: RGW: Cannot write to bucket anymore
On Thu, Mar 21, 2024 at 11:20:44AM +0100, Malte Stroem wrote: > Hello Robin, > > thanks a lot. > > Yes, I set debug to debug_rgw=20 & debug_ms=1. > > It's that 403 I always get. > > There is no versioning enabled. > > There is a lifecycle policy for removing the files after one day. Did the object stat call return anything? Can you show more of the debug output (redact the keys/hostname/filename)? -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation President & Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 signature.asc Description: PGP signature ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: RGW: Cannot write to bucket anymore
Hello Robin, thanks a lot. Yes, I set debug to debug_rgw=20 & debug_ms=1. It's that 403 I always get. There is no versioning enabled. There is a lifecycle policy for removing the files after one day. That's all I can find. Do you have any more ideas? Best, Malte On 19.03.24 17:23, Robin H. Johnson wrote: On Tue, Mar 19, 2024 at 01:19:34PM +0100, Malte Stroem wrote: I checked the policies, lifecycle and versioning. Nothing. The user has FULL_CONTROL. Same settings for the user's other buckets he can still write to. Wenn setting debugging to higher numbers all I can see is something like this while trying to write to the bucket: Did you get to debug_rgw=20 & debug_ms=1? s3:put_obj reading permissions s3:put_obj init op s3:put_obj verifying op mask s3:put_obj verifying op permissions op->ERRORHANDLER: err_no=-13 new_err_no=-13 cache get: name=default.rgw.log++script.postrequest. : hit (negative entry) s3:put_obj op status=0 s3:put_obj http status=403 1 == req done req=0x7fe8bb60a710 op status=0 http_status=403 latency=0.0s == Does an object of the same name exist, possibly versioned, somehow owned by a different user? `radosgw-admin object stat --bucket=... --object=...` IIRC there would be specific messages saying it was denied by policy, but I haven't checked that part of the codebase in some time. ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: RGW: Cannot write to bucket anymore
On Tue, Mar 19, 2024 at 01:19:34PM +0100, Malte Stroem wrote: > I checked the policies, lifecycle and versioning. > > Nothing. The user has FULL_CONTROL. Same settings for the user's other > buckets he can still write to. > > Wenn setting debugging to higher numbers all I can see is something like > this while trying to write to the bucket: Did you get to debug_rgw=20 & debug_ms=1? > > s3:put_obj reading permissions > > > s3:put_obj init op > s3:put_obj verifying op mask > s3:put_obj verifying op permissions > op->ERRORHANDLER: err_no=-13 new_err_no=-13 > cache get: name=default.rgw.log++script.postrequest. : hit (negative entry) > s3:put_obj op status=0 > s3:put_obj http status=403 > 1 == req done req=0x7fe8bb60a710 op status=0 http_status=403 > latency=0.0s == Does an object of the same name exist, possibly versioned, somehow owned by a different user? `radosgw-admin object stat --bucket=... --object=...` IIRC there would be specific messages saying it was denied by policy, but I haven't checked that part of the codebase in some time. -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation President & Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 signature.asc Description: PGP signature ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
