Re: [ceph-users] Broken Buckets after Jewel->Luminous Upgrade
On Tue, Jan 30, 2018 at 10:32:04AM +0100, Ingo Reimann wrote: > What could be the problem,and how may I solve that? For anybody else tracking this, the logs & debugging info are filed at http://tracker.ceph.com/issues/22928 -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 signature.asc Description: Digital signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Broken Buckets after Jewel->Luminous Upgrade
On Wed, Jan 31, 2018 at 07:39:02AM +0100, Ingo Reimann wrote: > Hi Robin, > > thanks for your reply. > > Concerning "https://tracker.ceph.com/issues/22756 - buckets showing as > empty": Our cluster is rather old - argonaut, but the affected bucket and > user are created under jewel. > > If you need more data, I may post it. Yes, please, I'm even more interested now, as I only have customer reports from old buckets! I'm wondering if it's slightly related to issue 22714 for some other object that starts with an underscore. -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 signature.asc Description: Digital signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Broken Buckets after Jewel->Luminous Upgrade
Hi Robin, thanks for your reply. Concerning "https://tracker.ceph.com/issues/22756 - buckets showing as empty": Our cluster is rather old - argonaut, but the affected bucket and user are created under jewel. If you need more data, I may post it. Best regards, Ingo -Ursprüngliche Nachricht- Von: Robin H. Johnson [mailto:robb...@gentoo.org] Gesendet: Dienstag, 30. Januar 2018 19:17 An: Ingo Reimann Cc: ceph-users Betreff: Re: [ceph-users] Broken Buckets after Jewel->Luminous Upgrade On Tue, Jan 30, 2018 at 10:32:04AM +0100, Ingo Reimann wrote: > The problem: > Some Buckets are not accessible from the luminous gateway. The > metadata for that buckets seemed ok, but listing was not possible. A > local s3cmd got "404 NoSuchKey". I exported and imported the metadata > for one instance and ran radosgw-admin --check. Now the bucket is > listable but empty under luminous and broken under jewel. The > corresponding directory object still contains the file in its omap. > > I am afraid to corrupt my cluster so I stopped the upgrade for the > other gateways. I have a couple of bugs open for possible the same issue: https://tracker.ceph.com/issues/22756 - buckets showing as empty http://tracker.ceph.com/issues/22714 - old AccessKeys not working One more to come after more diagnosis my side, where some old files don't work properly anymore (dropping off at a multiple of 512K) -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Broken Buckets after Jewel->Luminous Upgrade
On Tue, Jan 30, 2018 at 10:32:04AM +0100, Ingo Reimann wrote: > The problem: > Some Buckets are not accessible from the luminous gateway. The metadata > for that buckets seemed ok, but listing was not possible. A local s3cmd > got "404 NoSuchKey". I exported and imported the metadata for one instance > and ran radosgw-admin --check. Now the bucket is listable but empty under > luminous and broken under jewel. The corresponding directory object still > contains the file in its omap. > > I am afraid to corrupt my cluster so I stopped the upgrade for the other > gateways. I have a couple of bugs open for possible the same issue: https://tracker.ceph.com/issues/22756 - buckets showing as empty http://tracker.ceph.com/issues/22714 - old AccessKeys not working One more to come after more diagnosis my side, where some old files don't work properly anymore (dropping off at a multiple of 512K) -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 signature.asc Description: Digital signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Broken Buckets after Jewel->Luminous Upgrade
Hi, We got a nasty issue during our Jewel->Luminous upgrade. The Mon/MGR and OSD part went well, but the first of our rgws is incompatible to the others: The problem: Some Buckets are not accessible from the luminous gateway. The metadata for that buckets seemed ok, but listing was not possible. A local s3cmd got "404 NoSuchKey". I exported and imported the metadata for one instance and ran radosgw-admin --check. Now the bucket is listable but empty under luminous and broken under jewel. The corresponding directory object still contains the file in its omap. I am afraid to corrupt my cluster so I stopped the upgrade for the other gateways. What could be the problem,and how may I solve that? Any help is apreciated. Regards, Ingo Reimann Dunkel GmbH ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
