Hi Casey,
Thanks for you answer. Good news to know that in Squid it is possible to
unset the default realm definition. In fact I was running Reef when I
(or my colleagues) had all the last bad experiences but we upgraded to
Squid yesterday! Nonetheless, I just add a look at `radosgw-admin
--help` and the command `realm default rm` doesn't seems to be documented...
Improvements to the documentation are probably the easiest and here are
a few suggestions:
- Warn that, despite there is normally a hierarchical relationship
between realm, zonegroup and zone, radosgw-admin doesn't implement this
hierarchy with the result that for a zone or zonegroup specifying a
realm will not have any effect to restrict/select one (and the same with
zonegroup and zone).
- Mention that defining a default realm/zonegroup/zone is discouraged if
you have several of thems for the reason above. This can lead to
updating an object different from what you had in mind.
- Remove the --default from suggested commands. When you are new you
tend to add it, it you miss the note or didn't fully understand all the
things it implies.
As for small improvements to radosgw-admin that may avoid running into
troubles, I could suggest:
- Have a "default rm" command for zonegroup and zone too and document it
in the help for all the objects that have it
- Rewrite the help (may be not a small change in fact) so that you could
have a hierarchical help giving more details than the summary on a given
command (for example `realm zone modify --help` would tell you that if
you use --master, this will modify the zonegroup the zone belongs to and
that you have to mention it or the default zone will be used)
- Do a minimum of validation when modifications are done to prevent have
a configuration/period that cannot be commited anymore
Really RGW are a very nice service in Ceph, easy to deploy in various
scenarios when you use cephadm, but the radosgw-admin part of the
configuration is very risky. This is not the case in other parts of Ceph
and I think something should be done to put this part at the same level
as other Ceph management commands/tools (I don't have in mind any where
you can just by a small mistake break your configuration as there is
enough validation done to prevent it and the famous
--yes-i-really-want-it for the things that can make sense but are
potentially dangerous). An issue has been open by Frederic after the
discussion in another thread
(https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/6SD24SR7JBHFZ4E5YTTTQ7NZULHYVHRA/)
around this idea: https://tracker.ceph.com/issues/71406.
Best regards,
Michel
Le 27/05/2025 à 17:53, Casey Bodley a écrit :
On Mon, May 26, 2025 at 7:38 AM Michel Jouvin
<michel.jou...@ijclab.in2p3.fr> wrote:
Hi,
We have a Ceph cluster hosting ~10 RGW realms and in our experience,
radosgw-admin is making the Ceph admins life very hard... The opposite
experience to other Ceph management tools.
In addition to the lack of verifications before applying modifications
that leads to the issues describe in another thread were you can have a
configuration that cannot be commited and at the same time cannot be
fixed, I have the feeling that one of the major issue is that
radosgw-admin predates the rgw/zonegroup/zone paradigm and manages these
3 different kinds of objects as completely independent from the others.
It is made worst by the fact that radosgw-admin defines a default object
in each category, used implicitely (and slilently) if you don't select
one specific object. The consequence of these 2 problems is that if you
modify a zone, you can endup modifying the zonegroup of another realm
than the one the zone belongs to (as there is no concept of a zone
belonging to a zonegroup in radosgw-admin). It happens for example if
you modify a zone (with zone modify) to make it the master zone (of a
zonegroup in fact) and you don't specify the zonegroup the zone belongs
to/is associated with: it will in fact update the default zonegroup that
can be in another realm or at least not be the zonegroup the zone is
associated with. A less impactful but nevertheless unexpected result is
if you do something like: 'zonegroup list --rgw-realm my_realm'. You
would expect to see only the zonegroups associated with the realm you
specified where you'll get the list of all the zonegroups defined.
I can imagine that chaning this is difficult without breaking backward
compatibility. IMO, an improvement would be the ability not to have a
default realm, zonegroup, zone. At least you would have an error if you
don't specify explicitely the object to modify.
thanks Michel,
this 'default' stuff was meant to be a convenience for clusters with a
single zone each, but has indeed turned out to be a nightmare. prior
to squid, it wasn't even possible to leave the 'default realm' unset.
quoting the release note:
The radosgw-admin commands 'realm create' and 'realm pull' no longer set the
default realm without --default.
there's a 'radosgw-admin realm default rm' command that can clear that
default realm, if set. i don't know of a way to unset the default zone
or zonegroup, though you at least had to opt in to those with
--default
Am I the only one struggling with radosgw-admin? Is there any plan to
improve this?
as you say, it's hard to make significant changes here without
breaking existing deployments/automation. so there aren't any concrete
plans, but we're open to ideas
in the meantime, would you suggest any improvements to the
documentation? for example, the note about 'default' behavior under
https://docs.ceph.com/en/latest/radosgw/multisite/#create-a-realm
Best regards,
Michel
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
