Hi to all! I have 2 questions regarding host buckets in the crush map;
1. The name of a "host bucket" is just a label (which can have an arbitrary value) or is used in some way or form (so it has to be a real short or full hostname)?
2. The second question is more complex, how to avoid, upon host reboot (or during the OSD activation), the creation of a new host bucket? I'll try to explain myself better.
The host buckets in our environment have the name of the interface not the name of the host itself (if a node's "hostname -s" is "node01" this node is present in the crush map as "node01-int", to represent the interface where Ceph is communicating, cluster/private network). Furthermore each host bucket can be in a different crush root (the "default" crush root is empty).
I just reinstalled some nodes with Rocky 9 and Ceph Quincy (see my previous messages for more info) and upon activation of the OSDs a new host bucket is created and placed (with all the OSDs) in the default root (The name of this host bucket is node "hostname -s", like "node01"). Because it is in the wrong root, I've renamed it to the interface name ("node01-int"), moved it to the correct root and deleted the "wrong" host bucket ("node01").
Everything seemed to be working fine, but to be sure I've rebooted (again) a freshly reinstalled node. Upon reboot the "wrong" host bucket (node01) is recreated in the default root and all the OSDs are moved there leaving the host bucket "node01-int" empty.
Clearly upon boot the ceph-osd daemons pass the name "hostname -s" to the monitors, which (I suppose) check if this name exists in the crush map, if it doesn't a new host bucket is created (or something along this lines).
In fact, when I changed the hostname of my node to "node01-int", after reboot, the OSDs were placed under the correct host bucket (and, more importantly, under the correct root).
If the answer on my question number 1. is that the host bucket name is just a label, as long as its in the correct root, doesn't matter how the node is called (and I could just put all the host buckets name without the "*-int" suffix). But if is not a label I need to have it with the interface name.
So, the question remains, how to "pass" to ceph the correct host bucket name?
I think that the solution is to put "host=node01-int" in ceph.conf, but where? Under the "[osd]" sections ("[osd.*]"???) or I have to insert it under every single osd ([osd.12], [osd.13]...)?
If you read this line, congratulation you reached the finish! :-) Thank you! Iztok P.S. I'm not using cephadm, all the commands are manual. -- Iztok Gregori ICT Systems and Services Elettra - Sincrotrone Trieste S.C.p.A. http://www.elettra.eu _______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
