Hi again... I have reinstall all Proxmox nodes and install ceph on each node. Create the mons and mgr on eatch node. I have issue the command ceph-volume lvm activate --all, on each node, in order bring up the /var/lib/ceph/osd/<node> After that I ran this commands: ceph-volume lvm activate --all mkdir /root/mon-store ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-0 --no-mon-config --op update-mon-db --mon-store-path mon-store/ ceph-monstore-tool mon-store/ rebuild -- --keyring /etc/pve/priv/ceph.client.admin.keyring --mon-ids pve01 pve02 pve03 mv /var/lib/ceph/mon/ceph-pve01/store.db/ /var/lib/ceph/mon/ceph-pve01/store.db-bkp cp -rf mon-store/store.db/ /var/lib/ceph/mon/ceph-pve01/ chown -R ceph:ceph /var/lib/ceph/mon/ceph-pve01/store.db
But now I got nothing! No monitor, no manager, no osd, none! Perhaps somebody point me what I did wrong. Thanks Em qua., 20 de ago. de 2025 às 11:32, Gilberto Ferreira < gilberto.nune...@gmail.com> escreveu: > I can see the content of the mentioned folders just after issue the > command ceph-volume.... > Thanks anyway. > > > > Em qua., 20 de ago. de 2025 às 11:26, Eugen Block <ebl...@nde.ag> > escreveu: > >> I assume you're right. Do you see the OSD contents in >> /var/lib/ceph/osd/ceph-pve01 after activating? >> And remember to collect the clustermap from all OSDs for this >> procedure to succeed. >> >> Zitat von Gilberto Ferreira <gilberto.nune...@gmail.com>: >> >> > I see... >> > >> > But I had another problem. >> > The script from (0) indicate that should be exist a /var/lib/ceph/osd >> > folder, like: >> > /var/lib/ceph/osd/ceph-pve01 >> > /var/lib/ceph/osd/ceph-pve02 >> > and so on. >> > >> > But this folder appears only if I run ceph-volume lvm activate --all. >> > So my question is: when I should run this command: after or before use >> the >> > script? >> > I think I need to run ceph-volume lvm activate --all, right? >> > Just to clarify. >> > >> > Thanks >> > >> > Em qua., 20 de ago. de 2025 às 11:08, Eugen Block <ebl...@nde.ag> >> escreveu: >> > >> >> Yes, you need a monitor. The mgr is not required and can be deployed >> >> later. After you created the monitor, replace the mon store contents >> >> by the collected clustermaps from the mentioned procedure. Keep the >> >> ownerships of the directories/files in mind. If the monitor starts >> >> successfully (with the original FSID), you can try to start one of the >> >> OSDs. If that works, start the rest of them, wait for the peering >> >> storm to settle, create two more monitors and two mgr daemons. >> >> >> >> Note that if you lose the mon store and you had a CephFS, you'll need >> >> to recreate that from the existing pools. >> >> >> >> Zitat von Gilberto Ferreira <gilberto.nune...@gmail.com>: >> >> >> >> > Hi >> >> > >> >> > Do I need to create any mon and/or mgr in the new ceph cluster? >> >> > >> >> > >> >> > >> >> > Em seg., 18 de ago. de 2025 às 13:03, Eugen Block <ebl...@nde.ag> >> >> escreveu: >> >> > >> >> >> Hi, >> >> >> >> >> >> this sounds like you created a new cluster (new fsid), the OSDs >> still >> >> >> have the previous fsid configured. I'd rather recommend to follow >> this >> >> >> procedure [0] to restore the mon store utilizing OSDs rather than >> >> >> trying to manipulate otherwise intact OSDs to fit into the "new" >> >> >> cluster. That way you'll have "your" cluster back. I don't know if >> >> >> there are any specifics to using proxmox, though. But the mentioned >> >> >> procedure seems to work just fine, I've read multiple reports on >> this >> >> >> list. Luckily, I haven't had to use it myself. >> >> >> >> >> >> Regards, >> >> >> Eugen >> >> >> >> >> >> [0] >> >> >> >> >> >> >> >> >> https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-mon/#recovery-using-osds >> >> >> >> >> >> Zitat von Gilberto Ferreira <gilberto.nune...@gmail.com>: >> >> >> >> >> >> > Hi >> >> >> > >> >> >> > I have 3 nodes Proxmox Cluster with CEPH, and after a crash, I >> have to >> >> >> > reinstall Proxmox from scratch, along with Ceph. >> >> >> > OSD are intact. >> >> >> > I already did ceph-volume lvm activate --all and the OSD appears >> with >> >> >> > ceph-volum lvm list and I got a folder with the name of the OSD >> under >> >> >> > /var/lib/ceph/osd. >> >> >> > However is not appear in ceph osd tree or ceph -s or even in the >> web >> >> gui. >> >> >> > Is there any way to re-add this OSD to Proxmox CEPH? >> >> >> > >> >> >> > Thanks a lot for any help. >> >> >> > >> >> >> > >> >> >> > Best Regards >> >> >> > --- >> >> >> > Gilbert >> >> >> > _______________________________________________ >> >> >> > ceph-users mailing list -- ceph-users@ceph.io >> >> >> > To unsubscribe send an email to ceph-users-le...@ceph.io >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> >> ceph-users mailing list -- ceph-users@ceph.io >> >> >> To unsubscribe send an email to ceph-users-le...@ceph.io >> >> >> >> >> > _______________________________________________ >> >> > ceph-users mailing list -- ceph-users@ceph.io >> >> > To unsubscribe send an email to ceph-users-le...@ceph.io >> >> >> >> >> >> _______________________________________________ >> >> ceph-users mailing list -- ceph-users@ceph.io >> >> To unsubscribe send an email to ceph-users-le...@ceph.io >> >> >> > _______________________________________________ >> > ceph-users mailing list -- ceph-users@ceph.io >> > To unsubscribe send an email to ceph-users-le...@ceph.io >> >> >> _______________________________________________ >> ceph-users mailing list -- ceph-users@ceph.io >> To unsubscribe send an email to ceph-users-le...@ceph.io >> > _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
