On Tue, Jul 03, 2018 at 12:18:53PM +0100, Mark Adams wrote: > Hi Alwin, please see my response below. > > On 3 July 2018 at 10:07, Alwin Antreich <a.antre...@proxmox.com> wrote: > > > On Tue, Jul 03, 2018 at 01:05:51AM +0100, Mark Adams wrote: > > > Currently running the newest 5.2-1 version, I had a test cluster which > > was > > > working fine. I since added more disks, first stopping, then setting out, > > > then destroying each osd so I could recreate it all from scratch. > > > > > > However, when adding a new osd (either via GUI or pveceph CLI) it seems > > to > > > show a successful create, however does not show in the gui as an osd > > under > > > the host. > > > > > > It's like the osd information is being stored by proxmox/ceph somewhere > > > else and not being correctly removed and recreated? > > > > > > I can see that the newly created disk (after it being destroyed) is > > > down/out. > > > > > > Is this by design? is there a way to force the disk back? shouldn't it > > show > > > in the gui once you create it again? > > > > > Did you zero your disks after removal? On the first ~200 MB of the disk, > > there are leftovers that need to be zeroed before use. After that the > > OSD should be added fine. > > > > > I hadn't done this, no - it has helped with the majority of disks thanks > and I can now re-add them (I also had to remove the folders from > /var/lib/ceph/osd which had other osd names - not sure if the destroy > process is supposed to remove them also?) They will not interfere. ;)
> > However I have a strange problem on the 2nd host, where it will not make > osd.12 ... I get no error output from the gui or pveceph createosd /dev/sda > - it just doesn't appear as an osd. > > It successfully partitions the disk, but doesn't create a folder in > /var/lib/ceph/osd/ for the osd mount. I can see there is lock files in > /var/lib/ceph/tmp/ ... which I would think should only be there whilst the > creation is taking place? From the OSD creation or different lock? Shouldn't make troubles either. > > journalctl -xe is showing me the problem I think, "command_with_stdin: > Error EEXIST: entity osd.12 exists but key does not match" > > Where is this key? how should I be clearing it out so it will create? > 'ceph auth list' will show you all keys in ceph, there will be an orphan osd.12. Removal 'ceph auth del <osd.id>'. -- Cheers, Alwin _______________________________________________ pve-user mailing list pve-user@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user