Re: [ceph-users] OSD after OS reinstallation.
Mandi! Alfredo Deza In chel di` si favelave... > There are ways to create partitions without a PARTUUID. We have an > example in our docs with parted that will produce what is needed: > http://docs.ceph.com/docs/master/ceph-volume/lvm/prepare/#partitioning > But then again... I would strongly suggest avoiding all of this and > just using the new way of doing OSDs with LVM Ahem... i'm still on hammer... ;-))) -- dott. Marco Gaiarin GNUPG Key ID: 240A3D66 Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/ Polo FVG - Via della Bontà, 7 - 33078 - San Vito al Tagliamento (PN) marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797 Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA! http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000 (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA) ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] OSD after OS reinstallation.
On Fri, Feb 22, 2019 at 9:38 AM Marco Gaiarin wrote: > > Mandi! Alfredo Deza > In chel di` si favelave... > > > The problem is that if there is no PARTUUID ceph-volume can't ensure > > what device is the one actually pointing to data/journal. Being 'GPT' > > alone will not be enough here :( > > Ok. There's some way to 'force' a PARTUUID, in a GPT or non-GPT > partition, clearly even, if needed, destroying it? > > > I've tried also to create a GPT partition in a DOS partition (eg, in a > /dev/sda5), and seems that GPT partition get correctly created, but > still (sub) partition have no PARTUUID... There are ways to create partitions without a PARTUUID. We have an example in our docs with parted that will produce what is needed: http://docs.ceph.com/docs/master/ceph-volume/lvm/prepare/#partitioning But then again... I would strongly suggest avoiding all of this and just using the new way of doing OSDs with LVM > > -- > dott. Marco Gaiarin GNUPG Key ID: 240A3D66 > Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/ > Polo FVG - Via della Bontà, 7 - 33078 - San Vito al Tagliamento (PN) > marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797 > > Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA! > http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000 > (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA) > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] OSD after OS reinstallation.
Mandi! Alfredo Deza In chel di` si favelave... > The problem is that if there is no PARTUUID ceph-volume can't ensure > what device is the one actually pointing to data/journal. Being 'GPT' > alone will not be enough here :( Ok. There's some way to 'force' a PARTUUID, in a GPT or non-GPT partition, clearly even, if needed, destroying it? I've tried also to create a GPT partition in a DOS partition (eg, in a /dev/sda5), and seems that GPT partition get correctly created, but still (sub) partition have no PARTUUID... -- dott. Marco Gaiarin GNUPG Key ID: 240A3D66 Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/ Polo FVG - Via della Bontà, 7 - 33078 - San Vito al Tagliamento (PN) marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797 Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA! http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000 (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA) ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] OSD after OS reinstallation.
On Thu, Feb 21, 2019 at 2:34 AM Анатолий Фуников wrote: > > It's strange but parted output for this disk (/dev/sdf) show me that it's GPT: > > (parted) print > Model: ATA HGST HUS726020AL (scsi) > Disk /dev/sdf: 2000GB > Sector size (logical/physical): 512B/4096B > Partition Table: gpt > > Number Start End SizeType File system Flags > 2 1049kB 1075MB 1074MBceph journal > 1 1075MB 2000GB 1999GB xfs ceph data > The problem is that if there is no PARTUUID ceph-volume can't ensure what device is the one actually pointing to data/journal. Being 'GPT' alone will not be enough here :( > ср, 20 февр. 2019 г. в 17:11, Alfredo Deza : >> >> On Wed, Feb 20, 2019 at 8:40 AM Анатолий Фуников >> wrote: >> > >> > Thanks for the reply. >> > blkid -s PARTUUID -o value /dev/sdf1 shows me nothing, but blkid /dev/sdf1 >> > shows me this: /dev/sdf1: UUID="b03810e4-dcc1-46c2-bc31-a1e558904750" >> > TYPE="xfs" >> >> I think this is what happens with a non-gpt partition. GPT labels will >> use a PARTUUID to identify the partition, and I just confirmed that >> ceph-volume will enforce looking for PARTUUID if the JSON >> identified a partition (vs. an LV). >> >> From what I briefly researched it is not possible to add a GPT label >> on a non-gpt partition without losing data. >> >> My suggestion (if you confirm it is not possible to add the GPT label) >> is to start the migration towards the new way of creating OSDs >> >> > >> > ср, 20 февр. 2019 г. в 16:27, Alfredo Deza : >> >> >> >> On Wed, Feb 20, 2019 at 8:16 AM Анатолий Фуников >> >> wrote: >> >> > >> >> > Hello. I need to raise the OSD on the node after reinstalling the OS, >> >> > some OSD were made a long time ago, not even a ceph-disk, but a set of >> >> > scripts. >> >> > There was an idea to get their configuration in json via ceph-volume >> >> > simple scan, and then on a fresh system I can make a ceph-volume simple >> >> > activate --file >> >> > /etc/ceph/osd/31-46eacafe-22b6-4433-8e5c-e595612d8579.json >> >> > I do ceph-volume simple scan /var/lib/ceph/osd/ceph-31/, and got this >> >> > json: https://pastebin.com/uJ8WVZyV >> >> > It seems everything is not bad, but in the data section I see a direct >> >> > link to the device /dev/sdf1, and the uuid field is empty. At the same >> >> > time, in the /dev/disk/by-partuuid directory I can find and substitute >> >> > this UUID in this json, and delete the direct link to the device in >> >> > this json. >> >> > The question is: how correct is it and can I raise this OSD on a >> >> > freshly installed OS with this fixed json? >> >> >> >> It worries me that it is unable to find a uuid for the device. This is >> >> important because paths like /dev/sdf1 are ephemeral and can change >> >> after a reboot. The uuid is found by running the following: >> >> >> >> blkid -s PARTUUID -o value /dev/sdf1 >> >> >> >> If that is not returning anything, then ceph-volume will probably not >> >> be able to ensure this device is brought up correctly. You can correct >> >> or add to anything in the JSON after a scan and rely on that, but then >> >> again >> >> without a partuuid I don't think this will work nicely >> >> >> >> > ___ >> >> > ceph-users mailing list >> >> > ceph-users@lists.ceph.com >> >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] OSD after OS reinstallation.
It's strange but parted output for this disk (/dev/sdf) show me that it's GPT: (parted) print Model: ATA HGST HUS726020AL (scsi) Disk /dev/sdf: 2000GB Sector size (logical/physical): 512B/4096B Partition Table: gpt Number Start End SizeType File system Flags 2 1049kB 1075MB 1074MBceph journal 1 1075MB 2000GB 1999GB xfs ceph data ср, 20 февр. 2019 г. в 17:11, Alfredo Deza : > On Wed, Feb 20, 2019 at 8:40 AM Анатолий Фуников > wrote: > > > > Thanks for the reply. > > blkid -s PARTUUID -o value /dev/sdf1 shows me nothing, but blkid > /dev/sdf1 shows me this: /dev/sdf1: > UUID="b03810e4-dcc1-46c2-bc31-a1e558904750" TYPE="xfs" > > I think this is what happens with a non-gpt partition. GPT labels will > use a PARTUUID to identify the partition, and I just confirmed that > ceph-volume will enforce looking for PARTUUID if the JSON > identified a partition (vs. an LV). > > From what I briefly researched it is not possible to add a GPT label > on a non-gpt partition without losing data. > > My suggestion (if you confirm it is not possible to add the GPT label) > is to start the migration towards the new way of creating OSDs > > > > > ср, 20 февр. 2019 г. в 16:27, Alfredo Deza : > >> > >> On Wed, Feb 20, 2019 at 8:16 AM Анатолий Фуников > >> wrote: > >> > > >> > Hello. I need to raise the OSD on the node after reinstalling the OS, > some OSD were made a long time ago, not even a ceph-disk, but a set of > scripts. > >> > There was an idea to get their configuration in json via ceph-volume > simple scan, and then on a fresh system I can make a ceph-volume simple > activate --file /etc/ceph/osd/31-46eacafe-22b6-4433-8e5c-e595612d8579.json > >> > I do ceph-volume simple scan /var/lib/ceph/osd/ceph-31/, and got this > json: https://pastebin.com/uJ8WVZyV > >> > It seems everything is not bad, but in the data section I see a > direct link to the device /dev/sdf1, and the uuid field is empty. At the > same time, in the /dev/disk/by-partuuid directory I can find and substitute > this UUID in this json, and delete the direct link to the device in this > json. > >> > The question is: how correct is it and can I raise this OSD on a > freshly installed OS with this fixed json? > >> > >> It worries me that it is unable to find a uuid for the device. This is > >> important because paths like /dev/sdf1 are ephemeral and can change > >> after a reboot. The uuid is found by running the following: > >> > >> blkid -s PARTUUID -o value /dev/sdf1 > >> > >> If that is not returning anything, then ceph-volume will probably not > >> be able to ensure this device is brought up correctly. You can correct > >> or add to anything in the JSON after a scan and rely on that, but then > >> again > >> without a partuuid I don't think this will work nicely > >> > >> > ___ > >> > ceph-users mailing list > >> > ceph-users@lists.ceph.com > >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] OSD after OS reinstallation.
Mandi! Alfredo Deza In chel di` si favelave... > > Ahem, how can i add a GPT label to a non-GPT partition (even loosing > > data)? > If you are coming from ceph-disk (or something else custom-made) and > don't care about losing data, why not fully migrate to the > new OSDs? > http://docs.ceph.com/docs/master/rados/operations/add-or-rm-osds/#rados-replacing-an-osd I'm using proxmox, so 'pveceph' helper, but i've trouble with journal lables, indeed, not main filesystem labels... -- dott. Marco Gaiarin GNUPG Key ID: 240A3D66 Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/ Polo FVG - Via della Bontà, 7 - 33078 - San Vito al Tagliamento (PN) marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797 Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA! http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000 (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA) ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] OSD after OS reinstallation.
On Wed, Feb 20, 2019 at 10:21 AM Marco Gaiarin wrote: > > Mandi! Alfredo Deza > In chel di` si favelave... > > > I think this is what happens with a non-gpt partition. GPT labels will > > use a PARTUUID to identify the partition, and I just confirmed that > > ceph-volume will enforce looking for PARTUUID if the JSON > > identified a partition (vs. an LV). > > From what I briefly researched it is not possible to add a GPT label > > on a non-gpt partition without losing data. > > Ahem, how can i add a GPT label to a non-GPT partition (even loosing > data)? If you are coming from ceph-disk (or something else custom-made) and don't care about losing data, why not fully migrate to the new OSDs? http://docs.ceph.com/docs/master/rados/operations/add-or-rm-osds/#rados-replacing-an-osd > > Seems the culprit around my 'Proxmox 4.4, Ceph hammer, OSD cache > link...' thread... > > > Thanks. > > -- > dott. Marco Gaiarin GNUPG Key ID: 240A3D66 > Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/ > Polo FVG - Via della Bontà, 7 - 33078 - San Vito al Tagliamento (PN) > marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797 > > Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA! > http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000 > (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA) > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] OSD after OS reinstallation.
Mandi! Alfredo Deza In chel di` si favelave... > I think this is what happens with a non-gpt partition. GPT labels will > use a PARTUUID to identify the partition, and I just confirmed that > ceph-volume will enforce looking for PARTUUID if the JSON > identified a partition (vs. an LV). > From what I briefly researched it is not possible to add a GPT label > on a non-gpt partition without losing data. Ahem, how can i add a GPT label to a non-GPT partition (even loosing data)? Seems the culprit around my 'Proxmox 4.4, Ceph hammer, OSD cache link...' thread... Thanks. -- dott. Marco Gaiarin GNUPG Key ID: 240A3D66 Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/ Polo FVG - Via della Bontà, 7 - 33078 - San Vito al Tagliamento (PN) marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797 Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA! http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000 (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA) ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] OSD after OS reinstallation.
On Wed, Feb 20, 2019 at 8:40 AM Анатолий Фуников wrote: > > Thanks for the reply. > blkid -s PARTUUID -o value /dev/sdf1 shows me nothing, but blkid /dev/sdf1 > shows me this: /dev/sdf1: UUID="b03810e4-dcc1-46c2-bc31-a1e558904750" > TYPE="xfs" I think this is what happens with a non-gpt partition. GPT labels will use a PARTUUID to identify the partition, and I just confirmed that ceph-volume will enforce looking for PARTUUID if the JSON identified a partition (vs. an LV). From what I briefly researched it is not possible to add a GPT label on a non-gpt partition without losing data. My suggestion (if you confirm it is not possible to add the GPT label) is to start the migration towards the new way of creating OSDs > > ср, 20 февр. 2019 г. в 16:27, Alfredo Deza : >> >> On Wed, Feb 20, 2019 at 8:16 AM Анатолий Фуников >> wrote: >> > >> > Hello. I need to raise the OSD on the node after reinstalling the OS, some >> > OSD were made a long time ago, not even a ceph-disk, but a set of scripts. >> > There was an idea to get their configuration in json via ceph-volume >> > simple scan, and then on a fresh system I can make a ceph-volume simple >> > activate --file /etc/ceph/osd/31-46eacafe-22b6-4433-8e5c-e595612d8579.json >> > I do ceph-volume simple scan /var/lib/ceph/osd/ceph-31/, and got this >> > json: https://pastebin.com/uJ8WVZyV >> > It seems everything is not bad, but in the data section I see a direct >> > link to the device /dev/sdf1, and the uuid field is empty. At the same >> > time, in the /dev/disk/by-partuuid directory I can find and substitute >> > this UUID in this json, and delete the direct link to the device in this >> > json. >> > The question is: how correct is it and can I raise this OSD on a freshly >> > installed OS with this fixed json? >> >> It worries me that it is unable to find a uuid for the device. This is >> important because paths like /dev/sdf1 are ephemeral and can change >> after a reboot. The uuid is found by running the following: >> >> blkid -s PARTUUID -o value /dev/sdf1 >> >> If that is not returning anything, then ceph-volume will probably not >> be able to ensure this device is brought up correctly. You can correct >> or add to anything in the JSON after a scan and rely on that, but then >> again >> without a partuuid I don't think this will work nicely >> >> > ___ >> > ceph-users mailing list >> > ceph-users@lists.ceph.com >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] OSD after OS reinstallation.
Thanks for the reply. blkid -s PARTUUID -o value /dev/sdf1 shows me nothing, but blkid /dev/sdf1 shows me this: /dev/sdf1: UUID="b03810e4-dcc1-46c2-bc31-a1e558904750" TYPE="xfs" ср, 20 февр. 2019 г. в 16:27, Alfredo Deza : > On Wed, Feb 20, 2019 at 8:16 AM Анатолий Фуников > wrote: > > > > Hello. I need to raise the OSD on the node after reinstalling the OS, > some OSD were made a long time ago, not even a ceph-disk, but a set of > scripts. > > There was an idea to get their configuration in json via ceph-volume > simple scan, and then on a fresh system I can make a ceph-volume simple > activate --file /etc/ceph/osd/31-46eacafe-22b6-4433-8e5c-e595612d8579.json > > I do ceph-volume simple scan /var/lib/ceph/osd/ceph-31/, and got this > json: https://pastebin.com/uJ8WVZyV > > It seems everything is not bad, but in the data section I see a direct > link to the device /dev/sdf1, and the uuid field is empty. At the same > time, in the /dev/disk/by-partuuid directory I can find and substitute this > UUID in this json, and delete the direct link to the device in this json. > > The question is: how correct is it and can I raise this OSD on a freshly > installed OS with this fixed json? > > It worries me that it is unable to find a uuid for the device. This is > important because paths like /dev/sdf1 are ephemeral and can change > after a reboot. The uuid is found by running the following: > > blkid -s PARTUUID -o value /dev/sdf1 > > If that is not returning anything, then ceph-volume will probably not > be able to ensure this device is brought up correctly. You can correct > or add to anything in the JSON after a scan and rely on that, but then > again > without a partuuid I don't think this will work nicely > > > ___ > > ceph-users mailing list > > ceph-users@lists.ceph.com > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] OSD after OS reinstallation.
On Wed, Feb 20, 2019 at 8:16 AM Анатолий Фуников wrote: > > Hello. I need to raise the OSD on the node after reinstalling the OS, some > OSD were made a long time ago, not even a ceph-disk, but a set of scripts. > There was an idea to get their configuration in json via ceph-volume simple > scan, and then on a fresh system I can make a ceph-volume simple activate > --file /etc/ceph/osd/31-46eacafe-22b6-4433-8e5c-e595612d8579.json > I do ceph-volume simple scan /var/lib/ceph/osd/ceph-31/, and got this json: > https://pastebin.com/uJ8WVZyV > It seems everything is not bad, but in the data section I see a direct link > to the device /dev/sdf1, and the uuid field is empty. At the same time, in > the /dev/disk/by-partuuid directory I can find and substitute this UUID in > this json, and delete the direct link to the device in this json. > The question is: how correct is it and can I raise this OSD on a freshly > installed OS with this fixed json? It worries me that it is unable to find a uuid for the device. This is important because paths like /dev/sdf1 are ephemeral and can change after a reboot. The uuid is found by running the following: blkid -s PARTUUID -o value /dev/sdf1 If that is not returning anything, then ceph-volume will probably not be able to ensure this device is brought up correctly. You can correct or add to anything in the JSON after a scan and rely on that, but then again without a partuuid I don't think this will work nicely > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] OSD after OS reinstallation.
Hello. I need to raise the OSD on the node after reinstalling the OS, some OSD were made a long time ago, not even a ceph-disk, but a set of scripts. There was an idea to get their configuration in json via ceph-volume simple scan, and then on a fresh system I can make a ceph-volume simple activate --file /etc/ceph/osd/31-46eacafe-22b6-4433-8e5c-e595612d8579.json I do ceph-volume simple scan /var/lib/ceph/osd/ceph-31/, and got this json: https://pastebin.com/uJ8WVZyV It seems everything is not bad, but in the data section I see a direct link to the device /dev/sdf1, and the uuid field is empty. At the same time, in the /dev/disk/by-partuuid directory I can find and substitute this UUID in this json, and delete the direct link to the device in this json. The question is: how correct is it and can I raise this OSD on a freshly installed OS with this fixed json? ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
