Re: [PVE-User] [ceph-users] OSD won't start, even created ??

2017-09-10 Thread Phil Schwarz 

OK, a few more hints -for a potential helper ;-) :

- Did a complete cluster update (dist-upgrade)
- Installed a mgr for every of the 3 mons.
- plugged a new disk in the same server
- Using GUI, installed the disk as new OSD WITH Bluestore chosen.

The OSD isn't visible until
 ceph osd crush add osd.0 0 host=d
And then, the OSD appears in the GUI BUT:
-  it is seen as filestore one
- is marked as down

ceph-osd.admin.log ::
tail -f /var/log/ceph/ceph-osd.admin.log
2017-09-10 17:28:24.937583 7feace863e00  0 ceph version 12.2.0 
(36f6c5ea099d43087ff0276121fd34e71668ae0e) luminous (rc), process 
(unknown), pid 15859
2017-09-10 17:28:24.938042 7feace863e00 -1 bluestore(/dev/sdc2) 
_read_bdev_label unable to decode label at offset 62: 
buffer::malformed_input: void 
bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) no longer 
understand old encoding version 1 < struct_compat
2017-09-10 17:28:24.938140 7feace863e00  1 journal _open /dev/sdc2 fd 4: 
319967006720 bytes, block size 4096 bytes, directio = 0, aio = 0
2017-09-10 17:28:24.938425 7feace863e00 -1 journal read_header error 
decoding journal header

2017-09-10 17:28:24.938433 7feace863e00  1 journal close /dev/sdc2
2017-09-10 17:28:25.933794 7f00ad7d7e00  0 ceph version 12.2.0 
(36f6c5ea099d43087ff0276121fd34e71668ae0e) luminous (rc), process 
(unknown), pid 15938
2017-09-10 17:28:25.936225 7f00ad7d7e00 -1 bluestore(/dev/sdc2) 
_read_bdev_label unable to decode label at offset 62: 
buffer::malformed_input: void 
bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) no longer 
understand old encoding version 1 < struct_compat
2017-09-10 17:28:25.944991 7f00ad7d7e00  1 journal _open /dev/sdc2 fd 4: 
319967006720 bytes, block size 4096 bytes, directio = 0, aio = 0
2017-09-10 17:28:25.945369 7f00ad7d7e00 -1 journal read_header error 
decoding journal header

2017-09-10 17:28:25.945396 7f00ad7d7e00  1 journal close /dev/sdc2



sgdisk --print /dev/sdc

Disk /dev/sdc: 625142448 sectors, 298.1 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 093F0FB0-1F83-4E8F-825E-762DA535
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 625142414
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)End (sector)  Size   Code  Name
   12048  206847   100.0 MiB   F800  ceph data
   2  206848   625142414   298.0 GiB     ceph block

My cluster is getting near full (In fact it is full; i have to manually 
reweight some larger OSDs to decrease to Defcon 3...)



thanks by advance
Best regards


Le 09/09/2017 à 16:08, Phil Schwarz a écrit :

Did a few more tests :

Older Ceph server with a pveceph create osd command (

(pveceph create osd /dev/sdb

equivalent to

ceph-disk prepare --zap-disk --fs-type xfs --cluster ceph --cluster-uuid
a5c0cfed-...4bf939ed70 /dev/sdb )

sgdisk --print /dev/sdd

Disk /dev/sdd: 2930277168 sectors, 1.4 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 638646CF-..-62296C871132
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 2930277134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)End (sector)  Size   Code  Name
   110487808  2930277134   1.4 TiB F800  ceph data
   2204810487807   5.0 GiB F802  ceph journal


On a newer ceph server ( dpkg -l : 12.2.0-pve1 version)

sgdisk --print /dev/sdb

Disk /dev/sdb: 1465149168 sectors, 698.6 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): D63886B6-0.26-BCBCD6FFCA3C
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1465149134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)End (sector)  Size   Code  Name
   12048  206847   100.0 MiB   F800  ceph data
   2  206848  1465149134   698.5 GiB     ceph block


Related to the cep-osd.admin log , i think i used a osd creation process
leading to a bluestore osd (instead of a filestore one).
And seems that afterward the ceph server is unable to use the new
bluestore :

( bluestore(/dev/sdb2) _read_bdev_label unable to decode label at offset
102: buffer::malformed_input: void
bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode
past end of struct encoding
)
just before trying to use it as a filestore one :

( probe_block_device_fsid /dev/sdb2 is filestore )


Tried to use the --bluestore 0 flag when creating the osd, but the flag
is unknown.

thanks by advance for any hint.
Being ready to do a few more tests.
Best regards.

Le 08/09/2017 à 17:27, Phil Schwarz a écrit :

Hi,
any help would be really useful.
Does anyone got a clue with my issue ?

Thanks by advance.
Best regards;


Le 05/09/2017 à 20:25, Phil Schwarz a écrit :

Hi,
I come

Re: [PVE-User] proxmox boot from usb

2017-09-10 Thread Michael Rasmussen 
On Sun, 10 Sep 2017 14:41:20 +0200
Gregor Burck  wrote:

> Hi,
> 
> and now the next question:
> 
> Is it recommended to boot from a USB Stick?
> 
If your motherboard has a M.2 socket I would strongly recommend using
that. The default Debian distribution which Proxmox is build upon is
not designed to be used from a USB stick.

See more of M.2 here: https://en.wikipedia.org/wiki/M.2

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael  rasmussen  cc
http://pgp.mit.edu:11371/pks/lookup?op=get=0xD3C9A00E
mir  datanom  net
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE501F51C
mir  miras  org
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE3E80917
--
/usr/games/fortune -es says:
Mary Tyler Moore's SEVENTH HUSBAND is wearing my DACRON TANK TOP in a
cheap hotel in HONOLULU!


pgp3oC95jP4IG.pgp
Description: OpenPGP digital signature
___
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

[PVE-User] proxmox boot from usb

2017-09-10 Thread Gregor Burck 

Hi,

and now the next question:

Is it recommended to boot from a USB Stick?

Background: We've a setup with esxi Server, the nodes only boot from a  
USB Stick, so no installation on the Storage disk was necessary.


My idea is to pack all logik on the USB Stick and all storage hdd's  
over glusterfs in the cluster.


But in proxmox the logic is shared over all nodes, so the main idea  
from proxmox seam a other one.


Bye

Gregor

___
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] Some question to proxmox cluster

2017-09-10 Thread Dietmar Maurer 
> One first question: wenn a VM is assigned to an node an the node faile (some 
> hard case like power or an atomic bomb). Could I start the VM on one of the 
> other nodes?

You need to 'steal' the VM from that node. Please note that you need
to be absolutely sure that the node is down.

Then run

# mv /etc/pve/nodes//qemu-server/.conf
/etc/pve/nodes//qemu-server/

Note: HA manager can do that automaticaly

___
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] Some question to proxmox cluster

2017-09-10 Thread Gregor Burck 



You need to enable HA

Thank you, I guessed it,...

Bye

Gregor

___
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] Some question to proxmox cluster

2017-09-10 Thread Gilberto Nunes 
You need to enable HA

Em 10 de set de 2017 08:25, "Gregor Burck"  escreveu:

> Hi,
>
> I setup an pve cluster on three nodes. I manage to setup a
> glusterfs-cluster
> on the three nodes to. So far seem all work well, I could migrate the VM
> from
> one to other nodes, clone and things that bump ;-)
>
> One first question: wenn a VM is assigned to an node an the node faile
> (some
> hard case like power or an atomic bomb). Could I start the VM on one of the
> other nodes?
>
> I test to switch down the node1 where the VMs 're running. In the
> datacenterview I still see them. I try to start them or igrate them, but I
> got
> an connection error.
>
> The images exist in the gluster-cluster, so them could reach from every
> node.
>
> Is there a trick to get them running on the two other pve nodes?
>
> Bye
>
> Gregor
> ___
> pve-user mailing list
> pve-user@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>
___
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

[PVE-User] Some question to proxmox cluster

2017-09-10 Thread Gregor Burck 
Hi,

I setup an pve cluster on three nodes. I manage to setup a glusterfs-cluster 
on the three nodes to. So far seem all work well, I could migrate the VM from 
one to other nodes, clone and things that bump ;-)

One first question: wenn a VM is assigned to an node an the node faile (some 
hard case like power or an atomic bomb). Could I start the VM on one of the 
other nodes?

I test to switch down the node1 where the VMs 're running. In the 
datacenterview I still see them. I try to start them or igrate them, but I got 
an connection error.

The images exist in the gluster-cluster, so them could reach from every node.

Is there a trick to get them running on the two other pve nodes?

Bye

Gregor
___
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user