osd creation is hang and after some time it disconnects. ceph deploying on AWS
instances.
Having 800G ssd disk on siteAosd:/dev/xvdb
sdsuser@siteAmon:~/cluster$ ceph -v
ceph version 10.2.2 (45107e21c568dd033c2f0a3107dec8f0b0e58374)
sdsuser@siteAmon:~/cluster$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.4 LTS
Release: 14.04
Codename: trusty
sdsuser@siteAmon:~/cluster$
sdsuser@siteAmon:~/cluster$ ceph-deploy --overwrite-conf osd create
siteAosd:/dev/xvdb
[ceph_deploy.conf][DEBUG ] found configuration file at:
/home/sdsuser/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.34): /usr/bin/ceph-deploy
--overwrite-conf osd create siteAosd:/dev/xvdb
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] disk : [('siteAosd',
'/dev/xvdb', None)]
[ceph_deploy.cli][INFO ] dmcrypt : False
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] bluestore : None
[ceph_deploy.cli][INFO ] overwrite_conf : True
[ceph_deploy.cli][INFO ] subcommand : create
[ceph_deploy.cli][INFO ] dmcrypt_key_dir :
/etc/ceph/dmcrypt-keys
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf :
<ceph_deploy.conf.cephdeploy.Conf instance at 0x7f49f7665830>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] fs_type : xfs
[ceph_deploy.cli][INFO ] func : <function osd at
0x7f49f7abd848>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] zap_disk : False
[ceph_deploy.osd][DEBUG ] Preparing cluster ceph disks siteAosd:/dev/xvdb:
[siteAosd][DEBUG ] connection detected need for sudo
[siteAosd][DEBUG ] connected to host: siteAosd
[siteAosd][DEBUG ] detect platform information from remote host
[siteAosd][DEBUG ] detect machine type
[siteAosd][DEBUG ] find the location of an executable
[siteAosd][INFO ] Running command: sudo /sbin/initctl version
[siteAosd][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: Ubuntu 14.04 trusty
[ceph_deploy.osd][DEBUG ] Deploying osd to siteAosd
[siteAosd][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph_deploy.osd][DEBUG ] Preparing host siteAosd disk /dev/xvdb journal None
activate True
[siteAosd][DEBUG ] find the location of an executable
[siteAosd][INFO ] Running command: sudo /usr/sbin/ceph-disk -v prepare
--cluster ceph --fs-type xfs -- /dev/xvdb
[siteAosd][WARNIN] command: Running command: /usr/bin/ceph-osd --cluster=ceph
--show-config-value=fsid
[siteAosd][WARNIN] command: Running command: /usr/bin/ceph-osd
--check-allows-journal -i 0 --cluster ceph
[siteAosd][WARNIN] command: Running command: /usr/bin/ceph-osd
--check-wants-journal -i 0 --cluster ceph
[siteAosd][WARNIN] command: Running command: /usr/bin/ceph-osd
--check-needs-journal -i 0 --cluster ceph
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb uuid path is
/sys/dev/block/202:16/dm/uuid
[siteAosd][WARNIN] set_type: Will colocate journal with data on /dev/xvdb
[siteAosd][WARNIN] command: Running command: /usr/bin/ceph-osd --cluster=ceph
--show-config-value=osd_journal_size
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb uuid path is
/sys/dev/block/202:16/dm/uuid
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb uuid path is
/sys/dev/block/202:16/dm/uuid
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb uuid path is
/sys/dev/block/202:16/dm/uuid
[siteAosd][WARNIN] command: Running command: /usr/bin/ceph-conf --cluster=ceph
--name=osd. --lookup osd_mkfs_options_xfs
[siteAosd][WARNIN] command: Running command: /usr/bin/ceph-conf --cluster=ceph
--name=osd. --lookup osd_mount_options_xfs
[siteAosd][WARNIN] command: Running command: /usr/bin/ceph-conf --cluster=ceph
--name=osd. --lookup osd_fs_mount_options_xfs
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb uuid path is
/sys/dev/block/202:16/dm/uuid
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb uuid path is
/sys/dev/block/202:16/dm/uuid
[siteAosd][WARNIN] ptype_tobe_for_name: name = journal
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb uuid path is
/sys/dev/block/202:16/dm/uuid
[siteAosd][WARNIN] create_partition: Creating journal partition num 2 size 5120
on /dev/xvdb
[siteAosd][WARNIN] command_check_call: Running command: /sbin/sgdisk
--new=2:0:+5120M --change-name=2:ceph journal
--partition-guid=2:63b0f607-cf92-40c8- 91be-4e1058559157
--typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106 --mbrtogpt -- /dev/xvdb
[siteAosd][DEBUG ] The operation has completed successfully.
[siteAosd][WARNIN] update_partition: Calling partprobe on created device
/dev/xvdb
[siteAosd][WARNIN] command_check_call: Running command: /sbin/udevadm settle
--timeout=600
[siteAosd][WARNIN] command: Running command: /sbin/partprobe /dev/xvdb
[siteAosd][WARNIN] command_check_call: Running command: /sbin/udevadm settle
--timeout=600
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb uuid path is
/sys/dev/block/202:16/dm/uuid
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb uuid path is
/sys/dev/block/202:16/dm/uuid
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb2 uuid path is
/sys/dev/block/202:18/dm/uuid
[siteAosd][WARNIN] prepare_device: Journal is GPT partition
/dev/disk/by-partuuid/63b0f607-cf92-40c8-91be-4e1058559157
[siteAosd][WARNIN] prepare_device: Journal is GPT partition
/dev/disk/by-partuuid/63b0f607-cf92-40c8-91be-4e1058559157
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb uuid path is
/sys/dev/block/202:16/dm/uuid
[siteAosd][WARNIN] set_data_partition: Creating osd partition on /dev/xvdb
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb uuid path is
/sys/dev/block/202:16/dm/uuid
[siteAosd][WARNIN] ptype_tobe_for_name: name = data
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb uuid path is
/sys/dev/block/202:16/dm/uuid
[siteAosd][WARNIN] create_partition: Creating data partition num 1 size 0 on
/dev/xvdb
[siteAosd][WARNIN] command_check_call: Running command: /sbin/sgdisk
--largest-new=1 --change-name=1:ceph data
--partition-guid=1:59ec3569-5a6b-446f-bc5a -9764f32dbc8c
--typecode=1:89c57f98-2fe5-4dc0-89c1-f3ad0ceff2be --mbrtogpt -- /dev/xvdb
[siteAosd][DEBUG ] The operation has completed successfully.
[siteAosd][WARNIN] update_partition: Calling partprobe on created device
/dev/xvdb
[siteAosd][WARNIN] command_check_call: Running command: /sbin/udevadm settle
--timeout=600
[siteAosd][WARNIN] command: Running command: /sbin/partprobe /dev/xvdb
[siteAosd][WARNIN] command_check_call: Running command: /sbin/udevadm settle
--timeout=600
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb uuid path is
/sys/dev/block/202:16/dm/uuid
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb uuid path is
/sys/dev/block/202:16/dm/uuid
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb1 uuid path is
/sys/dev/block/202:17/dm/uuid
[siteAosd][WARNIN] populate_data_path_device: Creating xfs fs on /dev/xvdb1
[siteAosd][WARNIN] command_check_call: Running command: /sbin/mkfs -t xfs -f -K
-d agcount=24 -f -- /dev/xvdb1
[siteAosd][DEBUG ] meta-data=/dev/xvdb1 isize=256 agcount=24,
agsize=8085076 blks
[siteAosd][DEBUG ] = sectsz=512 attr=2,
projid32bit=0
[siteAosd][DEBUG ] data = bsize=4096
blocks=194041809, imaxpct=25
[siteAosd][DEBUG ] = sunit=0 swidth=0 blks
[siteAosd][DEBUG ] naming =version 2 bsize=4096 ascii-ci=0
[siteAosd][DEBUG ] log =internal log bsize=4096 blocks=94746,
version=2
[siteAosd][DEBUG ] = sectsz=512 sunit=0 blks,
lazy-count=1
[siteAosd][DEBUG ] realtime =none extsz=4096 blocks=0,
rtextents=0
[siteAosd][WARNIN] mount: Mounting /dev/xvdb1 on /var/lib/ceph/tmp/mnt.fohc2q
with options noatime,inode64
[siteAosd][WARNIN] command_check_call: Running command: /bin/mount -t xfs -o
noatime,inode64 -- /dev/xvdb1 /var/lib/ceph/tmp/mnt.fohc2q
[siteAosd][WARNIN] populate_data_path: Preparing osd data dir
/var/lib/ceph/tmp/mnt.fohc2q
[siteAosd][WARNIN] command: Running command: /bin/chown -R ceph:ceph
/var/lib/ceph/tmp/mnt.fohc2q/ceph_fsid.4361.tmp
[siteAosd][WARNIN] command: Running command: /bin/chown -R ceph:ceph
/var/lib/ceph/tmp/mnt.fohc2q/fsid.4361.tmp
[siteAosd][WARNIN] command: Running command: /bin/chown -R ceph:ceph
/var/lib/ceph/tmp/mnt.fohc2q/magic.4361.tmp
[siteAosd][WARNIN] command: Running command: /bin/chown -R ceph:ceph
/var/lib/ceph/tmp/mnt.fohc2q/journal_uuid.4361.tmp
[siteAosd][WARNIN] adjust_symlink: Creating symlink
/var/lib/ceph/tmp/mnt.fohc2q/journal ->
/dev/disk/by-partuuid/63b0f607-cf92-40c8-91be-4e1058559157
[siteAosd][WARNIN] command: Running command: /bin/chown -R ceph:ceph
/var/lib/ceph/tmp/mnt.fohc2q
[siteAosd][WARNIN] unmount: Unmounting /var/lib/ceph/tmp/mnt.fohc2q
[siteAosd][WARNIN] command_check_call: Running command: /bin/umount --
/var/lib/ceph/tmp/mnt.fohc2q
[siteAosd][WARNIN] get_dm_uuid: get_dm_uuid /dev/xvdb uuid path is
/sys/dev/block/202:16/dm/uuid
[siteAosd][WARNIN] command_check_call: Running command: /sbin/sgdisk
--typecode=1:4fbd7e29-9d25-41b8-afd0-062c0ceff05d -- /dev/xvdb
[siteAosd][DEBUG ] The operation has completed successfully.
[siteAosd][WARNIN] update_partition: Calling partprobe on prepared device
/dev/xvdb
[siteAosd][WARNIN] command_check_call: Running command: /sbin/udevadm settle
--timeout=600
[siteAosd][WARNIN] command: Running command: /sbin/partprobe /dev/xvdb
[siteAosd][WARNIN] command_check_call: Running command: /sbin/udevadm settle
--timeout=600
[siteAosd][WARNIN] command_check_call: Running command: /sbin/udevadm trigger
--action=add --sysname-match xvdb1
[siteAosd][INFO ] checking OSD status...
[siteAosd][DEBUG ] find the location of an executable
[siteAosd][INFO ] Running command: sudo /usr/bin/ceph --cluster=ceph osd stat
--format=json
sudo /usr/bin/ceph --cluster=ceph osd stat --format=json[siteAosd][WARNIN] No
data was received after 300 seconds, disconnecting...
[ceph_deploy.osd][DEBUG ] Host siteAosd is now ready for osd use.
sdsuser@siteAmon:~/cluster$ sudo /usr/bin/ceph --cluster=ceph osd stat
--format=json
{"epoch":1,"num_osds":0,"num_up_osds":0,"num_in_osds":0,"full":false,"nearfull":false,"num_remapped_pgs":0}sdsuser@siteAmon:~/cluster$
sdsuser@siteAmon:~/cluster$ ceph -s
cluster a9cd6ab8-2e9c-425b-a644-262a52eabf5e
health HEALTH_ERR
64 pgs are stuck inactive for more than 300 seconds
64 pgs stuck inactive
no osds
monmap e1: 1 mons at {siteAmon=10.10.100.4:6789/0}
election epoch 4, quorum 0 siteAmon
osdmap e1: 0 osds: 0 up, 0 in
flags sortbitwise
pgmap v2: 64 pgs, 1 pools, 0 bytes data, 0 objects
0 kB used, 0 kB / 0 kB avail
64 creating
sdsuser@siteAmon:~/cluster$ ceph osd tree
ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY
-1 0 root default
--Rakesh Parkiti
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
