Hi David,
I just add a new OSD with two disk (10 in total 24GB per osd.x)
But I get this error now:
[ceph@cephadm ceph-cloud]$ ceph -w
cluster 344c60e2-cef8-41f3-92ae-1995b0abc870
health HEALTH_ERR 4 pgs incomplete; 15 pgs peering; 19 pgs stale; 46
pgs stuck inactive; 19 pgs stuck stale; 47 pgs stuck unclean; 10 full
osd(s); pool rbd pg_num 256 > pgp_num 64
monmap e2: 3 mons at {ceph02=
10.10.3.2:6789/0,ceph03=10.10.3.3:6789/0,cephadm=10.10.3.1:6789/0},
election epoch 12, quorum 0,1,2 cephadm,ceph02,ceph03
osdmap e110: 10 osds: 10 up, 10 in
flags full
pgmap v458: 384 pgs, 3 pools, 0 bytes data, 0 objects
865 GB used, 0 kB / 885 GB avail
27 creating
4 stale+incomplete
337 active+clean
1 active+remapped
15 stale+peering
2014-06-25 23:31:09.363826 osd.5 [ERR] OSD full dropping all updates 100%
full
2014-06-25 23:31:19.946287 osd.4 [ERR] OSD full dropping all updates 100%
full
2014-06-25 23:31:24.439105 osd.6 [ERR] OSD full dropping all updates 100%
full
2014-06-25 23:31:23.230226 osd.9 [ERR] OSD full dropping all updates 100%
full
2014-06-25 23:31:22.088065 osd.3 [ERR] OSD full dropping all updates 100%
full
2014-06-25 23:31:27.695166 osd.0 [ERR] OSD full dropping all updates 100%
full
2014-06-25 23:31:29.681142 osd.1 [ERR] OSD full dropping all updates 100%
full
2014-06-25 23:31:32.730167 osd.2 [ERR] OSD full dropping all updates 100%
full
2014-06-25 23:31:36.036583 osd.8 [ERR] OSD full dropping all updates 100%
full
There is a couple of things taht I don't understand:jo
why I see 885 GB avail when I only add 10x24 =240GB, under /data/osd...?
^C[ceph@cephadm ceph-cloud]$ ceph osd tree
# id weight type name up/down reweight
-1 0.3199 root default
-2 0.14 host ceph02
0 0 osd.0 up 1
2 0.01999 osd.2 up 1
6 0.01999 osd.6 up 1
8 0.09999 osd.8 up 1
-3 0.14 host ceph03
7 0.01999 osd.7 up 1
1 0 osd.1 up 1
3 0.01999 osd.3 up 1
9 0.09999 osd.9 up 1
-4 0.03998 host ceph04
4 0.01999 osd.4 up 1
5 0.01999 osd.5 up 1
Why is the OSDs full?
Should be a minimal journal size for a test deployment?
Thanks in advance, I
2014-06-25 20:21 GMT+02:00 David Zafman <[email protected]>:
>
> Create a 3rd OSD. The default pool size is 3 replicas including the
> initial system created pools.
>
> David Zafman
> Senior Developer
> http://www.inktank.com
> http://www.redhat.com
>
> On Jun 25, 2014, at 3:04 AM, Iban Cabrillo <[email protected]>
> wrote:
>
> Dear,
> I am trying to deploy a new test following the instructions. for the
> latest firefly version under yum repo.
>
> Installing : ceph-libs-0.80.1-2.el6.x86_64
>
> Installing : ceph-0.80.1-2.el6.x86_64
>
> The initial setups contains 3 mon and little osds (1GB per journal)
>
> The cluster has been created correctly.
> The OSD's too with a little trouble (I had to make by hand the osd dir
> under /var/lib/ceph/, mayby this is a bug) :
> ....
> [ceph02][WARNIN] ceph-disk: Error: unable to create symlink
> /var/lib/ceph/osd/ceph-0 -> /data/osd0
> [ceph03][WARNIN] ceph-disk: Error: unable to create symlink
> /var/lib/ceph/osd/ceph-1 -> /data/osd1
> ....
> Now the status show:
>
> [ceph@cephadm ceph-cloud]$ sudo ceph status
> cluster 344c60e2-cef8-41f3-92ae-1995b0abc870
> *health HEALTH_WARN 192 pgs incomplete; 192 pgs stuck inactive; 192
> pgs stuck unclean*
> monmap e2: 3 mons at {ceph02=
> 10.10.3.2:6789/0,ceph03=10.10.3.3:6789/0,cephadm=10.10.3.1:6789/0},
> election epoch 6, quorum 0,1,2 cephadm,ceph02,ceph03
> osdmap e8: 2 osds: 2 up, 2 in
> pgmap v15: 192 pgs, 3 pools, 0 bytes data, 0 objects
> 2120 MB used, 1705 MB / 4030 MB avail
> 192 incomplete
>
> [ceph@cephadm ceph-cloud]$ ceph osd tree
> # id weight type name up/down reweight
> -1 0 root default
> -2 0 host ceph02
> 0 0 osd.0 up 1
> -3 0 host ceph03
> 1 0 osd.1 up 1
>
> [ceph@cephadm ceph-cloud]$ ceph osd dump
> epoch 8
> fsid 344c60e2-cef8-41f3-92ae-1995b0abc870
> created 2014-06-25 11:03:51.830572
> modified 2014-06-25 11:11:57.789185
> flags
> pool 0 'data' replicated size 3 min_size 2 crush_ruleset 0 object_hash
> rjenkins pg_num 64 pgp_num 64 last_change 1 owner 0 flags hashpspool
> crash_replay_interval 45 stripe_width 0
> pool 1 'metadata' replicated size 3 min_size 2 crush_ruleset 0 object_hash
> rjenkins pg_num 64 pgp_num 64 last_change 1 owner 0 flags hashpspool
> stripe_width 0
> pool 2 'rbd' replicated size 3 min_size 2 crush_ruleset 0 object_hash
> rjenkins pg_num 64 pgp_num 64 last_change 1 owner 0 flags hashpspool
> stripe_width 0
> max_osd 2
> osd.0 up in weight 1 up_from 4 up_thru 5 down_at 0 last_clean_interval
> [0,0) 10.10.3.2:6800/16295 10.10.3.2:6801/16295 10.10.3.2:6802/16295
> 10.10.3.2:6803/16295 exists,up ef41c1d6-4510-44fd-af48-81986c0f6a1e
> osd.1 up in weight 1 up_from 8 up_thru 0 down_at 0 last_clean_interval
> [0,0) 10.10.3.3:6800/14463 10.10.3.3:6801/14463 10.10.3.3:6802/14463
> 10.10.3.3:6803/14463 exists,up 6f2b4d9f-8b7
>
>
> [global]
> auth_service_required = cephx
> filestore_xattr_use_omap = true
> auth_client_required = cephx
> auth_cluster_required = cephx
> mon_host = 10.10.3.1,10.10.3.2,10.10.3.3
> mon_initial_members = cephadm, ceph02, ceph03
> fsid = 344c60e2-cef8-41f3-92ae-1995b0abc870
>
> [default]
> osd pool default size = 2
>
> [osd]
> osd journal size = 1024
>
> Any Idea why there is no active + clean state?
>
> I have tried to create a storage pool and mount from ceph client, but all
> the rbd commands hang forever...
>
> regards, I
> Bertrand Russell:
> *"El problema con el mundo es que los estúpidos están seguros de todo y
> los inteligentes están llenos de dudas*"
> _______________________________________________
> ceph-users mailing list
> [email protected]
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
>
--
############################################################################
Iban Cabrillo Bartolome
Instituto de Fisica de Cantabria (IFCA)
Santander, Spain
Tel: +34942200969
PGP PUBLIC KEY:
http://pgp.mit.edu/pks/lookup?op=get&search=0xD9DF0B3D6C8C08AC
############################################################################
Bertrand Russell:
*"El problema con el mundo es que los estúpidos están seguros de todo y los
inteligentes están llenos de dudas*"
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
