Thanks!
It seems to work!
I configure my cluster's crush rulesest according to
https://elkano.org/blog/ceph-sata-ssd-pools-server-editing-crushmap/.
Then restart my cluster, things looks like ok.
My tests have not finished.
Go on making tier-cache.
ceph osd tier add images images ssdpool
ceph osd tier cache-mode ssdpool writeback
ceph osd tier add images volumes ssdpool
ceph osd tier cache-mode ssdpool writeback
But 'ceph -s' replys:
1 cache pools are missing hit_sets
And then 'ceph osd tree' replys(a bit longer):
ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY -10
0.59999 root default-ssd -6
0.14000 host bjd-01-control1-ssd 3
0.06999 osd.3 down 1.00000 1.00000 4
0.06999 osd.4 down 1.00000 1.00000 -7
0.14000 host bjd-01-control2-ssd 11
0.06999 osd.11 down 1.00000 1.00000 12
0.06999 osd.12 down 1.00000 1.00000 -8
0.17999 host bjd-01-compute1-ssd 18
0.09000 osd.18 down 1.00000 1.00000 19
0.09000 osd.19 down 1.00000 1.00000 -9
0.14000 host bjd-01-compute2-ssd 28
0.06999 osd.28 down 1.00000 1.00000 29
0.06999 osd.29 down 1.00000 1.00000 -1
6.06000 root default -2
1.50000 host bjd-01-control1 0
0.25000 osd.0 up 1.00000 1.00000 2
0.25000 osd.2 up 1.00000 1.00000 5
0.25000 osd.5 up 1.00000 1.00000 6
0.25000 osd.6 up 1.00000 1.00000 22
0.25000 osd.22 up 1.00000 1.00000 23
0.25000 osd.23 up 1.00000 1.00000 -3
1.50000 host bjd-01-control2 7
0.25000 osd.7 up 1.00000 1.00000 8
0.25000 osd.8 up 1.00000 1.00000 9
0.25000 osd.9 up 1.00000 1.00000 10
0.25000 osd.10 up 1.00000 1.00000 13
0.25000 osd.13 up 1.00000 1.00000 14
0.25000 osd.14 up 1.00000 1.00000 -4
1.56000 host bjd-01-compute1 15
0.25000 osd.15 up 1.00000 1.00000 16
0.25000 osd.16 up 1.00000 1.00000 17
0.26999 osd.17 up 1.00000 1.00000 20
0.26999 osd.20 up 1.00000 1.00000 21
0.26999 osd.21 up 1.00000 1.00000 1
0.25000 osd.1 up 1.00000 1.00000 -5
1.50000 host bjd-01-compute2 24
0.25000 osd.24 up 1.00000 1.00000 25
0.25000 osd.25 up 1.00000 1.00000 26
0.25000 osd.26 up 1.00000 1.00000 27
0.25000 osd.27 up 1.00000 1.00000 30
0.25000 osd.30 up 1.00000 1.00000 31
0.25000 osd.31 up 1.00000 1.00000
In the end, i run ceph-osd manually, 'ceph-osd -i 12 -c /etc/ceph/ceph.conf -f':
SG_IO: bad/missing sense data, sb[]: 70 00 05 00 00 00 00 0d 00 00 00 00 20 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
SG_IO: bad/missing sense data, sb[]: 70 00 05 00 00 00 00 0d 00 00 00 00 20 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2016-06-08 06:21:58.335164 7fc376d74880 -1 osd.12 383 log_to_monitors
{default=true}
./include/interval_set.h: In function 'void interval_set<T>::erase(T, T) [with
T = snapid_t]' thread 7fc34dbe9700 time 2016-06-08 06:21:58.341270
./include/interval_set.h: 386: FAILED assert(_size >= 0)
./include/interval_set.h: In function 'void interval_set<T>::erase(T, T) [with
T = snapid_t]' thread 7fc34d3e8700 time 2016-06-08 06:21:58.341246
./include/interval_set.h: 386: FAILED assert(_size >= 0)
./include/interval_set.h: In function 'void interval_set<T>::erase(T, T) [with
T = snapid_t]' thread 7fc34c3e6700 time 2016-06-08 06:21:58.342349
./include/interval_set.h: 386: FAILED assert(_size >= 0)
./include/interval_set.h: In function 'void interval_set<T>::erase(T, T) [with
T = snapid_t]' thread 7fc34abe3700 time 2016-06-08 06:21:58.341558
./include/interval_set.h: 386: FAILED assert(_size >= 0)
./include/interval_set.h: In function 'void interval_set<T>::erase(T, T) [with
T = snapid_t]' thread 7fc34bbe5700 time 2016-06-08 06:21:58.342344
./include/interval_set.h: 386: FAILED assert(_size >= 0)
ceph version 0.94.6 (e832001feaf8c176593e0325c8298e3f16dfb403)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x85)
[0xbc9195]
2: (interval_set<snapid_t>::subtract(interval_set<snapid_t> const&)+0xc0)
[0x81ec80]
3: (PGPool::update(std::tr1::shared_ptr<OSDMap const>)+0x54e) [0x7f21be]
4: (PG::handle_advance_map(std::tr1::shared_ptr<OSDMap const>,
std::tr1::shared_ptr<OSDMap const>, std::vector<int, std::allocator<int> >&,
int, std::vector<int, std::allocator<int> >&, int, PG::RecoveryCtx*)+0x2a2)
[0x7f26f2]
5: (OSD::advance_pg(unsigned int, PG*, ThreadPool::TPHandle&,
PG::RecoveryCtx*, std::set<boost::intrusive_ptr<PG>,
std::less<boost::intrusive_ptr<PG> >, std::allocator<boost::intrusive_ptr<PG> >
>*)+0x2da) [0x6a591a]
6: (OSD::process_peering_events(std::list<PG*, std::allocator<PG*> > const&,
ThreadPool::TPHandle&)+0x22c) [0x6a643c]
7: (OSD::PeeringWQ::_process(std::list<PG*, std::allocator<PG*> > const&,
ThreadPool::TPHandle&)+0x28) [0x701d88]
8: (ThreadPool::worker(ThreadPool::WorkThread*)+0xa76) [0xbb9966]
9: (ThreadPool::WorkThread::entry()+0x10) [0xbba9f0]
10: (()+0x7dc5) [0x7fc3756ffdc5]
11: (clone()+0x6d) [0x7fc3741e228d]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to
interpret this.
2016-06-08 06:21:58.355662 7fc34d3e8700 -1 ./include/interval_set.h: In
function 'void interval_set<T>::erase(T, T) [with T = snapid_t]' thread
7fc34d3e8700 time 2016-06-08 06:21:58.341246
./include/interval_set.h: 386: FAILED assert(_size >= 0)
Q: can a pool tier to other pools more than once? like my case,
ssdpool--tier--images & ssdpool--tier--volumes.
Best regards,
Xiucai
------------------ ???????? ------------------
??????: "Christian Balzer";<[email protected]>;
????????: 2016??6??8??(??????) ????11:16
??????: "ceph-users"<[email protected]>;
????: "????"<[email protected]>;
????: Re: [ceph-users] Must host bucket name be the same with hostname ?
Hello,
you will want to read:
https://www.sebastien-han.fr/blog/2014/08/25/ceph-mix-sata-and-ssd-within-the-same-box/
especially section III and IV.
Another approach w/o editing the CRUSH map is here:
https://elkano.org/blog/ceph-sata-ssd-pools-server-editing-crushmap/
Christian
On Wed, 8 Jun 2016 10:54:36 +0800 ???? wrote:
> Hi all,
>
> There are SASes & SSDs in my nodes at the same time.
> Now i want divide them into 2 groups, one composed of SASes and one
> only contained SSDs. When i configure CRUSH rulesets, segment below:
>
>
> # buckets
> host robert-a {
> id -2 # do not change unnecessarily
> # weight 1.640
> alg straw
> hash 0 # rjenkins1
> item osd.0 weight 0.250 #SAS
> item osd.1 weight 0.250 #SAS
> item osd.2 weight 0.250 #SSD
> item osd.3 weight 0.250 #SSD
>
> }
>
>
> So, i am not sure must host bucket name be the same with hostname.
>
>
> Or host bucket name does no matter?
>
>
>
> Best regards,
>
> Xiucai
--
Christian Balzer Network/Systems Engineer
[email protected] Global OnLine Japan/Rakuten Communications
http://www.gol.com/_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
