Re: [ceph-users] RBD caching on 4K reads???

2015-02-02 Thread Somnath Roy
] On Behalf Of Bruce McFarland Sent: Monday, February 02, 2015 3:39 PM To: Mykola Golub; Udo Lembke Cc: ceph-us...@ceph.com; Prashanth Nednoor Subject: Re: [ceph-users] RBD caching on 4K reads??? I'm still missing something. I can check on the monitor to see that the running config on the cluster

Re: [ceph-users] RBD caching on 4K reads???

2015-02-02 Thread Bruce McFarland
-users] RBD caching on 4K reads??? On Fri, Jan 30, 2015 at 10:09:32PM +0100, Udo Lembke wrote: Hi Bruce, you can also look on the mon, like ceph --admin-daemon /var/run/ceph/ceph-mon.b.asok config show | grep cache rbd cache is a client setting, so you have to check this connecting to the client

Re: [ceph-users] RBD caching on 4K reads???

2015-02-02 Thread Nicheal
Message- From: Nicheal [mailto:zay11...@gmail.com] Sent: Monday, February 02, 2015 7:35 PM To: Bruce McFarland Cc: ceph-us...@ceph.com; Prashanth Nednoor Subject: Re: [ceph-users] RBD caching on 4K reads??? It seems you use the kernel rbd. So rbd_cache does not work, which is just designed

Re: [ceph-users] RBD caching on 4K reads???

2015-02-02 Thread Nicheal
It seems you use the kernel rbd. So rbd_cache does not work, which is just designed for librbd. Kernel rbd is directly using the system page cache. You said that you have already run like echo 3 /proc/sys/vm/drop_cache to invalidate all pages cached in kernel. So do you test the /dev/rbd1 based

Re: [ceph-users] RBD caching on 4K reads???

2015-02-02 Thread Bruce McFarland
. -Original Message- From: Nicheal [mailto:zay11...@gmail.com] Sent: Monday, February 02, 2015 7:35 PM To: Bruce McFarland Cc: ceph-us...@ceph.com; Prashanth Nednoor Subject: Re: [ceph-users] RBD caching on 4K reads??? It seems you use the kernel rbd. So rbd_cache does not work, which is just

Re: [ceph-users] RBD caching on 4K reads???

2015-02-02 Thread Bruce McFarland
and physically disabling kernel caching. -Original Message- From: Nicheal [mailto:zay11...@gmail.com] Sent: Monday, February 02, 2015 7:35 PM To: Bruce McFarland Cc: ceph-us...@ceph.com; Prashanth Nednoor Subject: Re: [ceph-users] RBD caching on 4K reads??? It seems you use the kernel rbd

Re: [ceph-users] RBD caching on 4K reads???

2015-02-01 Thread Mykola Golub
On Fri, Jan 30, 2015 at 10:09:32PM +0100, Udo Lembke wrote: Hi Bruce, you can also look on the mon, like ceph --admin-daemon /var/run/ceph/ceph-mon.b.asok config show | grep cache rbd cache is a client setting, so you have to check this connecting to the client admin socket. Its location is

Re: [ceph-users] RBD caching on 4K reads???

2015-01-30 Thread Udo Lembke
Hi Bruce, hmm, sounds for me like the rbd cache. Can you look, if the cache is realy disabled in the running config with ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config show | grep cache Udo On 30.01.2015 21:51, Bruce McFarland wrote: I have a cluster and have created a rbd device -

[ceph-users] RBD caching on 4K reads???

2015-01-30 Thread Bruce McFarland
I have a cluster and have created a rbd device - /dev/rbd1. It shows up as expected with 'rbd -image test info' and rbd showmapped. I have been looking at cluster performance with the usual Linux block device tools - fio and vdbench. When I look at writes and large block sequential reads I'm

Re: [ceph-users] RBD caching on 4K reads???

2015-01-30 Thread Bruce McFarland
[mailto:ulem...@polarzone.de] Sent: Friday, January 30, 2015 1:00 PM To: Bruce McFarland; ceph-us...@ceph.com Cc: Prashanth Nednoor Subject: Re: [ceph-users] RBD caching on 4K reads??? Hi Bruce, hmm, sounds for me like the rbd cache. Can you look, if the cache is realy disabled in the running config

Re: [ceph-users] RBD caching on 4K reads???

2015-01-30 Thread Udo Lembke
*To:* Bruce McFarland; ceph-us...@ceph.com *Cc:* Prashanth Nednoor *Subject:* Re: [ceph-users] RBD caching on 4K reads??? Hi Bruce, hmm, sounds for me like the rbd cache. Can you look, if the cache is realy disabled in the running config with ceph --admin-daemon /var/run/ceph/ceph-osd.0