[ceph-users] More writes on filestore than on journal ?
Hi, I'm still trying to find why there is much more write operations on filestore since Emperor/Firefly than from Dumpling. So, I add monitoring of all perf counters values from OSD. From what I see : «filestore.ops» reports an average of 78 operations per seconds. But, block device monitoring reports an average of 113 operations per seconds (+45%). please thoses 2 graphs : - https://daevel.fr/img/firefly/osd-70.filestore-ops.png - https://daevel.fr/img/firefly/osd-70.sda-ops.png Do you see what can explain this difference ? (this OSD use XFS) Thanks, Olivier ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] More writes on filestore than on journal ?
On Mon, Mar 23, 2015 at 6:21 AM, Olivier Bonvalet ceph.l...@daevel.fr wrote: Hi, I'm still trying to find why there is much more write operations on filestore since Emperor/Firefly than from Dumpling. Do you have any history around this? It doesn't sound familiar, although I bet it's because of the WBThrottle and flushing changes. So, I add monitoring of all perf counters values from OSD. From what I see : «filestore.ops» reports an average of 78 operations per seconds. But, block device monitoring reports an average of 113 operations per seconds (+45%). please thoses 2 graphs : - https://daevel.fr/img/firefly/osd-70.filestore-ops.png - https://daevel.fr/img/firefly/osd-70.sda-ops.png That's unfortunate but perhaps not surprising — any filestore op can change a backing file (which requires hitting both the file and the inode: potentially two disk seeks), as well as adding entries to the leveldb instance. -Greg Do you see what can explain this difference ? (this OSD use XFS) Thanks, Olivier ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] More writes on filestore than on journal ?
Hi, Le lundi 23 mars 2015 à 07:29 -0700, Gregory Farnum a écrit : On Mon, Mar 23, 2015 at 6:21 AM, Olivier Bonvalet ceph.l...@daevel.fr wrote: Hi, I'm still trying to find why there is much more write operations on filestore since Emperor/Firefly than from Dumpling. Do you have any history around this? It doesn't sound familiar, although I bet it's because of the WBThrottle and flushing changes. I only have history for block device stats and global stats reports by «ceph status». When I have upgrade from Dumpling to Firefly (via Emperor), write operations increased a lot on OSD. I suppose it's because of WBThrottle too, but can't find any parameter able to confirm that. So, I add monitoring of all perf counters values from OSD. From what I see : «filestore.ops» reports an average of 78 operations per seconds. But, block device monitoring reports an average of 113 operations per seconds (+45%). please thoses 2 graphs : - https://daevel.fr/img/firefly/osd-70.filestore-ops.png - https://daevel.fr/img/firefly/osd-70.sda-ops.png That's unfortunate but perhaps not surprising — any filestore op can change a backing file (which requires hitting both the file and the inode: potentially two disk seeks), as well as adding entries to the leveldb instance. -Greg Ok thanks, so this part can be «normal». Do you see what can explain this difference ? (this OSD use XFS) Thanks, Olivier ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
