RE: Memstore performance improvements v0.90 vs v0.87

2015-01-14 Thread Somnath Roy
Stephen, You may want to tweak the following parameter(s) in your ceph.conf file and see if it is further improving your performance or not. debug_lockdep = 0/0 debug_context = 0/0 debug_crush = 0/0 debug_buffer = 0/0 debug_timer = 0/0 debug_filer = 0/0 debug_objecter = 0/0 debug_rados = 0/0

v0.91 contributors credits

2015-01-14 Thread Loic Dachary
Hi Ceph, Here is a sorted list of authors and organizations who contributed to v0.91, by number of commits or reviews back to v0.90. The affiliation of authors to organizations can be updated by submitting a patch to https://github.com/ceph/ceph/blob/master/.organizationmap All commits are

RE: Memstore performance improvements v0.90 vs v0.87

2015-01-14 Thread Blinick, Stephen L
I went back and grabbed 87 and built it on RHEL7 as well, and performance is also similar (much better). I've also run it on a few systems (Dual socket 10-core E5v2, Dual socket 6-core E5v3). So, it's related to my switch to RHEL7, and not to the code changes between v0.90 and v0.87.

v0.80.8 Firefly released

2015-01-14 Thread Sage Weil
This is a long-awaited bugfix release for firefly. It has several imporant (but relatively rare) OSD peering fixes, performance issues when snapshots are trimmed, several RGW fixes, a paxos corner case fix, and some packaging updates. We recommend that all users for v0.80.x firefly upgrade

RE: Memstore performance improvements v0.90 vs v0.87

2015-01-14 Thread Blinick, Stephen L
Haha :) Well, my intuition is still pointing to something I've configured wrong (or had wrong).. but it will be interesting to see what it is. -Original Message- From: Mark Nelson [mailto:mark.nel...@inktank.com] Sent: Wednesday, January 14, 2015 3:43 PM To: Blinick, Stephen L; Ceph

RE: Memstore performance improvements v0.90 vs v0.87

2015-01-14 Thread Blinick, Stephen L
Somnath -- thanks for the info. . I held out for a while but eventually settled on running without debug as you've specified below.. I am not currently disabling crc signatures on messages, however. I can see if that makes an improvement. The latency reduction for single-client

v0.91 released

2015-01-14 Thread Sage Weil
We are quickly approaching the Hammer feature freeze but have a few more dev releases to go before we get there. The headline items are subtree-based quota support in CephFS (ceph-fuse/libcephfs client support only for now), a rewrite of the watch/notify librados API used by RBD and RGW,

Re: RGW's apache and mod_fastcgi

2015-01-14 Thread Ken Dreyer
On 01/06/2015 12:14 PM, Ken Dreyer wrote: There's the question of Unix Domain Sockets support. In Yehuda's testing with mod_proxy_fcgi, Unix Domain Sockets (UDS) provide a lot better performance in comparison to TCP sockets. Apache's UDS support shipped in Apache upstream version 2.4.9, but

Re: ERROR: assert(last_e.version.version e.version.version)

2015-01-14 Thread Samuel Just
Repairing the pg_log with kv-store-tool would probably be your best bet. -Sam On Wed, Jan 14, 2015 at 1:02 AM, Nicheal zay11...@gmail.com wrote: Yeah, I am convinced that the filesystem inconsistencies lead to a problem. Since in my log: 2015-01-06 00:03:06.145093 7fc6bf0c47a0 0 log [ERR] :

RE: Cache pool latency impact

2015-01-14 Thread Sage Weil
On Wed, 14 Jan 2015, Pavan Rallabhandi wrote: Thanks for the reply Sage; please ignore the same subject mails on ceph-users, they seem to have got delivered today. Hmm, we could have a 'noagent' option (similar to noout, nobackfill, noscrub, etc.) that lets the admin tell the system to

Re: Some gitbuilders not working

2015-01-14 Thread Ken Dreyer
On 01/09/2015 08:16 AM, Sage Weil wrote: On Fri, 9 Jan 2015, Ken Dreyer wrote: On 01/09/2015 03:33 AM, Loic Dachary wrote: On 09/01/2015 07:55, David Zafman wrote: We are seeing gitbuilder failures. This is what I saw on one. error: Failed build dependencies: xmlstarlet is needed by

Re: Some gitbuilders not working

2015-01-14 Thread Loic Dachary
Hi, On 14/01/2015 17:56, Ken Dreyer wrote: On 01/09/2015 08:16 AM, Sage Weil wrote: On Fri, 9 Jan 2015, Ken Dreyer wrote: On 01/09/2015 03:33 AM, Loic Dachary wrote: On 09/01/2015 07:55, David Zafman wrote: We are seeing gitbuilder failures. This is what I saw on one. error: Failed build

Re: Some gitbuilders not working

2015-01-14 Thread Sage Weil
On Wed, 14 Jan 2015, Ken Dreyer wrote: On 01/09/2015 08:16 AM, Sage Weil wrote: On Fri, 9 Jan 2015, Ken Dreyer wrote: On 01/09/2015 03:33 AM, Loic Dachary wrote: On 09/01/2015 07:55, David Zafman wrote: We are seeing gitbuilder failures. This is what I saw on one. error: Failed

RE: Cache pool latency impact

2015-01-14 Thread Pavan Rallabhandi
Thanks for the reply Sage; please ignore the same subject mails on ceph-users, they seem to have got delivered today. Hmm, we could have a 'noagent' option (similar to noout, nobackfill, noscrub, etc.) that lets the admin tell the system to stop tiering movements, but I'm not sure that's

Re: ERROR: assert(last_e.version.version e.version.version)

2015-01-14 Thread Nicheal
Yeah, I am convinced that the filesystem inconsistencies lead to a problem. Since in my log: 2015-01-06 00:03:06.145093 7fc6bf0c47a0 0 log [ERR] : 2.5b log bound mismatch, info (31150'2118828,31150'2121829] actual [31150'2118824,31150'2121829] 2015-01-06 00:03:06.596293 7fc6bf0c47a0 0 log

[PATCH 01/12] fs: deduplicate noop_backing_dev_info

2015-01-14 Thread Christoph Hellwig
hugetlbfs, kernfs and dlmfs can simply use noop_backing_dev_info instead of creating a local duplicate. Signed-off-by: Christoph Hellwig h...@lst.de Acked-by: Tejun Heo t...@kernel.org --- fs/hugetlbfs/inode.c| 14 +- fs/kernfs/inode.c | 14 +-

backing_dev_info cleanups lifetime rule fixes V2

2015-01-14 Thread Christoph Hellwig
The first 8 patches are unchanged from the series posted a week ago and cleans up how we use the backing_dev_info structure in preparation for fixing the life time rules for it. The most important change is to split the unrelated nommu mmap flags from it, but it also remove a backing_dev_info

[PATCH 12/12] fs: remove default_backing_dev_info

2015-01-14 Thread Christoph Hellwig
Now that default_backing_dev_info is not used for writeback purposes we can git rid of it easily: - instead of using it's name for tracing unregistered bdi we just use unknown - btrfs and ceph can just assign the default read ahead window themselves like several other filesystems already

[PATCH 06/12] nilfs2: set up s_bdi like the generic mount_bdev code

2015-01-14 Thread Christoph Hellwig
mapping-backing_dev_info will go away, so don't rely on it. Signed-off-by: Christoph Hellwig h...@lst.de Acked-by: Ryusuke Konishi konishi.ryus...@lab.ntt.co.jp Reviewed-by: Tejun Heo t...@kernel.org --- fs/nilfs2/super.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git

[PATCH 03/12] fs: introduce f_op-mmap_capabilities for nommu mmap support

2015-01-14 Thread Christoph Hellwig
Since BDI: Provide backing device capability information [try #3] the backing_dev_info structure also provides flags for the kind of mmap operation available in a nommu environment, which is entirely unrelated to it's original purpose. Introduce a new nommu-only file operation to provide this

[PATCH 11/12] fs: don't reassign dirty inodes to default_backing_dev_info

2015-01-14 Thread Christoph Hellwig
If we have dirty inodes we need to call the filesystem for it, even if the device has been removed and the filesystem will error out early. The current code does that by reassining all dirty inodes to the default backing_dev_info when a bdi is unlinked, but that's pretty pointless given that the

[PATCH 02/12] fs: kill BDI_CAP_SWAP_BACKED

2015-01-14 Thread Christoph Hellwig
This bdi flag isn't too useful - we can determine that a vma is backed by either swap or shmem trivially in the caller. This also allows removing the backing_dev_info instaces for swap and shmem in favor of noop_backing_dev_info. Signed-off-by: Christoph Hellwig h...@lst.de Reviewed-by: Tejun

[PATCH 07/12] fs: export inode_to_bdi and use it in favor of mapping-backing_dev_info

2015-01-14 Thread Christoph Hellwig
Now that we got rid of the bdi abuse on character devices we can always use sb-s_bdi to get at the backing_dev_info for a file, except for the block device special case. Export inode_to_bdi and replace uses of mapping-backing_dev_info with it to prepare for the removal of

[PATCH 04/12] block_dev: only write bdev inode on close

2015-01-14 Thread Christoph Hellwig
Since 018a17bdc865 (bdi: reimplement bdev_inode_switch_bdi()) the block device code writes out all dirty data whenever switching the backing_dev_info for a block device inode. But a block device inode can only be dirtied when it is in use, which means we only have to write it out on the final

[PATCH 09/12] ceph: remove call to bdi_unregister

2015-01-14 Thread Christoph Hellwig
bdi_destroy already does all the work, and if we delay freeing the anon bdev we can get away with just that single call. Signed-off-by: Christoph Hellwig h...@lst.de --- fs/ceph/super.c | 18 ++ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/fs/ceph/super.c

[PATCH 08/12] fs: remove mapping-backing_dev_info

2015-01-14 Thread Christoph Hellwig
Now that we never use the backing_dev_info pointer in struct address_space we can simply remove it and save 4 to 8 bytes in every inode. Signed-off-by: Christoph Hellwig h...@lst.de Acked-by: Ryusuke Konishi konishi.ryus...@lab.ntt.co.jp Reviewed-by: Tejun Heo t...@kernel.org ---

[PATCH 10/12] nfs: don't call bdi_unregister

2015-01-14 Thread Christoph Hellwig
bdi_destroy already does all the work, and if we delay freeing the anon bdev we can get away with just that single call. Addintionally remove the call during mount failure, as deactivate_super_locked will already call -kill_sb and clean up the bdi for us. Signed-off-by: Christoph Hellwig

[PATCH 05/12] block_dev: get bdev inode bdi directly from the block device

2015-01-14 Thread Christoph Hellwig
Directly grab the backing_dev_info from the request_queue instead of detouring through the address_space. Signed-off-by: Christoph Hellwig h...@lst.de Reviewed-by: Tejun Heo t...@kernel.org --- fs/fs-writeback.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git

Re: [PATCH 11/12] fs: don't reassign dirty inodes to default_backing_dev_info

2015-01-14 Thread Jan Kara
On Wed 14-01-15 10:42:40, Christoph Hellwig wrote: If we have dirty inodes we need to call the filesystem for it, even if the device has been removed and the filesystem will error out early. The current code does that by reassining all dirty inodes to the default backing_dev_info when a bdi

Re: [PATCH 09/12] ceph: remove call to bdi_unregister

2015-01-14 Thread Jan Kara
On Wed 14-01-15 10:42:38, Christoph Hellwig wrote: bdi_destroy already does all the work, and if we delay freeing the anon bdev we can get away with just that single call. Looks good. You can add: Reviewed-by: Jan Kara j...@suse.cz

Re: [PATCH 10/12] nfs: don't call bdi_unregister

2015-01-14 Thread Jan Kara
On Wed 14-01-15 10:42:39, Christoph Hellwig wrote: bdi_destroy already does all the work, and if we delay freeing the anon bdev we can get away with just that single call. Addintionally remove the call during mount failure, as deactivate_super_locked will already call -kill_sb and clean up

Re: [PATCH 04/12] block_dev: only write bdev inode on close

2015-01-14 Thread Jan Kara
On Wed 14-01-15 10:42:33, Christoph Hellwig wrote: Since 018a17bdc865 (bdi: reimplement bdev_inode_switch_bdi()) the block device code writes out all dirty data whenever switching the backing_dev_info for a block device inode. But a block device inode can only be dirtied when it is in use,

Re: [PATCH 05/12] block_dev: get bdev inode bdi directly from the block device

2015-01-14 Thread Jan Kara
On Wed 14-01-15 10:42:34, Christoph Hellwig wrote: Directly grab the backing_dev_info from the request_queue instead of detouring through the address_space. Signed-off-by: Christoph Hellwig h...@lst.de Reviewed-by: Tejun Heo t...@kernel.org Looks good. You can add: Reviewed-by: Jan Kara

Re: [PATCH 01/12] fs: deduplicate noop_backing_dev_info

2015-01-14 Thread Jan Kara
On Wed 14-01-15 10:42:30, Christoph Hellwig wrote: hugetlbfs, kernfs and dlmfs can simply use noop_backing_dev_info instead of creating a local duplicate. Looks good. Reviewed-by: Jan Kara j...@suse.cz Honza Signed-off-by:

Re: [PATCH 02/12] fs: kill BDI_CAP_SWAP_BACKED

2015-01-14 Thread Jan Kara
On Wed 14-01-15 10:42:31, Christoph Hellwig wrote: This bdi flag isn't too useful - we can determine that a vma is backed by either swap or shmem trivially in the caller. This also allows removing the backing_dev_info instaces for swap and shmem in favor of noop_backing_dev_info.

Re: [PATCH 06/12] nilfs2: set up s_bdi like the generic mount_bdev code

2015-01-14 Thread Jan Kara
On Wed 14-01-15 10:42:35, Christoph Hellwig wrote: mapping-backing_dev_info will go away, so don't rely on it. Signed-off-by: Christoph Hellwig h...@lst.de Acked-by: Ryusuke Konishi konishi.ryus...@lab.ntt.co.jp Reviewed-by: Tejun Heo t...@kernel.org --- fs/nilfs2/super.c | 4 +--- 1

Re: [PATCH 12/12] fs: remove default_backing_dev_info

2015-01-14 Thread Jan Kara
On Wed 14-01-15 10:42:41, Christoph Hellwig wrote: Now that default_backing_dev_info is not used for writeback purposes we can git rid of it easily: - instead of using it's name for tracing unregistered bdi we just use unknown - btrfs and ceph can just assign the default read ahead