Re: [PATCH 0/3] block I/O when cluster is full

2013-12-09 Thread Josh Durgin
On 12/09/2013 04:19 PM, Gregory Farnum wrote: On Mon, Dec 9, 2013 at 4:11 PM, Josh Durgin josh.dur...@inktank.com wrote: On 12/06/2013 06:24 PM, Gregory Farnum wrote: On Fri, Dec 6, 2013 at 6:16 PM, Josh Durgin josh.dur...@inktank.com wrote: Don't bother trying to stop ENOSPC on the client

Re: [PATCH 0/3] block I/O when cluster is full

2013-12-06 Thread Josh Durgin
On 12/05/2013 08:58 PM, Gregory Farnum wrote: On Thu, Dec 5, 2013 at 5:47 PM, Josh Durgin josh.dur...@inktank.com wrote: On 12/03/2013 03:12 PM, Josh Durgin wrote: These patches allow rbd to block writes instead of returning errors when OSDs are full enough that the FULL flag is set

Re: [PATCH 0/3] block I/O when cluster is full

2013-12-05 Thread Josh Durgin
On 12/03/2013 03:12 PM, Josh Durgin wrote: These patches allow rbd to block writes instead of returning errors when OSDs are full enough that the FULL flag is set in the osd map. This avoids filesystems on top of rbd getting confused by transient EIOs if the cluster oscillates between full

[PATCH 0/3] block I/O when cluster is full

2013-12-03 Thread Josh Durgin
-full branch of ceph-client.git. Josh Durgin (3): libceph: block I/O when PAUSE or FULL osd map flags are set libceph: add an option to configure client behavior when osds are full rbd: document rbd-specific options Documentation/ABI/testing/sysfs-bus-rbd | 19

[PATCH 3/3] rbd: document rbd-specific options

2013-12-03 Thread Josh Durgin
osd_full_behavior only affects rbd, so document it along with read-only and read-write. Signed-off-by: Josh Durgin josh.dur...@inktank.com --- Documentation/ABI/testing/sysfs-bus-rbd | 19 +++ 1 file changed, 19 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-bus

[PATCH 1/3] libceph: block I/O when PAUSE or FULL osd map flags are set

2013-12-03 Thread Josh Durgin
the monitor if any of these flags are set, so paused requests can be unblocked as soon as possible. Fixes: http://tracker.ceph.com/issues/6079 Signed-off-by: Josh Durgin josh.dur...@inktank.com --- include/linux/ceph/osd_client.h |1 + net/ceph/osd_client.c | 29

[PATCH 2/3] libceph: add an option to configure client behavior when osds are full

2013-12-03 Thread Josh Durgin
Default to blocking requests to be consistent with userspace. Some applications may prefer the previous behavior of returning an error instead, so make that an option. CephFS implements returning -ENOSPC at a higher level, so only rbd is really affected by this. Signed-off-by: Josh Durgin

Re: Osd magic detection failure in master

2013-12-02 Thread Josh Durgin
On 12/02/2013 06:42 PM, Mark Kirkwood wrote: I just updated master (a5eda4fcc34461dbc0fcc47448f8456097de15eb), and am seeing OSDs failing to start: 2013-12-03 15:37:01.291200 7f488e1157c0 -1 OSD magic != my ceph osd volume v026 failed: 'ulimit -n 32768; /usr/bin/ceph-osd -i 0 --pid-file

Re: /etc/init.d/ceph vs upstart

2013-11-25 Thread Josh Durgin
On 11/25/2013 11:01 AM, Tim Spriggs wrote: ... ping On Thu, Nov 7, 2013 at 3:31 PM, Tim Spriggs t...@uahirise.org wrote: Oops, I just realized I did the patch in the wrong direction :) On Thu, Nov 7, 2013 at 3:06 PM, Tim Spriggs t...@uahirise.org wrote: Hi All, I am battling extraneous

Re: [RBD][OpenStack]The way to solve problem when boot VM and root disk size is specified

2013-11-12 Thread Josh Durgin
On 11/11/2013 11:10 PM, Haomai Wang wrote: Hi all, Now OpenStack Nova master branch still exists a bug when you boot a VM which root disk size is specified. The storage backend of Nova also is rbd. For example, you boot a VM and specify 10G as root disk size. But the image is only 1G. Then

Ceph and OpenStack unconference on Friday

2013-11-04 Thread Josh Durgin
We're having an unconference session at the OpenStack design summit on Friday at 2:20pm. If you're interested in Ceph integration with OpenStack, please join us to discuss future development. Josh -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to

Re: feedback on supporting libc++

2013-10-30 Thread Josh Durgin
On 10/29/2013 03:51 PM, Noah Watkins wrote: Out of the box on OSX Mavericks libc++ [1] is being used as opposed to libstdc++. One of the issues is that stuff from tr1 isn't available (e.g. std::tr1::shared_ptr), as they have moved to std in c++11. I'm looking for any feedback on this patch set,

Re: [ceph-users] poor read performance on rbd+LVM, LVM overload

2013-10-20 Thread Josh Durgin
On 10/20/2013 08:18 AM, Ugis wrote: output follows: #pvs -o pe_start /dev/rbd1p1 1st PE 4.00m # cat /sys/block/rbd1/queue/minimum_io_size 4194304 # cat /sys/block/rbd1/queue/optimal_io_size 4194304 Well, the parameters are being set at least. Mike, is it possible that having

Re: Kernel panic putting zpool on iSCSI LUN using bs_rbd

2013-10-14 Thread Josh Durgin
On 10/07/2013 08:49 PM, Eric Eastman wrote: I am not sure if this is a bs_rbd, tgt or zfs issue, but I can reliably crash my Centos 6.4 system running tgt 1.0.40 using a bs_rbd backstore by creating a zpool. Using tgt with a file backed store does not panic the system when creating a zpool.

[PATCH 2/2] rbd: only set disk to read-only once

2013-10-01 Thread Josh Durgin
-off-by: Josh Durgin josh.dur...@inktank.com --- drivers/block/rbd.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index b3b1b57..fc3ebd9 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -490,7 +490,6 @@ static int

[PATCH 0/2] clean up read only mode

2013-10-01 Thread Josh Durgin
branch of ceph-client.git. Josh Durgin (2): rbd: move calls that may sleep out of spin lock range rbd: only set disk to read-only once drivers/block/rbd.c | 38 -- 1 files changed, 24 insertions(+), 14 deletions(-) -- 1.7.2.5 -- To unsubscribe from

[PATCH 1/2] rbd: move calls that may sleep out of spin lock range

2013-10-01 Thread Josh Durgin
get_user() and set_disk_ro() may allocate memory, leading to a potential deadlock if theye are called while a spin lock is held. Move the acquisition and release of rbd_dev-lock from rbd_ioctl() into rbd_ioctl_set_ro(), so it can occur between get_user() and set_disk_ro(). Signed-off-by: Josh

Re: [PATCH] fix librados aio read buffer handling

2013-09-30 Thread Josh Durgin
On 09/30/2013 08:38 AM, Sage Weil wrote: On Mon, 30 Sep 2013, Rutger ter Borg wrote: Dear all, please find attached a patch that enables a user to pass user-owned buffers into librados' aio_read. The patch (against dumpling) removes the buf and pbl data members in AioCompletionImpl. * The

Re: [PATCH v3] rbd: add ioctl for rbd

2013-09-23 Thread Josh Durgin
() is not called with the bdev lock held. With the change to rbd_request_fn() Alex mentioned, could you verify it has no problems with lockdep enabled? If so, it looks good to me. Reviewed-by: Josh Durgin josh.dur...@inktank.com + + switch (cmd) { + case BLKROSET: + ret

Re: Hiding auth key string for the qemu process

2013-09-23 Thread Josh Durgin
On 09/22/2013 12:54 PM, Andrey Korolyov wrote: Hello, Since it was a long time from enabling cephx by default and we may think that everyone using it, is seems worthy to introduce bits of code hiding the key from cmdline. First applicable place for such improvement is most-likely OpenStack envs

Re: [ceph-users] Scaling RBD module

2013-09-19 Thread Josh Durgin
-home on /home type ext4 (rw) Any idea what went wrong here ? Thanks Regards Somnath -Original Message- From: Josh Durgin [mailto:josh.dur...@inktank.com] Sent: Wednesday, September 18, 2013 6:10 PM To: Somnath Roy Cc: Sage Weil; ceph-devel@vger.kernel.org; Anirban Ray; ceph-us

Re: [ceph-users] Scaling RBD module

2013-09-18 Thread Josh Durgin
On 09/17/2013 03:30 PM, Somnath Roy wrote: Hi, I am running Ceph on a 3 node cluster and each of my server node is running 10 OSDs, one for each disk. I have one admin node and all the nodes are connected with 2 X 10G network. One network is for cluster and other one configured as public

Re: [PATCH v2] rbd: add ioctl for rbd

2013-09-17 Thread Josh Durgin
On 09/17/2013 07:42 AM, Alex Elder wrote: On 09/17/2013 02:04 AM, Guangliang Zhao wrote: When running the following commands: [root@ceph0 mnt]# blockdev --setro /dev/rbd2 [root@ceph0 mnt]# blockdev --getro /dev/rbd2 0 I think this is a good change to make, and I think what

Re: [PATCH v2] rbd: add ioctl for rbd

2013-09-17 Thread Josh Durgin
On 09/17/2013 09:11 AM, Alex Elder wrote: On 09/17/2013 10:24 AM, Josh Durgin wrote: block/ioctl.c will already call set_device_ro() for us after this driver-specific handling completes successfully, so we don't need to call it here. Also, it appears the block layer has a bug in that it does

[PATCH v3 2/5] rbd: make rbd_obj_notify_ack() synchronous

2013-09-09 Thread Josh Durgin
. Signed-off-by: Josh Durgin josh.dur...@inktank.com --- drivers/block/rbd.c | 11 ++- 1 files changed, 6 insertions(+), 5 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index bf89e34..9e5f07f 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -2808,7 +2808,7

[PATCH v3 1/5] rbd: complete notifies before cleaning up osd_client and rbd_dev

2013-09-09 Thread Josh Durgin
To ensure rbd_dev is not used after it's released, flush all pending notify callbacks before calling rbd_dev_image_release(). No new notifies can be added to the queue at this point because the watch has already be unregistered with the osd_client. Signed-off-by: Josh Durgin josh.dur

[PATCH v3 0/5] fix shutdown races and snapshot error handling

2013-09-09 Thread Josh Durgin
. It also restricts where it holds the lock to avoid an inversion with bdev-lock. Patch 4 is the same. Patch 5 fixes an inconsistency noticed by Alex in his review of patch 4. Josh Durgin (5): rbd: complete notifies before cleaning up osd_client and rbd_dev rbd: make rbd_obj_notify_ack

[PATCH v3 4/5] rbd: ignore unmapped snapshots that no longer exist

2013-09-09 Thread Josh Durgin
This prevents erroring out while adding a device when a snapshot unrelated to the current mapping is deleted between reading the snapshot context and reading the snapshot names. If the mapped snapshot name is not found an error still occurs as usual. Signed-off-by: Josh Durgin josh.dur

[PATCH v3 5/5] rbd: fix error handling from rbd_snap_name()

2013-09-09 Thread Josh Durgin
an ERR_PTR in rbd_dev_v1_snap_name() so that the specific error can be propagated, and it is consistent with rbd_dev_v2_snap_name(). Handle the ERR_PTR in the only rbd_snap_name() caller. Suggested-by: Alex Elder alex.el...@linaro.org Signed-off-by: Josh Durgin josh.dur...@inktank.com --- drivers

Re: [PATCH v2 1/3] rbd: fix use-after free of rbd_dev-disk

2013-09-09 Thread Josh Durgin
On 09/03/2013 05:41 AM, Alex Elder wrote: On 08/29/2013 07:57 PM, Josh Durgin wrote: Removing a device deallocates the disk, unschedules the watch, and finally cleans up the rbd_dev structure. rbd_dev_refresh(), called from the watch callback, updates the disk size and rbd_dev structure

Re: [PATCH] rbd: ignore unmapped snapshots that no longer exist

2013-09-09 Thread Josh Durgin
On 09/09/2013 12:31 AM, Josh Durgin wrote: On 09/03/2013 06:29 AM, Alex Elder wrote: On 08/29/2013 09:19 PM, Josh Durgin wrote: This prevents erroring out while adding a device when a snapshot unrelated to the current mapping is deleted between reading the snapshot context and reading

Re: [PATCH v3 3/5] rbd: fix use-after free of rbd_dev-disk

2013-09-09 Thread Josh Durgin
On 09/09/2013 06:37 AM, Alex Elder wrote: On 09/09/2013 02:17 AM, Josh Durgin wrote: Removing a device deallocates the disk, unschedules the watch, and finally cleans up the rbd_dev structure. rbd_dev_refresh(), called from the watch callback, updates the disk size and rbd_dev structure

Re: Add Metavariable $imagename

2013-09-05 Thread Josh Durgin
On 09/04/2013 09:22 AM, Sage Weil wrote: Hi Mike, On Wed, 4 Sep 2013, Mike Dawson wrote: Thanks for the feature request Mark! To illustrate where my understanding ends, here is a patch to stub in this functionality: From 137ebc2d5326ca710a5e99e2899fd851b02c10f7 Mon Sep 17 00:00:00 2001

Re: Add Metavariable $imagename

2013-09-05 Thread Josh Durgin
On 09/05/2013 07:59 AM, Sage Weil wrote: On Wed, 4 Sep 2013, Josh Durgin wrote: On 09/04/2013 09:22 AM, Sage Weil wrote: Hi Mike, On Wed, 4 Sep 2013, Mike Dawson wrote: Thanks for the feature request Mark! To illustrate where my understanding ends, here is a patch to stub

Re: libvirt: Using rbd_create3 to create format 2 images

2013-09-03 Thread Josh Durgin
On 09/02/2013 07:58 AM, Wido den Hollander wrote: On 08/30/2013 08:38 PM, Wido den Hollander wrote: On 08/30/2013 05:26 PM, Josh Durgin wrote: On 08/30/2013 02:42 AM, Wido den Hollander wrote: Hi, I created the attached patch to have libvirt create images with format 2 by default, this would

Re: libvirt: Using rbd_create3 to create format 2 images

2013-08-30 Thread Josh Durgin
On 08/30/2013 02:42 AM, Wido den Hollander wrote: Hi, I created the attached patch to have libvirt create images with format 2 by default, this would simplify the CloudStack code and could also help other projects. The problem with libvirt is that there is no mechanism to supply information

[PATCH 1/3] rbd: fix null dereference in dout

2013-08-29 Thread Josh Durgin
The order parameter is sometimes NULL in _rbd_dev_v2_snap_size(), but the dout() always derefences it. Move this to another dout() protected by a check that order is non-NULL. Signed-off-by: Josh Durgin josh.dur...@inktank.com --- drivers/block/rbd.c |8 +--- 1 files changed, 5

[PATCH 2/3] libceph: add function to ensure notifies are complete

2013-08-29 Thread Josh Durgin
for the event. If the queue is flushed after the event is unregistered, the caller can be sure no more watch callbacks will occur for the canceled watch. Signed-off-by: Josh Durgin josh.dur...@inktank.com --- include/linux/ceph/osd_client.h |2 ++ net/ceph/osd_client.c | 11

[PATCH 0/3] shutdown race and debug fix

2013-08-29 Thread Josh Durgin
These patches fix a race in rbd shutdown and a null deref when debugging is enabled. They're also available in the wip-rbd-bugs branch of ceph-client.git. Josh Durgin (3): rbd: fix null dereference in dout libceph: add function to ensure notifies are complete rbd: close remove vs. notify

[PATCH 3/3] rbd: close remove vs. notify race leading to use-after-free

2013-08-29 Thread Josh Durgin
the queue at this point since the watch has been canceled. Fixes: http://tracker.ceph.com/issues/5636 Signed-off-by: Josh Durgin josh.dur...@inktank.com --- drivers/block/rbd.c | 25 ++--- 1 files changed, 22 insertions(+), 3 deletions(-) diff --git a/drivers/block/rbd.c b

Re: [PATCH 3/3] rbd: close remove vs. notify race leading to use-after-free

2013-08-29 Thread Josh Durgin
On 08/29/2013 08:21 AM, Alex Elder wrote: On 08/29/2013 01:24 AM, Josh Durgin wrote: Removing a device deallocates the disk, unschedules the watch, and finally cleans up the rbd_dev structure. rbd_dev_refresh(), called from the watch callback, updates the disk size and rbd_dev structure

Re: [PATCH 3/3] rbd: close remove vs. notify race leading to use-after-free

2013-08-29 Thread Josh Durgin
On 08/29/2013 07:46 AM, Sage Weil wrote: On Wed, 28 Aug 2013, Josh Durgin wrote: Removing a device deallocates the disk, unschedules the watch, and finally cleans up the rbd_dev structure. rbd_dev_refresh(), called from the watch callback, updates the disk size and rbd_dev structure

[PATCH v2 2/3] rbd: complete notifies before cleaning up osd_client and rbd_dev

2013-08-29 Thread Josh Durgin
To ensure rbd_dev is not used after it's released, flush all pending notify callbacks before calling rbd_dev_image_release(). No new notifies can be added to the queue at this point because the watch has already be unregistered with the osd_client. Signed-off-by: Josh Durgin josh.dur

[PATCH v2 1/3] rbd: fix use-after free of rbd_dev-disk

2013-08-29 Thread Josh Durgin
been freed. To fix this, add a shutting_down flag and a mutex protecting it to rbd_dev. Take the mutex and check whether the flag is set before using rbd_dev-disk. Move this disk-updating to a separate function as well. Fixes: http://tracker.ceph.com/issues/5636 Signed-off-by: Josh Durgin josh.dur

[PATCH v2 3/3] rbd: make rbd_obj_notify_ack() synchronous

2013-08-29 Thread Josh Durgin
, waiting for the notify ack to complete before returning from the watch callback ensures there are no notify acks in flight during shutdown. Signed-off-by: Josh Durgin josh.dur...@inktank.com --- drivers/block/rbd.c |4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/drivers

[PATCH] rbd: ignore unmapped snapshots that no longer exist

2013-08-29 Thread Josh Durgin
This prevents erroring out while adding a device when a snapshot unrelated to the current mapping is deleted between reading the snapshot context and reading the snapshot names. If the mapped snapshot name is not found an error still occurs as usual. Signed-off-by: Josh Durgin josh.dur

Re: [PATCH] rbd: fix I/O error propagation for reads

2013-08-27 Thread Josh Durgin
. If you look at the admin socket's dump_historic_ops do you see higher op durations around the dips in performance? What about any correlation with underlying disk stats from iostat -x? Thanks, Mike Dawson On 8/26/2013 9:34 PM, Josh Durgin wrote: When a request returns an error, the driver needs

[PATCH] rbd: fix buffer size for writes to images with snapshots

2013-08-27 Thread Josh Durgin
: libceph: BUG_ON(p msg-front.iov_base + msg-front.iov_len); Fix this by associating the obj_request with its img_request immediately after it's created, before the osd request is created. Fixes: http://tracker.ceph.com/issues/5760 Suggested-by: Alex Elder alex.el...@linaro.org Signed-off-by: Josh

[PATCH] rbd: fix I/O error propagation for reads

2013-08-26 Thread Josh Durgin
incomplete. Fixes: http://tracker.ceph.com/issues/5647 Signed-off-by: Josh Durgin josh.dur...@inktank.com --- drivers/block/rbd.c | 14 +++--- 1 files changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 0d669ae..f8fd7d3 100644

Re: libvirt: Removing RBD volumes with snapshots, auto purge or not?

2013-08-20 Thread Josh Durgin
On 08/20/2013 08:36 AM, Wido den Hollander wrote: Hi, The current [0] libvirt storage pool code simply calls rbd_remove without anything else. As far as I know rbd_remove will fail if the image still has snapshots, you have to remove those snapshots first before you can remove the image. The

Re: Show outdated diffs on github

2013-08-16 Thread Josh Durgin
On 08/15/2013 11:45 PM, Loic Dachary wrote: Hi Josh, From https://github.com/ceph/ceph/pull/504 I see that your comments persisted despite a rebase and show as jdurgin discussed an outdated diff 8 hours ago I would like to do the same but when I comment inline, it shows differently and

Re: teuthology and code coverage

2013-08-14 Thread Josh Durgin
On 08/13/2013 10:01 AM, Loic Dachary wrote: Hi, When running teuthology from a laptop with the configuration below and ./virtualenv/bin/teuthology --archive=/tmp/teuthology try.yaml it then fails on ./virtualenv/bin/teuthology-coverage -v --html-output /tmp/html -o /tmp/lcov

Re: Correct usage of rbd_aio_release

2013-08-12 Thread Josh Durgin
On 08/12/2013 06:36 AM, Sylvain Munaut wrote: Hi, When should / can rbd_aio_release be called exactly ? It should be called whenever you're certain you won't be using the rbd_completion_t anymore (and an rbd_completion_t should not be reused after rbd_aio_{read,write,discard,flush} is called

Re: Internal Qemu snapshots with RBD and libvirt

2013-07-19 Thread Josh Durgin
On 07/18/2013 08:21 AM, Wido den Hollander wrote: Hi, I'm working on the RBD integration for CloudStack 4.2 and now I got to the point snapshotting. The problem is that CloudStack uses libvirt for snapshotting Instances, but Qemu/libvirt also tries to store the memory contents of the domain to

Re: Internal Qemu snapshots with RBD and libvirt

2013-07-19 Thread Josh Durgin
: On Fri, 19 Jul 2013, Josh Durgin wrote: On 07/18/2013 08:21 AM, Wido den Hollander wrote: Hi, I'm working on the RBD integration for CloudStack 4.2 and now I got to the point snapshotting. The problem is that CloudStack uses libvirt for snapshotting Instances, but Qemu/libvirt also tries to store

Re: [PATCH] Python Ioctx.get_xattrs does not take xattr_name

2013-07-10 Thread Josh Durgin
On 07/09/2013 08:15 AM, Johannes Erdfelt wrote: Fixes bug 5528 Since it returns all xattrs, it does not take an xattr_name. Also, add unit tests for the Object xattrs methods. Signed-off-by: Johannes Erdfelt johan...@erdfelt.com Applied to next and master, thanks! Josh -- To unsubscribe

Re: Modifying RBD image changes it's snapshot

2013-06-26 Thread Josh Durgin
On 06/26/2013 05:40 AM, Karol Jurak wrote: Hi, I'm using ceph 0.56.6 and kernel 3.9.7 and it looks like modifying RBD image also changes it's snapshot. I can reproduce this as follows: # create and map an image rbd create --size 128 test-1 rbd map test-1 # write some data to the image dd

[PATCH] rbd: send snapshot context with writes

2013-06-26 Thread Josh Durgin
, before the osd request formatting calls. This resolves: http://tracker.ceph.com/issues/5465 Reported-by: Karol Jurak karol.ju...@gmail.com Signed-off-by: Josh Durgin josh.dur...@inktank.com --- drivers/block/rbd.c |4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diff --git a/drivers

Re: Modifying RBD image changes it's snapshot

2013-06-26 Thread Josh Durgin
On 06/26/2013 11:15 AM, Josh Durgin wrote: On 06/26/2013 05:40 AM, Karol Jurak wrote: Hi, I'm using ceph 0.56.6 and kernel 3.9.7 and it looks like modifying RBD image also changes it's snapshot. I can reproduce this as follows: # create and map an image rbd create --size 128 test-1 rbd map

[PATCH 2/2] rbd: use the correct length for format 2 object names

2013-06-12 Thread Josh Durgin
an image written by one client look empty to the other client. Reported-by: Chris Dunlop ch...@onthe.net.au Signed-off-by: Josh Durgin josh.dur...@inktank.com --- drivers/block/rbd.c |6 +- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block

[PATCH 1/2] rbd: fetch object order before using it

2013-06-12 Thread Josh Durgin
size and object order before rbd_dev_v2_header_onetime() so it is set before use. Signed-off-by: Josh Durgin josh.dur...@inktank.com --- drivers/block/rbd.c |8 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index cecf5c6

Re: krbd + format=2 ?

2013-06-12 Thread Josh Durgin
On 06/11/2013 09:59 PM, Chris Dunlop wrote: On Sat, Jun 08, 2013 at 12:48:52PM +1000, Chris Dunlop wrote: On Fri, Jun 07, 2013 at 11:54:20AM -0500, Alex Elder wrote: On 06/03/2013 04:24 AM, Chris Dunlop wrote: I pulled the for-linus branch (@ 3abef3b) on top of 3.10.0-rc4, and it's letting me

Re: [PATCH 5/5] rbd: take a little credit

2013-06-07 Thread Josh Durgin
Reviewed-by: Josh Durgin josh.dur...@inktank.com Alex Elder el...@inktank.com wrote: Add a name to the list of authors. Signed-off-by: Alex Elder el...@inktank.com --- drivers/block/rbd.c |1 + 1 file changed, 1 insertion(+) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index

Re: [PATCH 2/5] rbd: protect against concurrent unmaps

2013-06-07 Thread Josh Durgin
Reviewed-by: Josh Durgin josh.dur...@inktank.com Alex Elder el...@inktank.com wrote: Make sure two concurrent unmap operations on the same rbd device won't collide, by only proceeding with the removal and cleanup of a device if is not already underway. Signed-off-by: Alex Elder el

Re: [PATCH 4/5] rbd: use rwsem to protect header updates

2013-06-07 Thread Josh Durgin
() so that reading the snapshot context via rbd_dev_snap_index() is protected. Looks good otherwise. Reviewed-by: Josh Durgin josh.dur...@inktank.com /* If it's a mapped snapshot, validate its EXISTS flag */ rbd_exists_validate(rbd_dev); - mutex_unlock(ctl_mutex

Re: flatten rbd export / export-diff ?

2013-06-04 Thread Josh Durgin
On 06/04/2013 11:04 AM, Stefan Priebe wrote: Am 04.06.2013 17:23, schrieb Sage Weil: On Tue, 4 Jun 2013, Stefan Priebe - Profihost AG wrote: Hi, is there a way to flatten the rbd export-diff to a new image FILE. Or do i always have to: rbd import OLD BASE IMAGE rbd import-diff diff1 rbd

Re: [PATCH 2/2] Add RADOS API lock tests

2013-05-31 Thread Josh Durgin
On 05/30/2013 06:02 AM, Filippos Giannakos wrote: Add tests for the advisory locking API calls. Signed-off-by: Filippos Giannakos philipg...@grnet.gr Reviewed-by: Josh Durgin josh.dur...@inktank.com --- src/Makefile.am |6 ++ src/test/librados/lock.cc | 236

Re: Speed up 'rbd rm'

2013-05-31 Thread Josh Durgin
On 05/30/2013 11:40 PM, Stefan Priebe - Profihost AG wrote: Am 31.05.2013 03:40, schrieb Chris Dunlop: On Thu, May 30, 2013 at 01:50:14PM -0700, Josh Durgin wrote: On 05/29/2013 07:23 PM, Chris Dunlop wrote: On Wed, May 29, 2013 at 12:21:07PM -0700, Josh Durgin wrote: On 05/28/2013 10:59 PM

Re: [PATCH] rbd: clean up a few things in the refresh path

2013-05-30 Thread Josh Durgin
there was + * no parent image. + */ + rbd_warn(rbd_dev, ignoring parent of + clone with overlap 0\n); + } } out: ret = 0; Reviewed-by: Josh Durgin josh.dur

Re: Speed up 'rbd rm'

2013-05-29 Thread Josh Durgin
On 05/28/2013 10:59 PM, Chris Dunlop wrote: I see there's a new commit to speed up an 'rbd rm': http://tracker.ceph.com/projects/ceph/repository/revisions/40956410169709c32a282d9b872cb5f618a48926 Is it safe to cherry-pick this commit on top of 0.56.6 (or, if not, v0.61.2) to speed up the

Re: [PATCH] rbd: wait for safe callback for write requests

2013-05-23 Thread Josh Durgin
Reviewed-by: Josh Durgin josh.dur...@inktank.com On 05/23/2013 05:11 AM, Alex Elder wrote: A write request sent to the osd will get either one or two responses. One is simply an acknowledgement of receipt, and another is an indication that the state change described by the request (typically

Re: [PATCH] libceph: add lingering request reference when registered

2013-05-23 Thread Josh Durgin
Reviewed-by: Josh Durgin josh.dur...@inktank.com On 05/23/2013 05:11 AM, Alex Elder wrote: When an osd request is set to linger, the osd client holds onto the request so it can be re-submitted following certain osd map changes. The osd client holds a reference to the request until

Re: [PATCH] rbd: flush dcache after zeroing page data

2013-05-23 Thread Josh Durgin
Reviewed-by: Josh Durgin josh.dur...@inktank.com On 05/23/2013 05:11 AM, Alex Elder wrote: Neither zero_bio_chain() nor zero_pages() contains a call to flush caches after zeroing a portion of a page. This can cause problems on architectures that have caches that allow virtual address aliasing

Re: [PATCH] rbd: don't destroy ceph_opts in rbd_add()

2013-05-16 Thread Josh Durgin
Reviewed-by: Josh Durgin josh.dur...@inktank.com On 05/16/2013 02:29 PM, Alex Elder wrote: (Note: this patch and the two I'm about to re-post are available in the review/wip-4559 branch of the ceph-client git repository.) Whether rbd_client_create() successfully creates a new client

Re: [PATCH, v2] rbd: fix cleanup in rbd_add()

2013-05-16 Thread Josh Durgin
the trick I think, since rbd_dev_image_release() is also used for parent images that don't call module_get() and module_put(). With that change: Reviewed-by: Josh Durgin josh.dur...@inktank.com + + return rc; + err_out_rbd_dev: rbd_dev_destroy(rbd_dev); err_out_client

Re: [PATCH, v2] rbd: drop original request earlier for existence check

2013-05-16 Thread Josh Durgin
Reviewed-by: Josh Durgin josh.dur...@inktank.com On 05/16/2013 02:29 PM, Alex Elder wrote: The reference to the original request dropped at the end of rbd_img_obj_exists_callback() corresponds to the reference taken in rbd_img_obj_exists_submit() to account for the stat request referring

Re: QCOW2 to RBD format 2 in one step

2013-05-14 Thread Josh Durgin
On 05/14/2013 03:16 AM, Wido den Hollander wrote: Hi, While working on the CloudStack code for 4.2 I'm running into some problems with QCOW2 and RBD format 2. A quick cap of the problem: CloudStack stores all the templates on Secondary Storage as QCOW2 files. When a VM is deployed, this

Re: QCOW2 to RBD format 2 in one step

2013-05-14 Thread Josh Durgin
On 05/14/2013 07:36 AM, Alexandre DERUMIER wrote: Hi, we do it with proxmox, we create the target rbd volume at format v2 with rbd .. commands, then, we use qemu-img convert to copy from qcow2 to rbd. We have patched qemu-img to be able to write to an existing volume. (By default qemu-img

Re: [PATCH] rbd: fix parent request size assumption

2013-05-13 Thread Josh Durgin
Reviewed-by: Josh Durgin josh.dur...@inktank.com On 05/13/2013 10:58 AM, Alex Elder wrote: I have inserted this patch into a series I posted over the weekend to address an issue that Josh's comment raised in the review for: [PATCH] rbd: support reading parent page data for writes

Re: [PATCH 1/5, v2] rbd: get parent info on refresh

2013-05-13 Thread Josh Durgin
Reviewed-by: Josh Durgin josh.dur...@inktank.com On 05/13/2013 10:58 AM, Alex Elder wrote: Get parent info for format 2 images on every refresh (rather than just during the initial probe). This will be needed to detect the disappearance of the parent image in the event a mapped image becomes

Re: [PATCH] rbd: support reading parent page data for writes

2013-05-11 Thread Josh Durgin
cross the overlap point will be shortened to less than an object size. The rest looks good. Reviewed-by: Josh Durgin josh.dur...@inktank.com - xferred = img_request-xferred; + rbd_assert(obj_size == img_request-xferred); rbd_img_request_put(img_request); rbd_assert

Re: [PATCH] rbd: ignore zero-overlap parent

2013-05-11 Thread Josh Durgin
Reviewed-by: Josh Durgin josh.dur...@inktank.com On 05/11/2013 10:38 AM, Alex Elder wrote: An rbd clone image that has an overlap with its parent of 0 is effectively not a layered image at all. Detect this case and treat such an image as non-layered. Issue a warning to be sure the user knows

Re: [PATCH 1/5] rbd: get parent info on refresh

2013-05-11 Thread Josh Durgin
On 05/11/2013 10:43 AM, Alex Elder wrote: Get parent info for format 2 images on every refresh (rather than just during the initial probe). This will be needed to detect the disappearance of the parent image in the event a mapped image becomes unlayered (i.e., flattened). Switch to using a

Re: [PATCH 0/5] rbd: prep work for flattening images

2013-05-11 Thread Josh Durgin
release write request until necessary [PATCH 3/5] rbd: define rbd_dev_unparent() [PATCH 4/5] rbd: define parent image request routines [PATCH 5/5] rbd: reference count parent requests 2-5 look good. Reviewed-by: Josh Durgin josh.dur...@inktank.com -- To unsubscribe from this list: send the line

Re: [PATCH 0/4] rbd: handle images flattened while mapped

2013-05-11 Thread Josh Durgin
[PATCH 2/4] rbd: re-submit read request for flattened clone [PATCH 3/4] rbd: re-submit write request for flattened clone [PATCH 4/4] rbd: re-submit flattened write request (part 2) These all look good. Reviewed-by: Josh Durgin josh.dur...@inktank.com -- To unsubscribe from this list: send the line

Re: [PATCH] libceph: init sent and completed when starting

2013-05-11 Thread Josh Durgin
Reviewed-by: Josh Durgin josh.dur...@inktank.com On 05/11/2013 10:37 AM, Alex Elder wrote: The rbd code has a need to be able to restart an osd request that has already been started and completed once before. This currently wouldn't work right because the osd client code assumes an osd request

Re: [PATCH 0/7] rbd: use common code for probe and refresh

2013-05-08 Thread Josh Durgin
already moved it in your upcoming flatten-handling patches. Reviewed-by: Josh Durgin josh.dur...@inktank.com -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo

Re: [PATCH 0/5] rbd: fix some bugs

2013-05-08 Thread Josh Durgin
[PATCH 1/5] rbd: fix an incorrect assertion condition [PATCH 2/5] rbd: support reading parent page data [PATCH 3/5] rbd: set mapping read-only flag in rbd_add() [PATCH 4/5] rbd: only set up watch for mapped images [PATCH 5/5] rbd: kill rbd_img_request_get() These all look good. Reviewed-by: Josh

Re: [PATCH 3/6] rbd: don't print warning if not mapping a parent

2013-05-07 Thread Josh Durgin
) rbd_warn(rbd_dev, WARNING: kernel layering is EXPERIMENTAL!); } Reviewed-by: Josh Durgin josh.dur...@inktank.com -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org

Re: [PATCH 5/6] rbd: kill rbd_dev_clear_mapping()

2013-05-07 Thread Josh Durgin
) rbd_free_disk(rbd_dev); clear_bit(RBD_DEV_FLAG_EXISTS, rbd_dev-flags); - rbd_dev_clear_mapping(rbd_dev); + rbd_dev_mapping_clear(rbd_dev); unregister_blkdev(rbd_dev-major, rbd_dev-name); rbd_dev-major = 0; rbd_dev_id_put(rbd_dev); Reviewed-by: Josh Durgin josh.dur

Re: [PATCH] rbd: revalidate only for mapping size changes

2013-05-07 Thread Josh Durgin
rbd_image_refresh(struct device *dev, int ret; ret = rbd_dev_refresh(rbd_dev); + if (ret) + rbd_warn(rbd_dev, : manual header refresh error (%d)\n, ret); return ret 0 ? ret : size; } Reviewed-by: Josh Durgin josh.dur...@inktank.com -- To unsubscribe from

Re: [PATCH 4/6] rbd: don't look up snapshot id in rbd_dev_mapping_set()

2013-05-07 Thread Josh Durgin
(rbd_dev, snap_name); - if (snap_id == CEPH_NOSNAP) - return -ENOENT; - } else { - snap_id = CEPH_NOSNAP; - } - ret = rbd_snap_size(rbd_dev, snap_id, size); if (ret) return ret; Reviewed-by: Josh Durgin josh.dur

Re: [PATCH 2/6] rbd: kill rbd_update_mapping_size()

2013-05-07 Thread Josh Durgin
-mapping.size != rbd_dev-header.image_size) + rbd_dev-mapping.size = rbd_dev-header.image_size; Using instead of two conditions would be a bit cleaner. Maybe you have later patches that depend on this though. Open coding looks fine to me in any case. Reviewed-by: Josh Durgin

Re: [PATCH 1/6] rbd: update capacity in rbd_dev_refresh()

2013-05-07 Thread Josh Durgin
a read anyway, so I think -EIO would be better. With that change: Reviewed-by: Josh Durgin josh.dur...@inktank.com + if (offset + length rbd_dev-mapping.size) { + rbd_warn(rbd_dev, beyond EOD (%llu~%llu %llu)\n, + offset, length

Re: [PATCH 6/6] rbd: always set read-only flag in rbd_add()

2013-05-07 Thread Josh Durgin
this was moved accidentally? Maybe needed by a later patch? Reviewed-by: Josh Durgin josh.dur...@inktank.com - - rc = rbd_dev_image_probe(rbd_dev); + rc = rbd_dev_image_probe(rbd_dev, read_only); if (rc 0) goto err_out_rbd_dev; -- To unsubscribe from

Re: [PATCH 1/4] rbd: allocate image requests with a slab allocator

2013-05-02 Thread Josh Durgin
It looks like the GFP flags used throughout request creation could be cleanup up (there's a mix of ATOMIC, KERNEL, and NOIO). I just noticed since this changes a GFP_ATOMIC to GFP_KERNEL. Making them consistent can be a later patch though. Reviewed-by: Josh Durgin josh.dur...@inktank.com On 05

Re: [PATCH 2/4] rbd: allocate name separate from obj_request

2013-05-02 Thread Josh Durgin
Reviewed-by: Josh Durgin josh.dur...@inktank.com On 05/01/2013 02:35 PM, Alex Elder wrote: The next patch will define a slab allocator for a object requests. To use that we'll need to allocate the name of an object separate from the request structure itself. Signed-off-by: Alex Elder el

Re: [PATCH 3/4] rbd: allocate object requests with a slab allocator

2013-05-02 Thread Josh Durgin
Reviewed-by: Josh Durgin josh.dur...@inktank.com On 05/01/2013 02:36 PM, Alex Elder wrote: Create a slab cache to manage rbd_obj_request allocation. We aren't using a constructor, and we'll zero-fill object request structures when they're allocated. This is part of: http

Re: [PATCH 4/4] rbd: allocate image object names with a slab allocator

2013-05-02 Thread Josh Durgin
; + if (rbd_img_request_cache) { + kmem_cache_destroy(rbd_obj_request_cache); + rbd_img_request_cache = NULL; Should be rbd_obj_request_cache here and in the condition two lines up. With that fixed: Reviewed-by: Josh Durgin josh.dur...@inktank.com + } + return -ENOMEM

Re: [PATCH 1/2] rbd: clear EXISTS flag if mapped snapshot disappears

2013-05-02 Thread Josh Durgin
Reviewed-by: Josh Durgin josh.dur...@inktank.com On 05/01/2013 02:32 PM, Alex Elder wrote: This functionality inadvertently disappeared in the last patch. Image snapshots can get removed at just about any time. In particular it can disappear even if it is in use by an rbd client as a mapped

<    1   2   3   4   5   6   7   8   >