Re: less cores more iops / speed

2012-11-09 Thread Stefan Priebe - Profihost AG
Am 08.11.2012 16:53, schrieb Alexandre DERUMIER: So it is a problem of KVM which let's the processes jump between cores a lot. maybe numad from redhat can help ? http://fedoraproject.org/wiki/Features/numad It's try to keep process on same numa node and I think it's also doing some dynamic

Re: extreme ceph-osd cpu load for rand. 4k write

2012-11-09 Thread Stefan Priebe - Profihost AG
Disabling the logging with: debug lockdep = 0/0 debug context = 0/0 debug crush = 0/0 debug buffer = 0/0 debug timer = 0/0 debug journaler = 0/0 debug osd = 0/0 debug optracker = 0/0 debug objclass = 0/0 debug filestore = 0/0 debug journal = 0/0 debug ms = 0/0 debug monc = 0/0

Re: extreme ceph-osd cpu load for rand. 4k write

2012-11-09 Thread Stefan Priebe - Profihost AG
New graph from today. fsetxattr seems to take a lot of CPU too. Am 09.11.2012 11:09, schrieb Stefan Priebe - Profihost AG: Disabling the logging with: debug lockdep = 0/0 debug context = 0/0 debug crush = 0/0 debug buffer = 0/0 debug timer = 0/0 debug journaler = 0/0 debug osd =

can't build the last master git (`configure.ac' or `configure.in' is required)

2012-11-09 Thread Alexandre DERUMIER
Hi,I need help to build last master git, I was building fine before, #git clone git://ceph.newdream.net/git/ceph.git #git submodule init #git module update #dpkg-buildpackage ... autoreconf: running: /usr/bin/autoconf --force autoreconf: running: /usr/bin/autoheader --force autoreconf: running:

Re: can't build the last master git (`configure.ac' or `configure.in' is required)

2012-11-09 Thread Joao Eduardo Luis
Hi Alexandre, Forgot to CC the list on the previous email. On 11/09/2012 01:05 PM, Alexandre DERUMIER wrote: # ./autogen.sh libtoolize: putting auxiliary files in `.'. libtoolize: copying file `./ltmain.sh' libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'. libtoolize: copying file

Re: can't build the last master git (`configure.ac' or `configure.in' is required)

2012-11-09 Thread Alexandre DERUMIER
Thanks, It's works ! I check my history, seem that git submodule init didn't retrieve correctly git://github.com/ceph/leveldb.git. (maybe a temp network problem). Thanks again, Regards, Alexandre - Mail original - De: Joao Eduardo Luis joao.l...@inktank.com À: Alexandre DERUMIER

Re: SSD journal suggestion / rsockets

2012-11-09 Thread Atchley, Scott
On Nov 8, 2012, at 5:00 PM, Joseph Glanville joseph.glanvi...@orionvm.com.au wrote: On 9 November 2012 08:21, Dieter Kasper d.kas...@kabelmail.de wrote: Joseph, I've downloaded and read the presentation from 'Sean Hefty / Intel Corporation' about rsockets, which sounds very promising to

``rbd mv'' crash when no destination issued

2012-11-09 Thread Andrey Korolyov
Hi, Please take a look, seems harmless: $ rbd mv vm0 terminate called after throwing an instance of 'std::logic_error' what(): basic_string::_S_construct null not valid *** Caught signal (Aborted) ** in thread 7f85f5981780 ceph version 0.53 (commit:2528b5ee105b16352c91af064af5c0b5a7d45d7c)

Re: cephx auth issue

2012-11-09 Thread Ben Poliakoff
* Travis Rhoden trho...@gmail.com [20121109 09:55]: I'm not sure what I've done wrong here: Things are okay as client.admin: # rbd -p images --id admin ls test But not as client.images: # rbd -p images --id images ls error: (1) Operation not permitted The privs/caps seem okay

Re: can't build the last master git (`configure.ac' or `configure.in' is required)

2012-11-09 Thread Gary Lowell
Just tried a fresh checkout and build and didn't see the issue. There may have been commits between your issue and my test this morning. On possible problem might be that your third command reads module instead of submodule. I assume that's a typo, but if not that would explain

Re: cephx auth issue

2012-11-09 Thread Travis Rhoden
Ben, That did it! Thank you so much, I owe you one. - Travis On Fri, Nov 9, 2012 at 1:08 PM, Ben Poliakoff b...@reed.edu wrote: * Travis Rhoden trho...@gmail.com [20121109 09:55]: I'm not sure what I've done wrong here: Things are okay as client.admin: # rbd -p images --id admin ls

rbd volume upgrades

2012-11-09 Thread Gregory Farnum
I was asked today if there's a way to upgrade RBD volumes from v1 to v2. I didn't think so, but wanted 1) to make sure I'm right, 2) to ask how hard it would be, 3) to ask if we haven't done it because it didn't occur to us or because it's too hard. -Greg -- To unsubscribe from this list: send the

Re: rbd volume upgrades

2012-11-09 Thread Josh Durgin
On 11/09/2012 11:01 AM, Gregory Farnum wrote: I was asked today if there's a way to upgrade RBD volumes from v1 to v2. I didn't think so, but wanted 1) to make sure I'm right, 2) to ask how hard it would be, 3) to ask if we haven't done it because it didn't occur to us or because it's too hard.

Re: rbd volume upgrades

2012-11-09 Thread Gregory Farnum
On Fri, Nov 9, 2012 at 8:04 PM, Josh Durgin josh.dur...@inktank.com wrote: On 11/09/2012 11:01 AM, Gregory Farnum wrote: I was asked today if there's a way to upgrade RBD volumes from v1 to v2. I didn't think so, but wanted 1) to make sure I'm right, 2) to ask how hard it would be, 3) to

Re: rbd volume upgrades

2012-11-09 Thread Yehuda Sadeh
On Fri, Nov 9, 2012 at 11:04 AM, Josh Durgin josh.dur...@inktank.com wrote: On 11/09/2012 11:01 AM, Gregory Farnum wrote: I was asked today if there's a way to upgrade RBD volumes from v1 to v2. I didn't think so, but wanted 1) to make sure I'm right, 2) to ask how hard it would be, 3) to

Re: rbd volume upgrades

2012-11-09 Thread Alex Elder
On 11/09/2012 01:08 PM, Yehuda Sadeh wrote: On Fri, Nov 9, 2012 at 11:04 AM, Josh Durgin josh.dur...@inktank.com wrote: On 11/09/2012 11:01 AM, Gregory Farnum wrote: I was asked today if there's a way to upgrade RBD volumes from v1 to v2. I didn't think so, but wanted 1) to make sure I'm

Re: rbd volume upgrades

2012-11-09 Thread Josh Durgin
On 11/09/2012 11:08 AM, Yehuda Sadeh wrote: On Fri, Nov 9, 2012 at 11:04 AM, Josh Durgin josh.dur...@inktank.com wrote: On 11/09/2012 11:01 AM, Gregory Farnum wrote: I was asked today if there's a way to upgrade RBD volumes from v1 to v2. I didn't think so, but wanted 1) to make sure I'm

Re: rbd volume upgrades

2012-11-09 Thread Yehuda Sadeh
On Fri, Nov 9, 2012 at 11:30 AM, Josh Durgin josh.dur...@inktank.com wrote: On 11/09/2012 11:08 AM, Yehuda Sadeh wrote: On Fri, Nov 9, 2012 at 11:04 AM, Josh Durgin josh.dur...@inktank.com wrote: On 11/09/2012 11:01 AM, Gregory Farnum wrote: I was asked today if there's a way to upgrade

Re: rbd volume upgrades

2012-11-09 Thread Alex Elder
On 11/09/2012 01:44 PM, Yehuda Sadeh wrote: On Fri, Nov 9, 2012 at 11:30 AM, Josh Durgin josh.dur...@inktank.com wrote: On 11/09/2012 11:08 AM, Yehuda Sadeh wrote: . . . You need to export and then import the volume as format 2. Format 2 uses different names for objects, so providing an

Re: rbd volume upgrades

2012-11-09 Thread Josh Durgin
On 11/09/2012 11:44 AM, Yehuda Sadeh wrote: On Fri, Nov 9, 2012 at 11:30 AM, Josh Durgin josh.dur...@inktank.com wrote: On 11/09/2012 11:08 AM, Yehuda Sadeh wrote: On Fri, Nov 9, 2012 at 11:04 AM, Josh Durgin josh.dur...@inktank.com wrote: On 11/09/2012 11:01 AM, Gregory Farnum wrote: I

Re: rbd volume upgrades

2012-11-09 Thread Alex Elder
On 11/09/2012 02:03 PM, Josh Durgin wrote: On 11/09/2012 11:44 AM, Yehuda Sadeh wrote: On Fri, Nov 9, 2012 at 11:30 AM, Josh Durgin josh.dur...@inktank.com wrote: On 11/09/2012 11:08 AM, Yehuda Sadeh wrote: On Fri, Nov 9, 2012 at 11:04 AM, Josh Durgin josh.dur...@inktank.com wrote: On

Re: rbd volume upgrades

2012-11-09 Thread Josh Durgin
On 11/09/2012 12:09 PM, Alex Elder wrote: On 11/09/2012 02:03 PM, Josh Durgin wrote: On 11/09/2012 11:44 AM, Yehuda Sadeh wrote: On Fri, Nov 9, 2012 at 11:30 AM, Josh Durgin josh.dur...@inktank.com wrote: On 11/09/2012 11:08 AM, Yehuda Sadeh wrote: On Fri, Nov 9, 2012 at 11:04 AM, Josh

Re: rbd volume upgrades

2012-11-09 Thread Yehuda Sadeh
On Fri, Nov 9, 2012 at 12:26 PM, Josh Durgin josh.dur...@inktank.com wrote: On 11/09/2012 12:09 PM, Alex Elder wrote: On 11/09/2012 02:03 PM, Josh Durgin wrote: On 11/09/2012 11:44 AM, Yehuda Sadeh wrote: On Fri, Nov 9, 2012 at 11:30 AM, Josh Durgin josh.dur...@inktank.com wrote: On

Re: rbd volume upgrades

2012-11-09 Thread Yehuda Sadeh
On Fri, Nov 9, 2012 at 12:30 PM, Yehuda Sadeh yeh...@inktank.com wrote: On Fri, Nov 9, 2012 at 12:26 PM, Josh Durgin josh.dur...@inktank.com wrote: On 11/09/2012 12:09 PM, Alex Elder wrote: On 11/09/2012 02:03 PM, Josh Durgin wrote: On 11/09/2012 11:44 AM, Yehuda Sadeh wrote: On Fri, Nov

Re: rbd volume upgrades

2012-11-09 Thread Josh Durgin
On 11/09/2012 12:31 PM, Yehuda Sadeh wrote: On Fri, Nov 9, 2012 at 12:30 PM, Yehuda Sadeh yeh...@inktank.com wrote: On Fri, Nov 9, 2012 at 12:26 PM, Josh Durgin josh.dur...@inktank.com wrote: On 11/09/2012 12:09 PM, Alex Elder wrote: On 11/09/2012 02:03 PM, Josh Durgin wrote: On 11/09/2012

Re: ``rbd mv'' crash when no destination issued

2012-11-09 Thread Josh Durgin
On 11/09/2012 08:57 AM, Andrey Korolyov wrote: Hi, Please take a look, seems harmless: $ rbd mv vm0 terminate called after throwing an instance of 'std::logic_error' Thanks for the report, it's fixed in the next and master branches, and is indeed harmless. Josh -- To unsubscribe from this

Re: extreme ceph-osd cpu load for rand. 4k write

2012-11-09 Thread Samuel Just
Can you describe the osd and client set up (number of nodes, number of osds per node, journal disks, replication level, and osd disks)? Looks like a lot of time is spent looking up objects in the filestore (lfn_open, etc). -Sam On Fri, Nov 9, 2012 at 2:21 AM, Stefan Priebe - Profihost AG

Re: some snapshot problems

2012-11-09 Thread Cláudio Martins
On Thu, 8 Nov 2012 09:30:55 -0800 (PST) Sage Weil s...@inktank.com wrote: Lots of different snapshots: - librados lets you do 'selfmanaged snaps' in its API, which let an application control which snapshots apply to which objects. - you can create a 'pool' snapshot on an entire

Re: extreme ceph-osd cpu load for rand. 4k write

2012-11-09 Thread Stefan Priebe - Profihost AG
Am 09.11.2012 um 22:21 schrieb Samuel Just sam.j...@inktank.com: Can you describe the osd and client set up (number of nodes, number of osds per node, journal disks, replication level, and osd disks)? Looks like a lot of time is spent looking up objects in the filestore (lfn_open, etc).

Re: some snapshot problems

2012-11-09 Thread Josh Durgin
On 11/09/2012 01:32 PM, Cláudio Martins wrote: On Thu, 8 Nov 2012 09:30:55 -0800 (PST) Sage Weil s...@inktank.com wrote: Lots of different snapshots: - librados lets you do 'selfmanaged snaps' in its API, which let an application control which snapshots apply to which objects. - you

Re: SSD journal suggestion / rsockets

2012-11-09 Thread Joseph Glanville
On 10 November 2012 01:43, Atchley, Scott atchle...@ornl.gov wrote: On Nov 8, 2012, at 5:00 PM, Joseph Glanville joseph.glanvi...@orionvm.com.au wrote: On 9 November 2012 08:21, Dieter Kasper d.kas...@kabelmail.de wrote: Joseph, I've downloaded and read the presentation from 'Sean Hefty /

[PATCH 0/2] rbd: snap context references

2012-11-09 Thread Alex Elder
These two patches are related to how snapshot context reference counting is handled in rbd_do_op(). -Alex [PATCH 1/2] rbd: fix reference leak in rbd_do_op() [PATCH 2/2] rbd: only get snap context for write requests -- To unsubscribe from this list: send

[PATCH 1/2] rbd: fix reference leak in rbd_do_op()

2012-11-09 Thread Alex Elder
This commit introduced a bug: 4634246d rbd: consolidate rbd_do_op() calls When a read request is being issued, the snapshot context provided isn't needed. But that snap context pointer was acquired in rbd_rq_fn() and it carries with it a reference to that structure. So before discarding it,

[PATCH 2/2] rbd: only get snap context for write requests

2012-11-09 Thread Alex Elder
Right now we get and release the header semaphore every time we process a request for an rbd image. We do this because for write requests we need to supply the snapshot context, and we can't safely get a reference to it without holding that semaphore. There's no need to get the snap context if

[PATCH 0/4] rbd: drop some unnecessary function parameters

2012-11-09 Thread Alex Elder
These four patches drop some parameters that are not needed from a number of functions. -Alex [PATCH 1/4] rbd: drop oid parameters from ceph_osdc_build_request() [PATCH 2/4] rbd: drop snapid parameter from rbd_req_sync_read() [PATCH 3/4] rbd: drop flags

[PATCH 1/4] rbd: drop oid parameters from ceph_osdc_build_request()

2012-11-09 Thread Alex Elder
The last two parameters to ceph_osd_build_request() describe the object id, but the values passed always come from the osd request structure whose address is also provided. Get rid of those last two parameters. Signed-off-by: Alex Elder el...@inktank.com --- drivers/block/rbd.c |

[PATCH 2/4] rbd: drop snapid parameter from rbd_req_sync_read()

2012-11-09 Thread Alex Elder
There is only one caller of rbd_req_sync_read(), and it passes CEPH_NOSNAP as the snapshot id argument. Delete that parameter and just use CEPH_NOSNAP within the function. Signed-off-by: Alex Elder el...@inktank.com --- drivers/block/rbd.c |6 ++ 1 file changed, 2 insertions(+), 4

[PATCH 3/4] rbd: drop flags parameter from rbd_req_sync_exec()

2012-11-09 Thread Alex Elder
All callers of rbd_req_sync_exec() pass CEPH_OSD_FLAG_READ as their flags argument. Delete that parameter and use CEPH_OSD_FLAG_READ within the function. If we find a need to support write operations we can add it back again. Signed-off-by: Alex Elder el...@inktank.com --- drivers/block/rbd.c

[PATCH 4/4] rbd: kill rbd_req_sync_op() snapc and snapid parameters

2012-11-09 Thread Alex Elder
The snapc and snapid parameters to rbd_req_sync_op() always take the values NULL and CEPH_NOSNAP, respectively. So just get rid of them and use those values where needed. Signed-off-by: Alex Elder el...@inktank.com --- drivers/block/rbd.c | 17 + 1 file changed, 5

[PATCH] rbd: don't bother setting snapid in rbd_do_request()

2012-11-09 Thread Alex Elder
For some reason, the snapid field of the osd request header is explicitly set to CEPH_NOSNAP in rbd_do_request(). Just a few lines later--with no code that would access this field in between--a call is made to ceph_calc_raw_layout() passing the snapid provided to rbd_do_request(), which encodes

[PATCH] rbd: get rid of RBD_MAX_SEG_NAME_LEN

2012-11-09 Thread Alex Elder
RBD_MAX_SEG_NAME_LEN represents the maximum length of an rbd object name (i.e., one of the objects providing storage backing an rbd image). Another symbol, MAX_OBJ_NAME_SIZE, is used in the osd client code to define the maximum length of any object name in an osd request. Right now they