Re: SIGSEGV in cephfs-java, but probably in Ceph

On May 31, 2012, at 9:44 PM, Nam Dang wrote: > I pulled the Java lib from > https://github.com/noahdesu/ceph/tree/wip-java-cephfs > However, I use ceph 0.47.1 installed directly from Ubuntu's repository > with apt-get, not the one that I built with the java library. I > assumed that since the ja

Re: SIGSEGV in cephfs-java, but probably in Ceph

I made a mistake in the previous email. As Noah said, this problem is due to the wrapper being used with an unsuccessfully mounted client. However, I think if the mount fails, the wrapper should throw an exception instead of letting the client continue. Best regards, Nam Dang Tokyo Institute of Te

Re: SIGSEGV in cephfs-java, but probably in Ceph

I pulled the Java lib from https://github.com/noahdesu/ceph/tree/wip-java-cephfs However, I use ceph 0.47.1 installed directly from Ubuntu's repository with apt-get, not the one that I built with the java library. I assumed that since the java lib is just a wrapper. >>There are only two segfaults

Re: "rbd rm image" slow with big images ?

>>That said, the current implementation is also stupid: it's doing a single >>io at a time. #2256 (next sprint) will parallelize this to make it go >>much faster (probably an order of magnitude?). Ah, ok, this is why is see low ios/network during delete. Thanks Sage and Wido for the explains,

Re: [PATCH 13/13] libceph: set CLOSED state bit in con_init

Reviewed-by: Sage Weil On Wed, 30 May 2012, Alex Elder wrote: > Once a connection is fully initialized, it is really in a CLOSED > state, so make that explicit by setting the bit in its state field. > > It is possible for a connection in NEGOTIATING state to get a > failure, leading to ceph_fau

Re: [PATCH 12/13] libceph: fully initialize connection in con_init()

Reviewed-by: Sage Weil On Wed, 30 May 2012, Alex Elder wrote: > Move the initialization of a ceph connection's private pointer, > operations vector pointer, and peer name information into > ceph_con_init(). Rearrange the arguments so the connection pointer > is first. Hide the byte-swapping of

Re: [PATCH 11/13] libceph: init monitor connection when opening

yep! On Wed, 30 May 2012, Alex Elder wrote: > Hold off initializing a monitor client's connection until just > before it gets opened for use. > > Signed-off-by: Alex Elder > --- > net/ceph/mon_client.c | 13 ++--- > 1 files changed, 6 insertions(+), 7 deletions(-) > > diff --git a/n

Re: [PATCH 10/13] libceph: provide osd number when creating osd

Looks good! Reviewed-by: Sage Weil On Wed, 30 May 2012, Alex Elder wrote: > Pass the osd number to the create_osd() routine, and move the > initialization of fields that depend on it therein. > > Signed-off-by: Alex Elder > --- > net/ceph/osd_client.c |8 > 1 files changed, 4 in

Re: [PATCH 09/13] libceph: start tracking connection socket state

On Wed, 30 May 2012, Alex Elder wrote: > Start explicitly keeping track of the state of a ceph connection's > socket, separate from the state of the connection itself. Create > placeholder functions to encapsulate the state transitions. > > > | NEW* | transient initial state >

Re: [PATCH 08/13] libceph: start separating connection flags from state

On Wed, 30 May 2012, Alex Elder wrote: > A ceph_connection holds a mixture of connection state (as in "state > machine" state) and connection flags in a single "state" field. To > make the distinction more clear, define a new "flags" field and use > it rather than the "state" field to hold Boolean

Re: [PATCH 07/13] libceph: embed ceph connection structure in mon_client

On Wed, 30 May 2012, Alex Elder wrote: > A monitor client has a pointer to a ceph connection structure in it. > This is the only one of the three ceph client types that do it this > way; the OSD and MDS clients embed the connection into their main > structures. There is always exactly one ceph con

Re: iozone test crashed on ceph

Hi, thanks for your reply. The output of 'modinfo ceph' is as follows: filename: /lib/modules/3.1.10-1.9-desktop/kernel/fs/ceph/ceph.ko license:GPL description:Ceph filesystem for Linux author: Patience Warnick author: Yehuda Sadeh author: Sage Weil s

Re: [PATCH 06/13] libceph: embed ceph messenger structure in ceph_client

Reviewed-by: Sage Weil On Wed, 30 May 2012, Alex Elder wrote: > A ceph client has a pointer to a ceph messenger structure in it. > There is always exactly one ceph messenger for a ceph client, so > there is no need to allocate it separate from the ceph client > structure. > > Switch the ceph_cl

Re: [PATCH 05/13] libceph: rename kvec_reset and kvec_add functions

Yep On Wed, 30 May 2012, Alex Elder wrote: > The functions ceph_con_out_kvec_reset() and ceph_con_out_kvec_add() > are entirely private functions, so drop the "ceph_" prefix in their > name to make them slightly more wieldy. > > Signed-off-by: Alex Elder > --- > net/ceph/messenger.c | 48 +++

Re: [PATCH 04/13] libceph: rename socket callbacks

On Wed, 30 May 2012, Alex Elder wrote: > Change the names of the three socket callback functions to make it > more obvious they're specifically associated with a connection's > socket (not the ceph connection that uses it). > > Signed-off-by: Alex Elder > --- > net/ceph/messenger.c | 28 ++

Re: iozone test crashed on ceph

Hi, Thanks for letting us know. What version are you running? Can you post your ceph.conf to give us an idea of how your cluster is configured? Also, did any of the daemons crash? If it's reproducible, it would help to turn up osd and mds debugging to 20 and post the logs. Thanks -Sam On Thu

Re: RBD operations, pinging client that serves lingering tid

Those messages are harmless. It's just debug output indicating that the objecter is maintaining a watch on an rbd image header. I'll tone down the debug verbosity tomorrow. -Sam On Wed, May 30, 2012 at 6:54 AM, Guido Winkelmann wrote: > Hi, > > Whenever I'm doing any operations on rbd volumes (

iozone test crashed on ceph

Hi, I have set up ceph system with a client, mon and mds on one system which is connected to 2 osds. I ran iozone test with a 10G file and it ran fine. But when I ran iozone test with a 5G file, the process got killed and our ceph system hanged. Can anyone please help me with this. Thanks in adv

Re: SIGSEGV in cephfs-java, but probably in Ceph

On May 31, 2012, at 3:39 PM, Greg Farnum wrote: >> >> Nevermind to my last comment. Hmm, I've seen this, but very rarely. > Noah, do you have any leads on this? Do you think it's a bug in your Java > code or in the C/++ libraries? I _think_ this is because the JVM uses its own threading library

Re: SIGSEGV in cephfs-java, but probably in Ceph

On Thursday, May 31, 2012 at 7:43 AM, Noah Watkins wrote: > > On May 31, 2012, at 6:20 AM, Nam Dang wrote: > > > > Stack: [0x7ff6aa828000,0x7ff6aa929000], > > > sp=0x7ff6aa9274f0, free space=1021k > > > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native > > > c

Re: "rbd rm image" slow with big images ?

On 05/31/2012 08:16 PM, Stefan Priebe wrote: One note: he has written: "then just delete it, without having writed nothing in image " That is true, but RBD doesn't know that. There is no record of which object got created and which didn't, so the removal process has to issue a removal for eac

Re: "rbd rm image" slow with big images ?

On Thu, 31 May 2012, Wido den Hollander wrote: > Hi, > > Is it the normal behaviour ? Maybe some xfs tuning could help ? > > It's in the nature of RBD. Yes. That said, the current implementation is also stupid: it's doing a single io at a time. #2256 (next sprint) will parallelize this to make

Re: "rbd rm image" slow with big images ?

One note: he has written: "then just delete it, without having writed nothing in image " Am 31.05.2012 20:15, schrieb Wido den Hollander: Hi, On 05/31/2012 09:12 AM, Alexandre DERUMIER wrote: Hi, I trying to delete some rbd images with rbd rm, and it seem to be "slow" with big images. I'm

Re: "rbd rm image" slow with big images ?

Hi, On 05/31/2012 09:12 AM, Alexandre DERUMIER wrote: Hi, I trying to delete some rbd images with rbd rm, and it seem to be "slow" with big images. I'm testing it with just create a new image (1TB): # time rbd -p pool1 create --size 100 image2 real0m0.031s user0m0.015s sys

Re: [PATCH 06/13] libceph: embed ceph messenger structure in ceph_client

Reviewed-by: Yehuda Sadeh On Wed, May 30, 2012 at 12:34 PM, Alex Elder wrote: > A ceph client has a pointer to a ceph messenger structure in it. > There is always exactly one ceph messenger for a ceph client, so > there is no need to allocate it separate from the ceph client > structure. > > Swi

Re: poor OSD performance using kernel 3.4 => problem found

Le 31/05/2012 18:29, Sage Weil a écrit : Can you post 'ceph osd dump | grep ^pool' so we can see which CRUSH rules the pools are mapped to? yes : root@label5:~# ceph osd dump | grep ^pool pool 0 'data' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num 576 pgp_num 576 last_change 816 ow

Re: [PATCH 05/13] libceph: rename kvec_reset and kvec_add functions

Reviewed-by: Yehuda Sadeh On Wed, May 30, 2012 at 12:34 PM, Alex Elder wrote: > The functions ceph_con_out_kvec_reset() and ceph_con_out_kvec_add() > are entirely private functions, so drop the "ceph_" prefix in their > name to make them slightly more wieldy. > > Signed-off-by: Alex Elder > ---

Re: [PATCH 04/13] libceph: rename socket callbacks

Reviewed-by: Yehuda Sadeh On Wed, May 30, 2012 at 12:34 PM, Alex Elder wrote: > Change the names of the three socket callback functions to make it > more obvious they're specifically associated with a connection's > socket (not the ceph connection that uses it). > > Signed-off-by: Alex Elder >

Re: [PATCH 02/13] libceph: kill bad_proto ceph connection op

Reviewed-by: Yehuda Sadeh On Wed, May 30, 2012 at 12:34 PM, Alex Elder wrote: > No code sets a bad_proto method in its ceph connection operations > vector, so just get rid of it. > > Signed-off-by: Alex Elder > --- >  include/linux/ceph/messenger.h |    3 --- >  net/ceph/messenger.c           |

Re: poor OSD performance using kernel 3.4 => problem found

On Thu, 31 May 2012, Yann Dupont wrote: > On 31/05/2012 17:32, Mark Nelson wrote: > > ceph osd pool get pg_num > > My setup is detailed in a previous mail , But as I changed some parameters > this morning, here we go : > > root@chichibu:~# ceph osd pool get data pg_num > PG_NUM: 576 > root@chich

Re: differents ip/network link for osd replication and client->osd ?

On Thu, 31 May 2012, Alexandre DERUMIER wrote: > Hi, > Is it possible to use differents ip / network link for > > - replication between osd > - network between client and osd > > ? > > I would like to use differents swichs/network card for osd replication. Yep: [osd] public network =

Re: [PATCH 01/13] libceph: eliminate connection state "DEAD"

Reviewed-by: Yehuda Sadeh On Wed, May 30, 2012 at 12:34 PM, Alex Elder wrote: > The ceph connection state "DEAD" is never set and is therefore not > needed.  Eliminate it. > > Signed-off-by: Alex Elder > --- >  include/linux/ceph/messenger.h |    1 - >  net/ceph/messenger.c           |    6 ---

Re: poor OSD performance using kernel 3.4 => problem found

On 05/31/2012 10:43 AM, Yann Dupont wrote: On 31/05/2012 17:32, Mark Nelson wrote: ceph osd pool get pg_num My setup is detailed in a previous mail , But as I changed some parameters this morning, here we go : root@chichibu:~# ceph osd pool get data pg_num PG_NUM: 576 root@chichibu:~# ceph os

Re: SIGSEGV in cephfs-java, but probably in Ceph

On May 31, 2012, at 8:48 AM, Nam Dang wrote: > Hi Noah, > > By the way, the test suite of cephfs-java has a bug. You should put > the permission value in the form of 0777 instead of 777 since the > number has to be octal. With 777 I got directories with weird > permission settings. Thanks Nam,

Re: SIGSEGV in cephfs-java, but probably in Ceph

Hi Noah, By the way, the test suite of cephfs-java has a bug. You should put the permission value in the form of 0777 instead of 777 since the number has to be octal. With 777 I got directories with weird permission settings. Best regards Nam Dang Tokyo Institute of Technology Tokyo, Japan On T

Re: poor OSD performance using kernel 3.4 => problem found

On 31/05/2012 17:32, Mark Nelson wrote: ceph osd pool get pg_num My setup is detailed in a previous mail , But as I changed some parameters this morning, here we go : root@chichibu:~# ceph osd pool get data pg_num PG_NUM: 576 root@chichibu:~# ceph osd pool get rbd pg_num PG_NUM: 576 The

Re: poor OSD performance using kernel 3.4 => problem found

On 05/31/2012 09:42 AM, Yann Dupont wrote: On 31/05/2012 15:45, Yann Dupont wrote: On 31/05/2012 15:37, Stefan Priebe - Profihost AG wrote: what puzzles me is that this morning, with 3.4.0 it was rbd that was stable, and now I have the exact contrary. I'll begin to reboot with old 3.4.0 kern

differents ip/network link for osd replication and client->osd ?

Hi, Is it possible to use differents ip / network link for - replication between osd - network between client and osd ? I would like to use differents swichs/network card for osd replication. Regards, Alexandre -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the b

Re: SIGSEGV in cephfs-java, but probably in Ceph

On May 31, 2012, at 6:20 AM, Nam Dang wrote: >> Stack: [0x7ff6aa828000,0x7ff6aa929000], >> sp=0x7ff6aa9274f0, free space=1021k >> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native >> code) >> C [libcephfs.so.1+0x139d39] Mutex::Lock(bool)+0x9 >> >> Java fra

Re: poor OSD performance using kernel 3.4 => problem found

On 31/05/2012 15:45, Yann Dupont wrote: On 31/05/2012 15:37, Stefan Priebe - Profihost AG wrote: what puzzles me is that this morning, with 3.4.0 it was rbd that was stable, and now I have the exact contrary. I'll begin to reboot with old 3.4.0 kernel to see if things are reproductible. Chee

Re: SIGSEGV in cephfs-java, but probably in Ceph

On May 31, 2012, at 6:20 AM, Nam Dang wrote: > It turned out my monitor went down without my knowing. > So my bad, it wasn't because of Ceph. I believe the segfault here is from client being null dereferenced in the c wrappers. Which patch set are you using? > > Best regards, > > Nam Dang >

Re: poor OSD performance using kernel 3.4 => problem found

On 31/05/2012 15:37, Stefan Priebe - Profihost AG wrote: Am 31.05.2012 15:21, schrieb Yann Dupont: On 31/05/2012 09:30, Yehuda Sadeh wrote: On Thu, May 31, 2012 at 12:10 AM, Stefan Priebe - Profihost AG wrote: But very strangely it's now rbd that isn't stable ?! root@label5:~# rados -p rbd b

Re: poor OSD performance using kernel 3.4 => problem found

Am 31.05.2012 15:21, schrieb Yann Dupont: On 31/05/2012 09:30, Yehuda Sadeh wrote: On Thu, May 31, 2012 at 12:10 AM, Stefan Priebe - Profihost AG wrote: But very strangely it's now rbd that isn't stable ?! root@label5:~# rados -p rbd bench 20 write -t 16 Maintaining 16 concurrent writes of 41

Re: poor OSD performance using kernel 3.4 => problem found

On 31/05/2012 09:30, Yehuda Sadeh wrote: On Thu, May 31, 2012 at 12:10 AM, Stefan Priebe - Profihost AG wrote: Hi Marc, Hi Stefan, Hello, back today Today, I upgraded my 2 last osd nodes with big storage, so now all my nodes are equivalent. Using 3.4.0 kernel, I still have good results w

Re: SIGSEGV in cephfs-java, but probably in Ceph

It turned out my monitor went down without my knowing. So my bad, it wasn't because of Ceph. Best regards, Nam Dang Tokyo Institute of Technology Tokyo, Japan On Thu, May 31, 2012 at 10:08 PM, Nam Dang wrote: > Dear all, > > I am running a small benchmark for Ceph with multithreading and cephf

SIGSEGV in cephfs-java, but probably in Ceph

Dear all, I am running a small benchmark for Ceph with multithreading and cephfs-java API. I encountered this issue even when I use only two threads, and I used only open file and creating directory operations. The piece of code is simply: String parent = filePath.substring(0, filePath.lastIndexO

Re: poor OSD performance using kernel 3.4 => problem found

Am 31.05.2012 14:31, schrieb Mark Nelson: Hi Stefan, Please do share! I was planning on starting out on the wiki and eventually getting these kinds of things into the master docs. If you (and others) have already done testing it would be really interesting to compare experiences. So far I've bee

Re: poor OSD performance using kernel 3.4 => problem found

Hi Stefan, Please do share! I was planning on starting out on the wiki and eventually getting these kinds of things into the master docs. If you (and others) have already done testing it would be really interesting to compare experiences. So far I've been just kind of throwing stuff into:

Re: poor OSD performance using kernel 3.4 => problem found

Hi Mark, Hi Stefan, i found a way to solve it by comparing /proc/sys/net with an patched and an unpatched kernel. Strangely the problem occours when the values are too big (in new kernel). With the smaller values everything works fine even under 3.4. Any ideas how that can be? I thought thes

Re: Wiki Spam

Doh! Thanks for the heads-up. We'll deal with it. Thanks, Mark On 5/31/12 2:05 AM, "SPONEM, Benoît" wrote: Dear all, Just for information, there are a lot of spam in Ceph's wiki (http://ceph.com/wiki/Special:RecentChanges, http://ceph.com/w/index.php?title=Special:LonelyPages&limit=250&offse

Re: poor OSD performance using kernel 3.4 => problem found

Am 31.05.2012 10:09, schrieb Stefan Majer: Hi Stefan, then you should probably describe this in a short mail to Jason Wang and ask him how to circumvent this commit with sysctl settings. done hopefully he can help I´m pretty sure my sysctl setting reverts the first part of the commit. So pro

Re: poor OSD performance using kernel 3.4 => problem found

Hi Stefan, then you should probably describe this in a short mail to Jason Wang and ask him how to circumvent this commit with sysctl settings. I´m pretty sure my sysctl setting reverts the first part of the commit. So probably the second part is the evil one ? Greetings Stefan On Thu, May 31, 2

Re: poor OSD performance using kernel 3.4 => problem found

Am 31.05.2012 09:27, schrieb Stefan Majer: > we have set them in /etc/sysctl.conf to: > net.ipv4.tcp_mem = 1000 1000 1000 This does not help ;-( > wow, this was fast ! > if i understand this commit correct it simply skips a in-kernel > configuration of network related sysctl parameter

Re: poor OSD performance using kernel 3.4 => problem found

On Thu, May 31, 2012 at 12:10 AM, Stefan Priebe - Profihost AG wrote: > Hi Marc, Hi Stefan, > > first thanks for all your help and time. > > I found the commit which results in this problem and it is TCP related > but i'm still wondering if the expected behaviour of this commit is > expected? > >

"rbd rm image" slow with big images ?

Hi, I trying to delete some rbd images with rbd rm, and it seem to be "slow" with big images. I'm testing it with just create a new image (1TB): # time rbd -p pool1 create --size 100 image2 real0m0.031s user0m0.015s sys 0m0.010s then just delete it, without having writed not

Re: poor OSD performance using kernel 3.4 => problem found

Hi Marc, Hi Stefan, first thanks for all your help and time. I found the commit which results in this problem and it is TCP related but i'm still wondering if the expected behaviour of this commit is expected? The commit in question is: git show c43b874d5d714f271b80d4c3f49e05d0cbf51ed2 commit c4

Wiki Spam

Dear all, Just for information, there are a lot of spam in Ceph's wiki (http://ceph.com/wiki/Special:RecentChanges, http://ceph.com/w/index.php?title=Special:LonelyPages&limit=250&offset=0 ). Regards, Benoit -- To unsubscri