Re: [PATCH 1/2] libceph: embed ceph connection structure in mon_client

2012-06-05 Thread Sage Weil
Reviewed-by: Sage Weil On Tue, 5 Jun 2012, Alex Elder wrote: > A monitor client has a pointer to a ceph connection structure in it. > This is the only one of the three ceph client types that do it this > way; the OSD and MDS clients embed the connection into their main > structures. There is al

Re: [PATCH 4/4] libceph: make ceph_con_revoke_message() a msg op

2012-06-05 Thread Sage Weil
On Tue, 5 Jun 2012, Alex Elder wrote: > ceph_con_revoke_message() is passed both a message and a ceph > connection. A ceph_msg allocated for incoming messages on a > connection always has a pointer to that connection, so there's no > need to provide the connection when revoking such a message. >

Re: [PATCH 3/4] libceph: make ceph_con_revoke() a msg operation

2012-06-05 Thread Sage Weil
Reviewed-by: Sage Weil I'm mildly concerned that if the msg isn't on the con the upper layer thinks it is it is a bug, and we might be better off keeping this arg and asserting that con == msg->con... On Tue, 5 Jun 2012, Alex Elder wrote: > ceph_con_revoke() is passed both a message and a ce

Re: [PATCH 1/4] libceph: have messages point to their connection

2012-06-05 Thread Sage Weil
Reviewed-by: Sage Weil On Tue, 5 Jun 2012, Alex Elder wrote: > When a ceph message is queued for sending it is placed on a list of > pending messages (ceph_connection->out_queue). When they are > actually sent over the wire, they are moved from that list to > another (ceph_connection->out_sent

Re: [PATCH] libceph: tweak ceph_alloc_msg()

2012-06-05 Thread Sage Weil
On Tue, 5 Jun 2012, Alex Elder wrote: > The function ceph_alloc_msg() is only used to allocate a message > that will be assigned to a connection's in_msg pointer. Rename the > function so this implied usage is more clear. > > In addition, make that assignment inside the function (again, since >

Fwd: mount: 10.0.6.10:/: can't read superblock

2012-06-05 Thread Martin Wilderoth
Trying to remove one of the folders made the mds.a and mds.b to stop. so somting is wrong in my mds. ceph -s gives 2012-06-06 06:19:19.899973 pg v1220573: 1152 pgs: 1152 active+clean; 191 GB data, 393 GB used, 973 GB / 1379 GB avail 2012-06-06 06:19:19.905097 mds e78: 1/1/1 up {0=c=up:active}

Re: iozone test crashed on ceph

2012-06-05 Thread udit agarwal
Hi, Thank you all for your support in resolving the issue. As I mentioned in my previous post that I ran my iozone test again on a 5G file excluding the 4k record size. And now, I have successfully completed this test with record sizes of 8k and 16k and hopefully all others will also work fine. Bu

Re: [PATCH] ceph.spec.in: Fix license to GPL-2.0 and use SPDX format

2012-06-05 Thread Matt W. Benjamin
That wouldn't seem to make a lot of sense, no. Matt - "Sage Weil" wrote: > On Tue, 5 Jun 2012, Holger Macht wrote: > > Minor spec file fix. LGPLv2 in spec file is not correct, because > some of > > the included packages/binaries are GPLv2. For example: > > > > src/os/btrfs_ioctl.h -> pack

[PATCH 4/4] libceph: make ceph_con_revoke_message() a msg op

2012-06-05 Thread Alex Elder
ceph_con_revoke_message() is passed both a message and a ceph connection. A ceph_msg allocated for incoming messages on a connection always has a pointer to that connection, so there's no need to provide the connection when revoking such a message. Note that the existing logic does not preclude t

[PATCH 3/4] libceph: make ceph_con_revoke() a msg operation

2012-06-05 Thread Alex Elder
ceph_con_revoke() is passed both a message and a ceph connection. Now that any message associated with a connection holds a pointer to that connection, there's no need to provide the connection when revoking a message. This has the added benefit of precluding the possibility of the providing the w

[PATCH 2/4] libceph: have messages take a connection reference

2012-06-05 Thread Alex Elder
There are essentially two types of ceph messages: incoming and outgoing. Outgoing messages are always allocated via ceph_msg_new(), and at the time of their allocation they are not associated with any particular connection. Incoming messages are always allocated via ceph_con_in_msg_alloc(), and t

[PATCH 1/4] libceph: have messages point to their connection

2012-06-05 Thread Alex Elder
When a ceph message is queued for sending it is placed on a list of pending messages (ceph_connection->out_queue). When they are actually sent over the wire, they are moved from that list to another (ceph_connection->out_sent). When acknowledgement for the message is received, it is removed from

[PATCH] libceph: tweak ceph_alloc_msg()

2012-06-05 Thread Alex Elder
The function ceph_alloc_msg() is only used to allocate a message that will be assigned to a connection's in_msg pointer. Rename the function so this implied usage is more clear. In addition, make that assignment inside the function (again, since that's precisely what it's intended to be used for)

[PATCH 2/2] libceph: fully initialize connection in con_init()

2012-06-05 Thread Alex Elder
Move the initialization of a ceph connection's private pointer, operations vector pointer, and peer name information into ceph_con_init(). Rearrange the arguments so the connection pointer is first. Hide the byte-swapping of the peer entity number inside ceph_con_init() Signed-off-by: Alex Elder

[PATCH 1/2] libceph: init monitor connection when opening

2012-06-05 Thread Alex Elder
Hold off initializing a monitor client's connection until just before it gets opened for use. Signed-off-by: Alex Elder Reviewed-by: Sage Weil --- net/ceph/mon_client.c | 13 ++--- 1 files changed, 6 insertions(+), 7 deletions(-) diff --git a/net/ceph/mon_client.c b/net/ceph/mon_clie

[PATCH 2/2] libceph: drop connection refcounting for mon_client

2012-06-05 Thread Alex Elder
>From Sage Weil All references to the embedded ceph_connection come from the msgr workqueue, which is drained prior to mon_client destruction. That means we can ignore con refcounting entirely. Signed-off-by: Sage Weil Reviewed-by: Alex Elder --- net/ceph/mon_client.c | 18

[PATCH 1/2] libceph: embed ceph connection structure in mon_client

2012-06-05 Thread Alex Elder
A monitor client has a pointer to a ceph connection structure in it. This is the only one of the three ceph client types that do it this way; the OSD and MDS clients embed the connection into their main structures. There is always exactly one ceph connection for a monitor client, so there is no ne

[PATCH] libceph: use con get/put ops from osd_client

2012-06-05 Thread Alex Elder
>From Sage Weil There were a few direct calls to ceph_con_{get,put}() instead of the con ops from osd_client.c. This is a bug since those ops aren't defined to be ceph_con_get/put. This breaks refcounting on the ceph_osd structs that contain the ceph_connections, and could lead to all manner of

[PATCH] libceph: osd_client: don't drop reply reference too early

2012-06-05 Thread Alex Elder
In ceph_osdc_release_request(), a reference to the r_reply message is dropped. But just after that, that same message is revoked if it was in use to receive an incoming reply. Reorder these so we are sure we hold a reference until we're actually done with the message. Signed-off-by: Alex Elder

[PATCH 00/11] continued messenger-related changes

2012-06-05 Thread Alex Elder
This is a series of 11 related patches but I've grouped them into smaller sub-sets as follows. A few of these were posted previously but their changes--if any--were merely related to adjustments to account for other patches inserted before them in the series. Two of them are being submitted on be

Re: iozone test crashed on ceph

2012-06-05 Thread udit agarwal
Hi, Thanks for your reply. After you hinted that this problem may have caused due to using record size of length 4k. Then, I ran iozone test on a 5m file with record length 4k and to my surprise, the ceph system again hanged up. Then, I tried same with record lengths from 8k to 1m and everything

Re: [PATCH] ceph.spec.in: Fix license to GPL-2.0 and use SPDX format

2012-06-05 Thread Sage Weil
On Tue, 5 Jun 2012, Holger Macht wrote: > Minor spec file fix. LGPLv2 in spec file is not correct, because some of > the included packages/binaries are GPLv2. For example: > > src/os/btrfs_ioctl.h -> package ceph, binary ceph-osd My understanding is that including a header file (in this case, to

Can we mount cephfs in centos 5.7 ?

2012-06-05 Thread Eric_YH_Chen
Dear All:   My ceph cluster is installed on Ubuntu 12.04 with kernel 3.2.0.    But I have a situation that I may need to mount cephfs on centos 5.7, which with kernel 2.6.18. (that is to say , I only need to execute mount.ceph on the centos 5.7, not the whole ceph system) I want to

Re: mount: 10.0.6.10:/: can't read superblock

2012-06-05 Thread Greg Farnum
For future reference, that error was because the active MDS server was in replay. I can't tell why it didn't move on to active from what you posted, but I imagine it just got a little stuck since restarting made it work out. -Greg On Tuesday, June 5, 2012 at 1:05 PM, Martin Wilderoth wrote: >

Re: mount: 10.0.6.10:/: can't read superblock

2012-06-05 Thread Martin Wilderoth
Hello Again, I restarted the mds on all servers and then it worked again /Regards Martin > Hello > > > Hi Martin, > > > > On 06/05/2012 08:07 PM, Martin Wilderoth wrote: > > > Hello > > > > > > Is there a way to recover this error. > > > > > > mount -t ceph 10.0.6.10:/ /mnt -vv -o > >

Re: mount: 10.0.6.10:/: can't read superblock

2012-06-05 Thread Martin Wilderoth
Hello > Hi Martin, > > On 06/05/2012 08:07 PM, Martin Wilderoth wrote: > > Hello > > > > Is there a way to recover this error. > > > > mount -t ceph 10.0.6.10:/ /mnt -vv -o > > name=admin,secret=XXX > > [ 506.640433] libceph: loaded (mon/osd proto 15/24, osdmap 5/6 5/6)

Re: mount: 10.0.6.10:/: can't read superblock

2012-06-05 Thread Wido den Hollander
Hi Martin, On 06/05/2012 08:07 PM, Martin Wilderoth wrote: Hello Is there a way to recover this error. mount -t ceph 10.0.6.10:/ /mnt -vv -o name=admin,secret=XXX [ 506.640433] libceph: loaded (mon/osd proto 15/24, osdmap 5/6 5/6) [ 506.650594] ceph: loaded (mds proto 32)

mount: 10.0.6.10:/: can't read superblock

2012-06-05 Thread Martin Wilderoth
Hello Is there a way to recover this error. mount -t ceph 10.0.6.10:/ /mnt -vv -o name=admin,secret=XXX [ 506.640433] libceph: loaded (mon/osd proto 15/24, osdmap 5/6 5/6) [ 506.650594] ceph: loaded (mds proto 32) [ 506.652353] libceph: client0 fsid a9d5f9e1-4bb9-4fab-b79b-

Re: iozone test crashed on ceph

2012-06-05 Thread Tommi Virtanen
On Mon, Jun 4, 2012 at 4:43 PM, udit agarwal wrote: > Sorry ,the link is: [...] If you run iozone again, does the bug happen again? Comparing your iozone run with our test suite, we don't currently do -i 2 (random read/write), and we only run a few specific record sizes to save time (-r 16K,1M);

Re: OSD per disk.

2012-06-05 Thread Tommi Virtanen
On Tue, Jun 5, 2012 at 5:34 AM, chandrashekhar wrote: >>> > I built ceph from source ( f859f25d7eaec01fb6a82b88409d005c8177fea2  branch). > > mkcephfs -a -c /etc/ceph/ceph.conf  -k /etc/ceph/keyring.admin -- is working > fine, but when I tried to start ceph its showing following error. I'm sorry,

Re: SIGSEGV in cephfs-java, but probably in Ceph

2012-06-05 Thread Noah Watkins
Thanks for testing this, Nam. On Tue, Jun 5, 2012 at 4:22 AM, Nam Dang wrote: > Hi Noah, > > Thank you for the push. Now I don't have SIGSEGV anymore. By the way, > is there any way to move the ceph_is_mounted() function into the java > library's code instead of putting it in the main Ceph's code

Re: [PATCH] ceph.spec.in: Fix license to GPL-2.0 and use SPDX format

2012-06-05 Thread Udo Seidel
Hi Holger, On Tue, 5 Jun 2012, Holger Macht wrote: Minor spec file fix. LGPLv2 in spec file is not correct, because some of the included packages/binaries are GPLv2. For example: cool. src/os/btrfs_ioctl.h -> package ceph, binary ceph-osd +License: GPL-2.0 Group: Syste

[PATCH] ceph.spec.in: Fix license to GPL-2.0 and use SPDX format

2012-06-05 Thread Holger Macht
Minor spec file fix. LGPLv2 in spec file is not correct, because some of the included packages/binaries are GPLv2. For example: src/os/btrfs_ioctl.h -> package ceph, binary ceph-osd src/mount/mtab.c -> package ceph, binary mount.ceph src/common/fiemap.cc -> package ceph, binary rbd Also us

Re: OSD per disk.

2012-06-05 Thread chandrashekhar
>> I built ceph from source ( f859f25d7eaec01fb6a82b88409d005c8177fea2 branch). mkcephfs -a -c /etc/ceph/ceph.conf -k /etc/ceph/keyring.admin -- is working fine, but when I tried to start ceph its showing following error. >strace ceph -a start rt_sigprocmask(SIG_SETMASK, [PIPE], NULL, 8) = 0 r

Re: SIGSEGV in cephfs-java, but probably in Ceph

2012-06-05 Thread Nam Dang
Hi Noah, Thank you for the push. Now I don't have SIGSEGV anymore. By the way, is there any way to move the ceph_is_mounted() function into the java library's code instead of putting it in the main Ceph's code? > The basic idea was that threads in Java did not map 1:1 with kernel > threads (think

Re: Infiniband 40GB

2012-06-05 Thread Stefan Priebe - Profihost AG
Hi Stefan, Am 04.06.2012 10:23, schrieb Stefan Majer: our production environment is running on 10GB infrastructure. We had a lot of troubles till we got to where we are today. We use Intel X520 D2 cards on our OSD´s and nexus switch infrastructure. All other cards we where testing failed horribl

Re: Infiniband 40GB

2012-06-05 Thread Stefan Priebe - Profihost AG
Am 05.06.2012 09:08, schrieb Amon Ott: On Monday 04 June 2012 wrote Stefan Priebe: Hi Amon, thanks for your backported patch. At least it doesn't cleanly apply to debian squeeze stable as it wants a glic 2.12 in Versions.def but Debian is only at 2.11? Do you use another patch too? I ripped t

Re: Infiniband 40GB

2012-06-05 Thread Amon Ott
On Monday 04 June 2012 wrote Stefan Priebe: > Hi Amon, > > thanks for your backported patch. At least it doesn't cleanly apply to > debian squeeze stable as it wants a glic 2.12 in Versions.def but Debian > is only at 2.11? Do you use another patch too? I ripped the patch right out of our previous