[Cluster-devel] [PATCH 13/18] [try #3] DLM: fix conversion deadlock when DLM_LKF_NODLCKWT flag is set

2017-09-14 Thread tsutomu.owa
Hi, Thanks for your comments. > From: David Teigland [mailto:teigl...@redhat.com] > Sent: Friday, September 15, 2017 12:14 AM > >> On Thu, Sep 14, 2017 at 09:31:59AM +, tsutomu@toshiba.co.jp wrote: >> Hi, >> >> > Thanks for sorting this out. Could we expand this with a description of >>

Re: [Cluster-devel] [PATCH 13/18] [try #2] DLM: fix conversion deadlock when DLM_LKF_NODLCKWT flag is set

2017-09-14 Thread tsutomu.owa
Hi, > Thanks for sorting this out. Could we expand this with a description of > the externally visible effects from the change? This fix has no effect except when using DLM_LKF_NODLCKWT flag. We are using fsdlm with ocfs2. Currently, DLM_LKF_NODLCKWT flag is used only by ocfs2. ocfs2 does not ex

[Cluster-devel] [PATCH 07/18] [try #2] DLM: fix to use sock_mutex correctly in xxx_accept_from_sock

2017-09-12 Thread tsutomu.owa
In the current implementation, we think that exclusion control for othercon in tcp_accept_from_sock() and sctp_accept_from_sock() was not enough. We fix them. Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/lowcomms.c | 6 ++ 1 file changed, 6 insertions(+) diff --git

[Cluster-devel] [PATCH 17/18] [try #2] DLM: fix to reschedule rwork

2017-09-12 Thread tsutomu.owa
When an error occurs in kernel_recvmsg or kernel_sendpage and close_connection is called and receive work is already scheduled, receive work is canceled. In that case, the receive work will not be scheduled forever after reconnection, because CF_READ_PENDING flag is established. Signed-off-by: Tad

[Cluster-devel] [PATCH 11/18] [try #2] DLM: Reanimate CF_WRITE_PENDING flag

2017-09-12 Thread tsutomu.owa
CF_WRITE_PENDING flag has been reanimated to make dlm_send stop properly when running dlm_lowcomms_stop. Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/lowcomms.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcom

[Cluster-devel] [PATCH 08/18] [try #2] DLM: retry rcom when dlm_wait_function is timed out.

2017-09-12 Thread tsutomu.owa
If a node sends a DLM_RCOM_STATUS command and an error occurs on the receiving side, the DLM_RCOM_STATUS_REPLY response may not be returned. We retransmitted the DLM_RCOM_STATUS command so that we do not wait for an infinite response. Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --

[Cluster-devel] [PATCH 02/18] [try #2] DLM: Eliminate CF_WRITE_PENDING flag

2017-09-12 Thread tsutomu.owa
--- This version implements Steve Whitehouse's suggestion to put cond_resched() after the queue_work in function send_to_sock. This was just a thinko; it does make more sense to do it afterward. Before this patch the CF_WRITE_PENDING flag was used to indicate when writes to the socket were pending

[Cluster-devel] [PATCH 06/18] [try #2] DLM: fix race condition between dlm_send and dlm_recv

2017-09-12 Thread tsutomu.owa
When kernel_sendpage(in send_to_sock) and kernel_recvmsg (in receive_from_sock) return error, close_connection may works at the same time. At that time, they may wait for each other by cancel_work_sync. Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/lowcomms.c | 8 ++-

[Cluster-devel] [PATCH 13/18] [try #2] DLM: fix conversion deadlock when DLM_LKF_NODLCKWT flag is set

2017-09-12 Thread tsutomu.owa
When the DLM_LKF_NODLCKWT flag was set, even if conversion deadlock was detected, the caller of can_be_granted() was unknown. We change the behavior of can_be_granted() and change it to detect conversion deadlock regardless of whether the DLM_LKF_NODLCKWT flag is set or not. And depending on whethe

[Cluster-devel] [PATCH 14/18] [try #2] DLM: fix memory leak in tcp_accept_from_sock()

2017-09-12 Thread tsutomu.owa
The sk member of the socket generated by sock_create_kern() is overwritten by ops->accept(). So the previous sk will not be released. We use kernel_accept() instead of sock_create_kern() and ops->accept(). Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/lowcomms.c | 21 +++

[Cluster-devel] [PATCH 18/18] [try #2] DLM: fix NULL pointer dereference in send_to_sock()

2017-09-12 Thread tsutomu.owa
The writequeue and writequeue_lock member of othercon was not initialized. If lowcomms_state_change() is called from network layer, othercon->swork may be scheduled. In this case, send_to_sock() will generate a NULL pointer reference. We avoid this problem by correctly initializing writequeue and w

[Cluster-devel] [PATCH 05/18] [try #2] DLM: fix double list_del()

2017-09-12 Thread tsutomu.owa
dlm_lowcomms_stop() was not functioning properly. Correctly, we have to wait until all processing is finished with send_workqueue and recv_workqueue. This problem causes the following issue. Senario is 1. dlm_send thread: send_to_sock refers con->writequeue 2. main thread: dlm_lowcomms_sto

[Cluster-devel] [PATCH 01/18] [try #2] DLM: Eliminate CF_CONNECT_PENDING flag

2017-09-12 Thread tsutomu.owa
Before this patch, there was a flag in the con structure that was used to determine whether or not a connect was needed. The bit was set here and there, and cleared here and there, so it left some race conditions: the bit was set, work was queued, then the worker cleared the bit, allowing someone e

[Cluster-devel] [PATCH 03/18] [try #2] DLM: Fix saving of NULL callbacks

2017-09-12 Thread tsutomu.owa
In a previous patch I noted that accept() often copies the struct sock (sk) which overwrites the sock callbacks. However, in testing we discovered that the dlm connection structures (con) are sometimes deleted and recreated as connections come and go, and since they're zeroed out by kmem_cache_zall

[Cluster-devel] [PATCH 10/18] [try #2] DLM: fix race condition between dlm_recoverd_stop and dlm_recoverd

2017-09-12 Thread tsutomu.owa
When dlm_recoverd_stop() is called between kthread_should_stop() and set_task_state(TASK_INTERRUPTIBLE), dlm_recoverd will not wake up. Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/recoverd.c | 11 ++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git

[Cluster-devel] [PATCH 16/18] [try #2] DLM: fix to use sk_callback_lock correctly

2017-09-12 Thread tsutomu.owa
In the current implementation, we think that exclusion control between processing to set the callback function to the connection structure and processing to refer to the connection structure from the callback function was not enough. We fix them. Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsu

[Cluster-devel] [PATCH 04/18] [try #2] DLM: fix remove save_cb argument from add_sock()

2017-09-12 Thread tsutomu.owa
save_cb argument is not used. We remove them. Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/lowcomms.c | 16 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcomms.c index f5eea54..585a327 100644 --- a/fs/dlm/lo

[Cluster-devel] [PATCH 09/18] [try #2] DLM: close othercon at send/receive error

2017-09-12 Thread tsutomu.owa
If an error occurs in the sending / receiving process, if othercon exists, sending / receiving processing using othercon may also result in an error. We fix to pre-close othercon as well. Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/lowcomms.c | 4 ++-- 1 file changed,

[Cluster-devel] [PATCH 15/18] [try #2] DLM: fix overflow dlm_cb_seq

2017-09-12 Thread tsutomu.owa
dlm_cb_seq is 64 bits. If dlm_cb_seq overflows and returns to 0, dlm_rem_lkb_callback() will not work properly. Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/ast.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/dlm/ast.c b/fs/dlm/ast.c index 07fed83..562fa8c 10

[Cluster-devel] [PATCH 12/18] [try #2] DLM: use CF_CLOSE flag to stop dlm_send correctly

2017-09-12 Thread tsutomu.owa
If reconnection fails while executing dlm_lowcomms_stop, dlm_send will not stop. Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/lowcomms.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcomms.c index 4b33614..e2067a6 100644 --- a/fs/dlm/

[Cluster-devel] [PATCH 00/18] [try #2] DLM: dlm patches need review

2017-09-12 Thread tsutomu.owa
Hi, This series of patches (2nd version after previous review on August) is to fix various bugs. This patch set is against the mainline kernel. We'd like reviewed to make sure those changes are fine. Patch number 01/02/03 were posted on this mailing list. However, I've modifed 03 to correct a bu

Re: [Cluster-devel] [PATCH 02/17] DLM: Eliminate CF_WRITE_PENDING flag

2017-08-27 Thread tsutomu.owa
Hi, Thanks a lot for your patch. Yes, we're thinking about sending the next revised patch set after fixing the [13/17] which has currenly been tested. We'll re-send the patch set in a few days. thanks, -- owa -Original Message- From: Bob Peterson [mailto:rpete...@redhat.com] Sent: Sa

Re: [Cluster-devel] [PATCH 16/17] dlm: fix to use sk_callback_lock correctly

2017-08-17 Thread tsutomu.owa
Hi, > This patch could do with a description, but the addition of locking for > the callbacks is definitely something that needs doing, The reason for this fix is not because something went wrong. In the current implementation, we think that exclusion control between processing to set the callb

Re: [Cluster-devel] [PATCH 13/17] dlm: fix _can_be_granted() for lock at the head of covert queue.

2017-08-17 Thread tsutomu.owa
Hi, >> After looking more closely, this is a subtle form of conversion deadlock, >> and this exact case is described in the comment here: thanks, we withdraw this patch for now since we need to look into more. -- owa -Original Message- From: David Teigland [mailto:teigl...@redhat.com]

Re: [Cluster-devel] [PATCH 10/17] dlm: use schedule_timeout instead of schedule in dlm_recoverd

2017-08-17 Thread tsutomu.owa
Hi, > This works, but have you looked elsewhere in the kernel for kthread > examples we could copy that do a race-free check without a timeout? If you refer to other implementations in kernel, the following modifications may be better. The important thing is to call kthread_should_stop() after se

Re: [Cluster-devel] [PATCH 08/17] dlm: retry rcom when dlm_wait_function is timed out.

2017-08-17 Thread tsutomu.owa
Hi, thank you for your review. > I'm curious to know more about this case. Could you describe a little > more about what happens? Errors (sk_err=104) occurred in fs/dlm/lowcomms.c::lowcomms_error_report(). In this case, DLM_RCOM_STATUS packet was lost at receiving side, as a result no DLM_RCOM_S

Re: [Cluster-devel] [PATCH 02/17] DLM: Eliminate CF_WRITE_PENDING flag

2017-08-17 Thread tsutomu.owa
Hi, thank you for your review. >> +cond_resched(); >> +queue_work(send_workqueue, &con->swork); > I think it would make more sense to call cond_resched() after the > queue_work() since we want the queued work to run soon after it has been > queued I think, Well, we're fine with that order

Re: [Cluster-devel] [PATCH 00/17] DLM: dlm patches need review

2017-08-09 Thread tsutomu.owa
Hi, Thanks a lot for your quick response and comment. We hope to make corrections better and incorporate them into mainline by proceeding with the review. The primary test which we've been using for those reported cases is to mount and umount from multiple nodes at the same time. i.e.; mkfs.oc

[Cluster-devel] [PATCH 16/17] dlm: fix to use sk_callback_lock correctly

2017-08-08 Thread tsutomu.owa
Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/lowcomms.c | 29 + 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcomms.c index 3e7c096..f4ba4a8 100644 --- a/fs/dlm/lowcomms.c +++ b/fs/dlm/lowcomms.c

[Cluster-devel] [PATCH 15/17] dlm: fix overflow dlm_cb_seq

2017-08-08 Thread tsutomu.owa
dlm_cb_seq is 64 bits. If dlm_cb_seq overflows and returns to 0, dlm_rem_lkb_callback() will not work properly. Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/ast.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/dlm/ast.c b/fs/dlm/ast.c index 07fed83..562fa8c 10

[Cluster-devel] [PATCH 17/17] dlm: fix to reschedule rwork

2017-08-08 Thread tsutomu.owa
When an error occurs in kernel_recvmsg or kernel_sendpage and close_connection is called and receive work is already scheduled, receive work is canceled. In that case, the receive work will not be scheduled forever after reconnection, because CF_READ_PENDING flag is established. Signed-off-by: Tad

[Cluster-devel] [PATCH 14/17] dlm: fix memory leak in tcp_accept_from_sock()

2017-08-08 Thread tsutomu.owa
The sk member of the socket generated by sock_create_kern() is overwritten by ops->accept(). So the previous sk will not be released. We use kernel_accept() instead of sock_create_kern() and ops->accept(). Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/lowcomms.c | 21 +++

[Cluster-devel] [PATCH 13/17] dlm: fix _can_be_granted() for lock at the head of covert queue.

2017-08-08 Thread tsutomu.owa
If there is a lock resource conflict on multiple nodes, the lock on convert queue may not be granted forever. EX.) grant queue: node0 grmode NL / rqmode IV node1 grmode NL / rqmode IV convert queue: node2 grmode NL / rqmode EX node3 grmode PR / rqmode EX wait queue: node4 grm

[Cluster-devel] [PATCH 12/17] dlm: use CF_CLOSE flag to stop dlm_send correctly

2017-08-08 Thread tsutomu.owa
If reconnection fails while executing dlm_lowcomms_stop, dlm_send will not stop. Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/lowcomms.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcomms.c index 438fa92..74ed41e 100644 --- a/fs/dlm/

[Cluster-devel] [PATCH 09/17] dlm: close othercon at send/receive error

2017-08-08 Thread tsutomu.owa
Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/lowcomms.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcomms.c index 4b53c32..76c6cdd 100644 --- a/fs/dlm/lowcomms.c +++ b/fs/dlm/lowcomms.c @@ -703,7 +703,7 @@ static

[Cluster-devel] [PATCH 10/17] dlm: use schedule_timeout instead of schedule in dlm_recoverd

2017-08-08 Thread tsutomu.owa
When dlm_recoverd_stop() is called between kthread_should_stop() and set_task_state(), dlm_recoverd will not wake up. Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/recoverd.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/dlm/recoverd.c b/fs/dlm

[Cluster-devel] [PATCH 11/17] dlm: Reanimate CF_WRITE_PENDING flag

2017-08-08 Thread tsutomu.owa
CF_WRITE_PENDING flag has been reanimated to make dlm_send stop properly when running dlm_lowcomms_stop. Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/lowcomms.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcom

[Cluster-devel] [PATCH 08/17] dlm: retry rcom when dlm_wait_function is timed out.

2017-08-08 Thread tsutomu.owa
If a node sends a DLM_RCOM_STATUS command and an error occurs on the receiving side, the DLM_RCOM_STATUS_REPLY response may not be returned. We retransmitted the DLM_RCOM_STATUS command so that we do not wait for an infinite response. Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --

[Cluster-devel] [PATCH 05/17] dlm: fix double list_del()

2017-08-08 Thread tsutomu.owa
dlm_lowcomms_stop() was not functioning properly. Correctly, we have to wait until all processing is finished with send_workqueue and recv_workqueue. This problem causes the following issue. Scenario is 1. dlm_send thread: send_to_sock refers con->writequeue 2. main thread: dlm_lowcomms_st

[Cluster-devel] [PATCH 07/17] dlm: fix to use sock_mutex correctly in xxx_accept_from_sock

2017-08-08 Thread tsutomu.owa
Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/lowcomms.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcomms.c index aec3c59..4b53c32 100644 --- a/fs/dlm/lowcomms.c +++ b/fs/dlm/lowcomms.c @@ -801,16 +801,19 @@ static int tcp_acce

[Cluster-devel] [PATCH 06/17] dlm: fix race condition between dlm_send and dlm_recv

2017-08-08 Thread tsutomu.owa
When kernel_sendpage (in send_to_sock) and kernel_recvmsg (in receive_from_sock) return error, close_connection may works at the same time. At that time, they may wait for each other by cancel_work_sync. Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/lowcomms.c | 8 ++

[Cluster-devel] [PATCH 02/17] DLM: Eliminate CF_WRITE_PENDING flag

2017-08-08 Thread tsutomu.owa
From: Bob Peterson Before this patch the CF_WRITE_PENDING flag was used to indicate when writes to the socket were pending. This caused race conditions whereby one process set the bit and another cleared it. Instead, we just check to see if there's anything there to be sent. This makes the code m

[Cluster-devel] [PATCH 03/17] DLM: Fix saving of NULL callbacks

2017-08-08 Thread tsutomu.owa
From: Bob Peterson In a previous patch I noted that accept() often copies the struct sock (sk) which overwrites the sock callbacks. However, in testing we discovered that the dlm connection structures (con) are sometimes deleted and recreated as connections come and go, and since they're zeroed o

[Cluster-devel] [PATCH 04/17] dlm: fix remove save_cb argument from add_sock()

2017-08-08 Thread tsutomu.owa
To remove an unused argument Signed-off-by: Tadashi Miyauchi Signed-off-by: Tsutomu Owa --- fs/dlm/lowcomms.c | 16 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/fs/dlm/lowcomms.c b/fs/dlm/lowcomms.c index ede4522..be62cf1 100644 --- a/fs/dlm/lowcomms.c +++ b/fs

[Cluster-devel] [PATCH 01/17] DLM: Eliminate CF_CONNECT_PENDING flag

2017-08-08 Thread tsutomu.owa
From: Bob Peterson Before this patch, there was a flag in the con structure that was used to determine whether or not a connect was needed. The bit was set here and there, and cleared here and there, so it left some race conditions: the bit was set, work was queued, then the worker cleared the bi

[Cluster-devel] [PATCH 00/17] DLM: dlm patches need review

2017-08-08 Thread tsutomu.owa
Hi, This series of patches is to fix various bugs. This patch set is against the mainline kernel. I'd like reviewed to make sure those changes are fine. Patch number 01/02/03 were posted on this mailing list by Bob Peterson . Patch 03 is modified to correct a bug acquiring rwlock multiple time