Re: Antw: Re: [PATCH] iscsi_tcp set SO_LINGER to abort connection for error handling

[Chris Leech]

On Fri, Mar 04, 2016 at 08:27:14AM +0100, Ulrich Windl wrote:
> Hi!
> 
> I always thought SO_LINGER only has an effect on connections that are
> (partially) closed only: So if there is some network outage on a TCP
> connection and the connection is still considered established, would
> it help?
> 
> Regards,
> Ulrich

It changes what happens during an active close. With SO_LINGER set,
close(2) will block until the connection is finalized or the l_linger
timeout is reached and the connection is aborted with a RST. The
important detail is that once close(2) returns, the (re)transmit queue
is purged, rather than being retried in the background.

- Chris

> 
> >>> Chris Leech  schrieb am 04.03.2016 um 06:00 in 
> >>> Nachricht
> <20160304050001.cohjmdvxyopzd...@straylight.hirudinean.org>:
> > On Thu, Mar 03, 2016 at 07:09:14PM -0600, Mike Christie wrote:
> >> On 03/03/2016 06:09 PM, Chris Leech wrote:
> >> > When requests are being failed it's important to abort the TCP
> >> > connection rather than let TCP wait and attempt a graceful shutdown.
> >> > 
> >> > That can be accomplished by setting the SO_LINGER socket option with a
> >> > linger time of 0 to drop queued data and close the connection with a RST
> >> > instead of a FIN.
> >> > 
> >> > Signed-off-by: Chris Leech 
> >> > ---
> >> >  usr/io.c | 15 +++
> >> >  1 file changed, 15 insertions(+)
> >> > 
> >> > diff --git a/usr/io.c b/usr/io.c
> >> > index f552e1e..48b233c 100644
> >> > --- a/usr/io.c
> >> > +++ b/usr/io.c
> >> > @@ -391,9 +391,24 @@ iscsi_io_tcp_poll(iscsi_conn_t *conn, int 
> >> > timeout_ms)
> >> >  void
> >> >  iscsi_io_tcp_disconnect(iscsi_conn_t *conn)
> >> >  {
> >> > +struct linger so_linger = { .l_onoff = 1, .l_linger = 0 };
> >> > +
> >> >  if (conn->socket_fd >= 0) {
> >> >  log_debug(1, "disconnecting conn %p, fd %d", conn,
> >> >   conn->socket_fd);
> >> > +
> >> > +/* If the state is not IN_LOGOUT, this isn't a clean 
> >> > shutdown
> >> > + * and there's some sort of error handling going on. In 
> >> > that
> >> > + * case, set a 0 SO_LINGER to force an abortive close 
> >> > (RST) and
> >> > + * free whatever is sitting in the TCP transmit queue. 
> >> > This is
> >> > + * done to prevent stale data from being sent should the
> >> > + * network connection be restored before TCP times out.
> >> > + */
> >> > +if (conn->state != ISCSI_CONN_STATE_IN_LOGOUT) {
> >> > +setsockopt(conn->socket_fd, SOL_SOCKET, 
> >> > SO_LINGER,
> >> > +   _linger, sizeof(so_linger));
> >> > +}
> >> > +
> >> >  close(conn->socket_fd);
> >> >  conn->socket_fd = -1;
> >> >  }
> >> > 
> >> 
> >> Nice.
> >> 
> >> For maybe a slightly different problem, but hoping I get lucky and your
> >> patch fixes it too, I thought the network layer was still accessing
> >> pages that we tried to send and was causing a oops. I get the part where
> >> with your patch the network layer will not try to send data anymore, but
> >> I guess I am asking if the network layer could still be doing some sort
> >> of delayed cleanup process after close() has returned?
> > 
> > From what I can tell in the tcp code, the zero linger handling purges
> > all of the socket queues freeing everything before close returns.
> > 
> > - Chris

-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Antw: Re: [PATCH] iscsi_tcp set SO_LINGER to abort connection for error handling

[Ulrich Windl]

Hi!

I always thought SO_LINGER only has an effect on connections that are 
(partially) closed only: So if there is some network outage on a TCP connection 
and the connection is still considered established, would it help?

Regards,
Ulrich

>>> Chris Leech  schrieb am 04.03.2016 um 06:00 in Nachricht
<20160304050001.cohjmdvxyopzd...@straylight.hirudinean.org>:
> On Thu, Mar 03, 2016 at 07:09:14PM -0600, Mike Christie wrote:
>> On 03/03/2016 06:09 PM, Chris Leech wrote:
>> > When requests are being failed it's important to abort the TCP
>> > connection rather than let TCP wait and attempt a graceful shutdown.
>> > 
>> > That can be accomplished by setting the SO_LINGER socket option with a
>> > linger time of 0 to drop queued data and close the connection with a RST
>> > instead of a FIN.
>> > 
>> > Signed-off-by: Chris Leech 
>> > ---
>> >  usr/io.c | 15 +++
>> >  1 file changed, 15 insertions(+)
>> > 
>> > diff --git a/usr/io.c b/usr/io.c
>> > index f552e1e..48b233c 100644
>> > --- a/usr/io.c
>> > +++ b/usr/io.c
>> > @@ -391,9 +391,24 @@ iscsi_io_tcp_poll(iscsi_conn_t *conn, int timeout_ms)
>> >  void
>> >  iscsi_io_tcp_disconnect(iscsi_conn_t *conn)
>> >  {
>> > +  struct linger so_linger = { .l_onoff = 1, .l_linger = 0 };
>> > +
>> >if (conn->socket_fd >= 0) {
>> >log_debug(1, "disconnecting conn %p, fd %d", conn,
>> > conn->socket_fd);
>> > +
>> > +  /* If the state is not IN_LOGOUT, this isn't a clean shutdown
>> > +   * and there's some sort of error handling going on. In that
>> > +   * case, set a 0 SO_LINGER to force an abortive close (RST) and
>> > +   * free whatever is sitting in the TCP transmit queue. This is
>> > +   * done to prevent stale data from being sent should the
>> > +   * network connection be restored before TCP times out.
>> > +   */
>> > +  if (conn->state != ISCSI_CONN_STATE_IN_LOGOUT) {
>> > +  setsockopt(conn->socket_fd, SOL_SOCKET, SO_LINGER,
>> > + _linger, sizeof(so_linger));
>> > +  }
>> > +
>> >close(conn->socket_fd);
>> >conn->socket_fd = -1;
>> >}
>> > 
>> 
>> Nice.
>> 
>> For maybe a slightly different problem, but hoping I get lucky and your
>> patch fixes it too, I thought the network layer was still accessing
>> pages that we tried to send and was causing a oops. I get the part where
>> with your patch the network layer will not try to send data anymore, but
>> I guess I am asking if the network layer could still be doing some sort
>> of delayed cleanup process after close() has returned?
> 
> From what I can tell in the tcp code, the zero linger handling purges
> all of the socket queues freeing everything before close returns.
> 
> - Chris
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "open-iscsi" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to open-iscsi+unsubscr...@googlegroups.com.
> To post to this group, send email to open-iscsi@googlegroups.com.
> Visit this group at https://groups.google.com/group/open-iscsi.
> For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Re: [PATCH] iscsi_tcp set SO_LINGER to abort connection for error handling

[Chris Leech]

On Thu, Mar 03, 2016 at 07:09:14PM -0600, Mike Christie wrote:
> On 03/03/2016 06:09 PM, Chris Leech wrote:
> > When requests are being failed it's important to abort the TCP
> > connection rather than let TCP wait and attempt a graceful shutdown.
> > 
> > That can be accomplished by setting the SO_LINGER socket option with a
> > linger time of 0 to drop queued data and close the connection with a RST
> > instead of a FIN.
> > 
> > Signed-off-by: Chris Leech 
> > ---
> >  usr/io.c | 15 +++
> >  1 file changed, 15 insertions(+)
> > 
> > diff --git a/usr/io.c b/usr/io.c
> > index f552e1e..48b233c 100644
> > --- a/usr/io.c
> > +++ b/usr/io.c
> > @@ -391,9 +391,24 @@ iscsi_io_tcp_poll(iscsi_conn_t *conn, int timeout_ms)
> >  void
> >  iscsi_io_tcp_disconnect(iscsi_conn_t *conn)
> >  {
> > +   struct linger so_linger = { .l_onoff = 1, .l_linger = 0 };
> > +
> > if (conn->socket_fd >= 0) {
> > log_debug(1, "disconnecting conn %p, fd %d", conn,
> >  conn->socket_fd);
> > +
> > +   /* If the state is not IN_LOGOUT, this isn't a clean shutdown
> > +* and there's some sort of error handling going on. In that
> > +* case, set a 0 SO_LINGER to force an abortive close (RST) and
> > +* free whatever is sitting in the TCP transmit queue. This is
> > +* done to prevent stale data from being sent should the
> > +* network connection be restored before TCP times out.
> > +*/
> > +   if (conn->state != ISCSI_CONN_STATE_IN_LOGOUT) {
> > +   setsockopt(conn->socket_fd, SOL_SOCKET, SO_LINGER,
> > +  _linger, sizeof(so_linger));
> > +   }
> > +
> > close(conn->socket_fd);
> > conn->socket_fd = -1;
> > }
> > 
> 
> Nice.
> 
> For maybe a slightly different problem, but hoping I get lucky and your
> patch fixes it too, I thought the network layer was still accessing
> pages that we tried to send and was causing a oops. I get the part where
> with your patch the network layer will not try to send data anymore, but
> I guess I am asking if the network layer could still be doing some sort
> of delayed cleanup process after close() has returned?

>From what I can tell in the tcp code, the zero linger handling purges
all of the socket queues freeing everything before close returns.

- Chris

-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Re: [PATCH] iscsi_tcp set SO_LINGER to abort connection for error handling

[Mike Christie]

On 03/03/2016 06:09 PM, Chris Leech wrote:
> When requests are being failed it's important to abort the TCP
> connection rather than let TCP wait and attempt a graceful shutdown.
> 
> That can be accomplished by setting the SO_LINGER socket option with a
> linger time of 0 to drop queued data and close the connection with a RST
> instead of a FIN.
> 
> Signed-off-by: Chris Leech 
> ---
>  usr/io.c | 15 +++
>  1 file changed, 15 insertions(+)
> 
> diff --git a/usr/io.c b/usr/io.c
> index f552e1e..48b233c 100644
> --- a/usr/io.c
> +++ b/usr/io.c
> @@ -391,9 +391,24 @@ iscsi_io_tcp_poll(iscsi_conn_t *conn, int timeout_ms)
>  void
>  iscsi_io_tcp_disconnect(iscsi_conn_t *conn)
>  {
> + struct linger so_linger = { .l_onoff = 1, .l_linger = 0 };
> +
>   if (conn->socket_fd >= 0) {
>   log_debug(1, "disconnecting conn %p, fd %d", conn,
>conn->socket_fd);
> +
> + /* If the state is not IN_LOGOUT, this isn't a clean shutdown
> +  * and there's some sort of error handling going on. In that
> +  * case, set a 0 SO_LINGER to force an abortive close (RST) and
> +  * free whatever is sitting in the TCP transmit queue. This is
> +  * done to prevent stale data from being sent should the
> +  * network connection be restored before TCP times out.
> +  */
> + if (conn->state != ISCSI_CONN_STATE_IN_LOGOUT) {
> + setsockopt(conn->socket_fd, SOL_SOCKET, SO_LINGER,
> +_linger, sizeof(so_linger));
> + }
> +
>   close(conn->socket_fd);
>   conn->socket_fd = -1;
>   }
> 

Nice.

For maybe a slightly different problem, but hoping I get lucky and your
patch fixes it too, I thought the network layer was still accessing
pages that we tried to send and was causing a oops. I get the part where
with your patch the network layer will not try to send data anymore, but
I guess I am asking if the network layer could still be doing some sort
of delayed cleanup process after close() has returned?

-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

[PATCH] iscsi_tcp set SO_LINGER to abort connection for error handling

[Chris Leech]

When requests are being failed it's important to abort the TCP
connection rather than let TCP wait and attempt a graceful shutdown.

That can be accomplished by setting the SO_LINGER socket option with a
linger time of 0 to drop queued data and close the connection with a RST
instead of a FIN.

Signed-off-by: Chris Leech 
---
 usr/io.c | 15 +++
 1 file changed, 15 insertions(+)

diff --git a/usr/io.c b/usr/io.c
index f552e1e..48b233c 100644
--- a/usr/io.c
+++ b/usr/io.c
@@ -391,9 +391,24 @@ iscsi_io_tcp_poll(iscsi_conn_t *conn, int timeout_ms)
 void
 iscsi_io_tcp_disconnect(iscsi_conn_t *conn)
 {
+   struct linger so_linger = { .l_onoff = 1, .l_linger = 0 };
+
if (conn->socket_fd >= 0) {
log_debug(1, "disconnecting conn %p, fd %d", conn,
 conn->socket_fd);
+
+   /* If the state is not IN_LOGOUT, this isn't a clean shutdown
+* and there's some sort of error handling going on. In that
+* case, set a 0 SO_LINGER to force an abortive close (RST) and
+* free whatever is sitting in the TCP transmit queue. This is
+* done to prevent stale data from being sent should the
+* network connection be restored before TCP times out.
+*/
+   if (conn->state != ISCSI_CONN_STATE_IN_LOGOUT) {
+   setsockopt(conn->socket_fd, SOL_SOCKET, SO_LINGER,
+  _linger, sizeof(so_linger));
+   }
+
close(conn->socket_fd);
conn->socket_fd = -1;
}
-- 
2.5.0

-- 
You received this message because you are subscribed to the Google Groups 
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.