Antw: [EXT] Re: Large Immediate and/or Unsolicted Data causes long delays on R2T responses
>>> The Lee-Man schrieb am 29.06.2020 um 20:59 in
Nachricht
<4792_1593457181_5EFA3A1C_4792_1489_1_4c70b62c-467c-4860-a951-663fb88158c7o@goog
egroups.com>:
> On Saturday, May 2, 2020 at 11:30:27 AM UTC-7, ajhutc...@gmail.com wrote:
>>
>> I am able to create a condition where the open-iscsi initiator fails to
>> respond to an R2T request if the immediate/unsolicited data support is
>> large ~128KB. I've seen instances where a delay on an R2T is only a few
>> seconds and other instances where no response is received in 180 seconds.
>>
>> If the host is doing a prefill operation with large writes that can be
>> completed with immediate data alone and a large write that requires an R2T
>> is sent, the open-iscsi initiator sometimes fails to respond to the
>> target's R2T.
>>
>> After inspecting the code, I am convinced it is caused by the lack of
>> fairness in the *libiscsi **iscsi_data_xmit* routine, which always
>> favors the sending a new command over responding to R2Ts.
>>
>> /**
>> * iscsi_data_xmit - xmit any command into the scheduled connection
>> * @conn: iscsi connection
>> *
>> * Notes:
>> * The function can return -EAGAIN in which case the caller must
>> * re-schedule it again later or recover. '0' return code means
>> * successful xmit.
>> **/
>> static int iscsi_data_xmit(struct iscsi_conn *conn)
>> {
>> ...
>> /*
>> * process mgmt pdus like nops before commands since we should
>> * only have one nop-out as a ping from us and targets should not
>> * overflow us with nop-ins
>> */
>> while (!list_empty(>mgmtqueue)) {
>> ...
>> /* process pending command queue */
>> while (!list_empty(>cmdqueue)) {
>> ...
>> while (!list_empty(>requeue)) {
>>
>>
>> Am I looking at this code correctly? I guess this order might be better
>> for parallelization at the target by getting more commands onboard before
>> responding to outstanding R2Ts. With immediate/unsolicited data enabled,
>> the overhead of transmitting a new commands if higher and probably
>> shouldn't come before responding to R2Ts.
>>
>>
>> Do you have NOPs enabled? If so, do you see this issue with them disabled?
> I seriously dislike and advise against NOPs. I've never seen them actually
> help anything.
They could keep an connection alive that is passing through some firewall when
there is no iSCSI I/O, maybe.
>
> Have you tried playing with this code, i.e. changing the order? Without
> looking deeply, are the R2Ts in the command queue and not in the requeue
> queue?
>
> What kind of load are you presenting to the server?
>
> What do you mean by "the immediate/unsolicited data support is large
> ~128KB"? What setting(s) did you change?
>
> --
> You received this message because you are subscribed to the Google Groups
> "open-iscsi" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to open-iscsi+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/open-iscsi/4c70b62c-467c-4860-a951-663fb881
> 58c7o%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to open-iscsi+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/open-iscsi/5EFADD5D02A100039CE2%40gwsmtp.uni-regensburg.de.
Re: Large Immediate and/or Unsolicted Data causes long delays on R2T responses
On Saturday, May 2, 2020 at 11:30:27 AM UTC-7, ajhutc...@gmail.com wrote:
>
> I am able to create a condition where the open-iscsi initiator fails to
> respond to an R2T request if the immediate/unsolicited data support is
> large ~128KB. I've seen instances where a delay on an R2T is only a few
> seconds and other instances where no response is received in 180 seconds.
>
> If the host is doing a prefill operation with large writes that can be
> completed with immediate data alone and a large write that requires an R2T
> is sent, the open-iscsi initiator sometimes fails to respond to the
> target's R2T.
>
> After inspecting the code, I am convinced it is caused by the lack of
> fairness in the *libiscsi **iscsi_data_xmit* routine, which always
> favors the sending a new command over responding to R2Ts.
>
> /**
> * iscsi_data_xmit - xmit any command into the scheduled connection
> * @conn: iscsi connection
> *
> * Notes:
> * The function can return -EAGAIN in which case the caller must
> * re-schedule it again later or recover. '0' return code means
> * successful xmit.
> **/
> static int iscsi_data_xmit(struct iscsi_conn *conn)
> {
> ...
> /*
> * process mgmt pdus like nops before commands since we should
> * only have one nop-out as a ping from us and targets should not
> * overflow us with nop-ins
> */
> while (!list_empty(>mgmtqueue)) {
> ...
> /* process pending command queue */
> while (!list_empty(>cmdqueue)) {
> ...
> while (!list_empty(>requeue)) {
>
>
> Am I looking at this code correctly? I guess this order might be better
> for parallelization at the target by getting more commands onboard before
> responding to outstanding R2Ts. With immediate/unsolicited data enabled,
> the overhead of transmitting a new commands if higher and probably
> shouldn't come before responding to R2Ts.
>
>
> Do you have NOPs enabled? If so, do you see this issue with them disabled?
I seriously dislike and advise against NOPs. I've never seen them actually
help anything.
Have you tried playing with this code, i.e. changing the order? Without
looking deeply, are the R2Ts in the command queue and not in the requeue
queue?
What kind of load are you presenting to the server?
What do you mean by "the immediate/unsolicited data support is large
~128KB"? What setting(s) did you change?
--
You received this message because you are subscribed to the Google Groups
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to open-iscsi+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/open-iscsi/4c70b62c-467c-4860-a951-663fb88158c7o%40googlegroups.com.
Large Immediate and/or Unsolicted Data causes long delays on R2T responses
I am able to create a condition where the open-iscsi initiator fails to
respond to an R2T request if the immediate/unsolicited data support is
large ~128KB. I've seen instances where a delay on an R2T is only a few
seconds and other instances where no response is received in 180 seconds.
If the host is doing a prefill operation with large writes that can be
completed with immediate data alone and a large write that requires an R2T
is sent, the open-iscsi initiator sometimes fails to respond to the
target's R2T.
After inspecting the code, I am convinced it is caused by the lack of
fairness in the *libiscsi **iscsi_data_xmit* routine, which always favors
the sending a new command over responding to R2Ts.
/**
* iscsi_data_xmit - xmit any command into the scheduled connection
* @conn: iscsi connection
*
* Notes:
* The function can return -EAGAIN in which case the caller must
* re-schedule it again later or recover. '0' return code means
* successful xmit.
**/
static int iscsi_data_xmit(struct iscsi_conn *conn)
{
...
/*
* process mgmt pdus like nops before commands since we should
* only have one nop-out as a ping from us and targets should not
* overflow us with nop-ins
*/
while (!list_empty(>mgmtqueue)) {
...
/* process pending command queue */
while (!list_empty(>cmdqueue)) {
...
while (!list_empty(>requeue)) {
Am I looking at this code correctly? I guess this order might be better
for parallelization at the target by getting more commands onboard before
responding to outstanding R2Ts. With immediate/unsolicited data enabled,
the overhead of transmitting a new commands if higher and probably
shouldn't come before responding to R2Ts.
--
You received this message because you are subscribed to the Google Groups
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to open-iscsi+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/open-iscsi/75d706c2-e331-45bf-b764-6aa77703a45a%40googlegroups.com.
