Re: [PATCH 0/2] Two refactoring patches for the qla2xxx driver

[Laurence Oberman]

On Tue, 2018-11-27 at 15:04 -0800, Bart Van Assche wrote:
> Hi Martin,
> 
> The two patches in this series make the qla2xxx driver source code
> easier
> to read without changing the driver functionality. Please consider
> these
> patches for kernel v4.21.
> 
> Thanks,
> 
> Bart.
> 
> Bart Van Assche (2):
>   qla2xxx: Introduce a switch/case statement in qlt_xmit_tm_rsp()
>   qla2xxx: Split the __qla2x00_abort_all_cmds() function
> 
>  drivers/scsi/qla2xxx/qla_os.c | 89 +++
> 
>  drivers/scsi/qla2xxx/qla_target.c | 14 ++---
>  2 files changed, 51 insertions(+), 52 deletions(-)
> 

These look fine to me and the switch change makes it much cleaner.
Reviewed-by: Laurence Oberman 

Re: [PATCH V2 RESEND] Timeouts occur on surprise removal of QLogic adapter

[Laurence Oberman]

On Mon, 2018-11-05 at 11:32 -0500, Laurence Oberman wrote:
> On Mon, 2018-11-05 at 11:23 -0500, Bill Kuzeja wrote:
> > When doing a surprise removal of an adapter, some in flight I/Os
> > can
> > get 
> > stuck and take a while to complete (they actually timeout and are 
> > retried). We are not handling an early error exit from 
> > qla2xxx_eh_abort properly.
> > 
> > Fixes: 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting
> > down chip") 
> > Signed-off-by: Bill Kuzeja 
> > 
> > ---
> > Resending...can somebody review this (pretty please)??
> > 
> > After a hot remove of a Qlogic adapter, the driver's remove
> > function
> > gets
> > called and we end up aborting all in progress I/Os. Here is the
> > code
> > flow:
> > 
> > qla2x00_remove_one
> >   qla2x00_abort_isp_cleanup
> > qla2x00_abort_all_cmds
> >   __qla2x00_abort_all_cmds
> > qla2xxx_eh_abort
> > 
> > At the start of qla2xxx_eh_abort, some sanity checks are done
> > before
> > actually sending the abort. One of these checks is a call to
> > fc_block_scsi_eh. In the case of a hot remove, it turns out that
> > this
> > routine can exit with FAST_IO_FAIL.
> > 
> > When this occurs, we return back to __qla2x00_abort_all_cmds with
> > an
> > extra reference on sp (because the abort never gets sent).
> > Originally,
> > this was addressed with another fix:
> > 
> > commit 4cd3b6ebff85 scsi: qla2xxx: Fix extraneous ref on sp's after
> > adapter break
> > 
> > But this later this added change complicated matters:
> > 
> > commit 45235022da99 scsi: qla2xxx: Fix driver unload by shutting
> > down
> > chip
> > 
> > Because the abort is now being done earlier in the teardown
> > (through
> > qla2x00_abort_isp_cleanup), in qla2xxx_eh_abort we make it past
> > the first check because qla2x00_isp_reg_stat(ha) returns zero. When
> > we
> > fail a few lines later in fc_block_scsi_eh, this error is not
> > handled
> > properly in __qla2x00_abort_all_cmds and the I/O ends up hanging
> > and
> > timing out because of the extra reference.
> > 
> > For this fix, a check for FAST_IO_FAIL is added to
> > __qla2x00_abort_all_cmds where we check to see if qla2xxx_eh_abort
> > succeeded or not.
> > 
> > This removes the extra reference in this additional early exit
> > case.
> > In
> > my testing (hw surprise removals and also adapter remove via
> > sysfs),
> > this
> > eliminates the timeouts and delays and the remove proceeds
> > smoothly.
> > 
> > ---
> >  drivers/scsi/qla2xxx/qla_os.c | 10 --
> >  1 file changed, 8 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/scsi/qla2xxx/qla_os.c
> > b/drivers/scsi/qla2xxx/qla_os.c
> > index 518f151..5261adf 100644
> > --- a/drivers/scsi/qla2xxx/qla_os.c
> > +++ b/drivers/scsi/qla2xxx/qla_os.c
> > @@ -1749,7 +1749,7 @@ uint32_t qla2x00_isp_reg_stat(struct
> > qla_hw_data *ha)
> >  static void
> >  __qla2x00_abort_all_cmds(struct qla_qpair *qp, int res)
> >  {
> > -   int cnt;
> > +   int cnt, status;
> >     unsigned long flags;
> >     srb_t *sp;
> >     scsi_qla_host_t *vha = qp->vha;
> > @@ -1799,10 +1799,16 @@ uint32_t qla2x00_isp_reg_stat(struct
> > qla_hw_data *ha)
> >     if (!sp_get(sp)) {
> >     spin_unlock_irqres
> > to
> > re
> >     (qp-
> > > qp_lock_ptr, flags);
> > 
> > -   qla2xxx_eh_abort(
> > +   status =
> > qla2xxx_eh_abort(
> >     GET_CMD_SP
> > (s
> > p));
> >     spin_lock_irqsave
> >     (qp-
> > > qp_lock_ptr, flags);
> > 
> > +   /*
> > +    * Get rid of
> > extra
> > reference caused
> > +    * by early exit
> > from qla2xxx_eh_abort
> > +    */
> > +   if (status ==
> > FAST_IO_FAIL)
> > +   atomic_dec
> > (&
> > sp->ref_count);
> >     }
> >     }
> >     sp->done(sp, res);
> 
> Bill 
> The patch looks fine to me but I still cant apply it
> I tried against linux-next which is 4.20
> What ree are you applying it against now
> 
> Regards
> Laurence
> 
I has a note from Bill saying it applies cleanly now to stable.

In that case
Reviewed-by: Laurence Oberman 

Re: [PATCH V2 RESEND] Timeouts occur on surprise removal of QLogic adapter

[Laurence Oberman]

On Mon, 2018-11-05 at 11:23 -0500, Bill Kuzeja wrote:
> When doing a surprise removal of an adapter, some in flight I/Os can
> get 
> stuck and take a while to complete (they actually timeout and are 
> retried). We are not handling an early error exit from 
> qla2xxx_eh_abort properly.
> 
> Fixes: 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting
> down chip") 
> Signed-off-by: Bill Kuzeja 
> 
> ---
> Resending...can somebody review this (pretty please)??
> 
> After a hot remove of a Qlogic adapter, the driver's remove function
> gets
> called and we end up aborting all in progress I/Os. Here is the code
> flow:
> 
> qla2x00_remove_one
>   qla2x00_abort_isp_cleanup
> qla2x00_abort_all_cmds
>   __qla2x00_abort_all_cmds
> qla2xxx_eh_abort
> 
> At the start of qla2xxx_eh_abort, some sanity checks are done before
> actually sending the abort. One of these checks is a call to
> fc_block_scsi_eh. In the case of a hot remove, it turns out that this
> routine can exit with FAST_IO_FAIL.
> 
> When this occurs, we return back to __qla2x00_abort_all_cmds with an
> extra reference on sp (because the abort never gets sent).
> Originally,
> this was addressed with another fix:
> 
> commit 4cd3b6ebff85 scsi: qla2xxx: Fix extraneous ref on sp's after
> adapter break
> 
> But this later this added change complicated matters:
> 
> commit 45235022da99 scsi: qla2xxx: Fix driver unload by shutting down
> chip
> 
> Because the abort is now being done earlier in the teardown (through
> qla2x00_abort_isp_cleanup), in qla2xxx_eh_abort we make it past
> the first check because qla2x00_isp_reg_stat(ha) returns zero. When
> we
> fail a few lines later in fc_block_scsi_eh, this error is not handled
> properly in __qla2x00_abort_all_cmds and the I/O ends up hanging and
> timing out because of the extra reference.
> 
> For this fix, a check for FAST_IO_FAIL is added to
> __qla2x00_abort_all_cmds where we check to see if qla2xxx_eh_abort
> succeeded or not.
> 
> This removes the extra reference in this additional early exit case.
> In
> my testing (hw surprise removals and also adapter remove via sysfs),
> this
> eliminates the timeouts and delays and the remove proceeds smoothly.
> 
> ---
>  drivers/scsi/qla2xxx/qla_os.c | 10 --
>  1 file changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/scsi/qla2xxx/qla_os.c
> b/drivers/scsi/qla2xxx/qla_os.c
> index 518f151..5261adf 100644
> --- a/drivers/scsi/qla2xxx/qla_os.c
> +++ b/drivers/scsi/qla2xxx/qla_os.c
> @@ -1749,7 +1749,7 @@ uint32_t qla2x00_isp_reg_stat(struct
> qla_hw_data *ha)
>  static void
>  __qla2x00_abort_all_cmds(struct qla_qpair *qp, int res)
>  {
> - int cnt;
> + int cnt, status;
>   unsigned long flags;
>   srb_t *sp;
>   scsi_qla_host_t *vha = qp->vha;
> @@ -1799,10 +1799,16 @@ uint32_t qla2x00_isp_reg_stat(struct
> qla_hw_data *ha)
>   if (!sp_get(sp)) {
>   spin_unlock_irqresto
> re
>   (qp-
> >qp_lock_ptr, flags);
> - qla2xxx_eh_abort(
> + status =
> qla2xxx_eh_abort(
>   GET_CMD_SP(s
> p));
>   spin_lock_irqsave
>   (qp-
> >qp_lock_ptr, flags);
> + /*
> +  * Get rid of extra
> reference caused
> +  * by early exit
> from qla2xxx_eh_abort
> +  */
> + if (status ==
> FAST_IO_FAIL)
> + atomic_dec(&
> sp->ref_count);
>   }
>   }
>   sp->done(sp, res);

Bill 
The patch looks fine to me but I still cant apply it
I tried against linux-next which is 4.20
What ree are you applying it against now

Regards
Laurence

Re: [PATCH 0/3] sd: Rely on the driver core for asynchronous probing

[Laurence Oberman]

On Tue, 2018-10-02 at 14:52 -0700, Bart Van Assche wrote:
> Hello Martin,
> 
> During the 2018 edition of LSF/MM there was a session about
> increasing SCSI
> disk probing concurrency. This patch series implements what has been
> proposed
> during that session, namely:
> - Make sure that the driver core is aware of asynchronous SCSI LUN
> probing.
> - Avoid unnecessary serialization between sd_probe() and sd_remove()
> because
>   this could lead to a deadlock.
> 
> Please consider this patch series for kernel v4.20.
> 
> Thanks,
> 
> Bart.
> 
> Bart Van Assche (3):
>   sd: Rely on the driver core for asynchronous probing
>   sd: Inline sd_probe_part2()
>   __device_release_driver(): Do not wait for asynchronous probing
> 
>  drivers/base/dd.c|   3 --
>  drivers/scsi/scsi.c  |  14 -
>  drivers/scsi/scsi_pm.c   |  22 +---
>  drivers/scsi/scsi_priv.h |   3 --
>  drivers/scsi/sd.c| 110 -
> --
>  5 files changed, 46 insertions(+), 106 deletions(-)
> 

I ran multiple tests for two days using the series from Bart's tree on
the SRP test bed here.

Multiple disconnects/reconnects during heavy IO activity on 32 SRP 64
LUNS/ 32 mpaths with no issues seen with probes and reconnects.

For the series.

Tested-by: Laurence Oberman 

Re: SCSI qla2xxx: tcm_qla2xxx target server code regession

[Laurence Oberman]

On Wed, 2018-10-17 at 16:42 -0400, Laurence Oberman wrote:
> On Mon, 2018-10-15 at 08:51 -0400, Laurence Oberman wrote:
> > On Sat, 2018-10-13 at 10:42 -0400, Laurence Oberman wrote:
> > > On Fri, 2018-10-12 at 17:51 -0700, Bart Van Assche wrote:
> > > > On 10/12/18 1:36 PM, Laurence Oberman wrote:
> > > >  > While I have for the longest time used 4.5 as a base for my
> > > > F/C
> > > > jammer
> > > >  > that I use every day here in our lab I recently added more
> > > > jammer
> > > > code
> > > >  > so I decided to test this all on latest upstream.
> > > >  >
> > > >  > Booting the target server on my 4.5 kernel with jammer code
> > > > is
> > > >  > flawless and serves LUNS with no issues and handles the
> > > > jamming
> > > > also
> > > >  > fine.
> > > >  >
> > > >  > However just building a 4.19.0_rc7+-1 (I left the jammer
> > > > stuff
> > > > out)
> > > >  > its pretty broken.
> > > > 
> > > > A large number of patches went upstream between these two
> > > > kernel 
> > > > versions for both the QLogic initiator and target drivers. From
> > > > the
> > > > logs 
> > > > it seems like you were using QLogic hardware at both the
> > > > initiator
> > > > and 
> > > > target side? If so, which kernel version was running at the
> > > > initiator 
> > > > side during these tests? 4.5, 4.19-rc7+ or yet another version?
> > > > 
> > > > Thanks,
> > > > 
> > > > Bart.
> > > > 
> > > > 
> > > 
> > > I had only replied to Bart, this was my reply, reply all now
> > > 
> > > Hi Bart
> > > Thank you for always being helpful.
> > > 
> > > I am using at the moment RHEL 7.5 for the initiator (based on
> > > kernel
> > > 3.10 but of course lots of backports)
> > > The exact same initiator is working fine with the 4.5 and I would
> > > not
> > > expect the target to  require the same kernel level.
> > > Of course I will try latest upstream on the initiator later and
> > > reply
> > > back.
> > > 
> > > I was thinking the target should adhere to the standards and
> > > support
> > > many types of kernels within reason for the initiator.
> > > 
> > > Thanks
> > 
> > Changed the Subject to match regression
> > 
> > I tested the following:
> > Target 
> > upstream 4.19_rc4
> > 
> > Inititiator
> > qla2xxx RHEL7.5 
> > lpfc RHEL7.5
> > qla2xxx Upstream 4.19_rc4 matching initiatior
> > 
> > All 3 are unstable and fail, with tag errors and aborts
> > 
> > I am not sure when the issue started so will work on a bisect
> > 
> > Oct 15 07:29:52 ml150 kernel: print_req_error: I/O error, dev sdj,
> > sector 128
> > Oct 15 07:29:52 ml150 kernel: print_req_error: I/O error, dev sds,
> > sector 16
> > ..
> > ..
> > Oct 15 07:30:11 ml150 kernel: sd 5:0:0:0: [sdi] tag#0 FAILED
> > Result:
> > hostbyte=DID_ERROR driverbyte=DRIVER_OK
> > Oct 15 07:30:11 ml150 kernel: sd 5:0:0:0: [sdi] tag#0 CDB: Read(10)
> > 28
> > 00 00 00 03 00 00 01 00 00
> > Oct 15 07:30:16 ml150 kernel: sd 5:0:0:0: [sdi] tag#1 FAILED
> > Result:
> > hostbyte=DID_ERROR driverbyte=DRIVER_OK
> > Oct 15 07:30:16 ml150 kernel: sd 5:0:0:0: [sdi] tag#1 CDB: Read(10)
> > 28
> > 00 00 00 00 40 00 00 38 00
> > Oct 15 07:30:17 ml150 kernel: sd 5:0:1:0: [sdp] tag#0 FAILED
> > Result: 
> 
> Following up here
> 
> I started with 4.9 and it was stable
> 
> I then tested 4.10 and its immediately broken and logs these when
> restoring the target configuration.
> This is a different failure to the upstream latest which takes the
> configuration but fails the I/O servicing.
> 
> This is logged on the target when restoring the target configuration
> for 4.10
> 
> Perhaps in this case its a target config mismatch. I did not manually
> add targets on 4.10 but I can certainly try it.
> Note that 4.18 for example takes the config with no complaints so I
> doubt its a mismatch.
> 
> [   95.668210] qla2xxx [:07:00.0]-5030:0: Error entry - invalid
> handle/queue (1c01).
> [   95.706773] qla2xxx [:07:00.0]-5030:0: Error entry - invalid
> handle/queue (0002).
> [   95.745912] qla2xxx [:07:00.0]-5030:0: Er

Re: SCSI qla2xxx: tcm_qla2xxx target server code regession

[Laurence Oberman]

On Mon, 2018-10-15 at 08:51 -0400, Laurence Oberman wrote:
> On Sat, 2018-10-13 at 10:42 -0400, Laurence Oberman wrote:
> > On Fri, 2018-10-12 at 17:51 -0700, Bart Van Assche wrote:
> > > On 10/12/18 1:36 PM, Laurence Oberman wrote:
> > >  > While I have for the longest time used 4.5 as a base for my
> > > F/C
> > > jammer
> > >  > that I use every day here in our lab I recently added more
> > > jammer
> > > code
> > >  > so I decided to test this all on latest upstream.
> > >  >
> > >  > Booting the target server on my 4.5 kernel with jammer code is
> > >  > flawless and serves LUNS with no issues and handles the
> > > jamming
> > > also
> > >  > fine.
> > >  >
> > >  > However just building a 4.19.0_rc7+-1 (I left the jammer stuff
> > > out)
> > >  > its pretty broken.
> > > 
> > > A large number of patches went upstream between these two kernel 
> > > versions for both the QLogic initiator and target drivers. From
> > > the
> > > logs 
> > > it seems like you were using QLogic hardware at both the
> > > initiator
> > > and 
> > > target side? If so, which kernel version was running at the
> > > initiator 
> > > side during these tests? 4.5, 4.19-rc7+ or yet another version?
> > > 
> > > Thanks,
> > > 
> > > Bart.
> > > 
> > > 
> > 
> > I had only replied to Bart, this was my reply, reply all now
> > 
> > Hi Bart
> > Thank you for always being helpful.
> > 
> > I am using at the moment RHEL 7.5 for the initiator (based on
> > kernel
> > 3.10 but of course lots of backports)
> > The exact same initiator is working fine with the 4.5 and I would
> > not
> > expect the target to  require the same kernel level.
> > Of course I will try latest upstream on the initiator later and
> > reply
> > back.
> > 
> > I was thinking the target should adhere to the standards and
> > support
> > many types of kernels within reason for the initiator.
> > 
> > Thanks
> 
> Changed the Subject to match regression
> 
> I tested the following:
> Target 
> upstream 4.19_rc4
> 
> Inititiator
> qla2xxx RHEL7.5 
> lpfc RHEL7.5
> qla2xxx Upstream 4.19_rc4 matching initiatior
> 
> All 3 are unstable and fail, with tag errors and aborts
> 
> I am not sure when the issue started so will work on a bisect
> 
> Oct 15 07:29:52 ml150 kernel: print_req_error: I/O error, dev sdj,
> sector 128
> Oct 15 07:29:52 ml150 kernel: print_req_error: I/O error, dev sds,
> sector 16
> ..
> ..
> Oct 15 07:30:11 ml150 kernel: sd 5:0:0:0: [sdi] tag#0 FAILED Result:
> hostbyte=DID_ERROR driverbyte=DRIVER_OK
> Oct 15 07:30:11 ml150 kernel: sd 5:0:0:0: [sdi] tag#0 CDB: Read(10)
> 28
> 00 00 00 03 00 00 01 00 00
> Oct 15 07:30:16 ml150 kernel: sd 5:0:0:0: [sdi] tag#1 FAILED Result:
> hostbyte=DID_ERROR driverbyte=DRIVER_OK
> Oct 15 07:30:16 ml150 kernel: sd 5:0:0:0: [sdi] tag#1 CDB: Read(10)
> 28
> 00 00 00 00 40 00 00 38 00
> Oct 15 07:30:17 ml150 kernel: sd 5:0:1:0: [sdp] tag#0 FAILED Result: 

Following up here

I started with 4.9 and it was stable

I then tested 4.10 and its immediately broken and logs these when
restoring the target configuration.
This is a different failure to the upstream latest which takes the
configuration but fails the I/O servicing.

This is logged on the target when restoring the target configuration
for 4.10

Perhaps in this case its a target config mismatch. I did not manually
add targets on 4.10 but I can certainly try it.
Note that 4.18 for example takes the config with no complaints so I
doubt its a mismatch.

[   95.668210] qla2xxx [:07:00.0]-5030:0: Error entry - invalid
handle/queue (1c01).
[   95.706773] qla2xxx [:07:00.0]-5030:0: Error entry - invalid
handle/queue (0002).
[   95.745912] qla2xxx [:07:00.0]-5030:0: Error entry - invalid
handle/queue (5838).

This comes from

/**
 * qla2x00_error_entry() - Process an error entry.
 * @ha: SCSI driver HA context
 * @pkt: Entry pointer
 */
static void
qla2x00_error_entry(scsi_qla_host_t *vha, struct rsp_que *rsp,
sts_entry_t *pkt)
{
srb_t *sp;
struct qla_hw_data *ha = vha->hw;
const char func[] = "ERROR-IOCB";
uint16_t que = MSW(pkt->handle);
struct req_que *req = NULL;
int res = DID_ERROR << 16;

ql_dbg(ql_dbg_async, vha, 0x502a,
"type of error status in response: 0x%x\n", pkt-
>entry_status);

if (que >= ha->max_req_queues || !ha->req_q_map[que])
goto fatal;

[PATCH] scsi_error: Remove scsi_block_when_processing_errors: message

[Laurence Oberman]

This message floods the log when enabling mask 0x7 for
 /proc/sys/dev/scsi/logging_level 
  kernel: scsi_block_when_processing_errors: rtn: 1 
Its not needed and makes tracing just scsi_eh* messages way too verbose
so get rid of it

Signed-off-by: Laurence Oberman 

---
 drivers/scsi/scsi_error.c | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/drivers/scsi/scsi_error.c b/drivers/scsi/scsi_error.c
index b7a8fdf..c736d61 100644
--- a/drivers/scsi/scsi_error.c
+++ b/drivers/scsi/scsi_error.c
@@ -338,9 +338,6 @@ int scsi_block_when_processing_errors(struct
scsi_device *sdev)
 
    online = scsi_device_online(sdev);
 
-   SCSI_LOG_ERROR_RECOVERY(5, sdev_printk(KERN_INFO, sdev,
-   "%s: rtn: %d\n", __func__, online));
-
    return online;
 }
 EXPORT_SYMBOL(scsi_block_when_processing_errors);
-- 
1.8.3.1

SCSI qla2xxx: tcm_qla2xxx target server code regession

[Laurence Oberman]

On Sat, 2018-10-13 at 10:42 -0400, Laurence Oberman wrote:
> On Fri, 2018-10-12 at 17:51 -0700, Bart Van Assche wrote:
> > On 10/12/18 1:36 PM, Laurence Oberman wrote:
> >  > While I have for the longest time used 4.5 as a base for my F/C
> > jammer
> >  > that I use every day here in our lab I recently added more
> > jammer
> > code
> >  > so I decided to test this all on latest upstream.
> >  >
> >  > Booting the target server on my 4.5 kernel with jammer code is
> >  > flawless and serves LUNS with no issues and handles the jamming
> > also
> >  > fine.
> >  >
> >  > However just building a 4.19.0_rc7+-1 (I left the jammer stuff
> > out)
> >  > its pretty broken.
> > 
> > A large number of patches went upstream between these two kernel 
> > versions for both the QLogic initiator and target drivers. From the
> > logs 
> > it seems like you were using QLogic hardware at both the initiator
> > and 
> > target side? If so, which kernel version was running at the
> > initiator 
> > side during these tests? 4.5, 4.19-rc7+ or yet another version?
> > 
> > Thanks,
> > 
> > Bart.
> > 
> > 
> 
> I had only replied to Bart, this was my reply, reply all now
> 
> Hi Bart
> Thank you for always being helpful.
> 
> I am using at the moment RHEL 7.5 for the initiator (based on kernel
> 3.10 but of course lots of backports)
> The exact same initiator is working fine with the 4.5 and I would not
> expect the target to  require the same kernel level.
> Of course I will try latest upstream on the initiator later and reply
> back.
> 
> I was thinking the target should adhere to the standards and support
> many types of kernels within reason for the initiator.
> 
> Thanks

Changed the Subject to match regression

I tested the following:
Target 
upstream 4.19_rc4

Inititiator
qla2xxx RHEL7.5 
lpfc RHEL7.5
qla2xxx Upstream 4.19_rc4 matching initiatior

All 3 are unstable and fail, with tag errors and aborts

I am not sure when the issue started so will work on a bisect

Oct 15 07:29:52 ml150 kernel: print_req_error: I/O error, dev sdj,
sector 128
Oct 15 07:29:52 ml150 kernel: print_req_error: I/O error, dev sds,
sector 16
..
..
Oct 15 07:30:11 ml150 kernel: sd 5:0:0:0: [sdi] tag#0 FAILED Result:
hostbyte=DID_ERROR driverbyte=DRIVER_OK
Oct 15 07:30:11 ml150 kernel: sd 5:0:0:0: [sdi] tag#0 CDB: Read(10) 28
00 00 00 03 00 00 01 00 00
Oct 15 07:30:16 ml150 kernel: sd 5:0:0:0: [sdi] tag#1 FAILED Result:
hostbyte=DID_ERROR driverbyte=DRIVER_OK
Oct 15 07:30:16 ml150 kernel: sd 5:0:0:0: [sdi] tag#1 CDB: Read(10) 28
00 00 00 00 40 00 00 38 00
Oct 15 07:30:17 ml150 kernel: sd 5:0:1:0: [sdp] tag#0 FAILED Result: 

Re: SCSI qla2xxx: tcm_qla2xxx server code seems to have regressed quite badly with latest testing

[Laurence Oberman]

On Fri, 2018-10-12 at 17:51 -0700, Bart Van Assche wrote:
> On 10/12/18 1:36 PM, Laurence Oberman wrote:
>  > While I have for the longest time used 4.5 as a base for my F/C
> jammer
>  > that I use every day here in our lab I recently added more jammer
> code
>  > so I decided to test this all on latest upstream.
>  >
>  > Booting the target server on my 4.5 kernel with jammer code is
>  > flawless and serves LUNS with no issues and handles the jamming
> also
>  > fine.
>  >
>  > However just building a 4.19.0_rc7+-1 (I left the jammer stuff
> out)
>  > its pretty broken.
> 
> A large number of patches went upstream between these two kernel 
> versions for both the QLogic initiator and target drivers. From the
> logs 
> it seems like you were using QLogic hardware at both the initiator
> and 
> target side? If so, which kernel version was running at the
> initiator 
> side during these tests? 4.5, 4.19-rc7+ or yet another version?
> 
> Thanks,
> 
> Bart.
> 
> 

I had only replied to Bart, this was my reply, reply all now

Hi Bart
Thank you for always being helpful.

I am using at the moment RHEL 7.5 for the initiator (based on kernel
3.10 but of course lots of backports)
The exact same initiator is working fine with the 4.5 and I would not
expect the target to  require the same kernel level.
Of course I will try latest upstream on the initiator later and reply
back.

I was thinking the target should adhere to the standards and support
many types of kernels within reason for the initiator.

Thanks

SCSI qla2xxx: tcm_qla2xxx server code seems to have regressed quite badly with latest testing

[Laurence Oberman]

Hello

While I have for the longest time used 4.5 as a base for my F/C jammer
that I use every day here in our lab I recently added more jammer code
so I decided to test this all on latest upstream.

Booting the target server on my 4.5 kernel with jammer code is flawless
and serves LUNS with no issues and handles the jamming also fine.

However just building a 4.19.0_rc7+-1 (I left the jammer stuff out) its
pretty broken.

So: 
Has anybody used this recently on the later kernels, if so when was it
last stable before I go starting to figure this out.

Some supporting Data
---
The server

The have seen the nacks below before and its seemed to work (been a
while though)

[  189.631036] qla2xxx [:07:00.1]-00af:7: Performing ISP error
recovery - ha=d1cfbc6b.
[  190.667982] qla2xxx [:07:00.1]-500a:7: LOOP UP detected (4
Gbps).
[  191.787048] qla2xxx [:07:00.0]-00af:1: Performing ISP error
recovery - ha=d30e7010.
[  192.824963] qla2xxx [:07:00.0]-500a:1: LOOP UP detected (4
Gbps).

[  389.510040] qla2xxx [:07:00.1]-d034:7: qla24xx_do_nack_work
create sess success 5c90391f
[  389.530212] qla2xxx [:07:00.0]-d034:1: qla24xx_do_nack_work
create sess success 83b2759c
[  389.987400] qla2xxx [:07:00.1]-d034:7: qla24xx_do_nack_work
create sess success 8498cfdd
[  390.068025] qla2xxx [:07:00.0]-d034:1: qla24xx_do_nack_work
create sess success 9af322af

Client however is badly broken (Its running RHEL 7.5 right now)

   24.884271] qla2xxx [:03:00.1]-801c:14: Abort command issued
nexus=14:0:0 --  1 2002.
[   24.899968] qla2xxx [:03:00.0]-801c:12: Abort command issued
nexus=12:0:0 --  1 2002.
[   24.915903] qla2xxx [:03:00.0]-801c:12: Abort command issued
nexus=12:0:10 --  1 2002.
[   25.004912] qla2xxx [:03:00.1]-801c:14: Abort command issued
nexus=14:0:10 --  1 2002.
[**] (7 of 10) A start job is running fo...on device 8:112 (17s /
no limit)[   35.955940] qla2xxx [:03:00.1]-801c:14: Abort command
issued nexus=14:0:8 --  1 2002.
[   35.994432] sd 14:0:0:0: Unexpected response from lun 8 while
scanning, scan aborted
[   36.031970] scsi 14:0:1:0: Direct-Access LIO-
ORG  sanboot694.0  PQ: 0 ANSI: 5
[   36.095961] scsi 14:0:1:0: alua: supports implicit and explicit TPGS
[   36.127023] scsi 14:0:1:0: alua: device
naa.60014055bfcdb7e1a7349c38126475e2 port group 0 rel port 2
[   36.170336] scsi 14:0:1:0: alua: Attached
[   36.189777] sd 14:0:1:0: Warning! Received an indication that the
LUN assignments on this target have changed. The Linux SCSI layer does
not automatical
[   36.189905] sd 14:0:1:0: Attached scsi generic sg15 type 0
[   36.283640] sd 14:0:1:0: alua: port group 00 state A non-preferred
supports TOlUSNA
[   36.283641] sd 14:0:1:0: [sdo] 209715200 512-byte logical blocks:
(107 GB/100 GiB)
[   36.283644] sd 14:0:1:0: [sdo] 4096-byte physical blocks
[   36.283664] scsi 14:0:1:10: Direct-Access LIO-ORG  block-
10 4.0  PQ: 0 ANSI: 5
[   36.284305] sd 14:0:1:0: [sdo] Write Protect is off
[   36.284454] sd 14:0:1:0: [sdo] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[   36.331321] sd 14:0:1:0: [sdo] Attached SCSI disk
[ *] (8 of 10) A start job is running fo...k in the kernel (19s /
no limit)[   36.524843] scsi 14:0:1:10: alua: supports implicit and
explicit TPGS
[   36.555474] scsi 14:0:1:10: alua: device
naa.60014052fedf4c41aeb439494ee641fe port group 0 rel port 2
[   36.599498] scsi 14:0:1:10: alua: Attached
[   36.619278] sd 14:0:1:10: alua: port group 00 state A non-preferred
supports TOlUSNA
[   36.619286] sd 14:0:1:10: Attached scsi generic sg16 type 0
[   36.68] sd 14:0:1:10: [sdp] 41943040 512-byte logical blocks:
(21.4 GB/20.0 GiB)
[   36.684447] sd 14:0:1:10: [sdp] 4096-byte physical blocks
[   36.684448] scsi 14:0:1:9: Direct-Access LIO-ORG  block-
9  4.0  PQ: 0 ANSI: 5
[   36.786318] sd 14:0:1:10: [sdp] Write Protect is off
[   36.810070] sd 14:0:1:10: [sdp] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[**] (8 of 10) A start job is running fo...k in the kernel (20s /
no limit)[   36.868799] scsi 14:0:1:9: alua: supports implicit and
explicit TPGS
[   36.899505] scsi 14:0:1:9: alua: device
naa.6001405b39403fdc0004075a9c1bfc7b port group 0 rel port 2
[   36.901649] sd 14:0:1:10: [sdp] Attached SCSI disk
[   36.967102] scsi 14:0:1:9: alua: Attached
[   36.986518] sd 14:0:1:9: Attached scsi generic sg17 type 0
[   36.986579] sd 14:0:1:9: Warning! Received an indication that the
LUN assignments on this target have changed. The Linux SCSI layer does
not automatical
[   37.078486] sd 14:0:1:9: [sdq] 41943040 512-byte logical blocks:
(21.4 GB/20.0 GiB)
[   37.115268] sd 14:0:1:9: [sdq] 4096-byte physical blocks
[   37.141405] scsi 14:0:1:8: Direct-Access LIO-ORG  block-
8  4.0  PQ: 0 ANSI: 5
[ *] (2 of 10) A start job is running fo... on device 8:48 (26s /
no limit)[   44.979937] 

Re: [PATCH 0/4] libfc/fcoe: disc_mutex fixes

[Laurence Oberman]

On Wed, 2018-07-11 at 10:09 +0200, Hannes Reinecke wrote:
> Hi all,
> 
> it turned out that my original patch for fixing up disc_mutex
> handling
> wasn't quite perfect, as it was causing 'scheduling while atomie'
> warnings.
> Upon inspection it turns out that we can't use RCU list traversal in
> most cases, as these list iterations would be calling
> fc_rport_login() /
> fc_rport_logoff(), which in turn would be acquiring a mutex.
> So this patchset fixes up the remoaining uses, and also adds a
> warining
> to catch reference counting issues.
> 
> As usual, comments and reviews are welcome.
> 
> Hannes Reinecke (4):
>   libfc: fixup lockdep annotations
>   libfc: hold disc_mutex in fc_disc_stop_rports()
>   libfc: Add WARN_ON() when deleting rports
>   fcoe: hold disc_mutex when traversing rport lists
> 
>  drivers/scsi/fcoe/fcoe_ctlr.c | 12 
>  drivers/scsi/libfc/fc_disc.c  | 10 --
>  drivers/scsi/libfc/fc_rport.c |  3 ++-
>  3 files changed, 10 insertions(+), 15 deletions(-)
> 
Hannes
Series looks fine to me, I will try get this tested as well.
Reviewed-by Laurence Oberman 

Re: Possible race in completion with SRP after abort with latest upstream kernel 4.17.0+

[Laurence Oberman]

On Thu, 2018-06-07 at 15:21 +, Bart Van Assche wrote:
> On Thu, 2018-06-07 at 09:23 -0400, Laurence Oberman wrote:
> > Have you seen this before, let me know what else you want from the
> > dump
> > while I look further.
> 
> Hello Laurence,
> 
> I haven't seen this before and I can't reproduce this by running srp-
> tests
> against Linus' latest master. So it would be appreciated if you could
> tell
> me how to reproduce this behavior or if you could run a bisect.
> 
> Thanks,
> 
> Bart.
> 
> 
> 
> 
> 
> NrybXǧv^)޺{.n+{ٚ{ayʇڙ,jfhzwj:+vwjmzZ+ݢj"!

OK, let me see if I can get it to fail reliably to narrow it down.
Thanks
Laurence

Possible race in completion with SRP after abort with latest upstream kernel 4.17.0+

[Laurence Oberman]

Hello Bart

Have not seen this more than twice but during testing of latest
upstream kernel with SRP I have had two of these completion races.

4.17.0+

[49945.984133] sd 2:0:0:29: alua: transition timeout set to 60 seconds
[49945.984136] sd 2:0:0:29: alua: port group 00 state A non-preferred
supports TOlUSNA
[49946.023273] sd 2:0:0:6: alua: port group 00 state A non-preferred
supports TOlUSNA
[49946.052514] sd 2:0:0:5: alua: port group 00 state A non-preferred
supports TOlUSNA
[49946.092895] sd 2:0:0:4: [sdl] Attached SCSI disk
[49946.093422] sd 2:0:0:6: alua: port group 00 state A non-preferred
supports TOlUSNA
[49953.156158] scsi host2: SRP abort called * Abort
[49953.187444] sd 2:0:0:5: [sdm] Attached SCSI disk
[49953.211545] BUG: unable to handle kernel NULL pointer dereference at
0008
[49965.632850] PGD 0 P4D 0 
[49965.644974] Oops: 0002 [#1] SMP PTI
[49965.661765] CPU: 11 PID: 2949 Comm: kworker/u64:0 Kdump: loaded
Tainted: G  I   4.17.0+ #1
[49965.711026] Hardware name: HP ProLiant DL380 G7, BIOS P67 08/16/2015
[49965.742461] Workqueue: scsi_tmf_2 scmd_eh_abort_handler
[49965.770633] RIP: 0010:_raw_spin_lock_irqsave+0x1e/0x40
[49965.795410] Code: 40 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90
53 9c 58 66 66 90 66 90 48 89 c3 fa 66 66 90 66 66 90 31 c0 ba 01 00 00
00  0f b1 17 85 c0 75 05 48 89 d8 5b c3 89 c6 e8 d7 26 92 ff eb f2 
[49965.892623] RSP: 0018:b75e4789fdc0 EFLAGS: 00010046
[49965.920553] RAX:  RBX: 0286 RCX:
0018
[49965.954952] RDX: 0001 RSI: 000a RDI:
0008
[49965.995180] RBP:  R08:  R09:
000a
[49966.033257] R10:  R11:  R12:
000a
[49966.073219] R13: 8d51f9041380 R14:  R15:
8d454df84d30
[49966.107885] FS:  () GS:8d52b334()
knlGS:
[49966.150490] CS:  0010 DS:  ES:  CR0: 80050033
[49966.177976] CR2: 0008 CR3: 00107300a005 CR4:
000206e0
[49966.216606] Call Trace:
[49966.228353]  complete+0x18/0x50
[49966.243410]  scsi_end_request+0x95/0x1e0
[49966.263891]  scsi_io_completion+0x1c1/0x680
[49966.286617]  process_one_work+0x171/0x370
[49966.305850]  worker_thread+0x49/0x3f0
[49966.323408]  kthread+0xf8/0x130
[49966.341046]  ? max_active_store+0x80/0x80
[49966.362901]  ? kthread_bind+0x10/0x10
[49966.382485]  ret_from_fork+0x35/0x40

Looks like a race in completion

Pull request off stack

struct request {
  q = 0x8d51f979b0c0, 
  mq_ctx = 0xd752442e0600, 
  cpu = -1, 
  cmd_flags = 0, 
  rq_flags = 139456, 
  internal_tag = -1, 
  __data_len = 0, 
  tag = 56, 
  __sector = 8191008, 
  bio = 0x0, 
  biotail = 0x8d46acd9e700, 
  queuelist = {
next = 0x8d454df84c40, 
prev = 0x8d454df84c40
  }, 

struct gendisk {
  major = 67, 
  first_minor = 64, 
  minors = 16, 
  disk_name =
"sdba\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\0
00\000\000\000\000\000\000\000\000ba", 


crash> scsi_device.sdev_state 0x8d52b1da4800
  sdev_state = SDEV_RUNNING

crash> Scsi_Host.shost_state 0x8d466b969000
  shost_state = SHOST_RUNNING


   if (scsi_target(sdev)->single_lun ||
!list_empty(>host->starved_list))
kblockd_schedule_work(>requeue_work);

Have you seen this before, let me know what else you want from the dump
while I look further.
I have not tested for a while so not sure where this crept in or if its
even an issue for others.

Thanks
Laurence

Re: [PATCH] scsi: qla2xxx: remove the unused tcm_qla2xxx_cmd_wq

[Laurence Oberman]

On Wed, 2018-05-02 at 13:31 -0700, Andrei Vagin wrote:
> Signed-off-by: Andrei Vagin <ava...@openvz.org>
> ---
>  drivers/scsi/qla2xxx/tcm_qla2xxx.c | 10 --
>  1 file changed, 10 deletions(-)
> 
> diff --git a/drivers/scsi/qla2xxx/tcm_qla2xxx.c
> b/drivers/scsi/qla2xxx/tcm_qla2xxx.c
> index aadfeaac3898..b63440fec18e 100644
> --- a/drivers/scsi/qla2xxx/tcm_qla2xxx.c
> +++ b/drivers/scsi/qla2xxx/tcm_qla2xxx.c
> @@ -48,7 +48,6 @@
>  #include "tcm_qla2xxx.h"
>  
>  static struct workqueue_struct *tcm_qla2xxx_free_wq;
> -static struct workqueue_struct *tcm_qla2xxx_cmd_wq;
>  
>  /*
>   * Parse WWN.
> @@ -1976,16 +1975,8 @@ static int tcm_qla2xxx_register_configfs(void)
>   goto out_fabric_npiv;
>   }
>  
> - tcm_qla2xxx_cmd_wq = alloc_workqueue("tcm_qla2xxx_cmd", 0,
> 0);
> - if (!tcm_qla2xxx_cmd_wq) {
> - ret = -ENOMEM;
> - goto out_free_wq;
> - }
> -
>   return 0;
>  
> -out_free_wq:
> - destroy_workqueue(tcm_qla2xxx_free_wq);
>  out_fabric_npiv:
>   target_unregister_template(_qla2xxx_npiv_ops);
>  out_fabric:
> @@ -1995,7 +1986,6 @@ static int tcm_qla2xxx_register_configfs(void)
>  
>  static void tcm_qla2xxx_deregister_configfs(void)
>  {
> - destroy_workqueue(tcm_qla2xxx_cmd_wq);
>   destroy_workqueue(tcm_qla2xxx_free_wq);
>  
>   target_unregister_template(_qla2xxx_ops);
When I first saw this I thought, no way, it must be used, but when I
had a closer look and checked its indeed not used or even made
external.

This looks OK to me.

Reviewed-by: Laurence Oberman <lober...@redhat.com>

Re: [PATCH 0/3] scsi: scsi-mq: don't hold host_busy in IO path

[Laurence Oberman]

On Fri, 2018-04-27 at 15:48 +, Bart Van Assche wrote:
> On Fri, 2018-04-27 at 09:39 -0600, Jens Axboe wrote:
> > blk_mq_tagset_busy_iter(>tag_set, scsi_host_check_in_flight,
> > _flight);
> > return in_flight.cnt + atomic_read(>host_busy);
> > 
> > The atomic read is basically free, once we get rid of the dirty of
> > that
> > variable on each IO.
> 
> Hello Jens,
> 
> What makes you think that " + atomic_read(>host_busy)" is
> necessary?
> I am not aware of any code outside the SCSI core that modifies the
> host_busy
> member.
> 
> Thanks,
> 
> Bart.
> 
> 
> 

As part of testing latest upstream in MQ and non-MQ I intend to test
this patch series fully on actual hardware
F/C 8G to memory backed array LUNS and of course SRP/RDMA
I have started working on this and will report back.
Thanks
Laurence

Re: [PATCH 13/25] tcm_qla2xxx: Do not allow aborted cmd to advance.

[Laurence Oberman]

On Sun, 2017-05-21 at 21:45 -0700, Nicholas A. Bellinger wrote:
> On Fri, 2017-05-19 at 14:53 -0700, Himanshu Madhani wrote:
> > From: Quinn Tran 
> > 
> > In case of hardware queue full, commands can loop between
> > TCM stack and tcm_qla2xx shim layers for retry. While command
> > is waiting for retry, task mgmt can get ahead and abort the
> > cmmand that encountered queue full condition. Fix this by
> > dropping the command, if task mgmt has already started the
> > command free process.
> > 
> > Cc: 
> > Signed-off-by: Quinn Tran 
> > Signed-off-by: Himanshu Madhani 
> > ---
> >  drivers/scsi/qla2xxx/tcm_qla2xxx.c | 14 ++
> >  1 file changed, 14 insertions(+)
> > 
> > diff --git a/drivers/scsi/qla2xxx/tcm_qla2xxx.c
> > b/drivers/scsi/qla2xxx/tcm_qla2xxx.c
> > index 7443e4efa3ae..07f8ad001bcb 100644
> > --- a/drivers/scsi/qla2xxx/tcm_qla2xxx.c
> > +++ b/drivers/scsi/qla2xxx/tcm_qla2xxx.c
> > @@ -686,6 +686,20 @@ static int tcm_qla2xxx_queue_status(struct
> > se_cmd *se_cmd)
> >     struct qla_tgt_cmd, se_cmd);
> >     int xmit_type = QLA_TGT_XMIT_STATUS;
> >  
> > +   if (cmd->aborted) {
> > +   /*
> > +    * Cmd can loop during Q-full.
> > tcm_qla2xxx_aborted_task
> > +    * can get ahead of this cmd.
> > tcm_qla2xxx_aborted_task
> > +    * already kick start the free.
> > +    */
> > +   pr_debug(
> > +   "queue_data_in aborted cmd[%p] refcount %d
> > transport_state %x, t_state %x, se_cmd_flags %x\n",
> > +   cmd, kref_read(>se_cmd.cmd_kref),
> > +   cmd->se_cmd.transport_state, cmd-
> > >se_cmd.t_state,
> > +   cmd->se_cmd.se_cmd_flags);
> > +   return 0;
> > +   }
> > +
> >     cmd->bufflen = se_cmd->data_length;
> >     cmd->sg = NULL;
> >     cmd->sg_cnt = 0;
> 
> As your comment above mentions, there is a known issue in target-core
> when queue-full occurs repeatably and a se_cmd descriptor is
> explicitly
> stopped via session shutdown, TMR ABORT_TASK / LUN_RESET or
> otherwise.
> 
> We hit this scenario a while back in iser-target with iw_cxgb hw,
> which
> due to a separate bug was constantly triggering queue-full under load
> to
> uncover this same case.
> 
> So I'm still considering the different approaches to address this in
> target-core proper, but don't have a problem with merging this as-is
> as
> it won't logically conflict with any of those changes.
> 
> That said:
> 
> Acked-by: Nicholas Bellinger 
> 
Indeed the queue-full issue has been there for quite a while in the
core.
Affects ISER, SRP and F/C (tcm_qla2xxx)

Re: [PATCH v4 1/7] scsi: hpsa: Eliminate duplicate barriers on weakly-ordered archs

[Laurence Oberman]

On Mon, 2018-03-19 at 22:50 -0400, Sinan Kaya wrote:
> Code includes wmb() followed by writel(). writel() already has a
> barrier on some architectures like arm64.
> 
> This ends up CPU observing two barriers back to back before executing
> the register write.
> 
> Since code already has an explicit barrier call, changing writel() to
> writel_relaxed().
> 
> Signed-off-by: Sinan Kaya <ok...@codeaurora.org>
> ---
>  drivers/scsi/hpsa.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/scsi/hpsa.h b/drivers/scsi/hpsa.h
> index 018f980..c7d7e6a 100644
> --- a/drivers/scsi/hpsa.h
> +++ b/drivers/scsi/hpsa.h
> @@ -599,7 +599,7 @@ static unsigned long
> SA5_ioaccel_mode1_completed(struct ctlr_info *h, u8 q)
>    * but with current driver design this is easiest.
>    */
>   wmb();
> - writel((q << 24) | rq->current_entry, h->vaddr +
> + writel_relaxed((q << 24) | rq->current_entry, h-
> >vaddr +
>   IOACCEL_MODE1_CONSUMER_INDEX);
>   atomic_dec(>commands_outstanding);
>   }

This looks like it would work for the x86_64 and arm because of how its
defined architecture specific for the x86_64 and the arm64

I guess its up to Don and the driver folks and if its worth the change.
I am generally not a fan of messing with these barrier things though.

Reviewed-by: Laurence Oberman <lober...@redhat.com>

Re: [PATCH] scsi: scsi_dh_alua: Correct comment for alua_alloc_pg()

[Laurence Oberman]

On Wed, 2018-03-14 at 09:52 -0400, John Pittman wrote:
> In the comment for function alua_alloc_pg() the argument '@h' is
> mistakenly referred to.  Fix this by replacing it with the correct
> argument reference, '@tpgs', and provide a short description.
> 
> Signed-off-by: John Pittman <jpitt...@redhat.com>
> ---
>  drivers/scsi/device_handler/scsi_dh_alua.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/scsi/device_handler/scsi_dh_alua.c
> b/drivers/scsi/device_handler/scsi_dh_alua.c
> index 4b44325..ab89e42 100644
> --- a/drivers/scsi/device_handler/scsi_dh_alua.c
> +++ b/drivers/scsi/device_handler/scsi_dh_alua.c
> @@ -214,8 +214,8 @@ static struct alua_port_group
> *alua_find_get_pg(char *id_str, size_t id_size,
>  /*
>   * alua_alloc_pg - Allocate a new port_group structure
>   * @sdev: scsi device
> - * @h: alua device_handler data
>   * @group_id: port group id
> + * @tpgs: target port group settings
>   *
>   * Allocate a new port_group structure for a given
>   * device.

Looks good to me, Thanks

Reviewed-by Laurence Oberman <lober...@redhat.com>

Re: [PATCH] scsi: sd: Remember that READ CAPACITY(16) succeeded

[Laurence Oberman]

On Wed, 2018-03-14 at 12:15 -0400, Martin K. Petersen wrote:
> The USB storage glue sets the try_rc_10_first flag in an attempt to
> avoid wedging poorly implemented legacy USB devices.
> 
> If the device capacity is too large to be expressed in the provided
> response buffer field of READ CAPACITY(10), a well-behaved device
> will
> set the reported capacity to 0x. We will then attempt to
> issue a
> READ CAPACITY(16) to obtain the real capacity.
> 
> Since this part of the discovery logic is not covered by the
> first_scan
> flag, a warning will be printed a couple of times times per
> revalidate
> attempt if we upgrade from READ CAPACITY(10) to READ CAPACITY(16).
> 
> Remember that we have successfully issued READ CAPACITY(16) so we can
> take the fast path on subsequent revalidate attempts.
> 
> Reported-by: Menion <men...@gmail.com>
> Signed-off-by: Martin K. Petersen <martin.peter...@oracle.com>
> ---
>  drivers/scsi/sd.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
> index bff21e636ddd..6e971b94af7d 100644
> --- a/drivers/scsi/sd.c
> +++ b/drivers/scsi/sd.c
> @@ -2484,6 +2484,8 @@ sd_read_capacity(struct scsi_disk *sdkp,
> unsigned char *buffer)
>   sector_size = old_sector_size;
>   goto got_data;
>   }
> + /* Remember that READ CAPACITY(16) succeeded
> */
> + sdp->try_rc_10_first = 0;
>   }
>   }
>  

Looks fine to me.
Reviewed-by: Laurence Oberman <lober...@redhat.com>

Re: [PATCH V3 1/8] scsi: hpsa: fix selection of reply queue

[Laurence Oberman]

On Thu, 2018-03-08 at 21:42 +0800, Ming Lei wrote:
> On Wed, Mar 07, 2018 at 09:11:37AM -0500, Laurence Oberman wrote:
> > On Tue, 2018-03-06 at 14:24 -0500, Martin K. Petersen wrote:
> > > Ming,
> > > 
> > > > Given both Don and Laurence have verified that patch 1 and
> > > > patch 2
> > > > does fix IO hang, could you consider to merge the two first?
> > > 
> > > Oh, and I would still need a formal Acked-by: from Don and
> > > Tested-by:
> > > from Laurence.
> > > 
> > > Also, for 4.16/scsi-fixes I would prefer verification to be done
> > > with
> > > just patch 1/8 and none of the subsequent changes in place. Just
> > > to
> > > make
> > > sure we're testing the right thing.
> > > 
> > > Thanks!
> > > 
> > 
> > Hello Martin
> > 
> > I tested just Patch 1/8 from the V3 series.
> > No issues running workload and no issues booting on the DL380G7.
> > Don can you ack this so we can at least get this one in.
> > 
> > Against: 4.16.0-rc4.v31of8+ on an x86_64
> > 
> > Tested-by: Laurence Oberman <lober...@redhat.com>
> 
> Hi Laurence,
> 
> Thanks for your test!
> 
> Could you test patch 2 too since you have megaraid_sas controller?
> 
> Looks it is better to split the fix patches from the current
> patchset,
> since these fixes should be for V4.16.
> 
> Thanks
> Ming

Ho Ming I see a V4 now so I am going to wait until you split these and
then I will test both HPSA and megaraid_sas once Kashyap agrees.

When I see a V4 show up with the split will pull and act on it.

Thanks
Laurence

Re: [PATCH V3 1/8] scsi: hpsa: fix selection of reply queue

[Laurence Oberman]

On Tue, 2018-03-06 at 14:24 -0500, Martin K. Petersen wrote:
> Ming,
> 
> > Given both Don and Laurence have verified that patch 1 and patch 2
> > does fix IO hang, could you consider to merge the two first?
> 
> Oh, and I would still need a formal Acked-by: from Don and Tested-by:
> from Laurence.
> 
> Also, for 4.16/scsi-fixes I would prefer verification to be done with
> just patch 1/8 and none of the subsequent changes in place. Just to
> make
> sure we're testing the right thing.
> 
> Thanks!
> 

Hello Martin

I tested just Patch 1/8 from the V3 series.
No issues running workload and no issues booting on the DL380G7.
Don can you ack this so we can at least get this one in.

Against: 4.16.0-rc4.v31of8+ on an x86_64

Tested-by: Laurence Oberman <lober...@redhat.com>

Thanks
Laurence

Re: [PATCH V3 1/8] scsi: hpsa: fix selection of reply queue

[Laurence Oberman]

On Fri, 2018-03-02 at 15:03 +, Don Brace wrote:
> > -Original Message-
> > From: Laurence Oberman [mailto:lober...@redhat.com]
> > Sent: Friday, March 02, 2018 8:09 AM
> > To: Ming Lei <ming@redhat.com>
> > Cc: Don Brace <don.br...@microsemi.com>; Jens Axboe <axboe@kernel.d
> > k>;
> > linux-bl...@vger.kernel.org; Christoph Hellwig <h...@infradead.org>;
> > Mike
> > Snitzer <snit...@redhat.com>; linux-scsi@vger.kernel.org; Hannes
> > Reinecke
> > <h...@suse.de>; Arun Easi <arun.e...@cavium.com>; Omar Sandoval
> > <osan...@fb.com>; Martin K . Petersen <martin.peter...@oracle.com>;
> > James
> > Bottomley <james.bottom...@hansenpartnership.com>; Christoph
> > Hellwig
> > <h...@lst.de>; Kashyap Desai <kashyap.de...@broadcom.com>; Peter
> > Rivera
> > <peter.riv...@broadcom.com>; Meelis Roos <mr...@linux.ee>
> > Subject: Re: [PATCH V3 1/8] scsi: hpsa: fix selection of reply
> > queue
> > 
> > EXTERNAL EMAIL
> > 
> > 
> > On Fri, 2018-03-02 at 10:16 +0800, Ming Lei wrote:
> > > On Thu, Mar 01, 2018 at 04:19:34PM -0500, Laurence Oberman wrote:
> > > > On Thu, 2018-03-01 at 14:01 -0500, Laurence Oberman wrote:
> > > > > On Thu, 2018-03-01 at 16:18 +, Don Brace wrote:
> > > > > > > -Original Message-
> > > > > > > From: Ming Lei [mailto:ming@redhat.com]
> > > > > > > Sent: Tuesday, February 27, 2018 4:08 AM
> > > > > > > To: Jens Axboe <ax...@kernel.dk>; linux-block@vger.kernel
> > > > > > > .org
> > > > > > > ;
> > > > > > > Christoph
> > > > > > > Hellwig <h...@infradead.org>; Mike Snitzer <snitzer@redhat
> > > > > > > .com
> > > > > > > > 
> > > > > > > 
> > > > > > > Cc: linux-scsi@vger.kernel.org; Hannes Reinecke <hare@sus
> > > > > > > e.de
> > > > > > > > ;
> > > > > > > 
> > > > > > > Arun Easi
> > > > > > > <arun.e...@cavium.com>; Omar Sandoval <osan...@fb.com>;
> > > > > > > Martin K
> > > > > > > .
> > > > > > > Petersen <martin.peter...@oracle.com>; James Bottomley
> > > > > > > <james.bottom...@hansenpartnership.com>; Christoph
> > > > > > > Hellwig  > > > > > > ch@l
> > > > > > > st
> > > > > > > .de>;
> > > > > > > Don Brace <don.br...@microsemi.com>; Kashyap Desai
> > > > > > > <kashyap.de...@broadcom.com>; Peter Rivera <peter.rivera@
> > > > > > > broa
> > > > > > > dcom
> > > > > > > .c
> > > > > > > om>;
> > > > > > > Laurence Oberman <lober...@redhat.com>; Ming Lei
> > > > > > > <ming@redhat.com>; Meelis Roos <mr...@linux.ee>
> > > > > > > Subject: [PATCH V3 1/8] scsi: hpsa: fix selection of
> > > > > > > reply
> > > > > > > queue
> > > > > > > 
> > > 
> > > Seems Don run into IO failure without blk-mq, could you run your
> > > tests again
> > > in legacy mode?
> > > 
> > > Thanks,
> > > Ming
> > 
> > Hello Ming
> > I ran multiple passes on Legacy and still see no issues in my test
> > bed
> > 
> > BOOT_IMAGE=/vmlinuz-4.16.0-rc2.ming+ root=UUID=43f86d71-b1bf-4789-
> > a28e-
> > 21c6ddc90195 ro crashkernel=256M@64M log_buf_len=64M
> > console=ttyS1,115200n8
> > 
> > HEAD of the git kernel I am using
> > 
> > 694e16f scsi: megaraid: improve scsi_mq performance via
> > .host_tagset
> > 793686c scsi: hpsa: improve scsi_mq performance via .host_tagset
> > 60d5b36 block: null_blk: introduce module parameter of
> > 'g_host_tags'
> > 8847067 scsi: Add template flag 'host_tagset'
> > a8fbdd6 blk-mq: introduce BLK_MQ_F_HOST_TAGS
> > 4710fab blk-mq: introduce 'start_tag' field to 'struct blk_mq_tags'
> > 09bb153 scsi: megaraid_sas: fix selection of reply queue
> > 52700d8 scsi: hpsa: fix selection of reply queue
> 
> I checkout out Linus's tree (4.16.0-rc3+) and re-applied the above
> patches.
> I  and have been running 24 hours with no issues.
> Evidently my forked copy was corrupted. 
> 
> So, my I/O testing has gone well. 
> 
> I'll run some performance numbers next.
> 
> Thanks,
> Don

Unless Kashyap is not happy we need to consider getting this in to
Linus now because we are seeing HPE servers that keep hanging now with
the original commit now upstream.

Kashyap, are you good with the v3 patchset or still concerned with
performance. I was getting pretty good IOPS/sec to individual SSD
drives set up as jbod devices on the megaraid_sas.

With larger I/O sizes like 1MB I was getting good MB/sec and not seeing
a measurable performance impact.

I dont have the hardware you have to mimic your configuration.

Thanks
Laurence

Re: [PATCH V3 1/8] scsi: hpsa: fix selection of reply queue

[Laurence Oberman]

On Fri, 2018-03-02 at 10:16 +0800, Ming Lei wrote:
> On Thu, Mar 01, 2018 at 04:19:34PM -0500, Laurence Oberman wrote:
> > On Thu, 2018-03-01 at 14:01 -0500, Laurence Oberman wrote:
> > > On Thu, 2018-03-01 at 16:18 +, Don Brace wrote:
> > > > > -Original Message-
> > > > > From: Ming Lei [mailto:ming@redhat.com]
> > > > > Sent: Tuesday, February 27, 2018 4:08 AM
> > > > > To: Jens Axboe <ax...@kernel.dk>; linux-bl...@vger.kernel.org
> > > > > ;
> > > > > Christoph
> > > > > Hellwig <h...@infradead.org>; Mike Snitzer <snit...@redhat.com
> > > > > >
> > > > > Cc: linux-scsi@vger.kernel.org; Hannes Reinecke <h...@suse.de
> > > > > >;
> > > > > Arun Easi
> > > > > <arun.e...@cavium.com>; Omar Sandoval <osan...@fb.com>;
> > > > > Martin K
> > > > > .
> > > > > Petersen <martin.peter...@oracle.com>; James Bottomley
> > > > > <james.bottom...@hansenpartnership.com>; Christoph Hellwig  > > > > ch@l
> > > > > st
> > > > > .de>;
> > > > > Don Brace <don.br...@microsemi.com>; Kashyap Desai
> > > > > <kashyap.de...@broadcom.com>; Peter Rivera <peter.rivera@broa
> > > > > dcom
> > > > > .c
> > > > > om>;
> > > > > Laurence Oberman <lober...@redhat.com>; Ming Lei
> > > > > <ming@redhat.com>; Meelis Roos <mr...@linux.ee>
> > > > > Subject: [PATCH V3 1/8] scsi: hpsa: fix selection of reply
> > > > > queue
> > > > > 
> > > > > EXTERNAL EMAIL
> > > > > 
> > > > > 
> > > > > From 84676c1f21 (genirq/affinity: assign vectors to all
> > > > > possible
> > > > > CPUs),
> > > > > one msix vector can be created without any online CPU mapped,
> > > > > then
> > > > > one
> > > > > command's completion may not be notified.
> > > > > 
> > > > > This patch setups mapping between cpu and reply queue
> > > > > according
> > > > > to
> > > > > irq
> > > > > affinity info retrived by pci_irq_get_affinity(), and uses
> > > > > this
> > > > > mapping
> > > > > table to choose reply queue for queuing one command.
> > > > > 
> > > > > Then the chosen reply queue has to be active, and fixes IO
> > > > > hang
> > > > > caused
> > > > > by using inactive reply queue which doesn't have any online
> > > > > CPU
> > > > > mapped.
> > > > > 
> > > > > Cc: Hannes Reinecke <h...@suse.de>
> > > > > Cc: Arun Easi <arun.e...@cavium.com>
> > > > > Cc: "Martin K. Petersen" <martin.peter...@oracle.com>,
> > > > > Cc: James Bottomley <james.bottom...@hansenpartnership.com>,
> > > > > Cc: Christoph Hellwig <h...@lst.de>,
> > > > > Cc: Don Brace <don.br...@microsemi.com>
> > > > > Cc: Kashyap Desai <kashyap.de...@broadcom.com>
> > > > > Cc: Peter Rivera <peter.riv...@broadcom.com>
> > > > > Cc: Laurence Oberman <lober...@redhat.com>
> > > > > Cc: Meelis Roos <mr...@linux.ee>
> > > > > Fixes: 84676c1f21e8 ("genirq/affinity: assign vectors to all
> > > > > possible CPUs")
> > > > > Signed-off-by: Ming Lei <ming@redhat.com>
> > > > 
> > > > I am getting some issues that need to be tracked down:
> > > > 
> > > > [ 1636.032984] hpsa :87:00.0: Acknowledging event:
> > > > 0xc032
> > > > (HP
> > > > SSD Smart Path configuration change)
> > > > [ 1638.510656] hpsa :87:00.0: scsi 3:0:8:0: updated Direct-
> > > > Access HP   MO0400JDVEU  PHYS DRV SSDSmartPathCap-
> > > > En-
> > > > Exp=0
> > > > [ 1653.967695] hpsa :87:00.0: Acknowledging event:
> > > > 0x8020
> > > > (HP
> > > > SSD Smart Path configuration change)
> > > > [ 1656.770377] hpsa :87:00.0: scsi 3:0:8:0: updated Direct-
> > > > Access HP   MO0400JDVEU  PHYS DRV SSDSmartPathCap-
> > > > En-
> > > > Exp=0
>

Re: [PATCH V3 0/8] blk-mq & scsi: fix reply queue selection and improve host wide tagset

[Laurence Oberman]

On Tue, 2018-02-27 at 18:07 +0800, Ming Lei wrote:
> Hi All,
> 
> The 1st two patches fixes reply queue selection, and this issue has
> been
> reported and can cause IO hang during booting, please consider the
> two
> for V4.16.
> 
> The following 6 patches try to improve hostwide tagset on hpsa and
> megaraid_sas by making hw queue per NUMA node.
> 
> I don't have high-performance hpsa and megaraid_sas device at hand.
> 
> Don Brace, could you test this patchset on concurrent IOs over you
> hpsa
> SSD and see if this approach is well?
> 
> Kashyap, could you test this patchset on your megaraid_sas SSDs?
> 
>   gitweb: https://github.com/ming1/linux/tree/v4.16-rc-host-tags-
> v3.2
> 
> thanks,
> Ming
> 
> Hannes Reinecke (1):
>   scsi: Add template flag 'host_tagset'
> 
> Ming Lei (7):
>   scsi: hpsa: fix selection of reply queue
>   scsi: megaraid_sas: fix selection of reply queue
>   blk-mq: introduce 'start_tag' field to 'struct blk_mq_tags'
>   blk-mq: introduce BLK_MQ_F_HOST_TAGS
>   block: null_blk: introduce module parameter of 'g_host_tags'
>   scsi: hpsa: improve scsi_mq performance via .host_tagset
>   scsi: megaraid: improve scsi_mq performance via .host_tagset
> 
>  block/blk-mq-debugfs.c  |  2 +
>  block/blk-mq-sched.c|  2 +-
>  block/blk-mq-tag.c  | 13 +++--
>  block/blk-mq-tag.h  | 11 ++--
>  block/blk-mq.c  | 50 +++---
>  block/blk-mq.h  |  3 +-
>  drivers/block/null_blk.c|  6 +++
>  drivers/scsi/hpsa.c | 79
> ++---
>  drivers/scsi/hpsa.h |  1 +
>  drivers/scsi/megaraid/megaraid_sas.h|  2 +-
>  drivers/scsi/megaraid/megaraid_sas_base.c   | 40 ++-
>  drivers/scsi/megaraid/megaraid_sas_fusion.c | 12 ++---
>  drivers/scsi/scsi_lib.c |  2 +
>  include/linux/blk-mq.h  |  2 +
>  include/scsi/scsi_host.h|  3 ++
>  15 files changed, 182 insertions(+), 46 deletions(-)
> 

For the patchset above
All functional I/O tests and boot tests passed with multiple concurrent
fio runs.

Original HPSA booting issue is also resolved and its important or we
will have to revert original genirq commit
commit 84676c1f21e8ff54befe985f4f14dc1edc10046b

Tested-by: Laurence Oberman <lober...@redhat.com>

Thanks
Laurence

Re: [PATCH V3 1/8] scsi: hpsa: fix selection of reply queue

[Laurence Oberman]

On Thu, 2018-03-01 at 14:01 -0500, Laurence Oberman wrote:
> On Thu, 2018-03-01 at 16:18 +, Don Brace wrote:
> > > -Original Message-
> > > From: Ming Lei [mailto:ming@redhat.com]
> > > Sent: Tuesday, February 27, 2018 4:08 AM
> > > To: Jens Axboe <ax...@kernel.dk>; linux-bl...@vger.kernel.org;
> > > Christoph
> > > Hellwig <h...@infradead.org>; Mike Snitzer <snit...@redhat.com>
> > > Cc: linux-scsi@vger.kernel.org; Hannes Reinecke <h...@suse.de>;
> > > Arun Easi
> > > <arun.e...@cavium.com>; Omar Sandoval <osan...@fb.com>; Martin K
> > > .
> > > Petersen <martin.peter...@oracle.com>; James Bottomley
> > > <james.bottom...@hansenpartnership.com>; Christoph Hellwig <hch@l
> > > st
> > > .de>;
> > > Don Brace <don.br...@microsemi.com>; Kashyap Desai
> > > <kashyap.de...@broadcom.com>; Peter Rivera <peter.rivera@broadcom
> > > .c
> > > om>;
> > > Laurence Oberman <lober...@redhat.com>; Ming Lei
> > > <ming@redhat.com>; Meelis Roos <mr...@linux.ee>
> > > Subject: [PATCH V3 1/8] scsi: hpsa: fix selection of reply queue
> > > 
> > > EXTERNAL EMAIL
> > > 
> > > 
> > > From 84676c1f21 (genirq/affinity: assign vectors to all possible
> > > CPUs),
> > > one msix vector can be created without any online CPU mapped,
> > > then
> > > one
> > > command's completion may not be notified.
> > > 
> > > This patch setups mapping between cpu and reply queue according
> > > to
> > > irq
> > > affinity info retrived by pci_irq_get_affinity(), and uses this
> > > mapping
> > > table to choose reply queue for queuing one command.
> > > 
> > > Then the chosen reply queue has to be active, and fixes IO hang
> > > caused
> > > by using inactive reply queue which doesn't have any online CPU
> > > mapped.
> > > 
> > > Cc: Hannes Reinecke <h...@suse.de>
> > > Cc: Arun Easi <arun.e...@cavium.com>
> > > Cc: "Martin K. Petersen" <martin.peter...@oracle.com>,
> > > Cc: James Bottomley <james.bottom...@hansenpartnership.com>,
> > > Cc: Christoph Hellwig <h...@lst.de>,
> > > Cc: Don Brace <don.br...@microsemi.com>
> > > Cc: Kashyap Desai <kashyap.de...@broadcom.com>
> > > Cc: Peter Rivera <peter.riv...@broadcom.com>
> > > Cc: Laurence Oberman <lober...@redhat.com>
> > > Cc: Meelis Roos <mr...@linux.ee>
> > > Fixes: 84676c1f21e8 ("genirq/affinity: assign vectors to all
> > > possible CPUs")
> > > Signed-off-by: Ming Lei <ming@redhat.com>
> > 
> > I am getting some issues that need to be tracked down:
> > 
> > [ 1636.032984] hpsa :87:00.0: Acknowledging event: 0xc032
> > (HP
> > SSD Smart Path configuration change)
> > [ 1638.510656] hpsa :87:00.0: scsi 3:0:8:0: updated Direct-
> > Access HP   MO0400JDVEU  PHYS DRV SSDSmartPathCap- En-
> > Exp=0
> > [ 1653.967695] hpsa :87:00.0: Acknowledging event: 0x8020
> > (HP
> > SSD Smart Path configuration change)
> > [ 1656.770377] hpsa :87:00.0: scsi 3:0:8:0: updated Direct-
> > Access HP   MO0400JDVEU  PHYS DRV SSDSmartPathCap- En-
> > Exp=0
> > [ 2839.762267] hpsa :87:00.0: Acknowledging event: 0x8020
> > (HP
> > SSD Smart Path configuration change)
> > [ 2840.841290] hpsa :87:00.0: scsi 3:0:8:0: updated Direct-
> > Access HP   MO0400JDVEU  PHYS DRV SSDSmartPathCap- En-
> > Exp=0
> > [ 2917.582653] hpsa :87:00.0: Acknowledging event: 0xc020
> > (HP
> > SSD Smart Path configuration change)
> > [ 2919.087191] hpsa :87:00.0: scsi 3:1:0:1: updated Direct-
> > Access HP   LOGICAL VOLUME   RAID-5 SSDSmartPathCap+ En+
> > Exp=1
> > [ 2919.142527] hpsa :87:00.0: hpsa_figure_phys_disk_ptrs:
> > [3:1:0:2] A phys disk component of LV is missing, turning off
> > offload_enabled for LV.
> > [ 2919.203915] hpsa :87:00.0: hpsa_figure_phys_disk_ptrs:
> > [3:1:0:2] A phys disk component of LV is missing, turning off
> > offload_enabled for LV.
> > [ 2919.266921] hpsa :87:00.0: hpsa_figure_phys_disk_ptrs:
> > [3:1:0:2] A phys disk component of LV is missing, turning off
> > offload_enabled for LV.
> > [ 2934.999629] hpsa :87:00.0: Acknowledging event: 0x4000
> > (HP
> > SSD 

Re: [PATCH V3 1/8] scsi: hpsa: fix selection of reply queue

[Laurence Oberman]

On Thu, 2018-03-01 at 16:18 +, Don Brace wrote:
> > -Original Message-
> > From: Ming Lei [mailto:ming@redhat.com]
> > Sent: Tuesday, February 27, 2018 4:08 AM
> > To: Jens Axboe <ax...@kernel.dk>; linux-bl...@vger.kernel.org;
> > Christoph
> > Hellwig <h...@infradead.org>; Mike Snitzer <snit...@redhat.com>
> > Cc: linux-scsi@vger.kernel.org; Hannes Reinecke <h...@suse.de>;
> > Arun Easi
> > <arun.e...@cavium.com>; Omar Sandoval <osan...@fb.com>; Martin K .
> > Petersen <martin.peter...@oracle.com>; James Bottomley
> > <james.bottom...@hansenpartnership.com>; Christoph Hellwig <hch@lst
> > .de>;
> > Don Brace <don.br...@microsemi.com>; Kashyap Desai
> > <kashyap.de...@broadcom.com>; Peter Rivera <peter.rivera@broadcom.c
> > om>;
> > Laurence Oberman <lober...@redhat.com>; Ming Lei
> > <ming@redhat.com>; Meelis Roos <mr...@linux.ee>
> > Subject: [PATCH V3 1/8] scsi: hpsa: fix selection of reply queue
> > 
> > EXTERNAL EMAIL
> > 
> > 
> > From 84676c1f21 (genirq/affinity: assign vectors to all possible
> > CPUs),
> > one msix vector can be created without any online CPU mapped, then
> > one
> > command's completion may not be notified.
> > 
> > This patch setups mapping between cpu and reply queue according to
> > irq
> > affinity info retrived by pci_irq_get_affinity(), and uses this
> > mapping
> > table to choose reply queue for queuing one command.
> > 
> > Then the chosen reply queue has to be active, and fixes IO hang
> > caused
> > by using inactive reply queue which doesn't have any online CPU
> > mapped.
> > 
> > Cc: Hannes Reinecke <h...@suse.de>
> > Cc: Arun Easi <arun.e...@cavium.com>
> > Cc: "Martin K. Petersen" <martin.peter...@oracle.com>,
> > Cc: James Bottomley <james.bottom...@hansenpartnership.com>,
> > Cc: Christoph Hellwig <h...@lst.de>,
> > Cc: Don Brace <don.br...@microsemi.com>
> > Cc: Kashyap Desai <kashyap.de...@broadcom.com>
> > Cc: Peter Rivera <peter.riv...@broadcom.com>
> > Cc: Laurence Oberman <lober...@redhat.com>
> > Cc: Meelis Roos <mr...@linux.ee>
> > Fixes: 84676c1f21e8 ("genirq/affinity: assign vectors to all
> > possible CPUs")
> > Signed-off-by: Ming Lei <ming@redhat.com>
> 
> I am getting some issues that need to be tracked down:
> 
> [ 1636.032984] hpsa :87:00.0: Acknowledging event: 0xc032 (HP
> SSD Smart Path configuration change)
> [ 1638.510656] hpsa :87:00.0: scsi 3:0:8:0: updated Direct-
> Access HP   MO0400JDVEU  PHYS DRV SSDSmartPathCap- En-
> Exp=0
> [ 1653.967695] hpsa :87:00.0: Acknowledging event: 0x8020 (HP
> SSD Smart Path configuration change)
> [ 1656.770377] hpsa :87:00.0: scsi 3:0:8:0: updated Direct-
> Access HP   MO0400JDVEU  PHYS DRV SSDSmartPathCap- En-
> Exp=0
> [ 2839.762267] hpsa :87:00.0: Acknowledging event: 0x8020 (HP
> SSD Smart Path configuration change)
> [ 2840.841290] hpsa :87:00.0: scsi 3:0:8:0: updated Direct-
> Access HP   MO0400JDVEU  PHYS DRV SSDSmartPathCap- En-
> Exp=0
> [ 2917.582653] hpsa :87:00.0: Acknowledging event: 0xc020 (HP
> SSD Smart Path configuration change)
> [ 2919.087191] hpsa :87:00.0: scsi 3:1:0:1: updated Direct-
> Access HP   LOGICAL VOLUME   RAID-5 SSDSmartPathCap+ En+
> Exp=1
> [ 2919.142527] hpsa :87:00.0: hpsa_figure_phys_disk_ptrs:
> [3:1:0:2] A phys disk component of LV is missing, turning off
> offload_enabled for LV.
> [ 2919.203915] hpsa :87:00.0: hpsa_figure_phys_disk_ptrs:
> [3:1:0:2] A phys disk component of LV is missing, turning off
> offload_enabled for LV.
> [ 2919.266921] hpsa :87:00.0: hpsa_figure_phys_disk_ptrs:
> [3:1:0:2] A phys disk component of LV is missing, turning off
> offload_enabled for LV.
> [ 2934.999629] hpsa :87:00.0: Acknowledging event: 0x4000 (HP
> SSD Smart Path state change)
> [ 2936.937333] hpsa :87:00.0: hpsa_figure_phys_disk_ptrs:
> [3:1:0:2] A phys disk component of LV is missing, turning off
> offload_enabled for LV.
> [ 2936.998707] hpsa :87:00.0: hpsa_figure_phys_disk_ptrs:
> [3:1:0:2] A phys disk component of LV is missing, turning off
> offload_enabled for LV.
> [ 2937.060101] hpsa :87:00.0: hpsa_figure_phys_disk_ptrs:
> [3:1:0:2] A phys disk component of LV is missing, turning off
> offload_enabled for LV.
> [ 3619.711122] sd 3:1:0:3: [sde] tag#436 FAILED Result:
> hostbyte=DID_OK driverbyte=DRIVER_SENSE

Re: [PATCH V3 8/8] scsi: megaraid: improve scsi_mq performance via .host_tagset

[Laurence Oberman]

On Wed, 2018-02-28 at 23:21 +0800, Ming Lei wrote:
> On Wed, Feb 28, 2018 at 08:28:48PM +0530, Kashyap Desai wrote:
> > Ming -
> > 
> > Quick testing on my setup -  Performance slightly degraded (4-5%
> > drop)for
> > megaraid_sas driver with this patch. (From 1610K IOPS it goes to
> > 1544K)
> > I confirm that after applying this patch, we have #queue = #numa
> > node.
> > 
> > ls -l
> > /sys/devices/pci:80/:80:02.0/:83:00.0/host10/target10:2
> > :23/10:
> > 2:23:0/block/sdy/mq
> > total 0
> > drwxr-xr-x. 18 root root 0 Feb 28 09:53 0
> > drwxr-xr-x. 18 root root 0 Feb 28 09:53 1
> 
> OK, thanks for your test.
> 
> As I mentioned to you, this patch should have improved performance on
> megaraid_sas, but the current slight degrade might be caused by
> scsi_host_queue_ready() in scsi_queue_rq(), I guess.
> 
> With .host_tagset enabled and use per-numa-node hw queue, request can
> be
> queued to lld more frequently/quick than single queue, then the cost
> of
> atomic_inc_return(>host_busy) may be increased much meantime,
> think about millions of such operations, and finally slight IOPS drop
> is observed when the hw queue depth becomes half of .can_queue.
> 
> > 
> > 
> > I would suggest to skip megaraid_sas driver changes using
> > shared_tagset
> > until and unless there is obvious gain. If overall interface of
> > using
> > shared_tagset is commit in kernel tree, we will investigate
> > (megaraid_sas
> > driver) in future about real benefit of using it.
> 
> I'd suggest to not merge it until it is proved that performance can
> be
> improved in real device.
> 
> I will try to work to remove the expensive atomic_inc_return(
> >host_busy)
> from scsi_queue_rq(), since it isn't needed for SCSI_MQ, once it is
> done, will
> ask you to test again.
> 
> 
> Thanks,
> Ming

I will test this here as well
I just put the Megaraid card in to my system here

Kashyap, do you have ssd's on the back-end and are you you using jbods
or virtual devices. Let me have your config.
I only have 6G sas shelves though.

Regards
Laurence

Re: Fwd: All scsi_debug devices in the scsi_debug driver share the same RAM space

[Laurence Oberman]

On Tue, 2018-02-13 at 21:14 -0500, Martin K. Petersen wrote:
> Lukas,
> 
> > I like the idea to have an option to separate the memory area for
> > different LUNs, while keeping the current behavior as default.
> 
> As the name implies, scsi_debug is mostly a tool for debugging the
> SCSI
> stack.
> 
> If you care about storing data, why not simply use target in loopback
> mode?
> 

Hi Martin

I think what Lukas was really looking for is what I started out
wanting.
Individual devices that could be put into configurations such as md-
raid for testing and could be used with fault injection.

In the end I had to use 5 (LIO or physical) devices and 1 scsi_debug
device to reproduce an issue for md-raid.
Target loopback was an option too.

I have fault injection in qla2xxx tcm already but not in the target
core.

I guess it time i look into that now so will proceed with that.

Regards
Laurence

Re: [PATCH V2 8/8] scsi: hpsa: use blk_mq to solve irq affinity issue

[Laurence Oberman]

On Mon, 2018-02-05 at 23:20 +0800, Ming Lei wrote:
> This patch uses .force_blk_mq to drive HPSA via SCSI_MQ, meantime
> maps
> each reply queue to blk_mq's hw queue, then .queuecommand can always
> choose the hw queue as the reply queue. And if no any online CPU is
> mapped to one hw queue, request can't be submitted to this hw queue
> at all, finally the irq affinity issue is solved.
> 
> Cc: Hannes Reinecke <h...@suse.de>
> Cc: Arun Easi <arun.e...@cavium.com>
> Cc: Omar Sandoval <osan...@fb.com>,
> Cc: "Martin K. Petersen" <martin.peter...@oracle.com>,
> Cc: James Bottomley <james.bottom...@hansenpartnership.com>,
> Cc: Christoph Hellwig <h...@lst.de>,
> Cc: Don Brace <don.br...@microsemi.com>
> Cc: Kashyap Desai <kashyap.de...@broadcom.com>
> Cc: Peter Rivera <peter.riv...@broadcom.com>
> Cc: Paolo Bonzini <pbonz...@redhat.com>
> Cc: Mike Snitzer <snit...@redhat.com>
> Tested-by: Laurence Oberman <lober...@redhat.com>
> Signed-off-by: Ming Lei <ming@redhat.com>
> ---
>  drivers/scsi/hpsa.c | 51 ++-
> 
>  1 file changed, 34 insertions(+), 17 deletions(-)
> 
> diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c
> index 443eabf63a9f..e517a4c74a28 100644
> --- a/drivers/scsi/hpsa.c
> +++ b/drivers/scsi/hpsa.c
> @@ -51,6 +51,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  #include 
>  #include 
>  #include "hpsa_cmd.h"
> @@ -956,6 +957,13 @@ static struct device_attribute
> *hpsa_shost_attrs[] = {
>  #define HPSA_NRESERVED_CMDS  (HPSA_CMDS_RESERVED_FOR_DRIVER +\
>    HPSA_MAX_CONCURRENT_PASSTHRUS)
>  
> +static int hpsa_map_queues(struct Scsi_Host *shost)
> +{
> +struct ctlr_info *h = shost_to_hba(shost);
> +
> +return blk_mq_pci_map_queues(>tag_set, h->pdev);
> +}
> +
>  static struct scsi_host_template hpsa_driver_template = {
>   .module = THIS_MODULE,
>   .name   = HPSA,
> @@ -974,10 +982,13 @@ static struct scsi_host_template
> hpsa_driver_template = {
>  #ifdef CONFIG_COMPAT
>   .compat_ioctl   = hpsa_compat_ioctl,
>  #endif
> + .map_queues = hpsa_map_queues,
>   .sdev_attrs = hpsa_sdev_attrs,
>   .shost_attrs = hpsa_shost_attrs,
>   .max_sectors = 1024,
>   .no_write_same = 1,
> + .force_blk_mq = 1,
> + .host_tagset = 1,
>  };
>  
>  static inline u32 next_command(struct ctlr_info *h, u8 q)
> @@ -1045,11 +1056,7 @@ static void set_performant_mode(struct
> ctlr_info *h, struct CommandList *c,
>   c->busaddr |= 1 | (h->blockFetchTable[c-
> >Header.SGList] << 1);
>   if (unlikely(!h->msix_vectors))
>   return;
> - if (likely(reply_queue == DEFAULT_REPLY_QUEUE))
> - c->Header.ReplyQueue =
> - raw_smp_processor_id() % h-
> >nreply_queues;
> - else
> - c->Header.ReplyQueue = reply_queue % h-
> >nreply_queues;
> + c->Header.ReplyQueue = reply_queue;
>   }
>  }
>  
> @@ -1063,10 +1070,7 @@ static void
> set_ioaccel1_performant_mode(struct ctlr_info *h,
>    * Tell the controller to post the reply to the queue for
> this
>    * processor.  This seems to give the best I/O throughput.
>    */
> - if (likely(reply_queue == DEFAULT_REPLY_QUEUE))
> - cp->ReplyQueue = smp_processor_id() % h-
> >nreply_queues;
> - else
> - cp->ReplyQueue = reply_queue % h->nreply_queues;
> + cp->ReplyQueue = reply_queue;
>   /*
>    * Set the bits in the address sent down to include:
>    *  - performant mode bit (bit 0)
> @@ -1087,10 +1091,7 @@ static void
> set_ioaccel2_tmf_performant_mode(struct ctlr_info *h,
>   /* Tell the controller to post the reply to the queue for
> this
>    * processor.  This seems to give the best I/O throughput.
>    */
> - if (likely(reply_queue == DEFAULT_REPLY_QUEUE))
> - cp->reply_queue = smp_processor_id() % h-
> >nreply_queues;
> - else
> - cp->reply_queue = reply_queue % h->nreply_queues;
> + cp->reply_queue = reply_queue;
>   /* Set the bits in the address sent down to include:
>    *  - performant mode bit not used in ioaccel mode 2
>    *  - pull count (bits 0-3)
> @@ -1109,10 +1110,7 @@ static void
> set_ioaccel2_performant_mode(struct ctlr_info *h,
>    * Tell the controller to post the reply to th

Re: All scsi_debug devices in the scsi_debug driver share the same RAM space

[Laurence Oberman]

On Sat, 2018-02-03 at 14:43 -0500, Laurence Oberman wrote:
> Hello Doug
> 
> I had emailed you earlier about this issue forgetting to copy others.
> 
> All test devices in the scsi_debug driver share the same ram space so
> we cannot really have individual devices for testing stuff like md-
> raid.
> 
> I bumped into this a few times already and I think it would be useful
> to make each device an individual RAM entry so we can create multiple
> unique devices.
> 
> Of course this means also now having to maybe start using attributes
> for each device do we can selectively choose which devices get the
> fault injections.
> 
> I know its adding complexity here but wanted to get input about the
> possibility of changing this.
> 
> For a particular fault injection case this morning I had to use 5
> regular drives and 1 scsi-debug drive to inject faults to reproduce
> an
> md-raid issue.
> 
> I also wanted a different start sector and range and sent an earlier
> patch for that.
> 
> So is this something worth considering ?
> If so, i will start working on it.
> 
> Regards
> Laurence
> 
> 
> 
> 

I chatted with Doug about this.

Seems that many of the use cases for scsi_debug involved creating high
count devices for testing etc. so having each with its own ram space
could land up chewing a bunch of memory.

I am going to drop this idea for now, but the patch I sent for the
choice of sector for MEDIUM errors and count is very useful so I hope
that one gets accepted.
That patch changes nothing if the two new parameters are not set so
should be low risk for acceptance.

Thanks
Laurence

All scsi_debug devices in the scsi_debug driver share the same RAM space

[Laurence Oberman]

Hello Doug

I had emailed you earlier about this issue forgetting to copy others.

All test devices in the scsi_debug driver share the same ram space so
we cannot really have individual devices for testing stuff like md-
raid.

I bumped into this a few times already and I think it would be useful
to make each device an individual RAM entry so we can create multiple
unique devices.

Of course this means also now having to maybe start using attributes
for each device do we can selectively choose which devices get the
fault injections.

I know its adding complexity here but wanted to get input about the
possibility of changing this.

For a particular fault injection case this morning I had to use 5
regular drives and 1 scsi-debug drive to inject faults to reproduce an
md-raid issue.

I also wanted a different start sector and range and sent an earlier
patch for that.

So is this something worth considering ?
If so, i will start working on it.

Regards
Laurence

[PATCH] scsi_debug: Add two new parameters to scsi_debug driver

[Laurence Oberman]

---
This patch adds two new parameters to the scsi_debug driver.
During various fault injection scenarios it would be useful to be able
to pick a specific starting sector and number of follow on sectors
where a MEDIUM ERROR for reads would be returned against a scsi-debug
device.

Right now this only works against sector 0x1234 and OPT_MEDIUM_ERR_NUM
follow on sectors.
However during testing of md-raid and other scenarios I wanted more
flexibility.

The idea is add 2 new parameters:
medium_error_start
medium_error_count

If medium_error_start is set then we don't use the default of
OPT_MEDIUM_ERR_ADDR, but use that set value.
If medium_error_count is set we use that value otherwise default to
OPT_MEDIUM_ERR_NUM.

Signed-off-by: Laurence Oberman <lober...@redhat.com>
Tested-by: Laurence Oberman <lober...@redhat.com>

 drivers/scsi/scsi_debug.c | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/scsi/scsi_debug.c b/drivers/scsi/scsi_debug.c
index a5986da..5a5aba0 100644
--- a/drivers/scsi/scsi_debug.c
+++ b/drivers/scsi/scsi_debug.c
@@ -616,6 +616,8 @@ enum sdeb_opcode_index {
 static int sdebug_lowest_aligned = DEF_LOWEST_ALIGNED;
 static int sdebug_max_luns = DEF_MAX_LUNS;
 static int sdebug_max_queue = SDEBUG_CANQUEUE; /* per submit queue */
+static unsigned int sdebug_medium_error_start = OPT_MEDIUM_ERR_ADDR;
+static int sdebug_medium_error_count = OPT_MEDIUM_ERR_NUM;
 static atomic_t retired_max_queue; /* if > 0 then was prior max_queue */
 static int sdebug_ndelay = DEF_NDELAY; /* if > 0 then unit is nanoseconds */
 static int sdebug_no_lun_0 = DEF_NO_LUN_0;
@@ -2712,8 +2714,8 @@ static int resp_read_dt0(struct scsi_cmnd *scp, struct 
sdebug_dev_info *devip)
}
 
if (unlikely((SDEBUG_OPT_MEDIUM_ERR & sdebug_opts) &&
-(lba <= (OPT_MEDIUM_ERR_ADDR + OPT_MEDIUM_ERR_NUM - 1)) &&
-((lba + num) > OPT_MEDIUM_ERR_ADDR))) {
+(lba <= (sdebug_medium_error_start + 
sdebug_medium_error_count - 1)) &&
+((lba + num) > sdebug_medium_error_start))) {
/* claim unrecoverable read error */
mk_sense_buffer(scp, MEDIUM_ERROR, UNRECOVERED_READ_ERR, 0);
/* set info field and valid bit for fixed descriptor */
@@ -4440,6 +4442,8 @@ static int schedule_resp(struct scsi_cmnd *cmnd, struct 
sdebug_dev_info *devip,
 module_param_named(lowest_aligned, sdebug_lowest_aligned, int, S_IRUGO);
 module_param_named(max_luns, sdebug_max_luns, int, S_IRUGO | S_IWUSR);
 module_param_named(max_queue, sdebug_max_queue, int, S_IRUGO | S_IWUSR);
+module_param_named(medium_error_start, sdebug_medium_error_start, int, S_IRUGO 
| S_IWUSR);
+module_param_named(medium_error_count, sdebug_medium_error_count, int, S_IRUGO 
| S_IWUSR);
 module_param_named(ndelay, sdebug_ndelay, int, S_IRUGO | S_IWUSR);
 module_param_named(no_lun_0, sdebug_no_lun_0, int, S_IRUGO | S_IWUSR);
 module_param_named(no_uld, sdebug_no_uld, int, S_IRUGO);
@@ -4497,6 +4501,8 @@ static int schedule_resp(struct scsi_cmnd *cmnd, struct 
sdebug_dev_info *devip,
 MODULE_PARM_DESC(lowest_aligned, "lowest aligned lba (def=0)");
 MODULE_PARM_DESC(max_luns, "number of LUNs per target to simulate(def=1)");
 MODULE_PARM_DESC(max_queue, "max number of queued commands (1 to max(def))");
+MODULE_PARM_DESC(medium_error_start, "starting sector number to return MEDIUM 
error");
+MODULE_PARM_DESC(medium_error_count, "count of sectors to return follow on 
MEDIUM error");
 MODULE_PARM_DESC(ndelay, "response delay in nanoseconds (def=0 -> ignore)");
 MODULE_PARM_DESC(no_lun_0, "no LU number 0 (def=0 -> have lun 0)");
 MODULE_PARM_DESC(no_uld, "stop ULD (e.g. sd driver) attaching (def=0))");
-- 
1.8.3.1

Request for input on new options for scsi_debug driver

[Laurence Oberman]

Hello

During various fault injection scenarios it would be useful to be able
to pick a specific starting sector and number of follow on sectors
where a MEDIUM ERROR for reads would be returned against a scsi-debug
device.

Right now this only works against sector 0x1234 and OPT_MEDIUM_ERR_NUM
follow on sectors.
However during testing of md-raid and other scenarios I wanted more
flexibility.

The idea is add 2 new parameters:
medium_error_start
medium_error_count

If medium_error_start is set then we don't use the default of
OPT_MEDIUM_ERR_ADDR, but use that set value.
If medium_error_count is set we use that value otherwise default to
OPT_MEDIUM_ERR_NUM.

Should I send a patch for this, wanted to find out if this would be
acceptable before I waste anybody's time having to review.

Regards
Laurence Oberman

Re: [dm-devel] [PATCH v5] blk-mq: introduce BLK_STS_DEV_RESOURCE

[Laurence Oberman]

On Tue, 2018-01-30 at 09:52 -0800, Bart Van Assche wrote:
> On 01/30/18 06:24, Mike Snitzer wrote:
> > +    *
> > +    * If driver returns BLK_STS_RESOURCE and
> > SCHED_RESTART
> > +    * bit is set, run queue after a delay to avoid IO
> > stalls
> > +    * that could otherwise occur if the queue is
> > idle.
> >      */
> > -   if (!blk_mq_sched_needs_restart(hctx) ||
> > +   needs_restart = blk_mq_sched_needs_restart(hctx);
> > +   if (!needs_restart ||
> >     (no_tag && list_empty_careful(
> > >dispatch_wait.entry)))
> >     blk_mq_run_hw_queue(hctx, true);
> > +   else if (needs_restart && (ret ==
> > BLK_STS_RESOURCE))
> > +   blk_mq_delay_run_hw_queue(hctx,
> > BLK_MQ_QUEUE_DELAY);
> >     }
> 
> If a request completes concurrently with execution of the above code 
> then the request completion will trigger a call of 
> blk_mq_sched_restart_hctx() and that call will clear the 
> BLK_MQ_S_SCHED_RESTART bit. If that bit is cleared before the above
> code 
> tests it then the above code will schedule an asynchronous call of 
> __blk_mq_run_hw_queue(). If the .queue_rq() call triggered by the
> new 
> queue run returns again BLK_STS_RESOURCE then the above code will be 
> executed again. In other words, a loop occurs. That loop will repeat
> as 
> long as the described race occurs. The current (kernel v4.15) block 
> layer behavior is simpler: only block drivers call 
> blk_mq_delay_run_hw_queue() and the block layer core never calls
> that 
> function. Hence that loop cannot occur with the v4.15 block layer
> core 
> and block drivers. A motivation of why that loop is preferred
> compared 
> to the current behavior (no loop) is missing. Does this mean that
> that 
> loop is a needless complication of the block layer core?
> 
> Sorry but I still prefer the v4.15 block layer approach because this 
> patch has in my view the following disadvantages:
> - It involves a blk-mq API change. API changes are always risky and
> need
>    some time to stabilize.
> - The delay after which to rerun the queue is moved from block layer
>    drivers into the block layer core. I think that's wrong because
> only
>    the block driver authors can make a good choice for this constant.
> - This patch makes block drivers harder to understand. Anyone who
> sees
>    return BLK_STS_RESOURCE / return BLK_STS_DEV_RESOURCE for the
> first
>    time will have to look up the meaning of these constants. An
> explicit
>    blk_mq_delay_run_hw_queue() call is easier to understand.
> - This patch makes the blk-mq core harder to understand because of
> the
>    loop mentioned above.
> - This patch does not fix any bugs nor makes block drivers easier to
>    read or to implement. So why is this patch considered useful?
> 
> Thanks,
> 
> Bart.

Hello Bart

What is the performance implication of your method versus this patch
above.
Is there more of a delay in your approach or in Ming's approach from
your own testing.
I guess it seems we will never get consensus on this so is it time to
take a vote. 

I respect and trust your inputs, you know that, but are you perhaps
prepared to accept the approach above and agree to it and if it turns
out to expose more issues it can be addressed later.

Is that not a better way to progress this because to me it looks like
you and Ming will continue to disagree on which is the better approach.

With much respect
Laurence

Re: [PATCH V3] blk-mq: introduce BLK_STS_DEV_RESOURCE

[Laurence Oberman]

On Sun, 2018-01-28 at 16:57 +, Bart Van Assche wrote:
> On Sat, 2018-01-27 at 23:58 -0500, Mike Snitzer wrote:
> > On Sat, Jan 27 2018 at 10:00pm -0500,
> > Bart Van Assche  wrote:
> > 
> > > On Sat, 2018-01-27 at 21:03 -0500, Mike Snitzer wrote:
> > > > You cannot even be forthcoming about the technical merit of a
> > > > change you
> > > > authored (commit 6077c2d70) that I'm left to clean up in the
> > > > face of
> > > > performance bottlenecks it unwittingly introduced?  If you were
> > > > being
> > > > honest: you'd grant that the random delay of 100ms is utterly
> > > > baseless
> > > > (not to mention that kicking the queue like you did is a
> > > > complete
> > > > hack).  So that 100ms delay is what my dm-4.16 commit is
> > > > talking about.
> > > 
> > > There are multiple errors in the above:
> > > 1. I have already explained in detail why commit 6077c2d70 is (a)
> > > correct
> > >    and (b) essential. See e.g. https://www.redhat.com/archives/dm
> > > -devel/2018-January/msg00168.html.
> > 
> > And you'd be wrong.  Again.  We've already established that commit
> > 6077c2d70 is _not_ essential.  Ming's V3, in conjunction with all
> > the
> > changes that already went into block and DM for 4.16, prove that.
> 
> What I wrote was right when commit 6077c2d70 got introduced. My
> explanation
> shows that at that time  was both correct and essential. Ming's v3 is
> in my
> opinion not relevant yet to this discussion because it introduces new
> bugs
> and so far no attempt has been made to address these bugs.
> 
> > > 2. With patch "blk-mq: Avoid that blk_mq_delay_run_hw_queue()
> > > introduces
> > >    unintended delays" applied, there is nothing to clean up
> > > anymore since
> > >    that patch eliminates the queue delays that were triggered by
> > >    blk_mq_delay_run_hw_queue().
> > 
> > The issue Ming fixed is that your random queue kicks break RESTART
> > on
> > dm-mq mpath.
> 
> That comment is too short to be comprehensible for anyone. Can you
> elaborate
> this further?
> 
> > > 3. You know that I'm honest. Suggesting that I'm not is wrong.
> > 
> > No, I trully do _not_ know you're always honest.  You've conflated
> > so
> > many details on this topic that it makes me have serious
> > doubts.  You're
> > lashing out so much, in defense of your dm_mq_queue_rq delayed
> > queue
> > kick, that you're missing there is a genuine generic blk-mq problem
> > that
> > is getting fixed in Ming's V3.
> > 
> > [ ... ]
> > 
> > Bart, the number of emails exchanged on this topic has exhausted
> > _everyone_.  Emotions get tense when things don't evolve despite
> > every
> > oppotunity for progress to be realized.  When people cling to
> > solutions
> > that are no longer relevant.  There is a very real need for
> > progress
> > here.  So either get out of the way or suggest a specific series of
> > change(s) that you feel better than Ming's V3.
> 
> If you and Ming really care about the approach in the patch at the
> start
> of this e-mail thread, what are you waiting for to address the
> technical
> shortcomings that I pointed out?
> 
> > With that, I'll also stop responding to your baiting now and
> > forevermore
> > (e.g. "maintainers should this.. and maintainers should not that"
> > or
> > "your employer is not investing adequately".  Next time you find
> > yourself typing drivel like that: spare us all and hit delete).
> 
> My opinion about what you wrote in this and the other e-mails you
> sent to
> me during the past months is as follows:
> 1. That I have always done my best to provide all the relevant
> technical
>    information.
> 2. That my focus was on the technical aspects of the discussion.
> 3. That you did not try to reach a consensus but rather to dominate
> the
>    discussion.
> 4. That the tone of your e-mails was very aggressive.
> 5. That you insulted me several times personally.
> 
> I believe that your behavior is a violation of the kernel code of
> conflict
> and sufficient to file a complaint with the TAB. From
> Documentation/process/code-of-conflict.rst:
> 
> "If however, anyone feels personally abused, threatened, or otherwise
> uncomfortable due to this process, that is not acceptable.  If so,
> please contact the Linux Foundation's Technical Advisory Board [ ...
> ]"
> 
> Additionally, since you are a U.S. Citizen, you should know that what
> you
> wrote in previous e-mails amounts to libel. A quote from a definition
> of
> libel: "to publish in print (including pictures), writing or
> broadcast
> through radio, television or film, an untruth about another which
> will do
> harm to that person or his/her reputation, by tending to bring the
> target
> into ridicule, hatred, scorn or contempt of others." You should be
> aware
> since I live in the U.S. that this makes it possible for me to sue
> you for
> defamation and to ask for a compensation.
> 
> Bart.

Folks

I think we need to all take some time, take a breath and lets see how
we can 

Re: [PATCH] qla2xxx: Avoid triggering undefined behavior in qla2x00_mbx_completion()

[Laurence Oberman]

On Thu, 2018-01-25 at 08:24 -0800, Bart Van Assche wrote:
> A left shift must shift less than the bit width of the left argument.
> Avoid triggering undefined behavior if ha->mbx_count == 32.
> 
> This patch avoids that UBSAN reports the following complaint:
> 
> UBSAN: Undefined behaviour in drivers/scsi/qla2xxx/qla_isr.c:275:14
> shift exponent 32 is too large for 32-bit type 'int'
> Call Trace:
>  dump_stack+0x4e/0x6c
>  ubsan_epilogue+0xd/0x3b
>  __ubsan_handle_shift_out_of_bounds+0x112/0x14c
>  qla2x00_mbx_completion+0x1c5/0x25d [qla2xxx]
>  qla2300_intr_handler+0x1ea/0x3bb [qla2xxx]
>  qla2x00_mailbox_command+0x77b/0x139a [qla2xxx]
>  qla2x00_mbx_reg_test+0x83/0x114 [qla2xxx]
>  qla2x00_chip_diag+0x354/0x45f [qla2xxx]
>  qla2x00_initialize_adapter+0x2c2/0xa4e [qla2xxx]
>  qla2x00_probe_one+0x1681/0x392e [qla2xxx]
>  pci_device_probe+0x10b/0x1f1
>  driver_probe_device+0x21f/0x3a4
>  __driver_attach+0xa9/0xe1
>  bus_for_each_dev+0x6e/0xb5
>  driver_attach+0x22/0x3c
>  bus_add_driver+0x1d1/0x2ae
>  driver_register+0x78/0x130
>  __pci_register_driver+0x75/0xa8
>  qla2x00_module_init+0x21b/0x267 [qla2xxx]
>  do_one_initcall+0x5a/0x1e2
>  do_init_module+0x9d/0x285
>  load_module+0x20db/0x38e3
>  SYSC_finit_module+0xa8/0xbc
>  SyS_finit_module+0x9/0xb
>  do_syscall_64+0x77/0x271
>  entry_SYSCALL64_slow_path+0x25/0x25
> 
> Reported-by: Meelis Roos <mr...@linux.ee>
> Signed-off-by: Bart Van Assche <bart.vanass...@wdc.com>
> Cc: Himanshu Madhani <himanshu.madh...@cavium.com>
> ---
>  drivers/scsi/qla2xxx/qla_isr.c | 6 --
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/scsi/qla2xxx/qla_isr.c
> b/drivers/scsi/qla2xxx/qla_isr.c
> index 16c43bd9bb83..79f538448364 100644
> --- a/drivers/scsi/qla2xxx/qla_isr.c
> +++ b/drivers/scsi/qla2xxx/qla_isr.c
> @@ -272,7 +272,8 @@ qla2x00_mbx_completion(scsi_qla_host_t *vha,
> uint16_t mb0)
>   struct device_reg_2xxx __iomem *reg = >iobase->isp;
>  
>   /* Read all mbox registers? */
> - mboxes = (1 << ha->mbx_count) - 1;
> + WARN_ON_ONCE(ha->mbx_count > 32);
> + mboxes = (1ULL << ha->mbx_count) - 1;
>   if (!ha->mcp)
>   ql_dbg(ql_dbg_async, vha, 0x5001, "MBX pointer
> ERROR.\n");
>   else
> @@ -2884,7 +2885,8 @@ qla24xx_mbx_completion(scsi_qla_host_t *vha,
> uint16_t mb0)
>   struct device_reg_24xx __iomem *reg = >iobase->isp24;
>  
>   /* Read all mbox registers? */
> - mboxes = (1 << ha->mbx_count) - 1;
> + WARN_ON_ONCE(ha->mbx_count > 32);
> + mboxes = (1ULL << ha->mbx_count) - 1;
>   if (!ha->mcp)
>   ql_dbg(ql_dbg_async, vha, 0x504e, "MBX pointer
> ERROR.\n");
>   else

This looks fine to me and I tested the earlier patch suggestion
supplied by Meelis.

Reviewed-by: Laurence Oberman <lober...@redhat.com>

Re: [PATCH] bnx2fc: Fix check in SCSI completion handler when reqeust has already timed out.

[Laurence Oberman]

On Wed, 2018-01-24 at 08:07 -0800, Chad Dupuis wrote:
> When a request times out we set the io_req flag BNX2FC_FLAG_IO_COMPL
> so
> that if a subsequent completion comes in on that task ID we will
> ignore
> it.  The issue is that in the check for this flag there is a missing
> return so we will continue to process a request which may have
> already
> been returned to the ownership of the SCSI layer.  This can cause
> unpredictable results.
> 
> Solution is to add in the missing return.
> 
> Signed-off-by: Chad Dupuis <chad.dup...@cavium.com>
> ---
>  drivers/scsi/bnx2fc/bnx2fc_io.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/scsi/bnx2fc/bnx2fc_io.c
> b/drivers/scsi/bnx2fc/bnx2fc_io.c
> index 8e2f767147cb..5a645b8b9af1 100644
> --- a/drivers/scsi/bnx2fc/bnx2fc_io.c
> +++ b/drivers/scsi/bnx2fc/bnx2fc_io.c
> @@ -1889,6 +1889,7 @@ void bnx2fc_process_scsi_cmd_compl(struct
> bnx2fc_cmd *io_req,
>   /* we will not receive ABTS response for this IO */
>   BNX2FC_IO_DBG(io_req, "Timer context finished
> processing "
>      "this scsi cmd\n");
> + return;
>   }
>  
>   /* Cancel the timeout_work, as we received IO completion */

Hello

This was added after the race showed up for the first issue we fixed.
Its been running stable in our lab and at the customer.

Reviewed-by: Laurence Oberman <lober...@redhat.com>
Tested-by:   Laurence Oberman <lober...@redhat.com>

Thanks
Laurence

Re: HP ProLiant DL360p Gen8 hangs with Linux 4.13+.

[Laurence Oberman]

On Mon, 2018-01-15 at 07:01 -0800, Hellwig, Christoph wrote:
> Laurence, I'm a little confused.  Is this the same issue we just
> fixed,
> or is this an issue showing up with the fix?
> 
> E.g. what kernel versions or trees are affected?

Hello Christoph

This showed up on a  combined tree of Mikes and Jens (4.15.0-
rc4.block.dm.4.16) I was testing this weekend but was not apparent on
the generic upstream 4.15-rc7 from Linus.
I have to admit that was puzzling me.

When I removed your commit the issue went away.

Ming has crafted a fix so that your original commit can remain in and I
am testing that now against the same tree that was hanging before.

Ming has a handle on the issue so I will report back after testing.

Kernel is building now

Thanks
Laurence

Re: HP ProLiant DL360p Gen8 hangs with Linux 4.13+.

[Laurence Oberman]

On Mon, 2018-01-15 at 20:17 +0800, Ming Lei wrote:
> On Sun, Jan 14, 2018 at 06:40:40PM -0500, Laurence Oberman wrote:
> > On Thu, 2018-01-04 at 14:32 -0800, Vinson Lee wrote:
> > > Hi.
> > > 
> > > HP ProLiant DL360p Gen8 with Smart Array P420i boots to the login
> > > prompt and hangs with Linux 4.13 or later. I cannot log in on
> > > console
> > > or SSH into the machine. Linux 4.12 and older boot fine.
> > > 
> > > 
> > 
> > ...
> > 
> > ...
> > 
> > This issue bit me for for two straight days.
> > I was testing Mike Snitzers combined tree and this commit crept
> > into
> > the latest combined tree.
> > 
> > commit 84676c1f21e8ff54befe985f4f14dc1edc10046b
> > Author: Christoph Hellwig <h...@lst.de>
> > Date:   Fri Jan 12 10:53:05 2018 +0800
> > 
> > genirq/affinity: assign vectors to all possible CPUs
> >    
> > Currently we assign managed interrupt vectors to all present
> > CPUs.  This
> > works fine for systems were we only online/offline CPUs.  But
> > in
> > case of
> > systems that support physical CPU hotplug (or the virtualized
> > version of
> > it) this means the additional CPUs covered for in the ACPI
> > tables
> > or on
> > the command line are not catered for.  To fix this we'd either
> > need
> > to
> > introduce new hotplug CPU states just for this case, or we can
> > start
> > assining vectors to possible but not present CPUs.
> >    
> > Reported-by: Christian Borntraeger <borntrae...@de.ibm.com>
> > Tested-by: Christian Borntraeger <borntrae...@de.ibm.com>
> > Tested-by: Stefan Haberland <s...@linux.vnet.ibm.com>
> > Fixes: 4b855ad37194 ("blk-mq: Create hctx for each present
> > CPU")
> > Cc: linux-ker...@vger.kernel.org
> > Cc: Thomas Gleixner <t...@linutronix.de>
> > Signed-off-by: Christoph Hellwig <h...@lst.de>
> > Signed-off-by: Jens Axboe <ax...@kernel.dk>
> > 
> > Reason I never thought about this being my reason for the latest
> > hang
> > is I have used Linus' tree all the way to 4.15-rc7 with no issues.
> > 
> > Vinson reporting it against 4.13 or later was not making sense
> > because
> > I had not seen the hang until this weekend.
> > 
> > I checked  and its in Linus's tree but its not an issue in the
> > generic
> > 4.15-rc7 for me.
> 
> Hi Laurence,
> 
> Wrt. your issue, I have investigated a bit and found that it is
> because
> one irq vector may be assigned to all offline CPUs, and it may not be
> same with Vinson's.
> 
> And the following patch can address your issue, I may prepare a
> formal
> version if no one objects this approach.
> 
> Thomas, Christoph, could you take a look this patch?
> 
> ---
>  kernel/irq/affinity.c | 69 +++
> 
>  1 file changed, 47 insertions(+), 22 deletions(-)
> 
> diff --git a/kernel/irq/affinity.c b/kernel/irq/affinity.c
> index a37a3b4b6342..dfc1f6a9c488 100644
> --- a/kernel/irq/affinity.c
> +++ b/kernel/irq/affinity.c
> @@ -94,6 +94,39 @@ static int get_nodes_in_cpumask(cpumask_var_t
> *node_to_possible_cpumask,
>   return nodes;
>  }
>  
> +/*
> + * Spread the affinity of @nmsk into @nr_vecs irq vectors, and the
> + * result is stored to @start_irqmsk.
> + */
> +static int irq_vecs_spread_affinity(struct cpumask *irqmsk,
> + int max_irqmsks,
> + struct cpumask *nmsk,
> + int max_ncpus)
> +{
> + int v, ncpus;
> + int vecs_to_assign, extra_vecs;
> +
> + /* Calculate the number of cpus per vector */
> + ncpus = cpumask_weight(nmsk);
> + vecs_to_assign = min(max_ncpus, ncpus);
> +
> + /* Account for rounding errors */
> + extra_vecs = ncpus - vecs_to_assign * (ncpus /
> vecs_to_assign);
> +
> + for (v = 0; v < min(max_irqmsks, vecs_to_assign); v++) {
> + int cpus_per_vec = ncpus / vecs_to_assign;
> +
> + /* Account for extra vectors to compensate rounding
> errors */
> + if (extra_vecs) {
> + cpus_per_vec++;
> + --extra_vecs;
> + }
> + irq_spread_init_one(irqmsk + v, nmsk, cpus_per_vec);
> + }
> +
> + return v;
> +}
> +
>  /**
>   * irq_create_affinity_masks - Create affinity masks for multiqueue
> spreading
>   * @nvecs:   The tot

Re: HP ProLiant DL360p Gen8 hangs with Linux 4.13+.

[Laurence Oberman]

On Thu, 2018-01-04 at 14:32 -0800, Vinson Lee wrote:
> Hi.
> 
> HP ProLiant DL360p Gen8 with Smart Array P420i boots to the login
> prompt and hangs with Linux 4.13 or later. I cannot log in on console
> or SSH into the machine. Linux 4.12 and older boot fine.
> 
> 
...

...

This issue bit me for for two straight days.
I was testing Mike Snitzers combined tree and this commit crept into
the latest combined tree.

commit 84676c1f21e8ff54befe985f4f14dc1edc10046b
Author: Christoph Hellwig 
Date:   Fri Jan 12 10:53:05 2018 +0800

genirq/affinity: assign vectors to all possible CPUs
   
Currently we assign managed interrupt vectors to all present
CPUs.  This
works fine for systems were we only online/offline CPUs.  But in
case of
systems that support physical CPU hotplug (or the virtualized
version of
it) this means the additional CPUs covered for in the ACPI tables
or on
the command line are not catered for.  To fix this we'd either need
to
introduce new hotplug CPU states just for this case, or we can
start
assining vectors to possible but not present CPUs.
   
Reported-by: Christian Borntraeger 
Tested-by: Christian Borntraeger 
Tested-by: Stefan Haberland 
Fixes: 4b855ad37194 ("blk-mq: Create hctx for each present CPU")
Cc: linux-ker...@vger.kernel.org
Cc: Thomas Gleixner 
Signed-off-by: Christoph Hellwig 
Signed-off-by: Jens Axboe 

Reason I never thought about this being my reason for the latest hang
is I have used Linus' tree all the way to 4.15-rc7 with no issues.

Vinson reporting it against 4.13 or later was not making sense because
I had not seen the hang until this weekend.

I checked  and its in Linus's tree but its not an issue in the generic
4.15-rc7 for me.

Anyway, its going to possibly bite anybody running HP DL servers with
HPSA boot devices. I have not tried the workaround below.
>From Vinsons message repeated here

"The machine still hangs with Linux 4.15-rc6.

I did a bisect. The hang is introduced with Linux 4.13-rc1 commit
c5cb83bb337c25caae995d992d1cdf9b317f83de "genirq/cpuhotplug: Handle
managed IRQs on CPU hotplug".

There is a startup script that disables hyperthreading by offlining
sibling CPUs.

for CPU in $(cut -s -d, -f2
$SYS_PATH/cpu*/topology/thread_siblings_list | sort -un); do
echo 0 > /sys/devices/system/cpu/cpu$CPU/online
done

If the above script is not run, the machine does not hang with Linux
4.13.

Cheers,
Vinson"

Thanks
Laurence

Re: HP ProLiant DL360p Gen8 hangs with Linux 4.13+.

[Laurence Oberman]

On Fri, 2018-01-05 at 16:32 +, Bart Van Assche wrote:
> On Thu, 2018-01-04 at 14:32 -0800, Vinson Lee wrote:
> > HP ProLiant DL360p Gen8 with Smart Array P420i boots to the login
> > prompt and hangs with Linux 4.13 or later. I cannot log in on
> > console
> > or SSH into the machine. Linux 4.12 and older boot fine.
> > 
> > I see these messages on the console.
> > 
> > [  242.843206] INFO: task scsi_eh_2:465 blocked for more than 120
> > seconds.
> > [  242.877835]   Not tainted 4.15.0-041500rc6-generic
> > #201712312330
> 
> It seems like something got stuck in the block layer. The traditional
> way to
> debug this is to analyze the information that is available under
> /sys/kernel/debug/block. However, since login is not possible we
> can't use
> that approach. Would it be possible for you to check whether this has
> been
> resolved in kernel v4.15-rc6, and if not, bisect this?
> 
> Thanks,
> 
> Bart.

One of the ways to debug this given its an HP DL380 is follow this.

1. Boot the working kernel
2, ensure kdump is activated and running on boot.
3. add these to the /etc/sysctl.conf file

   kernel.panic_on_io_nmi = 1
   kernel.panic_on_unrecovered_nmi = 1
   kernel.unknown_nmi_panic = 1

4, Once hung after boot, go to the ILO page under admin/diagnostics and
press the Virtual NMI button to generate a vmcore

When you have a vmcore, I will give you a place to upload it to so I
can look at it

Re: [PATCH] bnx2fc: Fix hung task messages when a cleanup response is not received during abort.

[Laurence Oberman]

uest */
>   set_bit(BNX2FC_FLAG_IO_CLEANUP, _req-
> >req_flags);
> - goto out;
> + goto done;
>   }
>  
>   /* Cancel the current timer running on this io_req */
> @@ -1221,7 +1238,11 @@ int bnx2fc_eh_abort(struct scsi_cmnd *sc_cmd)
>   }
>   spin_unlock_bh(>tgt_lock);
>  
> - wait_for_completion(_req->tm_done);
> + /* Wait 2 * RA_TOV + 1 to be sure timeout function hasn't
> fired */
> + time_left = wait_for_completion_timeout(_req->tm_done,
> + (2 * rp->r_a_tov + 1) * HZ);
> + if (time_left)
> + BNX2FC_IO_DBG(io_req, "Timed out in eh_abort waiting
> for tm_done");
>  
>   spin_lock_bh(>tgt_lock);
>   io_req->wait_for_comp = 0;
> @@ -1233,8 +1254,12 @@ int bnx2fc_eh_abort(struct scsi_cmnd *sc_cmd)
>   /* Let the scsi-ml try to recover this command */
>   printk(KERN_ERR PFX "abort failed, xid = 0x%x\n",
>      io_req->xid);
> + /*
> +  * Cleanup firmware residuals before returning
> control back
> +  * to SCSI ML.
> +  */
>   rc = bnx2fc_abts_cleanup(io_req);
> - goto out;
> + goto done;
>   } else {
>   /*
>        * We come here even when there was a race condition
> @@ -1249,7 +1274,6 @@ int bnx2fc_eh_abort(struct scsi_cmnd *sc_cmd)
>  done:
>   /* release the reference taken in eh_abort */
>   kref_put(_req->refcount, bnx2fc_cmd_release);
> -out:
>   spin_unlock_bh(>tgt_lock);
>   return rc;
>  }

We experienced this at a major customer and provided this patch after
working with Chad and backported to RHEL as a test kernel.

Its been stable while running a test kernel and the changes look good
ate least to me.

Reviewed-by: Laurence Oberman <lober...@redhat.com>
Tested-by:   Laurence Oberman <lober...@redhat.com>

Thanks
Laurence

Re: [PATCH] SCSI: don't get target/host busy_count in scsi_mq_get_budget()

[Laurence Oberman]

On Wed, 2017-11-08 at 10:57 -0700, Jens Axboe wrote:
> On 11/08/2017 09:41 AM, Bart Van Assche wrote:
> > On Tue, 2017-11-07 at 20:06 -0700, Jens Axboe wrote:
> > > At this point, I have no idea what Bart's setup looks like. Bart,
> > > it
> > > would be REALLY helpful if you could tell us how you are
> > > reproducing
> > > your hang. I don't know why this has to be dragged out.
> > 
> > Hello Jens,
> > 
> > It is a disappointment to me that you have allowed Ming to evaluate
> > other
> > approaches than reverting "blk-mq: don't handle TAG_SHARED in
> > restart". That
> > patch namely replaces an algorithm that is trusted by the community
> > with an
> > algorithm of which even Ming acknowledged that it is racy. A quote
> > from [1]:
> > "IO hang may be caused if all requests are completed just before
> > the current
> > SCSI device is added to shost->starved_list". I don't know of any
> > way to fix
> > that race other than serializing request submission and completion
> > by adding
> > locking around these actions, which is something we don't want.
> > Hence my
> > request to revert that patch.
> 
> I was reluctant to revert it, in case we could work out a better way
> of
> doing it. As I mentioned in the other replies, it's not exactly the
> prettiest or most efficient. However, since we currently don't have
> a good solution for the issue, I'm fine with reverting that patch.
> 
> > Regarding the test I run, here is a summary of what I mentioned in
> > previous
> > e-mails:
> > * I modified the SRP initiator such that the SCSI target queue
> > depth is
> >   reduced to one by setting starget->can_queue to 1 from inside
> >   scsi_host_template.target_alloc.
> > * With that modified SRP initiator I run the srp-test software as
> > follows
> >   until something breaks:
> >   while ./run_tests -f xfs -d -e deadline -r 60; do :; done
> 
> What kernel options are needed? Where do I download everything I
> need?
> 
> In other words, would it be possible to do a fuller guide for getting
> this setup and running?
> 
> I'll run my simple test case as well, since it's currently breaking
> basically everywhere.
> 
> > Today a system with at least one InfiniBand HCA is required to run
> > that test.
> > When I have the time I will post the SRP initiator and target
> > patches on the
> > linux-rdma mailing list that make it possible to run that test
> > against the
> > SoftRoCE driver (drivers/infiniband/sw/rxe). The only hardware
> > required to
> > use that driver is an Ethernet adapter.
> 
> OK, I guess I can't run it then... I'll have to rely on your testing.

Hello 

I agree with Bart in this case, we should revert this.
My test-bed is tied up and I have not been able to give it back to Ming
so he could follow up on Bart's last update.

Right now its safer to revert.

Thanks
Laurence
> 

[PATCH] Added notes to scsi-parameters.txt to show how to pass parameters on kernel line

[Laurence Oberman]

---
 Documentation/scsi/scsi-parameters.txt | 8 
 1 file changed, 8 insertions(+)

diff --git a/Documentation/scsi/scsi-parameters.txt 
b/Documentation/scsi/scsi-parameters.txt
index 8477655..5b87a11 100644
--- a/Documentation/scsi/scsi-parameters.txt
+++ b/Documentation/scsi/scsi-parameters.txt
@@ -116,3 +116,11 @@ parameters may be changed at runtime by the command
 
wd33c93=[HW,SCSI]
See header of drivers/scsi/wd33c93.c.
+
+Additional notes on passing parameters on the command line:
+
+It's not intuitively obvious how to pass certain parameters on the boot line.
+For parameters such as scsi_dev_flags, you need to pass these to the scsi_mod 
as 
+the following example for the scsi_dev_flags. 
+
+On kernel line add scsi_mod.dev_flags=LIO-ORG:thin2:0x800
-- 
1.8.3.1

Re: Patch: add to Documentation/scsi for scsi_dev_flags and command line passing

[Laurence Oberman]

Indeed it was Evolution and then linux-scsi rejected my fix.

Trying again, for some reason it thought I had html

Signed-off-by: Laurence Oberman <lober...@redhat.com>

--- a/Documentation/scsi/scsi-parameters.txt
+++ b/Documentation/scsi/scsi-parameters.txt
@@ -116,3 +116,11 @@ parameters may be changed at runtime by the command

  wd33c93= [HW,SCSI]
  See header of drivers/scsi/wd33c93.c.
+
+Additional notes on passing parameters on the command line:
+
+It's not intuitively obvious how to pass certain parameters on the boot line.
+For parameters such as scsi_dev_flags, you need to pass to the scsi_mod as
+follows.
+On kernel line add scsi_mod.dev_flags=LIO-ORG:thin2:0x800
+

On Wed, Oct 18, 2017 at 1:46 PM, Randy Dunlap <rdun...@infradead.org> wrote:
> On 10/18/17 10:28, Laurence Oberman wrote:
>> I just noticed that
>> Was taken straight out of git after changing the Doc file.
>> Let me see why git did that
>>
>
> It's more likely that Evolution did it.
>
>>
>> On Wed, Oct 18, 2017 at 1:25 PM, Randy Dunlap <rdun...@infradead.org 
>> <mailto:rdun...@infradead.org>> wrote:
>>
>> On 10/18/17 10:13, Laurence Oberman wrote:
>> > Tired of forgetting about this so documented it for others so they also
>> > don't spend unnecessary time figuring this out.
>> >
>> > Signed-off-by: Laurence Oberman <lober...@redhat.com 
>> <mailto:lober...@redhat.com>>
>> >
>> > diff --git a/Documentation/scsi/scsi-parameters.txt
>> > b/Documentation/scsi/scsi-parameters.txt
>> > index 8477655..c9e9ce8 100644
>> > --- a/Documentation/scsi/scsi-parameters.txt
>> > +++ b/Documentation/scsi/scsi-parameters.txt
>> > @@ -116,3 +116,9 @@ parameters may be changed at runtime by the command
>> >
>> >   wd33c93=[HW,SCSI]
>> >   See header of drivers/scsi/wd33c93.c.
>> > +
>> > +Additional notes on passing parameters on the command line.
>>
>>  line:
>>
>> > +Its not intuitively obvious how to pass certain parameters on the boot
>>
>>It's
>>
>> > line.
>> > +For parameters such as scsi_dev_flags, you need to pass to the
>> > scsi_mod as follows
>>
>>as follows:
>>
>> > +scsi_mod.dev_flags=LIO-ORG:thin2:0x8000
>> > +
>> > --
>> > 1.8.3.1
>>
>>
>> Also the patch has some weird line breaks.
>>
>>
>> --
>> ~Randy
>>
>>
>
>
> --
> ~Randy

Patch: add to Documentation/scsi for scsi_dev_flags and command line passing

[Laurence Oberman]

Tired of forgetting about this so documented it for others so they also
don't spend unnecessary time figuring this out.

Signed-off-by: Laurence Oberman <lober...@redhat.com>

diff --git a/Documentation/scsi/scsi-parameters.txt
b/Documentation/scsi/scsi-parameters.txt
index 8477655..c9e9ce8 100644
--- a/Documentation/scsi/scsi-parameters.txt
+++ b/Documentation/scsi/scsi-parameters.txt
@@ -116,3 +116,9 @@ parameters may be changed at runtime by the command
 
    wd33c93=[HW,SCSI]
    See header of drivers/scsi/wd33c93.c.
+
+Additional notes on passing parameters on the command line.
+Its not intuitively obvious how to pass certain parameters on the boot
line.
+For parameters such as scsi_dev_flags, you need to pass to the
scsi_mod as follows
+scsi_mod.dev_flags=LIO-ORG:thin2:0x8000
+
-- 
1.8.3.1

Re: [PATCH] scsi: sd: Implement blacklist option for WRITE SAME w/ UNMAP

[Laurence Oberman]

On Tue, 2017-10-17 at 10:26 -0400, Laurence Oberman wrote:
> On Fri, 2017-09-29 at 10:01 -0400, Laurence Oberman wrote:
> > On Fri, 2017-09-29 at 09:21 -0400, Martin K. Petersen wrote:
> > > Laurence,
> > > 
> > > > I am testing this but its not being picked up so I want to know
> > > > if
> > > > I
> > > > have the kernel command line wrong here.
> > > > 
> > > > scsi_dev_flags=LIO-ORG:thin2:0x8000
> > > > 
> > > > What am I doing wrong to pass the BLIST flags.
> > > 
> > > This worked for me:
> > > 
> > > [root@kvm ~]# echo "Linux:scsi_debug:0x8000" >
> > > /proc/scsi/device_info
> > > [root@kvm ~]# grep Linux /proc/scsi/device_info 
> > > 'Linux   ' 'scsi_debug  ' 0x8000
> > > [root@kvm ~]# modprobe scsi_debug unmap_max_blocks=10
> > > unmap_max_desc=1 write_same_length=20 lbpws=1
> > > [root@kvm ~]# lsblk -D
> > > NAME DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
> > > sda 0  512B   5K 0
> > > 
> > > (With the caveat that I tweaked scsi_debug to report the UNMAP
> > > parameters despite lbpu being 0).
> > > 
> > 
> > OK, Thanks, that is working now and I pick up the correct size now.
> > Its going to be very useful for these corner case array
> > inconsistencies.
> > 
> > Tested-by: Laurence Oberman <lober...@redhat.com>
> > 
> > Sep 29 09:56:11 localhost kernel: scsi 1:0:0:50: Direct-
> > Access LIO-
> > ORG  thin24.0  PQ: 0 ANSI: 5
> > Sep 29 09:56:11 localhost kernel: scsi 1:0:0:50: alua: supports
> > implicit and explicit TPGS
> > Sep 29 09:56:11 localhost kernel: scsi 1:0:0:50: alua: device
> > naa.6001405f7aa27ca453f4381a00f22ea6 port group 0 rel port 2
> > Sep 29 09:56:11 localhost kernel: sd 1:0:0:50: Attached scsi
> > generic
> > sg64 type 0
> > Sep 29 09:56:11 localhost kernel: RHDEBUG: unmap_limit_for_ws set
> > by
> > kernel flag for case SD_LBP_WS16
> > Sep 29 09:56:11 localhost kernel: sd 1:0:0:50: [sdbl] 8192 512-
> > byte 
> > logical blocks: (41.9 GB/39.1 GiB)
> > Sep 29 09:56:11 localhost kernel: sd 1:0:0:50: [sdbl] Write Protect
> > is
> > off
> > Sep 29 09:56:11 localhost kernel: sd 1:0:0:50: [sdbl] Write cache:
> > enabled, read cache: enabled, supports DPO and FUA
> > Sep 29 09:56:11 localhost kernel: sd 1:0:0:50: alua: transition
> > timeout
> > set to 60 seconds
> 
> 
> Hi Martin
> 
> We have the code accepted for the patch above and its working fine
> with
> echo after boot as you already know.
> 
> However I am still fighting with trying to pass this on the kernel
> command line. We need to be able to do this as a dynamic method for
> adding devices to the list so that the list is populated prior to the
> device scan.
> 
> Using scsi_dev_flags=LIO-ORG:thin2:0x8000 on the kernel line is
> ignored and not filled in.
> Its apparent to me that we have no longer have code to actually copy
> the string from the kernel line after boot.
> 
> I ran some tests and added a couple of printk;s to see if we have any
> capture and its NULL.
> 
> So when did this stop working, is what I am now chasing
> 
> [1.524595] RHDEBUG:In scsi_init_devinfo scsi_dev_flags=
> [1.524705] RHDEBUG: In scsi_init_devinfo error=0
> 
> We have this in  drivers/scsi/scsi_devinfo.c
> 
> module_param_string(dev_flags, scsi_dev_flags,
> sizeof(scsi_dev_flags),
> 0);
> MODULE_PARM_DESC(dev_flags,
> 
>  "Given scsi_dev_flags=vendor:model:flags[,v:m:f] add
> black/white"
>  " list entries for vendor and model with an integer value of
> flags"
>  " to the scsi device info list");
> 
> and we have:
> 
> /**
>  * scsi_init_devinfo - set up the dynamic device list.
>  *
>  * Description:
>  *  Add command line entries from scsi_dev_flags, then add
>  *  scsi_static_device_list entries to the scsi device info list.
>  */
> int __init scsi_init_devinfo(void)
> {
> #ifdef CONFIG_SCSI_PROC_FS
> struct proc_dir_entry *p;
> #endif
> int error, i;
> 
> printk("RHDEBUG:In scsi_init_devinfo
> scsi_dev_flags=%s\n",scsi_dev_flags);
> 
> error = scsi_dev_info_add_list(SCSI_DEVINFO_GLOBAL, NULL);
> printk("RHDEBUG: In scsi_init_devinfo error=%d\n",error);
> if (error) {
> printk("RHDEBUG: In scsi_init_devinfo, calling
> scsi_dev_info_add_list returning with error=%

Re: [PATCH] scsi: sd: Implement blacklist option for WRITE SAME w/ UNMAP

[Laurence Oberman]

On Fri, 2017-09-29 at 10:01 -0400, Laurence Oberman wrote:
> On Fri, 2017-09-29 at 09:21 -0400, Martin K. Petersen wrote:
> > Laurence,
> > 
> > > I am testing this but its not being picked up so I want to know
> > > if
> > > I
> > > have the kernel command line wrong here.
> > > 
> > > scsi_dev_flags=LIO-ORG:thin2:0x8000
> > > 
> > > What am I doing wrong to pass the BLIST flags.
> > 
> > This worked for me:
> > 
> > [root@kvm ~]# echo "Linux:scsi_debug:0x8000" >
> > /proc/scsi/device_info
> > [root@kvm ~]# grep Linux /proc/scsi/device_info 
> > 'Linux   ' 'scsi_debug  ' 0x8000
> > [root@kvm ~]# modprobe scsi_debug unmap_max_blocks=10
> > unmap_max_desc=1 write_same_length=20 lbpws=1
> > [root@kvm ~]# lsblk -D
> > NAME DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
> > sda 0  512B   5K 0
> > 
> > (With the caveat that I tweaked scsi_debug to report the UNMAP
> > parameters despite lbpu being 0).
> > 
> 
> OK, Thanks, that is working now and I pick up the correct size now.
> Its going to be very useful for these corner case array
> inconsistencies.
> 
> Tested-by: Laurence Oberman <lober...@redhat.com>
> 
> Sep 29 09:56:11 localhost kernel: scsi 1:0:0:50: Direct-
> Access LIO-
> ORG  thin24.0  PQ: 0 ANSI: 5
> Sep 29 09:56:11 localhost kernel: scsi 1:0:0:50: alua: supports
> implicit and explicit TPGS
> Sep 29 09:56:11 localhost kernel: scsi 1:0:0:50: alua: device
> naa.6001405f7aa27ca453f4381a00f22ea6 port group 0 rel port 2
> Sep 29 09:56:11 localhost kernel: sd 1:0:0:50: Attached scsi generic
> sg64 type 0
> Sep 29 09:56:11 localhost kernel: RHDEBUG: unmap_limit_for_ws set by
> kernel flag for case SD_LBP_WS16
> Sep 29 09:56:11 localhost kernel: sd 1:0:0:50: [sdbl] 8192 512-
> byte 
> logical blocks: (41.9 GB/39.1 GiB)
> Sep 29 09:56:11 localhost kernel: sd 1:0:0:50: [sdbl] Write Protect
> is
> off
> Sep 29 09:56:11 localhost kernel: sd 1:0:0:50: [sdbl] Write cache:
> enabled, read cache: enabled, supports DPO and FUA
> Sep 29 09:56:11 localhost kernel: sd 1:0:0:50: alua: transition
> timeout
> set to 60 seconds


Hi Martin

We have the code accepted for the patch above and its working fine with
echo after boot as you already know.

However I am still fighting with trying to pass this on the kernel
command line. We need to be able to do this as a dynamic method for
adding devices to the list so that the list is populated prior to the
device scan.

Using scsi_dev_flags=LIO-ORG:thin2:0x8000 on the kernel line is
ignored and not filled in.
Its apparent to me that we have no longer have code to actually copy
the string from the kernel line after boot.

I ran some tests and added a couple of printk;s to see if we have any
capture and its NULL.

So when did this stop working, is what I am now chasing

[1.524595] RHDEBUG:In scsi_init_devinfo scsi_dev_flags=
[1.524705] RHDEBUG: In scsi_init_devinfo error=0

We have this in  drivers/scsi/scsi_devinfo.c

module_param_string(dev_flags, scsi_dev_flags, sizeof(scsi_dev_flags),
0);
MODULE_PARM_DESC(dev_flags,

 "Given scsi_dev_flags=vendor:model:flags[,v:m:f] add
black/white"
 " list entries for vendor and model with an integer value of
flags"
 " to the scsi device info list");

and we have:

/**
 * scsi_init_devinfo - set up the dynamic device list.
 *
 * Description:
 *  Add command line entries from scsi_dev_flags, then add
 *  scsi_static_device_list entries to the scsi device info list.
 */
int __init scsi_init_devinfo(void)
{
#ifdef CONFIG_SCSI_PROC_FS
struct proc_dir_entry *p;
#endif
int error, i;

printk("RHDEBUG:In scsi_init_devinfo
scsi_dev_flags=%s\n",scsi_dev_flags);

error = scsi_dev_info_add_list(SCSI_DEVINFO_GLOBAL, NULL);
printk("RHDEBUG: In scsi_init_devinfo error=%d\n",error);
if (error) {
printk("RHDEBUG: In scsi_init_devinfo, calling
scsi_dev_info_add_list returning with error=%d\n",error);
return error;
}

error = scsi_dev_info_list_add_str(scsi_dev_flags);
if (error) {
printk("RHDEBUG: In scsi_init_devinfo, calling
scsi_info_list_add returning with error=%d\n",error);
goto out;
}

for (i = 0; scsi_static_device_list[i].vendor; i++) {
error = scsi_dev_info_list_add(1 /* compatibile */,
scsi_static_device_list[i].vendor,
scsi_static_device_list[i].model,
NULL,
scsi_static_device_list[i].flags);
i

Re: [PATCH] scsi: sd: Implement blacklist option for WRITE SAME w/ UNMAP

[Laurence Oberman]

On Fri, 2017-09-29 at 09:21 -0400, Martin K. Petersen wrote:
> Laurence,
> 
> > I am testing this but its not being picked up so I want to know if
> > I
> > have the kernel command line wrong here.
> > 
> > scsi_dev_flags=LIO-ORG:thin2:0x8000
> > 
> > What am I doing wrong to pass the BLIST flags.
> 
> This worked for me:
> 
> [root@kvm ~]# echo "Linux:scsi_debug:0x8000" >
> /proc/scsi/device_info
> [root@kvm ~]# grep Linux /proc/scsi/device_info 
> 'Linux   ' 'scsi_debug  ' 0x8000
> [root@kvm ~]# modprobe scsi_debug unmap_max_blocks=10
> unmap_max_desc=1 write_same_length=20 lbpws=1
> [root@kvm ~]# lsblk -D
> NAME DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
> sda 0  512B   5K 0
> 
> (With the caveat that I tweaked scsi_debug to report the UNMAP
> parameters despite lbpu being 0).
> 

OK, Thanks, that is working now and I pick up the correct size now.
Its going to be very useful for these corner case array
inconsistencies.

Tested-by: Laurence Oberman <lober...@redhat.com>

Sep 29 09:56:11 localhost kernel: scsi 1:0:0:50: Direct-Access LIO-
ORG  thin24.0  PQ: 0 ANSI: 5
Sep 29 09:56:11 localhost kernel: scsi 1:0:0:50: alua: supports
implicit and explicit TPGS
Sep 29 09:56:11 localhost kernel: scsi 1:0:0:50: alua: device
naa.6001405f7aa27ca453f4381a00f22ea6 port group 0 rel port 2
Sep 29 09:56:11 localhost kernel: sd 1:0:0:50: Attached scsi generic
sg64 type 0
Sep 29 09:56:11 localhost kernel: RHDEBUG: unmap_limit_for_ws set by
kernel flag for case SD_LBP_WS16
Sep 29 09:56:11 localhost kernel: sd 1:0:0:50: [sdbl] 8192 512-byte 
logical blocks: (41.9 GB/39.1 GiB)
Sep 29 09:56:11 localhost kernel: sd 1:0:0:50: [sdbl] Write Protect is
off
Sep 29 09:56:11 localhost kernel: sd 1:0:0:50: [sdbl] Write cache:
enabled, read cache: enabled, supports DPO and FUA
Sep 29 09:56:11 localhost kernel: sd 1:0:0:50: alua: transition timeout
set to 60 seconds

Re: [PATCH] scsi: sd: Implement blacklist option for WRITE SAME w/ UNMAP

[Laurence Oberman]

On Wed, 2017-09-27 at 21:35 -0400, Martin K. Petersen wrote:
> SBC-4 states:
> 
>   "A MAXIMUM UNMAP LBA COUNT field set to a non-zero value indicates
> the
>    maximum number of LBAs that may be unmapped by an UNMAP command"
> 
>   "A MAXIMUM WRITE SAME LENGTH field set to a non-zero value
> indicates
>    the maximum number of contiguous logical blocks that the device
> server
>    allows to be unmapped or written in a single WRITE SAME command."
> 
> Despite the spec being clear on the topic, some devices incorrectly
> expect WRITE SAME commands with the UNMAP bit set to be limited to
> the
> value reported in MAXIMUM UNMAP LBA COUNT in the Block Limits VPD.
> 
> Implement a blacklist option that can be used to accommodate devices
> with this behavior.
> 
> Reported-by: Bill Kuzeja 
> Reported-by: Ewan D. Milne 
> Signed-off-by: Martin K. Petersen 
> ---
>  drivers/scsi/scsi_scan.c|  3 +++
>  drivers/scsi/sd.c   | 16 
>  include/scsi/scsi_device.h  |  1 +
>  include/scsi/scsi_devinfo.h |  1 +
>  4 files changed, 17 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
> index e7818afeda2b..15590a063ad9 100644
> --- a/drivers/scsi/scsi_scan.c
> +++ b/drivers/scsi/scsi_scan.c
> @@ -956,6 +956,9 @@ static int scsi_add_lun(struct scsi_device *sdev,
> unsigned char *inq_result,
>   if (*bflags & BLIST_NO_DIF)
>   sdev->no_dif = 1;
>  
> + if (*bflags & BLIST_UNMAP_LIMIT_WS)
> + sdev->unmap_limit_for_ws = 1;
> +
>   sdev->eh_timeout = SCSI_DEFAULT_EH_TIMEOUT;
>  
>   if (*bflags & BLIST_TRY_VPD_PAGES)
> diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
> index b18ba3235900..347be7580181 100644
> --- a/drivers/scsi/sd.c
> +++ b/drivers/scsi/sd.c
> @@ -715,13 +715,21 @@ static void sd_config_discard(struct scsi_disk
> *sdkp, unsigned int mode)
>   break;
>  
>   case SD_LBP_WS16:
> - max_blocks = min_not_zero(sdkp->max_ws_blocks,
> -   (u32)SD_MAX_WS16_BLOCKS);
> + if (sdkp->device->unmap_limit_for_ws)
> + max_blocks = sdkp->max_unmap_blocks;
> + else
> + max_blocks = sdkp->max_ws_blocks;
> +
> + max_blocks = min_not_zero(max_blocks,
> (u32)SD_MAX_WS16_BLOCKS);
>   break;
>  
>   case SD_LBP_WS10:
> - max_blocks = min_not_zero(sdkp->max_ws_blocks,
> -   (u32)SD_MAX_WS10_BLOCKS);
> + if (sdkp->device->unmap_limit_for_ws)
> + max_blocks = sdkp->max_unmap_blocks;
> + else
> + max_blocks = sdkp->max_ws_blocks;
> +
> + max_blocks = min_not_zero(max_blocks,
> (u32)SD_MAX_WS10_BLOCKS);
>   break;
>  
>   case SD_LBP_ZERO:
> diff --git a/include/scsi/scsi_device.h b/include/scsi/scsi_device.h
> index 82e93ee94708..67c5a9f223f7 100644
> --- a/include/scsi/scsi_device.h
> +++ b/include/scsi/scsi_device.h
> @@ -192,6 +192,7 @@ struct scsi_device {
>   unsigned no_dif:1;  /* T10 PI (DIF) should be disabled
> */
>   unsigned broken_fua:1;  /* Don't set FUA bit
> */
>   unsigned lun_in_cdb:1;  /* Store LUN bits in
> CDB[1] */
> + unsigned unmap_limit_for_ws:1;  /* Use the UNMAP limit
> for WRITE SAME */
>  
>   atomic_t disk_events_disable_depth; /* disable depth for
> disk events */
>  
> diff --git a/include/scsi/scsi_devinfo.h
> b/include/scsi/scsi_devinfo.h
> index 9592570e092a..36b03013d629 100644
> --- a/include/scsi/scsi_devinfo.h
> +++ b/include/scsi/scsi_devinfo.h
> @@ -29,5 +29,6 @@
>  #define BLIST_TRY_VPD_PAGES  0x1000 /* Attempt to read VPD
> pages */
>  #define BLIST_NO_RSOC0x2000 /* don't try to
> issue RSOC */
>  #define BLIST_MAX_1024   0x4000 /* maximum 1024
> sector cdb length */
> +#define BLIST_UNMAP_LIMIT_WS 0x8000 /* Use UNMAP limit
> for WRITE SAME */
>  
>  #endif

Hello Martin
I am testing this but its not being picked up so I want to know if I
have the kernel command line wrong here.

scsi_dev_flags=LIO-ORG:thin2:0x8000

Device is here
[   16.853083] scsi 4:0:0:50: Direct-Access LIO-
ORG  thin24.0  PQ: 0 ANSI: 5

Have a couple of printk's in now to see if I see the flags and they
dont trigger

case SD_LBP_WS16:
if (sdkp->device->unmap_limit_for_ws) {
max_blocks = sdkp->max_unmap_blocks;
printk("RHDEBUG: unmap_limit_for_ws set by
kernel flag for case SD_LBP_WS16\n");
}
else
max_blocks = sdkp->max_ws_blocks;

max_blocks = min_not_zero(max_blocks,
(u32)SD_MAX_WS16_BLOCKS);
break;

case SD_LBP_WS10:
if (sdkp->device->unmap_limit_for_ws) {
   

Re: Sniffing FC traffic

[Laurence Oberman]

On Fri, Aug 18, 2017 at 8:37 AM, Thomas Glanzmann  wrote:
> Hello,
> I would like to create a setup that allows me to sniff FC traffic. Is it
> possible with Linux or can someone recommend a setup that works. I want
> to avoid buying a 120kUSD fabric analyzer.
>
> Cheers,
> Thomas

There is no way to do this using adapters and generic F/C with Linux as the O/S.
The ability to enable debugging in the F/C drivers will expose some of
the internals but there is no way to sniff directly as far as I am
aware.

Many switches allow port level tracing facilities but inline sniffing
using Linux and generic hosts is not possible.

We have Finisars for inline tracing when we have to debug host and
fabric issues.

Software based FCOE using libfc and the Intel cards for example will
allow Wireshark tracing but that is encapsulated F/C in Ethernet
packets hence the Wireshark ability.

Thanks
Laurence

Re: HBA recommended as FC target

[Laurence Oberman]

On Fri, Aug 18, 2017 at 8:36 AM, Thomas Glanzmann  wrote:
> Hello,
> I look for a FC HBA that works with the Linux target. Can somone
> recommend a HBA type to me?
>
> Cheers,
> Thomas

Hello

Any of the Qlogic qla24xx or qla25xx and higher that allow you to
disable initiator mode will work.
I use qla25xx 8Gbit in all my Target arrays.

Thanks
Laurence

Re: [v4.13-rc BUG] system lockup when running big buffered write(4M) to IB SRP via mpath

[Laurence Oberman]

On 08/08/2017 10:28 PM, Laurence Oberman wrote:

On Tue, 2017-08-08 at 20:11 -0400, Laurence Oberman wrote:

On Tue, 2017-08-08 at 22:17 +0800, Ming Lei wrote:

Hi Guys,
  
Laurence and I see a system lockup issue when running

concurrent

big buffered write(4M bytes) to IB SRP on v4.13-rc3.
  
1 how to reproduce
  
1) setup IB_SRR & multi path
  
  	#./start_opensm.sh

#./start_srp.sh
  
  	# cat start_opensm.sh

#!/bin/bash
rmmod ib_srpt
opensm -F opensm.1.conf &
opensm -F opensm.2.conf &
  
  	# cat start_srp.sh

run_srp_daemon  -f /etc/ddn/srp_daemon.conf -R 30 -T 10

-t 7000

-ance -i mlx5_0 -p 1 1>/root/srp1.log 2>&1 &
run_srp_daemon  -f /etc/ddn/srp_daemon.conf -R 30 -T 10

-t 7000

-ance -i mlx5_1 -p 1 1>/root/srp2.log 2>&1 &

  
2) run the following test script
  
  	#./test-mp.sh
  
  	#cat test-mp.sh

#!/bin/sh
DEVS="360014051186d24cc27b4d04994e3
36001405b2b5c6c24c084b6fa4d55da2f
36001405b26ebe76dcb94a489f6f245f8"
  
  	for i in $DEVS; do

for j in `seq 0 15`; do
./hammer_write.sh $i 1>/dev/null 2>&1 &
done
done
  
  	#cat hammer_write.sh

#!/bin/bash
while true; do
dd if=/dev/zero of=/dev/mapper/$1 bs=4096k

count=800

done
  
  
2 lockup log

[root@ibclient ~]# ./test-mp.sh
[root@ibclient ~]# [  169.095494] perf: interrupt took too long
(2575

2500), lowering kernel.perf_event_max_sample_rate to 77000


  
[  178.569124] perf: interrupt took too long (3249 > 3218),

lowering
kernel.perf_event_max_sample_rate to 61000
[  190.586624] perf: interrupt took too long (4066 > 4061),
lowering
kernel.perf_event_max_sample_rate to 49000
[  210.381836] perf: interrupt took too long (5083 > 5082),
lowering
kernel.perf_event_max_sample_rate to 39000
[  240.498659] perf: interrupt took too long (6373 > 6353),
lowering
kernel.perf_event_max_sample_rate to 31000
  
[root@ibclient ~]#

[root@ibclient ~]# [  292.252795] perf: interrupt took too long
(8006

7966), lowering kernel.perf_event_max_sample_rate to 24000


  
[  410.354443] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  410.432725] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  410.499530] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  410.564952] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  410.644493] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  410.698091] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  410.755175] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  410.796784] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  410.837690] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  410.878743] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  463.476823] multipath_clone_and_map: 32 callbacks suppressed
[  463.506773] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  463.586752] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  463.656880] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  463.698919] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  463.723572] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  463.751820] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  463.781110] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  463.828267] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  463.856997] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  463.885998] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  468.572060] multipath_clone_and_map: 5201 callbacks

suppressed

[  468.602191] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  468.655761] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  468.697540] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  468.738610] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  468.779597] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  468.820705] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  468.862712] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  468.904662] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  468.945132] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  468.985128] device-mapper: multipath: blk_get_request()

returned

-11 - requeuing
[  473.643013] multipath_clone_and_map: 6569 callbacks

suppresse

Re: [v4.13-rc BUG] system lockup when running big buffered write(4M) to IB SRP via mpath

[Laurence Oberman]

On Tue, 2017-08-08 at 20:11 -0400, Laurence Oberman wrote:
> > On Tue, 2017-08-08 at 22:17 +0800, Ming Lei wrote:
> > > > Hi Guys,
> > > > 
> > > > Laurence and I see a system lockup issue when running
> > concurrent
> > > > big buffered write(4M bytes) to IB SRP on v4.13-rc3.
> > > > 
> > > > 1 how to reproduce
> > > > 
> > > > 1) setup IB_SRR & multi path
> > > > 
> > > > #./start_opensm.sh
> > > > #./start_srp.sh 
> > > > 
> > > > # cat start_opensm.sh 
> > > > #!/bin/bash
> > > > rmmod ib_srpt
> > > > opensm -F opensm.1.conf &
> > > > opensm -F opensm.2.conf &
> > > > 
> > > > # cat start_srp.sh
> > > > run_srp_daemon  -f /etc/ddn/srp_daemon.conf -R 30 -T 10
> > -t 7000
> > > > -ance -i mlx5_0 -p 1 1>/root/srp1.log 2>&1 &
> > > > run_srp_daemon  -f /etc/ddn/srp_daemon.conf -R 30 -T 10
> > -t 7000
> > > > -ance -i mlx5_1 -p 1 1>/root/srp2.log 2>&1 &
> > > > 
> > > > 
> > > > 2) run the following test script
> > > > 
> > > > #./test-mp.sh
> > > >  
> > > > #cat test-mp.sh 
> > > > #!/bin/sh
> > > > DEVS="360014051186d24cc27b4d04994e3
> > > > 36001405b2b5c6c24c084b6fa4d55da2f
> > > > 36001405b26ebe76dcb94a489f6f245f8"
> > > > 
> > > > for i in $DEVS; do
> > > > for j in `seq 0 15`; do
> > > > ./hammer_write.sh $i 1>/dev/null 2>&1 &
> > > > done
> > > > done
> > > > 
> > > > #cat hammer_write.sh
> > > > #!/bin/bash
> > > > while true; do
> > > > dd if=/dev/zero of=/dev/mapper/$1 bs=4096k
> > count=800 
> > > > done
> > > > 
> > > > 
> > > > 2 lockup log
> > > > [root@ibclient ~]# ./test-mp.sh 
> > > > [root@ibclient ~]# [  169.095494] perf: interrupt took too long
> > > > (2575
> > > > > > 2500), lowering kernel.perf_event_max_sample_rate to 77000
> > 
> > > > 
> > > > [  178.569124] perf: interrupt took too long (3249 > 3218),
> > > > lowering
> > > > kernel.perf_event_max_sample_rate to 61000
> > > > [  190.586624] perf: interrupt took too long (4066 > 4061),
> > > > lowering
> > > > kernel.perf_event_max_sample_rate to 49000
> > > > [  210.381836] perf: interrupt took too long (5083 > 5082),
> > > > lowering
> > > > kernel.perf_event_max_sample_rate to 39000
> > > > [  240.498659] perf: interrupt took too long (6373 > 6353),
> > > > lowering
> > > > kernel.perf_event_max_sample_rate to 31000
> > > > 
> > > > [root@ibclient ~]# 
> > > > [root@ibclient ~]# [  292.252795] perf: interrupt took too long
> > > > (8006
> > > > > > 7966), lowering kernel.perf_event_max_sample_rate to 24000
> > 
> > > > 
> > > > [  410.354443] device-mapper: multipath: blk_get_request()
> > returned
> > > > -11 - requeuing
> > > > [  410.432725] device-mapper: multipath: blk_get_request()
> > returned
> > > > -11 - requeuing
> > > > [  410.499530] device-mapper: multipath: blk_get_request()
> > returned
> > > > -11 - requeuing
> > > > [  410.564952] device-mapper: multipath: blk_get_request()
> > returned
> > > > -11 - requeuing
> > > > [  410.644493] device-mapper: multipath: blk_get_request()
> > returned
> > > > -11 - requeuing
> > > > [  410.698091] device-mapper: multipath: blk_get_request()
> > returned
> > > > -11 - requeuing
> > > > [  410.755175] device-mapper: multipath: blk_get_request()
> > returned
> > > > -11 - requeuing
> > > > [  410.796784] device-mapper: multipath: blk_get_request()
> > returned
> > > > -11 - requeuing
> > > > [  410.837690] device-mapper: multipath: blk_get_request()
> > returned
> > > > -11 - requeuing
> > > > [  410.878743] device-mapper: multipath: blk_get_request()
> > returned
> > &

Re: [v4.13-rc BUG] system lockup when running big buffered write(4M) to IB SRP via mpath

[Laurence Oberman]

On Tue, 2017-08-08 at 22:17 +0800, Ming Lei wrote:
> Hi Guys,
> 
> Laurence and I see a system lockup issue when running concurrent
> big buffered write(4M bytes) to IB SRP on v4.13-rc3.
> 
> 1 how to reproduce
> 
> 1) setup IB_SRR & multi path
> 
>   #./start_opensm.sh
>   #./start_srp.sh 
> 
>   # cat start_opensm.sh 
>   #!/bin/bash
>   rmmod ib_srpt
>   opensm -F opensm.1.conf &
>   opensm -F opensm.2.conf &
> 
>   # cat start_srp.sh
>   run_srp_daemon  -f /etc/ddn/srp_daemon.conf -R 30 -T 10 -t 7000
> -ance -i mlx5_0 -p 1 1>/root/srp1.log 2>&1 &
>   run_srp_daemon  -f /etc/ddn/srp_daemon.conf -R 30 -T 10 -t 7000
> -ance -i mlx5_1 -p 1 1>/root/srp2.log 2>&1 &
>   
> 
> 2) run the following test script
> 
>   #./test-mp.sh
>  
>   #cat test-mp.sh 
>   #!/bin/sh
>   DEVS="360014051186d24cc27b4d04994e3
> 36001405b2b5c6c24c084b6fa4d55da2f 36001405b26ebe76dcb94a489f6f245f8"
> 
>   for i in $DEVS; do
>   for j in `seq 0 15`; do
>   ./hammer_write.sh $i 1>/dev/null 2>&1 &
>   done
>   done
> 
>   #cat hammer_write.sh
>   #!/bin/bash
>   while true; do
>   dd if=/dev/zero of=/dev/mapper/$1 bs=4096k count=800 
>   done
> 
> 
> 2 lockup log
> [root@ibclient ~]# ./test-mp.sh 
> [root@ibclient ~]# [  169.095494] perf: interrupt took too long (2575
> > 2500), lowering kernel.perf_event_max_sample_rate to 77000
> [  178.569124] perf: interrupt took too long (3249 > 3218), lowering
> kernel.perf_event_max_sample_rate to 61000
> [  190.586624] perf: interrupt took too long (4066 > 4061), lowering
> kernel.perf_event_max_sample_rate to 49000
> [  210.381836] perf: interrupt took too long (5083 > 5082), lowering
> kernel.perf_event_max_sample_rate to 39000
> [  240.498659] perf: interrupt took too long (6373 > 6353), lowering
> kernel.perf_event_max_sample_rate to 31000
> 
> [root@ibclient ~]# 
> [root@ibclient ~]# [  292.252795] perf: interrupt took too long (8006
> > 7966), lowering kernel.perf_event_max_sample_rate to 24000
> [  410.354443] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  410.432725] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  410.499530] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  410.564952] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  410.644493] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  410.698091] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  410.755175] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  410.796784] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  410.837690] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  410.878743] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  463.476823] multipath_clone_and_map: 32 callbacks suppressed
> [  463.506773] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  463.586752] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  463.656880] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  463.698919] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  463.723572] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  463.751820] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  463.781110] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  463.828267] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  463.856997] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  463.885998] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  468.572060] multipath_clone_and_map: 5201 callbacks suppressed
> [  468.602191] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  468.655761] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  468.697540] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  468.738610] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  468.779597] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  468.820705] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  468.862712] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  468.904662] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  468.945132] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  468.985128] device-mapper: multipath: blk_get_request() returned
> -11 - requeuing
> [  473.643013] multipath_clone_and_map: 6569 callbacks suppressed
> [  473.673466] device-mapper: multipath: blk_get_request() 

Re: device support in hpsa, was: Re: OOPS from cciss_ioctl in 4.12+git

[Laurence Oberman]

On 07/07/2017 02:08 PM, Christoph Hellwig wrote:

On Fri, Jul 07, 2017 at 11:42:38AM -0400, Laurence Oberman wrote:

What happens when  hpsa_allow_any=1 with the Smart Array 64xx
It should probe.


But only if it has a HP vendor ID as far as I can tell.  We'd
still need to add the compaq ids so that these controllers get
probed.  But maybe it's time to add them and flip the hpsa_allow_any
default (maybe conditionally on a config option?) and mark cciss
deprecated.


Agreed, I vote yes.

Re: device support in hpasa, was: Re: OOPS from cciss_ioctl in 4.12+git

[Laurence Oberman]

On 07/07/2017 11:03 AM, Jens Axboe wrote:

On 07/07/2017 09:00 AM, Christoph Hellwig wrote:

On Thu, Jul 06, 2017 at 12:55:04PM +0300, Meelis Roos wrote:

Also we're trying to move people away from the cciss driver, can you
check if the hpsa SCSI driver works for you as well?


I have older adapter:

Compaq Computer Corporation Smart Array 64xx [0e11:0046] (rev 01)

That does not seem to be supported by hpsa AFAICS.


Looks like.  Although hpsa has support for various SA5 controllers
it seems like it decided to skip all Compaq branded controllers.

As far as I can tell we could simply add support for those to
hpsa.  Ccing hpsa folks to figure out if that's the case.


Pretty sure Hannes had a patch he tested for that, he talked about
that back at LSFMM earlier this year. Hannes?



What happens when  hpsa_allow_any=1 with the Smart Array 64xx
It should probe.

Re: [PATCH RESEND] scsi: Add STARGET_CREATED_REMOVE state to scsi_target_state

[Laurence Oberman]

(struct device *dev)
spin_lock_irqsave(shost->host_lock, flags);
list_for_each_entry(starget, >__targets, siblings) {
if (starget->state == STARGET_DEL ||
-   starget->state == STARGET_REMOVE)
+   starget->state == STARGET_REMOVE ||
+   starget->state == STARGET_CREATED_REMOVE)
continue;
if (starget->dev.parent == dev || >dev == dev) {
kref_get(>reap_ref);
-   starget->state = STARGET_REMOVE;
+   if (starget->state == STARGET_CREATED)
+   starget->state = STARGET_CREATED_REMOVE;
+   else
+   starget->state = STARGET_REMOVE;
spin_unlock_irqrestore(shost->host_lock, flags);
__scsi_remove_target(starget);
scsi_target_reap(starget);
diff --git a/include/scsi/scsi_device.h b/include/scsi/scsi_device.h
index d3fb98f..b41ee9d 100644
--- a/include/scsi/scsi_device.h
+++ b/include/scsi/scsi_device.h
@@ -248,6 +248,7 @@ enum scsi_target_state {
STARGET_CREATED = 1,
STARGET_RUNNING,
STARGET_REMOVE,
+   STARGET_CREATED_REMOVE,
STARGET_DEL,
  };
  



This patch (similar patch modified for RHEL) has been tested and 
validated here for multiple customers.


Reviewed-by: Laurence Oberman <lober...@redhat.com>
Tested-by:   Laurence Oberman <lober...@redhat.com>

Thanks
Laurence

Re: [PATCH v2] lpfc: Fix panic on BFS configuration.

[Laurence Oberman]

- Original Message -
> From: jsmart2...@gmail.com
> To: linux-scsi@vger.kernel.org, sta...@vger.kernel.org
> Cc: jthumsh...@suse.de, linux-n...@lists.infradead.org, emi...@redhat.com, 
> "James Smart" ,
> "Dick Kennedy" , "James Smart" 
> 
> Sent: Wednesday, April 26, 2017 3:19:26 PM
> Subject: [PATCH v2] lpfc: Fix panic on BFS configuration.
> 
> From: James Smart 
> 
> To select the appropriate shost template, the driver is issuing
> a mailbox command to retrieve the wwn. Turns out the sending of
> the command precedes the reset of the function.  On SLI-4 adapters,
> this is inconsequential as the mailbox command location is specified
> by dma via the BMBX register. However, on SLI-3 adapters, the
> location of the mailbox command submission area changes. When the
> function is first powered on or reset, the cmd is submitted via PCI
> bar memory. Later the driver changes the function config to use
> host memory and DMA. The request to start a mailbox command is the
> same, a simple doorbell write, regardless of submission area.
> So.. if there has not been a boot driver run against the adapter,
> the mailbox command works as defaults are ok. But, if the boot
> driver has configured the card and, and if no platform pci
> function/slot reset occurs as the os starts, the mailbox command
> will fail. The SLI-3 device will use the stale boot driver dma
> location. This can cause PCI eeh errors.
> 
> Fix is to reset the sli-3 function before sending the
> mailbox command, thus synchronizing the function/driver on mailbox
> location.
> 
> This issue was introduced by this patch:
> http://www.spinics.net/lists/linux-scsi/msg105908.html
> which is in the stable pools with commit id:
> 96418b5e2c8867da3279d877f5d1ffabfe460c3d
> 
> This patch was cut against the scsi.git tree, misc branch and should
> be pulled in via the scsi tree.
> 
> This patch needs to be applied to the stable trees where ever the
> introducing patch exists.
> 
> Signed-off-by: Dick Kennedy 
> Signed-off-by: James Smart 
> Cc: sta...@vger.kernel.org
> ---
>  drivers/scsi/lpfc/lpfc_crtn.h |  1 +
>  drivers/scsi/lpfc/lpfc_init.c |  7 +++
>  drivers/scsi/lpfc/lpfc_sli.c  | 19 ---
>  3 files changed, 20 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/scsi/lpfc/lpfc_crtn.h b/drivers/scsi/lpfc/lpfc_crtn.h
> index 843dd73..4295ef1 100644
> --- a/drivers/scsi/lpfc/lpfc_crtn.h
> +++ b/drivers/scsi/lpfc/lpfc_crtn.h
> @@ -289,6 +289,7 @@ int lpfc_selective_reset(struct lpfc_hba *);
>  void lpfc_reset_barrier(struct lpfc_hba *);
>  int lpfc_sli_brdready(struct lpfc_hba *, uint32_t);
>  int lpfc_sli_brdkill(struct lpfc_hba *);
> +int lpfc_sli_chipset_init(struct lpfc_hba *);
>  int lpfc_sli_brdreset(struct lpfc_hba *);
>  int lpfc_sli_brdrestart(struct lpfc_hba *);
>  int lpfc_sli_hba_setup(struct lpfc_hba *);
> diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
> index 0ee429d..4b47708 100644
> --- a/drivers/scsi/lpfc/lpfc_init.c
> +++ b/drivers/scsi/lpfc/lpfc_init.c
> @@ -1422,6 +1422,13 @@ lpfc_handle_deferred_eratt(struct lpfc_hba *phba)
>   psli->sli_flag &= ~LPFC_SLI_ACTIVE;
>   spin_unlock_irq(>hbalock);
>  
> + if (phba->sli_rev < LPFC_SLI_REV4) {
> + /* Reset the port first */
> + lpfc_sli_brdrestart(phba);
> + rc = lpfc_sli_chipset_init(phba);
> + if (rc)
> + return (uint64_t)-1;
> + }
>  
>   /*
>* Firmware stops when it triggred erratt. That could cause the I/Os
> diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c
> index e43e5e2..0296c47 100644
> --- a/drivers/scsi/lpfc/lpfc_sli.c
> +++ b/drivers/scsi/lpfc/lpfc_sli.c
> @@ -4203,13 +4203,16 @@ lpfc_sli_brdreset(struct lpfc_hba *phba)
>   /* Reset HBA */
>   lpfc_printf_log(phba, KERN_INFO, LOG_SLI,
>   "0325 Reset HBA Data: x%x x%x\n",
> - phba->pport->port_state, psli->sli_flag);
> + (phba->pport) ? phba->pport->port_state : 0,
> + psli->sli_flag);
>  
>   /* perform board reset */
>   phba->fc_eventTag = 0;
>   phba->link_events = 0;
> - phba->pport->fc_myDID = 0;
> - phba->pport->fc_prevDID = 0;
> + if (phba->pport) {
> + phba->pport->fc_myDID = 0;
> + phba->pport->fc_prevDID = 0;
> + }
>  
>   /* Turn off parity checking and serr during the physical reset */
>   pci_read_config_word(phba->pcidev, PCI_COMMAND, _value);
> @@ -4335,7 +4338,8 @@ lpfc_sli_brdrestart_s3(struct lpfc_hba *phba)
>   /* Restart HBA */
>   lpfc_printf_log(phba, KERN_INFO, LOG_SLI,
>   "0337 Restart HBA Data: x%x x%x\n",
> - phba->pport->port_state, psli->sli_flag);
> + 

tcm_qla2xxx and target mode not stable at all on 4.10+

[Laurence Oberman]

Hello

I have had issues with the target mode working since moving to 4.10+.
I am using a qla25xx card at 8Gbit
Latest testing with 4.11 RC6 sees the same issue.

Going back to 4.10.4 I can map targets but when I use my jammer I get into 
other issues.

Its rock solid on 4.9 with the jammer.

I will be going back now and testing from 4.9+ and later and report back but 
wanted
to make the folks on this list aware of it. 

We loop in 

[ 1654.895663] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue ().


Log

[ 1617.374267] Rounding down aligned max_sectors from 4294967295 to 4294967288
[ 1617.627264] ignoring deprecated emulate_fua_read attribute
[ 1617.654380] ignoring deprecated emulate_dpo attribute
[ 1622.822301] Rounding down aligned max_sectors from 4294967295 to 4294967288
[ 1627.522678] Rounding down aligned max_sectors from 4294967295 to 4294967288
[ 1632.209884] Rounding down aligned max_sectors from 4294967295 to 4294967288
[ 1636.895416] Rounding down aligned max_sectors from 4294967295 to 4294967288
[ 1642.271975] Rounding down aligned max_sectors from 4294967295 to 4294967288
[ 1645.478152] Rounding down aligned max_sectors from 4294967295 to 4294967288
[ 1648.609477] Rounding down aligned max_sectors from 4294967295 to 4294967288
[ 1650.192484] Rounding down aligned max_sectors from 4294967295 to 4294967288
[ 1651.771651] Rounding down aligned max_sectors from 4294967295 to 4294967288
[ 1653.350823] Rounding down aligned max_sectors from 4294967295 to 4294967288
[ 1653.552445] qla2xxx [:0a:00.0]-00af:4: Performing ISP error recovery - 
ha=8817c8e0.
[ 1654.802967] qla2xxx [:0a:00.0]-500a:4: LOOP UP detected (4 Gbps).
[ 1654.857970] qla2xxx [:0a:00.0]-5032:4: Invalid completion handle (0) -- 
timed-out.
[ 1654.895663] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue ().
[ 1654.933551] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (0501).
[ 1654.971256] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (5c03).
[ 1655.009369] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (6400).
[ 1655.047874] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (1c01).
[ 1655.086435] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (5834).
[ 1655.124292] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (6407).
[ 1655.162292] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (1c01).
[ 1655.200088] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (0002).
[ 1655.237939] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (5838).
[ 1684.982064] qla2xxx [:0a:00.0]-1119:4: MBX Command timeout for cmd 27, 
iocontrol=8 jiffies=1001521c0 mb[0-3]=[0x4000 0x0 0xc8f1 0xc8] mb7 0x17 
host_status 0x14 hccr 0x40
[ 1685.070674] qla2xxx [:0a:00.0]-d001:4: Firmware dump saved to temp 
buffer (4/c9001bf31000), dump status flags (0x3f).
[ 1685.126059] qla2xxx [:0a:00.0]-8033:4: Unable to reinitialize FCE (256).
[ 1685.172066] qla2xxx [:0a:00.0]-8034:4: Unable to reinitialize EFT (258).
[ 1715.253118] qla2xxx [:0a:00.0]-1015:4: cmd=0x69, waited 30049 msecs
[ 1715.284860] qla2xxx [:0a:00.0]-1119:4: MBX Command timeout for cmd 69, 
iocontrol=0 jiffies=10015981f mb[0-3]=[0x4001 0x4953 0x5020 0x2532] mb7 0x1 
host_status 0x40018002 hccr 0x0
[ 1715.362716] qla2xxx [:0a:00.0]-d009:4: Firmware has been previously 
dumped (c9001bf31000) -- ignoring request.
[ 1715.414674] qla2xxx [:0a:00.0]-101e:4: Mailbox cmd timeout occurred, 
cmd=0x69, mb[0]=0x69. Scheduling ISP abort 
[ 1715.414679] qla2xxx [:0a:00.0]-00af:4: Performing ISP error recovery - 
ha=8817c8e0.
[ 1716.713072] qla2xxx [:0a:00.0]-500a:4: LOOP UP detected (4 Gbps).
[ 1716.803096] qla2xxx [:0a:00.0]-803b:4: Firmware ready  FAILED .
[ 1717.202630] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (1c01).
[ 1717.210468] qla2xxx [:0a:00.1]-00af:6: Performing ISP error recovery - 
ha=8817c88e.
[ 1717.282428] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (5834).
[ 1717.31] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (6407).
[ 1717.359180] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (1c01).
[ 1717.396856] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (0002).
[ 1717.434517] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (5838).
[ 1717.472756] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (0502).
[ 1717.511664] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (4c0c).
[ 1717.549481] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (580c).
[ 1717.587457] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 
handle/queue (0105).
[ 1717.625668] qla2xxx [:0a:00.0]-5030:4: Error entry - invalid 

Re: [PATCH] sd: Consider max_xfer_blocks if opt_xfer_blocks is unusable

[Laurence Oberman]

- Original Message -
> From: "Fam Zheng" <f...@redhat.com>
> To: linux-ker...@vger.kernel.org
> Cc: "Martin K. Petersen" <martin.peter...@oracle.com>, f...@redhat.com, 
> linux-scsi@vger.kernel.org, "James E.J.
> Bottomley" <j...@linux.vnet.ibm.com>
> Sent: Monday, March 27, 2017 10:18:31 AM
> Subject: [PATCH] sd: Consider max_xfer_blocks if opt_xfer_blocks is unusable
> 
> If device reports a small max_xfer_blocks and a zero opt_xfer_blocks, we
> end up using BLK_DEF_MAX_SECTORS, which is wrong and r/w of that size
> may get error.
> 
> Fixes: ca369d51b3e ("block/sd: Fix device-imposed transfer length limits")
> Signed-off-by: Fam Zheng <f...@redhat.com>
> ---
>  drivers/scsi/sd.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
> index fcfeddc..e2e21ab 100644
> --- a/drivers/scsi/sd.c
> +++ b/drivers/scsi/sd.c
> @@ -2957,6 +2957,7 @@ static int sd_revalidate_disk(struct gendisk *disk)
>   rw_max = logical_to_sectors(sdp, sdkp->opt_xfer_blocks);
>   } else
>   rw_max = BLK_DEF_MAX_SECTORS;
> + rw_max = min_not_zero(rw_max, dev_max);
>  
>   /* Combine with controller limits */
>   q->limits.max_sectors = min(rw_max, queue_max_hw_sectors(q));
> --
> 2.9.3
> 
> 
Looks good to me and addresses the issue we faced.
FInal confirmation this change is OK to come from Martin or James.

Reviewed-by: Laurence Oberman <lober...@redhat.com>

Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9

[Laurence Oberman]

- Original Message -
> From: "Laurence Oberman" <lober...@redhat.com>
> To: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List" 
> <linux-scsi@vger.kernel.org>
> Sent: Tuesday, March 14, 2017 11:49:51 PM
> Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based 8Gb) 
> with BAR 3 error, work fine on 4.9
> 
> 
> 
> - Original Message -
> > From: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > To: "Laurence Oberman" <lober...@redhat.com>
> > Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List"
> > <linux-scsi@vger.kernel.org>
> > Sent: Tuesday, March 14, 2017 8:32:25 PM
> > Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based
> > 8Gb) with BAR 3 error, work fine on 4.9
> > 
> > 
> > - Original Message -
> > > From: "Laurence Oberman" <lober...@redhat.com>
> > > To: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > > Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List"
> > > <linux-scsi@vger.kernel.org>
> > > Sent: Tuesday, March 14, 2017 8:02:32 PM
> > > Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx
> > > (ISP2532-based
> > > 8Gb) with BAR 3 error, work fine on 4.9
> > > 
> > > 
> > > 
> > > - Original Message -
> > > > From: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > > > To: "Laurence Oberman" <lober...@redhat.com>
> > > > Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List"
> > > > <linux-scsi@vger.kernel.org>
> > > > Sent: Tuesday, March 14, 2017 5:11:13 PM
> > > > Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx
> > > > (ISP2532-based
> > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > > 
> > > > 
> > > > - Original Message -
> > > > > From: "Laurence Oberman" <lober...@redhat.com>
> > > > > To: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > > > > Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List"
> > > > > <linux-scsi@vger.kernel.org>
> > > > > Sent: Monday, March 13, 2017 9:06:38 PM
> > > > > Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx
> > > > > (ISP2532-based
> > > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > > > 
> > > > > 
> >     > > > 
> > > > > - Original Message -
> > > > > > From: "Laurence Oberman" <lober...@redhat.com>
> > > > > > To: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > > > > > Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI
> > > > > > List"
> > > > > > <linux-scsi@vger.kernel.org>
> > > > > > Sent: Monday, March 13, 2017 12:54:12 PM
> > > > > > Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx
> > > > > > (ISP2532-based
> > > > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > - Original Message -
> > > > > > > From: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > > > > > > To: "Laurence Oberman" <lober...@redhat.com>, "Chad
> > > > > > > Dupuis"
> > > > > > > <chad.dup...@cavium.com>
> > > > > > > Cc: "Linux SCSI List" <linux-scsi@vger.kernel.org>
> > > > > > > Sent: Monday, March 13, 2017 12:39:03 PM
> > > > > > > Subject: RE: 4.10+ qla2xxx  driver wont load for qla2xxx
> > > > > > > (ISP2532-based
> > > > > > > 8Gb

Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9

[Laurence Oberman]

- Original Message -
> From: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> To: "Laurence Oberman" <lober...@redhat.com>
> Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List" 
> <linux-scsi@vger.kernel.org>
> Sent: Tuesday, March 14, 2017 8:32:25 PM
> Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based 8Gb) 
> with BAR 3 error, work fine on 4.9
> 
> 
> - Original Message -
> > From: "Laurence Oberman" <lober...@redhat.com>
> > To: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List"
> > <linux-scsi@vger.kernel.org>
> > Sent: Tuesday, March 14, 2017 8:02:32 PM
> > Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based
>     > 8Gb) with BAR 3 error, work fine on 4.9
> > 
> > 
> > 
> > - Original Message -
> > > From: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > > To: "Laurence Oberman" <lober...@redhat.com>
> > > Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List"
> > > <linux-scsi@vger.kernel.org>
> > > Sent: Tuesday, March 14, 2017 5:11:13 PM
> > > Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx
> > > (ISP2532-based
> > > 8Gb) with BAR 3 error, work fine on 4.9
> > > 
> > > 
> > > - Original Message -
> > > > From: "Laurence Oberman" <lober...@redhat.com>
> > > > To: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > > > Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List"
> > > > <linux-scsi@vger.kernel.org>
> > > > Sent: Monday, March 13, 2017 9:06:38 PM
> > > > Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx
> > > > (ISP2532-based
> > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > > 
> > > > 
> > > > 
> > > > - Original Message -
> > > > > From: "Laurence Oberman" <lober...@redhat.com>
> > > > > To: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > > > > Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List"
> > > > > <linux-scsi@vger.kernel.org>
> > > > > Sent: Monday, March 13, 2017 12:54:12 PM
> > > > > Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx
> > > > > (ISP2532-based
> > > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > > > 
> > > > > 
> > > > > 
> > > > > - Original Message -
> > > > > > From: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > > > > > To: "Laurence Oberman" <lober...@redhat.com>, "Chad Dupuis"
> > > > > > <chad.dup...@cavium.com>
> > > > > > Cc: "Linux SCSI List" <linux-scsi@vger.kernel.org>
> > > > > > Sent: Monday, March 13, 2017 12:39:03 PM
> > > > > > Subject: RE: 4.10+ qla2xxx  driver wont load for qla2xxx
> > > > > > (ISP2532-based
> > > > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > > > > 
> > > > > > Hi Laurence,
> > > > > > 
> > > > > > > -Original Message-
> > > > > > > From: Laurence Oberman [mailto:lober...@redhat.com]
> > > > > > > Sent: Sunday, March 12, 2017 11:31 AM
> > > > > > > To: Dupuis, Chad <chad.dup...@cavium.com>; Madhani,
> > > > > > > Himanshu
> > > > > > > <himanshu.madh...@cavium.com>
> > > > > > > Cc: Linux SCSI List <linux-scsi@vger.kernel.org>
> > > > > > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx
> > > > > > > (ISP2532-based
> > &

Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9

[Laurence Oberman]

- Original Message -
> From: "Laurence Oberman" <lober...@redhat.com>
> To: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List" 
> <linux-scsi@vger.kernel.org>
> Sent: Tuesday, March 14, 2017 8:02:32 PM
> Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based 8Gb) 
> with BAR 3 error, work fine on 4.9
> 
> 
> 
> - Original Message -
> > From: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > To: "Laurence Oberman" <lober...@redhat.com>
> > Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List"
> > <linux-scsi@vger.kernel.org>
> > Sent: Tuesday, March 14, 2017 5:11:13 PM
> > Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based
> > 8Gb) with BAR 3 error, work fine on 4.9
> > 
> > 
> > - Original Message -
> > > From: "Laurence Oberman" <lober...@redhat.com>
> > > To: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > > Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List"
> > > <linux-scsi@vger.kernel.org>
> > > Sent: Monday, March 13, 2017 9:06:38 PM
> > > Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx
> > > (ISP2532-based
> > > 8Gb) with BAR 3 error, work fine on 4.9
> > > 
> > > 
> > > 
> > > - Original Message -
> > > > From: "Laurence Oberman" <lober...@redhat.com>
> > > > To: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > > > Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List"
> > > > <linux-scsi@vger.kernel.org>
> > > > Sent: Monday, March 13, 2017 12:54:12 PM
> > > > Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx
> > > > (ISP2532-based
> > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > > 
> > > > 
> > > > 
> > > > - Original Message -
> > > > > From: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > > > > To: "Laurence Oberman" <lober...@redhat.com>, "Chad Dupuis"
> > > > > <chad.dup...@cavium.com>
> > > > > Cc: "Linux SCSI List" <linux-scsi@vger.kernel.org>
> > > > > Sent: Monday, March 13, 2017 12:39:03 PM
> > > > > Subject: RE: 4.10+ qla2xxx  driver wont load for qla2xxx
> > > > > (ISP2532-based
> > > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > > > 
> > > > > Hi Laurence,
> > > > > 
> > > > > > -Original Message-
> > > > > > From: Laurence Oberman [mailto:lober...@redhat.com]
> > > > > > Sent: Sunday, March 12, 2017 11:31 AM
> > > > > > To: Dupuis, Chad <chad.dup...@cavium.com>; Madhani, Himanshu
> > > > > > <himanshu.madh...@cavium.com>
> > > > > > Cc: Linux SCSI List <linux-scsi@vger.kernel.org>
> > > > > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx
> > > > > > (ISP2532-based
> > > > > > 8Gb)
> > > > > > with BAR 3 error, work fine on 4.9
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > - Original Message -
> > > > > > > From: "Laurence Oberman" <lober...@redhat.com>
> > > > > > > To: "Chad Dupuis" <chad.dup...@cavium.com>, "Himanshu
> > > > > > > Madhani"
> > > > > > > <himanshu.madh...@cavium.com>
> > > > > > > Cc: "Linux SCSI List" <linux-scsi@vger.kernel.org>
> > > > > > > Sent: Sunday, March 12, 2017 7:39:23 AM
> > > > > > > Subject: 4.10+ qla2xxx  driver wont load for qla2xxx
> > > > > > > (ISP2532-based
> > > > > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > > > > >
> > > > > > > Chad, Himanshu
> > > > > > >
>

Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9

[Laurence Oberman]

- Original Message -
> From: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> To: "Laurence Oberman" <lober...@redhat.com>
> Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List" 
> <linux-scsi@vger.kernel.org>
> Sent: Tuesday, March 14, 2017 5:11:13 PM
> Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based 8Gb) 
> with BAR 3 error, work fine on 4.9
> 
> 
> - Original Message -
> > From: "Laurence Oberman" <lober...@redhat.com>
> > To: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List"
> > <linux-scsi@vger.kernel.org>
> > Sent: Monday, March 13, 2017 9:06:38 PM
> > Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based
> > 8Gb) with BAR 3 error, work fine on 4.9
> > 
> > 
> > 
> > - Original Message -
> > > From: "Laurence Oberman" <lober...@redhat.com>
> > > To: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > > Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List"
> > > <linux-scsi@vger.kernel.org>
> > > Sent: Monday, March 13, 2017 12:54:12 PM
> > > Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx
> > > (ISP2532-based
> > > 8Gb) with BAR 3 error, work fine on 4.9
> > > 
> > > 
> > > 
> > > - Original Message -
> > > > From: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > > > To: "Laurence Oberman" <lober...@redhat.com>, "Chad Dupuis"
> > > > <chad.dup...@cavium.com>
> > > > Cc: "Linux SCSI List" <linux-scsi@vger.kernel.org>
> > > > Sent: Monday, March 13, 2017 12:39:03 PM
> > > > Subject: RE: 4.10+ qla2xxx  driver wont load for qla2xxx
> > > > (ISP2532-based
> > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > > 
> > > > Hi Laurence,
> > > > 
> > > > > -Original Message-
> > > > > From: Laurence Oberman [mailto:lober...@redhat.com]
> > > > > Sent: Sunday, March 12, 2017 11:31 AM
> > > > > To: Dupuis, Chad <chad.dup...@cavium.com>; Madhani, Himanshu
> > > > > <himanshu.madh...@cavium.com>
> > > > > Cc: Linux SCSI List <linux-scsi@vger.kernel.org>
> > > > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx
> > > > > (ISP2532-based
> > > > > 8Gb)
> > > > > with BAR 3 error, work fine on 4.9
> > > > > 
> > > > > 
> > > > > 
> > > > > - Original Message -
> > > > > > From: "Laurence Oberman" <lober...@redhat.com>
> > > > > > To: "Chad Dupuis" <chad.dup...@cavium.com>, "Himanshu Madhani"
> > > > > > <himanshu.madh...@cavium.com>
> > > > > > Cc: "Linux SCSI List" <linux-scsi@vger.kernel.org>
> > > > > > Sent: Sunday, March 12, 2017 7:39:23 AM
> > > > > > Subject: 4.10+ qla2xxx  driver wont load for qla2xxx
> > > > > > (ISP2532-based
> > > > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > > > >
> > > > > > Chad, Himanshu
> > > > > >
> > > > > > Before I bisect or go chase changes, wanted to reach out
> > > > > > because the
> > > > > > driver seems to be the same version.
> > > > > > Perhaps this is a PCIE change in the kernel for 4.10 affecting
> > > > > > the
> > > > > > load.
> > > > > > Its the same targetLIO server I have been using for a long time
> > > > > > with
> > > > > > 4.9
> > > > > >
> > > > > > 27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre
> > > > > > Channel
> > > > > > to
> > > > > > PCI Express HBA (rev 02)
> > > > > >
> > > > > > With 4.9 I have

Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9

[Laurence Oberman]

- Original Message -
> From: "Laurence Oberman" <lober...@redhat.com>
> To: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List" 
> <linux-scsi@vger.kernel.org>
> Sent: Monday, March 13, 2017 9:06:38 PM
> Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based 8Gb) 
> with BAR 3 error, work fine on 4.9
> 
> 
> 
> - Original Message -
> > From: "Laurence Oberman" <lober...@redhat.com>
> > To: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List"
> > <linux-scsi@vger.kernel.org>
> > Sent: Monday, March 13, 2017 12:54:12 PM
> > Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based
> > 8Gb) with BAR 3 error, work fine on 4.9
> > 
> > 
> > 
> > - Original Message -
> > > From: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > > To: "Laurence Oberman" <lober...@redhat.com>, "Chad Dupuis"
> > > <chad.dup...@cavium.com>
> > > Cc: "Linux SCSI List" <linux-scsi@vger.kernel.org>
> > > Sent: Monday, March 13, 2017 12:39:03 PM
> > > Subject: RE: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based
> > > 8Gb) with BAR 3 error, work fine on 4.9
> > > 
> > > Hi Laurence,
> > > 
> > > > -Original Message-
> > > > From: Laurence Oberman [mailto:lober...@redhat.com]
> > > > Sent: Sunday, March 12, 2017 11:31 AM
> > > > To: Dupuis, Chad <chad.dup...@cavium.com>; Madhani, Himanshu
> > > > <himanshu.madh...@cavium.com>
> > > > Cc: Linux SCSI List <linux-scsi@vger.kernel.org>
> > > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based
> > > > 8Gb)
> > > > with BAR 3 error, work fine on 4.9
> > > > 
> > > > 
> > > > 
> > > > - Original Message -
> > > > > From: "Laurence Oberman" <lober...@redhat.com>
> > > > > To: "Chad Dupuis" <chad.dup...@cavium.com>, "Himanshu Madhani"
> > > > > <himanshu.madh...@cavium.com>
> > > > > Cc: "Linux SCSI List" <linux-scsi@vger.kernel.org>
> > > > > Sent: Sunday, March 12, 2017 7:39:23 AM
> > > > > Subject: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based
> > > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > > >
> > > > > Chad, Himanshu
> > > > >
> > > > > Before I bisect or go chase changes, wanted to reach out because the
> > > > > driver seems to be the same version.
> > > > > Perhaps this is a PCIE change in the kernel for 4.10 affecting the
> > > > > load.
> > > > > Its the same targetLIO server I have been using for a long time with
> > > > > 4.9
> > > > >
> > > > > 27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel
> > > > > to
> > > > > PCI Express HBA (rev 02)
> > > > >
> > > > > With 4.9 I have no issues loading the driver for my targetLIO server.
> > > > > (DL380G8)
> > > > >
> > > > > # modinfo qla2xxx | more
> > > > > filename:
> > > > > /lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko
> > > > > firmware:   ql2500_fw.bin
> > > > > version:8.07.00.38-k
> > > > > license:GPL
> > > > > description:QLogic Fibre Channel HBA Driver
> > > > > author: QLogic Corporation
> > > > > srcversion: 94A8431A85BFF854B97B02D
> > > > >
> > > > > [8.906351] qla2xxx [:00:00.0]-0005: : QLogic Fibre Channel
> > > > > HBA
> > > > > Driver: 8.07.00.38-k.
> > > > > [   10.014052] qla2xxx [:27:00.0]-001d: : Found an ISP2532 irq
> > > > > 106
> > > > > iobase
> > > > > 0xadce989a1000.
> > > > > [   10.455108] scsi host4: qla2xxx
> > > > > [   10.460206] qla2xxx [:27:00.0]-00fb:4: QLogic QLE2562 -
> > > > > PCI-Express
> > > > > Dual Channel 8Gb Fibre Channel HBA.
> > > > > [   10.460215] qla2xxx [

Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9

[Laurence Oberman]

- Original Message -
> From: "Laurence Oberman" <lober...@redhat.com>
> To: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> Cc: "Chad Dupuis" <chad.dup...@cavium.com>, "Linux SCSI List" 
> <linux-scsi@vger.kernel.org>
> Sent: Monday, March 13, 2017 12:54:12 PM
> Subject: Re: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based 8Gb) 
> with BAR 3 error, work fine on 4.9
> 
> 
> 
> - Original Message -
> > From: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> > To: "Laurence Oberman" <lober...@redhat.com>, "Chad Dupuis"
> > <chad.dup...@cavium.com>
> > Cc: "Linux SCSI List" <linux-scsi@vger.kernel.org>
> > Sent: Monday, March 13, 2017 12:39:03 PM
> > Subject: RE: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based
> > 8Gb) with BAR 3 error, work fine on 4.9
> > 
> > Hi Laurence,
> > 
> > > -Original Message-
> > > From: Laurence Oberman [mailto:lober...@redhat.com]
> > > Sent: Sunday, March 12, 2017 11:31 AM
> > > To: Dupuis, Chad <chad.dup...@cavium.com>; Madhani, Himanshu
> > > <himanshu.madh...@cavium.com>
> > > Cc: Linux SCSI List <linux-scsi@vger.kernel.org>
> > > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based
> > > 8Gb)
> > > with BAR 3 error, work fine on 4.9
> > > 
> > > 
> > > 
> > > - Original Message -
> > > > From: "Laurence Oberman" <lober...@redhat.com>
> > > > To: "Chad Dupuis" <chad.dup...@cavium.com>, "Himanshu Madhani"
> > > > <himanshu.madh...@cavium.com>
> > > > Cc: "Linux SCSI List" <linux-scsi@vger.kernel.org>
> > > > Sent: Sunday, March 12, 2017 7:39:23 AM
> > > > Subject: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based
> > > > 8Gb) with BAR 3 error, work fine on 4.9
> > > >
> > > > Chad, Himanshu
> > > >
> > > > Before I bisect or go chase changes, wanted to reach out because the
> > > > driver seems to be the same version.
> > > > Perhaps this is a PCIE change in the kernel for 4.10 affecting the
> > > > load.
> > > > Its the same targetLIO server I have been using for a long time with
> > > > 4.9
> > > >
> > > > 27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to
> > > > PCI Express HBA (rev 02)
> > > >
> > > > With 4.9 I have no issues loading the driver for my targetLIO server.
> > > > (DL380G8)
> > > >
> > > > # modinfo qla2xxx | more
> > > > filename:
> > > > /lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko
> > > > firmware:   ql2500_fw.bin
> > > > version:8.07.00.38-k
> > > > license:GPL
> > > > description:QLogic Fibre Channel HBA Driver
> > > > author: QLogic Corporation
> > > > srcversion: 94A8431A85BFF854B97B02D
> > > >
> > > > [8.906351] qla2xxx [:00:00.0]-0005: : QLogic Fibre Channel HBA
> > > > Driver: 8.07.00.38-k.
> > > > [   10.014052] qla2xxx [:27:00.0]-001d: : Found an ISP2532 irq 106
> > > > iobase
> > > > 0xadce989a1000.
> > > > [   10.455108] scsi host4: qla2xxx
> > > > [   10.460206] qla2xxx [:27:00.0]-00fb:4: QLogic QLE2562 -
> > > > PCI-Express
> > > > Dual Channel 8Gb Fibre Channel HBA.
> > > > [   10.460215] qla2xxx [:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s
> > > > x8)
> > > > @
> > > > :27:00.0 hdma+ host#=4 fw=8.03.00 (90d5).
> > > > [   10.460545] qla2xxx [:27:00.1]-001d: : Found an ISP2532 irq 110
> > > > iobase
> > > > 0xadce989a9000.
> > > > [   10.662120] scsi host5: qla2xxx
> > > > [   11.007841] qla2xxx [:27:00.1]-00fb:5: QLogic QLE2562 -
> > > > PCI-Express
> > > > Dual Channel 8Gb Fibre Channel HBA.
> > > > [   11.007849] qla2xxx [:27:00.1]-00fc:5: ISP2532: PCIe (5.0GT/s
> > > > x8)
> > > > @
> > > > :27:00.1 hdma+ host#=5 fw=8.03.00 (90d5).
> > > >
> > > > Rebooting on the same server with 4.10 fails to load
> > > >
> > > > Linux  4.10.0+
> > > > # modinfo qla2xxx | more
> > > > filen

Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9

[Laurence Oberman]

- Original Message -
> From: "Himanshu Madhani" <himanshu.madh...@cavium.com>
> To: "Laurence Oberman" <lober...@redhat.com>, "Chad Dupuis" 
> <chad.dup...@cavium.com>
> Cc: "Linux SCSI List" <linux-scsi@vger.kernel.org>
> Sent: Monday, March 13, 2017 12:39:03 PM
> Subject: RE: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based 8Gb) 
> with BAR 3 error, work fine on 4.9
> 
> Hi Laurence,
> 
> > -Original Message-
> > From: Laurence Oberman [mailto:lober...@redhat.com]
> > Sent: Sunday, March 12, 2017 11:31 AM
> > To: Dupuis, Chad <chad.dup...@cavium.com>; Madhani, Himanshu
> > <himanshu.madh...@cavium.com>
> > Cc: Linux SCSI List <linux-scsi@vger.kernel.org>
> > Subject: Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb)
> > with BAR 3 error, work fine on 4.9
> > 
> > 
> > 
> > - Original Message -
> > > From: "Laurence Oberman" <lober...@redhat.com>
> > > To: "Chad Dupuis" <chad.dup...@cavium.com>, "Himanshu Madhani"
> > > <himanshu.madh...@cavium.com>
> > > Cc: "Linux SCSI List" <linux-scsi@vger.kernel.org>
> > > Sent: Sunday, March 12, 2017 7:39:23 AM
> > > Subject: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based
> > > 8Gb) with BAR 3 error, work fine on 4.9
> > >
> > > Chad, Himanshu
> > >
> > > Before I bisect or go chase changes, wanted to reach out because the
> > > driver seems to be the same version.
> > > Perhaps this is a PCIE change in the kernel for 4.10 affecting the load.
> > > Its the same targetLIO server I have been using for a long time with
> > > 4.9
> > >
> > > 27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to
> > > PCI Express HBA (rev 02)
> > >
> > > With 4.9 I have no issues loading the driver for my targetLIO server.
> > > (DL380G8)
> > >
> > > # modinfo qla2xxx | more
> > > filename:
> > > /lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko
> > > firmware:   ql2500_fw.bin
> > > version:8.07.00.38-k
> > > license:GPL
> > > description:QLogic Fibre Channel HBA Driver
> > > author: QLogic Corporation
> > > srcversion: 94A8431A85BFF854B97B02D
> > >
> > > [8.906351] qla2xxx [:00:00.0]-0005: : QLogic Fibre Channel HBA
> > > Driver: 8.07.00.38-k.
> > > [   10.014052] qla2xxx [:27:00.0]-001d: : Found an ISP2532 irq 106
> > > iobase
> > > 0xadce989a1000.
> > > [   10.455108] scsi host4: qla2xxx
> > > [   10.460206] qla2xxx [:27:00.0]-00fb:4: QLogic QLE2562 -
> > > PCI-Express
> > > Dual Channel 8Gb Fibre Channel HBA.
> > > [   10.460215] qla2xxx [:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s x8)
> > > @
> > > :27:00.0 hdma+ host#=4 fw=8.03.00 (90d5).
> > > [   10.460545] qla2xxx [:27:00.1]-001d: : Found an ISP2532 irq 110
> > > iobase
> > > 0xadce989a9000.
> > > [   10.662120] scsi host5: qla2xxx
> > > [   11.007841] qla2xxx [:27:00.1]-00fb:5: QLogic QLE2562 -
> > > PCI-Express
> > > Dual Channel 8Gb Fibre Channel HBA.
> > > [   11.007849] qla2xxx [:27:00.1]-00fc:5: ISP2532: PCIe (5.0GT/s x8)
> > > @
> > > :27:00.1 hdma+ host#=5 fw=8.03.00 (90d5).
> > >
> > > Rebooting on the same server with 4.10 fails to load
> > >
> > > Linux  4.10.0+
> > > # modinfo qla2xxx | more
> > > filename:
> > > /lib/modules/4.10.0+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko
> > > firmware:   ql2500_fw.bin
> > > version:8.07.00.38-k
> > > license:GPL
> > > description:QLogic Fibre Channel HBA Driver
> > > author: QLogic Corporation
> > > srcversion: 939E0595E8A3C2E1BE94392
> > >
> > > [8.754040] qla2xxx [:00:00.0]-0005: : QLogic Fibre Channel HBA
> > > Driver: 8.07.00.38-k.
> > > [9.979523] qla2xxx [:27:00.0]-001b: : BAR 3 not enabled.
> > > [   10.201268] qla2xxx [:27:00.0]-001d: : Found an ISP2532 irq 110
> > > iobase
> > > 0xacbf189b1000.
> > > [   10.407865] scsi host5: qla2xxx
> > > [   10.444281] qla2xxx: probe of :27:00.0 failed with error -22
> > > [   10.444519] qla2xxx [:27:00.1]-001b: : BAR 3 not enabled.
> > > [   10.444522] qla2x

Re: 4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9

[Laurence Oberman]

- Original Message -
> From: "Laurence Oberman" <lober...@redhat.com>
> To: "Chad Dupuis" <chad.dup...@cavium.com>, "Himanshu Madhani" 
> <himanshu.madh...@cavium.com>
> Cc: "Linux SCSI List" <linux-scsi@vger.kernel.org>
> Sent: Sunday, March 12, 2017 7:39:23 AM
> Subject: 4.10+ qla2xxx  driver wont load for qla2xxx (ISP2532-based 8Gb) with 
> BAR 3 error, work fine on 4.9
> 
> Chad, Himanshu
> 
> Before I bisect or go chase changes, wanted to reach out because the driver
> seems to be the same version.
> Perhaps this is a PCIE change in the kernel for 4.10 affecting the load.
> Its the same targetLIO server I have been using for a long time with 4.9
> 
> 27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI
> Express HBA (rev 02)
> 
> With 4.9 I have no issues loading the driver for my targetLIO server.
> (DL380G8)
> 
> # modinfo qla2xxx | more
> filename:
> /lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko
> firmware:   ql2500_fw.bin
> version:8.07.00.38-k
> license:GPL
> description:QLogic Fibre Channel HBA Driver
> author: QLogic Corporation
> srcversion: 94A8431A85BFF854B97B02D
> 
> [8.906351] qla2xxx [:00:00.0]-0005: : QLogic Fibre Channel HBA
> Driver: 8.07.00.38-k.
> [   10.014052] qla2xxx [:27:00.0]-001d: : Found an ISP2532 irq 106 iobase
> 0xadce989a1000.
> [   10.455108] scsi host4: qla2xxx
> [   10.460206] qla2xxx [:27:00.0]-00fb:4: QLogic QLE2562 - PCI-Express
> Dual Channel 8Gb Fibre Channel HBA.
> [   10.460215] qla2xxx [:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s x8) @
> :27:00.0 hdma+ host#=4 fw=8.03.00 (90d5).
> [   10.460545] qla2xxx [:27:00.1]-001d: : Found an ISP2532 irq 110 iobase
> 0xadce989a9000.
> [   10.662120] scsi host5: qla2xxx
> [   11.007841] qla2xxx [:27:00.1]-00fb:5: QLogic QLE2562 - PCI-Express
> Dual Channel 8Gb Fibre Channel HBA.
> [   11.007849] qla2xxx [:27:00.1]-00fc:5: ISP2532: PCIe (5.0GT/s x8) @
> :27:00.1 hdma+ host#=5 fw=8.03.00 (90d5).
> 
> Rebooting on the same server with 4.10 fails to load
> 
> Linux  4.10.0+
> # modinfo qla2xxx | more
> filename:   /lib/modules/4.10.0+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko
> firmware:   ql2500_fw.bin
> version:8.07.00.38-k
> license:GPL
> description:QLogic Fibre Channel HBA Driver
> author: QLogic Corporation
> srcversion: 939E0595E8A3C2E1BE94392
> 
> [8.754040] qla2xxx [:00:00.0]-0005: : QLogic Fibre Channel HBA
> Driver: 8.07.00.38-k.
> [9.979523] qla2xxx [:27:00.0]-001b: : BAR 3 not enabled.
> [   10.201268] qla2xxx [:27:00.0]-001d: : Found an ISP2532 irq 110 iobase
> 0xacbf189b1000.
> [   10.407865] scsi host5: qla2xxx
> [   10.444281] qla2xxx: probe of :27:00.0 failed with error -22
> [   10.444519] qla2xxx [:27:00.1]-001b: : BAR 3 not enabled.
> [   10.444522] qla2xxx [:27:00.1]-001d: : Found an ISP2532 irq 110 iobase
> 0xacbf189b9000.
> [   10.645932] scsi host5: qla2xxx
> [   10.682233] qla2xxx: probe of :27:00.1 failed with error -22
> 
> Thanks
> Laurence
> 

I started bisecting this, cannot believe others have not bumped into this on 
4.10.
This is a generic QLE2562 and firmware is loaded by the driver so wondering why 
I am seeing this and other are not.
There is nothing special with the PCIE bus on this DL380G8.

Anyway during the bisect I got to a point where in the 4.10 commits I still saw 
the "BAR 3" message
but the probe worked.

[7.208237] qla2xxx [:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 
8.07.00.38-k.
[7.208492] qla2xxx [:27:00.0]-001b: : BAR 3 not enabled.

   see this above but probe did not fail

[7.208494] qla2xxx [:27:00.0]-001d: : Found an ISP2532 irq 97 iobase 
0xc02f98989000.
[7.414738] scsi host4: qla2xxx

[7.419267] qla2xxx [:27:00.0]-00fb:4: QLogic QLE2562 - PCI-Express Dual 
Channel 8Gb Fibre Channel HBA.
[7.419278] qla2xxx [:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s x8) @ 
:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5).
[7.419698] qla2xxx [:27:00.1]-001b: : BAR 3 not enabled.
[7.419701] qla2xxx [:27:00.1]-001d: : Found an ISP2532 irq 100 iobase 
0xc02f989b1000.
[7.625691] scsi host6: qla2xxx
[7.629218] qla2xxx [:27:00.1]-00fb:6: QLogic QLE2562 - PCI-Express Dual 
Channel 8Gb Fibre Channel HBA.
[7.629222] qla2xxx [:27:00.1]-00fc:6: ISP2532: PCIe (5.0GT/s x8) @ 
:27:00.1 hdma+ host#=6 

I marked that still as bad and am continuing, I have 9 builds to go.

Thanks
Laurence

4.10+ qla2xxx driver wont load for qla2xxx (ISP2532-based 8Gb) with BAR 3 error, work fine on 4.9

[Laurence Oberman]

Chad, Himanshu

Before I bisect or go chase changes, wanted to reach out because the driver 
seems to be the same version.
Perhaps this is a PCIE change in the kernel for 4.10 affecting the load.
Its the same targetLIO server I have been using for a long time with 4.9

27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI 
Express HBA (rev 02)

With 4.9 I have no issues loading the driver for my targetLIO server. (DL380G8)

# modinfo qla2xxx | more
filename:   
/lib/modules/4.9.0.lobetcm+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko
firmware:   ql2500_fw.bin
version:8.07.00.38-k
license:GPL
description:QLogic Fibre Channel HBA Driver
author: QLogic Corporation
srcversion: 94A8431A85BFF854B97B02D

[8.906351] qla2xxx [:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 
8.07.00.38-k.
[   10.014052] qla2xxx [:27:00.0]-001d: : Found an ISP2532 irq 106 iobase 
0xadce989a1000.
[   10.455108] scsi host4: qla2xxx
[   10.460206] qla2xxx [:27:00.0]-00fb:4: QLogic QLE2562 - PCI-Express Dual 
Channel 8Gb Fibre Channel HBA.
[   10.460215] qla2xxx [:27:00.0]-00fc:4: ISP2532: PCIe (5.0GT/s x8) @ 
:27:00.0 hdma+ host#=4 fw=8.03.00 (90d5).
[   10.460545] qla2xxx [:27:00.1]-001d: : Found an ISP2532 irq 110 iobase 
0xadce989a9000.
[   10.662120] scsi host5: qla2xxx
[   11.007841] qla2xxx [:27:00.1]-00fb:5: QLogic QLE2562 - PCI-Express Dual 
Channel 8Gb Fibre Channel HBA.
[   11.007849] qla2xxx [:27:00.1]-00fc:5: ISP2532: PCIe (5.0GT/s x8) @ 
:27:00.1 hdma+ host#=5 fw=8.03.00 (90d5).

Rebooting on the same server with 4.10 fails to load 

Linux  4.10.0+ 
# modinfo qla2xxx | more
filename:   /lib/modules/4.10.0+/kernel/drivers/scsi/qla2xxx/qla2xxx.ko
firmware:   ql2500_fw.bin
version:8.07.00.38-k
license:GPL
description:QLogic Fibre Channel HBA Driver
author: QLogic Corporation
srcversion: 939E0595E8A3C2E1BE94392

[8.754040] qla2xxx [:00:00.0]-0005: : QLogic Fibre Channel HBA Driver: 
8.07.00.38-k.
[9.979523] qla2xxx [:27:00.0]-001b: : BAR 3 not enabled.
[   10.201268] qla2xxx [:27:00.0]-001d: : Found an ISP2532 irq 110 iobase 
0xacbf189b1000.
[   10.407865] scsi host5: qla2xxx
[   10.444281] qla2xxx: probe of :27:00.0 failed with error -22
[   10.444519] qla2xxx [:27:00.1]-001b: : BAR 3 not enabled.
[   10.444522] qla2xxx [:27:00.1]-001d: : Found an ISP2532 irq 110 iobase 
0xacbf189b9000.
[   10.645932] scsi host5: qla2xxx
[   10.682233] qla2xxx: probe of :27:00.1 failed with error -22

Thanks
Laurence

Chasing possible Intel driver issue on Intel Soft FCOE CNA card

[Laurence Oberman]

Hello Nicholas

I am chasing the possibility of an OX_ID being prematurely re-used in the ixgbe 
and software fcoe stack
when we have congestion.
We have a situation where we have F/C traces showing this happening and in 
trying to track this down using LIO target as the array.

I bumped into this stack below for tcm_qla2xxx during the efforts.

Running performance tests via FCOE on the Intel card due to known issues in the 
resources of this card.
  See prior http://www.spinics.net/lists/linux-scsi/msg100505.html

Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network 
Connection (rev 01)

We will get into blk_requeue_request() here because of the limited pool which 
is not unexpected

On the LIO array I see this and trying to understand if this is the software 
fcoe stack initiator causing this.
I can add whatever instrumentation needed to make progress.


static int tcm_qla2xxx_queue_status(struct se_cmd *se_cmd)
{
struct qla_tgt_cmd *cmd = container_of(se_cmd,
struct qla_tgt_cmd, se_cmd);
int xmit_type = QLA_TGT_XMIT_STATUS;

cmd->bufflen = se_cmd->data_length;
cmd->sg = NULL;
cmd->sg_cnt = 0;
cmd->offset = 0;
cmd->dma_data_direction = target_reverse_dma_direction(se_cmd);
if (cmd->cmd_flags &  BIT_5) {
pr_crit("Bit_5 already set for cmd = %p.\n", cmd);  

dump_stack();
}

[ 3216.602652] Bit_5 already set for cmd = 928007b95a50.
[ 3216.628736] CPU: 8 PID: 396 Comm: kworker/8:1 Not tainted 4.9.0tcm_debug+ #1
[ 3216.662893] Hardware name: HP ProLiant DL380p Gen8, BIOS P70 07/01/2015
[ 3216.694389] Workqueue: events target_qf_do_work [target_core_mod]
[ 3216.723595]  b75c9a157db0 bb3e924e 928007b95a50 
927584b8e058
[ 3216.758911]  b75c9a157dc8 c098fa57 928007b95a50 
b75c9a157e18
[ 3216.794563]  c0af2ca1 b75c9a157dd8 b75c9a157dd8 
a47f17db
[ 3216.829762] Call Trace:
[ 3216.841338]  [] dump_stack+0x63/0x85
[ 3216.865694]  [] tcm_qla2xxx_queue_status+0xf7/0x110 
[tcm_qla2xxx]
[ 3216.902302]  [] target_qf_do_work+0x1b1/0x300 
[target_core_mod]
[ 3216.938148]  [] process_one_work+0x15f/0x430
[ 3216.966349]  [] worker_thread+0x4e/0x490
[ 3216.993094]  [] ? process_one_work+0x430/0x430
[ 3217.021825]  [] ? process_one_work+0x430/0x430
[ 3217.050700]  [] kthread+0xd9/0xf0
[ 3217.073733]  [] ? kthread_park+0x60/0x60
[ 3217.100765]  [] ret_from_fork+0x25/0x30

typedef enum {
/*
 * BIT_0 - Atio Arrival / schedule to work
 * BIT_1 - qlt_do_work
 * BIT_2 - qlt_do work failed
 * BIT_3 - xfer rdy/tcm_qla2xxx_write_pending
 * BIT_4 - read respond/tcm_qla2xx_queue_data_in
 * BIT_5 - status respond / tcm_qla2xx_queue_status
*
 * BIT_6 - tcm request to abort/Term exchange.
 *  pre_xmit_response->qlt_send_term_exchange
 * BIT_7 - SRR received (qlt_handle_srr->qlt_xmit_response)
 * BIT_8 - SRR received (qlt_handle_srr->qlt_rdy_to_xfer)
 * BIT_9 - SRR received (qla_handle_srr->qlt_send_term_exchange)
 * BIT_10 - Data in - hanlde_data->tcm_qla2xxx_handle_data

 * BIT_12 - good completion - qlt_ctio_do_completion -->free_cmd
 * BIT_13 - Bad completion -
 *  qlt_ctio_do_completion --> qlt_term_ctio_exchange
 * BIT_14 - Back end data received/sent.
 * BIT_15 - SRR prepare ctio
 * BIT_16 - complete free
 * BIT_17 - flush - qlt_abort_cmd_on_host_reset
 * BIT_18 - completion w/abort status
 * BIT_19 - completion w/unknown status
 * BIT_20 - tcm_qla2xxx_free_cmd
 */
CMD_FLAG_DATA_WORK = BIT_11,
CMD_FLAG_DATA_WORK_FREE = BIT_21,
} cmd_flags_t;

Thanks
Laurence

Re: [PATCH 11/20] lpfc: Fix eh_deadline setting for sli3 adapters.

[Laurence Oberman]

rivers/scsi/lpfc/lpfc_init.c
> +++ b/drivers/scsi/lpfc/lpfc_init.c
> @@ -3555,6 +3555,44 @@ lpfc_sli4_scsi_sgl_update(struct lpfc_hba *phba)
>   return rc;
>  }
>  
> +static uint64_t
> +lpfc_get_wwpn(struct lpfc_hba *phba)
> +{
> + uint64_t wwn;
> + int rc;
> + LPFC_MBOXQ_t *mboxq;
> + MAILBOX_t *mb;
> +
> +
> + mboxq = (LPFC_MBOXQ_t *) mempool_alloc(phba->mbox_mem_pool,
> + GFP_KERNEL);
> + if (!mboxq)
> + return (uint64_t)-1;
> +
> + /* First get WWN of HBA instance */
> + lpfc_read_nv(phba, mboxq);
> + rc = lpfc_sli_issue_mbox(phba, mboxq, MBX_POLL);
> + if (rc != MBX_SUCCESS) {
> + lpfc_printf_log(phba, KERN_ERR, LOG_SLI,
> + "6019 Mailbox failed , mbxCmd x%x "
> + "READ_NV, mbxStatus x%x\n",
> + bf_get(lpfc_mqe_command, >u.mqe),
> + bf_get(lpfc_mqe_status, >u.mqe));
> + mempool_free(mboxq, phba->mbox_mem_pool);
> + return (uint64_t) -1;
> + }
> + mb = >u.mb;
> + memcpy(, (char *)mb->un.varRDnvp.portname, sizeof(uint64_t));
> + /* wwn is WWPN of HBA instance */
> + mempool_free(mboxq, phba->mbox_mem_pool);
> + if (phba->sli_rev == LPFC_SLI_REV4)
> + return be64_to_cpu(wwn);
> + else
> + return (((wwn & 0x) >> 32) |
> + ((wwn & 0x) << 32));
> +
> +}
> +
>  /**
>   * lpfc_sli4_nvme_sgl_update - update xri-sgl sizing and mapping
>   * @phba: pointer to lpfc hba data structure.
> @@ -3676,17 +3714,32 @@ lpfc_create_port(struct lpfc_hba *phba, int instance,
> struct device *dev)
>   struct lpfc_vport *vport;
>   struct Scsi_Host  *shost = NULL;
>   int error = 0;
> + int i;
> + uint64_t wwn;
> + bool use_no_reset_hba = false;
> +
> + wwn = lpfc_get_wwpn(phba);
> +
> + for (i = 0; i < lpfc_no_hba_reset_cnt; i++) {
> + if (wwn == lpfc_no_hba_reset[i]) {
> + lpfc_printf_log(phba, KERN_ERR, LOG_SLI,
> + "6020 Setting use_no_reset port=%llx\n",
> + wwn);
> + use_no_reset_hba = true;
> + break;
> + }
> + }
>  
>   if (phba->cfg_enable_fc4_type & LPFC_ENABLE_FCP) {
>   if (dev != >pcidev->dev) {
>   shost = scsi_host_alloc(_vport_template,
>   sizeof(struct lpfc_vport));
>   } else {
> - if (phba->sli_rev == LPFC_SLI_REV4)
> + if (!use_no_reset_hba)
>   shost = scsi_host_alloc(_template,
>   sizeof(struct lpfc_vport));
>   else
> - shost = scsi_host_alloc(_template_s3,
> + shost = scsi_host_alloc(_template_no_hr,
>   sizeof(struct lpfc_vport));
>   }
>   } else if (phba->cfg_enable_fc4_type & LPFC_ENABLE_NVME) {
> @@ -5482,7 +5535,8 @@ lpfc_sli_driver_resource_setup(struct lpfc_hba *phba)
>  
>   /* Initialize the host templates the configured values. */
>   lpfc_vport_template.sg_tablesize = phba->cfg_sg_seg_cnt;
> - lpfc_template_s3.sg_tablesize = phba->cfg_sg_seg_cnt;
> + lpfc_template_no_hr.sg_tablesize = phba->cfg_sg_seg_cnt;
> + lpfc_template.sg_tablesize = phba->cfg_sg_seg_cnt;
>  
>   /* There are going to be 2 reserved BDEs: 1 FCP cmnd + 1 FCP rsp */
>   if (phba->cfg_enable_bg) {
> @@ -5706,6 +5760,7 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba)
>   /* Initialize the host templates with the updated values. */
>   lpfc_vport_template.sg_tablesize = phba->cfg_sg_seg_cnt;
>   lpfc_template.sg_tablesize = phba->cfg_sg_seg_cnt;
> + lpfc_template_no_hr.sg_tablesize = phba->cfg_sg_seg_cnt;
>  
>   if (phba->cfg_sg_dma_buf_size  <= LPFC_MIN_SG_SLI4_BUF_SZ)
>   phba->cfg_sg_dma_buf_size = LPFC_MIN_SG_SLI4_BUF_SZ;
> diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c
> index 9d6384a..bcfd6d6 100644
> --- a/drivers/scsi/lpfc/lpfc_scsi.c
> +++ b/drivers/scsi/lpfc/lpfc_scsi.c
> @@ -5953,7 +5953,7 @@ struct scsi_host_template lpfc_template_nvme = {
>   .track_queue_depth  = 0,
>  };
>  
> -struct scsi_host_template lpfc_template_s3 = {

Re: sg driver, sg_io and sg tablesize

[Laurence Oberman]

- Original Message -
> From: "Laurence Oberman" <lober...@redhat.com>
> To: dgilb...@interlog.com, "Linux SCSI List" <linux-scsi@vger.kernel.org>
> Sent: Wednesday, March 1, 2017 10:44:32 AM
> Subject: sg driver, sg_io and sg tablesize
> 
> Hi Doug
> I have a quick question here regarding the sg tablesize and retrieval of the
> supported size via a userspace ioctl.
> 
> lpfc defaults to
> lpfc_sg_seg_cnt:Max Scatter Gather Segment Count (uint) = 64
> 
> For sg_io sent from qemu if we exceed 512K I/O we fail, qlogic allows 128 by
> default.
> 
> 7:0:2:13 /dev/sdbb 67:80   /dev/sg55   Emulex lpfc LUN
> 
> [root@dhcp40-131 ~]# sg_dd if=/dev/zero of=/dev/sdbb bpt=0x400 count=0x400
> blk_sgio=1
> Assume default 'bs' (block size) of 512 bytes
> writing (SG_IO) on sg device, error: Invalid argument
> sg_write failed, seek=0
> Some error occurred,  remaining block count=1024
> 1024+0 records in
> 0+0 records out
> 
> 9:0:1:13 /dev/sdcm 71:240  /dev/sg92 Qlogic qla2xxx LUN
> 
> [root@dhcp40-131 ~]# sg_dd if=/dev/zero of=/dev/sdcm bpt=0x400 count=0x400
> blk_sgio=1
> Assume default 'bs' (block size) of 512 bytes
> 1024+0 records in
> 1024+0 records out
> 
> Using ioctl(sg_fd, BLKSECTGET, _sectors) from userspace correctly returns
> the /sys/block/sdcm/queue/max_sectors_kb
> 
> How can we query the sg tablesize using a userspace ioctl for the sg driver.
> 
> Figured I would ask you first.
> 
> Thanks
> Laurence
> 

Answering my own email,
I chatted with Ewan and we decided we would point our Virt/Qemu team to simply 
go after max_segments in sysfs
This was suggested by Ewan and is the best way to know what the actual max I/O 
size for sg_io would be for the LPFC driver.

Thanks
Laurence

sg driver, sg_io and sg tablesize

[Laurence Oberman]

Hi Doug
I have a quick question here regarding the sg tablesize and retrieval of the 
supported size via a userspace ioctl.

lpfc defaults to 
lpfc_sg_seg_cnt:Max Scatter Gather Segment Count (uint) = 64

For sg_io sent from qemu if we exceed 512K I/O we fail, qlogic allows 128 by 
default.

7:0:2:13 /dev/sdbb 67:80   /dev/sg55   Emulex lpfc LUN

[root@dhcp40-131 ~]# sg_dd if=/dev/zero of=/dev/sdbb bpt=0x400 count=0x400 
blk_sgio=1
Assume default 'bs' (block size) of 512 bytes
writing (SG_IO) on sg device, error: Invalid argument
sg_write failed, seek=0
Some error occurred,  remaining block count=1024
1024+0 records in
0+0 records out

9:0:1:13 /dev/sdcm 71:240  /dev/sg92 Qlogic qla2xxx LUN

[root@dhcp40-131 ~]# sg_dd if=/dev/zero of=/dev/sdcm bpt=0x400 count=0x400 
blk_sgio=1
Assume default 'bs' (block size) of 512 bytes
1024+0 records in
1024+0 records out

Using ioctl(sg_fd, BLKSECTGET, _sectors) from userspace correctly returns
the /sys/block/sdcm/queue/max_sectors_kb 

How can we query the sg tablesize using a userspace ioctl for the sg driver.

Figured I would ask you first.

Thanks
Laurence

Re: [PATCH] scsi_error: count medium access timeout only once per EH run

[Laurence Oberman]

   write_prot;
>   u8  protection_type;/* Data Integrity Field */
> diff --git a/include/scsi/scsi.h b/include/scsi/scsi.h
> index a1e1930..b6c750f 100644
> --- a/include/scsi/scsi.h
> +++ b/include/scsi/scsi.h
> @@ -185,6 +185,7 @@ static inline int scsi_is_wlun(u64 lun)
>  #define TIMEOUT_ERROR   0x2007
>  #define SCSI_RETURN_NOT_HANDLED   0x2008
>  #define FAST_IO_FAIL 0x2009
> +#define NEEDS_RESET 0x2010
>  
>  /*
>   * Midlevel queue return values.
> --
> 1.8.5.6
> 
> 

Hello Hannes 
This makes sense to me what you are doing here.
I will also wait for Ewan to weigh in but I wonder if we should make a simple 
change.
Maybe good to clarify the RESET here by simply changing the name.

Change 
+#define NEEDS_RESET 0x2010
to
+#define MAX_MEDIUM_ERROR_NEEDS_RESET

Of course then also change
+   if (eh_disp == NEEDS_RESET) {
to
+   if (eh_disp == MAX_MEDIUM_ERROR_NEEDS_RESET) {

Reviewed-by: Laurence Oberman <lober...@redhat.com

Re: Patch: lpfc: Modify Emulex lpfc LPFC_DEFAULT_SG_SEG_CNT to 128 for default larger sg_io capability

[Laurence Oberman]

- Original Message -
> From: "Laurence Oberman" <lober...@redhat.com>
> To: "Linux SCSI List" <linux-scsi@vger.kernel.org>
> Sent: Wednesday, February 8, 2017 3:07:12 PM
> Subject: Patch: lpfc: Modify Emulex lpfc LPFC_DEFAULT_SG_SEG_CNT to 128 for 
> default larger sg_io capability
> 
> We have been seeing issues with qemu_kvm virtual guests where if they issue
> I/O to direct attached SCSI passthrough LUNS with large I/O sizes such as
> 512K
> the sg_io will fail with -EINVAL.
> 
> Simple reproducer is here.
> 
> strace -e ioctl sg_dd if=/dev/zero of=/dev/sdae bpt=0x400 count=0x400
> blk_sgio=1
> Assume default 'bs' (block size) of 512 bytes
> ioctl(4, SG_IO, {'S', SG_DXFER_FROM_DEV, cmd[6]=[12, 00, 00, 00, 24, 00],
> mx_sb_len=64, iovec_count=0, dxfer_len=36, timeout=6, flags=0,
> data[36]=["\0\0\5\2\37\270\0\2LIO-ORG block-15"...], status=00,
> masked_status=00, sb[0]=[], host_status=0, driver_status=0, resid=0,
> duration=0, info=0}) = 0
> ioctl(4, SG_IO, {'S', SG_DXFER_TO_DEV, cmd[10]=[2a, 00, 00, 00, 00, 00, 00,
> 04, 00, 00], mx_sb_len=64, iovec_count=0, dxfer_len=524288, timeout=6,
> flags=0,
> data[524288]=["\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...]})
> = -1 EINVAL (Invalid argument)
> writing (SG_IO) on sg device, error: Invalid argument
> sg_write failed, seek=0
> Some error occurred,  remaining block count=1024
> 1024+0 records in
> 0+0 records out
> 
> Qlogic defaults to 128 so has not been seeing this but lpfc does not.
> Would like to avoid having to set this in the module parameters to 128.
> 
> Tested-by: Laurence Oberman <lober...@redhat.com>
> Signed-off-by: Laurence Oberman <lober...@redhat.com>
> 
> From 23996dcbdd9b505d18d59ecde961c87f76fc9c2e Mon Sep 17 00:00:00 2001
> From: Laurence Oberman <lober...@redhat.com>
> Date: Wed, 8 Feb 2017 14:49:05 -0500
> Subject: [PATCH] Modified LPFC_DEFAULT_SG_SEG_CNT to 128 to match Qlogic.
> This
>  will allow larger sg_io passthrough for qemu_kvm guests that expect the
>  larger setting for I/O large rthen 512k.
> 
> ---
>  drivers/scsi/lpfc/lpfc.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h
> index 8a20b4e..8f86d41 100644
> --- a/drivers/scsi/lpfc/lpfc.h
> +++ b/drivers/scsi/lpfc/lpfc.h
> @@ -42,7 +42,7 @@
>  #define LPFC_MAX_NS_RETRY3   /* Number of retry attempts to contact
>  the NameServer  before giving up. */
>  #define LPFC_CMD_PER_LUN 3   /* max outstanding cmds per lun */
> -#define LPFC_DEFAULT_SG_SEG_CNT 64   /* sg element count per scsi cmnd */
> +#define LPFC_DEFAULT_SG_SEG_CNT 128  /* sg element count per scsi cmnd */
>  #define LPFC_DEFAULT_MENLO_SG_SEG_CNT 128/* sg element count per scsi
>   cmnd for menlo needs nearly twice as for firmware
>   downloads using bsg */
> --
> 1.8.3.1
> 
> 
> 
>  
> 

Replying to my own email

I was wondering if we could set it to 256 as the default.
This seems to pass sg_io all the way to 0x800

bpt=0x200
Assume default 'bs' (block size) of 512 bytes
ioctl(4, SG_IO, {'S', SG_DXFER_FROM_DEV, cmd[6]=[12, 00, 00, 00, 24, 00], 
mx_sb_len=64, iovec_count=0, dxfer_len=36, timeout=6, flags=0, 
data[36]=["\0\0\5\2\37\270\0\2LIO-ORG block-21"...], status=00, 
masked_status=00, sb[0]=[], host_status=0, driver_status=0, resid=0, 
duration=0, info=0}) = 0
ioctl(4, SG_IO, {'S', SG_DXFER_TO_DEV, cmd[10]=[2a, 00, 00, 00, 00, 00, 00, 02, 
00, 00], mx_sb_len=64, iovec_count=0, dxfer_len=262144, timeout=6, flags=0, 
data[262144]=["\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...],
 status=00, masked_status=00, sb[0]=[], host_status=0, driver_status=0, 
resid=0, duration=30, info=0}) = 0
512+0 records in
512+0 records out
+++ exited with 0 +++

bpt=0x400
Assume default 'bs' (block size) of 512 bytes
ioctl(4, SG_IO, {'S', SG_DXFER_FROM_DEV, cmd[6]=[12, 00, 00, 00, 24, 00], 
mx_sb_len=64, iovec_count=0, dxfer_len=36, timeout=6, flags=0, 
data[36]=["\0\0\5\2\37\270\0\2LIO-ORG block-21"...], status=00, 
masked_status=00, sb[0]=[], host_status=0, driver_status=0, resid=0, 
duration=0, info=0}) = 0
ioctl(4, SG_IO, {'S', SG_DXFER_TO_DEV, cmd[10]=[2a, 00, 00, 00, 00, 00, 00, 04, 
00, 00], mx_sb_len=64, iovec_count=0, dxfer_len=524288, timeout=6, flags=0, 
data[524288]=["\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...],
 status=00, masked_status=00, sb[0]=[], host_status=0, driver_status=0, 
resid=0, duration=43, info=0}) = 0
1024+0 records in
1024+0 records out
+++ exited with 0 +++

bpt=0x800
Assume default 'bs' (b

Patch: lpfc: Modify Emulex lpfc LPFC_DEFAULT_SG_SEG_CNT to 128 for default larger sg_io capability

[Laurence Oberman]

We have been seeing issues with qemu_kvm virtual guests where if they issue 
I/O to direct attached SCSI passthrough LUNS with large I/O sizes such as 512K
the sg_io will fail with -EINVAL.

Simple reproducer is here.

strace -e ioctl sg_dd if=/dev/zero of=/dev/sdae bpt=0x400 count=0x400 blk_sgio=1
Assume default 'bs' (block size) of 512 bytes
ioctl(4, SG_IO, {'S', SG_DXFER_FROM_DEV, cmd[6]=[12, 00, 00, 00, 24, 00], 
mx_sb_len=64, iovec_count=0, dxfer_len=36, timeout=6, flags=0, 
data[36]=["\0\0\5\2\37\270\0\2LIO-ORG block-15"...], status=00, 
masked_status=00, sb[0]=[], host_status=0, driver_status=0, resid=0, 
duration=0, info=0}) = 0
ioctl(4, SG_IO, {'S', SG_DXFER_TO_DEV, cmd[10]=[2a, 00, 00, 00, 00, 00, 00, 04, 
00, 00], mx_sb_len=64, iovec_count=0, dxfer_len=524288, timeout=6, flags=0, 
data[524288]=["\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...]})
 = -1 EINVAL (Invalid argument)
writing (SG_IO) on sg device, error: Invalid argument
sg_write failed, seek=0
Some error occurred,  remaining block count=1024
1024+0 records in
0+0 records out

Qlogic defaults to 128 so has not been seeing this but lpfc does not.
Would like to avoid having to set this in the module parameters to 128.

Tested-by: Laurence Oberman <lober...@redhat.com>
Signed-off-by: Laurence Oberman <lober...@redhat.com>

>From 23996dcbdd9b505d18d59ecde961c87f76fc9c2e Mon Sep 17 00:00:00 2001
From: Laurence Oberman <lober...@redhat.com>
Date: Wed, 8 Feb 2017 14:49:05 -0500
Subject: [PATCH] Modified LPFC_DEFAULT_SG_SEG_CNT to 128 to match Qlogic. This
 will allow larger sg_io passthrough for qemu_kvm guests that expect the
 larger setting for I/O large rthen 512k.

---
 drivers/scsi/lpfc/lpfc.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h
index 8a20b4e..8f86d41 100644
--- a/drivers/scsi/lpfc/lpfc.h
+++ b/drivers/scsi/lpfc/lpfc.h
@@ -42,7 +42,7 @@
 #define LPFC_MAX_NS_RETRY  3   /* Number of retry attempts to contact
   the NameServer  before giving up. */
 #define LPFC_CMD_PER_LUN   3   /* max outstanding cmds per lun */
-#define LPFC_DEFAULT_SG_SEG_CNT 64 /* sg element count per scsi cmnd */
+#define LPFC_DEFAULT_SG_SEG_CNT 128/* sg element count per scsi cmnd */
 #define LPFC_DEFAULT_MENLO_SG_SEG_CNT 128  /* sg element count per scsi
cmnd for menlo needs nearly twice as for firmware
downloads using bsg */
-- 
1.8.3.1



 

Re: iscsi_trx going into D state

[Laurence Oberman]

- Original Message -
> From: "Robert LeBlanc" <rob...@leblancnet.us>
> To: "Laurence Oberman" <lober...@redhat.com>
> Cc: "Doug Ledford" <dledf...@redhat.com>, "Nicholas A. Bellinger" 
> <n...@linux-iscsi.org>, "Zhu Lingshan"
> <ls...@suse.com>, "linux-rdma" <linux-r...@vger.kernel.org>, 
> linux-scsi@vger.kernel.org, "Sagi Grimberg"
> <s...@grimberg.me>, "Christoph Hellwig" <h...@lst.de>
> Sent: Friday, January 13, 2017 6:38:33 PM
> Subject: Re: iscsi_trx going into D state
> 
> Laurance,
> 
> I'm really starting to think that the stars aligned with the phase of
> the moon or something when I reproduced this in my lab before because
> I've been unable to reproduce it on Infiniband the last two days. The
> problem with this issue is that it is so hard to trigger, but causes a
> lot of problems when it does happen. I really hate wasting people's
> time when I can't reproduce it myself reliably. Please don't waste too
> much time if you can't get it reproduced on Infiniband, I'll have to
> wait until someone with the ConnectX-4-LX cards can replicate it.
> 
> Hmmm you do have ConnectX-4 cards which may have the same bug it
> Ethernet mode. I don't see the RoCE bug on my ConnectX-3 cards, but
> your ConnectX-4 cards may work. Try putting the cards into Ethernet
> mode, set the speed and advertised speed to something lower than the
> max speed and verify that the link speed is that (ethtool). On the
> ConnectX-4-LX cards, I just had to set both interfaces down and then
> back up at the same time, on the ConnectX-3 I had to pull the cable
> (shutting down the client might have worked). Then set up target and
> client with iSER, format and run the test and it should trigger
> automatically.
> 
> Looking at release notes on the ConnectX-4-LX cards, the latest
> firmware may fix the bug that so easily exposes the problem with that
> card. My cards are SuperMicro branded cards and don't have the new
> firmware available yet.
> 
> Good luck.
> 
> Robert LeBlanc
> PGP Fingerprint 79A2 9CA4 6CC4 45DD A904  C70E E654 3BB2 FA62 B9F1
> 
> 
> On Fri, Jan 13, 2017 at 8:10 AM, Laurence Oberman <lober...@redhat.com>
> wrote:
> >
> >
> > - Original Message -
> >> From: "Robert LeBlanc" <rob...@leblancnet.us>
> >> To: "Laurence Oberman" <lober...@redhat.com>
> >> Cc: "Doug Ledford" <dledf...@redhat.com>, "Nicholas A. Bellinger"
> >> <n...@linux-iscsi.org>, "Zhu Lingshan"
> >> <ls...@suse.com>, "linux-rdma" <linux-r...@vger.kernel.org>,
> >> linux-scsi@vger.kernel.org, "Sagi Grimberg"
> >> <s...@grimberg.me>, "Christoph Hellwig" <h...@lst.de>
> >> Sent: Thursday, January 12, 2017 4:26:05 PM
> >> Subject: Re: iscsi_trx going into D state
> >>
> >> Sorry sent prematurely...
> >>
> >> On Thu, Jan 12, 2017 at 2:22 PM, Robert LeBlanc <rob...@leblancnet.us>
> >> wrote:
> >> > I'm having trouble replicating the D state issue on Infiniband (I was
> >> > able to trigger it reliably a couple weeks back, I don't know if OFED
> >> > to verify the same results happen there as well.
> >>
> >> I'm having trouble replicating the D state issue on Infiniband (I was
> >> able to trigger it reliably a couple weeks back, I don't know if OFED
> >> being installed is altering things but it only installed for 3.10. The
> >> ConnectX-4-LX exposes the issue easily if you have those cards.) to
> >> verify the same results happen there as well.
> >>
> >> 
> >> Robert LeBlanc
> >> PGP Fingerprint 79A2 9CA4 6CC4 45DD A904  C70E E654 3BB2 FA62 B9F1
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> >> the body of a message to majord...@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>
> >
> > I am only back in the office next Wednesday.
> > I have this all setup using ConnectX-4 with IB/ISER but have no way of
> > remotely creating the disconnect as I currently have it back-to-back.
> > Have run multiple tests with IB and ISER hard resting the client to break
> > the IB connection but have not been able to reproduce as yet.
> > So it will have to wait until I can pull cables next week as that seemed to
> > be the way you have been reproducing this.
> >
> > This is in a code area I also don't have a lot of knowledge of the flow but
> > have started trying to understand it better.
> >
> > Thanks
> > Laurence
> >
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
Hello Robert

I will try this sometime tomorrow by running in ethernet mode.
Its been days of resets with no reproduction so I agree, very hard ro trproduce 
with Infiniband.

Thanks
Laurence
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: iscsi_trx going into D state

[Laurence Oberman]

- Original Message -
> From: "Robert LeBlanc" <rob...@leblancnet.us>
> To: "Laurence Oberman" <lober...@redhat.com>
> Cc: "Doug Ledford" <dledf...@redhat.com>, "Nicholas A. Bellinger" 
> <n...@linux-iscsi.org>, "Zhu Lingshan"
> <ls...@suse.com>, "linux-rdma" <linux-r...@vger.kernel.org>, 
> linux-scsi@vger.kernel.org, "Sagi Grimberg"
> <s...@grimberg.me>, "Christoph Hellwig" <h...@lst.de>
> Sent: Thursday, January 12, 2017 4:26:05 PM
> Subject: Re: iscsi_trx going into D state
> 
> Sorry sent prematurely...
> 
> On Thu, Jan 12, 2017 at 2:22 PM, Robert LeBlanc <rob...@leblancnet.us> wrote:
> > I'm having trouble replicating the D state issue on Infiniband (I was
> > able to trigger it reliably a couple weeks back, I don't know if OFED
> > to verify the same results happen there as well.
> 
> I'm having trouble replicating the D state issue on Infiniband (I was
> able to trigger it reliably a couple weeks back, I don't know if OFED
> being installed is altering things but it only installed for 3.10. The
> ConnectX-4-LX exposes the issue easily if you have those cards.) to
> verify the same results happen there as well.
> 
> 
> Robert LeBlanc
> PGP Fingerprint 79A2 9CA4 6CC4 45DD A904  C70E E654 3BB2 FA62 B9F1
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

I am only back in the office next Wednesday.
I have this all setup using ConnectX-4 with IB/ISER but have no way of remotely 
creating the disconnect as I currently have it back-to-back.
Have run multiple tests with IB and ISER hard resting the client to break the 
IB connection but have not been able to reproduce as yet.
So it will have to wait until I can pull cables next week as that seemed to be 
the way you have been reproducing this.

This is in a code area I also don't have a lot of knowledge of the flow but 
have started trying to understand it better.

Thanks
Laurence
 
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: iscsi_trx going into D state

[Laurence Oberman]

- Original Message -
> From: "Robert LeBlanc" 
> To: "Doug Ledford" 
> Cc: "Nicholas A. Bellinger" , "Zhu Lingshan" 
> , "linux-rdma"
> , linux-scsi@vger.kernel.org, "Sagi Grimberg" 
> , "Christoph Hellwig"
> 
> Sent: Tuesday, January 3, 2017 7:11:40 PM
> Subject: Re: iscsi_trx going into D state
> 
> With the last patch it is getting hung up on wait_for_completion in
> target_wait_for_sess_cmds. I don't know what t_state or fabric state
> mean. To me it looks like a queue is not being emptied, but it would
> help if someone confirmed this and has some pointers on how to
> properly flush them when the communication is interrupted.
> 
> [  222.989134] Starting iscsit_close_connection.
> [  222.993555] Calling flush_workqueue.
> [  222.997703] Returned from flush_workqueue.
> [  223.005802] isert_wait_conn calling ib_close_qp/ib_drain_qp.
> [  223.011892] isert_wait_conn finished ib_close_qp/ib_drain_qp.
> [  223.018063] isert_wait_conn calling isert_put_unsol_pending_cmds.
> [  223.024574] isert_wait_conn returned from
> isert_put_unsol_pending_cmds.
> [  223.031582] isert_wait_conn calling isert_wait4cmds.
> [  223.036942] isert_wait4cmds calling
> target_sess_cmd_list_set_waiting.
> [  223.043789] isert_wait4cmds returned from
> target_sess_cmd_list_set_waiting.
> [  223.051135] isert_wait4cmds calling target_wait_for_sess_cmds.
> [  223.057362] Waiting for se_cmd: 887ebf88bd00 t_state: 6, fabric
> state: 29
> [  223.064893] target_wait_for_sess_cmds calling
> spin_unlock_irqrestore.
> [  223.071748] target_wait_for_sess_cmds calling wait_for_completion.
> [  224.997636] Calling wait_for_common.
> [  225.001936] Starting __wait_for_common.
> [  225.006226] Calling do_wait_for_common.
> 
> Thanks
> 
> Robert LeBlanc
> PGP Fingerprint 79A2 9CA4 6CC4 45DD A904  C70E E654 3BB2 FA62 B9F1
> 
> 
> On Tue, Jan 3, 2017 at 1:07 PM, Robert LeBlanc  wrote:
> > With this patch I'm not seeing the __ib_drain_sq backtraces, but I'm
> > still seeing the previous backtraces.
> >
> > diff --git a/drivers/infiniband/ulp/isert/ib_isert.c
> > b/drivers/infiniband/ulp/isert/ib_isert.c
> > index 6dd43f6..1e53502 100644
> > --- a/drivers/infiniband/ulp/isert/ib_isert.c
> > +++ b/drivers/infiniband/ulp/isert/ib_isert.c
> > @@ -2595,7 +2595,7 @@ static void isert_wait_conn(struct iscsi_conn *conn)
> >isert_conn_terminate(isert_conn);
> >mutex_unlock(_conn->mutex);
> >
> > -   ib_drain_qp(isert_conn->qp);
> > +   ib_close_qp(isert_conn->qp);
> >isert_put_unsol_pending_cmds(conn);
> >isert_wait4cmds(conn);
> >isert_wait4logout(isert_conn);
> >
> > I was thinking that if the connection was brought down uncleanly then
> > there may be messages(??) it the send queue that would never be
> > consumed by the application, so it would never drain and would have to
> > be forcibly emptied. Maybe there is something stuck in the command
> > queue as well?
> >
> > [(support-1.0) root@prv-0-13-roberttest ~]# ps aux | grep " D "
> > root 15426  0.0  0.0  0 0 ?D12:48   0:00 [iscsi_np]
> > root 15429  0.0  0.0  0 0 ?D12:48   0:00
> > [iscsi_ttx]
> > root 16077  0.0  0.0 112656  2216 pts/0S+   12:55   0:00 grep
> > --color=auto  D
> > [(support-1.0) root@prv-0-13-roberttest ~]# cat /proc/15426/stack
> > [] iscsit_stop_session+0x1b1/0x1c0
> > [] iscsi_check_for_session_reinstatement+0x1e2/0x270
> > [] iscsi_target_check_for_existing_instances+0x30/0x40
> > [] iscsi_target_do_login+0x138/0x630
> > [] iscsi_target_start_negotiation+0x4e/0xa0
> > [] __iscsi_target_login_thread+0x83e/0xf20
> > [] iscsi_target_login_thread+0x24/0x30
> > [] kthread+0xd9/0xf0
> > [] ret_from_fork+0x25/0x30
> > [] 0x
> > [(support-1.0) root@prv-0-13-roberttest ~]# cat /proc/15429/stack
> > [] target_wait_for_sess_cmds+0x49/0x190
> > [] isert_wait_conn+0x1a4/0x2d0 [ib_isert]
> > [] iscsit_close_connection+0x157/0x860
> > [] iscsit_take_action_for_connection_exit+0x7b/0xf0
> > [] iscsi_target_tx_thread+0x150/0x1d0
> > [] kthread+0xd9/0xf0
> > [] ret_from_fork+0x25/0x30
> > [] 0x
> > 
> > Robert LeBlanc
> > PGP Fingerprint 79A2 9CA4 6CC4 45DD A904  C70E E654 3BB2 FA62 B9F1
> >
> >
> > On Fri, Dec 30, 2016 at 4:07 PM, Robert LeBlanc 
> > wrote:
> >> I decided to try something completely different... Running the stock
> >> CentOS 3.10 kernel and OFED 3.4 on both hosts, I'm not seeing the hung
> >> processes and the tests complete successfully. The same seems to be
> >> true for the target on 4.9 and the initiator on 3.10.
> >>
> >> However, with the target on 3.10 and the initiator on 4.9, I get this
> >> on the target:
> >>
> >> [(support-1.0) root@prv-0-13-roberttest ~]# ps aux | grep " D "
> >> root 14791  0.0  

Re: [4.10, panic, regression] iscsi: null pointer deref at iscsi_tcp_segment_done+0x20d/0x2e0

[Laurence Oberman]

- Original Message -
> From: "Laurence Oberman" <lober...@redhat.com>
> To: "Jan Kara" <j...@suse.cz>
> Cc: "Johannes Weiner" <han...@cmpxchg.org>, "Hugh Dickins" 
> <hu...@google.com>, "Linus Torvalds"
> <torva...@linux-foundation.org>, "Dave Chinner" <da...@fromorbit.com>, "Chris 
> Leech" <cle...@redhat.com>, "Linux
> Kernel Mailing List" <linux-ker...@vger.kernel.org>, "Lee Duncan" 
> <ldun...@suse.com>, open-is...@googlegroups.com,
> "Linux SCSI List" <linux-scsi@vger.kernel.org>, linux-bl...@vger.kernel.org, 
> "Christoph Hellwig" <h...@lst.de>,
> "Andrea Arcangeli" <aarca...@redhat.com>
> Sent: Wednesday, January 4, 2017 10:26:09 AM
> Subject: Re: [4.10, panic, regression] iscsi: null pointer deref at 
> iscsi_tcp_segment_done+0x20d/0x2e0
> 
> 
> 
> - Original Message -
> > From: "Jan Kara" <j...@suse.cz>
> > To: "Johannes Weiner" <han...@cmpxchg.org>
> > Cc: "Hugh Dickins" <hu...@google.com>, "Linus Torvalds"
> > <torva...@linux-foundation.org>, "Dave Chinner"
> > <da...@fromorbit.com>, "Chris Leech" <cle...@redhat.com>, "Linux Kernel
> > Mailing List"
> > <linux-ker...@vger.kernel.org>, "Lee Duncan" <ldun...@suse.com>,
> > open-is...@googlegroups.com, "Linux SCSI List"
> > <linux-scsi@vger.kernel.org>, linux-bl...@vger.kernel.org, "Christoph
> > Hellwig" <h...@lst.de>, "Jan Kara"
> > <j...@suse.cz>, "Andrea Arcangeli" <aarca...@redhat.com>
> > Sent: Tuesday, January 3, 2017 7:28:25 AM
> > Subject: Re: [4.10, panic, regression] iscsi: null pointer deref at
> > iscsi_tcp_segment_done+0x20d/0x2e0
> > 
> > On Mon 02-01-17 16:11:36, Johannes Weiner wrote:
> > > On Fri, Dec 23, 2016 at 03:33:29AM -0500, Johannes Weiner wrote:
> > > > On Fri, Dec 23, 2016 at 02:32:41AM -0500, Johannes Weiner wrote:
> > > > > On Thu, Dec 22, 2016 at 12:22:27PM -0800, Hugh Dickins wrote:
> > > > > > On Wed, 21 Dec 2016, Linus Torvalds wrote:
> > > > > > > On Wed, Dec 21, 2016 at 9:13 PM, Dave Chinner
> > > > > > > <da...@fromorbit.com>
> > > > > > > wrote:
> > > > > > > > I unmounted the fs, mkfs'd it again, ran the
> > > > > > > > workload again and about a minute in this fired:
> > > > > > > >
> > > > > > > > [628867.607417] [ cut here ]
> > > > > > > > [628867.608603] WARNING: CPU: 2 PID: 16925 at
> > > > > > > > mm/workingset.c:461
> > > > > > > > shadow_lru_isolate+0x171/0x220
> > > > > > > 
> > > > > > > Well, part of the changes during the merge window were the shadow
> > > > > > > entry tracking changes that came in through Andrew's tree. Adding
> > > > > > > Johannes Weiner to the participants.
> > > > > > > 
> > > > > > > > Now, this workload does not touch the page cache at all - it's
> > > > > > > > entirely an XFS metadata workload, so it should not really be
> > > > > > > > affecting the working set code.
> > > > > > > 
> > > > > > > Well, I suspect that anything that creates memory pressure will
> > > > > > > end
> > > > > > > up
> > > > > > > triggering the working set code, so ..
> > > > > > > 
> > > > > > > That said, obviously memory corruption could be involved and
> > > > > > > result
> > > > > > > in
> > > > > > > random issues too, but I wouldn't really expect that in this
> > > > > > > code.
> > > > > > > 
> > > > > > > It would probably be really useful to get more data points - is
> > > > > > > the
> > > > > > > problem reliably in this area, or is it going to be random and
> > > > > > > all
> > > > > > > over the place.
> > > > > > 
> > > > > > Data point: kswapd got WARNING on mm/workingset.c:457 in
> > > > > > shadow_lru_isolate,
> > > > > > soon fo

Re: [4.10, panic, regression] iscsi: null pointer deref at iscsi_tcp_segment_done+0x20d/0x2e0

[Laurence Oberman]

- Original Message -
> From: "Jan Kara" 
> To: "Johannes Weiner" 
> Cc: "Hugh Dickins" , "Linus Torvalds" 
> , "Dave Chinner"
> , "Chris Leech" , "Linux Kernel 
> Mailing List"
> , "Lee Duncan" , 
> open-is...@googlegroups.com, "Linux SCSI List"
> , linux-bl...@vger.kernel.org, "Christoph 
> Hellwig" , "Jan Kara"
> , "Andrea Arcangeli" 
> Sent: Tuesday, January 3, 2017 7:28:25 AM
> Subject: Re: [4.10, panic, regression] iscsi: null pointer deref at 
> iscsi_tcp_segment_done+0x20d/0x2e0
> 
> On Mon 02-01-17 16:11:36, Johannes Weiner wrote:
> > On Fri, Dec 23, 2016 at 03:33:29AM -0500, Johannes Weiner wrote:
> > > On Fri, Dec 23, 2016 at 02:32:41AM -0500, Johannes Weiner wrote:
> > > > On Thu, Dec 22, 2016 at 12:22:27PM -0800, Hugh Dickins wrote:
> > > > > On Wed, 21 Dec 2016, Linus Torvalds wrote:
> > > > > > On Wed, Dec 21, 2016 at 9:13 PM, Dave Chinner 
> > > > > > wrote:
> > > > > > > I unmounted the fs, mkfs'd it again, ran the
> > > > > > > workload again and about a minute in this fired:
> > > > > > >
> > > > > > > [628867.607417] [ cut here ]
> > > > > > > [628867.608603] WARNING: CPU: 2 PID: 16925 at mm/workingset.c:461
> > > > > > > shadow_lru_isolate+0x171/0x220
> > > > > > 
> > > > > > Well, part of the changes during the merge window were the shadow
> > > > > > entry tracking changes that came in through Andrew's tree. Adding
> > > > > > Johannes Weiner to the participants.
> > > > > > 
> > > > > > > Now, this workload does not touch the page cache at all - it's
> > > > > > > entirely an XFS metadata workload, so it should not really be
> > > > > > > affecting the working set code.
> > > > > > 
> > > > > > Well, I suspect that anything that creates memory pressure will end
> > > > > > up
> > > > > > triggering the working set code, so ..
> > > > > > 
> > > > > > That said, obviously memory corruption could be involved and result
> > > > > > in
> > > > > > random issues too, but I wouldn't really expect that in this code.
> > > > > > 
> > > > > > It would probably be really useful to get more data points - is the
> > > > > > problem reliably in this area, or is it going to be random and all
> > > > > > over the place.
> > > > > 
> > > > > Data point: kswapd got WARNING on mm/workingset.c:457 in
> > > > > shadow_lru_isolate,
> > > > > soon followed by NULL pointer deref in list_lru_isolate, one time
> > > > > when
> > > > > I tried out Sunday's git tree.  Not seen since, I haven't had time to
> > > > > investigate, just set it aside as something to worry about if it
> > > > > happens
> > > > > again.  But it looks like shadow_lru_isolate() has issues beyond
> > > > > Dave's
> > > > > case (I've no XFS and no iscsi), suspect unrelated to his other
> > > > > problems.
> > > > 
> > > > This seems consistent with what Dave observed: we encounter regular
> > > > pages in radix tree nodes on the shadow LRU that should only contain
> > > > nodes full of exceptional shadow entries. It could be an issue in the
> > > > new slot replacement code and the node tracking callback.
> > > 
> > > Both encounters seem to indicate use-after-free. Dave's node didn't
> > > warn about an unexpected node->count / node->exceptional state, but
> > > had entries that were inconsistent with that. Hugh got the counter
> > > warning but crashed on a list_head that's not NULLed in a live node.
> > > 
> > > workingset_update_node() should be called on page cache radix tree
> > > leaf nodes that go empty. I must be missing an update_node callback
> > > where a leaf node gets freed somewhere.
> > 
> > Sorry for dropping silent on this. I'm traveling over the holidays
> > with sporadic access to my emails and no access to real equipment.
> > 
> > The times I managed to sneak away to look at the code didn't turn up
> > anything useful yet.
> > 
> > Andrea encountered the warning as well and I gave him a debugging
> > patch (attached below), but he hasn't been able to reproduce this
> > condition. I've personally never seen the warning trigger, even though
> > the patches have been running on my main development machine for quite
> > a while now. Albeit against an older base; I've updated to Linus's
> > master branch now in case it's an interaction with other new code.
> > 
> > If anybody manages to reproduce this, that would be helpful. Any extra
> > eyes on this would be much appreciated too until I'm back at my desk.
> 
> I was looking into this but I didn't find a way how we could possibly leave
> radix tree node on LRU. So your debug patch looks like a good way forward.
> 
>   Honza
> --
> Jan Kara 
> SUSE Labs, CR
> --
> To unsubscribe from this list: send 

Re: scsi: qla2xxx: Fix apparent cut-n-paste error.

[Laurence Oberman]

- Original Message -
> From: "Dave Jones" <da...@codemonkey.org.uk>
> To: linux-scsi@vger.kernel.org
> Cc: "Quinn Tran" <quinn.t...@cavium.com>
> Sent: Tuesday, December 27, 2016 1:13:21 PM
> Subject: scsi: qla2xxx: Fix apparent cut-n-paste error.
> 
> commit 093df73771bac8a37d607c0e705d157a8cef4c5c introduces two bodies of
> code that look similar but with s/req/rsp/ in the second instance.
> But in one case, it looks like this conversion was missed.
> 
> Signed-off-by: Dave Jones <da...@codemonkey.org.uk>
> 
> diff --git a/drivers/scsi/qla2xxx/qla_os.c b/drivers/scsi/qla2xxx/qla_os.c
> index 8521cfe302e9..ad4edc13ebcf 100644
> --- a/drivers/scsi/qla2xxx/qla_os.c
> +++ b/drivers/scsi/qla2xxx/qla_os.c
> @@ -466,7 +466,7 @@ static void qla2x00_free_queues(struct qla_hw_data *ha)
>   continue;
>  
>   rsp = ha->rsp_q_map[cnt];
> - clear_bit(cnt, ha->req_qid_map);
> + clear_bit(cnt, ha->rsp_qid_map);
>   ha->rsp_q_map[cnt] =  NULL;
>   spin_unlock_irqrestore(>hardware_lock, flags);
>   qla2x00_free_rsp_que(ha, rsp);
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Hi Dave

This had me going for a while but I figured out what you were pointing out.
Your change seems correct to me.

Reviewed-by: Laurence Oberman <lober...@redhat.com>
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] qla2xxx: Get mutex lock before checking optrom_state

[Laurence Oberman]

- Original Message -
> From: "Milan P. Gandhi" <mgan...@redhat.com>
> To: linux-scsi@vger.kernel.org
> Cc: "Laurence Oberman" <lober...@redhat.com>, "chad dupuis" 
> <chad.dup...@cavium.com>
> Sent: Saturday, December 24, 2016 11:32:46 AM
> Subject: [PATCH] qla2xxx: Get mutex lock before checking optrom_state
> 
> Hello,
> 
> There is a race condition with qla2xxx optrom functions where
> one thread might modify optrom buffer, optrom_state while
> other thread is still reading from it.
> 
> In couple of crashes, it was found that we had successfully
> passed the following 'if' check where we confirm optrom_state
> to be QLA_SREADING. But by the time we acquired mutex lock
> to proceed with memory_read_from_buffer function, some other
> thread/process had already modified that option rom buffer
> and optrom_state from QLA_SREADING to QLA_SWAITING. Then
> we got ha->optrom_buffer 0x0 and crashed the system:
> 
> if (ha->optrom_state != QLA_SREADING)
> return 0;
> 
> mutex_lock(>optrom_mutex);
> rval = memory_read_from_buffer(buf, count, , ha->optrom_buffer,
> ha->optrom_region_size);
> mutex_unlock(>optrom_mutex);
> 
> 
> With current optrom function we get following crash due to
> a race condition:
> 
> [ 1479.466679] BUG: unable to handle kernel NULL pointer dereference at
> (null)
> [ 1479.466707] IP: [] memcpy+0x6/0x110
> [...]
> [ 1479.473673] Call Trace:
> [ 1479.474296]  [] ? memory_read_from_buffer+0x3c/0x60
> [ 1479.474941]  [] qla2x00_sysfs_read_optrom+0x9c/0xc0
> [qla2xxx]
> [ 1479.475571]  [] read+0xdb/0x1f0
> [ 1479.476206]  [] vfs_read+0x9e/0x170
> [ 1479.476839]  [] SyS_read+0x7f/0xe0
> [ 1479.477466]  [] system_call_fastpath+0x16/0x1b
> 
> 
> Below patch modifies qla2x00_sysfs_read_optrom,
> qla2x00_sysfs_write_optrom functions to get the mutex_lock
> before checking ha->optrom_state to avoid similar crashes.
> 
> The patch was applied and tested and same crashes were no
> longer observed again.
> 
> 
> Tested-by: Milan P. Gandhi <mgan...@redhat.com>
> Signed-off-by: Milan P. Gandhi <mgan...@redhat.com>
> ---
>  drivers/scsi/qla2xxx/qla_attr.c | 18 +-
>  1 file changed, 13 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/scsi/qla2xxx/qla_attr.c
> b/drivers/scsi/qla2xxx/qla_attr.c
> index da5ae11..47ea164 100644
> --- a/drivers/scsi/qla2xxx/qla_attr.c
> +++ b/drivers/scsi/qla2xxx/qla_attr.c
> @@ -329,12 +329,15 @@ qla2x00_sysfs_read_optrom(struct file *filp, struct
> kobject *kobj,
>   struct qla_hw_data *ha = vha->hw;
>   ssize_t rval = 0;
>  
> + mutex_lock(>optrom_mutex);
> +
>   if (ha->optrom_state != QLA_SREADING)
> - return 0;
> + goto out;
>  
> - mutex_lock(>optrom_mutex);
>   rval = memory_read_from_buffer(buf, count, , ha->optrom_buffer,
>   ha->optrom_region_size);
> +
> +out:
>   mutex_unlock(>optrom_mutex);
>  
>   return rval;
> @@ -349,14 +352,19 @@ qla2x00_sysfs_write_optrom(struct file *filp, struct
> kobject *kobj,
>   struct device, kobj)));
>   struct qla_hw_data *ha = vha->hw;
>  
> - if (ha->optrom_state != QLA_SWRITING)
> + mutex_lock(>optrom_mutex);
> +
> + if (ha->optrom_state != QLA_SWRITING) {
> + mutex_unlock(>optrom_mutex);
>   return -EINVAL;
> - if (off > ha->optrom_region_size)
> + }
> + if (off > ha->optrom_region_size) {
> + mutex_unlock(>optrom_mutex);
>   return -ERANGE;
> + }
>   if (off + count > ha->optrom_region_size)
>   count = ha->optrom_region_size - off;
>  
> - mutex_lock(>optrom_mutex);
>   memcpy(>optrom_buffer[off], buf, count);
>   mutex_unlock(>optrom_mutex);
>  
> 
Looks good, and I know it fixed the issue.
Milan, Thank you for this work.

Reviewed-by: Laurence Oberman <lober...@redhat.com> 
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH ] tcm_qla2xxx - Enhancements to the tcm_debug jammer code to jam only data movement commands

[Laurence Oberman]

- Original Message -
> From: "Bart Van Assche" <bart.vanass...@sandisk.com>
> To: "himanshu madhani" <himanshu.madh...@cavium.com>, lober...@redhat.com, 
> n...@daterainc.com
> Cc: linux-scsi@vger.kernel.org
> Sent: Thursday, December 22, 2016 4:54:30 PM
> Subject: Re: [PATCH ] tcm_qla2xxx - Enhancements to the tcm_debug jammer code 
> to jam only data movement commands
> 
> On Thu, 2016-12-22 at 16:46 -0500, Laurence Oberman wrote:
> > Added the possibility of blocking only specific SCSI data movement commands
> > but allowing TUR'S to pass.
> > This has been helpful for debugging many driver/array interoperabilty
> > issues.
> 
> Hello Laurence,
> 
> Your work is appreciated and also that you are posting this patch for
> inclusion
> in the kernel. However, since this is functionality of which I think that it
> is
> useful for all target drivers, shouldn't this kind of functionality be added
> to
> the target core instead of one specific target driver?
> 
> Bart.

Hello Bart

Thank you for this.

Indeed, That's on my plate for next year to try rework this into the target 
core.
Its used a lot here at Red Hat for testing the F/C drivers and multipath via 
the LIO target server.
The first version of this is already in that allows jamming of all commands.
I wanted to share the enhancement which is very useful and has been working 
well here but indeed if we have this in the core its better.

I may have some questions for you as I work on that next year.
Many Thanks Bart for all your help this year.

Regards
Laurence
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH ] tcm_qla2xxx - Enhancements to the tcm_debug jammer code to jam only data movement commands

[Laurence Oberman]

Hello Himanshu and Nicholas,

Was going over my patches and remembered this one got lost.

Himanshu, you acked this already at some point.
Nicholas can we get this on next submission maybe.

Thanks
Laurence

Added the possibility of blocking only specific SCSI data movement commands
but allowing TUR'S to pass.
This has been helpful for debugging many driver/array interoperabilty
issues.

Folded documentation and code into single patch now

Tested by: Laurence Oberman <lober...@redhat.com>
Signed-off-by: Laurence Oberman <lober...@redhat.com>

 Documentation/scsi/tcm_qla2xxx.txt | 16 
 drivers/scsi/qla2xxx/tcm_qla2xxx.c |  7 +++
 drivers/scsi/qla2xxx/tcm_qla2xxx.h |  1 +
 3 files changed, 24 insertions(+)

diff --git a/Documentation/scsi/tcm_qla2xxx.txt 
b/Documentation/scsi/tcm_qla2xxx.txt
index c3a670a..8d26d29 100644
--- a/Documentation/scsi/tcm_qla2xxx.txt
+++ b/Documentation/scsi/tcm_qla2xxx.txt
@@ -20,3 +20,19 @@ echo 1 > 
/sys/kernel/config/target/qla2xxx/21:00:00:24:ff:27:8f:ae/tpgt_1/attrib
 
 Disable jamming on host 4
 echo 0 > 
/sys/kernel/config/target/qla2xxx/21:00:00:24:ff:27:8f:ae/tpgt_1/attrib/jam_host
+
+New feature added with a new attribute called jam_data.
+
+Setting a boolean of 1 for jam_data will allow the allow the dropping of 
data-only SCSI 
+ commands but allow TUR commands to pass.
+Note that for this to work jam_host needs to be set to 0
+
+This has proven very useful for testing Low Level driver response.
+
+Enable host 4 for only data commands to be jammed
+echo 0 > 
/sys/kernel/config/target/qla2xxx/21:00:00:24:ff:27:8f:ae/tpgt_1/attrib/jam_host
+echo 1 > 
/sys/kernel/config/target/qla2xxx/21:00:00:24:ff:27:8f:ae/tpgt_1/attrib/jam_data
+
+Disable jamming on host 4
+echo 0 > 
/sys/kernel/config/target/qla2xxx/21:00:00:24:ff:27:8f:ae/tpgt_1/attrib/jam_host
+echo 0 > 
/sys/kernel/config/target/qla2xxx/21:00:00:24:ff:27:8f:ae/tpgt_1/attrib/jam_data

diff --git a/drivers/scsi/qla2xxx/tcm_qla2xxx.c 
b/drivers/scsi/qla2xxx/tcm_qla2xxx.c
index 6643f6f..1cb3f02 100644
--- a/drivers/scsi/qla2xxx/tcm_qla2xxx.c
+++ b/drivers/scsi/qla2xxx/tcm_qla2xxx.c
@@ -473,6 +473,10 @@ static int tcm_qla2xxx_handle_cmd(scsi_qla_host_t *vha, 
struct qla_tgt_cmd *cmd,
/* return, and dont run target_submit_cmd,discarding command */
return 0;
}
+   if (unlikely(tpg->tpg_attrib.jam_data && (cdb[0]==0x08 || cdb[0]==0x0A 
|| cdb[0]==0x28 || cdb[0]==0x2A))) {
+/* return, and dont run target_submit_cmd,discarding command 
if not TUR*/
+return 0;
+}
 #endif
 
cmd->vha->tgt_counters.qla_core_sbt_cmd++;
@@ -827,6 +831,7 @@ DEF_QLA_TPG_ATTRIB(prod_mode_write_protect);
 DEF_QLA_TPG_ATTRIB(demo_mode_login_only);
 #ifdef CONFIG_TCM_QLA2XXX_DEBUG
 DEF_QLA_TPG_ATTRIB(jam_host);
+DEF_QLA_TPG_ATTRIB(jam_data);
 #endif
 
 static struct configfs_attribute *tcm_qla2xxx_tpg_attrib_attrs[] = {
@@ -837,6 +842,7 @@ static struct configfs_attribute 
*tcm_qla2xxx_tpg_attrib_attrs[] = {
_qla2xxx_tpg_attrib_attr_demo_mode_login_only,
 #ifdef CONFIG_TCM_QLA2XXX_DEBUG
_qla2xxx_tpg_attrib_attr_jam_host,
+   _qla2xxx_tpg_attrib_attr_jam_data,
 #endif
NULL,
 };
@@ -1011,6 +1017,7 @@ static struct se_portal_group *tcm_qla2xxx_make_tpg(
tpg->tpg_attrib.cache_dynamic_acls = 1;
tpg->tpg_attrib.demo_mode_login_only = 1;
tpg->tpg_attrib.jam_host = 0;
+   tpg->tpg_attrib.jam_data = 0;
 
ret = core_tpg_register(wwn, >se_tpg, SCSI_PROTOCOL_FCP);
if (ret < 0) {
diff --git a/drivers/scsi/qla2xxx/tcm_qla2xxx.h 
b/drivers/scsi/qla2xxx/tcm_qla2xxx.h
index 37e026a..789bdeb 100644
--- a/drivers/scsi/qla2xxx/tcm_qla2xxx.h
+++ b/drivers/scsi/qla2xxx/tcm_qla2xxx.h
@@ -35,6 +35,7 @@ struct tcm_qla2xxx_tpg_attrib {
int demo_mode_login_only;
int fabric_prot_type;
int jam_host;
+   int jam_data;
 };
 
 struct tcm_qla2xxx_tpg {
-- 
2.5.5
--
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Testing latest linux-next 4.9 ib_srp and ib_srpt sees I/O capped at 1MB and no merging

[Laurence Oberman]

- Original Message -
> From: "Christoph Hellwig" <h...@infradead.org>
> To: "Laurence Oberman" <lober...@redhat.com>
> Cc: "Christoph Hellwig" <h...@infradead.org>, "Bart Van Assche" 
> <bart.vanass...@sandisk.com>,
> linux-r...@vger.kernel.org, linux-scsi@vger.kernel.org
> Sent: Thursday, December 22, 2016 10:40:49 AM
> Subject: Re: Testing latest linux-next 4.9 ib_srp and ib_srpt sees I/O capped 
> at 1MB and no merging
> 
> Hi Laurance,
> 
> please try the patch below:
> 
> ---
> From 69febe1cfb55844862f768447432249781001f9c Mon Sep 17 00:00:00 2001
> From: Christoph Hellwig <h...@lst.de>
> Date: Thu, 22 Dec 2016 16:38:29 +0100
> Subject: block: add back plugging in __blkdev_direct_IO
> 
> This allows sending larger than 1 MB requess to devices that support
> large I/O sizes.
> 
> Signed-off-by: Christoph Hellwig <h...@lst.de>
> Reported-by: Laurence Oberman <lober...@redhat.com>
> ---
>  fs/block_dev.c | 3 +++
>  fs/iomap.c | 1 -
>  2 files changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/fs/block_dev.c b/fs/block_dev.c
> index 7c45072..206a92a 100644
> --- a/fs/block_dev.c
> +++ b/fs/block_dev.c
> @@ -328,6 +328,7 @@ __blkdev_direct_IO(struct kiocb *iocb, struct iov_iter
> *iter, int nr_pages)
>   struct file *file = iocb->ki_filp;
>   struct inode *inode = bdev_file_inode(file);
>   struct block_device *bdev = I_BDEV(inode);
> + struct blk_plug plug;
>   struct blkdev_dio *dio;
>   struct bio *bio;
>   bool is_read = (iov_iter_rw(iter) == READ);
> @@ -353,6 +354,7 @@ __blkdev_direct_IO(struct kiocb *iocb, struct iov_iter
> *iter, int nr_pages)
>   dio->multi_bio = false;
>   dio->should_dirty = is_read && (iter->type == ITER_IOVEC);
>  
> + blk_start_plug();
>   for (;;) {
>   bio->bi_bdev = bdev;
>   bio->bi_iter.bi_sector = pos >> 9;
> @@ -394,6 +396,7 @@ __blkdev_direct_IO(struct kiocb *iocb, struct iov_iter
> *iter, int nr_pages)
>   submit_bio(bio);
>   bio = bio_alloc(GFP_KERNEL, nr_pages);
>   }
> + blk_finish_plug();
>  
>   if (!dio->is_sync)
>   return -EIOCBQUEUED;
> diff --git a/fs/iomap.c b/fs/iomap.c
> index 354a123..3adf1e1 100644
> --- a/fs/iomap.c
> +++ b/fs/iomap.c
> @@ -844,7 +844,6 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,
> struct iomap_ops *ops,
>   size_t count = iov_iter_count(iter);
>   loff_t pos = iocb->ki_pos, end = iocb->ki_pos + count - 1, ret = 0;
>   unsigned int flags = IOMAP_DIRECT;
> - struct blk_plug plug;
>   struct iomap_dio *dio;
>  
>   lockdep_assert_held(>i_rwsem);
> --
> 2.1.4
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Hello Christoph

The patch works and I now see 4MB I/O

# DISK STATISTICS (/sec)
#   
<-reads-><-writes-><averages> 
Pct
#Time Name   KBytes Merged  IOs Size  KBytes Merged  IOs Size  RWSize  
QLen  Wait SvcTim Util
11:53:58 sdah143360105   35 4096   0  0004096   
  128 28   99
11:53:59 sdah139264102   34 4096   0  0004096   
  129 29   99
11:54:00 sdah143360105   35 4096   0  0004096   
  128 28   99

I think you forgot to remove calls to blk_start_plug and blk_finish_plug in 
fs/iomap.c in your patch.
I took them out and built the test kernel that way.

Let me know if you will just remove those in the final or want a patch.


Thanks fo rthe super quick response

Regards
Laurence
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Testing latest linux-next 4.9 ib_srp and ib_srpt sees I/O capped at 1MB and no merging

[Laurence Oberman]

Hello Christoph, apologies, here is a clear summary of the issue.

During testing of the latest linux-next with rc-10 block layer changes I 
noticed that I/O was being capped at 1MB size and no merging was seen.

The issue was not apparent on 4.8.0-rc8 or earlier.

dd if=/dev/sdw bs=4096k of=/dev/null iflag=direct

### RECORD6 >>> ibclient <<< (1482266543.001) (Tue Dec 20 15:42:23 2016) ###
# DISK STATISTICS (/sec)
#   
<-reads---><-writes-->
 Pct
#Time Name   KBytes Merged  IOs Size  Wait  KBytes Merged  IOs Size  
Wait  RWSize  QLen  Wait SvcTim Util
15:42:23 sdw 278528201   68 4096 2   0  000 
04096 1 2  2   206

Then git checkout master and build final test kernel
4.9.0+

This one clearly shows the I.O at 1MB and no merging.

# DISK STATISTICS (/sec)
#   
<-reads---><-writes-->
 Pct
#Time Name   KBytes Merged  IOs Size  Wait  KBytes Merged  IOs Size  
Wait  RWSize  QLen  Wait SvcTim Util
22:12:48 sdw 1413120  0 1380 1024 2   0  000
 01024 3 2  0   99
22:12:49 sdw 1409024  0 1376 1024 2   0  000
 01024 3 2  0   98
22:12:50 sdw 1445888  0 1412 1024 2   0  000
 01024 3 2  0   98
22:12:51 sdw 1429504  0 1396 1024 2   0  000
 01024 3 2  0   98
22:12:52 sdw 1426432  0 1393 1024 2   0  000
 01024 3 2  0   98
22:12:53 sdw 1408000  0 1375 1024 2   0  000
 01024 3 2  0   98
***     
  

After multiple bisects (6000 revisions, 13 cycles), I got to this one.
Of course there are a huge amount of block layer changes as we know in rc10.

[loberman@ibclient linux-next.orig]$ git bisect bad
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[542ff7bf18c63cf403e36a4a1c71d86dc120d924] block: new direct I/O implementation

This commit is the one that seems to have changed the behavior.
Max I/O size restricted 1MB, even when 4MB I/O is requested, no merging seen.

This is not going to only affect SRP targets.

I will be happy to test any patches and the test bed is always in place.


commit 542ff7bf18c63cf403e36a4a1c71d86dc120d924
Author: Christoph Hellwig 
Date:   Wed Nov 16 23:14:22 2016 -0700

block: new direct I/O implementation

Similar to the simple fast path, but we now need a dio structure to
track multiple-bio completions.  It's basically a cut-down version
of the new iomap-based direct I/O code for filesystems, but without
all the logic to call into the filesystem for extent lookup or
allocation, and without the complex I/O completion workqueue handler
for AIO - instead we just use the FUA bit on the bios to ensure
data is flushed to stable storage.

Signed-off-by: Christoph Hellwig 
Signed-off-by: Jens Axboe 

Many Thanks
Laurence
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Testing latest linux-next 4.9 ib_srp and ib_srpt

[Laurence Oberman]

- Original Message -
> From: "Laurence Oberman" <lober...@redhat.com>
> To: "Bart Van Assche" <bart.vanass...@sandisk.com>
> Cc: linux-r...@vger.kernel.org
> Sent: Wednesday, December 21, 2016 1:34:01 AM
> Subject: Re: Testing latest linux-next 4.9 ib_srp and ib_srpt
> 
> 
> 
> - Original Message -
> > From: "Laurence Oberman" <lober...@redhat.com>
> > To: "Bart Van Assche" <bart.vanass...@sandisk.com>
> > Cc: linux-r...@vger.kernel.org
> > Sent: Tuesday, December 20, 2016 10:31:34 PM
> > Subject: Re: Testing latest linux-next 4.9 ib_srp and ib_srpt
> > 
> > 
> > 
> > - Original Message -
> > > From: "Laurence Oberman" <lober...@redhat.com>
> > > To: "Bart Van Assche" <bart.vanass...@sandisk.com>
> > > Cc: linux-r...@vger.kernel.org
> > > Sent: Tuesday, December 20, 2016 3:44:42 PM
> > > Subject: Re: Testing latest linux-next 4.9 ib_srp and ib_srpt
> > > 
> > > 
> > > 
> > > - Original Message -
> > > > From: "Laurence Oberman" <lober...@redhat.com>
> > > > To: "Bart Van Assche" <bart.vanass...@sandisk.com>
> > > > Cc: linux-r...@vger.kernel.org
> > > > Sent: Tuesday, December 20, 2016 2:43:48 PM
> > > > Subject: Re: Testing latest linux-next 4.9 ib_srp and ib_srpt
> > > > 
> > > > 
> > > > 
> > > > - Original Message -
> > > > > From: "Laurence Oberman" <lober...@redhat.com>
> > > > > To: "Bart Van Assche" <bart.vanass...@sandisk.com>
> > > > > Cc: linux-r...@vger.kernel.org
> > > > > Sent: Tuesday, December 20, 2016 2:33:26 PM
> > > > > Subject: Re: Testing latest linux-next 4.9 ib_srp and ib_srpt
> > > > > 
> > > > > Hello Bart
> > > > > 
> > > > > I pulled the latest linux-next and built kernels for both server and
> > > > > client
> > > > > to rerun all my EDR tests for srp.
> > > > > 
> > > > > For some reason the I/O size is being capped again to 1MB in my
> > > > > testing.
> > > > > Using my same testbed.
> > > > > Remember we spent a lot of time making sure we could do 4MB I/O :)
> > > > > 
> > > > > Its working fine in the RHEL 7.3 kernel so before I start going back
> > > > > testing
> > > > > upstream kernels decided to ask.
> > > > > 
> > > > > Have you tested large I/O with latest linux-next
> > > > > 
> > > > > Server Configuration
> > > > > -
> > > > > Linux fedstorage.bos.redhat.com 4.9.0+
> > > > > 
> > > > > [root@fedstorage modprobe.d]# cat ib_srp.conf
> > > > > options ib_srp cmd_sg_entries=64 indirect_sg_entries=2048
> > > > > 
> > > > > [root@fedstorage modprobe.d]# cat ib_srpt.conf
> > > > > options ib_srpt srp_max_req_size=8296
> > > > > 
> > > > > Also Using
> > > > > 
> > > > > # Set the srp_sq_size
> > > > > for i in
> > > > > /sys/kernel/config/target/srpt/0xfe807cfe900300726e4e
> > > > > /sys/kernel/config/target/srpt/0xfe807cfe900300726e4f
> > > > > do
> > > > >   echo 16384 > $i/tpgt_1/attrib/srp_sq_size
> > > > > done
> > > > > 
> > > > > Client Configuration
> > > > > 
> > > > > Linux ibclient 4.9.0+
> > > > > 
> > > > > [root@ibclient modprobe.d]# cat ib_srp.conf
> > > > > options ib_srp cmd_sg_entries=255 indirect_sg_entries=2048
> > > > > 
> > > > > dd if=/dev/sdw bs=4096k of=/dev/null iflag=direct
> > > > > 
> > > > > ### RECORD4 >>> ibclient <<< (1482261733.001) (Tue Dec 20
> > > > > 14:22:13
> > > > > 2016)
> > > > > ###
> > > > > # DISK STATISTICS (/sec)
> > > > > #
> > > > > <-reads---><-writes--><averages>
> > > > > Pct
> > > > > #Time Name   KBytes Merged  IOs Size  Wait  KBytes Merged
> > > > > IOs
> > >

Re: SG does not ignore dxferp (direct io + mmap)

[Laurence Oberman]

- Original Message -
> From: "Hannes Reinecke" <h...@suse.de>
> To: emi...@redhat.com, "Christoph Hellwig" <h...@infradead.org>
> Cc: "Martin K. Petersen" <martin.peter...@oracle.com>, "Johannes Thumshirn" 
> <jthumsh...@suse.de>, "Laurence Oberman"
> <lober...@redhat.com>, "Eyal Ben David" <bde...@gmail.com>, 
> dgilb...@interlog.com, linux-scsi@vger.kernel.org
> Sent: Friday, December 2, 2016 9:10:01 AM
> Subject: Re: SG does not ignore dxferp (direct io + mmap)
> 
> On 12/02/2016 02:29 PM, Ewan D. Milne wrote:
> > On Fri, 2016-12-02 at 04:21 -0800, Christoph Hellwig wrote:
> >> On Thu, Dec 01, 2016 at 08:40:31AM -0500, Martin K. Petersen wrote:
> >>> Specifically, the problem appears to be caused by the removal of
> >>> the setting of bio->bi_bdev, which would previously be set to NULL.
> >>> If I add:
> >>
> >> Very odd.  For one I would expect it to be NULL anyway, second
> >> I don't see why the behavior changed.  But given that this reverts
> >> to the original assignment and makes things work I'll happily hack it
> >> to get things working again:
> >>
> >> Acked-by: Christoph Hellwig <h...@lst.de>
> >
> > Yeah, I'm not sure I understand this either, apart from the change
> > adjusting the code to effectively do what it used to and making the
> > test case work.  I'm reluctant to cc: stable yet, let me look at this
> > a bit more and I'll post the actual patch soon.
> >
> Plus we found that this is basically a timing issue; we've found that
> supposedly fixed bugs will crop up after ~4k iterations.
> (Johannes did a _lot_ of testing here :-)
> So just because the bug failed to materialize can also mean that you
> simply didn't test long enough.
> 
> Cheers,
> 
> Hannes
> --
> Dr. Hannes Reinecke  Teamlead Storage & Networking
> h...@suse.de +49 911 74053 688
> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
> GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
> HRB 21284 (AG Nürnberg)
> 
Hannes, 
Just to clarify, my original test reverting  commit 
3fa6c507319c897598512da91c010a4ad2ed682c,
what Ewan originally bisected to and running > 10 iterations never failed.

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] bnx2fc: shift wrapping bug in bnx2fc_process_unsol_compl()

[Laurence Oberman]

- Original Message -
> From: "Christophe JAILLET" <christophe.jail...@wanadoo.fr>
> To: qlogic-storage-upstr...@qlogic.com, j...@linux.vnet.ibm.com, "martin 
> petersen" <martin.peter...@oracle.com>
> Cc: linux-scsi@vger.kernel.org, linux-ker...@vger.kernel.org, 
> kernel-janit...@vger.kernel.org, "Christophe JAILLET"
> <christophe.jail...@wanadoo.fr>
> Sent: Saturday, November 26, 2016 1:36:29 PM
> Subject: [PATCH] bnx2fc: shift wrapping bug in bnx2fc_process_unsol_compl()
> 
> BNX2FC_NUM_ERR_BITS is 63. err_warn_bit_map is a u64. So, to make sure that
> no shift wrapping will occur, we need need additionnal casting.
> 
> The same test is already done a few lines above and '(u64)1' is already
> used there. So just do the same here.
> 
> Signed-off-by: Christophe JAILLET <christophe.jail...@wanadoo.fr>
> ---
> I guess that this could also be written with a '1ULL << i' which would be
> cleaner and less verbose IMHO, but apparently this driver does not use
> such things yet. So keep the current style with casting.
> ---
>  drivers/scsi/bnx2fc/bnx2fc_hwi.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/scsi/bnx2fc/bnx2fc_hwi.c
> b/drivers/scsi/bnx2fc/bnx2fc_hwi.c
> index 5ff9f89c17c7..a9dccb3b49cc 100644
> --- a/drivers/scsi/bnx2fc/bnx2fc_hwi.c
> +++ b/drivers/scsi/bnx2fc/bnx2fc_hwi.c
> @@ -829,7 +829,7 @@ static void bnx2fc_process_unsol_compl(struct
> bnx2fc_rport *tgt, u16 wqe)
>   ((u64)err_entry->data.err_warn_bitmap_hi << 32) |
>   (u64)err_entry->data.err_warn_bitmap_lo;
>   for (i = 0; i < BNX2FC_NUM_ERR_BITS; i++) {
> - if (err_warn_bit_map & (u64) (1 << i)) {
> + if (err_warn_bit_map & (u64)((u64)1 << i)) {
>   err_warn = i;
>   break;
>   }
> --
> 2.9.3
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Looks fine to me.
Reviewed-by: Laurence Oberman <lober...@redhat.com>
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SG does not ignore dxferp (direct io + mmap)

[Laurence Oberman]

- Original Message -
> From: "Ewan Milne" <emi...@redhat.com>
> To: "Johannes Thumshirn" <jthumsh...@suse.de>
> Cc: "Laurence Oberman" <lober...@redhat.com>, "Eyal Ben David" 
> <bde...@gmail.com>, dgilb...@interlog.com,
> linux-scsi@vger.kernel.org
> Sent: Friday, November 25, 2016 12:56:16 PM
> Subject: Re: SG does not ignore dxferp (direct io + mmap)
> 
> >> ---
> >> 
> >> In other words, this commit made the bad behavior go away in 4.8.
> >> Need to look at this in more detail, it doesn't appear as if this patch
> >> was intended to fix such a problem.
> >> 
> >> -Ewan
> >
> >Are you sure it did? I can repropduce copy_to_user() errors with 4.8 as
> >well.
> >Using the very same reproducer. On 4.8 it's just harder to trigger and
> >doesn't trigger on AHCI as fas as I can telli (maybe I just haven't hit
> >it hard enough). I can trigger it on QEMUs SCSI CDROM emulation and hpsa
> >though. I cannot however trigger this with a minimal config inside an
> >initrd.
> 
> It did for Eyal's supplied test case on my machine, but that was not an
> exhaustive test, and I am a little suspicious that the behavior change was
> due to a side-effect of the patch rather than actually fixing something.
> 
> I think what we need to understand is what caused the regression in the
> first place, I probably should have been bisecting the original failure
> rather than trying to find where it started working.
> 
> I was running against an internal (physical) drive.
> 
> -Ewan
> 

My 10 loop was against an HPSA target and passed all tests.
Again, all I did was patch the 4.7.9 with the 2 line changes, the rest of the 
patch was line breaks.

I guess we need to understand when it first broke and what caused that, versus 
what seems to correct it.
Thanks
Laurence
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SG does not ignore dxferp (direct io + mmap)

[Laurence Oberman]

- Original Message -
> From: "Eyal Ben David" <bde...@gmail.com>
> To: "Johannes Thumshirn" <jthumsh...@suse.de>
> Cc: "Ewan D. Milne" <emi...@redhat.com>, "Laurence Oberman" 
> <lober...@redhat.com>, dgilb...@interlog.com,
> linux-scsi@vger.kernel.org
> Sent: Friday, November 25, 2016 7:36:34 AM
> Subject: Re: SG does not ignore dxferp (direct io + mmap)
> 
> On Fri, Nov 25, 2016 at 1:53 PM, Johannes Thumshirn <jthumsh...@suse.de>
> wrote:
> > On Fri, Nov 25, 2016 at 01:20:34PM +0200, Eyal Ben David wrote:
> >> Note that sg_mmap_read does not parse the SCSI sense, so the script
> >> might fail for other reasons (some SCSI error) and think its a zero
> >> byte corruption.
> >
> > But SCSI generic checks for errors and returns -EINVAL on CHECK_CONDITION
> > or
> > DRIVER_SENSE (and sets SG_INFO_CHECK in hdr.info).
> >
> Ah OK. We use async write/read instead of ioctl and forgot that ioctl
> checks the read.
> 
> > Anyways, can you test the patch Ewan found on one of your kernel's that are
> > known to fail?
> 
> All the examples I gave before were on physical hosts and storage at a
> testing lab.
> That would be difficult to do. Sorry.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

I applied just the patch myself and Ewan isolated to the 4.7.9 kernel and the 
issue is resolved for me.
I have 10 loops of the test with no issue.

I repeated the test flushing cache each time as well to make sure

for i in `seq 1 10`; do ./sg_mmap_read -d /dev/sg1 -l 0 -m -b | hexdump | 
grep 6300; done

for i in `seq 1 10`; do ./sg_mmap_read -d /dev/sg1 -l 0 -m -b | hexdump | 
grep 6300; echo 3 > /proc/sys/vm/drop_caches;done

Johannes, you are reproducing another race in your test I think.

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SG does not ignore dxferp (direct io + mmap)

[Laurence Oberman]

- Original Message -
> From: "Eyal Ben David" <bde...@gmail.com>
> To: "Ewan D. Milne" <emi...@redhat.com>
> Cc: "Johannes Thumshirn" <jthumsh...@suse.de>, dgilb...@interlog.com, 
> "Laurence Oberman" <lober...@redhat.com>,
> linux-scsi@vger.kernel.org
> Sent: Tuesday, November 22, 2016 3:55:44 PM
> Subject: Re: SG does not ignore dxferp (direct io + mmap)
> 
> On Tue, Nov 22, 2016 at 8:30 PM, Ewan D. Milne <emi...@redhat.com> wrote:
> >
> > I see the behavior (zero byte) on the 4.4.34, 4.5.7, 4.6.7, and 4.7.10
> > -stable kernels.  But not (of course) on 4.8.10 -stable.
> >
> > It doesn't look like the sg driver, might be something in the mmap code?
> 
> 
> A kernel guy colleague suggested to look at copy_from_user / copy_to_user
> code.
> It was changed in 4.8
> 
> It was OK with 3.13  (Ubuntu 14.04) but from some kernel (prior or equal to
> 4.4)
> until 4.7 we see the bug. It was somehow fixed at 4.8.
> 
> In order to fully understand what happened, there are two changes to find.
> They might not even be related.
> 
> Thanks!
> Eyal
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

So 4.7.9 fails and 4.8.0 works and 4.8.0 is a rebase so we have 

[loberman@localhost linux-stable-4.8.10]$ git log --oneline v4.7.9..v4.8 | wc -l
14552

No obvious single commits stand out for me for copy_from* or copy_to*
There is this:

3fa6c50 mm: optimize copy_page_to/from_iter_iovec
6e05050 sh: fix copy_from_user()
e697100 x86/uaccess: force copy_*_user() to be inlined

I will have to do this the hard way with bisects to figure out which commit 
addresses this.

Back when I have had enough time to do it.

Thanks
Laurence
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SG does not ignore dxferp (direct io + mmap)

[Laurence Oberman]

- Original Message -
> From: "Ewan D. Milne" <emi...@redhat.com>
> To: "Johannes Thumshirn" <jthumsh...@suse.de>
> Cc: dgilb...@interlog.com, "Eyal Ben David" <bde...@gmail.com>, "Laurence 
> Oberman" <lober...@redhat.com>,
> linux-scsi@vger.kernel.org
> Sent: Tuesday, November 22, 2016 1:30:07 PM
> Subject: Re: SG does not ignore dxferp (direct io + mmap)
> 
> On Tue, 2016-11-22 at 09:37 +0100, Johannes Thumshirn wrote:
> > On Mon, Nov 21, 2016 at 01:24:02PM -0500, Ewan Milne wrote:
> > > On Mon, 2016-11-21 at 12:34 -0500, Douglas Gilbert wrote:
> > > > There was also this change which seems closer to the problem area:
> > > > 
> > > > commit 461c7fa126794157484dca48e88effa4963e3af3
> > > > Author: Kirill A. Shutemov <kirill.shute...@linux.intel.com>
> > > > Date:   Tue Feb 2 16:57:35 2016 -0800
> > > > 
> > > >  drivers/scsi/sg.c: mark VMA as VM_IO to prevent migration
> > > > ...
> > > > 
> > > > diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c
> > > > index 503ab8b..5e82067 100644
> > > > --- a/drivers/scsi/sg.c
> > > > +++ b/drivers/scsi/sg.c
> > > > @@ -1261,7 +1261,7 @@ sg_mmap(struct file *filp, struct vm_area_struct
> > > > *vma)
> > > >  }
> > > > 
> > > >  sfp->mmap_called = 1;
> > > > -   vma->vm_flags |= VM_DONTEXPAND | VM_DONTDUMP;
> > > > +   vma->vm_flags |= VM_IO | VM_DONTEXPAND | VM_DONTDUMP;
> > > >  vma->vm_private_data = sfp;
> > > >  vma->vm_ops = _mmap_vm_ops;
> > > >  return 0;
> > > > 
> > > > Doug Gilbert
> > > > 
> > > 
> > > Neither this change nor "sg: fix dxferp in from_to case" appears to
> > > fix the issue when applied on top of 4.4.  Still looking...
> > 
> > This brings bad memories from commit 2d99b55d3 back to live, but this is
> > applied on all test kernels I have.
> > 
> > I too will run some bisection as well now that we have an easy reproducer
> > and
> > my timezone is somewhat ahead. Let's see if we can stretch the workday a
> > bit.
> 
> I see the behavior (zero byte) on the 4.4.34, 4.5.7, 4.6.7, and 4.7.10
> -stable kernels.  But not (of course) on 4.8.10 -stable.
> 
> It doesn't look like the sg driver, might be something in the mmap code?
> 
> -Ewan
> 
> 
> 
> 
> 
> 
I have just re-installed my system here with a much larger /boot partition in 
preparation for the bisect builds.
My prior /boot was too small to hold multiple upstream kernels during the 
bisect.
I expect to have time Friday to Monday to do the bisects and hopefully narrow 
it down to what mmap code changes affect us here.

I will narrow the bisect starting with and 4.7.10 bad and 4.8.10 good.
Thanks
Laurence
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SG does not ignore dxferp (direct io + mmap)

[Laurence Oberman]

- Original Message -
> From: "Eyal Ben David" <bde...@gmail.com>
> To: "Johannes Thumshirn" <jthumsh...@suse.de>
> Cc: "Ewan D. Milne" <emi...@redhat.com>, dgilb...@interlog.com, "Laurence 
> Oberman" <lober...@redhat.com>,
> linux-scsi@vger.kernel.org
> Sent: Tuesday, November 22, 2016 8:48:06 AM
> Subject: Re: SG does not ignore dxferp (direct io + mmap)
> 
> Same problem on Fedora 23
> 
> $ uname -r
> 4.7.10-100.fc23.x86_64
> 
> $ sudo ./sg_mmap_read -d /dev/sg0 -l 0 | od -t x1
> 000 eb 63 90 10 8e d0 bc 00 b0 b8 00 00 8e d8 8e c0
> ...
> 
> $ sudo ./sg_mmap_read -d /dev/sg0 -l 0 -m | od -t x1
> 000 eb 63 90 10 8e d0 bc 00 b0 b8 00 00 8e d8 8e c0
> ...
> 
> $ sudo ./sg_mmap_read -d /dev/sg0 -l 0 -m -b | od -t x1
> 000 00 63 90 10 8e d0 bc 00 b0 b8 00 00 8e d8 8e c0
> ...
> 
> 2016-11-22 10:37 GMT+02:00 Johannes Thumshirn <jthumsh...@suse.de>:
> > On Mon, Nov 21, 2016 at 01:24:02PM -0500, Ewan Milne wrote:
> >> On Mon, 2016-11-21 at 12:34 -0500, Douglas Gilbert wrote:
> >> > There was also this change which seems closer to the problem area:
> >> >
> >> > commit 461c7fa126794157484dca48e88effa4963e3af3
> >> > Author: Kirill A. Shutemov <kirill.shute...@linux.intel.com>
> >> > Date:   Tue Feb 2 16:57:35 2016 -0800
> >> >
> >> >  drivers/scsi/sg.c: mark VMA as VM_IO to prevent migration
> >> > ...
> >> >
> >> > diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c
> >> > index 503ab8b..5e82067 100644
> >> > --- a/drivers/scsi/sg.c
> >> > +++ b/drivers/scsi/sg.c
> >> > @@ -1261,7 +1261,7 @@ sg_mmap(struct file *filp, struct vm_area_struct
> >> > *vma)
> >> >  }
> >> >
> >> >  sfp->mmap_called = 1;
> >> > -   vma->vm_flags |= VM_DONTEXPAND | VM_DONTDUMP;
> >> > +   vma->vm_flags |= VM_IO | VM_DONTEXPAND | VM_DONTDUMP;
> >> >  vma->vm_private_data = sfp;
> >> >  vma->vm_ops = _mmap_vm_ops;
> >> >  return 0;
> >> >
> >> > Doug Gilbert
> >> >
> >>
> >> Neither this change nor "sg: fix dxferp in from_to case" appears to
> >> fix the issue when applied on top of 4.4.  Still looking...
> >
> > This brings bad memories from commit 2d99b55d3 back to live, but this is
> > applied on all test kernels I have.
> >
> > I too will run some bisection as well now that we have an easy reproducer
> > and
> > my timezone is somewhat ahead. Let's see if we can stretch the workday a
> > bit.
> >
> > Byte,
> > Johannes
> >
> > --
> > Johannes Thumshirn  Storage
> > jthumsh...@suse.de+49 911 74053 689
> > SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
> > GF: Felix Imendörffer, Jane Smithard, Graham Norton
> > HRB 21284 (AG Nürnberg)
> > Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Its not failing on 4.8-rc2 so working on tracing rgw ioctl SG_DXFER_FROM_DEV as 
well and will bisect this weekend and/or add some kernel probes.
Just need to get some time.
Johannes or Ewan may beat me to it.

Working RHEL7 

open("/dev/sg1", O_RDWR)= 3
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_SHARED, 3, 0) = 0x7f5645d4a000
ioctl(3, SG_IO, {'S', SG_DXFER_FROM_DEV, cmd[16]=[88, 00, 00, 00, 00, 00, 00, 
00, 00, 00, 00, 00, 00, 01, 00, 00], mx_sb_len=96, iovec_count=0, 
dxfer_len=512, timeout=3, flags=0x4, data[512]=[0x7f5645d4a000], status=00, 
masked_status=00, sb[0]=[], host_status=0, driver_status=0, resid=0, 
duration=1, info=0}) = 0
write(1, 0x7f5645d4a000, 64000 63eb 1090 d08e 00bc b8b0  d88e c08e
)= 64
010 befb 7c00 00bf b906 0200 a4f3 21ea 0006
020 be00 07be 0438 0b75 c683 8110 fefe 7507
030 ebf3 b416 b002 bb01 7c00 80b2 748a 8b01
munmap(0x7f5645d4a000, 8192)= 0
close(3)= 0
exit_group(0)   = ?
+++ exited with 0 +++
040

Failing 4.4

open("/dev/sg1", O_RDWR)= 3
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_SHARED, 3, 0) = 0x7f2951de3000
ioctl(3, SG_IO, {'S', SG_DXFER_FROM_DEV, cmd[16]=[88, 00, 00, 00, 00, 00, 00, 
00, 00, 00, 00, 00, 00, 01, 00, 00]

Re: SG does not ignore dxferp (direct io + mmap)

[Laurence Oberman]

- Original Message -
> From: "Eyal Ben David" <bde...@gmail.com>
> To: emi...@redhat.com
> Cc: "Laurence Oberman" <lober...@redhat.com>, dgilb...@interlog.com, 
> linux-scsi@vger.kernel.org
> Sent: Monday, November 21, 2016 9:55:29 AM
> Subject: Re: SG does not ignore dxferp (direct io + mmap)
> 
> Thanks for your reply,
> 
> On RHEL system it does not occur.
> 
> So far I have seen the problem on Ubuntu 16.04 and Fedora 22 (both
> with kernel 4.4.x)
> 
> 2016-11-21 16:24 GMT+02:00 Ewan D. Milne <emi...@redhat.com>:
> > On Mon, 2016-11-21 at 11:23 +0200, Eyal Ben David wrote:
> >> Hi,
> >>
> >> The utility I mentioned is just a small program that I wrote to learn
> >> more about the problem.
> >>
> >> It is a very simple read16 with options for mmap and dxferp as null or
> >> other.
> >>
> >> Here is the source code:
> >>
> >> == cut here ==
> >>
> >
> > 
> >
> >> 2016-11-21 2:04 GMT+02:00 Laurence Oberman <lober...@redhat.com>:
> >> >
> >> >
> >> > - Original Message -
> >> >> From: "Eyal Ben David" <bde...@gmail.com>
> >> >> To: linux-scsi@vger.kernel.org
> >> >> Sent: Sunday, November 20, 2016 11:02:49 AM
> >> >> Subject: SG does not ignore dxferp (direct io + mmap)
> >> >>
> >> >> Hi all,
> >> >>
> >> >> We have some IO utility that perform the IOs using sg and direct io
> >> >> with
> >> >> mmap.
> >> >> Our current systems are Ubuntu 14.04, RHEL 6,7
> >> >> The IO utility always set dxferp to either the address or mmap of
> >> >> other allocation (valloc)
> >> >> Setting dxferp was harmless since SG is supposed to ignore the address
> >> >> if mmap IO is selected.
> >> >> When porting to Ubuntu 16.04, we had a corruption problem - first byte
> >> >> of a read task is always 0.
> >> >> When setting dxferp as NULL the corruption does not occur any more.
> >> >> This is a regression and not according to SCSI generic documentation.
> >> >>
> >> >> I wrote a small program that shows the change:
> >> >>
> >> >> Read indirect (no mmap), lba=0:
> >> >> ===
> >> >> $ ./sg_mmap_read -d /dev/sg0 -l 0
> >> >> 000 eb 63 90 10 8e d0 bc 00 b0 b8 00 00 8e d8 8e c0
> >> >>
> >> >> Read with mmap, lba=0, dxferp=NULL:
> >> >> 
> >> >> $ ./sg_mmap_read -d /dev/sg0 -l 0 -m
> >> >> 000 eb 63 90 10 8e d0 bc 00 b0 b8 00 00 8e d8 8e c0
> >> >>
> >> >> Read with mmap, lba=0, dxferp=address from mmap
> >> >> ==
> >> >> $ ./sg_mmap_read -d /dev/sg0 -l 0 -m -b
> >> >> 000 00 63 90 10 8e d0 bc 00 b0 b8 00 00 8e d8 8e c0
> >> >>
> >> >> On the older systems all results are the same.
> >> >>
> >> >> Thanks for any answer!
> >> >> --
> >
> > For what it's worth, I ran this on a 4.8 kernel and did not see your
> > problem.  I couldn't reproduce it on a RHEL kernel either.
> >
> > # ./sg_mmap_read -d /dev/sg0 -l 0 | od -x
> > 000 63eb 1090 d08e 00bc b8b0  d88e c08e
> > 020 befb 7c00 00bf b906 0200 a4f3 21ea 0006
> > 040 be00 07be 0438 0b75 c683 8110 fefe 7507
> > 060 ebf3 b416 b002 bb01 7c00 80b2 748a 8b01
> > 100
> > # ./sg_mmap_read -d /dev/sg0 -l 0 -m | od -x
> > 000 63eb 1090 d08e 00bc b8b0  d88e c08e
> > 020 befb 7c00 00bf b906 0200 a4f3 21ea 0006
> > 040 be00 07be 0438 0b75 c683 8110 fefe 7507
> > 060 ebf3 b416 b002 bb01 7c00 80b2 748a 8b01
> > 100
> > # ./sg_mmap_read -d /dev/sg0 -l 0 -m -b | od -x
> > 000 63eb 1090 d08e 00bc b8b0  d88e c08e
> > 020 befb 7c00 00bf b906 0200 a4f3 21ea 0006
> > 040 be00 07be 0438 0b75 c683 8110 fefe 7507
> > 060 ebf3 b416 b002 bb01 7c00 80b2 748a 8b01
> > 100
> >
> > The only recent relevant change I see is:
> >
> > commit 5ecee0a3ee8d74b6950cb41e8989b0c2174568d4
> > Author: Douglas Gilbert <dgilb...@interlog.com>
> > Date:   Thu Mar 3 00:31:29 2016 -0500
> >
> > sg: fix dxferp in from_to case
> >
> > However the kernel I used has tha

Re: SG does not ignore dxferp (direct io + mmap)

[Laurence Oberman]

- Original Message -
> From: "Ewan D. Milne" <emi...@redhat.com>
> To: "Eyal Ben David" <bde...@gmail.com>
> Cc: "Laurence Oberman" <lober...@redhat.com>, dgilb...@interlog.com, 
> linux-scsi@vger.kernel.org
> Sent: Monday, November 21, 2016 9:24:35 AM
> Subject: Re: SG does not ignore dxferp (direct io + mmap)
> 
> On Mon, 2016-11-21 at 11:23 +0200, Eyal Ben David wrote:
> > Hi,
> > 
> > The utility I mentioned is just a small program that I wrote to learn
> > more about the problem.
> > 
> > It is a very simple read16 with options for mmap and dxferp as null or
> > other.
> > 
> > Here is the source code:
> > 
> > == cut here ==
> > 
> 
> 
> 
> > 2016-11-21 2:04 GMT+02:00 Laurence Oberman <lober...@redhat.com>:
> > >
> > >
> > > - Original Message -
> > >> From: "Eyal Ben David" <bde...@gmail.com>
> > >> To: linux-scsi@vger.kernel.org
> > >> Sent: Sunday, November 20, 2016 11:02:49 AM
> > >> Subject: SG does not ignore dxferp (direct io + mmap)
> > >>
> > >> Hi all,
> > >>
> > >> We have some IO utility that perform the IOs using sg and direct io with
> > >> mmap.
> > >> Our current systems are Ubuntu 14.04, RHEL 6,7
> > >> The IO utility always set dxferp to either the address or mmap of
> > >> other allocation (valloc)
> > >> Setting dxferp was harmless since SG is supposed to ignore the address
> > >> if mmap IO is selected.
> > >> When porting to Ubuntu 16.04, we had a corruption problem - first byte
> > >> of a read task is always 0.
> > >> When setting dxferp as NULL the corruption does not occur any more.
> > >> This is a regression and not according to SCSI generic documentation.
> > >>
> > >> I wrote a small program that shows the change:
> > >>
> > >> Read indirect (no mmap), lba=0:
> > >> ===
> > >> $ ./sg_mmap_read -d /dev/sg0 -l 0
> > >> 000 eb 63 90 10 8e d0 bc 00 b0 b8 00 00 8e d8 8e c0
> > >>
> > >> Read with mmap, lba=0, dxferp=NULL:
> > >> 
> > >> $ ./sg_mmap_read -d /dev/sg0 -l 0 -m
> > >> 000 eb 63 90 10 8e d0 bc 00 b0 b8 00 00 8e d8 8e c0
> > >>
> > >> Read with mmap, lba=0, dxferp=address from mmap
> > >> ==
> > >> $ ./sg_mmap_read -d /dev/sg0 -l 0 -m -b
> > >> 000 00 63 90 10 8e d0 bc 00 b0 b8 00 00 8e d8 8e c0
> > >>
> > >> On the older systems all results are the same.
> > >>
> > >> Thanks for any answer!
> > >> --
> 
> For what it's worth, I ran this on a 4.8 kernel and did not see your
> problem.  I couldn't reproduce it on a RHEL kernel either.
> 
> # ./sg_mmap_read -d /dev/sg0 -l 0 | od -x
> 000 63eb 1090 d08e 00bc b8b0  d88e c08e
> 020 befb 7c00 00bf b906 0200 a4f3 21ea 0006
> 040 be00 07be 0438 0b75 c683 8110 fefe 7507
> 060 ebf3 b416 b002 bb01 7c00 80b2 748a 8b01
> 100
> # ./sg_mmap_read -d /dev/sg0 -l 0 -m | od -x
> 000 63eb 1090 d08e 00bc b8b0  d88e c08e
> 020 befb 7c00 00bf b906 0200 a4f3 21ea 0006
> 040 be00 07be 0438 0b75 c683 8110 fefe 7507
> 060 ebf3 b416 b002 bb01 7c00 80b2 748a 8b01
> 100
> # ./sg_mmap_read -d /dev/sg0 -l 0 -m -b | od -x
> 000 63eb 1090 d08e 00bc b8b0  d88e c08e
> 020 befb 7c00 00bf b906 0200 a4f3 21ea 0006
> 040 be00 07be 0438 0b75 c683 8110 fefe 7507
> 060 ebf3 b416 b002 bb01 7c00 80b2 748a 8b01
> 100
> 
> The only recent relevant change I see is:
> 
> commit 5ecee0a3ee8d74b6950cb41e8989b0c2174568d4
> Author: Douglas Gilbert <dgilb...@interlog.com>
> Date:   Thu Mar 3 00:31:29 2016 -0500
> 
> sg: fix dxferp in from_to case
> 
> However the kernel I used has that change, and the
> change purposely does not clear hp->dxferp if
> SG_DXFER_TO_FROM_DEV is specified, which your program
> does not, and the behavior is correct regardless.
> 
> Can you modify your program to output the userspace
> address of your buffer?
> 
> -Ewan
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

This is from a current RHEL6.9 kernel and seems to be unable to reproduce as 
well

Re: SG does not ignore dxferp (direct io + mmap)

[Laurence Oberman]

- Original Message -
> From: "Eyal Ben David" 
> To: linux-scsi@vger.kernel.org
> Sent: Sunday, November 20, 2016 11:02:49 AM
> Subject: SG does not ignore dxferp (direct io + mmap)
> 
> Hi all,
> 
> We have some IO utility that perform the IOs using sg and direct io with
> mmap.
> Our current systems are Ubuntu 14.04, RHEL 6,7
> The IO utility always set dxferp to either the address or mmap of
> other allocation (valloc)
> Setting dxferp was harmless since SG is supposed to ignore the address
> if mmap IO is selected.
> When porting to Ubuntu 16.04, we had a corruption problem - first byte
> of a read task is always 0.
> When setting dxferp as NULL the corruption does not occur any more.
> This is a regression and not according to SCSI generic documentation.
> 
> I wrote a small program that shows the change:
> 
> Read indirect (no mmap), lba=0:
> ===
> $ ./sg_mmap_read -d /dev/sg0 -l 0
> 000 eb 63 90 10 8e d0 bc 00 b0 b8 00 00 8e d8 8e c0
> 
> Read with mmap, lba=0, dxferp=NULL:
> 
> $ ./sg_mmap_read -d /dev/sg0 -l 0 -m
> 000 eb 63 90 10 8e d0 bc 00 b0 b8 00 00 8e d8 8e c0
> 
> Read with mmap, lba=0, dxferp=address from mmap
> ==
> $ ./sg_mmap_read -d /dev/sg0 -l 0 -m -b
> 000 00 63 90 10 8e d0 bc 00 b0 b8 00 00 8e d8 8e c0
> 
> On the older systems all results are the same.
> 
> Thanks for any answer!
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
Hello

Given that we cannot see what your utility (sg_mmap_read) is doing, can we get 
the source for that or an strace of your test here.
I am sure Doug will then be able to help you.

Thanks
Laurence
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] scsi: mpt3sas: fix some spelling mistakes in message and comments

[Laurence Oberman]

- Original Message -
> From: "Alexander Alemayhu" 
> To: "Colin King" 
> Cc: "Sathya Prakash" , "Chaitra P B" 
> , "Suganath Prabu
> Subramani" , "James E . J . Bottomley" 
> , "Martin K .
> Petersen" , "MPT-FusionLinux pdl" 
> ,
> linux-scsi@vger.kernel.org, linux-ker...@vger.kernel.org
> Sent: Wednesday, November 2, 2016 10:07:19 AM
> Subject: Re: [PATCH] scsi: mpt3sas: fix some spelling mistakes in message and 
> comments
> 
> On Mon, Oct 31, 2016 at 09:34:46AM -0600, Colin King wrote:
> > From: Colin Ian King 
> > 
> > Trival fixes, minor spelling mistakes in comments and in a KERN_INFO
> > message.
> 
> It's hard to see which words changed. A summary would have been nice,
> something
> like:
> 
> o "appropiately" -> "appropriately"
> o "becuase" -> "because"
> o "beleive" -> "believe"
> o "hidding" -> "hiding"
> o "managment" -> "management"
> o "reqest" -> "request"
> 
> Thanks.
> 
> --
> Mit freundlichen Grüßen
> 
> Alexander Alemayhu
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
Hello Alexander

Surely the patch shows whats changed exactly,
Its implied in the patch itself.

Thanks
Laurence
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html