On 09/18/2018 03:39 PM, Ming Lei wrote:
> On Tue, Sep 18, 2018 at 09:22:32AM +0800, jianchao.wang wrote:
>> Hi Ming
>>
>> On 09/17/2018 07:35 PM, Ming Lei wrote:
>>> On Mon, Sep 17, 2018 at 10:46:34AM +0800, jianchao.wang wrote:
>>>> Hi Ming
>>>>
>>>> On 09/14/2018 07:33 PM, Ming Lei wrote:
>>>>> On Fri, Sep 14, 2018 at 11:40 AM jianchao.wang
>>>>> <[email protected]> wrote:
>>>>>>
>>>>>> Hi Ming
>>>>>>
>>>>>> On 09/13/2018 08:15 PM, Ming Lei wrote:
>>>>>>> EXPORT_SYMBOL(__scsi_execute);
>>>>>>> @@ -3246,6 +3251,7 @@ static int scsi_internal_device_block(struct
>>>>>>> scsi_device *sdev)
>>>>>>> else
>>>>>>> scsi_wait_for_queuecommand(sdev);
>>>>>>> }
>>>>>>> + wait_event(sdev->admin_wq, !atomic_read(&sdev->nr_admin_pending));
>>>>>>> mutex_unlock(&sdev->state_mutex);
>>>>>>>
>>>>>>> return err;
>>>>>> ...
>>>>>>> diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
>>>>>>> index 3aee9464a7bf..8bcb7ecc0c06 100644
>>>>>>> --- a/drivers/scsi/scsi_sysfs.c
>>>>>>> +++ b/drivers/scsi/scsi_sysfs.c
>>>>>>> @@ -1393,6 +1393,7 @@ void __scsi_remove_device(struct scsi_device
>>>>>>> *sdev)
>>>>>>>
>>>>>>> blk_cleanup_queue(sdev->request_queue);
>>>>>>> cancel_work_sync(&sdev->requeue_work);
>>>>>>> + wait_event(sdev->admin_wq, !atomic_read(&sdev->nr_admin_pending))
>>>>>>
>>>>>> This nr_admin_pending could drain the ongoing
>>>>>> scsi_request_fn/scsi_queue_rq,
>>>>>> but I'm afraid it cannot stop new ones coming in, such as the ones that
>>>>>> have passed
>>>>>> the sdev state checking and have not crossed the
>>>>>> atomic_inc(&sdev->nr_admin_pending).
>>>>>>
>>>>>
>>>>> The counter of .nr_admin_pending is introduced for draining queued
>>>>> admin requests to this scsi device.
>>>>>
>>>>> Actually new requests have been prevented from entering scsi_queue_rq(),
>>>>> please see the two callers of wait_event(sdev->admin_wq,
>>>>> !atomic_read(&sdev->nr_admin_pending)).
>>>>>
>>>> For example
>>>>
>>>> _scsi_execute
>>>> ...
>>>> scsi_internal_device_block
>>>>
>>>> scsi_internal_device_block_nowait
>>>> blk_mq_quiesce_queue
>>>> wait_event(sdev->admin_wq,
>>>> !atomic_read(&sdev->nr_admin_pending))
>>>> &sdev->nr_admin_pending;
>>>>
>>>> blk_execute_rq（...）
>>>>
>>>> atomic_dec(&sdev->nr_admin_pending);
>>>> wake_up_all(&sdev->admin_wq);
>>>>
>>>> Or do you mean the scsi_queue_rq -> scsi_prep_state_check could gate out
>>>> of ?
>>>
>>> I got it, then this issue can be fixed simply by moving
>>> atomic_inc/dec(&sdev->nr_admin_pending)
>>> and related wake_up_all(&sdev->admin_wq) into scsi_admin_queue_rq().
>>>
>>
>> I don't think so. It is a similar scenario.
>>
>> I guess a state checking is needed after
>> atomic_inc(&sdev->nr_admin_pending), like:
>>
>> _scsi_execute
>> ...
>> scsi_internal_device_block
>>
>> scsi_internal_device_block_nowait
>> blk_mq_quiesce_queue
>> wait_event(sdev->admin_wq,
>> !atomic_read(&sdev->nr_admin_pending))
>> atomic_inc(&sdev->nr_admin_pending);
>> if state checking fails
>> goto done
>
> The check will be done in scsi_admin_queue_rq().
>
>>
>> blk_execute_rq（...）
>>
>> atomic_dec(&sdev->nr_admin_pending);
>> wake_up_all(&sdev->admin_wq);
>
> I guess you may misunderstand the purpose of .nr_admin_pending, which is
> for draining requests to .queue_rq(). So it is enough to just move the
> inc/dec of .nr_admin_pending into scsi_admin_queue_rq(), right?
Yes.
But I just think of how to assign with queue quiesce.
The existence of per-host adminq seems to break it.
Thanks
Jianchao
