Re: [developer] implement soft-hba property

[Richard Elling]

> On Feb 3, 2018, at 8:18 AM, Garrett D'Amore  wrote:
> 
> sd.conf is ugly as hell though.

agree 100%

> 
> What we *really* need is a better tunable system — more like what we have 
> with dladm for NICs, but intended for storage.  I think we’d like to be able 
> to set tunables for both targets and HBAs.  I have some ideas here, but 
> precious little time to do anything about it.

yep. when we made the sd changes we also added kstats to count retries and the 
various
causes for timeouts. In the end, it begins to look more like network -- a good 
thing. Multipathing
futher complicates things, but the heavy lifting is in sd.

FWIW, it is relatively easy to use mdb to change the pertinent values 
on-the-fly for testing. One
reason this is important is because, as-is, there is little visibility and 
dtracing to get internal state
of sd is tricky because the function boundaries aren't always where you need 
them to be. For AoE
in particular, you want to move towards faster timeouts and more retries before 
failing the I/O.
 -- richard

> 
>  - Garrett
> 
> On Sat, Feb 3, 2018 at 8:11 AM Richard Elling 
> > 
> wrote:
> 
> On Feb 3, 2018, at 7:57 AM, Igor Kozhukhov  > wrote:
> 
>> Hi All,
>> 
>> i’d like to propose implenetation for soft-hba property for scsi layer with 
>> timing updates.
>> 
>> Problem: we have hw hba like LSI, where we are using direct connetion of 
>> drives to HBA with low latency.
> 
> In the past, we’ve done this with an override property, similar to the other 
> properties for sd devices like physical block size, retries, etc. In other 
> words, adding a new global (aka /etc/system virus) is not as elegant as 
> adding an override in sd.conf
> 
>   -- richard
> 
>> 
>> We have iSCSI drives with sd module usage where we try to use timing related 
>> to HW HBA.
>> but, latency to denwork drives are different with direct connected local 
>> drives.
>> 
>> I have impleneted my idea to split sd_soft_io_time for hw nad soft drives:
>> https://bitbucket.org/dilos/dilos-illumos/commits/b82c668f9e837864afca983809c4d4a28f5051b0
>>  
>> 
>> https://bitbucket.org/dilos/dilos-illumos/commits/4d805e127dc4211671ad70c2e026014e6de4a991
>>  
>> 
>> 
>> and i’m using it with iscsi and aoe drives.
>> 
>> right now we can use /etc/system vriable sd_soft_io_time
>> for soft drives and it is not impacts for real hw drives.
>> 
>> it is example of my idea and if it is can be interest to others i can try 
>> contribute it ot you can try port it to illumos tree.
>> i have no time try to setup and use original build env and can provide 
>> reports based on DilOS, but i hope change sin sd module based on smartos 
>> changes and can be usable/applicable for illumos.
>> 
>> Best regards,
>> -Igor
>> 
> 
> openzfs-developer | Archives 
> 
>  | Powered by Topicbox 

--
openzfs-developer
Archives: 
https://openzfs.topicbox.com/groups/developer/discussions/Ta14496565d3bc26c-Mb87b51ac094daaeb8e8e8561
Powered by Topicbox: https://topicbox.com

Re: [developer] implement soft-hba property

[Garrett D'Amore]

sd.conf is ugly as hell though.

What we *really* need is a better tunable system — more like what we have
with dladm for NICs, but intended for storage.  I think we’d like to be
able to set tunables for both targets and HBAs.  I have some ideas here,
but precious little time to do anything about it.

 - Garrett

On Sat, Feb 3, 2018 at 8:11 AM Richard Elling <
richard.ell...@richardelling.com> wrote:

>
> On Feb 3, 2018, at 7:57 AM, Igor Kozhukhov  wrote:
>
> Hi All,
>
> i’d like to propose implenetation for soft-hba property for scsi layer
> with timing updates.
>
> Problem: we have hw hba like LSI, where we are using direct connetion of
> drives to HBA with low latency.
>
>
> In the past, we’ve done this with an override property, similar to the
> other properties for sd devices like physical block size, retries, etc. In
> other words, adding a new global (aka /etc/system virus) is not as elegant
> as adding an override in sd.conf
>
>   -- richard
>
>
> We have iSCSI drives with sd module usage where we try to use timing
> related to HW HBA.
> but, latency to denwork drives are different with direct connected local
> drives.
>
> I have impleneted my idea to split sd_soft_io_time for hw nad soft drives:
>
> https://bitbucket.org/dilos/dilos-illumos/commits/b82c668f9e837864afca983809c4d4a28f5051b0
>
> https://bitbucket.org/dilos/dilos-illumos/commits/4d805e127dc4211671ad70c2e026014e6de4a991
>
> and i’m using it with iscsi and aoe drives.
>
> right now we can use /etc/system vriable sd_soft_io_time
> for soft drives and it is not impacts for real hw drives.
>
> it is example of my idea and if it is can be interest to others i can try
> contribute it ot you can try port it to illumos tree.
> i have no time try to setup and use original build env and can provide
> reports based on DilOS, but i hope change sin sd module based on smartos
> changes and can be usable/applicable for illumos.
>
> Best regards,
> -Igor
>
> *openzfs-developer* | Archives
> 
> | Powered by Topicbox 

--
openzfs-developer
Archives: 
https://openzfs.topicbox.com/groups/developer/discussions/Ta14496565d3bc26c-M4f82719b034d4d8e242e40bb
Powered by Topicbox: https://topicbox.com

Re: [developer] implement soft-hba property

[Richard Elling]

> On Feb 3, 2018, at 7:57 AM, Igor Kozhukhov  wrote:
> 
> Hi All,
> 
> i’d like to propose implenetation for soft-hba property for scsi layer with 
> timing updates.
> 
> Problem: we have hw hba like LSI, where we are using direct connetion of 
> drives to HBA with low latency.

In the past, we’ve done this with an override property, similar to the other 
properties for sd devices like physical block size, retries, etc. In other 
words, adding a new global (aka /etc/system virus) is not as elegant as adding 
an override in sd.conf

  -- richard

> 
> We have iSCSI drives with sd module usage where we try to use timing related 
> to HW HBA.
> but, latency to denwork drives are different with direct connected local 
> drives.
> 
> I have impleneted my idea to split sd_soft_io_time for hw nad soft drives:
> https://bitbucket.org/dilos/dilos-illumos/commits/b82c668f9e837864afca983809c4d4a28f5051b0
> https://bitbucket.org/dilos/dilos-illumos/commits/4d805e127dc4211671ad70c2e026014e6de4a991
> 
> and i’m using it with iscsi and aoe drives.
> 
> right now we can use /etc/system vriable sd_soft_io_time
> for soft drives and it is not impacts for real hw drives.
> 
> it is example of my idea and if it is can be interest to others i can try 
> contribute it ot you can try port it to illumos tree.
> i have no time try to setup and use original build env and can provide 
> reports based on DilOS, but i hope change sin sd module based on smartos 
> changes and can be usable/applicable for illumos.
> 
> Best regards,
> -Igor
> 
> openzfs-developer | Archives | Powered by Topicbox

--
openzfs-developer
Archives: 
https://openzfs.topicbox.com/groups/developer/discussions/Ta14496565d3bc26c-M8f8a4e821a54690ec36d2f07
Powered by Topicbox: https://topicbox.com

[developer] implement soft-hba property

[Igor Kozhukhov]

Hi All,

i’d like to propose implenetation for soft-hba property for scsi layer with 
timing updates.

Problem: we have hw hba like LSI, where we are using direct connetion of drives 
to HBA with low latency.

We have iSCSI drives with sd module usage where we try to use timing related to 
HW HBA.
but, latency to denwork drives are different with direct connected local drives.

I have impleneted my idea to split sd_soft_io_time for hw nad soft drives:
https://bitbucket.org/dilos/dilos-illumos/commits/b82c668f9e837864afca983809c4d4a28f5051b0
https://bitbucket.org/dilos/dilos-illumos/commits/4d805e127dc4211671ad70c2e026014e6de4a991

and i’m using it with iscsi and aoe drives.

right now we can use /etc/system vriable sd_soft_io_time
for soft drives and it is not impacts for real hw drives.

it is example of my idea and if it is can be interest to others i can try 
contribute it ot you can try port it to illumos tree.
i have no time try to setup and use original build env and can provide reports 
based on DilOS, but i hope change sin sd module based on smartos changes and 
can be usable/applicable for illumos.

Best regards,
-Igor


--
openzfs-developer
Archives: 
https://openzfs.topicbox.com/groups/developer/discussions/Ta14496565d3bc26c-M30b74fb75652c97ec3059901
Powered by Topicbox: https://topicbox.com