Furthermore, drivers that don't want to re-spin for this should continue
to function normally. The only reason to avoid use of the IOMMU is
either performance optimization or to workaround incorrect driver
implementation.
As indicated, I don't mind have have some properties "undocumented"
usage, but we really should be encouraging folks that care to avoid the
IOMMU to use the documented API.
(And, IMO, there really should be no reason to avoid the IOMMU if you've
written your code to perform well under regular DDI compliant DMA. The
IOMMU itself affords some significant benefits, making it much easier to
get single contiguous DMA segments, reducing the need to deal with
multiple cookies -- scatter gather -- or DMA windows. Never mind the
huge benefits it affords for virtualization and the fault tolerance
benefits that come for free as well.)
-- Garrett
Wesley Shao wrote:
> FORCE_PHYSICAL allows finer granularity and the proposed properties
> make the drivers non-portable to SPARC platforms. They should really be
> used for internal debug.
>
> Besides, they make the drivers hard to run on virtualized platforms.
>
> Wes
>
> Vikram Hedgde wrote:
>> Hi,
>>
>>> Yes. I'd prefer not to "document" the alternative mechanism, since
>>> I view that as a hack/workaround rather than the correct behavior of
>>> using the DMA attributes.
>> How about 3rd party drivers that driver vendors don't want to rev and
>> re-release solely for this. Shouldn't there be a switch that they can
>> suggest to customers to use with their existing driver binaries ?
>>
>> Vikram
>>
>>
>> Garrett D'Amore wrote:
>> Vikram Hegde wrote:
>>> Garrett D'Amore wrote:
>>>> It would be cooler if this could be dealt with using ordinary DMA
>>>> attributes. Won't DDI_DMA_FORCE_PHYSICAL (see the dma_attr_flags
>>>> for ddi_dma_attr(9s)) work for this?
>>>>
>>> Yes, that would work as well and is the preferred and established
>>> solution. This flag will be respected by the IOMMU. However, it
>>> seems that an alternative mechanism that involves no touching of the
>>> driver code is desirable as well. Do you agree ?
>>
>> Yes. I'd prefer not to "document" the alternative mechanism, since I
>> view that as a hack/workaround rather than the correct behavior of
>> using the DMA attributes.
>>
>> -- Garrett
>>>
>>> Thanks,
>>> Vikram
>>>> - Garrett
>>>>
>>>>
>>>> Vikram Hegde wrote:
>>>>> Hi Kais,
>>>>>
>>>>> Below is the text I am planning to add to the PSARC case. And yes,
>>>>> the disable properties will be a part of this PSARC case.
>>>>>
>>>>> Vikram
>>>>>
>>>>> =====================================================================
>>>>> Enabling/Disabling the IOMMU
>>>>> =============================
>>>>> It is recognized that driver or platform developers may not in
>>>>> certain cases desire the
>>>>> IOMMU to be enabled. To accomodate such needs we propose the
>>>>> following global properties
>>>>> in the iommu driver.conf file
>>>>>
>>>>> "global-disable" - can be set to 1 to disable IOMMU platform wide.
>>>>> "exclude-list" - list of driver names that want the IOMMU disabled.
>>>>> ======================================================================
>>>>>
>>>>>
>>>>> Vikram
>>>>>
>>>>> Kais Belgaied wrote:
>>>>>> Vikram, Jerry,
>>>>>>
>>>>>> So, is the ability to disable the IOMMU on a per driver basis in
>>>>>> this case's scope?
>>>>>> what interface is being proposed for doing this?
>>>>>>
>>>>>> Jerry, assuming the answer the to first question above is 'yes',
>>>>>> the case cannot be in
>>>>>> a "closed approved automatic" state while waiting for some of its
>>>>>> spec.
>>>>>>
>>>>>> Kais.
>>>>>>
>>>>>> On 09/03/08 23:16, Matthew Jacob wrote:
>>>>>>> Garrett D'Amore wrote:
>>>>>>>> Vikram Hegde wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>>> Abilities to disable the IOMMU as a choice? Performance
>>>>>>>>>> impact on various systems and interactions with other
>>>>>>>>>> projects that have gone to some length to work within S/G and
>>>>>>>>>> its performance characteristics. How will this affect them?
>>>>>>>>>> Cache effects? Interactions with some VM work currently being
>>>>>>>>>> undertaken?
>>>>>>>>>>
>>>>>>>>>> I can't imagine such a major change being
>>>>>>>>>> closed/approved/automatic.
>>>>>>>>> Performance is driver dependant. There is no way a single
>>>>>>>>> project team can test the gamut of drivers out there and
>>>>>>>>> certify that the IOMMU will give acceptable performance for
>>>>>>>>> all needs (which is itself a subjective opinion). We have
>>>>>>>>> always planned to provide the ability to disable the IOMMU on
>>>>>>>>> a per driver basis. Does that address your performance concerns ?
>>>>>>>>
>>>>>>>> Actually, for many cases, IOMMU may *improve* performance.
>>>>>>>> This is because it will make it easier to satisfy device
>>>>>>>> constraints for DMA resources without requiring the use of a
>>>>>>>> bounce buffer or dma windows.
>>>>>>>
>>>>>>> That is almost always the exception now. The other issue IIRC is
>>>>>>> the interaction between the IOMMU and memory caches. In many
>>>>>>> data movement appliance contexts you use scatter-gather
>>>>>>> (scatter-gather in name only- contiguous memory is good) to
>>>>>>> physical pages which are never mapped in.
>>>>>>>
>>>>>>> The main point here is to avoid a situation that we're currently
>>>>>>> in now with respect MSI-X and Interrupt/CPU binding and make
>>>>>>> sure that there are sufficient control surfaces to allow for a
>>>>>>> broad range of possible implementation choices.
>>>>>>>
>>>>>>>>
>>>>>>>> That said, drivers which assume that setup and tear down of DMA
>>>>>>>> is "cheap" may take a perf. hit when using an IOMMU. This is
>>>>>>>> already the case for SPARC systems, btw.
>>>>>>>
>>>>>>> Except for those (ancient) ones that used 36-bit MBus physical
>>>>>>> addresses and bypassed the IOMMU.
>
>
