On 2020/4/13 下午9:30, Rob Miller wrote:
On Mon, Apr 13, 2020 at 8:16 AM Eugenio Perez Martin
<[email protected] <mailto:[email protected]>> wrote:
On Fri, Apr 10, 2020 at 4:40 AM Jason Wang <[email protected]
<mailto:[email protected]>> wrote:
>
>
> On 2020/4/10 上午5:06, Michael S. Tsirkin wrote:
> > On Tue, Apr 07, 2020 at 11:52:46AM +0200, Eugenio Perez Martin
wrote:
> >> Hi!
> >>
> >> So, from the previous mails, it seems that monitoring the
used ring
> >> (and the packed descriptors) is a good first step in that
direction,
> >> as DPDK did. This way, the device does not need to worry
about the
> >> dirty page tracking using a bitmap and the PCI writes
limitation, and
> >> we can evaluate later the proposed alternatives:
> >> * Alternate used descriptors in packed.
> >> * vDPA interface for vDPA devices in a convenient format.
> >>
> >> Any thoughts? Do you think that we should start with another way?
> >>
> >> Thanks!
> > I am concerned that with software in data path, we'll hit RX queue
> > underruns, won't we?
>
>
> Do you mean it will lose some performance? If yes, I think so.
>
>
> > Two ways to avoid underruns:
> > - dirty page tracking
> > - page faults
>
>
> It looks to me this will lead even worse performance than
software path?
> There will be lots of page faults during RX.
>
> Another direction is to track dirty pages via IOMMU. E.g recent
Intel
> IOMMU has EA and D bit which could be used for tracking pages
wrote by
> devices but not CPU.
>
So this could be added on top of the dirty tracking mechanism, isn't?
or would it be easier to start another-way around, and to start using
modern IOMMU and then extend to old generic code?
>
> >
> > I'm working on a proposal for page faults now.
>
>
> I guess it's better to have a transport independent method.
>
>
> > If someone wants
> > to work on dirty tracking in addition, that's also an option.
>
>
> I remember Rob mention some challenges of implementing dirty
bitmap, I
> wonder something like queue based interface would be better
(similar to
> Peter did for KVM)?
>
I think that the main challenge was to write bits instead of writes
using PCI bus.
From a conversation with Juan, another solution could be to do a byte
map DPT, where a byte represents a page, not a bit. While I find this
solution simpler, I'm not sure about the performance implications of
track this way (memory/TLB pressure, etc). Rob and Juan, please
correct me if I'm wrong or missed something.
It would lead large working set, but the actual impact may require
benchmark.
RJM>] You captured it correctly. We (Broadcom) discussed the idea of
using a byte (being 0xFF) which effectively indicate that 64KB were dirty,
this would cause potentially refreshing pages to the remote that weren't
needed to be refreshed, slowing down the migration. I like the idea of
mapping
a byte to a page, and teach QEMU (or vDPA, or ...) how to interpret.
That's a way to go, I think we can try to propose an API and move the
discussion upstream.
Compared to the used ring interposition, much more logic needs to be
in the device driver for both alternatives (byte mapping and queue),
isn't it?
RJM> Yes, but this would (should) lead to best performance.
I agree, I think we can start from software support and discuss the
hardware support in parallel.
Thanks
> Thanks
>
---------------------------------------------------------------------
To unsubscribe, e-mail:
[email protected]
<mailto:[email protected]>
For additional commands, e-mail:
[email protected]
<mailto:[email protected]>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
