On Mon, Apr 13, 2020 at 8:16 AM Eugenio Perez Martin <[email protected]> wrote:
> On Fri, Apr 10, 2020 at 4:40 AM Jason Wang <[email protected]> wrote: > > > > > > On 2020/4/10 上午5:06, Michael S. Tsirkin wrote: > > > On Tue, Apr 07, 2020 at 11:52:46AM +0200, Eugenio Perez Martin wrote: > > >> Hi! > > >> > > >> So, from the previous mails, it seems that monitoring the used ring > > >> (and the packed descriptors) is a good first step in that direction, > > >> as DPDK did. This way, the device does not need to worry about the > > >> dirty page tracking using a bitmap and the PCI writes limitation, and > > >> we can evaluate later the proposed alternatives: > > >> * Alternate used descriptors in packed. > > >> * vDPA interface for vDPA devices in a convenient format. > > >> > > >> Any thoughts? Do you think that we should start with another way? > > >> > > >> Thanks! > > > I am concerned that with software in data path, we'll hit RX queue > > > underruns, won't we? > > > > > > Do you mean it will lose some performance? If yes, I think so. > > > > > > > Two ways to avoid underruns: > > > - dirty page tracking > > > - page faults > > > > > > It looks to me this will lead even worse performance than software path? > > There will be lots of page faults during RX. > > > > Another direction is to track dirty pages via IOMMU. E.g recent Intel > > IOMMU has EA and D bit which could be used for tracking pages wrote by > > devices but not CPU. > > > > So this could be added on top of the dirty tracking mechanism, isn't? > or would it be easier to start another-way around, and to start using > modern IOMMU and then extend to old generic code? > > > > > > > > > I'm working on a proposal for page faults now. > > > > > > I guess it's better to have a transport independent method. > > > > > > > If someone wants > > > to work on dirty tracking in addition, that's also an option. > > > > > > I remember Rob mention some challenges of implementing dirty bitmap, I > > wonder something like queue based interface would be better (similar to > > Peter did for KVM)? > > > > I think that the main challenge was to write bits instead of writes > using PCI bus. > > From a conversation with Juan, another solution could be to do a byte > map DPT, where a byte represents a page, not a bit. While I find this > solution simpler, I'm not sure about the performance implications of > track this way (memory/TLB pressure, etc). Rob and Juan, please > correct me if I'm wrong or missed something. > RJM>] You captured it correctly. We (Broadcom) discussed the idea of using a byte (being 0xFF) which effectively indicate that 64KB were dirty, this would cause potentially refreshing pages to the remote that weren't needed to be refreshed, slowing down the migration. I like the idea of mapping a byte to a page, and teach QEMU (or vDPA, or ...) how to interpret. > > Compared to the used ring interposition, much more logic needs to be > in the device driver for both alternatives (byte mapping and queue), > isn't it? > RJM> Yes, but this would (should) lead to best performance. > > > Thanks > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
