On Mon, 6 Mar 2023 12:05:06 +0100 Cédric Le Goater <c...@redhat.com> wrote:
> On 3/6/23 10:45, Joao Martins wrote: > > On 06/03/2023 02:19, Alex Williamson wrote: > >> On Sun, 5 Mar 2023 23:33:35 +0000 > >> Joao Martins <joao.m.mart...@oracle.com> wrote: > >> > >>> On 05/03/2023 20:57, Alex Williamson wrote: > >>>> On Sat, 4 Mar 2023 01:43:30 +0000 > >>>> Joao Martins <joao.m.mart...@oracle.com> wrote: > >>>> > >>>>> Hey, > >>>>> > >>>>> Presented herewith a series based on the basic VFIO migration protocol > >>>>> v2 > >>>>> implementation [1]. > >>>>> > >>>>> It is split from its parent series[5] to solely focus on device dirty > >>>>> page tracking. Device dirty page tracking allows the VFIO device to > >>>>> record its DMAs and report them back when needed. This is part of VFIO > >>>>> migration and is used during pre-copy phase of migration to track the > >>>>> RAM pages that the device has written to and mark those pages dirty, so > >>>>> they can later be re-sent to target. > >>>>> > >>>>> Device dirty page tracking uses the DMA logging uAPI to discover device > >>>>> capabilities, to start and stop tracking, and to get dirty page bitmap > >>>>> report. Extra details and uAPI definition can be found here [3]. > >>>>> > >>>>> Device dirty page tracking operates in VFIOContainer scope. I.e., When > >>>>> dirty tracking is started, stopped or dirty page report is queried, all > >>>>> devices within a VFIOContainer are iterated and for each of them device > >>>>> dirty page tracking is started, stopped or dirty page report is queried, > >>>>> respectively. > >>>>> > >>>>> Device dirty page tracking is used only if all devices within a > >>>>> VFIOContainer support it. Otherwise, VFIO IOMMU dirty page tracking is > >>>>> used, and if that is not supported as well, memory is perpetually marked > >>>>> dirty by QEMU. Note that since VFIO IOMMU dirty page tracking has no HW > >>>>> support, the last two usually have the same effect of perpetually > >>>>> marking all pages dirty. > >>>>> > >>>>> Normally, when asked to start dirty tracking, all the currently DMA > >>>>> mapped ranges are tracked by device dirty page tracking. If using a > >>>>> vIOMMU we block live migration. It's temporary and a separate series is > >>>>> going to add support for it. Thus this series focus on getting the > >>>>> ground work first. > >>>>> > >>>>> The series is organized as follows: > >>>>> > >>>>> - Patches 1-7: Fix bugs and do some preparatory work required prior to > >>>>> adding device dirty page tracking. > >>>>> - Patches 8-10: Implement device dirty page tracking. > >>>>> - Patch 11: Blocks live migration with vIOMMU. > >>>>> - Patches 12-13 enable device dirty page tracking and document it. > >>>>> > >>>>> Comments, improvements as usual appreciated. > >>>> > >>>> Still some CI failures: > >>>> > >>>> https://gitlab.com/alex.williamson/qemu/-/pipelines/796657474 > >>>> > >>>> The docker failures are normal, afaict the rest are not. Thanks, > >>>> > >>> > >>> Ugh, sorry > >>> > >>> The patch below scissors mark (and also attached as a file) fixes those > >>> build > >>> issues. I managed to reproduce on i386 target builds, and these changes > >>> fix my > >>> 32-bit build. > >>> > >>> I don't have a working Gitlab setup[*] though to trigger the CI to enable > >>> to > >>> wealth of targets it build-tests. If you could kindly test the patch > >>> attached in > >>> a new pipeline (applied on top of the branch you just build) below to > >>> understand > >>> if the CI gets happy. I will include these changes in the right patches > >>> (patch 8 > >>> and 10) for the v4 spin. > >> > >> Looks like this passes: > >> > >> https://gitlab.com/alex.williamson/qemu/-/pipelines/796750136 > >> > > Great, I've staged this fixes in patches 8&10! > > > > I have a sliver of hope that we might still make it by soft freeze > > (tomorrow?). > > If you think it can still make it, should the rest of the series is good, > > then I > > can follow up v4 today/tomorrow. Thoughts? > > I would say, wait and see if a v4 is needed first. These changes are > relatively easy to fold in. I think we have enough changes and fixes to post a v4 once you're happy with it. We should have tomorrow, the 7th to get final reviews and post a pull request. Thanks, Alex