Re: Ack to merge through DRM? WAS Re: [PATCH v2 1/5] mm: Add write-protect and clean utilities for address space ranges
On 10/2/19 3:18 PM, Kirill A. Shutemov wrote: On Wed, Oct 02, 2019 at 11:21:01AM +0200, Thomas Hellström (VMware) wrote: On 9/26/19 10:16 PM, Linus Torvalds wrote: On Thu, Sep 26, 2019 at 1:09 PM Thomas Hellström (VMware) wrote: That said, if people are OK with me modifying the assert in pud_trans_huge_lock() and make __walk_page_range non-static, it should probably be possible to make it work, yes. I don't think you need to modify that assert at all. That thing only exists when there's a "pud_entry" op in the walker, and then you absolutely need to have that mmap_lock. As far as I can tell, you fundamentally only ever work on a pte level in your address space walker already and actually have a WARN_ON() on the pud_huge thing, so no pud entry can possibly apply. So no, the assert in pud_trans_huge_lock() does not seem to be a reason not to just use the existing page table walkers. And once you get rid of the walking, what is left? Just the "iterate over the inode mappings" part. Which could just be done in mm/pagewalk.c, and then you don't even need to remove the static. So making it be just another walking in pagewalk.c would seem to be the simplest model. Call it "walk_page_mapping()". And talk extensively about how the locking differs a lot from the usual "walk_page_vma()" things. The then actual "apply" functions (what a horrid name) could be in the users. They shouldn't be mixed in with the walking functions anyway. They are callbacks, not walkers. Linus Linus, Kirill I've pushed a reworked version based on the pagewalk code here: https://cgit.freedesktop.org/~thomash/linux/log/?h=pagewalk (top three patched) with users included here: https://cgit.freedesktop.org/~thomash/linux/log/?h=coherent-rebased Do you think this could work? The reason that the "mm: Add write-protect and clean.." code is still in mm as a set of helpers, is of course that much of the needed functionality is not exported, presumably since we want to keep page table manipulation in mm. Could you post it to the mailing list? It's easier to review this way. Sure. /Thomas ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: Ack to merge through DRM? WAS Re: [PATCH v2 1/5] mm: Add write-protect and clean utilities for address space ranges
On Wed, Oct 02, 2019 at 11:21:01AM +0200, Thomas Hellström (VMware) wrote: > On 9/26/19 10:16 PM, Linus Torvalds wrote: > > On Thu, Sep 26, 2019 at 1:09 PM Thomas Hellström (VMware) > > wrote: > > > That said, if people are OK with me modifying the assert in > > > pud_trans_huge_lock() and make __walk_page_range non-static, it should > > > probably be possible to make it work, yes. > > I don't think you need to modify that assert at all. > > > > That thing only exists when there's a "pud_entry" op in the walker, > > and then you absolutely need to have that mmap_lock. > > > > As far as I can tell, you fundamentally only ever work on a pte level > > in your address space walker already and actually have a WARN_ON() on > > the pud_huge thing, so no pud entry can possibly apply. > > > > So no, the assert in pud_trans_huge_lock() does not seem to be a > > reason not to just use the existing page table walkers. > > > > And once you get rid of the walking, what is left? Just the "iterate > > over the inode mappings" part. Which could just be done in > > mm/pagewalk.c, and then you don't even need to remove the static. > > > > So making it be just another walking in pagewalk.c would seem to be > > the simplest model. > > > > Call it "walk_page_mapping()". And talk extensively about how the > > locking differs a lot from the usual "walk_page_vma()" things. > > > > The then actual "apply" functions (what a horrid name) could be in the > > users. They shouldn't be mixed in with the walking functions anyway. > > They are callbacks, not walkers. > > > > Linus > > Linus, Kirill > > I've pushed a reworked version based on the pagewalk code here: > > https://cgit.freedesktop.org/~thomash/linux/log/?h=pagewalk > > (top three patched) > > with users included here: > > https://cgit.freedesktop.org/~thomash/linux/log/?h=coherent-rebased > > Do you think this could work? The reason that the "mm: Add write-protect and > clean.." code is still in mm as a set of helpers, is of course that much of > the needed functionality is not exported, presumably since we want to keep > page table manipulation in mm. Could you post it to the mailing list? It's easier to review this way. -- Kirill A. Shutemov ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: Ack to merge through DRM? WAS Re: [PATCH v2 1/5] mm: Add write-protect and clean utilities for address space ranges
On 9/26/19 10:16 PM, Linus Torvalds wrote: On Thu, Sep 26, 2019 at 1:09 PM Thomas Hellström (VMware) wrote: That said, if people are OK with me modifying the assert in pud_trans_huge_lock() and make __walk_page_range non-static, it should probably be possible to make it work, yes. I don't think you need to modify that assert at all. That thing only exists when there's a "pud_entry" op in the walker, and then you absolutely need to have that mmap_lock. As far as I can tell, you fundamentally only ever work on a pte level in your address space walker already and actually have a WARN_ON() on the pud_huge thing, so no pud entry can possibly apply. So no, the assert in pud_trans_huge_lock() does not seem to be a reason not to just use the existing page table walkers. And once you get rid of the walking, what is left? Just the "iterate over the inode mappings" part. Which could just be done in mm/pagewalk.c, and then you don't even need to remove the static. So making it be just another walking in pagewalk.c would seem to be the simplest model. Call it "walk_page_mapping()". And talk extensively about how the locking differs a lot from the usual "walk_page_vma()" things. The then actual "apply" functions (what a horrid name) could be in the users. They shouldn't be mixed in with the walking functions anyway. They are callbacks, not walkers. Linus Linus, Kirill I've pushed a reworked version based on the pagewalk code here: https://cgit.freedesktop.org/~thomash/linux/log/?h=pagewalk (top three patched) with users included here: https://cgit.freedesktop.org/~thomash/linux/log/?h=coherent-rebased Do you think this could work? The reason that the "mm: Add write-protect and clean.." code is still in mm as a set of helpers, is of course that much of the needed functionality is not exported, presumably since we want to keep page table manipulation in mm. Thanks, Thomas ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: Ack to merge through DRM? WAS Re: [PATCH v2 1/5] mm: Add write-protect and clean utilities for address space ranges
On 9/30/19 7:12 PM, Linus Torvalds wrote: On Mon, Sep 30, 2019 at 6:04 AM Kirill A. Shutemov wrote: Have you seen page_vma_mapped_walk()? I made it specifically for rmap code to cover cases when a THP is mapped with PTEs. To me it's not a big stretch to make it cover multiple pages too. I agree that is closer, but it does make for calling that big complex function for every iteration step. Of course, you are right that the callback approach is problematic too, now that we have retpoline issues, making those very expensive. But at least that hopefully gets fixed some day and gets to be a rare problem. Matter ot taste, I guess. Linus Matthew Wilcox suggested something similar before the pagewalk.c rewrite: https://lore.kernel.org/lkml/20190806190937.gd30...@bombadil.infradead.org/ Still, In this case I'd opt for using the pagewalk code: In the dirty helpers we don't ever use a struct page, but only deal with PTE entries, same as the pagewalk code does, but not page_vma_mapped_walk(). The underlying memory may well be IO memory. /Thomas ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: Ack to merge through DRM? WAS Re: [PATCH v2 1/5] mm: Add write-protect and clean utilities for address space ranges
On Mon, Sep 30, 2019 at 6:04 AM Kirill A. Shutemov wrote: > > Have you seen page_vma_mapped_walk()? I made it specifically for rmap code > to cover cases when a THP is mapped with PTEs. To me it's not a big > stretch to make it cover multiple pages too. I agree that is closer, but it does make for calling that big complex function for every iteration step. Of course, you are right that the callback approach is problematic too, now that we have retpoline issues, making those very expensive. But at least that hopefully gets fixed some day and gets to be a rare problem. Matter ot taste, I guess. Linus ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: Ack to merge through DRM? WAS Re: [PATCH v2 1/5] mm: Add write-protect and clean utilities for address space ranges
On Fri, Sep 27, 2019 at 09:39:48AM -0700, Linus Torvalds wrote: > On Fri, Sep 27, 2019 at 5:17 AM Kirill A. Shutemov > wrote: > > > > > Call it "walk_page_mapping()". And talk extensively about how the > > > locking differs a lot from the usual "walk_page_vma()" things. > > > > Walking mappings of a page is what rmap does. This code thas to be > > integrated there. > > Well, that's very questionable. > > The rmap code mainly does the "page -> virtual" mapping. One page at a time. > > The page walker code does the "virtual -> pte" mapping. Always a whole > range at a time. Have you seen page_vma_mapped_walk()? I made it specifically for rmap code to cover cases when a THP is mapped with PTEs. To me it's not a big stretch to make it cover multiple pages too. > So I think conceptually, mm/memory.c and unmap_mapping_range() is > closest but I don't think it's practical to share code. > > And between mm/pagewalk.c and mm/rmap.c, I think the page walking has > way more of actual practical code sharing, and is also conceptually > closer because most of the code is about walking a range, not looking > up the mapping of one page. I guess it's matter of personal preferences, but page table walkers based on callback always felt wrong to me. -- Kirill A. Shutemov ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: Ack to merge through DRM? WAS Re: [PATCH v2 1/5] mm: Add write-protect and clean utilities for address space ranges
On Fri, Sep 27, 2019 at 5:17 AM Kirill A. Shutemov wrote: > > > Call it "walk_page_mapping()". And talk extensively about how the > > locking differs a lot from the usual "walk_page_vma()" things. > > Walking mappings of a page is what rmap does. This code thas to be > integrated there. Well, that's very questionable. The rmap code mainly does the "page -> virtual" mapping. One page at a time. The page walker code does the "virtual -> pte" mapping. Always a whole range at a time. The new code wants a combination of both. It very much is about walking ranges - as in mm/pagewalk.c. It's just that it walks potentially multiple ranges, based on where the address space is mapped. I think it has way more commonalities with the page walking code than it has with the rmap code. But yes, there is some of that "look up mappings based on address space" in there too, but it's the least part of it And as Thomas pointed out, it also has commonalities with unmap_mapping_pages() in mm/memory.c. In many ways that part is the closest. I'd say that from a code sharing standpoint, mm/rmap.c is absolutely the wrong place. It's the furthest away from what Thomas wants to do. The mm/pagewalk.c code has the most actual code that could be shared, and the addition would be smallest there. And conceptually the closest analogue in terms of what it _does_ is unmap_mapping_range() in mm/memory.c, but I see no room for sharing actual code there unless we completely change how we do zap_page_range() and add a lot of configurability there (which we don't want, because page table teardown at exit is really a pretty critical operation - I commonly see copy_page_range() and zap_page_range() on profiles if you have things like script-heavyu traditional UNIX loads). So I think conceptually, mm/memory.c and unmap_mapping_range() is closest but I don't think it's practical to share code. And between mm/pagewalk.c and mm/rmap.c, I think the page walking has way more of actual practical code sharing, and is also conceptually closer because most of the code is about walking a range, not looking up the mapping of one page. Linus ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: Ack to merge through DRM? WAS Re: [PATCH v2 1/5] mm: Add write-protect and clean utilities for address space ranges
On Thu, Sep 26, 2019 at 03:20:42PM -0700, Linus Torvalds wrote: > On Thu, Sep 26, 2019 at 1:55 PM Thomas Hellström (VMware) > wrote: > > > > Well, we're working on supporting huge puds and pmds in the graphics > > VMAs, although in the write-notify cases we're looking at here, we would > > probably want to split them down to PTE level. > > Well, that's what the existing walker code does if you don't have that > "pud_entry()" callback. > > That said, I assume you would *not* want to do that if the huge > pud/pmd is already clean and read-only, but just continue. > > So you may want to have a special pud_entry() that handles that case. > Eventually. Maybe. Although honestly, if you're doing dirty tracking, > I doubt it makes much sense to use largepages. > > > Looking at zap_pud_range() which when called from unmap_mapping_pages() > > uses identical locking (no mmap_sem), it seems we should be able to get > > away with i_mmap_lock(), making sure the whole page table doesn't > > disappear under us. So it's not clear to me why the mmap_sem is strictly > > needed here. Better to sort those restrictions out now rather than when > > huge entries start appearing. > > zap_pud_range()actually does have that > >VM_BUG_ON_VMA(!rwsem_is_locked(>mm->mmap_sem), vma); The VM_BUG is a blind copy from PMD layer and it's bogus. i_mmap_lock() works fine for file mappings. The PMD was intended for THP case at the time when there were only anon-THP. The check was relaxed and later dropped for file-THP on PMD level. It has to be dropped on PUD too. We don't have anon-THP on PUD level at all, only DAX played with them. -- Kirill A. Shutemov ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: Ack to merge through DRM? WAS Re: [PATCH v2 1/5] mm: Add write-protect and clean utilities for address space ranges
On Thu, Sep 26, 2019 at 01:16:55PM -0700, Linus Torvalds wrote: > On Thu, Sep 26, 2019 at 1:09 PM Thomas Hellström (VMware) > wrote: > > > > That said, if people are OK with me modifying the assert in > > pud_trans_huge_lock() and make __walk_page_range non-static, it should > > probably be possible to make it work, yes. > > I don't think you need to modify that assert at all. > > That thing only exists when there's a "pud_entry" op in the walker, > and then you absolutely need to have that mmap_lock. > > As far as I can tell, you fundamentally only ever work on a pte level > in your address space walker already and actually have a WARN_ON() on > the pud_huge thing, so no pud entry can possibly apply. > > So no, the assert in pud_trans_huge_lock() does not seem to be a > reason not to just use the existing page table walkers. > > And once you get rid of the walking, what is left? Just the "iterate > over the inode mappings" part. Which could just be done in > mm/pagewalk.c, and then you don't even need to remove the static. > > So making it be just another walking in pagewalk.c would seem to be > the simplest model. > > Call it "walk_page_mapping()". And talk extensively about how the > locking differs a lot from the usual "walk_page_vma()" things. Walking mappings of a page is what rmap does. This code thas to be integrated there. -- Kirill A. Shutemov ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: Ack to merge through DRM? WAS Re: [PATCH v2 1/5] mm: Add write-protect and clean utilities for address space ranges
On 9/27/19 7:55 AM, Thomas Hellström (VMware) wrote: On 9/27/19 12:20 AM, Linus Torvalds wrote: On Thu, Sep 26, 2019 at 1:55 PM Thomas Hellström (VMware) wrote: Well, we're working on supporting huge puds and pmds in the graphics VMAs, although in the write-notify cases we're looking at here, we would probably want to split them down to PTE level. Well, that's what the existing walker code does if you don't have that "pud_entry()" callback. That said, I assume you would *not* want to do that if the huge pud/pmd is already clean and read-only, but just continue. So you may want to have a special pud_entry() that handles that case. Eventually. Maybe. Although honestly, if you're doing dirty tracking, I doubt it makes much sense to use largepages. The approach we're looking at in this case is to keep huge entries write-protected and split them in the wp_huge_xxx() code's fallback path with the mmap_sem held. This means that there will actually be huge entries in the page-walking code soon, but as you say, only entries that we want to ignore and not split. So we'd also need a way to avoid the pagewalk splitting for the situation when someone faults a huge entry in just before the call to split_huge_xxx. Looking at zap_pud_range() which when called from unmap_mapping_pages() uses identical locking (no mmap_sem), it seems we should be able to get away with i_mmap_lock(), making sure the whole page table doesn't disappear under us. So it's not clear to me why the mmap_sem is strictly needed here. Better to sort those restrictions out now rather than when huge entries start appearing. zap_pud_range()actually does have that VM_BUG_ON_VMA(!rwsem_is_locked(>mm->mmap_sem), vma); exactly for the case where it might have to split the pud entry. Yes. My take on this is that locking the PUD ptl can be done either with the mmap_sem or the i_mmap_lock if present and that we should update the asserts in xxx_trans_huge_lock to reflect that. But when actually splitting transhuge pages you don't want to race with khugepaged, so you need the mmap_sem. For the graphics VMAs (MIXEDMAP), khugepaged never touches them. Yet. It's why they've never gotten translated to use the generic walker code. OK. Yes there are a number of various specialized pagewalks all over the mm code. But another thing that worries me is that the page-table modifications that happen in the callback use functionality that is not guaranteed to be exported, and that mm people don't want them to be exported because you don't want the drivers to go hacking around in page tables, which means that the two callbacks used here would need to be a set of core helpers anyway. So I figure what I would end up with would actually be an extern __walk_page_range anyway, and slightly modified asserts. Do you think that could be acceptible? Actually, I'll give your original suggestion a try and see what I come up with. Thanks, Thomas
Re: Ack to merge through DRM? WAS Re: [PATCH v2 1/5] mm: Add write-protect and clean utilities for address space ranges
On 9/27/19 12:20 AM, Linus Torvalds wrote: On Thu, Sep 26, 2019 at 1:55 PM Thomas Hellström (VMware) wrote: Well, we're working on supporting huge puds and pmds in the graphics VMAs, although in the write-notify cases we're looking at here, we would probably want to split them down to PTE level. Well, that's what the existing walker code does if you don't have that "pud_entry()" callback. That said, I assume you would *not* want to do that if the huge pud/pmd is already clean and read-only, but just continue. So you may want to have a special pud_entry() that handles that case. Eventually. Maybe. Although honestly, if you're doing dirty tracking, I doubt it makes much sense to use largepages. The approach we're looking at in this case is to keep huge entries write-protected and split them in the wp_huge_xxx() code's fallback path with the mmap_sem held. This means that there will actually be huge entries in the page-walking code soon, but as you say, only entries that we want to ignore and not split. So we'd also need a way to avoid the pagewalk splitting for the situation when someone faults a huge entry in just before the call to split_huge_xxx. Looking at zap_pud_range() which when called from unmap_mapping_pages() uses identical locking (no mmap_sem), it seems we should be able to get away with i_mmap_lock(), making sure the whole page table doesn't disappear under us. So it's not clear to me why the mmap_sem is strictly needed here. Better to sort those restrictions out now rather than when huge entries start appearing. zap_pud_range()actually does have that VM_BUG_ON_VMA(!rwsem_is_locked(>mm->mmap_sem), vma); exactly for the case where it might have to split the pud entry. Yes. My take on this is that locking the PUD ptl can be done either with the mmap_sem or the i_mmap_lock if present and that we should update the asserts in xxx_trans_huge_lock to reflect that. But when actually splitting transhuge pages you don't want to race with khugepaged, so you need the mmap_sem. For the graphics VMAs (MIXEDMAP), khugepaged never touches them. Yet. It's why they've never gotten translated to use the generic walker code. OK. Yes there are a number of various specialized pagewalks all over the mm code. But another thing that worries me is that the page-table modifications that happen in the callback use functionality that is not guaranteed to be exported, and that mm people don't want them to be exported because you don't want the drivers to go hacking around in page tables, which means that the two callbacks used here would need to be a set of core helpers anyway. So I figure what I would end up with would actually be an extern __walk_page_range anyway, and slightly modified asserts. Do you think that could be acceptible? Thanks, Thomas Linus
Re: Ack to merge through DRM? WAS Re: [PATCH v2 1/5] mm: Add write-protect and clean utilities for address space ranges
On Thu, Sep 26, 2019 at 1:55 PM Thomas Hellström (VMware) wrote: > > Well, we're working on supporting huge puds and pmds in the graphics > VMAs, although in the write-notify cases we're looking at here, we would > probably want to split them down to PTE level. Well, that's what the existing walker code does if you don't have that "pud_entry()" callback. That said, I assume you would *not* want to do that if the huge pud/pmd is already clean and read-only, but just continue. So you may want to have a special pud_entry() that handles that case. Eventually. Maybe. Although honestly, if you're doing dirty tracking, I doubt it makes much sense to use largepages. > Looking at zap_pud_range() which when called from unmap_mapping_pages() > uses identical locking (no mmap_sem), it seems we should be able to get > away with i_mmap_lock(), making sure the whole page table doesn't > disappear under us. So it's not clear to me why the mmap_sem is strictly > needed here. Better to sort those restrictions out now rather than when > huge entries start appearing. zap_pud_range()actually does have that VM_BUG_ON_VMA(!rwsem_is_locked(>mm->mmap_sem), vma); exactly for the case where it might have to split the pud entry. Zapping the whole thing it does do without the assert. I'm not going to swear the mmap_sem is absolutely required, since a shared vma should be stable due to the i_mmap_lock, but splitting the hugepage really is a fairly big deal. It can't happen if you zap the *whole* mapping, but it can happen if you have a start/end range. Like you do. Also, in general it's probably not a great idea to look at zap_page_range() (and copy_page_range()) for ideas. They are kind of special, since they tend to be used for fundamental whole-address-space operations (ie fork/exit) and so as a result they get to do special things that a normal page walker generally shouldn't do. It's why they've never gotten translated to use the generic walker code. Linus
Re: Ack to merge through DRM? WAS Re: [PATCH v2 1/5] mm: Add write-protect and clean utilities for address space ranges
On 9/26/19 10:16 PM, Linus Torvalds wrote: On Thu, Sep 26, 2019 at 1:09 PM Thomas Hellström (VMware) wrote: That said, if people are OK with me modifying the assert in pud_trans_huge_lock() and make __walk_page_range non-static, it should probably be possible to make it work, yes. I don't think you need to modify that assert at all. That thing only exists when there's a "pud_entry" op in the walker, and then you absolutely need to have that mmap_lock. As far as I can tell, you fundamentally only ever work on a pte level in your address space walker already and actually have a WARN_ON() on the pud_huge thing, so no pud entry can possibly apply. Well, we're working on supporting huge puds and pmds in the graphics VMAs, although in the write-notify cases we're looking at here, we would probably want to split them down to PTE level. But if we would want to extend that or if we would want to make this interface general, we'd probably want to support also a pud_entry callback. Looking at zap_pud_range() which when called from unmap_mapping_pages() uses identical locking (no mmap_sem), it seems we should be able to get away with i_mmap_lock(), making sure the whole page table doesn't disappear under us. So it's not clear to me why the mmap_sem is strictly needed here. Better to sort those restrictions out now rather than when huge entries start appearing. So no, the assert in pud_trans_huge_lock() does not seem to be a reason not to just use the existing page table walkers. And once you get rid of the walking, what is left? Just the "iterate over the inode mappings" part. Which could just be done in mm/pagewalk.c, and then you don't even need to remove the static. So making it be just another walking in pagewalk.c would seem to be the simplest model. Call it "walk_page_mapping()". And talk extensively about how the locking differs a lot from the usual "walk_page_vma()" things. Sure. Can do that. Thanks, Thomas ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: Ack to merge through DRM? WAS Re: [PATCH v2 1/5] mm: Add write-protect and clean utilities for address space ranges
On Thu, Sep 26, 2019 at 1:09 PM Thomas Hellström (VMware) wrote: > > That said, if people are OK with me modifying the assert in > pud_trans_huge_lock() and make __walk_page_range non-static, it should > probably be possible to make it work, yes. I don't think you need to modify that assert at all. That thing only exists when there's a "pud_entry" op in the walker, and then you absolutely need to have that mmap_lock. As far as I can tell, you fundamentally only ever work on a pte level in your address space walker already and actually have a WARN_ON() on the pud_huge thing, so no pud entry can possibly apply. So no, the assert in pud_trans_huge_lock() does not seem to be a reason not to just use the existing page table walkers. And once you get rid of the walking, what is left? Just the "iterate over the inode mappings" part. Which could just be done in mm/pagewalk.c, and then you don't even need to remove the static. So making it be just another walking in pagewalk.c would seem to be the simplest model. Call it "walk_page_mapping()". And talk extensively about how the locking differs a lot from the usual "walk_page_vma()" things. The then actual "apply" functions (what a horrid name) could be in the users. They shouldn't be mixed in with the walking functions anyway. They are callbacks, not walkers. Linus
Re: Ack to merge through DRM? WAS Re: [PATCH v2 1/5] mm: Add write-protect and clean utilities for address space ranges
Hi, On 9/26/19 9:19 PM, Linus Torvalds wrote: On Thu, Sep 26, 2019 at 5:03 AM Thomas Hellström (VMware) wrote: I wonder if I can get an ack from an mm maintainer to merge this through DRM along with the vmwgfx patches? Andrew? Matthew? It would have helped to actually point to the patch itself, instead of just quoting the commit message. Looks like this: https://lore.kernel.org/lkml/20190926115548.44000-2-thomas...@shipmail.org/ but why is the code in question not just using the regular page walkers. The commit log shows no explanation of what's so special about this? Is the only reason the locking magic? Because if that's the reason, then afaik we already have a function for that: it's __walk_page_range(). Yes, it's static right now, but that's imho not a reason to duplicate all the walking (badly). Is there some other magic reason that isn't documented? Linus There is a discussion around this subject in https://lore.kernel.org/lkml/20190926115548.44000-1-thomas...@shipmail.org/ The main point is that there is an assert in pud_trans_huge_lock() that the mmap_sem is held, and we don't have it. Presumably we should be able to get away with the i_mmap_lock, but in addition I would have had to include the walk_as_pte_range() as the walk::pmd_entry anyway, so the amount of duplicated page walk code isn't that big. That said, if people are OK with me modifying the assert in pud_trans_huge_lock() and make __walk_page_range non-static, it should probably be possible to make it work, yes. Thanks, Thomas
Re: Ack to merge through DRM? WAS Re: [PATCH v2 1/5] mm: Add write-protect and clean utilities for address space ranges
On Thu, Sep 26, 2019 at 5:03 AM Thomas Hellström (VMware) wrote: > > I wonder if I can get an ack from an mm maintainer to merge this through > DRM along with the vmwgfx patches? Andrew? Matthew? It would have helped to actually point to the patch itself, instead of just quoting the commit message. Looks like this: https://lore.kernel.org/lkml/20190926115548.44000-2-thomas...@shipmail.org/ but why is the code in question not just using the regular page walkers. The commit log shows no explanation of what's so special about this? Is the only reason the locking magic? Because if that's the reason, then afaik we already have a function for that: it's __walk_page_range(). Yes, it's static right now, but that's imho not a reason to duplicate all the walking (badly). Is there some other magic reason that isn't documented? Linus
