Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Tue, Jul 25, 2017 at 03:45:14PM -0700, Evgeny Baskakov wrote: > On 7/20/17 6:33 PM, Jerome Glisse wrote: > > > So i pushed an updated hmm-next branch it should have all fixes so far, > > including > > something that should fix this issue. I still want to go over all emails > > again > > to

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Tue, Jul 25, 2017 at 03:45:14PM -0700, Evgeny Baskakov wrote: > On 7/20/17 6:33 PM, Jerome Glisse wrote: > > > So i pushed an updated hmm-next branch it should have all fixes so far, > > including > > something that should fix this issue. I still want to go over all emails > > again > > to

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/20/17 6:33 PM, Jerome Glisse wrote: So i pushed an updated hmm-next branch it should have all fixes so far, including something that should fix this issue. I still want to go over all emails again to make sure i am not forgetting anything. Cheers, Jérôme Hi Jerome, Thanks for updating

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/20/17 6:33 PM, Jerome Glisse wrote: So i pushed an updated hmm-next branch it should have all fixes so far, including something that should fix this issue. I still want to go over all emails again to make sure i am not forgetting anything. Cheers, Jérôme Hi Jerome, Thanks for updating

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/20/17 6:33 PM, Jerome Glisse wrote: So i pushed an updated hmm-next branch it should have all fixes so far, including something that should fix this issue. I still want to go over all emails again to make sure i am not forgetting anything. Cheers, Jérôme Hi Jerome, The issues I

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/20/17 6:33 PM, Jerome Glisse wrote: So i pushed an updated hmm-next branch it should have all fixes so far, including something that should fix this issue. I still want to go over all emails again to make sure i am not forgetting anything. Cheers, Jérôme Hi Jerome, The issues I

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Thu, Jul 20, 2017 at 06:00:08PM -0700, Evgeny Baskakov wrote: > On 7/14/17 5:55 PM, Jerome Glisse wrote: > Hi Jerome, > > I think I just found a couple of new issues, now related to fork/execve. > > 1) With a fork() followed by execve(), the child process makes a copy of the > parent

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Thu, Jul 20, 2017 at 06:00:08PM -0700, Evgeny Baskakov wrote: > On 7/14/17 5:55 PM, Jerome Glisse wrote: > Hi Jerome, > > I think I just found a couple of new issues, now related to fork/execve. > > 1) With a fork() followed by execve(), the child process makes a copy of the > parent

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/14/17 5:55 PM, Jerome Glisse wrote: ... Cheers, Jérôme Hi Jerome, I think I just found a couple of new issues, now related to fork/execve. 1) With a fork() followed by execve(), the child process makes a copy of the parent mm_struct object, including the "hmm" pointer. Later on, an

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/14/17 5:55 PM, Jerome Glisse wrote: ... Cheers, Jérôme Hi Jerome, I think I just found a couple of new issues, now related to fork/execve. 1) With a fork() followed by execve(), the child process makes a copy of the parent mm_struct object, including the "hmm" pointer. Later on, an

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/10/17 5:54 PM, Jerome Glisse wrote: On Mon, Jul 10, 2017 at 05:17:23PM -0700, Evgeny Baskakov wrote: On 7/10/17 4:43 PM, Jerome Glisse wrote: On Mon, Jul 10, 2017 at 03:59:37PM -0700, Evgeny Baskakov wrote: ... Horrible stupid bug in the code, most likely from cut and paste. Attached

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/10/17 5:54 PM, Jerome Glisse wrote: On Mon, Jul 10, 2017 at 05:17:23PM -0700, Evgeny Baskakov wrote: On 7/10/17 4:43 PM, Jerome Glisse wrote: On Mon, Jul 10, 2017 at 03:59:37PM -0700, Evgeny Baskakov wrote: ... Horrible stupid bug in the code, most likely from cut and paste. Attached

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/14/17 5:55 PM, Jerome Glisse wrote: So pushed an updated hmm-next branch this should fix all issues you had. Thought i am not sure about the test in this mail, all i see is that it continously spit error messages but it does not hang (i let it run 20min or so). Dunno if that is what

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/14/17 5:55 PM, Jerome Glisse wrote: So pushed an updated hmm-next branch this should fix all issues you had. Thought i am not sure about the test in this mail, all i see is that it continously spit error messages but it does not hang (i let it run 20min or so). Dunno if that is what

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Fri, Jul 14, 2017 at 12:43:51PM -0700, Evgeny Baskakov wrote: > On 7/13/17 1:16 PM, Jerome Glisse wrote: > Hi Jerome, > > I have hit another kind of hang. Briefly, if a not yet allocated page faults > on CPU during migration to device memory, any subsequent migration will fail > for such page.

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Fri, Jul 14, 2017 at 12:43:51PM -0700, Evgeny Baskakov wrote: > On 7/13/17 1:16 PM, Jerome Glisse wrote: > Hi Jerome, > > I have hit another kind of hang. Briefly, if a not yet allocated page faults > on CPU during migration to device memory, any subsequent migration will fail > for such page.

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/13/17 1:16 PM, Jerome Glisse wrote: ... Hi Jerome, I have hit another kind of hang. Briefly, if a not yet allocated page faults on CPU during migration to device memory, any subsequent migration will fail for such page. Such a situation can trigger if a CPU page fault happens just

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/13/17 1:16 PM, Jerome Glisse wrote: ... Hi Jerome, I have hit another kind of hang. Briefly, if a not yet allocated page faults on CPU during migration to device memory, any subsequent migration will fail for such page. Such a situation can trigger if a CPU page fault happens just

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Tue, Jul 11, 2017 at 12:35:03PM -0700, Evgeny Baskakov wrote: > On 7/11/17 11:49 AM, Jerome Glisse wrote: > > > > > What are the symptoms ? The program just stop making any progress and you > > trigger a sysrequest to dump current states of each threads ? In this > > log i don't see

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Tue, Jul 11, 2017 at 12:35:03PM -0700, Evgeny Baskakov wrote: > On 7/11/17 11:49 AM, Jerome Glisse wrote: > > > > > What are the symptoms ? The program just stop making any progress and you > > trigger a sysrequest to dump current states of each threads ? In this > > log i don't see

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/11/17 11:49 AM, Jerome Glisse wrote: What are the symptoms ? The program just stop making any progress and you trigger a sysrequest to dump current states of each threads ? In this log i don't see migration_entry_wait() anymore but it seems to be waiting on page lock so there might be 2

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/11/17 11:49 AM, Jerome Glisse wrote: What are the symptoms ? The program just stop making any progress and you trigger a sysrequest to dump current states of each threads ? In this log i don't see migration_entry_wait() anymore but it seems to be waiting on page lock so there might be 2

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Tue, Jul 11, 2017 at 11:42:20AM -0700, Evgeny Baskakov wrote: > On 7/11/17 11:29 AM, Jerome Glisse wrote: > > Can you test if attached patch helps ? I am having trouble reproducing > > this > > from inside a vm. > > > > My theory is that 2 concurrent CPU page fault happens. First one manage to

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Tue, Jul 11, 2017 at 11:42:20AM -0700, Evgeny Baskakov wrote: > On 7/11/17 11:29 AM, Jerome Glisse wrote: > > Can you test if attached patch helps ? I am having trouble reproducing > > this > > from inside a vm. > > > > My theory is that 2 concurrent CPU page fault happens. First one manage to

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/11/17 11:29 AM, Jerome Glisse wrote: Can you test if attached patch helps ? I am having trouble reproducing this from inside a vm. My theory is that 2 concurrent CPU page fault happens. First one manage to start the migration back to system memory but second one see the migration special

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/11/17 11:29 AM, Jerome Glisse wrote: Can you test if attached patch helps ? I am having trouble reproducing this from inside a vm. My theory is that 2 concurrent CPU page fault happens. First one manage to start the migration back to system memory but second one see the migration special

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Mon, Jul 10, 2017 at 04:44:38PM -0700, Evgeny Baskakov wrote: > On 6/30/17 5:57 PM, Jerome Glisse wrote: > > ... > > Hi Jerome, > > I am working on a sporadic data corruption seen in highly contented use > cases. So far, I've been able to re-create a sporadic hang that happens when >

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Mon, Jul 10, 2017 at 04:44:38PM -0700, Evgeny Baskakov wrote: > On 6/30/17 5:57 PM, Jerome Glisse wrote: > > ... > > Hi Jerome, > > I am working on a sporadic data corruption seen in highly contented use > cases. So far, I've been able to re-create a sporadic hang that happens when >

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Mon, Jul 10, 2017 at 05:17:23PM -0700, Evgeny Baskakov wrote: > On 7/10/17 4:43 PM, Jerome Glisse wrote: > > > On Mon, Jul 10, 2017 at 03:59:37PM -0700, Evgeny Baskakov wrote: > > ... > > Horrible stupid bug in the code, most likely from cut and paste. Attached > > patch should fix it. I don't

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Mon, Jul 10, 2017 at 05:17:23PM -0700, Evgeny Baskakov wrote: > On 7/10/17 4:43 PM, Jerome Glisse wrote: > > > On Mon, Jul 10, 2017 at 03:59:37PM -0700, Evgeny Baskakov wrote: > > ... > > Horrible stupid bug in the code, most likely from cut and paste. Attached > > patch should fix it. I don't

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/10/17 4:43 PM, Jerome Glisse wrote: On Mon, Jul 10, 2017 at 03:59:37PM -0700, Evgeny Baskakov wrote: ... Horrible stupid bug in the code, most likely from cut and paste. Attached patch should fix it. I don't know how long it took for you to trigger it. Jérôme Thanks, this indeed fixes the

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 7/10/17 4:43 PM, Jerome Glisse wrote: On Mon, Jul 10, 2017 at 03:59:37PM -0700, Evgeny Baskakov wrote: ... Horrible stupid bug in the code, most likely from cut and paste. Attached patch should fix it. I don't know how long it took for you to trigger it. Jérôme Thanks, this indeed fixes the

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 6/30/17 5:57 PM, Jerome Glisse wrote: ... Hi Jerome, I am working on a sporadic data corruption seen in highly contented use cases. So far, I've been able to re-create a sporadic hang that happens when multiple threads compete to migrate the same page to and from device memory. The

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 6/30/17 5:57 PM, Jerome Glisse wrote: ... Hi Jerome, I am working on a sporadic data corruption seen in highly contented use cases. So far, I've been able to re-create a sporadic hang that happens when multiple threads compete to migrate the same page to and from device memory. The

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Mon, Jul 10, 2017 at 03:59:37PM -0700, Evgeny Baskakov wrote: > On 6/30/17 5:57 PM, Jerome Glisse wrote: > ... > > Hi Jerome, > > I am seeing a strange crash in our code that uses the hmm_device_new() > helper. After the driver is repeatedly loaded/unloaded, hmm_device_new() > suddenly

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Mon, Jul 10, 2017 at 03:59:37PM -0700, Evgeny Baskakov wrote: > On 6/30/17 5:57 PM, Jerome Glisse wrote: > ... > > Hi Jerome, > > I am seeing a strange crash in our code that uses the hmm_device_new() > helper. After the driver is repeatedly loaded/unloaded, hmm_device_new() > suddenly

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 6/30/17 5:57 PM, Jerome Glisse wrote: ... Hi Jerome, I am seeing a strange crash in our code that uses the hmm_device_new() helper. After the driver is repeatedly loaded/unloaded, hmm_device_new() suddenly returns NULL. I have reproduced this with the dummy driver from the hmm-next

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 6/30/17 5:57 PM, Jerome Glisse wrote: ... Hi Jerome, I am seeing a strange crash in our code that uses the hmm_device_new() helper. After the driver is repeatedly loaded/unloaded, hmm_device_new() suddenly returns NULL. I have reproduced this with the dummy driver from the hmm-next

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 6/30/17 5:57 PM, Jerome Glisse wrote: On Fri, Jun 30, 2017 at 04:19:25PM -0700, Evgeny Baskakov wrote: Hi Jerome, It seems that the kernel can pass 0 in src_pfns for pages that it cannot migrate (i.e. the kernel knows that they cannot migrate prior to calling alloc_and_copy). So, a zero

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 6/30/17 5:57 PM, Jerome Glisse wrote: On Fri, Jun 30, 2017 at 04:19:25PM -0700, Evgeny Baskakov wrote: Hi Jerome, It seems that the kernel can pass 0 in src_pfns for pages that it cannot migrate (i.e. the kernel knows that they cannot migrate prior to calling alloc_and_copy). So, a zero

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Fri, Jun 30, 2017 at 04:19:25PM -0700, Evgeny Baskakov wrote: > Hi Jerome, > > It seems that the kernel can pass 0 in src_pfns for pages that it cannot > migrate (i.e. the kernel knows that they cannot migrate prior to calling > alloc_and_copy). > > So, a zero in src_pfns can mean either "the

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Fri, Jun 30, 2017 at 04:19:25PM -0700, Evgeny Baskakov wrote: > Hi Jerome, > > It seems that the kernel can pass 0 in src_pfns for pages that it cannot > migrate (i.e. the kernel knows that they cannot migrate prior to calling > alloc_and_copy). > > So, a zero in src_pfns can mean either "the

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 6/26/17 5:07 PM, Evgeny Baskakov wrote: > Hi Jerome, > > The documentation shown above doesn't tell what the alloc_and_copy callback should do for source pages that have not been allocated yet. Instead, it unconditionally suggests checking if the MIGRATE_PFN_VALID and MIGRATE_PFN_MIGRATE

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On 6/26/17 5:07 PM, Evgeny Baskakov wrote: > Hi Jerome, > > The documentation shown above doesn't tell what the alloc_and_copy callback should do for source pages that have not been allocated yet. Instead, it unconditionally suggests checking if the MIGRATE_PFN_VALID and MIGRATE_PFN_MIGRATE

RE: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Monday, May 22, 2017 9:52 AM, Jérôme Glisse wrote: [...] + * The alloc_and_copy() callback happens once all source pages have +been locked, + * unmapped and checked (checked whether pinned or not). All pages that +can be + * migrated will have an entry in the src array set with the pfn value

RE: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Monday, May 22, 2017 9:52 AM, Jérôme Glisse wrote: [...] + * The alloc_and_copy() callback happens once all source pages have +been locked, + * unmapped and checked (checked whether pinned or not). All pages that +can be + * migrated will have an entry in the src array set with the pfn value

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

> On Fri, Jun 2, 2017 at 8:35 AM, Jerome Glisse wrote: > > On Wed, May 31, 2017 at 01:59:54PM +1000, Balbir Singh wrote: > >> On Wed, 24 May 2017 13:20:21 -0400 > >> Jérôme Glisse wrote: > >> > >> > This patch add a new memory migration helpers, which

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

> On Fri, Jun 2, 2017 at 8:35 AM, Jerome Glisse wrote: > > On Wed, May 31, 2017 at 01:59:54PM +1000, Balbir Singh wrote: > >> On Wed, 24 May 2017 13:20:21 -0400 > >> Jérôme Glisse wrote: > >> > >> > This patch add a new memory migration helpers, which migrate memory > >> > backing a range of

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Fri, Jun 2, 2017 at 8:35 AM, Jerome Glisse wrote: > On Wed, May 31, 2017 at 01:59:54PM +1000, Balbir Singh wrote: >> On Wed, 24 May 2017 13:20:21 -0400 >> Jérôme Glisse wrote: >> >> > This patch add a new memory migration helpers, which migrate memory

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Fri, Jun 2, 2017 at 8:35 AM, Jerome Glisse wrote: > On Wed, May 31, 2017 at 01:59:54PM +1000, Balbir Singh wrote: >> On Wed, 24 May 2017 13:20:21 -0400 >> Jérôme Glisse wrote: >> >> > This patch add a new memory migration helpers, which migrate memory >> > backing a range of virtual address

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Wed, May 31, 2017 at 01:59:54PM +1000, Balbir Singh wrote: > On Wed, 24 May 2017 13:20:21 -0400 > Jérôme Glisse wrote: > > > This patch add a new memory migration helpers, which migrate memory > > backing a range of virtual address of a process to different memory > >

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Wed, May 31, 2017 at 01:59:54PM +1000, Balbir Singh wrote: > On Wed, 24 May 2017 13:20:21 -0400 > Jérôme Glisse wrote: > > > This patch add a new memory migration helpers, which migrate memory > > backing a range of virtual address of a process to different memory > > (which can be allocated

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Wed, 24 May 2017 13:20:21 -0400 Jérôme Glisse wrote: > This patch add a new memory migration helpers, which migrate memory > backing a range of virtual address of a process to different memory > (which can be allocated through special allocator). It differs from > numa

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Wed, 24 May 2017 13:20:21 -0400 Jérôme Glisse wrote: > This patch add a new memory migration helpers, which migrate memory > backing a range of virtual address of a process to different memory > (which can be allocated through special allocator). It differs from > numa migration by working on

[HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

This patch add a new memory migration helpers, which migrate memory backing a range of virtual address of a process to different memory (which can be allocated through special allocator). It differs from numa migration by working on a range of virtual address and thus by doing migration in chunk

[HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

This patch add a new memory migration helpers, which migrate memory backing a range of virtual address of a process to different memory (which can be allocated through special allocator). It differs from numa migration by working on a range of virtual address and thus by doing migration in chunk

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Mon, May 22, 2017 at 12:52:03PM -0400, Jérôme Glisse wrote: This patch add a new memory migration helpers, which migrate memory backing a range of virtual address of a process to different memory (which can be allocated through special allocator). It differs from numa migration by working on

Re: [HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

On Mon, May 22, 2017 at 12:52:03PM -0400, Jérôme Glisse wrote: This patch add a new memory migration helpers, which migrate memory backing a range of virtual address of a process to different memory (which can be allocated through special allocator). It differs from numa migration by working on

[HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

This patch add a new memory migration helpers, which migrate memory backing a range of virtual address of a process to different memory (which can be allocated through special allocator). It differs from numa migration by working on a range of virtual address and thus by doing migration in chunk

[HMM 12/15] mm/migrate: new memory migration helper for use with device memory v4

This patch add a new memory migration helpers, which migrate memory backing a range of virtual address of a process to different memory (which can be allocated through special allocator). It differs from numa migration by working on a range of virtual address and thus by doing migration in chunk