ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V7

2011-12-03 Thread Jerome Glisse
On Sat, Dec 3, 2011 at 4:41 PM, Jerome Glisse  wrote:
> On Sat, Dec 3, 2011 at 9:52 AM, David Airlie  wrote:
>>
>>
>> - Original Message -
>>> From: "Konrad Rzeszutek Wilk" 
>>> To: "Jerome Glisse" , dri-devel at 
>>> lists.freedesktop.org, "konrad wilk" ,
>>> thellstrom at vmware.com, airlied at gmail.com, "Jerome Glisse" >> redhat.com>
>>> Sent: Friday, 2 December, 2011 2:19:01 PM
>>> Subject: Re: ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V7
>>>
>>> On Fri, Nov 18, 2011 at 08:08:36PM -0500, Jerome Glisse wrote:
>>> > On Fri, Nov 18, 2011 at 08:04:47PM -0500, j.glisse at gmail.com
>>> > wrote:
>>> > > Important fix to patch 14, fix accounting of ghost bo. When
>>> > > creating a
>>> > > ghost bo we don't account it, so set its acc_size to 0 so that
>>> > > when
>>> > > ghost is release we don't overfree.
>>> > >
>>> > > I wonder how i didn't run into this before.
>>> > >
>>> > > Patch are also at
>>> > >
>>> > > http://people.freedesktop.org/~glisse/ttmdma/
>>> > >
>>> > > Cheers,
>>> > > Jerome
>>> > >
>>> >
>>> > Oh i forgot to add some of the reviewed by, i updated patches on
>>> > fdo,
>>> > i don't resend to the ml.
>>>
>>> Great! How should we ask Dave to pull them? Does he prefer to do it
>>> via git
>>> tree? In which I can create a branch with those patches and send him
>>> a GIT PULL
>>> email? Or is there a more convient way?
>>
>> If someone could suck them all into a git tree + all the Reviewed-by tags 
>> from the people who reviewed them it would make it a lot easier,
>>
>> I lost track of where this ended up as Jerome had a few balls in the air and 
>> I know Thomas wasn't liking some of them.
>>
>> So please send me a git url and I'll go review that next week.
>>
>> Dave.
>
> http://cgit.freedesktop.org/~glisse/linux/log/
>
> I need to add last reviewed by from Thomas. Note this tree also has
> other patches (multi ring lastest patchset) which we want for next too
> i believe.
>
> I will update this tree on monday with all the missing reviewed by
>
> Cheers,
> Jerome

Just to be extra informative my tree has:

drm/ttm: callback move_notify any time bo placement change v4
Got the reviewed by recently from Thomas

drm/ttm: simplify memory accounting for ttm user v2
Thomas reviewed version1, version2 only fix a bug so i think it's safe
to assume i have reviewed by Thomas for v2.

Cheers,
Jerome


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V7

2011-12-03 Thread Jerome Glisse
On Sat, Dec 3, 2011 at 9:52 AM, David Airlie  wrote:
>
>
> - Original Message -
>> From: "Konrad Rzeszutek Wilk" 
>> To: "Jerome Glisse" , dri-devel at 
>> lists.freedesktop.org, "konrad wilk" ,
>> thellstrom at vmware.com, airlied at gmail.com, "Jerome Glisse" > redhat.com>
>> Sent: Friday, 2 December, 2011 2:19:01 PM
>> Subject: Re: ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V7
>>
>> On Fri, Nov 18, 2011 at 08:08:36PM -0500, Jerome Glisse wrote:
>> > On Fri, Nov 18, 2011 at 08:04:47PM -0500, j.glisse at gmail.com
>> > wrote:
>> > > Important fix to patch 14, fix accounting of ghost bo. When
>> > > creating a
>> > > ghost bo we don't account it, so set its acc_size to 0 so that
>> > > when
>> > > ghost is release we don't overfree.
>> > >
>> > > I wonder how i didn't run into this before.
>> > >
>> > > Patch are also at
>> > >
>> > > http://people.freedesktop.org/~glisse/ttmdma/
>> > >
>> > > Cheers,
>> > > Jerome
>> > >
>> >
>> > Oh i forgot to add some of the reviewed by, i updated patches on
>> > fdo,
>> > i don't resend to the ml.
>>
>> Great! How should we ask Dave to pull them? Does he prefer to do it
>> via git
>> tree? In which I can create a branch with those patches and send him
>> a GIT PULL
>> email? Or is there a more convient way?
>
> If someone could suck them all into a git tree + all the Reviewed-by tags 
> from the people who reviewed them it would make it a lot easier,
>
> I lost track of where this ended up as Jerome had a few balls in the air and 
> I know Thomas wasn't liking some of them.
>
> So please send me a git url and I'll go review that next week.
>
> Dave.

http://cgit.freedesktop.org/~glisse/linux/log/

I need to add last reviewed by from Thomas. Note this tree also has
other patches (multi ring lastest patchset) which we want for next too
i believe.

I will update this tree on monday with all the missing reviewed by

Cheers,
Jerome


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V7

2011-12-03 Thread David Airlie


- Original Message -
> From: "Konrad Rzeszutek Wilk" 
> To: "Jerome Glisse" , dri-devel at 
> lists.freedesktop.org, "konrad wilk" ,
> thellstrom at vmware.com, airlied at gmail.com, "Jerome Glisse"  redhat.com>
> Sent: Friday, 2 December, 2011 2:19:01 PM
> Subject: Re: ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V7
> 
> On Fri, Nov 18, 2011 at 08:08:36PM -0500, Jerome Glisse wrote:
> > On Fri, Nov 18, 2011 at 08:04:47PM -0500, j.glisse at gmail.com
> > wrote:
> > > Important fix to patch 14, fix accounting of ghost bo. When
> > > creating a
> > > ghost bo we don't account it, so set its acc_size to 0 so that
> > > when
> > > ghost is release we don't overfree.
> > > 
> > > I wonder how i didn't run into this before.
> > > 
> > > Patch are also at
> > > 
> > > http://people.freedesktop.org/~glisse/ttmdma/
> > > 
> > > Cheers,
> > > Jerome
> > > 
> > 
> > Oh i forgot to add some of the reviewed by, i updated patches on
> > fdo,
> > i don't resend to the ml.
> 
> Great! How should we ask Dave to pull them? Does he prefer to do it
> via git
> tree? In which I can create a branch with those patches and send him
> a GIT PULL
> email? Or is there a more convient way?

If someone could suck them all into a git tree + all the Reviewed-by tags from 
the people who reviewed them it would make it a lot easier,

I lost track of where this ended up as Jerome had a few balls in the air and I 
know Thomas wasn't liking some of them.

So please send me a git url and I'll go review that next week.

Dave.


Re: ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator V7

2011-12-03 Thread Konrad Rzeszutek Wilk
On Fri, Nov 18, 2011 at 08:08:36PM -0500, Jerome Glisse wrote:
 On Fri, Nov 18, 2011 at 08:04:47PM -0500, j.glisse at gmail.com wrote:
  Important fix to patch 14, fix accounting of ghost bo. When creating a
  ghost bo we don't account it, so set its acc_size to 0 so that when
  ghost is release we don't overfree.
  
  I wonder how i didn't run into this before.
  
  Patch are also at
  
  http://people.freedesktop.org/~glisse/ttmdma/
  
  Cheers,
  Jerome
  
 
 Oh i forgot to add some of the reviewed by, i updated patches on fdo,
 i don't resend to the ml.

Great! How should we ask Dave to pull them? Does he prefer to do it via git
tree? In which I can create a branch with those patches and send him a GIT PULL
email? Or is there a more convient way?

 Cheers,
 Jerome
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


Re: ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator V7

2011-12-03 Thread Jerome Glisse
On Sat, Dec 3, 2011 at 9:52 AM, David Airlie airl...@redhat.com wrote:


 - Original Message -
 From: Konrad Rzeszutek Wilk konrad.w...@oracle.com
 To: Jerome Glisse j.gli...@gmail.com, dri-devel@lists.freedesktop.org, 
 konrad wilk konrad.w...@oracle.com,
 thellst...@vmware.com, airl...@gmail.com, Jerome Glisse 
 jgli...@redhat.com
 Sent: Friday, 2 December, 2011 2:19:01 PM
 Subject: Re: ttm: merge ttm_backend  ttm_tt, introduce ttm dma allocator V7

 On Fri, Nov 18, 2011 at 08:08:36PM -0500, Jerome Glisse wrote:
  On Fri, Nov 18, 2011 at 08:04:47PM -0500, j.glisse at gmail.com
  wrote:
   Important fix to patch 14, fix accounting of ghost bo. When
   creating a
   ghost bo we don't account it, so set its acc_size to 0 so that
   when
   ghost is release we don't overfree.
  
   I wonder how i didn't run into this before.
  
   Patch are also at
  
   http://people.freedesktop.org/~glisse/ttmdma/
  
   Cheers,
   Jerome
  
 
  Oh i forgot to add some of the reviewed by, i updated patches on
  fdo,
  i don't resend to the ml.

 Great! How should we ask Dave to pull them? Does he prefer to do it
 via git
 tree? In which I can create a branch with those patches and send him
 a GIT PULL
 email? Or is there a more convient way?

 If someone could suck them all into a git tree + all the Reviewed-by tags 
 from the people who reviewed them it would make it a lot easier,

 I lost track of where this ended up as Jerome had a few balls in the air and 
 I know Thomas wasn't liking some of them.

 So please send me a git url and I'll go review that next week.

 Dave.

http://cgit.freedesktop.org/~glisse/linux/log/

I need to add last reviewed by from Thomas. Note this tree also has
other patches (multi ring lastest patchset) which we want for next too
i believe.

I will update this tree on monday with all the missing reviewed by

Cheers,
Jerome
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V7

2011-12-02 Thread Konrad Rzeszutek Wilk
On Fri, Nov 18, 2011 at 08:08:36PM -0500, Jerome Glisse wrote:
> On Fri, Nov 18, 2011 at 08:04:47PM -0500, j.glisse at gmail.com wrote:
> > Important fix to patch 14, fix accounting of ghost bo. When creating a
> > ghost bo we don't account it, so set its acc_size to 0 so that when
> > ghost is release we don't overfree.
> > 
> > I wonder how i didn't run into this before.
> > 
> > Patch are also at
> > 
> > http://people.freedesktop.org/~glisse/ttmdma/
> > 
> > Cheers,
> > Jerome
> > 
> 
> Oh i forgot to add some of the reviewed by, i updated patches on fdo,
> i don't resend to the ml.

Great! How should we ask Dave to pull them? Does he prefer to do it via git
tree? In which I can create a branch with those patches and send him a GIT PULL
email? Or is there a more convient way?

> Cheers,
> Jerome


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V7

2011-11-18 Thread Jerome Glisse
On Fri, Nov 18, 2011 at 08:04:47PM -0500, j.glisse at gmail.com wrote:
> Important fix to patch 14, fix accounting of ghost bo. When creating a
> ghost bo we don't account it, so set its acc_size to 0 so that when
> ghost is release we don't overfree.
> 
> I wonder how i didn't run into this before.
> 
> Patch are also at
> 
> http://people.freedesktop.org/~glisse/ttmdma/
> 
> Cheers,
> Jerome
> 

Oh i forgot to add some of the reviewed by, i updated patches on fdo,
i don't resend to the ml.

Cheers,
Jerome


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V7

2011-11-18 Thread j.gli...@gmail.com
Important fix to patch 14, fix accounting of ghost bo. When creating a
ghost bo we don't account it, so set its acc_size to 0 so that when
ghost is release we don't overfree.

I wonder how i didn't run into this before.

Patch are also at

http://people.freedesktop.org/~glisse/ttmdma/

Cheers,
Jerome



ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator V7

2011-11-18 Thread j . glisse
Important fix to patch 14, fix accounting of ghost bo. When creating a
ghost bo we don't account it, so set its acc_size to 0 so that when
ghost is release we don't overfree.

I wonder how i didn't run into this before.

Patch are also at

http://people.freedesktop.org/~glisse/ttmdma/

Cheers,
Jerome

___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V6

2011-11-17 Thread Konrad Rzeszutek Wilk
On Wed, Nov 16, 2011 at 11:57:24AM -0500, j.glisse at gmail.com wrote:
> Respin some of the patch with syntax/typo fix + patch 10 with proper
> memory accounting of page being free.

Ran it yesterday all day and as well today using the TTM DMA (so SWIOTLB is on)
and had no trouble. This is with a Radeon HD 4870 on a AMD Phenom 6-core thingy.

Next will, when I am back, will run it with some more hardware - some ATI 
low-end,
NVidia low and high-end and try as well on a G5 which has a radeon card in it.

Oh, didn't even try to revert the patch you mentioned - I just used your
latest posting code and ran with that.


Re: ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator V6

2011-11-17 Thread Konrad Rzeszutek Wilk
On Wed, Nov 16, 2011 at 11:57:24AM -0500, j.gli...@gmail.com wrote:
 Respin some of the patch with syntax/typo fix + patch 10 with proper
 memory accounting of page being free.

Ran it yesterday all day and as well today using the TTM DMA (so SWIOTLB is on)
and had no trouble. This is with a Radeon HD 4870 on a AMD Phenom 6-core thingy.

Next will, when I am back, will run it with some more hardware - some ATI 
low-end,
NVidia low and high-end and try as well on a G5 which has a radeon card in it.

Oh, didn't even try to revert the patch you mentioned - I just used your
latest posting code and ran with that.
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V6

2011-11-16 Thread j.gli...@gmail.com
Respin some of the patch with syntax/typo fix + patch 10 with proper
memory accounting of page being free.

Cheers,
Jerome



ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V5

2011-11-16 Thread Konrad Rzeszutek Wilk
> > which I think was your V4 posting (or earlier) (the last patch is something 
> > I added
> > to toggle it off/on to test).
> 
> You have to allocate like a million of gem object to trigger it.
> 

Which I certainly did not, but I did find an accounting error in the
rebased v5 version of the TTM DMA that might be the culprit. Will build
a kernel shortly with those changes and see how it fares.

> >
> > The latest (V5) hit the OOM quite fast - took about an hour or two of me
> > answering emails (mutt inside gnome-terminal) and using both chrome and 
> > firefox,
> > and running a make -j10 on the Linux kernel.
> >
> > Time to turn on more debugging :-)
> >
> 
> Hhhhmm weird i did not run into that. will try to run heavy make with
> graphic along side. Does it happen without
> 0014-drm-ttm-simplify-memory-accounting-for-ttm-user.patch ?

Will try reverting it out and see what happends.


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V5

2011-11-16 Thread Jerome Glisse
On Wed, Nov 16, 2011 at 10:15 AM, Konrad Rzeszutek Wilk
 wrote:
> On Mon, Nov 14, 2011 at 01:54:27PM -0500, Jerome Glisse wrote:
>> On Mon, Nov 14, 2011 at 05:06:42PM +0100, Thomas Hellstrom wrote:
>> > On 11/14/2011 05:02 PM, Jerome Glisse wrote:
>> > >On Mon, Nov 14, 2011 at 9:49 AM, Thomas Hellstrom> > >vmware.com> ?wrote:
>> > >>On 11/11/2011 11:47 PM, j.glisse at gmail.com wrote:
>> > >>>So attached is updated serie of patch with fixes for swaping issue
>> > >>>that also impacted memory accounting.
>> > >>Jerome,
>> > >>Out of interest, what was the swapping issue?
>> > >>
>> > >>/Thomas
>> > >>
>> > >>
>> > >The page list was NULL and some access ended up freaking the kernel
>> > >and oom kicked in for some reason i still don't understand. Bottom
>
> Was it like this:
>
> Nov 16 10:02:02 phenom kernel: [ 4477.232933] [TTM] Out of kernel memory.
> Nov 16 10:02:02 phenom kernel: [ 4477.232958] radeon :01:00.0: 
> object_init failed for (12288, 0x0002)
> Nov 16 10:02:02 phenom kernel: [ 4477.232963] [drm:radeon_gem_object_create] 
> *ERROR* Failed to allocate GEM object (12288, 2, 4096, -12)
> Nov 16 10:02:02 phenom kernel: [ 4477.248329] [TTM] Out of kernel memory.
>
> followed up with an OOM?

Can't say want to fast last message i got on screen was oom and then
complete freeze.

>> > >line things were going south in a split second leaving me with no log
>> > >or access, i thought it was accounting not working as it was what i
>> > >tested at the time. I should have tested accouting without the
>> > >patchset
>
> Weirdly enough, I did _not_ see this with the patchset:
>
> 592d002 drm/ttm: remove userspace backed ttm object support
> 9f1cf44 drm/ttm: remove split btw highmen and lowmem page
> d27ea32 drm/ttm: remove unused backend flags field
> dad5ef9 drm/ttm: use ttm put pages function to properly restore cache 
> attribute
> 6aa902d drm/ttm: overhaul memory accounting
> 60d0fa6 drm/ttm: convert page allocation to use page ptr array instead of 
> list V4
> abde3ec drm/ttm: test for dma_address array allocation failure
> 8145582 drm/ttm: merge ttm_backend and ttm_tt V2
> 0216e52 drm/ttm: introduce callback for ttm_tt populate & unpopulate V2
> ecb0d22 ttm: Provide DMA aware TTM page pool code. V5
> 793dc40 swiotlb: Expose swiotlb_nr_tlb function to modules
> 767aa47 drm/radeon/kms: Enable the TTM DMA pool if swiotlb is on V2
> e91d0f0 drm/nouveau: enable the TTM DMA pool on 32-bit DMA only device V2
> 6eda9c3 ttm:dma: Add 'ttm_dma' module to radeon and nouveau to force enable 
> the TTM DMA
>
> which I think was your V4 posting (or earlier) (the last patch is something I 
> added
> to toggle it off/on to test).

You have to allocate like a million of gem object to trigger it.

>
> The latest (V5) hit the OOM quite fast - took about an hour or two of me
> answering emails (mutt inside gnome-terminal) and using both chrome and 
> firefox,
> and running a make -j10 on the Linux kernel.
>
> Time to turn on more debugging :-)
>

Hhhhmm weird i did not run into that. will try to run heavy make with
graphic along side. Does it happen without
0014-drm-ttm-simplify-memory-accounting-for-ttm-user.patch ?

Cheers,
Jerome


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V5

2011-11-16 Thread Konrad Rzeszutek Wilk
On Mon, Nov 14, 2011 at 01:54:27PM -0500, Jerome Glisse wrote:
> On Mon, Nov 14, 2011 at 05:06:42PM +0100, Thomas Hellstrom wrote:
> > On 11/14/2011 05:02 PM, Jerome Glisse wrote:
> > >On Mon, Nov 14, 2011 at 9:49 AM, Thomas Hellstrom > >vmware.com>  wrote:
> > >>On 11/11/2011 11:47 PM, j.glisse at gmail.com wrote:
> > >>>So attached is updated serie of patch with fixes for swaping issue
> > >>>that also impacted memory accounting.
> > >>Jerome,
> > >>Out of interest, what was the swapping issue?
> > >>
> > >>/Thomas
> > >>
> > >>
> > >The page list was NULL and some access ended up freaking the kernel
> > >and oom kicked in for some reason i still don't understand. Bottom

Was it like this:

Nov 16 10:02:02 phenom kernel: [ 4477.232933] [TTM] Out of kernel memory.
Nov 16 10:02:02 phenom kernel: [ 4477.232958] radeon :01:00.0: object_init 
failed for (12288, 0x0002)
Nov 16 10:02:02 phenom kernel: [ 4477.232963] [drm:radeon_gem_object_create] 
*ERROR* Failed to allocate GEM object (12288, 2, 4096, -12)
Nov 16 10:02:02 phenom kernel: [ 4477.248329] [TTM] Out of kernel memory.

followed up with an OOM?

> > >line things were going south in a split second leaving me with no log
> > >or access, i thought it was accounting not working as it was what i
> > >tested at the time. I should have tested accouting without the
> > >patchset

Weirdly enough, I did _not_ see this with the patchset:

592d002 drm/ttm: remove userspace backed ttm object support
9f1cf44 drm/ttm: remove split btw highmen and lowmem page
d27ea32 drm/ttm: remove unused backend flags field
dad5ef9 drm/ttm: use ttm put pages function to properly restore cache attribute
6aa902d drm/ttm: overhaul memory accounting
60d0fa6 drm/ttm: convert page allocation to use page ptr array instead of list 
V4
abde3ec drm/ttm: test for dma_address array allocation failure
8145582 drm/ttm: merge ttm_backend and ttm_tt V2
0216e52 drm/ttm: introduce callback for ttm_tt populate & unpopulate V2
ecb0d22 ttm: Provide DMA aware TTM page pool code. V5
793dc40 swiotlb: Expose swiotlb_nr_tlb function to modules
767aa47 drm/radeon/kms: Enable the TTM DMA pool if swiotlb is on V2
e91d0f0 drm/nouveau: enable the TTM DMA pool on 32-bit DMA only device V2
6eda9c3 ttm:dma: Add 'ttm_dma' module to radeon and nouveau to force enable the 
TTM DMA

which I think was your V4 posting (or earlier) (the last patch is something I 
added
to toggle it off/on to test).

The latest (V5) hit the OOM quite fast - took about an hour or two of me
answering emails (mutt inside gnome-terminal) and using both chrome and firefox,
and running a make -j10 on the Linux kernel.

Time to turn on more debugging :-)


Re: ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator V5

2011-11-16 Thread Konrad Rzeszutek Wilk
On Mon, Nov 14, 2011 at 01:54:27PM -0500, Jerome Glisse wrote:
 On Mon, Nov 14, 2011 at 05:06:42PM +0100, Thomas Hellstrom wrote:
  On 11/14/2011 05:02 PM, Jerome Glisse wrote:
  On Mon, Nov 14, 2011 at 9:49 AM, Thomas Hellstromthellst...@vmware.com  
  wrote:
  On 11/11/2011 11:47 PM, j.gli...@gmail.com wrote:
  So attached is updated serie of patch with fixes for swaping issue
  that also impacted memory accounting.
  Jerome,
  Out of interest, what was the swapping issue?
  
  /Thomas
  
  
  The page list was NULL and some access ended up freaking the kernel
  and oom kicked in for some reason i still don't understand. Bottom

Was it like this:

Nov 16 10:02:02 phenom kernel: [ 4477.232933] [TTM] Out of kernel memory.
Nov 16 10:02:02 phenom kernel: [ 4477.232958] radeon :01:00.0: object_init 
failed for (12288, 0x0002)
Nov 16 10:02:02 phenom kernel: [ 4477.232963] [drm:radeon_gem_object_create] 
*ERROR* Failed to allocate GEM object (12288, 2, 4096, -12)
Nov 16 10:02:02 phenom kernel: [ 4477.248329] [TTM] Out of kernel memory.

followed up with an OOM?

  line things were going south in a split second leaving me with no log
  or access, i thought it was accounting not working as it was what i
  tested at the time. I should have tested accouting without the
  patchset

Weirdly enough, I did _not_ see this with the patchset:

592d002 drm/ttm: remove userspace backed ttm object support
9f1cf44 drm/ttm: remove split btw highmen and lowmem page
d27ea32 drm/ttm: remove unused backend flags field
dad5ef9 drm/ttm: use ttm put pages function to properly restore cache attribute
6aa902d drm/ttm: overhaul memory accounting
60d0fa6 drm/ttm: convert page allocation to use page ptr array instead of list 
V4
abde3ec drm/ttm: test for dma_address array allocation failure
8145582 drm/ttm: merge ttm_backend and ttm_tt V2
0216e52 drm/ttm: introduce callback for ttm_tt populate  unpopulate V2
ecb0d22 ttm: Provide DMA aware TTM page pool code. V5
793dc40 swiotlb: Expose swiotlb_nr_tlb function to modules
767aa47 drm/radeon/kms: Enable the TTM DMA pool if swiotlb is on V2
e91d0f0 drm/nouveau: enable the TTM DMA pool on 32-bit DMA only device V2
6eda9c3 ttm:dma: Add 'ttm_dma' module to radeon and nouveau to force enable the 
TTM DMA

which I think was your V4 posting (or earlier) (the last patch is something I 
added
to toggle it off/on to test).

The latest (V5) hit the OOM quite fast - took about an hour or two of me
answering emails (mutt inside gnome-terminal) and using both chrome and firefox,
and running a make -j10 on the Linux kernel.

Time to turn on more debugging :-)
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


Re: ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator V5

2011-11-16 Thread Jerome Glisse
On Wed, Nov 16, 2011 at 10:15 AM, Konrad Rzeszutek Wilk
konrad.w...@oracle.com wrote:
 On Mon, Nov 14, 2011 at 01:54:27PM -0500, Jerome Glisse wrote:
 On Mon, Nov 14, 2011 at 05:06:42PM +0100, Thomas Hellstrom wrote:
  On 11/14/2011 05:02 PM, Jerome Glisse wrote:
  On Mon, Nov 14, 2011 at 9:49 AM, Thomas Hellstromthellst...@vmware.com  
  wrote:
  On 11/11/2011 11:47 PM, j.gli...@gmail.com wrote:
  So attached is updated serie of patch with fixes for swaping issue
  that also impacted memory accounting.
  Jerome,
  Out of interest, what was the swapping issue?
  
  /Thomas
  
  
  The page list was NULL and some access ended up freaking the kernel
  and oom kicked in for some reason i still don't understand. Bottom

 Was it like this:

 Nov 16 10:02:02 phenom kernel: [ 4477.232933] [TTM] Out of kernel memory.
 Nov 16 10:02:02 phenom kernel: [ 4477.232958] radeon :01:00.0: 
 object_init failed for (12288, 0x0002)
 Nov 16 10:02:02 phenom kernel: [ 4477.232963] [drm:radeon_gem_object_create] 
 *ERROR* Failed to allocate GEM object (12288, 2, 4096, -12)
 Nov 16 10:02:02 phenom kernel: [ 4477.248329] [TTM] Out of kernel memory.

 followed up with an OOM?

Can't say want to fast last message i got on screen was oom and then
complete freeze.

  line things were going south in a split second leaving me with no log
  or access, i thought it was accounting not working as it was what i
  tested at the time. I should have tested accouting without the
  patchset

 Weirdly enough, I did _not_ see this with the patchset:

 592d002 drm/ttm: remove userspace backed ttm object support
 9f1cf44 drm/ttm: remove split btw highmen and lowmem page
 d27ea32 drm/ttm: remove unused backend flags field
 dad5ef9 drm/ttm: use ttm put pages function to properly restore cache 
 attribute
 6aa902d drm/ttm: overhaul memory accounting
 60d0fa6 drm/ttm: convert page allocation to use page ptr array instead of 
 list V4
 abde3ec drm/ttm: test for dma_address array allocation failure
 8145582 drm/ttm: merge ttm_backend and ttm_tt V2
 0216e52 drm/ttm: introduce callback for ttm_tt populate  unpopulate V2
 ecb0d22 ttm: Provide DMA aware TTM page pool code. V5
 793dc40 swiotlb: Expose swiotlb_nr_tlb function to modules
 767aa47 drm/radeon/kms: Enable the TTM DMA pool if swiotlb is on V2
 e91d0f0 drm/nouveau: enable the TTM DMA pool on 32-bit DMA only device V2
 6eda9c3 ttm:dma: Add 'ttm_dma' module to radeon and nouveau to force enable 
 the TTM DMA

 which I think was your V4 posting (or earlier) (the last patch is something I 
 added
 to toggle it off/on to test).

You have to allocate like a million of gem object to trigger it.


 The latest (V5) hit the OOM quite fast - took about an hour or two of me
 answering emails (mutt inside gnome-terminal) and using both chrome and 
 firefox,
 and running a make -j10 on the Linux kernel.

 Time to turn on more debugging :-)


Hhhhmm weird i did not run into that. will try to run heavy make with
graphic along side. Does it happen without
0014-drm-ttm-simplify-memory-accounting-for-ttm-user.patch ?

Cheers,
Jerome
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


Re: ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator V5

2011-11-16 Thread Konrad Rzeszutek Wilk
  which I think was your V4 posting (or earlier) (the last patch is something 
  I added
  to toggle it off/on to test).
 
 You have to allocate like a million of gem object to trigger it.
 

Which I certainly did not, but I did find an accounting error in the
rebased v5 version of the TTM DMA that might be the culprit. Will build
a kernel shortly with those changes and see how it fares.

 
  The latest (V5) hit the OOM quite fast - took about an hour or two of me
  answering emails (mutt inside gnome-terminal) and using both chrome and 
  firefox,
  and running a make -j10 on the Linux kernel.
 
  Time to turn on more debugging :-)
 
 
 Hhhhmm weird i did not run into that. will try to run heavy make with
 graphic along side. Does it happen without
 0014-drm-ttm-simplify-memory-accounting-for-ttm-user.patch ?

Will try reverting it out and see what happends.
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator V6

2011-11-16 Thread j . glisse
Respin some of the patch with syntax/typo fix + patch 10 with proper
memory accounting of page being free.

Cheers,
Jerome

___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V5

2011-11-14 Thread Thomas Hellstrom
On 11/14/2011 07:54 PM, Jerome Glisse wrote:
> On Mon, Nov 14, 2011 at 05:06:42PM +0100, Thomas Hellstrom wrote:
>
>> On 11/14/2011 05:02 PM, Jerome Glisse wrote:
>>  
>>> On Mon, Nov 14, 2011 at 9:49 AM, Thomas Hellstrom 
>>>   wrote:
>>>
 On 11/11/2011 11:47 PM, j.glisse at gmail.com wrote:
  
> So attached is updated serie of patch with fixes for swaping issue
> that also impacted memory accounting.
>
 Jerome,
 Out of interest, what was the swapping issue?

 /Thomas


  
>>> The page list was NULL and some access ended up freaking the kernel
>>> and oom kicked in for some reason i still don't understand. Bottom
>>> line things were going south in a split second leaving me with no log
>>> or access, i thought it was accounting not working as it was what i
>>> tested at the time. I should have tested accouting without the
>>> patchset
>>>
>>> Cheers,
>>> Jerome
>>>
>> Still, I think you have a point in the fact that *when* finally the
>> OOM killer kicks in,
>> it probably doesn't have a way to associate user-space created bos
>> with processes
>> and kill the right process.
>>
>> /Thomas
>>
>>  
> Btw can you review patch 14 as it touch vmwgfx, i am pretty sure
> i got it right.
>
> Cheers,
> Jerome
>
Yes I will do a more thorough review tomorrow of all unreviewed stuff.

/Thomas








ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V5

2011-11-14 Thread Thomas Hellstrom
On 11/14/2011 05:02 PM, Jerome Glisse wrote:
> On Mon, Nov 14, 2011 at 9:49 AM, Thomas Hellstrom  
> wrote:
>
>> On 11/11/2011 11:47 PM, j.glisse at gmail.com wrote:
>>  
>>> So attached is updated serie of patch with fixes for swaping issue
>>> that also impacted memory accounting.
>>>
>> Jerome,
>> Out of interest, what was the swapping issue?
>>
>> /Thomas
>>
>>
>>  
> The page list was NULL and some access ended up freaking the kernel
> and oom kicked in for some reason i still don't understand. Bottom
> line things were going south in a split second leaving me with no log
> or access, i thought it was accounting not working as it was what i
> tested at the time. I should have tested accouting without the
> patchset
>
> Cheers,
> Jerome
>

Still, I think you have a point in the fact that *when* finally the OOM 
killer kicks in,
it probably doesn't have a way to associate user-space created bos with 
processes
and kill the right process.

/Thomas



ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V5

2011-11-14 Thread Thomas Hellstrom
On 11/11/2011 11:47 PM, j.glisse at gmail.com wrote:
> So attached is updated serie of patch with fixes for swaping issue
> that also impacted memory accounting.

Jerome,
Out of interest, what was the swapping issue?

/Thomas



>   Last patch fix memory accounting
> for radeon&  nouveau. I think it's ready to go into drm-next, patchset
> is against linus tree as there is thing there not in next that conflict.
> (in radeon iirc)
>
> Cheers,
> Jerome
>
>



ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V5

2011-11-14 Thread Jerome Glisse
On Mon, Nov 14, 2011 at 05:06:42PM +0100, Thomas Hellstrom wrote:
> On 11/14/2011 05:02 PM, Jerome Glisse wrote:
> >On Mon, Nov 14, 2011 at 9:49 AM, Thomas Hellstrom  
> >wrote:
> >>On 11/11/2011 11:47 PM, j.glisse at gmail.com wrote:
> >>>So attached is updated serie of patch with fixes for swaping issue
> >>>that also impacted memory accounting.
> >>Jerome,
> >>Out of interest, what was the swapping issue?
> >>
> >>/Thomas
> >>
> >>
> >The page list was NULL and some access ended up freaking the kernel
> >and oom kicked in for some reason i still don't understand. Bottom
> >line things were going south in a split second leaving me with no log
> >or access, i thought it was accounting not working as it was what i
> >tested at the time. I should have tested accouting without the
> >patchset
> >
> >Cheers,
> >Jerome
> 
> Still, I think you have a point in the fact that *when* finally the
> OOM killer kicks in,
> it probably doesn't have a way to associate user-space created bos
> with processes
> and kill the right process.
> 
> /Thomas
> 

Btw can you review patch 14 as it touch vmwgfx, i am pretty sure
i got it right.

Cheers,
Jerome


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V5

2011-11-14 Thread Jerome Glisse
On Mon, Nov 14, 2011 at 9:49 AM, Thomas Hellstrom  
wrote:
> On 11/11/2011 11:47 PM, j.glisse at gmail.com wrote:
>>
>> So attached is updated serie of patch with fixes for swaping issue
>> that also impacted memory accounting.
>
> Jerome,
> Out of interest, what was the swapping issue?
>
> /Thomas
>
>

The page list was NULL and some access ended up freaking the kernel
and oom kicked in for some reason i still don't understand. Bottom
line things were going south in a split second leaving me with no log
or access, i thought it was accounting not working as it was what i
tested at the time. I should have tested accouting without the
patchset

Cheers,
Jerome


Re: ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator V5

2011-11-14 Thread Thomas Hellstrom

On 11/11/2011 11:47 PM, j.gli...@gmail.com wrote:

So attached is updated serie of patch with fixes for swaping issue
that also impacted memory accounting.


Jerome,
Out of interest, what was the swapping issue?

/Thomas




  Last patch fix memory accounting
for radeon  nouveau. I think it's ready to go into drm-next, patchset
is against linus tree as there is thing there not in next that conflict.
(in radeon iirc)

Cheers,
Jerome

   


___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


Re: ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator V5

2011-11-14 Thread Jerome Glisse
On Mon, Nov 14, 2011 at 9:49 AM, Thomas Hellstrom thellst...@vmware.com wrote:
 On 11/11/2011 11:47 PM, j.gli...@gmail.com wrote:

 So attached is updated serie of patch with fixes for swaping issue
 that also impacted memory accounting.

 Jerome,
 Out of interest, what was the swapping issue?

 /Thomas



The page list was NULL and some access ended up freaking the kernel
and oom kicked in for some reason i still don't understand. Bottom
line things were going south in a split second leaving me with no log
or access, i thought it was accounting not working as it was what i
tested at the time. I should have tested accouting without the
patchset

Cheers,
Jerome
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


Re: ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator V5

2011-11-14 Thread Thomas Hellstrom

On 11/14/2011 05:02 PM, Jerome Glisse wrote:

On Mon, Nov 14, 2011 at 9:49 AM, Thomas Hellstromthellst...@vmware.com  wrote:
   

On 11/11/2011 11:47 PM, j.gli...@gmail.com wrote:
 

So attached is updated serie of patch with fixes for swaping issue
that also impacted memory accounting.
   

Jerome,
Out of interest, what was the swapping issue?

/Thomas


 

The page list was NULL and some access ended up freaking the kernel
and oom kicked in for some reason i still don't understand. Bottom
line things were going south in a split second leaving me with no log
or access, i thought it was accounting not working as it was what i
tested at the time. I should have tested accouting without the
patchset

Cheers,
Jerome
   


Still, I think you have a point in the fact that *when* finally the OOM 
killer kicks in,
it probably doesn't have a way to associate user-space created bos with 
processes

and kill the right process.

/Thomas

___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


Re: ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator V5

2011-11-14 Thread Jerome Glisse
On Mon, Nov 14, 2011 at 05:06:42PM +0100, Thomas Hellstrom wrote:
 On 11/14/2011 05:02 PM, Jerome Glisse wrote:
 On Mon, Nov 14, 2011 at 9:49 AM, Thomas Hellstromthellst...@vmware.com  
 wrote:
 On 11/11/2011 11:47 PM, j.gli...@gmail.com wrote:
 So attached is updated serie of patch with fixes for swaping issue
 that also impacted memory accounting.
 Jerome,
 Out of interest, what was the swapping issue?
 
 /Thomas
 
 
 The page list was NULL and some access ended up freaking the kernel
 and oom kicked in for some reason i still don't understand. Bottom
 line things were going south in a split second leaving me with no log
 or access, i thought it was accounting not working as it was what i
 tested at the time. I should have tested accouting without the
 patchset
 
 Cheers,
 Jerome
 
 Still, I think you have a point in the fact that *when* finally the
 OOM killer kicks in,
 it probably doesn't have a way to associate user-space created bos
 with processes
 and kill the right process.
 
 /Thomas
 

Btw can you review patch 14 as it touch vmwgfx, i am pretty sure
i got it right.

Cheers,
Jerome
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


Re: ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator V5

2011-11-14 Thread Thomas Hellstrom

On 11/14/2011 07:54 PM, Jerome Glisse wrote:

On Mon, Nov 14, 2011 at 05:06:42PM +0100, Thomas Hellstrom wrote:
   

On 11/14/2011 05:02 PM, Jerome Glisse wrote:
 

On Mon, Nov 14, 2011 at 9:49 AM, Thomas Hellstromthellst...@vmware.com   
wrote:
   

On 11/11/2011 11:47 PM, j.gli...@gmail.com wrote:
 

So attached is updated serie of patch with fixes for swaping issue
that also impacted memory accounting.
   

Jerome,
Out of interest, what was the swapping issue?

/Thomas


 

The page list was NULL and some access ended up freaking the kernel
and oom kicked in for some reason i still don't understand. Bottom
line things were going south in a split second leaving me with no log
or access, i thought it was accounting not working as it was what i
tested at the time. I should have tested accouting without the
patchset

Cheers,
Jerome
   

Still, I think you have a point in the fact that *when* finally the
OOM killer kicks in,
it probably doesn't have a way to associate user-space created bos
with processes
and kill the right process.

/Thomas

 

Btw can you review patch 14 as it touch vmwgfx, i am pretty sure
i got it right.

Cheers,
Jerome
   

Yes I will do a more thorough review tomorrow of all unreviewed stuff.

/Thomas






___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V5

2011-11-11 Thread j.gli...@gmail.com
So attached is updated serie of patch with fixes for swaping issue
that also impacted memory accounting. Last patch fix memory accounting
for radeon & nouveau. I think it's ready to go into drm-next, patchset
is against linus tree as there is thing there not in next that conflict.
(in radeon iirc)

Cheers,
Jerome



ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator V5

2011-11-11 Thread j . glisse
So attached is updated serie of patch with fixes for swaping issue
that also impacted memory accounting. Last patch fix memory accounting
for radeon  nouveau. I think it's ready to go into drm-next, patchset
is against linus tree as there is thing there not in next that conflict.
(in radeon iirc)

Cheers,
Jerome

___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator V4

2011-11-10 Thread j.gli...@gmail.com
So squeezed all to avoid any memory accouting messing, seems to work ok
so far.

Cheers,
Jerome



ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator V4

2011-11-10 Thread j . glisse
So squeezed all to avoid any memory accouting messing, seems to work ok
so far.

Cheers,
Jerome

___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator

2011-11-09 Thread Thomas Hellstrom
On 11/09/2011 09:22 PM, j.glisse at gmail.com wrote:
> So i did an overhaul of ttm_memory, i believe the simplification i did
> make sense. See patch 5 for a longer explanation.
>
>
> Thomas with the ttm_memory change the allocation of pages won't happen
> if the accounting report that we are going over the limit and bo shrinker
> failed to free any memory to make room.
>
> The handling of dma32 zone is done as post pass of ttm memory accounting.
>

OK. I'll take a deeper look into this.

> Regarding the pagefault comment i removed, it doesn't make sense anymore
> because now we populate the whole page table in one shot. So there is
> no more prefaulting few pages but a full prefaulting. Thought i can
> add a comment stating that if you like.
>

It's important that we distinguish between populating, which populates 
pages,
and faulting, which add ptes pointing to those pages.

Previously populating happened as a side effect of faulting, but now 
that populating is done
in a single step, faulting (adding ptes) is still not. Usually a fault() 
handler adds a single pte,
but TTM is different and tries to prefault more, but it is important 
that we only error on the first
pte, so that's why the comment should stay.

> For the ttm_tt_dma struct to hold page allocator specific informations
> i think it can be done as an followup patch but if you prefer to have
> that in this patchset let me know i will respin with such changes.
>
>

I'm fine with having it as a separate patchset as long as it gets done :).


> I am in the process of retesting this whole serie and especialy the
> while memory accounting.
>
> Cheers,
> Jerome
>

/Thomas



ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator

2011-11-09 Thread Jerome Glisse
On Wed, Nov 09, 2011 at 09:25:20PM +0100, Thomas Hellstrom wrote:
> On 11/09/2011 09:22 PM, j.glisse at gmail.com wrote:
> >So i did an overhaul of ttm_memory, i believe the simplification i did
> >make sense. See patch 5 for a longer explanation.
> >
> >Thomas with the ttm_memory change the allocation of pages won't happen
> >if the accounting report that we are going over the limit and bo shrinker
> >failed to free any memory to make room.
> >
> >The handling of dma32 zone is done as post pass of ttm memory accounting.
> 
> OK. I'll take a deeper look into this.
> 
> >Regarding the pagefault comment i removed, it doesn't make sense anymore
> >because now we populate the whole page table in one shot. So there is
> >no more prefaulting few pages but a full prefaulting. Thought i can
> >add a comment stating that if you like.
> It's important that we distinguish between populating, which
> populates pages,
> and faulting, which add ptes pointing to those pages.
> 
> Previously populating happened as a side effect of faulting, but now
> that populating is done
> in a single step, faulting (adding ptes) is still not. Usually a
> fault() handler adds a single pte,
> but TTM is different and tries to prefault more, but it is important
> that we only error on the first
> pte, so that's why the comment should stay.
> 

Well yes it only fill numprefault pte, but no error can happen in the
prefault loop except for vm_insert_mixed failure, it's why i kept the
report error only if it fails on the first page. I actually did a full
pte populate at one point while working on that, dunno if that would
make sense.

> >For the ttm_tt_dma struct to hold page allocator specific informations
> >i think it can be done as an followup patch but if you prefer to have
> >that in this patchset let me know i will respin with such changes.
> >
> 
> I'm fine with having it as a separate patchset as long as it gets done :).
> 

I will spin a patch for that on top of the patchset.

Cheers,
Jerome


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator

2011-11-09 Thread j.gli...@gmail.com
So i did an overhaul of ttm_memory, i believe the simplification i did
make sense. See patch 5 for a longer explanation.

Thomas with the ttm_memory change the allocation of pages won't happen
if the accounting report that we are going over the limit and bo shrinker
failed to free any memory to make room.

The handling of dma32 zone is done as post pass of ttm memory accounting.

Regarding the pagefault comment i removed, it doesn't make sense anymore
because now we populate the whole page table in one shot. So there is
no more prefaulting few pages but a full prefaulting. Thought i can
add a comment stating that if you like.

For the ttm_tt_dma struct to hold page allocator specific informations
i think it can be done as an followup patch but if you prefer to have
that in this patchset let me know i will respin with such changes.

I am in the process of retesting this whole serie and especialy the
while memory accounting.

Cheers,
Jerome


ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator

2011-11-09 Thread j . glisse
So i did an overhaul of ttm_memory, i believe the simplification i did
make sense. See patch 5 for a longer explanation.

Thomas with the ttm_memory change the allocation of pages won't happen
if the accounting report that we are going over the limit and bo shrinker
failed to free any memory to make room.

The handling of dma32 zone is done as post pass of ttm memory accounting.

Regarding the pagefault comment i removed, it doesn't make sense anymore
because now we populate the whole page table in one shot. So there is
no more prefaulting few pages but a full prefaulting. Thought i can
add a comment stating that if you like.

For the ttm_tt_dma struct to hold page allocator specific informations
i think it can be done as an followup patch but if you prefer to have
that in this patchset let me know i will respin with such changes.

I am in the process of retesting this whole serie and especialy the
while memory accounting.

Cheers,
Jerome
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


Re: ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator

2011-11-09 Thread Thomas Hellstrom

On 11/09/2011 09:22 PM, j.gli...@gmail.com wrote:

So i did an overhaul of ttm_memory, i believe the simplification i did
make sense. See patch 5 for a longer explanation.

   
Thomas with the ttm_memory change the allocation of pages won't happen

if the accounting report that we are going over the limit and bo shrinker
failed to free any memory to make room.

The handling of dma32 zone is done as post pass of ttm memory accounting.
   


OK. I'll take a deeper look into this.


Regarding the pagefault comment i removed, it doesn't make sense anymore
because now we populate the whole page table in one shot. So there is
no more prefaulting few pages but a full prefaulting. Thought i can
add a comment stating that if you like.
   


It's important that we distinguish between populating, which populates 
pages,

and faulting, which add ptes pointing to those pages.

Previously populating happened as a side effect of faulting, but now 
that populating is done
in a single step, faulting (adding ptes) is still not. Usually a fault() 
handler adds a single pte,
but TTM is different and tries to prefault more, but it is important 
that we only error on the first

pte, so that's why the comment should stay.


For the ttm_tt_dma struct to hold page allocator specific informations
i think it can be done as an followup patch but if you prefer to have
that in this patchset let me know i will respin with such changes.

   


I'm fine with having it as a separate patchset as long as it gets done :).



I am in the process of retesting this whole serie and especialy the
while memory accounting.

Cheers,
Jerome
   


/Thomas

___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


Re: ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator

2011-11-09 Thread Jerome Glisse
On Wed, Nov 09, 2011 at 09:25:20PM +0100, Thomas Hellstrom wrote:
 On 11/09/2011 09:22 PM, j.gli...@gmail.com wrote:
 So i did an overhaul of ttm_memory, i believe the simplification i did
 make sense. See patch 5 for a longer explanation.
 
 Thomas with the ttm_memory change the allocation of pages won't happen
 if the accounting report that we are going over the limit and bo shrinker
 failed to free any memory to make room.
 
 The handling of dma32 zone is done as post pass of ttm memory accounting.
 
 OK. I'll take a deeper look into this.
 
 Regarding the pagefault comment i removed, it doesn't make sense anymore
 because now we populate the whole page table in one shot. So there is
 no more prefaulting few pages but a full prefaulting. Thought i can
 add a comment stating that if you like.
 It's important that we distinguish between populating, which
 populates pages,
 and faulting, which add ptes pointing to those pages.
 
 Previously populating happened as a side effect of faulting, but now
 that populating is done
 in a single step, faulting (adding ptes) is still not. Usually a
 fault() handler adds a single pte,
 but TTM is different and tries to prefault more, but it is important
 that we only error on the first
 pte, so that's why the comment should stay.
 

Well yes it only fill numprefault pte, but no error can happen in the
prefault loop except for vm_insert_mixed failure, it's why i kept the
report error only if it fails on the first page. I actually did a full
pte populate at one point while working on that, dunno if that would
make sense.

 For the ttm_tt_dma struct to hold page allocator specific informations
 i think it can be done as an followup patch but if you prefer to have
 that in this patchset let me know i will respin with such changes.
 
 
 I'm fine with having it as a separate patchset as long as it gets done :).
 

I will spin a patch for that on top of the patchset.

Cheers,
Jerome
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator [FULL]

2011-11-08 Thread Konrad Rzeszutek Wilk
On Mon, Nov 07, 2011 at 06:40:20PM -0500, j.glisse at gmail.com wrote:
> Ok so here is full patchset, including nouveau support, Ben if you
> could review (if change to nouveau in patch 7 are correct then others
> change to nouveau are more than likely 100% correct :)).

You are missing one patch (thought I think the 'dma_bits = 32' part can easily
be moved to the "drm/radeon/kms: Enable the TTM DMA pool if swiotlb is on" 
patch.




ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator [FULL]

2011-11-07 Thread j.gli...@gmail.com
Ok so here is full patchset, including nouveau support, Ben if you
could review (if change to nouveau in patch 7 are correct then others
change to nouveau are more than likely 100% correct :)).

So been tested on R7XX,EVERGREEN,CAICOS,CAYMAN with SWIOTLB. Also
tested on NV50. I still need to test AGP configuration but i am quite
confident that there is no regression for PCIE/PCI.

Cheers,
Jerome



ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator

2011-11-07 Thread Konrad Rzeszutek Wilk
On Thu, Nov 03, 2011 at 07:39:04PM -0400, j.glisse at gmail.com wrote:
> Hi,
> 
> So updated patchset, only patch 5 seen change since last set.
> Last 3 patch are from your patchset, modified on top of mine.

Yup, and I've tested it on my radeon box. Going to test it
on the nvidia and some of the 32-bit troublesome cards.

Will need a patch to enable the TTM DMA under the nouveau
driver too, but I can spin that up.

> 
> Konrad so i added you dma pool allocator on top of that
> and added support for it to radeon. All in all it's slightly
> smaller than your patchset.

It is! Thank you.
> 
> Biggest change is use of a list_head in ttm_tt to keep the
> dma_page list inside the ttm_tt object allowing faster and
> lot simpler deallocation of page.

Yup.
> 
> I only briefly test this code, it seems ok so far. Did you
> tested booting kernel with swiotlb=force and with your patchset ?
> Because here it doesn't work. I still don't understand why
> swiotlb want to create a bounce page when the page supplied
> fit the constraint. Need to dig into kernel history to see if
> there is any good reasons for that.

I think you found out why.. the force, does force every page.

> 
> Otherwise i believe this whole patchset make things cleaner
> and simpler for ttm.

 So do I.

Let me run it throught some of the machines this week to get
that extra comfortable feeling.

> 
> Cheers,
> Jerome Glisse


Re: ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator

2011-11-07 Thread Konrad Rzeszutek Wilk
On Thu, Nov 03, 2011 at 07:39:04PM -0400, j.gli...@gmail.com wrote:
 Hi,
 
 So updated patchset, only patch 5 seen change since last set.
 Last 3 patch are from your patchset, modified on top of mine.

Yup, and I've tested it on my radeon box. Going to test it
on the nvidia and some of the 32-bit troublesome cards.

Will need a patch to enable the TTM DMA under the nouveau
driver too, but I can spin that up.

 
 Konrad so i added you dma pool allocator on top of that
 and added support for it to radeon. All in all it's slightly
 smaller than your patchset.

It is! Thank you.
 
 Biggest change is use of a list_head in ttm_tt to keep the
 dma_page list inside the ttm_tt object allowing faster and
 lot simpler deallocation of page.

Yup.
 
 I only briefly test this code, it seems ok so far. Did you
 tested booting kernel with swiotlb=force and with your patchset ?
 Because here it doesn't work. I still don't understand why
 swiotlb want to create a bounce page when the page supplied
 fit the constraint. Need to dig into kernel history to see if
 there is any good reasons for that.

I think you found out why.. the force, does force every page.

 
 Otherwise i believe this whole patchset make things cleaner
 and simpler for ttm.

nods So do I.

Let me run it throught some of the machines this week to get
that extra comfortable feeling.

 
 Cheers,
 Jerome Glisse
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator [FULL]

2011-11-07 Thread j . glisse
Ok so here is full patchset, including nouveau support, Ben if you
could review (if change to nouveau in patch 7 are correct then others
change to nouveau are more than likely 100% correct :)).

So been tested on R7XX,EVERGREEN,CAICOS,CAYMAN with SWIOTLB. Also
tested on NV50. I still need to test AGP configuration but i am quite
confident that there is no regression for PCIE/PCI.

Cheers,
Jerome

___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel


ttm: merge ttm_backend & ttm_tt, introduce ttm dma allocator

2011-11-03 Thread j.gli...@gmail.com
Hi,

So updated patchset, only patch 5 seen change since last set.
Last 3 patch are from your patchset, modified on top of mine.

Konrad so i added you dma pool allocator on top of that
and added support for it to radeon. All in all it's slightly
smaller than your patchset.

Biggest change is use of a list_head in ttm_tt to keep the
dma_page list inside the ttm_tt object allowing faster and
lot simpler deallocation of page.

I only briefly test this code, it seems ok so far. Did you
tested booting kernel with swiotlb=force and with your patchset ?
Because here it doesn't work. I still don't understand why
swiotlb want to create a bounce page when the page supplied
fit the constraint. Need to dig into kernel history to see if
there is any good reasons for that.

Otherwise i believe this whole patchset make things cleaner
and simpler for ttm.

Cheers,
Jerome Glisse



ttm: merge ttm_backend ttm_tt, introduce ttm dma allocator

2011-11-03 Thread j . glisse
Hi,

So updated patchset, only patch 5 seen change since last set.
Last 3 patch are from your patchset, modified on top of mine.

Konrad so i added you dma pool allocator on top of that
and added support for it to radeon. All in all it's slightly
smaller than your patchset.

Biggest change is use of a list_head in ttm_tt to keep the
dma_page list inside the ttm_tt object allowing faster and
lot simpler deallocation of page.

I only briefly test this code, it seems ok so far. Did you
tested booting kernel with swiotlb=force and with your patchset ?
Because here it doesn't work. I still don't understand why
swiotlb want to create a bounce page when the page supplied
fit the constraint. Need to dig into kernel history to see if
there is any good reasons for that.

Otherwise i believe this whole patchset make things cleaner
and simpler for ttm.

Cheers,
Jerome Glisse

___
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel