Re: [PATCH v10 0/3] Secure Memory Allocation Framework

[John Einar Reitan]

On Fri, Oct 07, 2016 at 10:42:17AM -0400, Rob Clark wrote:
> probably should keep the discussion on github (USAGE.md was updated a
> bit more and merged into https://github.com/cubanismo/allocator so
> look there for the latest)..
> 
> but briefly:
> 
> 1) my expectation is if the user is implementing some use-case, it
> knows what devices and APIs are involved, otherwise it wouldn't be
> able to pass a buffer to that device/API..

As I described at Linaro Connect late-connected devices could cause new
constrains to appear. I.e. some (additonal) HDMI connection or WiFi Display etc.
Including all the might-happen devices might lead to unoptimal buffers
just to be able to handle some rarely-happen events.

I guess the easy resolve here is for the user to do a reallocation with
the new constraints added and replace the buffer(s) in question, but
with a slight lag in enabling the new device.

John
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v10 0/3] Secure Memory Allocation Framework

[Rob Clark]

probably should keep the discussion on github (USAGE.md was updated a
bit more and merged into https://github.com/cubanismo/allocator so
look there for the latest)..

but briefly:

1) my expectation is if the user is implementing some use-case, it
knows what devices and APIs are involved, otherwise it wouldn't be
able to pass a buffer to that device/API..

2) if assertion/usage/devices haven't changed, you can re-use the
merged caps for however many buffer allocations

BR,
-R

On Fri, Oct 7, 2016 at 9:54 AM, Benjamin Gaignard
 wrote:
> Rob,
>
> how do you know which devices are concerned when listing the constraints ?
> Does combine_capabilities is done from each allocation or can it be cached ?
>
> Regards,
> Benjmain
>
> 2016-10-06 18:54 GMT+02:00 Rob Clark :
>> so there is discussion about a "central userspace allocator" (ie. more
>> like a common userspace API that could be implemented on top of
>> various devices/APIs) to decide in a generic way which device could
>> allocate.
>>
>>   https://github.com/cubanismo/allocator
>>
>> and I wrote up some rough thoughts/proposal about how the usage might
>> look.. just rough, so don't try to compile it or anything, and not
>> consensus yet so it will probably change/evolve..
>>
>>   https://github.com/robclark/allocator/blob/master/USAGE.md
>>
>> I think ion could be just another device to share buffers with, which
>> happens to not impose any specific constraints.  How "liballoc-ion.so"
>> backend figures out how to map constraints/usage to a heap is a bit
>> hand-wavey at the moment.
>>
>> BR,
>> -R
>>
>> On Wed, Oct 5, 2016 at 9:40 AM, Benjamin Gaignard
>>  wrote:
>>> because with ion it is up to userland to decide which heap to use
>>> and until now userland doesn't have any way to get device constraints...
>>>
>>> I will prefer let a central allocator (in kernel) decide from the
>>> attached devices
>>> which allocator is the best. It is what I have implemented in smaf.
>>>
>>> Benjamin
>>>
>>>
>>> 2016-10-05 15:19 GMT+02:00 Daniel Vetter :
 On Tue, Oct 04, 2016 at 01:47:21PM +0200, Benjamin Gaignard wrote:
> version 10 changes:
>  - rebased on kernel 4.8 tag
>  - minor typo fix
>
> version 9 changes:
>  - rebased on 4.8-rc5
>  - struct dma_attrs doesn't exist anymore so update CMA allocator
>to compile with new dma_*_attr functions
>  - add example SMAF use case in cover letter
>
> version 8 changes:
>  - rework of the structures used within ioctl
>by adding a version field and padding to be futur proof
>  - rename fake secure moduel to test secure module
>  - fix the various remarks done on the previous patcheset
>
> version 7 changes:
>  - rebased on kernel 4.6-rc7
>  - simplify secure module API
>  - add vma ops to be able to detect mmap/munmap calls
>  - add ioctl to get number and allocator names
>  - update libsmaf with adding tests
>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
>  - add debug log in fake secure module
>
> version 6 changes:
>  - rebased on kernel 4.5-rc4
>  - fix mmapping bug while requested allocation size isn't a a multiple of
>PAGE_SIZE (add a test for this in libsmaf)
>
> version 5 changes:
>  - rebased on kernel 4.3-rc6
>  - rework locking schema and make handle status use an atomic_t
>  - add a fake secure module to allow performing tests without trusted
>environment
>
> version 4 changes:
>  - rebased on kernel 4.3-rc3
>  - fix missing EXPORT_SYMBOL for smaf_create_handle()
>
> version 3 changes:
>  - Remove ioctl for allocator selection instead provide the name of
>the targeted allocator with allocation request.
>Selecting allocator from userland isn't the prefered way of working
>but is needed when the first user of the buffer is a software 
> component.
>  - Fix issues in case of error while creating smaf handle.
>  - Fix module license.
>  - Update libsmaf and tests to care of the SMAF API evolution
>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
>
> version 2 changes:
>  - Add one ioctl to allow allocator selection from userspace.
>This is required for the uses case where the first user of
>the buffer is a software IP which can't perform dma_buf attachement.
>  - Add name and ranking to allocator structure to be able to sort them.
>  - Create a tiny library to test SMAF:
>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
>  - Fix one issue when try to secure buffer without secure module 
> registered
>
> SMAF aim to solve two problems: allocating memory that fit with hardware 
> IPs
> constraints and secure those data from bus point of view.
>
> One example of SMAF usage is 

Re: [PATCH v10 0/3] Secure Memory Allocation Framework

[Benjamin Gaignard]

Rob,

how do you know which devices are concerned when listing the constraints ?
Does combine_capabilities is done from each allocation or can it be cached ?

Regards,
Benjmain

2016-10-06 18:54 GMT+02:00 Rob Clark :
> so there is discussion about a "central userspace allocator" (ie. more
> like a common userspace API that could be implemented on top of
> various devices/APIs) to decide in a generic way which device could
> allocate.
>
>   https://github.com/cubanismo/allocator
>
> and I wrote up some rough thoughts/proposal about how the usage might
> look.. just rough, so don't try to compile it or anything, and not
> consensus yet so it will probably change/evolve..
>
>   https://github.com/robclark/allocator/blob/master/USAGE.md
>
> I think ion could be just another device to share buffers with, which
> happens to not impose any specific constraints.  How "liballoc-ion.so"
> backend figures out how to map constraints/usage to a heap is a bit
> hand-wavey at the moment.
>
> BR,
> -R
>
> On Wed, Oct 5, 2016 at 9:40 AM, Benjamin Gaignard
>  wrote:
>> because with ion it is up to userland to decide which heap to use
>> and until now userland doesn't have any way to get device constraints...
>>
>> I will prefer let a central allocator (in kernel) decide from the
>> attached devices
>> which allocator is the best. It is what I have implemented in smaf.
>>
>> Benjamin
>>
>>
>> 2016-10-05 15:19 GMT+02:00 Daniel Vetter :
>>> On Tue, Oct 04, 2016 at 01:47:21PM +0200, Benjamin Gaignard wrote:
 version 10 changes:
  - rebased on kernel 4.8 tag
  - minor typo fix

 version 9 changes:
  - rebased on 4.8-rc5
  - struct dma_attrs doesn't exist anymore so update CMA allocator
to compile with new dma_*_attr functions
  - add example SMAF use case in cover letter

 version 8 changes:
  - rework of the structures used within ioctl
by adding a version field and padding to be futur proof
  - rename fake secure moduel to test secure module
  - fix the various remarks done on the previous patcheset

 version 7 changes:
  - rebased on kernel 4.6-rc7
  - simplify secure module API
  - add vma ops to be able to detect mmap/munmap calls
  - add ioctl to get number and allocator names
  - update libsmaf with adding tests
https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
  - add debug log in fake secure module

 version 6 changes:
  - rebased on kernel 4.5-rc4
  - fix mmapping bug while requested allocation size isn't a a multiple of
PAGE_SIZE (add a test for this in libsmaf)

 version 5 changes:
  - rebased on kernel 4.3-rc6
  - rework locking schema and make handle status use an atomic_t
  - add a fake secure module to allow performing tests without trusted
environment

 version 4 changes:
  - rebased on kernel 4.3-rc3
  - fix missing EXPORT_SYMBOL for smaf_create_handle()

 version 3 changes:
  - Remove ioctl for allocator selection instead provide the name of
the targeted allocator with allocation request.
Selecting allocator from userland isn't the prefered way of working
but is needed when the first user of the buffer is a software component.
  - Fix issues in case of error while creating smaf handle.
  - Fix module license.
  - Update libsmaf and tests to care of the SMAF API evolution
https://git.linaro.org/people/benjamin.gaignard/libsmaf.git

 version 2 changes:
  - Add one ioctl to allow allocator selection from userspace.
This is required for the uses case where the first user of
the buffer is a software IP which can't perform dma_buf attachement.
  - Add name and ranking to allocator structure to be able to sort them.
  - Create a tiny library to test SMAF:
https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
  - Fix one issue when try to secure buffer without secure module registered

 SMAF aim to solve two problems: allocating memory that fit with hardware 
 IPs
 constraints and secure those data from bus point of view.

 One example of SMAF usage is camera preview: on SoC you may use either an 
 USB
 webcam or the built-in camera interface and the frames could be send 
 directly
 to the dipslay Ip or handle by GPU.
 Most of USB interfaces and GPU have mmu but almost all built-in camera
 interace and display Ips don't have mmu so when selecting how allocate
 buffer you need to be aware of each devices constraints (contiguous memroy,
 stride, boundary, alignment ...).
 ION has solve this problem by let userland decide which allocator (heap) 
 to use
 but this require to adapt userland for each platform and sometime for each
 use case.

 To be sure to select the best allocation 

Re: [PATCH v10 0/3] Secure Memory Allocation Framework

[Rob Clark]

so there is discussion about a "central userspace allocator" (ie. more
like a common userspace API that could be implemented on top of
various devices/APIs) to decide in a generic way which device could
allocate.

  https://github.com/cubanismo/allocator

and I wrote up some rough thoughts/proposal about how the usage might
look.. just rough, so don't try to compile it or anything, and not
consensus yet so it will probably change/evolve..

  https://github.com/robclark/allocator/blob/master/USAGE.md

I think ion could be just another device to share buffers with, which
happens to not impose any specific constraints.  How "liballoc-ion.so"
backend figures out how to map constraints/usage to a heap is a bit
hand-wavey at the moment.

BR,
-R

On Wed, Oct 5, 2016 at 9:40 AM, Benjamin Gaignard
 wrote:
> because with ion it is up to userland to decide which heap to use
> and until now userland doesn't have any way to get device constraints...
>
> I will prefer let a central allocator (in kernel) decide from the
> attached devices
> which allocator is the best. It is what I have implemented in smaf.
>
> Benjamin
>
>
> 2016-10-05 15:19 GMT+02:00 Daniel Vetter :
>> On Tue, Oct 04, 2016 at 01:47:21PM +0200, Benjamin Gaignard wrote:
>>> version 10 changes:
>>>  - rebased on kernel 4.8 tag
>>>  - minor typo fix
>>>
>>> version 9 changes:
>>>  - rebased on 4.8-rc5
>>>  - struct dma_attrs doesn't exist anymore so update CMA allocator
>>>to compile with new dma_*_attr functions
>>>  - add example SMAF use case in cover letter
>>>
>>> version 8 changes:
>>>  - rework of the structures used within ioctl
>>>by adding a version field and padding to be futur proof
>>>  - rename fake secure moduel to test secure module
>>>  - fix the various remarks done on the previous patcheset
>>>
>>> version 7 changes:
>>>  - rebased on kernel 4.6-rc7
>>>  - simplify secure module API
>>>  - add vma ops to be able to detect mmap/munmap calls
>>>  - add ioctl to get number and allocator names
>>>  - update libsmaf with adding tests
>>>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
>>>  - add debug log in fake secure module
>>>
>>> version 6 changes:
>>>  - rebased on kernel 4.5-rc4
>>>  - fix mmapping bug while requested allocation size isn't a a multiple of
>>>PAGE_SIZE (add a test for this in libsmaf)
>>>
>>> version 5 changes:
>>>  - rebased on kernel 4.3-rc6
>>>  - rework locking schema and make handle status use an atomic_t
>>>  - add a fake secure module to allow performing tests without trusted
>>>environment
>>>
>>> version 4 changes:
>>>  - rebased on kernel 4.3-rc3
>>>  - fix missing EXPORT_SYMBOL for smaf_create_handle()
>>>
>>> version 3 changes:
>>>  - Remove ioctl for allocator selection instead provide the name of
>>>the targeted allocator with allocation request.
>>>Selecting allocator from userland isn't the prefered way of working
>>>but is needed when the first user of the buffer is a software component.
>>>  - Fix issues in case of error while creating smaf handle.
>>>  - Fix module license.
>>>  - Update libsmaf and tests to care of the SMAF API evolution
>>>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
>>>
>>> version 2 changes:
>>>  - Add one ioctl to allow allocator selection from userspace.
>>>This is required for the uses case where the first user of
>>>the buffer is a software IP which can't perform dma_buf attachement.
>>>  - Add name and ranking to allocator structure to be able to sort them.
>>>  - Create a tiny library to test SMAF:
>>>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
>>>  - Fix one issue when try to secure buffer without secure module registered
>>>
>>> SMAF aim to solve two problems: allocating memory that fit with hardware IPs
>>> constraints and secure those data from bus point of view.
>>>
>>> One example of SMAF usage is camera preview: on SoC you may use either an 
>>> USB
>>> webcam or the built-in camera interface and the frames could be send 
>>> directly
>>> to the dipslay Ip or handle by GPU.
>>> Most of USB interfaces and GPU have mmu but almost all built-in camera
>>> interace and display Ips don't have mmu so when selecting how allocate
>>> buffer you need to be aware of each devices constraints (contiguous memroy,
>>> stride, boundary, alignment ...).
>>> ION has solve this problem by let userland decide which allocator (heap) to 
>>> use
>>> but this require to adapt userland for each platform and sometime for each
>>> use case.
>>>
>>> To be sure to select the best allocation method for devices SMAF implement
>>> deferred allocation mechanism: memory allocation is only done when the first
>>> device effectively required it.
>>> Allocator modules have to implement a match() to let SMAF know if they are
>>> compatibles with devices needs.
>>> This patch set provide an example of allocator module which use
>>> dma_{alloc/free/mmap}_attrs() and 

Re: [PATCH v10 0/3] Secure Memory Allocation Framework

[Benjamin Gaignard]

When using dmabuf the devices have to attach themselves on the buffer
before map it
so you can know which devices will use the buffer before allocate it
when the first
dma_buf_map_attachment() is called (defered allocation)
Split dmabuf attach/map_attachment is not really done in drm or v4l2
but patch like this one
are going in this direction:
http://www.spinics.net/lists/linux-media/msg104480.html

I agree that device constraint enumeration is still missing, for smaf
I have just to test
by using device dma_coherent_mask field to check if cma allocator can
be used for
allocation. I guess that if you can export more device information to
userland we can
get access to them in kernel too.


2016-10-05 15:43 GMT+02:00 Daniel Vetter :
> On Wed, Oct 05, 2016 at 03:40:14PM +0200, Benjamin Gaignard wrote:
>> because with ion it is up to userland to decide which heap to use
>> and until now userland doesn't have any way to get device constraints...
>>
>> I will prefer let a central allocator (in kernel) decide from the
>> attached devices
>> which allocator is the best. It is what I have implemented in smaf.
>
> And how does that work? Atm there's no interfaces at all in the kernel to
> allocate a buffer suitable for 2 devices at the same time. Seems
> incomplete if this is the direction you want to go to. Also, it's against
> the direction we all discussed ad XDC, where the clear consensus was to
> have most of that haggling in userspace (with the kernel exporting a
> little bit more information to userspace than what it does now).
> -Daniel
>
>>
>> Benjamin
>>
>>
>> 2016-10-05 15:19 GMT+02:00 Daniel Vetter :
>> > On Tue, Oct 04, 2016 at 01:47:21PM +0200, Benjamin Gaignard wrote:
>> >> version 10 changes:
>> >>  - rebased on kernel 4.8 tag
>> >>  - minor typo fix
>> >>
>> >> version 9 changes:
>> >>  - rebased on 4.8-rc5
>> >>  - struct dma_attrs doesn't exist anymore so update CMA allocator
>> >>to compile with new dma_*_attr functions
>> >>  - add example SMAF use case in cover letter
>> >>
>> >> version 8 changes:
>> >>  - rework of the structures used within ioctl
>> >>by adding a version field and padding to be futur proof
>> >>  - rename fake secure moduel to test secure module
>> >>  - fix the various remarks done on the previous patcheset
>> >>
>> >> version 7 changes:
>> >>  - rebased on kernel 4.6-rc7
>> >>  - simplify secure module API
>> >>  - add vma ops to be able to detect mmap/munmap calls
>> >>  - add ioctl to get number and allocator names
>> >>  - update libsmaf with adding tests
>> >>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
>> >>  - add debug log in fake secure module
>> >>
>> >> version 6 changes:
>> >>  - rebased on kernel 4.5-rc4
>> >>  - fix mmapping bug while requested allocation size isn't a a multiple of
>> >>PAGE_SIZE (add a test for this in libsmaf)
>> >>
>> >> version 5 changes:
>> >>  - rebased on kernel 4.3-rc6
>> >>  - rework locking schema and make handle status use an atomic_t
>> >>  - add a fake secure module to allow performing tests without trusted
>> >>environment
>> >>
>> >> version 4 changes:
>> >>  - rebased on kernel 4.3-rc3
>> >>  - fix missing EXPORT_SYMBOL for smaf_create_handle()
>> >>
>> >> version 3 changes:
>> >>  - Remove ioctl for allocator selection instead provide the name of
>> >>the targeted allocator with allocation request.
>> >>Selecting allocator from userland isn't the prefered way of working
>> >>but is needed when the first user of the buffer is a software 
>> >> component.
>> >>  - Fix issues in case of error while creating smaf handle.
>> >>  - Fix module license.
>> >>  - Update libsmaf and tests to care of the SMAF API evolution
>> >>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
>> >>
>> >> version 2 changes:
>> >>  - Add one ioctl to allow allocator selection from userspace.
>> >>This is required for the uses case where the first user of
>> >>the buffer is a software IP which can't perform dma_buf attachement.
>> >>  - Add name and ranking to allocator structure to be able to sort them.
>> >>  - Create a tiny library to test SMAF:
>> >>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
>> >>  - Fix one issue when try to secure buffer without secure module 
>> >> registered
>> >>
>> >> SMAF aim to solve two problems: allocating memory that fit with hardware 
>> >> IPs
>> >> constraints and secure those data from bus point of view.
>> >>
>> >> One example of SMAF usage is camera preview: on SoC you may use either an 
>> >> USB
>> >> webcam or the built-in camera interface and the frames could be send 
>> >> directly
>> >> to the dipslay Ip or handle by GPU.
>> >> Most of USB interfaces and GPU have mmu but almost all built-in camera
>> >> interace and display Ips don't have mmu so when selecting how allocate
>> >> buffer you need to be aware of each devices constraints (contiguous 
>> >> memroy,
>> >> stride, boundary, 

Re: [PATCH v10 0/3] Secure Memory Allocation Framework

[Daniel Vetter]

On Wed, Oct 05, 2016 at 03:40:14PM +0200, Benjamin Gaignard wrote:
> because with ion it is up to userland to decide which heap to use
> and until now userland doesn't have any way to get device constraints...
> 
> I will prefer let a central allocator (in kernel) decide from the
> attached devices
> which allocator is the best. It is what I have implemented in smaf.

And how does that work? Atm there's no interfaces at all in the kernel to
allocate a buffer suitable for 2 devices at the same time. Seems
incomplete if this is the direction you want to go to. Also, it's against
the direction we all discussed ad XDC, where the clear consensus was to
have most of that haggling in userspace (with the kernel exporting a
little bit more information to userspace than what it does now).
-Daniel

> 
> Benjamin
> 
> 
> 2016-10-05 15:19 GMT+02:00 Daniel Vetter :
> > On Tue, Oct 04, 2016 at 01:47:21PM +0200, Benjamin Gaignard wrote:
> >> version 10 changes:
> >>  - rebased on kernel 4.8 tag
> >>  - minor typo fix
> >>
> >> version 9 changes:
> >>  - rebased on 4.8-rc5
> >>  - struct dma_attrs doesn't exist anymore so update CMA allocator
> >>to compile with new dma_*_attr functions
> >>  - add example SMAF use case in cover letter
> >>
> >> version 8 changes:
> >>  - rework of the structures used within ioctl
> >>by adding a version field and padding to be futur proof
> >>  - rename fake secure moduel to test secure module
> >>  - fix the various remarks done on the previous patcheset
> >>
> >> version 7 changes:
> >>  - rebased on kernel 4.6-rc7
> >>  - simplify secure module API
> >>  - add vma ops to be able to detect mmap/munmap calls
> >>  - add ioctl to get number and allocator names
> >>  - update libsmaf with adding tests
> >>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
> >>  - add debug log in fake secure module
> >>
> >> version 6 changes:
> >>  - rebased on kernel 4.5-rc4
> >>  - fix mmapping bug while requested allocation size isn't a a multiple of
> >>PAGE_SIZE (add a test for this in libsmaf)
> >>
> >> version 5 changes:
> >>  - rebased on kernel 4.3-rc6
> >>  - rework locking schema and make handle status use an atomic_t
> >>  - add a fake secure module to allow performing tests without trusted
> >>environment
> >>
> >> version 4 changes:
> >>  - rebased on kernel 4.3-rc3
> >>  - fix missing EXPORT_SYMBOL for smaf_create_handle()
> >>
> >> version 3 changes:
> >>  - Remove ioctl for allocator selection instead provide the name of
> >>the targeted allocator with allocation request.
> >>Selecting allocator from userland isn't the prefered way of working
> >>but is needed when the first user of the buffer is a software component.
> >>  - Fix issues in case of error while creating smaf handle.
> >>  - Fix module license.
> >>  - Update libsmaf and tests to care of the SMAF API evolution
> >>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
> >>
> >> version 2 changes:
> >>  - Add one ioctl to allow allocator selection from userspace.
> >>This is required for the uses case where the first user of
> >>the buffer is a software IP which can't perform dma_buf attachement.
> >>  - Add name and ranking to allocator structure to be able to sort them.
> >>  - Create a tiny library to test SMAF:
> >>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
> >>  - Fix one issue when try to secure buffer without secure module registered
> >>
> >> SMAF aim to solve two problems: allocating memory that fit with hardware 
> >> IPs
> >> constraints and secure those data from bus point of view.
> >>
> >> One example of SMAF usage is camera preview: on SoC you may use either an 
> >> USB
> >> webcam or the built-in camera interface and the frames could be send 
> >> directly
> >> to the dipslay Ip or handle by GPU.
> >> Most of USB interfaces and GPU have mmu but almost all built-in camera
> >> interace and display Ips don't have mmu so when selecting how allocate
> >> buffer you need to be aware of each devices constraints (contiguous memroy,
> >> stride, boundary, alignment ...).
> >> ION has solve this problem by let userland decide which allocator (heap) 
> >> to use
> >> but this require to adapt userland for each platform and sometime for each
> >> use case.
> >>
> >> To be sure to select the best allocation method for devices SMAF implement
> >> deferred allocation mechanism: memory allocation is only done when the 
> >> first
> >> device effectively required it.
> >> Allocator modules have to implement a match() to let SMAF know if they are
> >> compatibles with devices needs.
> >> This patch set provide an example of allocator module which use
> >> dma_{alloc/free/mmap}_attrs() and check if at least one device have
> >> coherent_dma_mask set to DMA_BIT_MASK(32) in match function.
> >>
> >> In the same camera preview use case, SMAF allow to protect the data from 
> >> being
> >> read by unauthorized IPs (i.e. a malware to 

Re: [PATCH v10 0/3] Secure Memory Allocation Framework

[Benjamin Gaignard]

because with ion it is up to userland to decide which heap to use
and until now userland doesn't have any way to get device constraints...

I will prefer let a central allocator (in kernel) decide from the
attached devices
which allocator is the best. It is what I have implemented in smaf.

Benjamin


2016-10-05 15:19 GMT+02:00 Daniel Vetter :
> On Tue, Oct 04, 2016 at 01:47:21PM +0200, Benjamin Gaignard wrote:
>> version 10 changes:
>>  - rebased on kernel 4.8 tag
>>  - minor typo fix
>>
>> version 9 changes:
>>  - rebased on 4.8-rc5
>>  - struct dma_attrs doesn't exist anymore so update CMA allocator
>>to compile with new dma_*_attr functions
>>  - add example SMAF use case in cover letter
>>
>> version 8 changes:
>>  - rework of the structures used within ioctl
>>by adding a version field and padding to be futur proof
>>  - rename fake secure moduel to test secure module
>>  - fix the various remarks done on the previous patcheset
>>
>> version 7 changes:
>>  - rebased on kernel 4.6-rc7
>>  - simplify secure module API
>>  - add vma ops to be able to detect mmap/munmap calls
>>  - add ioctl to get number and allocator names
>>  - update libsmaf with adding tests
>>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
>>  - add debug log in fake secure module
>>
>> version 6 changes:
>>  - rebased on kernel 4.5-rc4
>>  - fix mmapping bug while requested allocation size isn't a a multiple of
>>PAGE_SIZE (add a test for this in libsmaf)
>>
>> version 5 changes:
>>  - rebased on kernel 4.3-rc6
>>  - rework locking schema and make handle status use an atomic_t
>>  - add a fake secure module to allow performing tests without trusted
>>environment
>>
>> version 4 changes:
>>  - rebased on kernel 4.3-rc3
>>  - fix missing EXPORT_SYMBOL for smaf_create_handle()
>>
>> version 3 changes:
>>  - Remove ioctl for allocator selection instead provide the name of
>>the targeted allocator with allocation request.
>>Selecting allocator from userland isn't the prefered way of working
>>but is needed when the first user of the buffer is a software component.
>>  - Fix issues in case of error while creating smaf handle.
>>  - Fix module license.
>>  - Update libsmaf and tests to care of the SMAF API evolution
>>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
>>
>> version 2 changes:
>>  - Add one ioctl to allow allocator selection from userspace.
>>This is required for the uses case where the first user of
>>the buffer is a software IP which can't perform dma_buf attachement.
>>  - Add name and ranking to allocator structure to be able to sort them.
>>  - Create a tiny library to test SMAF:
>>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
>>  - Fix one issue when try to secure buffer without secure module registered
>>
>> SMAF aim to solve two problems: allocating memory that fit with hardware IPs
>> constraints and secure those data from bus point of view.
>>
>> One example of SMAF usage is camera preview: on SoC you may use either an USB
>> webcam or the built-in camera interface and the frames could be send directly
>> to the dipslay Ip or handle by GPU.
>> Most of USB interfaces and GPU have mmu but almost all built-in camera
>> interace and display Ips don't have mmu so when selecting how allocate
>> buffer you need to be aware of each devices constraints (contiguous memroy,
>> stride, boundary, alignment ...).
>> ION has solve this problem by let userland decide which allocator (heap) to 
>> use
>> but this require to adapt userland for each platform and sometime for each
>> use case.
>>
>> To be sure to select the best allocation method for devices SMAF implement
>> deferred allocation mechanism: memory allocation is only done when the first
>> device effectively required it.
>> Allocator modules have to implement a match() to let SMAF know if they are
>> compatibles with devices needs.
>> This patch set provide an example of allocator module which use
>> dma_{alloc/free/mmap}_attrs() and check if at least one device have
>> coherent_dma_mask set to DMA_BIT_MASK(32) in match function.
>>
>> In the same camera preview use case, SMAF allow to protect the data from 
>> being
>> read by unauthorized IPs (i.e. a malware to dump camera stream).
>> Until now I have only see access rights protection at process/thread level
>> (PKeys/MPK) or on file (SELinux) but nothing allow to drive data bus 
>> firewalls.
>> SMAF propose an interface to control and implement those firewalls.
>> Like IOMMU, firewalls IPs can help to protect memory from malicious/faulty 
>> devices
>> that are attempting DMA attacks.
>>
>> Secure modules are responsibles of granting and revoking devices access 
>> rights
>> on the memory. Secure module is also called to check if CPU map memory into
>> kernel and user address spaces.
>> An example of secure module implementation can be found here:
>> http://git.linaro.org/people/benjamin.gaignard/optee-sdp.git
>> 

Re: [PATCH v10 0/3] Secure Memory Allocation Framework

[Daniel Vetter]

On Tue, Oct 04, 2016 at 01:47:21PM +0200, Benjamin Gaignard wrote:
> version 10 changes:
>  - rebased on kernel 4.8 tag
>  - minor typo fix
> 
> version 9 changes:
>  - rebased on 4.8-rc5
>  - struct dma_attrs doesn't exist anymore so update CMA allocator
>to compile with new dma_*_attr functions
>  - add example SMAF use case in cover letter
> 
> version 8 changes:
>  - rework of the structures used within ioctl
>by adding a version field and padding to be futur proof
>  - rename fake secure moduel to test secure module
>  - fix the various remarks done on the previous patcheset
> 
> version 7 changes:
>  - rebased on kernel 4.6-rc7
>  - simplify secure module API
>  - add vma ops to be able to detect mmap/munmap calls
>  - add ioctl to get number and allocator names
>  - update libsmaf with adding tests
>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
>  - add debug log in fake secure module
> 
> version 6 changes:
>  - rebased on kernel 4.5-rc4
>  - fix mmapping bug while requested allocation size isn't a a multiple of
>PAGE_SIZE (add a test for this in libsmaf)
> 
> version 5 changes:
>  - rebased on kernel 4.3-rc6
>  - rework locking schema and make handle status use an atomic_t
>  - add a fake secure module to allow performing tests without trusted
>environment
> 
> version 4 changes:
>  - rebased on kernel 4.3-rc3
>  - fix missing EXPORT_SYMBOL for smaf_create_handle()
> 
> version 3 changes:
>  - Remove ioctl for allocator selection instead provide the name of
>the targeted allocator with allocation request.
>Selecting allocator from userland isn't the prefered way of working
>but is needed when the first user of the buffer is a software component.
>  - Fix issues in case of error while creating smaf handle.
>  - Fix module license.
>  - Update libsmaf and tests to care of the SMAF API evolution
>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
> 
> version 2 changes:
>  - Add one ioctl to allow allocator selection from userspace.
>This is required for the uses case where the first user of
>the buffer is a software IP which can't perform dma_buf attachement.
>  - Add name and ranking to allocator structure to be able to sort them.
>  - Create a tiny library to test SMAF:
>https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
>  - Fix one issue when try to secure buffer without secure module registered
> 
> SMAF aim to solve two problems: allocating memory that fit with hardware IPs
> constraints and secure those data from bus point of view.
> 
> One example of SMAF usage is camera preview: on SoC you may use either an USB
> webcam or the built-in camera interface and the frames could be send directly
> to the dipslay Ip or handle by GPU.
> Most of USB interfaces and GPU have mmu but almost all built-in camera
> interace and display Ips don't have mmu so when selecting how allocate
> buffer you need to be aware of each devices constraints (contiguous memroy,
> stride, boundary, alignment ...).
> ION has solve this problem by let userland decide which allocator (heap) to 
> use
> but this require to adapt userland for each platform and sometime for each
> use case.
> 
> To be sure to select the best allocation method for devices SMAF implement
> deferred allocation mechanism: memory allocation is only done when the first
> device effectively required it.
> Allocator modules have to implement a match() to let SMAF know if they are
> compatibles with devices needs.
> This patch set provide an example of allocator module which use
> dma_{alloc/free/mmap}_attrs() and check if at least one device have
> coherent_dma_mask set to DMA_BIT_MASK(32) in match function.
> 
> In the same camera preview use case, SMAF allow to protect the data from being
> read by unauthorized IPs (i.e. a malware to dump camera stream).
> Until now I have only see access rights protection at process/thread level 
> (PKeys/MPK) or on file (SELinux) but nothing allow to drive data bus 
> firewalls.
> SMAF propose an interface to control and implement those firewalls.
> Like IOMMU, firewalls IPs can help to protect memory from malicious/faulty 
> devices
> that are attempting DMA attacks.
> 
> Secure modules are responsibles of granting and revoking devices access rights
> on the memory. Secure module is also called to check if CPU map memory into
> kernel and user address spaces.
> An example of secure module implementation can be found here:
> http://git.linaro.org/people/benjamin.gaignard/optee-sdp.git
> This code isn't yet part of the patch set because it depends on generic TEE
> which is still under discussion (https://lwn.net/Articles/644646/)
> 
> For allocation part of SMAF code I get inspirated by Sumit Semwal work about
> constraint aware allocator.

semi-random review comment, and a bit late: Why not implement smaf as a
new heap in ion? I think consensus is pretty much that we'll be stuck with
ion forever, and I think it's better to have 1 

[PATCH v10 0/3] Secure Memory Allocation Framework

[Benjamin Gaignard]

version 10 changes:
 - rebased on kernel 4.8 tag
 - minor typo fix

version 9 changes:
 - rebased on 4.8-rc5
 - struct dma_attrs doesn't exist anymore so update CMA allocator
   to compile with new dma_*_attr functions
 - add example SMAF use case in cover letter

version 8 changes:
 - rework of the structures used within ioctl
   by adding a version field and padding to be futur proof
 - rename fake secure moduel to test secure module
 - fix the various remarks done on the previous patcheset

version 7 changes:
 - rebased on kernel 4.6-rc7
 - simplify secure module API
 - add vma ops to be able to detect mmap/munmap calls
 - add ioctl to get number and allocator names
 - update libsmaf with adding tests
   https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
 - add debug log in fake secure module

version 6 changes:
 - rebased on kernel 4.5-rc4
 - fix mmapping bug while requested allocation size isn't a a multiple of
   PAGE_SIZE (add a test for this in libsmaf)

version 5 changes:
 - rebased on kernel 4.3-rc6
 - rework locking schema and make handle status use an atomic_t
 - add a fake secure module to allow performing tests without trusted
   environment

version 4 changes:
 - rebased on kernel 4.3-rc3
 - fix missing EXPORT_SYMBOL for smaf_create_handle()

version 3 changes:
 - Remove ioctl for allocator selection instead provide the name of
   the targeted allocator with allocation request.
   Selecting allocator from userland isn't the prefered way of working
   but is needed when the first user of the buffer is a software component.
 - Fix issues in case of error while creating smaf handle.
 - Fix module license.
 - Update libsmaf and tests to care of the SMAF API evolution
   https://git.linaro.org/people/benjamin.gaignard/libsmaf.git

version 2 changes:
 - Add one ioctl to allow allocator selection from userspace.
   This is required for the uses case where the first user of
   the buffer is a software IP which can't perform dma_buf attachement.
 - Add name and ranking to allocator structure to be able to sort them.
 - Create a tiny library to test SMAF:
   https://git.linaro.org/people/benjamin.gaignard/libsmaf.git
 - Fix one issue when try to secure buffer without secure module registered

SMAF aim to solve two problems: allocating memory that fit with hardware IPs
constraints and secure those data from bus point of view.

One example of SMAF usage is camera preview: on SoC you may use either an USB
webcam or the built-in camera interface and the frames could be send directly
to the dipslay Ip or handle by GPU.
Most of USB interfaces and GPU have mmu but almost all built-in camera
interace and display Ips don't have mmu so when selecting how allocate
buffer you need to be aware of each devices constraints (contiguous memroy,
stride, boundary, alignment ...).
ION has solve this problem by let userland decide which allocator (heap) to use
but this require to adapt userland for each platform and sometime for each
use case.

To be sure to select the best allocation method for devices SMAF implement
deferred allocation mechanism: memory allocation is only done when the first
device effectively required it.
Allocator modules have to implement a match() to let SMAF know if they are
compatibles with devices needs.
This patch set provide an example of allocator module which use
dma_{alloc/free/mmap}_attrs() and check if at least one device have
coherent_dma_mask set to DMA_BIT_MASK(32) in match function.

In the same camera preview use case, SMAF allow to protect the data from being
read by unauthorized IPs (i.e. a malware to dump camera stream).
Until now I have only see access rights protection at process/thread level 
(PKeys/MPK) or on file (SELinux) but nothing allow to drive data bus firewalls.
SMAF propose an interface to control and implement those firewalls.
Like IOMMU, firewalls IPs can help to protect memory from malicious/faulty 
devices
that are attempting DMA attacks.

Secure modules are responsibles of granting and revoking devices access rights
on the memory. Secure module is also called to check if CPU map memory into
kernel and user address spaces.
An example of secure module implementation can be found here:
http://git.linaro.org/people/benjamin.gaignard/optee-sdp.git
This code isn't yet part of the patch set because it depends on generic TEE
which is still under discussion (https://lwn.net/Articles/644646/)

For allocation part of SMAF code I get inspirated by Sumit Semwal work about
constraint aware allocator.

Benjamin Gaignard (3):
  create SMAF module
  SMAF: add CMA allocator
  SMAF: add test secure module

 drivers/Kconfig|   2 +
 drivers/Makefile   |   1 +
 drivers/smaf/Kconfig   |  17 +
 drivers/smaf/Makefile  |   3 +
 drivers/smaf/smaf-cma.c| 186 ++
 drivers/smaf/smaf-core.c   | 818 +
 drivers/smaf/smaf-testsecure.c |  90 +
 include/linux/smaf-allocator.h |  45