Memory Management Mini-Summit
Linaro Developer Summit, Budapest, May 9-11, 2011
=================================================

Hi all.  Apologies for this report being so long in coming.  I know
others have thrown in their perceptions and opinions on how the
mini-summit went, so I suppose it's my turn.

Outcomes:
---------
 * Approach (full proposal under draft, to be sent to the lists below)
  - Modified CMA for additional physically contiguous buffer support.
  - dma-mapping API changes, enhancements and ARM architecture support.
  - "struct dma_buf" based buffer sharing infrastructure with support
from device drivers.
  - Pick any "low-hanging fruit" with respect to consolidation
(supporting the ARM arch/sub-arch goals).

 * Proposal for work around allocation, mapping and buffer sharing to
be announced on:
  - dri-devel
  - linux-arm-kernel
  - linux-kernel
  - linux-media
  - linux-mm
  - linux-mm-sig

 * Communication and Meetings
  - New IRC channel #linaro-mm-sig for meetings and general
communication between those working on and interested in these topics
(already created).
  - IRC meetings will be weekly with an option for the consituency to
decide on ultimate frequency (logs to be emailed to linaro-mm-sig
list).
  - Linaro can provide wiki services and any dial-in needed.
  - Next face-to-face meetings:
   . Linaro mid-cycle summit (August 1-5, see
https://wiki.linaro.org/Events/2011-08-LDS)
   . Linux Plumbers Conference (September 7-9, see
http://www.linuxplumbersconf.org/2011/ocw/proposals/567)
   . V4L2 brainstorm meeting (Hans Verkuil to update with details)

Overview and Goals for the 3 days:
----------------------------------
 * Day 1 - Component overviews, expected to spill over into day 2
 * Day 2 - Concrete use case that outlines a definition of the problem
that we are trying to solve, and shows that we have solved it.
 * Day 3 - Dig into the lower level details of the current
implementations.   What do we have, what's missing, what's not
implemented for ARM.

This is about memory management, zero-copy pipelines, kernel/userspace
interfaces, memory management, memory reservations and much more  :-)
In particular, what we would like to end up with is:

 * Understand who is working on what; avoid work duplication.
 * Focus on a specific problem we want to solve and discuss possible solutions.
 * Come up with a plan to fix this specific problem.
 * Start enumerating work items that the Linaro Graphics WG can work
on in this cycle.

Day 1:
------
The first day got off to a little bit of a stutter start as the summit
scheduler would not let us indicate that our desired starting time was
immediately after lunch, during the plenaries.  However, that didn't
stop people from flocking to the session in droves.  By the time I
made the kickoff comments on why we were there, and what we were there
to accomplish (see "Overview and Goals for the 3 days" above), we had
brought in an extra 10 chairs and there were people on the floor and
spilling out into the hallway.

Based upon our experiences from the birds-of-a-feather at the Embedded
Linux Conference, 2 things dominated day 1.  First things first, I
assigned someone to take notes ;-).  Etherpad made it really easy for
people to take notes collectively, including those participating
remotely, and for everyone to see who was writing what, but we
definitely needed someone whose focus would be capturing the
proceedings, so thanks to Dave Rusling for shouldering that burden.
The second thing was that we desperately needed an education in each
others components and subsystems.  Without this, we would risk missing
significant areas of discussion, or possibly even be violently
agreeing on something without realizing it.  So, we started with a
series of component overviews.  These were presentations on the order
of 20 minutes with some room for Q&A.  On day 1, we had:

 * V4L2 - Hans Verkuil
 * DRM/GEM/KMS - Daniel Vetter
 * TTM - Thomas Hellstrom
 * CMA - Marek Szyprowski
 * VCMM - Zach Pfeffer

All of these (as well as the ones from day 2) are available through
links on the mini-summit wiki
(https://wiki.linaro.org/Events/2011-05-MM).

Day 2:
------
The second day got off to a bit better a start than did day 1 as we
more clearly communicated the start time to everyone involved, and
forgot about the summit scheduler.  We (conceptually) picked up where
day 1 left off with one more component overview:

 * UMP - Ketil Johnson

and, covered the MediaController API for good measure.  From there, we
spent a fair amount of time discussing use cases to illustrate our
problem space.  We started (via pre-summit submissions) with a couple
of variations on what amounted to basically the same thing.  I think
the actual case is probably best illustrated by the pdf slides from
Sakari Ailus (see the link on the mini-summit wiki).  Basically, we
want to take a video input, either from a camera or from a file,
decode it, process it, render to it and/or with it and display it.
These pipeline stages may be handled by hardware, by software on the
CPU or some combination of the two; each stage should be handled by
accepting a buffer from the last stage and operating on it in some
fashion (no copies wherever possible).  It turned out that still image
capture can actually be a more complicated version of this use case,
but even something as simple as taking input through the camera and
displaying it (image preview) can involve much of the underpinnings
required to support the more complicated cases.  We may indeed start
with this simple case as a proof-of-concept.

Once we had the use case nailed down, we moved onto the actual
components/subsystems that would need to share buffers in order for
the use case to work properly with the zero-copy (or at least
minimal-copy) requirement.  We had:

 * DRM
 * V4L2
 * fbdev
 * ALSA
 * DSP
 * User-space (kind of all encompassing and could include things like
OpenCL, which also makes an interesting use case).
 * DVB
 * Out-of-tree GPU drivers

We wound out the day by discussing exactly what metadata we would want
to track in order to enable the desired levels of sharing with
simultaneous device mappings, cache management and other
considerations (e.g., device peculiarities).  What we came up with is
a struct (we called it "dma_buf") that has the following info:

 * Size
 * Creator/Allocator
 * Attributes:
   - sharable?
   - contiguous?
   - device-local?
 * Reference count
 * Pinning reference count
 * CPU cache management data
 * Device private data (e.g., quirky tiling modes)
 * Scatter list
 * Synchronization data (for managing in-flight device transactions)
 * Mapping data

These last few (device privates through mapping data) are lists of
data, one for each device that has a mapping of the buffer.  The
mapping data is nominally an address and per-device cache management
data.  We actually got through the this part fairly quickly.  The
biggest part of the discussion was what to use for handles/identifiers
in the buffer sharing scheme.  The discussion was between global
identifiers like GEM uses, or file descriptors as favored by Android.
Initially, there was an informal consensus around unique IDs, though
it was not a definitive decision (yet).  The atomicity of passing file
descriptors between processes makes them quite attractive for the
task.

Day 3:
------
By the third day, there was a sense of running out of time and really
needing to ensure that we left with a reasonable set of outcomes (see
the overview and goals section above).  In short, we wanted to make
sure that we had a plan/roadmap, a reasonably actionable set of tasks
that could be picked up by Linaro engineers and community members
alike, and that we would not only avoid duplicating new work, but also
reduce some of the existing code duplication that got us to this point
in the first place.

But, we weren't done.  We still had to cover the requirements around
allocation and explore the dma-mapping and IOMMU APIs.

This took most of the day, but was a quite fruitful set of
discussions.  As with the rest of the discussions, we focused on
leveraging existing technologies as much as possible.  With
allocations, however, this wasn't entirely possible as we have devices
on ARM SoCs that do not have an IOMMU and require physically
contiguous buffers in order to operate.  After a fair amount of
discussion, it was decided that a modified version of the current CMA
(see Marek's slides linked from the wiki).  It assumes the pages are
movable and manages them and not the mappings.  There was concern that
the API didn't quite fit with other related API, so the changes from
the current state will be around those details.

On the mapping side, we focused on the dma-mapping API with
appropriate layering on the IOMMU API where appropriate.  Without
going into crazy detail, we are looking at something like 4
implementation s of the dma_map_ops functions for ARM: with and
without IOMMU, with and without bounce buffer (these last two exist,
but not using the dma_map_ops API).  Marek has put out patches for
comment on the IOMMU based implementation of this based upon work he
had in progress.  Also in the area of dma_map_ops, the sync related
API need a start address and offset, and the alloc and free need
attribute parameters like map and unmap already have (to support
cacheable/coherent/write-combined).  In the "not involving
dma_map_ops" category, we have a couple of changes that are likely to
be non-trivial (not that any of the other proposed work is).  It was
proposed to modify (actually, the word thrown about in the discussions
was "fix") dma_alloc_coherent for ARM to support unmapping from the
kernel linear mapping and the use of HIGHMEM; two separate
implementations, configured at build-time.  And, last but not least,
there was a fair amount of concern over the cache management API and
its ability to live cleanly with the IOMMU code and to resist breakage
from other architecture implementations.

At this point, we reviewed what we had done and finalized the outcomes
(see the outcomes section at the top).  And, with a half an hour to
spare, I re-instigated the file descriptors versus unique identifiers
discussion from day 2.  I think file descriptors were winning by the
end (especially after people started posting pointers to code samples
of how to actually pass them between processes)....

Attendees:
----------
I will likely miss people here trying to list out everyone, especially
given that some of the sessions were quite literally overflowing the
room we were in.  For as accurate an account of attendance as I can
muster, check out the list of attendees on the mini-summit wiki page
or the discussion blueprints we used for scheduling:

https://wiki.linaro.org/Events/2011-05-MM#Attendees
https://blueprints.launchpad.net/linaro-graphics-wg/+spec/linaro-graphics-memory-management-summit-1
https://blueprints.launchpad.net/linaro-graphics-wg/+spec/linaro-graphics-memory-management-summit-2
https://blueprints.launchpad.net/linaro-graphics-wg/+spec/linaro-graphics-memory-management-summit-3

The occupants of the fishbowl (the front/center of the room in closest
proximity to the microphones) were primarily:

Arnd Bergmann
Laurent Pinchart
Hans Verkuil
Mauro Chehab
Daniel Vetter
Sakari Ailus
Thomas Hellstrom
Marek Szyprowski
Jesse Barker

The IRC fishbowl seemed to consist of:

Rob Morell
Jordan Crouse
David Brown

There were certainly others both local and remote participating to
varying degrees that I do not intend to omit, and a special thanks
goes out to Joey Stanford for arranging a larger room for us on days 2
and 3 when we had people sitting on the floor and spilling into the
hallway during day 1.

_______________________________________________
linaro-dev mailing list
linaro-dev@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-dev

Reply via email to