GSOC '08 hardware accelerated video decoding

2008-03-28 Thread Younes M

I recently posted to the Nouveau mailing list about this, but for
those who don't participate in that one I thought I would also post
here since it seems to concern DRI as much as Nouveau. I intend to
submit an application for a project that will attempt to implement
XvMC in terms of Gallium3D. I've come up with a preliminary proposal
and was hoping people would be willing to give it a quick read and
give me some feedback; opinions, corrections, concerns, etc. An HTML
version is here: and
a text version is below.

Also, something I forgot to mention in my Nouveau email is that I
would need a mentor and a mentoring organization. I've been told that
this can be pursued under since DRI isn't a mentoring org this
year, hopefully this is OK,

Thank you kindly.

Younes Manton
younes.m at gmail

Generic GPU-Accelerated Video Decoding


The purpose of this project is to produce a video decoding solution
for GPUs that are supported by the Gallium3D driver framework. The
project will attempt to implement the XvMC API using the programmable
pipeline of a typical GPU, thereby providing accelerated video
decoding to a wide variety of hardware. Since the decoding will be
implemented using the GPU's programmable pipeline, it is important to
note that this solution should support all recent GPUs regardless of
whether or not they include dedicated video decoding hardware. It is
hoped that this GPU-based acceleration will allow for real-time play
back of HD video streams on mid-range and possibly low-end hardware.
The implementation will be developed and tested using Gallium3D's
SoftPipe driver, a stable software reference implementation, and later
on Nvidia hardware and the nouveau driver.


Video media has become a pervasive part of the computing landscape and
encompasses a variety of formats and resolutions, from low-res MPEG2
streams to HD MPEG4 content. From the point of view of the end-user,
accelerated video decoding offers potentially better quality, smoother
multi-tasking (by way of unburdening the main CPU) and the extension
of the lifespan of current mid-range and low-end hardware. From the
point of view of the OSS community, accelerated video decoding will
offer an incentive to the end-user to adopt open-source drivers, which
have traditionally not provided significant video acceleration on the
most popular GPUs. This particular project will also provide a
multi-vendor solution, as most GPUs supported by the Gallium3D
framework can be targeted with the same code base, including some
current Nvidia hardware via the nouveau driver, AMD/ATI hardware, and
Intel hardware, amongst others.


The deliverables for this project have been organized into two
categories: the minimum set of deliverables that would make this
project worthwhile for all involved (must-haves), and a larger set of
goals that would make good contributions to the community and offer
greater benefit to the end-user (nice-to-haves).

   * An XvMC implementation that handles the color space conversion
(CSC) and motion compensation (MC) stages of the video decoding
pipeline. These two stages represent the bulk of the processing and
are good candidates for being handled by the GPU. This should allow
for real-time play back of HD video streams according to [1]. As part
of this goal it is expected some work will have to be done with the
nouveau driver to address possible bugs and add required

   * Handling of the inverse discrete cosine transform (IDCT) stage
of the video decoding pipeline. This stage does not map optimally to
the GPU pipeline but represents a large percentage of the processing
and would also allow for the preceding stage (inverse quantization -
IQ) to also be handled by the GPU without introducing an extra
GPU-CPU-GPU round trip between the IQ, IDCT, and MC stages. XvMC was
originally intended to handle the MC stage, but has been extended to
support IDCT.

A preliminary timeline with milestones is presented below:

   Preliminary research  experimentation - April weeks 1  2
   mplement CSC with SoftPipe - April weeks 3  4
   Implement MC, IDCT with SoftPipe - May to mid-June (1)
   Preliminary hardware research  experimentation - June
   Test with real hardware, add required functionality, fix bugs -
mid-June to Aug (2)
   Bug fixes, performance testing  tuning, documentation - Aug

  1. Working implementation with the SoftPipe driver by mid-June
  2. Working implementation with nouveau driver by end-July


   * Support for other video formats. XvMC was originally intended
for MPEG2 video, but has been extended to support other formats such
as MPEG4.

   * An implementation of the Video Acceleration API (VAAPI) which is
similar to XvMC but has been designed to support off-loading more
stages and more video formats. VAAPI is not currently supported widely
by user applications, but 

Re: GSOC '08 hardware accelerated video decoding

2008-03-29 Thread Younes M
Tom Cooksey wrote:
 Isn't XvMC going to be depricated in favor of VAAPI? Not sure if it's of any
 to you, but Qt 4.4.0 has an OpenGL playback widget for Phonon GStreamer
 backend. It uses a shader to do the color-space conversion and is avaliable
 under the GPLv2/GPLv3.

 If your interest is XcMV because of client-side support, I may be able to
 find someone in Trolltech willing to write a Phonon/GStreamer playback
 widget which uses VAAPI. It may not be able to make use of all the VAAPI
 features as many of them need to be supported in the decoder elements
 (I think anyway). Ping me if this is of interest.



Hi Tom,

I went with XvMC because it has a lot of client support as you said. I
think it would be immediately useful to a lot of people, plus there
are lots of resources out there. Implementing VAAPI is on my list of
nice-to-haves if I have time, but at the very least I'll try to keep
the code modularized enough to make a VAAPI implementation as painless
as possible. I think having VAAPI support would kick start things and
get more clients to support it, so it's definitely worth the effort.
If I have time at towards the end to do a basic VAAPI implementation
I'll definitely keep your offer in mind.

Thank you very much,


Check out the new Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.;164216239;13503038;w?
Dri-devel mailing list

Writing a state tracker for Gallium3D/SoftPipe

2008-04-16 Thread Younes M
I'm trying to get up and running writing a state tracker for libXvMC
using SoftPipe, but looking at the Mesa src a few things are unclear
to me.

Looking at root/src/gallium/winsys/xlib I can't quite figure out where
Mesa ends and where Gallium starts. It's my understanding that the
winsys creates the pipe_context that I need, but is there a generic X
winsys driver? The xm_* sources where softpipe_create() is eventually
called (in xm_winsys.c) look very tied to Mesa, even though they
appear to be in a generic directory, so do all state trackers have to
implement something similar to use SoftPipe? Is this also the case for
using hw drivers? Looking at the the aub src files and the dri
directory it looks like that stuff is tied to the hw and does not have
to be provided for each state tracker.

This email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2.;198757673;13503038;p?
Dri-devel mailing list