Re: [PATCH v2] media: cedrus: don't initialize pointers with zero
Hi,
On Fri, 2018-12-07 at 08:03 -0500, Mauro Carvalho Chehab wrote:
> A common mistake is to assume that initializing a var with:
> struct foo f = { 0 };
>
> Would initialize a zeroed struct. Actually, what this does is
> to initialize the first element of the struct to zero.
>
> According to C99 Standard 6.7.8.21:
>
> "If there are fewer initializers in a brace-enclosed
> list than there are elements or members of an aggregate,
> or fewer characters in a string literal used to initialize
> an array of known size than there are elements in the array,
> the remainder of the aggregate shall be initialized implicitly
> the same as objects that have static storage duration."
>
> So, in practice, it could zero the entire struct, but, if the
> first element is not an integer, it will produce warnings:
>
>
> drivers/staging/media/sunxi/cedrus/cedrus.c:drivers/staging/media/sunxi/cedrus/cedrus.c:78:49:
> warning: Using plain integer as NULL pointer
>
> drivers/staging/media/sunxi/cedrus/cedrus_dec.c:drivers/staging/media/sunxi/cedrus/cedrus_dec.c:29:35:
> warning: Using plain integer as NULL pointer
>
> As the right initialization would be, instead:
>
> struct foo f = { NULL };
Thanks for sharing these details, it's definitely interesting and good
to know :)
> Another way to initialize it with gcc is to use:
>
> struct foo f = {};
>
> That seems to be a gcc extension, but clang also does the right thing,
> and that's a clean way for doing it.
>
> Anyway, I decided to check upstream what's the most commonly pattern.
> The "= {}" pattern has about 2000 entries:
>
> $ git grep -E "=\s*\{\s*\}"|wc -l
> 1951
>
> The standard-C compliant pattern has about 2500 entries:
>
> $ git grep -E "=\s*\{\s*NULL\s*\}"|wc -l
> 137
> $ git grep -E "=\s*\{\s*0\s*\}"|wc -l
> 2323
>
> Meaning that developers have split options on that.
>
> So, let's opt to the simpler form.
Acked-by: Paul Kocialkowski
> Signed-off-by: Mauro Carvalho Chehab
> ---
> drivers/staging/media/sunxi/cedrus/cedrus.c | 2 +-
> drivers/staging/media/sunxi/cedrus/cedrus_dec.c | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c
> b/drivers/staging/media/sunxi/cedrus/cedrus.c
> index b538eb0321d8..b7c918fa5fd1 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus.c
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c
> @@ -75,7 +75,7 @@ static int cedrus_init_ctrls(struct cedrus_dev *dev, struct
> cedrus_ctx *ctx)
> memset(ctx->ctrls, 0, ctrl_size);
>
> for (i = 0; i < CEDRUS_CONTROLS_COUNT; i++) {
> - struct v4l2_ctrl_config cfg = { 0 };
> + struct v4l2_ctrl_config cfg = {};
>
> cfg.elem_size = cedrus_controls[i].elem_size;
> cfg.id = cedrus_controls[i].id;
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> index e40180a33951..f10c25f5460e 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> @@ -26,7 +26,7 @@ void cedrus_device_run(void *priv)
> {
> struct cedrus_ctx *ctx = priv;
> struct cedrus_dev *dev = ctx->dev;
> - struct cedrus_run run = { 0 };
> + struct cedrus_run run = {};
> struct media_request *src_req;
> unsigned long flags;
>
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
Re: [PATCH] media: cetrus: return an error if alloc fails
Hi,
On Fri, 2018-12-07 at 06:13 -0500, Mauro Carvalho Chehab wrote:
> As warned by smatch:
>
> drivers/staging/media/sunxi/cedrus/cedrus.c:
> drivers/staging/media/sunxi/cedrus/cedrus.c:93 cedrus_init_ctrls() error:
> potential null dereference 'ctx->ctrls'. (kzalloc returns null)
>
> While here, remove the memset(), as kzalloc() already zeroes the
> struct.
Good catch, thanks for the patch!
> Signed-off-by: Mauro Carvalho Chehab
Acked-by: Paul Kocialkowski
> ---
> drivers/staging/media/sunxi/cedrus/cedrus.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c
> b/drivers/staging/media/sunxi/cedrus/cedrus.c
> index 44c45c687503..24b89cd2b692 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus.c
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c
> @@ -72,7 +72,8 @@ static int cedrus_init_ctrls(struct cedrus_dev *dev, struct
> cedrus_ctx *ctx)
> ctrl_size = sizeof(ctrl) * CEDRUS_CONTROLS_COUNT + 1;
>
> ctx->ctrls = kzalloc(ctrl_size, GFP_KERNEL);
> - memset(ctx->ctrls, 0, ctrl_size);
> + if (!ctx->ctrls)
> + return -ENOMEM;
>
> for (i = 0; i < CEDRUS_CONTROLS_COUNT; i++) {
> struct v4l2_ctrl_config cfg = { NULL };
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
Re: [PATCH for v4.20 0/2] cedrus: move MPEG controls out of the uAPI
Hi, On Wed, 2018-12-05 at 13:09 +0100, hverkuil-ci...@xs4all.nl wrote: > From: Hans Verkuil > > The expectation was that the MPEG-2 state controls used by the staging > cedrus driver were stable, or would only require one final change. However, > it turns out that more changes are required, and that means that it is not > such a good idea to have these controls in the public kernel API. > > This patch series moves all the MPEG-2 state control data to a new > media/mpeg2-ctrls.h header. So none of this is available from the public > API. > > However, v4l2-ctrls.h includes it for now so the kAPI still knows about it > allowing the cedrus driver to use it without changes. > > The second patch adds a note to these two controls, mentioning that they > are likely to change. > > Moving forward, this allows us to take more time in getting the MPEG-2 > (and later H264/5) state controls right. Thanks a lot for this change, I'm glad we can take time to properly stabilize these controls! For the whole series: Reviewed-by: Paul Kocialkowski Cheers, Paul > Regards, > > Hans > > Hans Verkuil (2): > mpeg2-ctrls.h: move MPEG2 state controls to non-public header > extended-controls.rst: add note to the MPEG2 state controls > > .../media/uapi/v4l/extended-controls.rst | 10 +++ > drivers/media/v4l2-core/v4l2-ctrls.c | 4 +- > include/media/mpeg2-ctrls.h | 86 +++ > include/media/v4l2-ctrls.h| 6 ++ > include/uapi/linux/v4l2-controls.h| 68 --- > include/uapi/linux/videodev2.h| 4 - > 6 files changed, 104 insertions(+), 74 deletions(-) > create mode 100644 include/media/mpeg2-ctrls.h > -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com
Re: [PATCHv3 9/9] cedrus: add tag support
Hi,
On Mon, 2018-12-03 at 14:51 +0100, hverkuil-ci...@xs4all.nl wrote:
> From: Hans Verkuil
>
> Replace old reference frame indices by new tag method.
> Signed-off-by: Hans Verkuil
> Reviewed-by: Paul Kocialkowski
I missed it earlier, but we should remember to update the MPEG-2
controls documentation to mention the use of tags instead of buffer
indices.
Cheers,
Paul
> Reviewed-by: Alexandre Courbot
> ---
> drivers/media/v4l2-core/v4l2-ctrls.c | 9
> drivers/staging/media/sunxi/cedrus/cedrus.h | 9 +---
> .../staging/media/sunxi/cedrus/cedrus_dec.c | 2 ++
> .../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 21 ---
> .../staging/media/sunxi/cedrus/cedrus_video.c | 2 ++
> include/uapi/linux/v4l2-controls.h| 14 +
> 6 files changed, 24 insertions(+), 33 deletions(-)
>
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> b/drivers/media/v4l2-core/v4l2-ctrls.c
> index 5f2b033a7a42..b854cceb19dc 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -1660,15 +1660,6 @@ static int std_validate(const struct v4l2_ctrl *ctrl,
> u32 idx,
> return -EINVAL;
> }
>
> - if (p_mpeg2_slice_params->backward_ref_index >= VIDEO_MAX_FRAME
> ||
> - p_mpeg2_slice_params->forward_ref_index >= VIDEO_MAX_FRAME)
> - return -EINVAL;
> -
> - if (p_mpeg2_slice_params->pad ||
> - p_mpeg2_slice_params->picture.pad ||
> - p_mpeg2_slice_params->sequence.pad)
> - return -EINVAL;
> -
> return 0;
>
> case V4L2_CTRL_TYPE_MPEG2_QUANTIZATION:
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.h
> b/drivers/staging/media/sunxi/cedrus/cedrus.h
> index 3f61248c57ac..781676b55a1b 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus.h
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus.h
> @@ -142,11 +142,14 @@ static inline dma_addr_t cedrus_buf_addr(struct
> vb2_buffer *buf,
> }
>
> static inline dma_addr_t cedrus_dst_buf_addr(struct cedrus_ctx *ctx,
> - unsigned int index,
> - unsigned int plane)
> + int index, unsigned int plane)
> {
> - struct vb2_buffer *buf = ctx->dst_bufs[index];
> + struct vb2_buffer *buf;
>
> + if (index < 0)
> + return 0;
> +
> + buf = ctx->dst_bufs[index];
> return buf ? cedrus_buf_addr(buf, >dst_fmt, plane) : 0;
> }
>
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> index e40180a33951..0cfd6036d0cd 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
> @@ -53,6 +53,8 @@ void cedrus_device_run(void *priv)
> break;
> }
>
> + v4l2_m2m_buf_copy_data(run.src, run.dst, true);
> +
> dev->dec_ops[ctx->current_codec]->setup(ctx, );
>
> spin_unlock_irqrestore(>dev->irq_lock, flags);
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c
> b/drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c
> index 9abd39cae38c..fdde9a099153 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c
> @@ -82,7 +82,10 @@ static void cedrus_mpeg2_setup(struct cedrus_ctx *ctx,
> struct cedrus_run *run)
> dma_addr_t fwd_luma_addr, fwd_chroma_addr;
> dma_addr_t bwd_luma_addr, bwd_chroma_addr;
> struct cedrus_dev *dev = ctx->dev;
> + struct vb2_queue *cap_q = >fh.m2m_ctx->cap_q_ctx.q;
> const u8 *matrix;
> + int forward_idx;
> + int backward_idx;
> unsigned int i;
> u32 reg;
>
> @@ -156,23 +159,17 @@ static void cedrus_mpeg2_setup(struct cedrus_ctx *ctx,
> struct cedrus_run *run)
> cedrus_write(dev, VE_DEC_MPEG_PICBOUNDSIZE, reg);
>
> /* Forward and backward prediction reference buffers. */
> + forward_idx = vb2_find_tag(cap_q, slice_params->forward_ref_tag, 0);
>
> - fwd_luma_addr = cedrus_dst_buf_addr(ctx,
> - slice_params->forward_ref_index,
> - 0);
> - fwd_chroma_addr = cedrus_dst_buf_addr(ctx,
> - slice_params->forward_ref_index,
> - 1);
> + fwd_luma_addr = cedrus
Re: [PATCHv2 0/9] vb2/cedrus: add tag support
Hi, On Mon, 2018-11-19 at 12:18 +0100, Hans Verkuil wrote: > On 11/14/2018 02:47 PM, Hans Verkuil wrote: > > From: Hans Verkuil > > > > As was discussed here (among other places): > > > > https://lkml.org/lkml/2018/10/19/440 > > > > using capture queue buffer indices to refer to reference frames is > > not a good idea. A better idea is to use a 'tag' where the > > application can assign a u64 tag to an output buffer, which is then > > copied to the capture buffer(s) derived from the output buffer. > > > > A u64 is chosen since this allows userspace to also use pointers to > > internal structures as 'tag'. > > > > The first three patches add core tag support, the next patch document > > the tag support, then a new helper function is added to v4l2-mem2mem.c > > to easily copy data from a source to a destination buffer that drivers > > can use. > > > > Next a new supports_tags vb2_queue flag is added to indicate that > > the driver supports tags. Ideally this should not be necessary, but > > that would require that all m2m drivers are converted to using the > > new helper function introduced in the previous patch. That takes more > > time then I have now since we want to get this in for 4.20. > > > > Finally the vim2m, vicodec and cedrus drivers are converted to support > > tags. > > > > I also removed the 'pad' fields from the mpeg2 control structs (it > > should never been added in the first place) and aligned the structs > > to a u32 boundary (u64 for the tag values). > > > > Note that this might change further (Paul suggested using bitfields). > > > > Also note that the cedrus code doesn't set the sequence counter, that's > > something that should still be added before this driver can be moved > > out of staging. > > > > Note: if no buffer is found for a certain tag, then the dma address > > is just set to 0. That happened before as well with invalid buffer > > indices. This should be checked in the driver! > > > > The previous RFC series was tested successfully with the cedrus driver. > > > > Regards, > > > > Hans > > I'd like to get some Acked-by or Reviewed-by replies for this series. > Or comments if you don't like something. The series looks good to me, so consider it: Reviewed-by: Paul Kocialkowski Also, I'm glad you made the v4l2_m2m_buf_copy_data helper function, since all drivers will need to do these same operations anyway. > I would really like to get this in for 4.20 so the cedrus API is stable > (hopefully), since this is a last outstanding API item. I see a few more items that we need to tackle before we can consider the MPEG-2 stateless API as stable. * I mentionned using flags in the mpeg2 structures to solve the alignment issues, but I failed to provide a patch implementing it at this point. This would make the MPEG-2 controls more similar to the proposed H.264 ones and generally makes more sense than using a u8 for a binary value. * Some time ago, we also discussed adding extra fields to the structures for later use. Do you think that is still relevant? Adding a flags elements (with unused bits) would certainly help in this direction anyway. * During the discussions on the spec, the concensus was also to use one slice_params per-slice (instead of per-picture) which requires splitting the per-picture and the per-slice parameters. On this as well, I fell behind and didn't send a proposal yet. So my question is: does it need to be done for 4.20 or can we affoard going through another version cycle for these changes? It was my impression that we wanted to wait for another driver to use the API to declare it stable (and move the driver out of staging). Cheers, Paul > Tomasz: you commented that the text still referred to the tag as a u64, > but that was only in the cover letter, not the patches themselves. So > I don't plan to post a v3 just for that. > > Regards, > > Hans > > > Changes since v1: > > > > - changed to a u32 tag. Using a 64 bit tag was overly complicated due > > to the bad layout of the v4l2_buffer struct, and there is no real > > need for it by applications. > > > > Main changes since the RFC: > > > > - Added new buffer capability flag > > - Added m2m helper to copy data between buffers > > - Added documentation > > - Added tag logging in v4l2-ioctl.c > > > > Hans Verkuil (9): > > videodev2.h: add tag support > > vb2: add tag support > > v4l2-ioctl.c: log v4l2_buffer tag > > buffer.rst: document the new buffer tag feature. > > v4l2-mem2mem: add v4l2_m2m_buf_copy_data helper funct
Re: [PATCH] cedrus: add action item to the TODO
Hi, On Thu, 2018-11-15 at 08:49 +0100, Hans Verkuil wrote: > Mention that the request validation should increment the memory refcount > of reference buffers so we don't forget to do this. Thanks for adding this item, we should definitely take care of it before unstaging. Acked-by: Paul Kocialkowski > Signed-off-by: Hans Verkuil > --- > diff --git a/drivers/staging/media/sunxi/cedrus/TODO > b/drivers/staging/media/sunxi/cedrus/TODO > index ec277ece47af..a951b3fd1ea1 100644 > --- a/drivers/staging/media/sunxi/cedrus/TODO > +++ b/drivers/staging/media/sunxi/cedrus/TODO > @@ -5,3 +5,8 @@ Before this stateless decoder driver can leave the staging > area: > * Userspace support for the Request API needs to be reviewed; > * Another stateless decoder driver should be submitted; > * At least one stateless encoder driver should be submitted. > +* When queueing a request containing references to I frames, the > + refcount of the memory for those I frames needs to be incremented > + and decremented when the request is completed. This will likely > + require some help from vb2. The driver should fail the request > + if the memory/buffer is gone. -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [PATCH v5 0/5] Make sure .device_run is always called in non-atomic context
Hi, On Mon, 2018-11-12 at 18:05 -0300, Ezequiel Garcia wrote: > On Mon, 12 Nov 2018 at 13:52, Paul Kocialkowski > wrote: > > Hi, > > > > On Sun, 2018-11-11 at 18:26 -0300, Ezequiel Garcia wrote: > > > On Thu, 2018-10-18 at 15:02 -0300, Ezequiel Garcia wrote: > > > > This series goal is to avoid drivers from having ad-hoc code > > > > to call .device_run in non-atomic context. Currently, .device_run > > > > can be called via v4l2_m2m_job_finish(), not only running > > > > in interrupt context, but also creating a nasty re-entrant > > > > path into mem2mem drivers. > > > > > > > > The proposed solution is to add a per-device worker that is scheduled > > > > by v4l2_m2m_job_finish, which replaces drivers having a threaded > > > > interrupt > > > > or similar. > > > > > > > > This change allows v4l2_m2m_job_finish() to be called in interrupt > > > > context, separating .device_run and v4l2_m2m_job_finish() contexts. > > > > > > > > It's worth mentioning that v4l2_m2m_cancel_job() doesn't need > > > > to flush or cancel the new worker, because the job_spinlock > > > > synchronizes both and also because the core prevents simultaneous > > > > jobs. Either v4l2_m2m_cancel_job() will wait for the worker, or the > > > > worker will be unable to run a new job. > > > > > > > > Patches apply on top of Request API and the Cedrus VPU > > > > driver. > > > > > > > > Tested with cedrus driver using v4l2-request-test and > > > > vicodec driver using gstreamer. > > > > > > > > Ezequiel Garcia (4): > > > > mem2mem: Require capture and output mutexes to match > > > > v4l2-ioctl.c: Simplify locking for m2m devices > > > > v4l2-mem2mem: Avoid calling .device_run in v4l2_m2m_job_finish > > > > media: cedrus: Get rid of interrupt bottom-half > > > > > > > > Sakari Ailus (1): > > > > v4l2-mem2mem: Simplify exiting the function in __v4l2_m2m_try_schedule > > > > > > > > drivers/media/v4l2-core/v4l2-ioctl.c | 47 + > > > > drivers/media/v4l2-core/v4l2-mem2mem.c| 66 --- > > > > .../staging/media/sunxi/cedrus/cedrus_hw.c| 26 ++-- > > > > 3 files changed, 51 insertions(+), 88 deletions(-) > > > > > > > > > > Hans, Maxime: > > > > > > Any feedback for this? > > > > I just tested the whole series with the cedrus driver and everything > > looks good! > > > > Good! So this means we can add a Tested-by for the entire series? Definitely, I just sent the tested-by line on the cover letter! > > Removing the interrupt bottom-half in favor of a workqueue in the core > > seems like a good way to simplify m2m driver development by avoiding > > per-driver workqueues or threaded irqs. > > > > Thanks for the test! Cheers, Paul -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [PATCH v5 0/5] Make sure .device_run is always called in non-atomic context
Hi, On Thu, 2018-10-18 at 15:02 -0300, Ezequiel Garcia wrote: > This series goal is to avoid drivers from having ad-hoc code > to call .device_run in non-atomic context. Currently, .device_run > can be called via v4l2_m2m_job_finish(), not only running > in interrupt context, but also creating a nasty re-entrant > path into mem2mem drivers. > > The proposed solution is to add a per-device worker that is scheduled > by v4l2_m2m_job_finish, which replaces drivers having a threaded interrupt > or similar. > > This change allows v4l2_m2m_job_finish() to be called in interrupt > context, separating .device_run and v4l2_m2m_job_finish() contexts. > > It's worth mentioning that v4l2_m2m_cancel_job() doesn't need > to flush or cancel the new worker, because the job_spinlock > synchronizes both and also because the core prevents simultaneous > jobs. Either v4l2_m2m_cancel_job() will wait for the worker, or the > worker will be unable to run a new job. > > Patches apply on top of Request API and the Cedrus VPU > driver. > > Tested with cedrus driver using v4l2-request-test and > vicodec driver using gstreamer. For the entire series, tested with the cedrus driver: Tested-by: Paul Kocialkowski Cheers, Paul > Ezequiel Garcia (4): > mem2mem: Require capture and output mutexes to match > v4l2-ioctl.c: Simplify locking for m2m devices > v4l2-mem2mem: Avoid calling .device_run in v4l2_m2m_job_finish > media: cedrus: Get rid of interrupt bottom-half > > Sakari Ailus (1): > v4l2-mem2mem: Simplify exiting the function in __v4l2_m2m_try_schedule > > drivers/media/v4l2-core/v4l2-ioctl.c | 47 + > drivers/media/v4l2-core/v4l2-mem2mem.c| 66 --- > .../staging/media/sunxi/cedrus/cedrus_hw.c| 26 ++-- > 3 files changed, 51 insertions(+), 88 deletions(-) > -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [PATCH v5 5/5] media: cedrus: Get rid of interrupt bottom-half
Hi,
On Thu, 2018-10-18 at 15:02 -0300, Ezequiel Garcia wrote:
> Now that the mem2mem framework guarantees that .device_run
> won't be called from interrupt context, it is safe to call
> v4l2_m2m_job_finish directly in the top-half.
>
> So this means the bottom-half is no longer needed and we
> can get rid of it.
>
> Signed-off-by: Ezequiel Garcia
Acked-by: Paul Kocialkowski
Cheers,
Paul
> ---
> .../staging/media/sunxi/cedrus/cedrus_hw.c| 26 ---
> 1 file changed, 5 insertions(+), 21 deletions(-)
>
> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c
> b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c
> index 32adbcbe6175..493e65b17b30 100644
> --- a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c
> +++ b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c
> @@ -98,23 +98,6 @@ void cedrus_dst_format_set(struct cedrus_dev *dev,
> }
> }
>
> -static irqreturn_t cedrus_bh(int irq, void *data)
> -{
> - struct cedrus_dev *dev = data;
> - struct cedrus_ctx *ctx;
> -
> - ctx = v4l2_m2m_get_curr_priv(dev->m2m_dev);
> - if (!ctx) {
> - v4l2_err(>v4l2_dev,
> - "Instance released before the end of transaction\n");
> - return IRQ_HANDLED;
> - }
> -
> - v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx);
> -
> - return IRQ_HANDLED;
> -}
> -
> static irqreturn_t cedrus_irq(int irq, void *data)
> {
> struct cedrus_dev *dev = data;
> @@ -165,7 +148,9 @@ static irqreturn_t cedrus_irq(int irq, void *data)
>
> spin_unlock_irqrestore(>irq_lock, flags);
>
> - return IRQ_WAKE_THREAD;
> + v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx);
> +
> + return IRQ_HANDLED;
> }
>
> int cedrus_hw_probe(struct cedrus_dev *dev)
> @@ -187,9 +172,8 @@ int cedrus_hw_probe(struct cedrus_dev *dev)
>
> return irq_dec;
> }
> - ret = devm_request_threaded_irq(dev->dev, irq_dec, cedrus_irq,
> - cedrus_bh, 0, dev_name(dev->dev),
> - dev);
> + ret = devm_request_irq(dev->dev, irq_dec, cedrus_irq,
> +0, dev_name(dev->dev), dev);
> if (ret) {
> v4l2_err(>v4l2_dev, "Failed to request IRQ\n");
>
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [PATCH v5 0/5] Make sure .device_run is always called in non-atomic context
Hi, On Sun, 2018-11-11 at 18:26 -0300, Ezequiel Garcia wrote: > On Thu, 2018-10-18 at 15:02 -0300, Ezequiel Garcia wrote: > > This series goal is to avoid drivers from having ad-hoc code > > to call .device_run in non-atomic context. Currently, .device_run > > can be called via v4l2_m2m_job_finish(), not only running > > in interrupt context, but also creating a nasty re-entrant > > path into mem2mem drivers. > > > > The proposed solution is to add a per-device worker that is scheduled > > by v4l2_m2m_job_finish, which replaces drivers having a threaded interrupt > > or similar. > > > > This change allows v4l2_m2m_job_finish() to be called in interrupt > > context, separating .device_run and v4l2_m2m_job_finish() contexts. > > > > It's worth mentioning that v4l2_m2m_cancel_job() doesn't need > > to flush or cancel the new worker, because the job_spinlock > > synchronizes both and also because the core prevents simultaneous > > jobs. Either v4l2_m2m_cancel_job() will wait for the worker, or the > > worker will be unable to run a new job. > > > > Patches apply on top of Request API and the Cedrus VPU > > driver. > > > > Tested with cedrus driver using v4l2-request-test and > > vicodec driver using gstreamer. > > > > Ezequiel Garcia (4): > > mem2mem: Require capture and output mutexes to match > > v4l2-ioctl.c: Simplify locking for m2m devices > > v4l2-mem2mem: Avoid calling .device_run in v4l2_m2m_job_finish > > media: cedrus: Get rid of interrupt bottom-half > > > > Sakari Ailus (1): > > v4l2-mem2mem: Simplify exiting the function in __v4l2_m2m_try_schedule > > > > drivers/media/v4l2-core/v4l2-ioctl.c | 47 + > > drivers/media/v4l2-core/v4l2-mem2mem.c| 66 --- > > .../staging/media/sunxi/cedrus/cedrus_hw.c| 26 ++-- > > 3 files changed, 51 insertions(+), 88 deletions(-) > > > > Hans, Maxime: > > Any feedback for this? I just tested the whole series with the cedrus driver and everything looks good! Removing the interrupt bottom-half in favor of a workqueue in the core seems like a good way to simplify m2m driver development by avoiding per-driver workqueues or threaded irqs. Cheers, Paul -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [RFC PATCHv2 5/5] cedrus: add tag support
PICBOUNDSIZE, reg);
>
> /* Forward and backward prediction reference buffers. */
> + forward_idx = vb2_find_tag(cap_q, slice_params->forward_ref_tag, 0);
>
> - fwd_luma_addr = cedrus_dst_buf_addr(ctx,
> - slice_params->forward_ref_index,
> - 0);
> - fwd_chroma_addr = cedrus_dst_buf_addr(ctx,
> - slice_params->forward_ref_index,
> - 1);
> + fwd_luma_addr = cedrus_dst_buf_addr(ctx, forward_idx, 0);
> + fwd_chroma_addr = cedrus_dst_buf_addr(ctx, forward_idx, 1);
>
> cedrus_write(dev, VE_DEC_MPEG_FWD_REF_LUMA_ADDR, fwd_luma_addr);
> cedrus_write(dev, VE_DEC_MPEG_FWD_REF_CHROMA_ADDR, fwd_chroma_addr);
>
> - bwd_luma_addr = cedrus_dst_buf_addr(ctx,
> - slice_params->backward_ref_index,
> - 0);
> - bwd_chroma_addr = cedrus_dst_buf_addr(ctx,
> - slice_params->backward_ref_index,
> - 1);
> + backward_idx = vb2_find_tag(cap_q, slice_params->backward_ref_tag, 0);
> + bwd_luma_addr = cedrus_dst_buf_addr(ctx, backward_idx, 0);
> + bwd_chroma_addr = cedrus_dst_buf_addr(ctx, backward_idx, 1);
>
> cedrus_write(dev, VE_DEC_MPEG_BWD_REF_LUMA_ADDR, bwd_luma_addr);
> cedrus_write(dev, VE_DEC_MPEG_BWD_REF_CHROMA_ADDR, bwd_chroma_addr);
> diff --git a/include/uapi/linux/v4l2-controls.h
> b/include/uapi/linux/v4l2-controls.h
> index 998983a6e6b7..43f2f9148b3c 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -1109,10 +1109,9 @@ struct v4l2_mpeg2_sequence {
> __u32 vbv_buffer_size;
>
> /* ISO/IEC 13818-2, ITU-T Rec. H.262: Sequence extension */
> - __u8profile_and_level_indication;
> + __u16 profile_and_level_indication;
> __u8progressive_sequence;
> __u8chroma_format;
> - __u8pad;
> };
>
> struct v4l2_mpeg2_picture {
> @@ -1130,23 +1129,20 @@ struct v4l2_mpeg2_picture {
> __u8intra_vlc_format;
> __u8alternate_scan;
> __u8repeat_first_field;
> - __u8progressive_frame;
> - __u8pad;
> + __u16 progressive_frame;
> };
>
> struct v4l2_ctrl_mpeg2_slice_params {
> __u32 bit_size;
> __u32 data_bit_offset;
> + __u64 backward_ref_tag;
> + __u64 forward_ref_tag;
>
> struct v4l2_mpeg2_sequence sequence;
> struct v4l2_mpeg2_picture picture;
>
> /* ISO/IEC 13818-2, ITU-T Rec. H.262: Slice */
> - __u8quantiser_scale_code;
> -
> - __u8backward_ref_index;
> - __u8forward_ref_index;
> - __u8pad;
> + __u32 quantiser_scale_code;
> };
>
> struct v4l2_ctrl_mpeg2_quantization {
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [RFC PATCHv2 0/5] vb2/cedrus: add tag support
Hi, On Mon, 2018-11-12 at 09:33 +0100, Hans Verkuil wrote: > As was discussed here (among other places): > > https://lkml.org/lkml/2018/10/19/440 > > using capture queue buffer indices to refer to reference frames is > not a good idea. A better idea is to use a 'tag' (thanks to Alexandre > for the excellent name; it's much better than 'cookie') where the > application can assign a u64 tag to an output buffer, which is then > copied to the capture buffer(s) derived from the output buffer. > > A u64 is chosen since this allows userspace to also use pointers to > internal structures as 'tag'. As I mentionned in the dedicated patch, this approach is troublesome on 32-bit platforms. Do we really need this equivalency? > The first two patches add core tag support, the next two patches > add tag support to vim2m and vicodec, and the final patch (compile > tested only!) adds support to the cedrus driver. > > I also removed the 'pad' fields from the mpeg2 control structs (it > should never been added in the first place) and aligned the structs > to a u32 boundary (u64 for the tag values). > > The cedrus code now also copies the timestamps (didn't happen before) > but the sequence counter is still not set, that's something that should > still be added. > > Note: if no buffer is found for a certain tag, then the dma address > is just set to 0. That happened before as well with invalid buffer > indices. This should be checked in the driver! Thanks for making these changes! > Also missing in this series are documentation updates, which is why > it is marked RFC. > > I would very much appreciate it if someone can test the cedrus driver > with these changes. If it works, then I can prepare a real patch series > for 4.20. It would be really good if the API is as stable as we can make > it before 4.20 is released. I just had a go at testing the patches on cedrus with minimal userspace adaptation to deal with the tags and everything looks good! I only set the tag when queing each OUTPUT buffer and the driver properly matched the CAPTURE reference buffer. I think we should make it clear in the stateless spec that multiple OUTPUT buffers can be allowed for the same tag, but that a single CAPTURE buffer should be used. Otherwise, the hardware can't use different partly-decoded buffers as references (and the tag API doesn't allow that either, since a single buffer index is returned for a tag). What do you think? Cheers, Paul > Regards, > > Hans > > Changes since v1: > > - cookie -> tag > - renamed v4l2_tag to v4l2_buffer_tag > - dropped spurious 'to' in the commit log of patch 1 > > Hans Verkuil (5): > videodev2.h: add tag support > vb2: add tag support > vim2m: add tag support > vicodec: add tag support > cedrus: add tag support > > .../media/common/videobuf2/videobuf2-v4l2.c | 43 --- > drivers/media/platform/vicodec/vicodec-core.c | 3 ++ > drivers/media/platform/vim2m.c| 3 ++ > drivers/media/v4l2-core/v4l2-ctrls.c | 9 > drivers/staging/media/sunxi/cedrus/cedrus.h | 8 ++-- > .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 + > .../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 21 - > include/media/videobuf2-v4l2.h| 18 > include/uapi/linux/v4l2-controls.h| 14 +++--- > include/uapi/linux/videodev2.h| 37 +++- > 10 files changed, 127 insertions(+), 39 deletions(-) > -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [RFC PATCHv2 1/5] videodev2.h: add tag support
Hi,
On Mon, 2018-11-12 at 09:33 +0100, Hans Verkuil wrote:
> From: Hans Verkuil
>
> Add support for 'tags' to struct v4l2_buffer. These can be used
> by m2m devices so userspace can set a tag for an output buffer and
> this value will then be copied to the capture buffer(s).
>
> This tag can be used to refer to capture buffers, something that
> is needed by stateless HW codecs.
I am getting the following warning when building with this patch:
videodev2.h:1044:9: warning: cast to pointer from integer of different
size [-Wint-to-pointer-cast]
return (void *)v4l2_buffer_get_tag(buf);
That is because we are on a 32-bit architecture here, so pointers are
not 64-bit long, thus we can't have an equivalency between tag and
pointer.
It looks like this isn't used in the following patches, so perhaps it
should be dropped?
Cheers,
Paul
> Signed-off-by: Hans Verkuil
> ---
> include/uapi/linux/videodev2.h | 37 +-
> 1 file changed, 36 insertions(+), 1 deletion(-)
>
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index c8e8ff810190..a6f81f368e01 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -912,6 +912,11 @@ struct v4l2_plane {
> __u32 reserved[11];
> };
>
> +struct v4l2_buffer_tag {
> + __u32 low;
> + __u32 high;
> +};
> +
> /**
> * struct v4l2_buffer - video buffer info
> * @index: id number of the buffer
> @@ -950,7 +955,10 @@ struct v4l2_buffer {
> __u32 flags;
> __u32 field;
> struct timeval timestamp;
> - struct v4l2_timecodetimecode;
> + union {
> + struct v4l2_timecodetimecode;
> + struct v4l2_buffer_tag tag;
> + };
> __u32 sequence;
>
> /* memory location */
> @@ -988,6 +996,8 @@ struct v4l2_buffer {
> #define V4L2_BUF_FLAG_IN_REQUEST 0x0080
> /* timecode field is valid */
> #define V4L2_BUF_FLAG_TIMECODE 0x0100
> +/* tag field is valid */
> +#define V4L2_BUF_FLAG_TAG0x0200
> /* Buffer is prepared for queuing */
> #define V4L2_BUF_FLAG_PREPARED 0x0400
> /* Cache handling flags */
> @@ -1007,6 +1017,31 @@ struct v4l2_buffer {
> /* request_fd is valid */
> #define V4L2_BUF_FLAG_REQUEST_FD 0x0080
>
> +static inline void v4l2_buffer_set_tag(struct v4l2_buffer *buf, __u64 tag)
> +{
> + buf->tag.high = tag >> 32;
> + buf->tag.low = tag & 0xULL;
> + buf->flags |= V4L2_BUF_FLAG_TAG;
> +}
> +
> +static inline void v4l2_buffer_set_tag_ptr(struct v4l2_buffer *buf,
> +const void *tag)
> +{
> + v4l2_buffer_set_tag(buf, (__u64)tag);
> +}
> +
> +static inline __u64 v4l2_buffer_get_tag(const struct v4l2_buffer *buf)
> +{
> + if (!(buf->flags & V4L2_BUF_FLAG_TAG))
> + return 0;
> + return (((__u64)buf->tag.high) << 32) | (__u64)buf->tag.low;
> +}
> +
> +static inline void *v4l2_buffer_get_tag_ptr(const struct v4l2_buffer *buf)
> +{
> + return (void *)v4l2_buffer_get_tag(buf);
> +}
> +
> /**
> * struct v4l2_exportbuffer - export of video buffer as DMABUF file
> descriptor
> *
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [PATCH] media: v4l: v4l2-controls.h must include types.h
Hi,
On Mon, 2018-11-12 at 11:01 +0100, Jean Delvare wrote:
> Fix the following build-time warning:
> ./usr/include/linux/v4l2-controls.h:1105: found __[us]{8,16,32,64} type
> without #include
We already have a similar fix in the media tree:
https://git.linuxtv.org/media_tree.git/commit/?h=request_api=dafb7f9aef2fd44991ff1691721ff765a23be27b
So it looks like we won't be needing this one!
Cheers,
Paul
> Signed-off-by: Jean Delvare
> Fixes: c27bb30e7b6d ("media: v4l: Add definitions for MPEG-2 slice format and
> metadata")
> Cc: Paul Kocialkowski
> Cc: Mauro Carvalho Chehab
> ---
> include/uapi/linux/v4l2-controls.h |2 ++
> 1 file changed, 2 insertions(+)
>
> --- linux-4.20-rc2.orig/include/uapi/linux/v4l2-controls.h2018-11-12
> 09:34:20.869048454 +0100
> +++ linux-4.20-rc2/include/uapi/linux/v4l2-controls.h 2018-11-12
> 10:54:38.864904194 +0100
> @@ -50,6 +50,8 @@
> #ifndef __LINUX_V4L2_CONTROLS_H
> #define __LINUX_V4L2_CONTROLS_H
>
> +#include /* for __u* typedefs */
> +
> /* Control classes */
> #define V4L2_CTRL_CLASS_USER 0x0098 /* Old-style 'user'
> controls */
> #define V4L2_CTRL_CLASS_MPEG 0x0099 /* MPEG-compression
> controls */
>
>
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [PATCH v5 5/6] media: Add controls for JPEG quantization tables
Hi, Le mercredi 19 septembre 2018 à 13:28 +0900, Tomasz Figa a écrit : > On Thu, Sep 13, 2018 at 9:15 PM Paul Kocialkowski wrote: > > Hi, > > > > On Wed, 2018-09-05 at 19:00 -0300, Ezequiel Garcia wrote: > > > From: Shunqian Zheng > > > > > > Add V4L2_CID_JPEG_QUANTIZATION compound control to allow userspace > > > configure the JPEG quantization tables. > > > > > > Signed-off-by: Shunqian Zheng > > > Signed-off-by: Ezequiel Garcia > > > --- > > > .../media/uapi/v4l/extended-controls.rst | 31 +++ > > > .../media/videodev2.h.rst.exceptions | 1 + > > > drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++ > > > include/uapi/linux/v4l2-controls.h| 12 +++ > > > include/uapi/linux/videodev2.h| 1 + > > > 5 files changed, 55 insertions(+) > > > > > > diff --git a/Documentation/media/uapi/v4l/extended-controls.rst > > > b/Documentation/media/uapi/v4l/extended-controls.rst > > > index 9f7312bf3365..1335d27d30f3 100644 > > > --- a/Documentation/media/uapi/v4l/extended-controls.rst > > > +++ b/Documentation/media/uapi/v4l/extended-controls.rst > > > @@ -3354,7 +3354,38 @@ JPEG Control IDs > > > Specify which JPEG markers are included in compressed stream. This > > > control is valid only for encoders. > > > > > > +.. _jpeg-quant-tables-control: > > > > I just had a look at how the Allwinner VPU handles JPEG decoding and it > > seems to require the following information (in addition to > > quantization): > > I assume the hardware doesn't have the ability to parse those from the > stream and so they need to be parsed by user space and given to the > driver? That's correct, we are also dealing with a stateless decoder here. It's actually the same hardware engine that's used for MPEG2 decoding, only configured differently. So we will need to introduce a pixfmt for compressed JPEG data without headers, reuse JPEG controls that apply and perhaps introduce new ones too if needed. I am also wondering about how MJPEG support should fit into this. As far as I understood, it shouldn't be very different from JPEG so we might want to have common controls for both. > > * Horizontal and vertical sampling factors for each Y/U/V component: > > > > The number of components and sampling factors are coded separately in > > the bitstream, but it's probably easier to use the already-existing > > V4L2_CID_JPEG_CHROMA_SUBSAMPLING control for specifying that. > > > > However, this is potentially very much related to the destination > > format. If we decide that this format should match the format resulting > > from decompression, we don't need to specify it through an external > > control. On the other hand, it's possible that the VPU has format > > conversion block integrated in its pipeline so it would also make sense > > to consider the destination format as independent. > > +1 for keeping it separate. Just like for the stateless decoding API, it would make sense to expect userspace to set those before enumerating CAPTURE formats in order to determine what the hardware can output. > > * Custom Huffman tables (DC and AC), both for luma and chroma > > > > It seems that there is a default table when no Huffman table is provided > > in the bitstream (I'm not too sure how standard that is, just started > > learning about JPEG). We probably need a specific control for that. > > What happens if there is one in the bitstream? Would the hardware pick > it automatically? In our case, this part of the bitstream wouldn't be sent to the hardware anyway. > I think it might make sense to just have a general control for Huffman > table, which would be always provided by the user space, regardless of > whether it's parsed from the stream or default, so that drivers don't > have to care and could just always use it. For MPEG-2 support (and probably also H.265), we have considered the quantization tables optional and kept a default value in the driver. That's because said tables are not supported in all profiles, so they are de-facto optional. I think it's fair to consider that userspace does not need to implement more than what is needed for decoding. This makes our interface closer to the data obtained from the bitstream. However, having one copy of the default table per driver is far from optimal. I would suggest moving it to common v4l2 functions instead, but keeping it in kernel space. What do you think? Cheers, Paul -- Developer of free digital technology and hardware support. Website: https://www.paulk.fr/ Coding blog: https://code.paulk.fr/ Git repositories: https://git.paulk.fr/ https://git.code.paulk.fr/ signature.asc Description: This is a digitally signed message part
Re: [PATCH v5 5/6] media: Add controls for JPEG quantization tables
Hi, On Mon, 2018-09-10 at 10:25 -0300, Ezequiel Garcia wrote: > Hi Hans, > > Thanks for the review. > > On Mon, 2018-09-10 at 14:42 +0200, Hans Verkuil wrote: > > On 09/06/2018 12:00 AM, Ezequiel Garcia wrote: > > > From: Shunqian Zheng > > > > > > Add V4L2_CID_JPEG_QUANTIZATION compound control to allow userspace > > > configure the JPEG quantization tables. > > > > > > Signed-off-by: Shunqian Zheng > > > Signed-off-by: Ezequiel Garcia > > > --- > > > .../media/uapi/v4l/extended-controls.rst | 31 +++ > > > .../media/videodev2.h.rst.exceptions | 1 + > > > drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++ > > > include/uapi/linux/v4l2-controls.h| 12 +++ > > > include/uapi/linux/videodev2.h| 1 + > > > 5 files changed, 55 insertions(+) > > > > > > diff --git a/Documentation/media/uapi/v4l/extended-controls.rst > > > b/Documentation/media/uapi/v4l/extended-controls.rst > > > index 9f7312bf3365..1335d27d30f3 100644 > > > --- a/Documentation/media/uapi/v4l/extended-controls.rst > > > +++ b/Documentation/media/uapi/v4l/extended-controls.rst > > > @@ -3354,7 +3354,38 @@ JPEG Control IDs > > > Specify which JPEG markers are included in compressed stream. This > > > control is valid only for encoders. > > > > > > +.. _jpeg-quant-tables-control: > > > > > > +``V4L2_CID_JPEG_QUANTIZATION (struct)`` > > > +Specifies the luma and chroma quantization matrices for encoding > > > +or decoding a V4L2_PIX_FMT_JPEG_RAW format buffer. The :ref:`itu-t81` > > > +specification allows 8-bit quantization coefficients for > > > +baseline profile images, and 8-bit or 16-bit for extended profile > > > +images. Supporting or not 16-bit precision coefficients is > > > driver-specific. > > > +Coefficients must be set in JPEG zigzag scan order. > > > + > > > + > > > +.. c:type:: struct v4l2_ctrl_jpeg_quantization > > > + > > > +.. cssclass:: longtable > > > + > > > +.. flat-table:: struct v4l2_ctrl_jpeg_quantization > > > +:header-rows: 0 > > > +:stub-columns: 0 > > > +:widths: 1 1 2 > > > + > > > +* - __u8 > > > + - ``precision`` > > > + - Specifies the coefficient precision. User shall set 0 > > > +for 8-bit, and 1 for 16-bit. > > > > So does specifying 1 here switch the HW encoder to use extended profile? > > What if the HW only supports baseline? The rockchip driver doesn't appear > > to check the precision field at all... > > > > The driver is missing to check that, when the user sets this control. > > > I think this needs a bit more thought. > > > > I am not at all sure that this is the right place for the precision field. > > This is really about JPEG profiles, so I would kind of expect a JPEG PROFILE > > control (just like other codec profiles), or possibly a new pixelformat for > > extended profiles. > > > > And based on that the driver would interpret these matrix values as 8 or > > 16 bits. > > > > Right, the JPEG profile control is definitely needed. I haven't add it because > it wouldn't be used, since this VPU can only do baseline. Well, I suppose it would still be relevant that you add it for the encoder and only report baseline there. > However, the problem is that some JPEGs in the wild have with 8-bit data and > 16-bit quantization coefficients, as per [1] and [2]: > > [1] https://github.com/martinhath/jpeg-rust/issues/1 > [2] https://github.com/libjpeg-turbo/libjpeg-turbo/pull/90 > > So, in order to support decoding of these images, I've added the precision > field to the quantization control. The user would be able to set a baseline > or extended profile thru a (future) profile control, and if 16-bit > tables are found, and if the hardware supports them, the driver > would be able to support them. > > Another option, which might be even better, is have explicit baseline > and extended quantization tables controls, e.g.: V4L2_CID_JPEG_QUANT > and V4L2_CID_JPEG_EXT_QUANT. I think this makes more sense than a common structure with an indication bit on how to interpret the data. However, it seems problematic that userspace can't figure out whether 16-bit quant tables are supported with a baseline profile and just has to try and see. Hans, do you think this is an acceptable approach or should we rather stick to the standard here, at the cost of not supporting these pictures that were encoded with this common abuse of the standard? Cheers, Paul -- Developer of free digital technology and hardware support. Website: https://www.paulk.fr/ Coding blog: https://code.paulk.fr/ Git repositories: https://git.paulk.fr/ https://git.code.paulk.fr/
Re: [PATCH v5 5/6] media: Add controls for JPEG quantization tables
Hi, On Thu, 2018-09-13 at 14:14 +0200, Paul Kocialkowski wrote: > Hi, > > On Wed, 2018-09-05 at 19:00 -0300, Ezequiel Garcia wrote: > > From: Shunqian Zheng > > > > Add V4L2_CID_JPEG_QUANTIZATION compound control to allow userspace > > configure the JPEG quantization tables. > > > > Signed-off-by: Shunqian Zheng > > Signed-off-by: Ezequiel Garcia > > --- > > .../media/uapi/v4l/extended-controls.rst | 31 +++ > > .../media/videodev2.h.rst.exceptions | 1 + > > drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++ > > include/uapi/linux/v4l2-controls.h| 12 +++ > > include/uapi/linux/videodev2.h| 1 + > > 5 files changed, 55 insertions(+) > > > > diff --git a/Documentation/media/uapi/v4l/extended-controls.rst > > b/Documentation/media/uapi/v4l/extended-controls.rst > > index 9f7312bf3365..1335d27d30f3 100644 > > --- a/Documentation/media/uapi/v4l/extended-controls.rst > > +++ b/Documentation/media/uapi/v4l/extended-controls.rst > > @@ -3354,7 +3354,38 @@ JPEG Control IDs > > Specify which JPEG markers are included in compressed stream. This > > control is valid only for encoders. > > > > +.. _jpeg-quant-tables-control: > > I just had a look at how the Allwinner VPU handles JPEG decoding and it > seems to require the following information (in addition to > quantization): > > * Horizontal and vertical sampling factors for each Y/U/V component: > > The number of components and sampling factors are coded separately in > the bitstream, but it's probably easier to use the already-existing > V4L2_CID_JPEG_CHROMA_SUBSAMPLING control for specifying that. > > However, this is potentially very much related to the destination > format. If we decide that this format should match the format resulting > from decompression, we don't need to specify it through an external > control. On the other hand, it's possible that the VPU has format > conversion block integrated in its pipeline so it would also make sense > to consider the destination format as independent. > > * Custom Huffman tables (DC and AC), both for luma and chroma > > It seems that there is a default table when no Huffman table is provided > in the bitstream (I'm not too sure how standard that is, just started > learning about JPEG). We probably need a specific control for that. > > * Reset interval > > That's extracted from the bitstream as well and there's a > V4L2_CID_JPEG_RESTART_INTERVAL control already. > > In addition to these points, I see that among all the JPEG profiles, > some have to do with arithmetic coding which will probably require a > specific control on its own (not sure how it should look at this point > though). > > What do you think? For clarification: these are required for decoding, not for encoding. For encoding, it seems that the Allwinner hardware only requires quantization tables. Cheers, Paul > > +``V4L2_CID_JPEG_QUANTIZATION (struct)`` > > +Specifies the luma and chroma quantization matrices for encoding > > +or decoding a V4L2_PIX_FMT_JPEG_RAW format buffer. The :ref:`itu-t81` > > +specification allows 8-bit quantization coefficients for > > +baseline profile images, and 8-bit or 16-bit for extended profile > > +images. Supporting or not 16-bit precision coefficients is > > driver-specific. > > +Coefficients must be set in JPEG zigzag scan order. > > + > > + > > +.. c:type:: struct v4l2_ctrl_jpeg_quantization > > + > > +.. cssclass:: longtable > > + > > +.. flat-table:: struct v4l2_ctrl_jpeg_quantization > > +:header-rows: 0 > > +:stub-columns: 0 > > +:widths: 1 1 2 > > + > > +* - __u8 > > + - ``precision`` > > + - Specifies the coefficient precision. User shall set 0 > > +for 8-bit, and 1 for 16-bit. > > + > > +* - __u16 > > + - ``luma_quantization_matrix[64]`` > > + - Sets the luma quantization table. > > + > > +* - __u16 > > + - ``chroma_quantization_matrix[64]`` > > + - Sets the chroma quantization table. > > > > .. flat-table:: > > :header-rows: 0 > > diff --git a/Documentation/media/videodev2.h.rst.exceptions > > b/Documentation/media/videodev2.h.rst.exceptions > > index ca9f0edc579e..a0a38e92bf38 100644 > > --- a/Documentation/media/videodev2.h.rst.exceptions > > +++ b/Documentation/media/videodev2.h.rst.exceptions > > @@ -129,6 +129,7 @@ replace symbol V4L2_CTRL_TYPE_STRING > > :c:type:`v4l2_ctrl_type`
Re: [PATCH v2] staging: cedrus: Fix checkpatch issues
Hi,
On Thu, 2018-09-13 at 11:53 -0300, Mauro Carvalho Chehab wrote:
> Em Thu, 13 Sep 2018 16:40:47 +0200
> Maxime Ripard escreveu:
>
>
> > --- a/drivers/staging/media/sunxi/cedrus/cedrus_video.c
> > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_video.c
> > @@ -82,10 +82,7 @@ static struct cedrus_format *cedrus_find_format(u32
> > pixelformat, u32 directions,
> > static bool cedrus_check_format(u32 pixelformat, u32 directions,
> > unsigned int capabilities)
> > {
> > - struct cedrus_format *fmt = cedrus_find_format(pixelformat, directions,
> > - capabilities);
> > -
> > - return fmt != NULL;
> > + return cedrus_find_format(pixelformat, directions, capabilities);
> > }
>
> Hmm... just occurred to me... Why do you need this? I mean, you
> could simply do something like:
>
> $ git filter-branch -f --tree-filter 'for i in $(git grep -l
> cedrus_check_format); do \
> sed -E s,\\bcedrus_check_format\\b,cedrus_find_format,g -i $i; done '
> origin/master..
>
> (or just do a sed -E s,\\bcedrus_check_format\\b,cedrus_find_format,g as
> a separate patch)
>
> and get rid of cedrus_check_format() for good.
Agreed, the name is probably explicit enough anyway. I probably should
have done that in the first place anyway.
Cheers,
Paul
--
Developer of free digital technology and hardware support.
Website: https://www.paulk.fr/
Coding blog: https://code.paulk.fr/
Git repositories: https://git.paulk.fr/ https://git.code.paulk.fr/
signature.asc
Description: This is a digitally signed message part
Re: [PATCH v5 5/6] media: Add controls for JPEG quantization tables
Hi, On Wed, 2018-09-05 at 19:00 -0300, Ezequiel Garcia wrote: > From: Shunqian Zheng > > Add V4L2_CID_JPEG_QUANTIZATION compound control to allow userspace > configure the JPEG quantization tables. > > Signed-off-by: Shunqian Zheng > Signed-off-by: Ezequiel Garcia > --- > .../media/uapi/v4l/extended-controls.rst | 31 +++ > .../media/videodev2.h.rst.exceptions | 1 + > drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++ > include/uapi/linux/v4l2-controls.h| 12 +++ > include/uapi/linux/videodev2.h| 1 + > 5 files changed, 55 insertions(+) > > diff --git a/Documentation/media/uapi/v4l/extended-controls.rst > b/Documentation/media/uapi/v4l/extended-controls.rst > index 9f7312bf3365..1335d27d30f3 100644 > --- a/Documentation/media/uapi/v4l/extended-controls.rst > +++ b/Documentation/media/uapi/v4l/extended-controls.rst > @@ -3354,7 +3354,38 @@ JPEG Control IDs > Specify which JPEG markers are included in compressed stream. This > control is valid only for encoders. > > +.. _jpeg-quant-tables-control: I just had a look at how the Allwinner VPU handles JPEG decoding and it seems to require the following information (in addition to quantization): * Horizontal and vertical sampling factors for each Y/U/V component: The number of components and sampling factors are coded separately in the bitstream, but it's probably easier to use the already-existing V4L2_CID_JPEG_CHROMA_SUBSAMPLING control for specifying that. However, this is potentially very much related to the destination format. If we decide that this format should match the format resulting from decompression, we don't need to specify it through an external control. On the other hand, it's possible that the VPU has format conversion block integrated in its pipeline so it would also make sense to consider the destination format as independent. * Custom Huffman tables (DC and AC), both for luma and chroma It seems that there is a default table when no Huffman table is provided in the bitstream (I'm not too sure how standard that is, just started learning about JPEG). We probably need a specific control for that. * Reset interval That's extracted from the bitstream as well and there's a V4L2_CID_JPEG_RESTART_INTERVAL control already. In addition to these points, I see that among all the JPEG profiles, some have to do with arithmetic coding which will probably require a specific control on its own (not sure how it should look at this point though). What do you think? Cheers, Paul > +``V4L2_CID_JPEG_QUANTIZATION (struct)`` > +Specifies the luma and chroma quantization matrices for encoding > +or decoding a V4L2_PIX_FMT_JPEG_RAW format buffer. The :ref:`itu-t81` > +specification allows 8-bit quantization coefficients for > +baseline profile images, and 8-bit or 16-bit for extended profile > +images. Supporting or not 16-bit precision coefficients is > driver-specific. > +Coefficients must be set in JPEG zigzag scan order. > + > + > +.. c:type:: struct v4l2_ctrl_jpeg_quantization > + > +.. cssclass:: longtable > + > +.. flat-table:: struct v4l2_ctrl_jpeg_quantization > +:header-rows: 0 > +:stub-columns: 0 > +:widths: 1 1 2 > + > +* - __u8 > + - ``precision`` > + - Specifies the coefficient precision. User shall set 0 > +for 8-bit, and 1 for 16-bit. > + > +* - __u16 > + - ``luma_quantization_matrix[64]`` > + - Sets the luma quantization table. > + > +* - __u16 > + - ``chroma_quantization_matrix[64]`` > + - Sets the chroma quantization table. > > .. flat-table:: > :header-rows: 0 > diff --git a/Documentation/media/videodev2.h.rst.exceptions > b/Documentation/media/videodev2.h.rst.exceptions > index ca9f0edc579e..a0a38e92bf38 100644 > --- a/Documentation/media/videodev2.h.rst.exceptions > +++ b/Documentation/media/videodev2.h.rst.exceptions > @@ -129,6 +129,7 @@ replace symbol V4L2_CTRL_TYPE_STRING > :c:type:`v4l2_ctrl_type` > replace symbol V4L2_CTRL_TYPE_U16 :c:type:`v4l2_ctrl_type` > replace symbol V4L2_CTRL_TYPE_U32 :c:type:`v4l2_ctrl_type` > replace symbol V4L2_CTRL_TYPE_U8 :c:type:`v4l2_ctrl_type` > +replace symbol V4L2_CTRL_TYPE_JPEG_QUANTIZATION :c:type:`v4l2_ctrl_type` > > # V4L2 capability defines > replace define V4L2_CAP_VIDEO_CAPTURE device-capabilities > diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c > b/drivers/media/v4l2-core/v4l2-ctrls.c > index 599c1cbff3b9..305bd7a9b7f1 100644 > --- a/drivers/media/v4l2-core/v4l2-ctrls.c > +++ b/drivers/media/v4l2-core/v4l2-ctrls.c > @@ -999,6 +999,7 @@ const char *v4l2_ctrl_get_name(u32 id) > case V4L2_CID_JPEG_RESTART_INTERVAL:return "Restart Interval"; > case V4L2_CID_JPEG_COMPRESSION_QUALITY: return "Compression Quality"; > case V4L2_CID_JPEG_ACTIVE_MARKER: return "Active Markers"; > + case V4L2_CID_JPEG_QUANTIZATION:return "JPEG Quantization > Tables";
[PATCH 0/2] HEVC/H.265 stateless support for V4L2 and Cedrus
This introduces the required bits for supporting HEVC/H.265 both in the V4L2 framework and the Cedrus VPU driver that concerns Allwinner devices. A specific pixel format is introduced for the HEVC slice format and controls are provided to pass the bitstream metadata to the decoder. Some bitstream extensions are knowingly not supported at this point. Since this is the first proposal for stateless HEVC/H.265 support in V4L2, reviews and comments about the controls definitions are particularly welcome. On the Cedrus side, the H.265 implementation covers frame pictures with both uni-directional and bi-direction prediction modes (P/B slices). Field pictures (interleaved), scaling lists and 10-bit output are not supported at this point. This series is based upon the following series: * Cedrus driver for the Allwinner Video Engine, using media requests * media: cedrus: Add H264 decoding support Cheers! Paul Kocialkowski (2): media: v4l: Add definitions for the HEVC slice format and controls media: cedrus: Add HEVC/H.265 decoding support .../media/uapi/v4l/extended-controls.rst | 416 ++ .../media/uapi/v4l/pixfmt-compressed.rst | 15 + .../media/uapi/v4l/vidioc-queryctrl.rst | 18 + .../media/videodev2.h.rst.exceptions | 3 + drivers/media/v4l2-core/v4l2-ctrls.c | 26 + drivers/media/v4l2-core/v4l2-ioctl.c | 1 + drivers/staging/media/sunxi/cedrus/Makefile | 2 +- drivers/staging/media/sunxi/cedrus/cedrus.c | 19 + drivers/staging/media/sunxi/cedrus/cedrus.h | 20 +- .../staging/media/sunxi/cedrus/cedrus_dec.c | 9 + .../staging/media/sunxi/cedrus/cedrus_h265.c | 540 ++ .../staging/media/sunxi/cedrus/cedrus_hw.c| 4 + .../staging/media/sunxi/cedrus/cedrus_regs.h | 290 ++ .../staging/media/sunxi/cedrus/cedrus_video.c | 13 + include/media/v4l2-ctrls.h| 6 + include/uapi/linux/v4l2-controls.h| 155 + include/uapi/linux/videodev2.h| 7 + 17 files changed, 1542 insertions(+), 2 deletions(-) create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_h265.c -- 2.18.0
Re: [PATCH v3 6/7] media: Add controls for JPEG quantization tables
Hi,
On Wed, 2018-08-22 at 13:59 -0300, Ezequiel Garcia wrote:
> From: Shunqian Zheng
>
> Add V4L2_CID_JPEG_LUMA/CHROMA_QUANTIZATION controls to allow userspace
> configure the JPEG quantization tables.
How about having a single control for quantization?
In MPEG-2/H.264/H.265, we have a single control exposed as a structure,
which contains the tables for both luma and chroma. In the case of JPEG,
it's not that big a deal, but for advanced video formats, it would be
too much hassle to have one control per table.
In order to keep the interface consistent, I think it'd be best to merge
both matrices into a single control.
What do you think?
Paul
> Signed-off-by: Shunqian Zheng
> Signed-off-by: Ezequiel Garcia
> ---
> Documentation/media/uapi/v4l/extended-controls.rst | 13 +
> drivers/media/v4l2-core/v4l2-ctrls.c | 13 +
> include/uapi/linux/v4l2-controls.h | 3 +++
> 3 files changed, 29 insertions(+)
>
> diff --git a/Documentation/media/uapi/v4l/extended-controls.rst
> b/Documentation/media/uapi/v4l/extended-controls.rst
> index 9f7312bf3365..1189750a022c 100644
> --- a/Documentation/media/uapi/v4l/extended-controls.rst
> +++ b/Documentation/media/uapi/v4l/extended-controls.rst
> @@ -3354,6 +3354,19 @@ JPEG Control IDs
> Specify which JPEG markers are included in compressed stream. This
> control is valid only for encoders.
>
> +.. _jpeg-quant-tables-control:
> +
> +``V4L2_CID_JPEG_LUMA_QUANTIZATION (__u8 matrix)``
> +Sets the luma quantization table to be used for encoding
> +or decoding a V4L2_PIX_FMT_JPEG_RAW format buffer. This table is
> +a 8x8-byte matrix, and it's expected to be in JPEG zigzag order,
> +as per the JPEG specification.
> +
> +``V4L2_CID_JPEG_CHROMA_QUANTIZATION (__u8 matrix)``
> +Sets the chroma quantization table to be used for encoding
> +or decoding a V4L2_PIX_FMT_JPEG_RAW format buffer. This table is
> +a 8x8-byte matrix, and it's expected to be in JPEG zigzag order,
> +as per the JPEG specification.
>
>
> .. flat-table::
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> b/drivers/media/v4l2-core/v4l2-ctrls.c
> index 6ab15f4a0fea..677af8069bfc 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -999,6 +999,8 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_JPEG_RESTART_INTERVAL:return "Restart Interval";
> case V4L2_CID_JPEG_COMPRESSION_QUALITY: return "Compression Quality";
> case V4L2_CID_JPEG_ACTIVE_MARKER: return "Active Markers";
> + case V4L2_CID_JPEG_LUMA_QUANTIZATION: return "Luminance Quantization
> Matrix";
> + case V4L2_CID_JPEG_CHROMA_QUANTIZATION: return "Chrominance
> Quantization Matrix";
>
> /* Image source controls */
> /* Keep the order of the 'case's the same as in v4l2-controls.h! */
> @@ -1286,6 +1288,17 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
> v4l2_ctrl_type *type,
> case V4L2_CID_DETECT_MD_REGION_GRID:
> *type = V4L2_CTRL_TYPE_U8;
> break;
> + case V4L2_CID_JPEG_LUMA_QUANTIZATION:
> + case V4L2_CID_JPEG_CHROMA_QUANTIZATION:
> + *min = *def = 0;
> + *max = 0xff;
> + *step = 1;
> + *type = V4L2_CTRL_TYPE_U8;
> + if (dims) {
> + memset(dims, 0, V4L2_CTRL_MAX_DIMS * sizeof(dims[0]));
> + dims[0] = dims[1] = 8;
> + }
> + break;
> case V4L2_CID_DETECT_MD_THRESHOLD_GRID:
> *type = V4L2_CTRL_TYPE_U16;
> break;
> diff --git a/include/uapi/linux/v4l2-controls.h
> b/include/uapi/linux/v4l2-controls.h
> index e4ee10ee917d..a466acf40625 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -987,6 +987,9 @@ enum v4l2_jpeg_chroma_subsampling {
> #define V4L2_JPEG_ACTIVE_MARKER_DQT (1 << 17)
> #define V4L2_JPEG_ACTIVE_MARKER_DHT (1 << 18)
>
> +#define V4L2_CID_JPEG_LUMA_QUANTIZATION
> (V4L2_CID_JPEG_CLASS_BASE + 5)
> +#define V4L2_CID_JPEG_CHROMA_QUANTIZATION(V4L2_CID_JPEG_CLASS_BASE + 6)
> +
>
> /* Image source controls */
> #define V4L2_CID_IMAGE_SOURCE_CLASS_BASE (V4L2_CTRL_CLASS_IMAGE_SOURCE |
> 0x900)
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [RFC] Request API and V4L2 capabilities
Hi, On Fri, 2018-08-24 at 09:29 +0200, Hans Verkuil wrote: > On 08/23/2018 07:37 PM, Nicolas Dufresne wrote: > > Le jeudi 23 août 2018 à 16:31 +0200, Hans Verkuil a écrit : > > > > I propose adding these capabilities: > > > > > > > > #define V4L2_BUF_CAP_HAS_REQUESTS 0x0001 > > > > #define V4L2_BUF_CAP_REQUIRES_REQUESTS0x0002 > > > > #define V4L2_BUF_CAP_HAS_MMAP 0x0100 > > > > #define V4L2_BUF_CAP_HAS_USERPTR 0x0200 > > > > #define V4L2_BUF_CAP_HAS_DMABUF 0x0400 > > > > > > I substituted SUPPORTS for HAS and dropped the REQUIRES_REQUESTS > > > capability. > > > As Tomasz mentioned, technically (at least for stateless codecs) you could > > > handle just one frame at a time without using requests. It's very > > > inefficient, > > > but it would work. > > > > I thought the request was providing a data structure to refer back to > > the frames, so each codec don't have to implement one. Do you mean that > > the framework will implicitly request requests in that mode ? or simply > > that there is no such helper ? > > Yes, that's done through controls as well. > > The idea would be that you set the necessary controls, queue a buffer to > the output queue, dequeue a buffer from the output queue, read back any > new state information and repeat the process. > > That said, I'm not sure if the cedrus driver for example can handle this > at the moment. It is also inefficient and it won't work if codecs require > more than one buffer in the queue for whatever reason. > > Tomasz, Paul, please correct me if I am wrong. I haven't tested decoding without requests, but I suppose it should work when submitting only one source buffer at a time. By the way, note that our VAAPI backend for the request API gets the slice data and metadata for each frame sequentially and we are not yet starting to fill the next request before the current one was completed (fences would probably help implement that). > In any case, I think we can do without this proposed capability since it is > simply a requirement when implementing the pixelformat for the stateless > codec that the Request API will be available and it should be documented > as such in the spec. Agreed. Cheers, Paul -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [RFC] Request API and V4L2 capabilities
Hi, On Wed, 2018-08-22 at 14:33 -0300, Ezequiel Garcia wrote: > On Wed, 2018-08-22 at 16:10 +0200, Paul Kocialkowski wrote: > > Hi, > > > > On Tue, 2018-08-21 at 17:52 +0900, Tomasz Figa wrote: > > > Hi Hans, Paul, > > > > > > On Mon, Aug 6, 2018 at 6:29 PM Paul Kocialkowski > > > wrote: > > > > > > > > On Mon, 2018-08-06 at 11:23 +0200, Hans Verkuil wrote: > > > > > On 08/06/2018 11:13 AM, Paul Kocialkowski wrote: > > > > > > Hi, > > > > > > > > > > > > On Mon, 2018-08-06 at 10:32 +0200, Hans Verkuil wrote: > > > > > > > On 08/06/2018 10:16 AM, Paul Kocialkowski wrote: > > > > > > > > On Sat, 2018-08-04 at 15:50 +0200, Hans Verkuil wrote: > > > > > > > > > Regarding point 3: I think this should be documented next to > > > > > > > > > the pixel format. I.e. > > > > > > > > > the MPEG-2 Slice format used by the stateless cedrus codec > > > > > > > > > requires the request API > > > > > > > > > and that two MPEG-2 controls (slice params and quantization > > > > > > > > > matrices) must be present > > > > > > > > > in each request. > > > > > > > > > > > > > > > > > > I am not sure a control flag (e.g. > > > > > > > > > V4L2_CTRL_FLAG_REQUIRED_IN_REQ) is needed here. > > > > > > > > > It's really implied by the fact that you use a stateless > > > > > > > > > codec. It doesn't help > > > > > > > > > generic applications like v4l2-ctl or qv4l2 either since in > > > > > > > > > order to support > > > > > > > > > stateless codecs they will have to know about the details of > > > > > > > > > these controls anyway. > > > > > > > > > > > > > > > > > > So I am inclined to say that it is not necessary to expose > > > > > > > > > this information in > > > > > > > > > the API, but it has to be documented together with the pixel > > > > > > > > > format documentation. > > > > > > > > > > > > > > > > I think this is affected by considerations about codec > > > > > > > > profile/level > > > > > > > > support. More specifically, some controls will only be required > > > > > > > > for > > > > > > > > supporting advanced codec profiles/levels, so they can only be > > > > > > > > explicitly marked with appropriate flags by the driver when the > > > > > > > > target > > > > > > > > profile/level is known. And I don't think it would be sane for > > > > > > > > userspace > > > > > > > > to explicitly set what profile/level it's aiming at. As a > > > > > > > > result, I > > > > > > > > don't think we can explicitly mark controls as required or > > > > > > > > optional. > > > > > > I'm not sure this is entirely true. The hardware may need to be > > > explicitly told what profile the video is. It may even not be the > > > hardware, but the driver itself too, given that the profile may imply > > > the CAPTURE pixel format, e.g. for VP9 profiles: > > > > > > profile 0 > > > color depth: 8 bit/sample, chroma subsampling: 4:2:0 > > > profile 1 > > > color depth: 8 bit, chroma subsampling: 4:2:0, 4:2:2, 4:4:4 > > > profile 2 > > > color depth: 10–12 bit, chroma subsampling: 4:2:0 > > > profile 3 > > > color depth: 10–12 bit, chroma subsampling: 4:2:0, 4:2:2, 4:4:4 > > > > > > (reference: https://en.wikipedia.org/wiki/VP9#Profiles) > > > > I think it would be fair to expect userspace to select the right > > destination format (and maybe have the driver error if there's a > > mismatch with the meta-data) instead of having the driver somewhat > > expose what format should be used. > > > > But maybe this would be an API violation, since all the enumerated > > formats are probably supposed to be selectable? > > > > We could also look at it the other way round and consider that selecting > > an exposed format is always legit, but t
Re: [RFC] Request API and V4L2 capabilities
Hi, On Wed, 2018-08-15 at 09:57 -0400, Nicolas Dufresne wrote: > Le lundi 06 août 2018 à 10:16 +0200, Paul Kocialkowski a écrit : > > Hi Hans and all, > > > > On Sat, 2018-08-04 at 15:50 +0200, Hans Verkuil wrote: > > > Hi all, > > > > > > While the Request API patch series addresses all the core API issues, > > > there > > > are some high-level considerations as well: > > > > > > 1) How can the application tell that the Request API is supported and for > > >which buffer types (capture/output) and pixel formats? > > > > > > 2) How can the application tell if the Request API is required as opposed > > > to being > > >optional? > > > > > > 3) Some controls may be required in each request, how to let userspace > > > know this? > > >Is it even necessary to inform userspace? > > > > > > 4) (For bonus points): How to let the application know which streaming > > > I/O modes > > >are available? That's never been possible before, but it would be very > > > nice > > >indeed if that's made explicit. > > > > Thanks for bringing up these considerations and questions, which perhaps > > cover the last missing bits for streamlined use of the request API by > > userspace. I would suggest another item, related to 3): > > > > 5) How can applications tell whether the driver supports a specific > > codec profile/level, not only for encoding but also for decoding? It's > > common for low-end embedded hardware to not support the most advanced > > profiles (e.g. H264 high profile). > > Hi Paul, after some discussion with Philip, he sent a proposal patch > that enables profile/level extended CID support to decoders too. The > control is made read-only, the point is not really the CID get/set but > that the controls allow enumerating the supported values. This seems > quite straightforward and easy to use. > > This enumeration is already provided this way some of the existing > sate-full encoders. Sounds great, thanks for looking into it! I looked for the patch in the list, but couldn't find it off-hand. Do you have a link to it? I would like to bind it to the Cedrus VPU driver eventually. Cheers, Paul > > > Since the Request API associates data with frame buffers it makes sense > > > to expose > > > this as a new capability field in struct v4l2_requestbuffers and struct > > > v4l2_create_buffers. > > > > > > The first struct has 2 reserved fields, the second has 8, so it's not a > > > problem to > > > take one for a capability field. Both structs also have a buffer type, so > > > we know > > > if this is requested for a capture or output buffer type. The pixel > > > format is known > > > in the driver, so HAS/REQUIRES_REQUESTS can be set based on that. I doubt > > > we'll have > > > drivers where the request caps would actually depend on the pixel format, > > > but it > > > theoretically possible. For both ioctls you can call them with count=0 at > > > the start > > > of the application. REQBUFS has of course the side-effect of deleting all > > > buffers, > > > but at the start of your application you don't have any yet. CREATE_BUFS > > > has no > > > side-effects. > > > > My initial thoughts on this point were to have flags exposed in > > v4l2_capability, but now that you're saying it, it does make sense for > > the flag to be associated with a buffer rather than the global device. > > > > In addition, I've heard of cases (IIRC it was some Rockchip platforms) > > where the platform has both stateless and stateful VPUs (I think it was > > stateless up to H264 and stateful for H265). This would allow supporting > > these two hardware blocks under the same video device (if that makes > > sense anyway). And even if there's no immediate need, it's always good > > to have this level of granularity (with little drawbacks). > > > > > I propose adding these capabilities: > > > > > > #define V4L2_BUF_CAP_HAS_REQUESTS 0x0001 > > > #define V4L2_BUF_CAP_REQUIRES_REQUESTS0x0002 > > > #define V4L2_BUF_CAP_HAS_MMAP 0x0100 > > > #define V4L2_BUF_CAP_HAS_USERPTR 0x0200 > > > #define V4L2_BUF_CAP_HAS_DMABUF 0x0400 > > > > > > If REQUIRES_REQUESTS is set, then HAS_REQUESTS is also set. > > > > > > At this time I think that REQUIRES_REQUESTS would only need to be set for > > > th
Re: [RFC] Request API and V4L2 capabilities
Hi, On Tue, 2018-08-21 at 17:52 +0900, Tomasz Figa wrote: > Hi Hans, Paul, > > On Mon, Aug 6, 2018 at 6:29 PM Paul Kocialkowski > wrote: > > > > On Mon, 2018-08-06 at 11:23 +0200, Hans Verkuil wrote: > > > On 08/06/2018 11:13 AM, Paul Kocialkowski wrote: > > > > Hi, > > > > > > > > On Mon, 2018-08-06 at 10:32 +0200, Hans Verkuil wrote: > > > > > On 08/06/2018 10:16 AM, Paul Kocialkowski wrote: > > > > > > On Sat, 2018-08-04 at 15:50 +0200, Hans Verkuil wrote: > > > > > > > Regarding point 3: I think this should be documented next to the > > > > > > > pixel format. I.e. > > > > > > > the MPEG-2 Slice format used by the stateless cedrus codec > > > > > > > requires the request API > > > > > > > and that two MPEG-2 controls (slice params and quantization > > > > > > > matrices) must be present > > > > > > > in each request. > > > > > > > > > > > > > > I am not sure a control flag (e.g. > > > > > > > V4L2_CTRL_FLAG_REQUIRED_IN_REQ) is needed here. > > > > > > > It's really implied by the fact that you use a stateless codec. > > > > > > > It doesn't help > > > > > > > generic applications like v4l2-ctl or qv4l2 either since in order > > > > > > > to support > > > > > > > stateless codecs they will have to know about the details of > > > > > > > these controls anyway. > > > > > > > > > > > > > > So I am inclined to say that it is not necessary to expose this > > > > > > > information in > > > > > > > the API, but it has to be documented together with the pixel > > > > > > > format documentation. > > > > > > > > > > > > I think this is affected by considerations about codec profile/level > > > > > > support. More specifically, some controls will only be required for > > > > > > supporting advanced codec profiles/levels, so they can only be > > > > > > explicitly marked with appropriate flags by the driver when the > > > > > > target > > > > > > profile/level is known. And I don't think it would be sane for > > > > > > userspace > > > > > > to explicitly set what profile/level it's aiming at. As a result, I > > > > > > don't think we can explicitly mark controls as required or optional. > > I'm not sure this is entirely true. The hardware may need to be > explicitly told what profile the video is. It may even not be the > hardware, but the driver itself too, given that the profile may imply > the CAPTURE pixel format, e.g. for VP9 profiles: > > profile 0 > color depth: 8 bit/sample, chroma subsampling: 4:2:0 > profile 1 > color depth: 8 bit, chroma subsampling: 4:2:0, 4:2:2, 4:4:4 > profile 2 > color depth: 10–12 bit, chroma subsampling: 4:2:0 > profile 3 > color depth: 10–12 bit, chroma subsampling: 4:2:0, 4:2:2, 4:4:4 > > (reference: https://en.wikipedia.org/wiki/VP9#Profiles) I think it would be fair to expect userspace to select the right destination format (and maybe have the driver error if there's a mismatch with the meta-data) instead of having the driver somewhat expose what format should be used. But maybe this would be an API violation, since all the enumerated formats are probably supposed to be selectable? We could also look at it the other way round and consider that selecting an exposed format is always legit, but that it implies passing a bitstream that matches it or the driver will error (because of an invalid bitstream passed, not because of a "wrong" selected format). As far as I understood, the profile/level information is there to indicate a set of supported features by the decoder, not as an information used for the decoding process. Each corresponding feature is enabled or not in the bitstream meta-data and that's all the information the decoder really needs. This is why I think that setting the profile/level explicitly is not justified by the nature of the process and adding it only for convenience or marking whether controls are optional doesn't seem justified at this point, in my opinion. > > > > > > I also like the idea that it should instead be implicit and that the > > > > > > documentation should detail which specific stateless metadata > > > > > > controls > > > > > > are required for a given profile/level. > &g
Re: [RFC] Request API and V4L2 capabilities
Hi, [...] On Wed, 2018-08-15 at 14:51 +0200, Maxime Jourdan wrote: > Hi Paul, I think we need to go deeper than just exposing the supported > profiles/levels and also include a way to query the CAPTURE pixel > formats that are supported for each profile. > > Maybe HEVC Main produces yuv420p but HEVC Main10 gives you > yuv420p10le. Maybe H.264 HiP produces NV12 but H.264 Hi422P produces > YUYV while also supporting down-sampling to NV12. Well, I think we're looking at this backwards. Userspace certainly known what destination format is relevant for the video, so it shouldn't have to query the driver about it except to check that the format is indeed supported. > I don't know the specifics of each platform and the only example I can > think of is the Amlogic HEVC decoder that can produce NV12 for Main, > but only outputs a compressed proprietary format for Main10. > > I unfortunately don't have an idea about how to implement that, but > I'll think about it. On the first generations of Allwinner platforms, we also have a vendor- specific format as output, that we expose with a dedicated format. There's no particular interfacing issue with that. Only that userspace has to be aware of the format and how to deal with it. Cheers, Paul > > Cheers, > > > > Paul > > > > -- > > Paul Kocialkowski, Bootlin (formerly Free Electrons) > > Embedded Linux and kernel engineering > > https://bootlin.com -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [RFC] Request API and V4L2 capabilities
On Mon, 2018-08-06 at 11:23 +0200, Hans Verkuil wrote: > On 08/06/2018 11:13 AM, Paul Kocialkowski wrote: > > Hi, > > > > On Mon, 2018-08-06 at 10:32 +0200, Hans Verkuil wrote: > > > On 08/06/2018 10:16 AM, Paul Kocialkowski wrote: > > > > On Sat, 2018-08-04 at 15:50 +0200, Hans Verkuil wrote: > > > > > Regarding point 3: I think this should be documented next to the > > > > > pixel format. I.e. > > > > > the MPEG-2 Slice format used by the stateless cedrus codec requires > > > > > the request API > > > > > and that two MPEG-2 controls (slice params and quantization matrices) > > > > > must be present > > > > > in each request. > > > > > > > > > > I am not sure a control flag (e.g. V4L2_CTRL_FLAG_REQUIRED_IN_REQ) is > > > > > needed here. > > > > > It's really implied by the fact that you use a stateless codec. It > > > > > doesn't help > > > > > generic applications like v4l2-ctl or qv4l2 either since in order to > > > > > support > > > > > stateless codecs they will have to know about the details of these > > > > > controls anyway. > > > > > > > > > > So I am inclined to say that it is not necessary to expose this > > > > > information in > > > > > the API, but it has to be documented together with the pixel format > > > > > documentation. > > > > > > > > I think this is affected by considerations about codec profile/level > > > > support. More specifically, some controls will only be required for > > > > supporting advanced codec profiles/levels, so they can only be > > > > explicitly marked with appropriate flags by the driver when the target > > > > profile/level is known. And I don't think it would be sane for userspace > > > > to explicitly set what profile/level it's aiming at. As a result, I > > > > don't think we can explicitly mark controls as required or optional. > > > > > > > > I also like the idea that it should instead be implicit and that the > > > > documentation should detail which specific stateless metadata controls > > > > are required for a given profile/level. > > > > > > > > As for controls validation, the approach followed in the Cedrus driver > > > > is to check that the most basic controls are filled and allow having > > > > missing controls for those that match advanced profiles. > > > > > > > > Since this approach feels somewhat generic enough to be applied to all > > > > stateless VPU drivers, maybe this should be made a helper in the > > > > framework? > > > > > > Sounds reasonable. Not sure if it will be in the first version, but it is > > > easy to add later. > > > > Definitely, I don't think this is such a high priority for now either. > > > > > > In addition, I see a need for exposing the maximum profile/level that > > > > the driver supports for decoding. I would suggest reusing the already- > > > > existing dedicated controls used for encoding for this purpose. For > > > > decoders, they would be used to expose the (read-only) maximum > > > > profile/level that is supported by the hardware and keep using them as a > > > > settable value in a range (matching the level of support) for encoders. > > > > > > > > This is necessary for userspace to determine whether a given video can > > > > be decoded in hardware or not. Instead of half-way decoding the video > > > > (ending up in funky results), this would easily allow skipping hardware > > > > decoding and e.g. falling back on software decoding. > > > > > > I think it might be better to expose this through new read-only bitmask > > > controls: i.e. a bitmask containing the supported profiles and levels. > > > > It seems that this is more or less what the coda driver is doing for > > decoding actually, although it uses a menu control between min/max > > supported profile/levels, with a mask to "blacklist" the unsupported > > values. Then, the V4L2_CTRL_FLAG_READ_ONLY flag is set to keep the > > control read-only. > > > > > Reusing the existing controls for a decoder is odd since there is not > > > really a concept of a 'current' value since you just want to report what > > > is supported. And I am not sure if all decoders can repo
Re: [RFC] Request API and V4L2 capabilities
Hi, On Mon, 2018-08-06 at 10:32 +0200, Hans Verkuil wrote: > On 08/06/2018 10:16 AM, Paul Kocialkowski wrote: > > On Sat, 2018-08-04 at 15:50 +0200, Hans Verkuil wrote: > > > Regarding point 3: I think this should be documented next to the pixel > > > format. I.e. > > > the MPEG-2 Slice format used by the stateless cedrus codec requires the > > > request API > > > and that two MPEG-2 controls (slice params and quantization matrices) > > > must be present > > > in each request. > > > > > > I am not sure a control flag (e.g. V4L2_CTRL_FLAG_REQUIRED_IN_REQ) is > > > needed here. > > > It's really implied by the fact that you use a stateless codec. It > > > doesn't help > > > generic applications like v4l2-ctl or qv4l2 either since in order to > > > support > > > stateless codecs they will have to know about the details of these > > > controls anyway. > > > > > > So I am inclined to say that it is not necessary to expose this > > > information in > > > the API, but it has to be documented together with the pixel format > > > documentation. > > > > I think this is affected by considerations about codec profile/level > > support. More specifically, some controls will only be required for > > supporting advanced codec profiles/levels, so they can only be > > explicitly marked with appropriate flags by the driver when the target > > profile/level is known. And I don't think it would be sane for userspace > > to explicitly set what profile/level it's aiming at. As a result, I > > don't think we can explicitly mark controls as required or optional. > > > > I also like the idea that it should instead be implicit and that the > > documentation should detail which specific stateless metadata controls > > are required for a given profile/level. > > > > As for controls validation, the approach followed in the Cedrus driver > > is to check that the most basic controls are filled and allow having > > missing controls for those that match advanced profiles. > > > > Since this approach feels somewhat generic enough to be applied to all > > stateless VPU drivers, maybe this should be made a helper in the > > framework? > > Sounds reasonable. Not sure if it will be in the first version, but it is > easy to add later. Definitely, I don't think this is such a high priority for now either. > > In addition, I see a need for exposing the maximum profile/level that > > the driver supports for decoding. I would suggest reusing the already- > > existing dedicated controls used for encoding for this purpose. For > > decoders, they would be used to expose the (read-only) maximum > > profile/level that is supported by the hardware and keep using them as a > > settable value in a range (matching the level of support) for encoders. > > > > This is necessary for userspace to determine whether a given video can > > be decoded in hardware or not. Instead of half-way decoding the video > > (ending up in funky results), this would easily allow skipping hardware > > decoding and e.g. falling back on software decoding. > > I think it might be better to expose this through new read-only bitmask > controls: i.e. a bitmask containing the supported profiles and levels. It seems that this is more or less what the coda driver is doing for decoding actually, although it uses a menu control between min/max supported profile/levels, with a mask to "blacklist" the unsupported values. Then, the V4L2_CTRL_FLAG_READ_ONLY flag is set to keep the control read-only. > Reusing the existing controls for a decoder is odd since there is not > really a concept of a 'current' value since you just want to report what > is supported. And I am not sure if all decoders can report the profile > or level that they detect. Is that really a problem when the READ_ONLY flag is set? I thought it was designed to fit this specific case, when the driver reports a value that userspace cannot affect. Otherwise, I agree that having a bitmask type would be a better fit, but I think it would be beneficial to keep the already-defined control and associated values, which implies using the menu control type for both encoders and decoders. If this is not an option, I would be in favour of adding per-codec read- only bitmask controls (e.g. for H264 something like V4L2_CID_MPEG_VIDEO_H264_PROFILE_SUPPORT) that expose the already- existing profile/level definitions as bit identifiers (a bit like coda is using them to craft a mask for the menu items to blacklist) for decoding only. What do you think? Cheers, Paul -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [RFC] Request API and V4L2 capabilities
pose this information in > the API, but it has to be documented together with the pixel format > documentation. I think this is affected by considerations about codec profile/level support. More specifically, some controls will only be required for supporting advanced codec profiles/levels, so they can only be explicitly marked with appropriate flags by the driver when the target profile/level is known. And I don't think it would be sane for userspace to explicitly set what profile/level it's aiming at. As a result, I don't think we can explicitly mark controls as required or optional. I also like the idea that it should instead be implicit and that the documentation should detail which specific stateless metadata controls are required for a given profile/level. As for controls validation, the approach followed in the Cedrus driver is to check that the most basic controls are filled and allow having missing controls for those that match advanced profiles. Since this approach feels somewhat generic enough to be applied to all stateless VPU drivers, maybe this should be made a helper in the framework? In addition, I see a need for exposing the maximum profile/level that the driver supports for decoding. I would suggest reusing the already- existing dedicated controls used for encoding for this purpose. For decoders, they would be used to expose the (read-only) maximum profile/level that is supported by the hardware and keep using them as a settable value in a range (matching the level of support) for encoders. This is necessary for userspace to determine whether a given video can be decoded in hardware or not. Instead of half-way decoding the video (ending up in funky results), this would easily allow skipping hardware decoding and e.g. falling back on software decoding. What do you think? Cheers, Paul -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
[PATCH v6 8/8] ARM: dts: sun8i-h3: Add Video Engine and reserved memory nodes
This adds nodes for the Video Engine and the associated reserved memory
for the H3. Up to 96 MiB of memory are dedicated to the CMA pool.
Signed-off-by: Paul Kocialkowski
---
arch/arm/boot/dts/sun8i-h3.dtsi | 25 +
1 file changed, 25 insertions(+)
diff --git a/arch/arm/boot/dts/sun8i-h3.dtsi b/arch/arm/boot/dts/sun8i-h3.dtsi
index c93f6be40533..c1375e72bb12 100644
--- a/arch/arm/boot/dts/sun8i-h3.dtsi
+++ b/arch/arm/boot/dts/sun8i-h3.dtsi
@@ -110,6 +110,20 @@
;
};
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ cma_pool: cma@4a00 {
+ compatible = "shared-dma-pool";
+ size = <0x600>;
+ alloc-ranges = <0x4a00 0x600>;
+ reusable;
+ linux,cma-default;
+ };
+ };
+
soc {
system-control@1c0 {
compatible = "allwinner,sun8i-h3-system-control",
@@ -134,6 +148,17 @@
};
};
+ video-codec@01c0e000 {
+ compatible = "allwinner,sun8i-h3-video-engine";
+ reg = <0x01c0e000 0x1000>;
+ clocks = < CLK_BUS_VE>, < CLK_VE>,
+< CLK_DRAM_VE>;
+ clock-names = "ahb", "mod", "ram";
+ resets = < RST_BUS_VE>;
+ interrupts = ;
+ allwinner,sram = <_sram 1>;
+ };
+
mali: gpu@1c4 {
compatible = "allwinner,sun8i-h3-mali", "arm,mali-400";
reg = <0x01c4 0x1>;
--
2.18.0
[PATCH v6 7/8] ARM: dts: sun8i-a33: Add Video Engine and reserved memory nodes
This adds nodes for the Video Engine and the associated reserved memory
for the A33. Up to 96 MiB of memory are dedicated to the CMA pool.
The VPU can only map the first 256 MiB of DRAM, so the reserved memory
pool has to be located in that area. Following Allwinner's decision in
downstream software, the last 96 MiB of the first 256 MiB of RAM are
reserved for this purpose.
Signed-off-by: Paul Kocialkowski
---
arch/arm/boot/dts/sun8i-a33.dtsi | 26 ++
1 file changed, 26 insertions(+)
diff --git a/arch/arm/boot/dts/sun8i-a33.dtsi b/arch/arm/boot/dts/sun8i-a33.dtsi
index 8d278ee001e9..a212fbee14bc 100644
--- a/arch/arm/boot/dts/sun8i-a33.dtsi
+++ b/arch/arm/boot/dts/sun8i-a33.dtsi
@@ -181,6 +181,21 @@
reg = <0x4000 0x8000>;
};
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ /* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+ cma_pool: cma@4a00 {
+ compatible = "shared-dma-pool";
+ size = <0x600>;
+ alloc-ranges = <0x4a00 0x600>;
+ reusable;
+ linux,cma-default;
+ };
+ };
+
sound: sound {
compatible = "simple-audio-card";
simple-audio-card,name = "sun8i-a33-audio";
@@ -245,6 +260,17 @@
};
};
+ video-codec@01c0e000 {
+ compatible = "allwinner,sun8i-a33-video-engine";
+ reg = <0x01c0e000 0x1000>;
+ clocks = < CLK_BUS_VE>, < CLK_VE>,
+< CLK_DRAM_VE>;
+ clock-names = "ahb", "mod", "ram";
+ resets = < RST_BUS_VE>;
+ interrupts = ;
+ allwinner,sram = <_sram 1>;
+ };
+
crypto: crypto-engine@1c15000 {
compatible = "allwinner,sun4i-a10-crypto";
reg = <0x01c15000 0x1000>;
--
2.18.0
[PATCH v6 3/8] dt-bindings: media: Document bindings for the Cedrus VPU driver
This adds a device-tree binding document that specifies the properties
used by the Cedurs VPU driver, as well as examples.
Signed-off-by: Paul Kocialkowski
Reviewed-by: Rob Herring
---
.../devicetree/bindings/media/cedrus.txt | 54 +++
1 file changed, 54 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/cedrus.txt
diff --git a/Documentation/devicetree/bindings/media/cedrus.txt
b/Documentation/devicetree/bindings/media/cedrus.txt
new file mode 100644
index ..a089a0c1ff05
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/cedrus.txt
@@ -0,0 +1,54 @@
+Device-tree bindings for the VPU found in Allwinner SoCs, referred to as the
+Video Engine (VE) in Allwinner literature.
+
+The VPU can only access the first 256 MiB of DRAM, that are DMA-mapped starting
+from the DRAM base. This requires specific memory allocation and handling.
+
+Required properties:
+- compatible : must be one of the following compatibles:
+ - "allwinner,sun4i-a10-video-engine"
+ - "allwinner,sun5i-a13-video-engine"
+ - "allwinner,sun7i-a20-video-engine"
+ - "allwinner,sun8i-a33-video-engine"
+ - "allwinner,sun8i-h3-video-engine"
+- reg : register base and length of VE;
+- clocks : list of clock specifiers, corresponding to entries in
+ the clock-names property;
+- clock-names : should contain "ahb", "mod" and "ram" entries;
+- resets : phandle for reset;
+- interrupts : VE interrupt number;
+- allwinner,sram : SRAM region to use with the VE.
+
+Optional properties:
+- memory-region: CMA pool to use for buffers allocation
instead of the
+ default CMA pool.
+
+Example:
+
+reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ /* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+ cma_pool: cma@4a00 {
+ compatible = "shared-dma-pool";
+ size = <0x600>;
+ alloc-ranges = <0x4a00 0x600>;
+ reusable;
+ linux,cma-default;
+ };
+};
+
+video-codec@1c0e000 {
+ compatible = "allwinner,sun7i-a20-video-engine";
+ reg = <0x01c0e000 0x1000>;
+
+ clocks = < CLK_AHB_VE>, < CLK_VE>,
+< CLK_DRAM_VE>;
+ clock-names = "ahb", "mod", "ram";
+
+ resets = < RST_VE>;
+ interrupts = ;
+ allwinner,sram = <_sram 1>;
+};
--
2.18.0
[PATCH v6 6/8] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes
This adds nodes for the Video Engine and the associated reserved memory
for the A20. Up to 96 MiB of memory are dedicated to the CMA pool.
The VPU can only map the first 256 MiB of DRAM, so the reserved memory
pool has to be located in that area. Following Allwinner's decision in
downstream software, the last 96 MiB of the first 256 MiB of RAM are
reserved for this purpose.
Signed-off-by: Paul Kocialkowski
---
arch/arm/boot/dts/sun7i-a20.dtsi | 26 ++
1 file changed, 26 insertions(+)
diff --git a/arch/arm/boot/dts/sun7i-a20.dtsi b/arch/arm/boot/dts/sun7i-a20.dtsi
index 38999d791cb5..55517b068009 100644
--- a/arch/arm/boot/dts/sun7i-a20.dtsi
+++ b/arch/arm/boot/dts/sun7i-a20.dtsi
@@ -161,6 +161,21 @@
reg = <0x4000 0x8000>;
};
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ /* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+ cma_pool: cma@4a00 {
+ compatible = "shared-dma-pool";
+ size = <0x600>;
+ alloc-ranges = <0x4a00 0x600>;
+ reusable;
+ linux,cma-default;
+ };
+ };
+
timer {
compatible = "arm,armv7-timer";
interrupts = ,
@@ -466,6 +481,17 @@
};
};
+ video-codec@1c0e000 {
+ compatible = "allwinner,sun7i-a20-video-engine";
+ reg = <0x01c0e000 0x1000>;
+ clocks = < CLK_AHB_VE>, < CLK_VE>,
+< CLK_DRAM_VE>;
+ clock-names = "ahb", "mod", "ram";
+ resets = < RST_VE>;
+ interrupts = ;
+ allwinner,sram = <_sram 1>;
+ };
+
mmc0: mmc@1c0f000 {
compatible = "allwinner,sun7i-a20-mmc";
reg = <0x01c0f000 0x1000>;
--
2.18.0
[PATCH v6 1/8] media: v4l: Add definitions for MPEG2 slice format and metadata
Stateless video decoding engines require both the MPEG slices and
associated metadata from the video stream in order to decode frames.
This introduces definitions for a new pixel format, describing buffers
with MPEG2 slice data, as well as a control structure for passing the
frame metadata to drivers.
This is based on work from both Florent Revest and Hugues Fruchet.
Signed-off-by: Paul Kocialkowski
---
.../media/uapi/v4l/extended-controls.rst | 122 ++
.../media/uapi/v4l/pixfmt-compressed.rst | 5 +
drivers/media/v4l2-core/v4l2-ctrls.c | 54
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/media/v4l2-ctrls.h| 18 ++-
include/uapi/linux/v4l2-controls.h| 43 ++
include/uapi/linux/videodev2.h| 5 +
7 files changed, 241 insertions(+), 7 deletions(-)
diff --git a/Documentation/media/uapi/v4l/extended-controls.rst
b/Documentation/media/uapi/v4l/extended-controls.rst
index 9f7312bf3365..4a29d89fd9ac 100644
--- a/Documentation/media/uapi/v4l/extended-controls.rst
+++ b/Documentation/media/uapi/v4l/extended-controls.rst
@@ -1497,6 +1497,128 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
+.. _v4l2-mpeg-mpeg2:
+
+``V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS (struct)``
+Specifies the slice parameters (also known as slice header) for the
+associated MPEG-2 slice data. This includes all the necessary
+parameters for configuring a hardware decoder pipeline for MPEG-2.
+
+.. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.0cm}|
+
+.. c:type:: v4l2_ctrl_mpeg2_slice_params
+
+.. cssclass:: longtable
+
+.. flat-table:: struct v4l2_ctrl_mpeg2_slice_params
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - __u32
+ - ``slice_len``
+ - Length (in bits) of the current slice data.
+* - __u32
+ - ``slice_pos``
+ - Position (in bits) of the current slice data, relative to the
+frame start.
+* - __u16
+ - ``width``
+ - Width of the corresponding output frame for the current slice.
+* - __u16
+ - ``height``
+ - Height of the corresponding output frame for the current slice.
+* - __u8
+ - ``slice_type``
+ - Picture coding type for the frame covered by the current slice
+(V4L2_MPEG2_SLICE_TYPE_I, V4L2_MPEG2_SLICE_TYPE_P or
+V4L2_MPEG2_SLICE_PCT_B).
+* - __u8
+ - ``f_code[2][2]``
+ - Motion vector codes.
+* - __u8
+ - ``intra_dc_precision``
+ - Precision of Discrete Cosine transform (0: 8 bits precision,
+1: 9 bits precision, 2: 10 bits precision, 11: 11 bits precision).
+* - __u8
+ - ``picture_structure``
+ - Picture structure (1: interlaced top field,
+2: interlaced bottom field, 3: progressive frame).
+* - __u8
+ - ``top_field_first``
+ - If set to 1 and interlaced stream, top field is output first.
+* - __u8
+ - ``frame_pred_frame_dct``
+ - If set to 1, only frame-DCT and frame prediction are used.
+* - __u8
+ - ``concealment_motion_vectors``
+ - If set to 1, motion vectors are coded for intra macroblocks.
+* - __u8
+ - ``q_scale_type``
+ - This flag affects the inverse quantisation process.
+* - __u8
+ - ``intra_vlc_format``
+ - This flag affects the decoding of transform coefficient data.
+* - __u8
+ - ``alternate_scan``
+ - This flag affects the decoding of transform coefficient data.
+* - __u8
+ - ``backward_ref_index``
+ - Index for the V4L2 buffer to use as backward reference, used with
+B-coded and P-coded frames.
+* - __u8
+ - ``forward_ref_index``
+ - Index for the V4L2 buffer to use as forward reference, used with
+P-coded frames.
+* - :cspan:`2`
+
+``V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION (struct)``
+Specifies quantization matrices for the associated MPEG-2 slice data.
+
+.. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.0cm}|
+
+.. c:type:: v4l2_ctrl_mpeg2_quantization
+
+.. cssclass:: longtable
+
+.. flat-table:: struct v4l2_ctrl_mpeg2_quantization
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - __u8
+ - ``load_intra_quantiser_matrix``
+ - One bit to indicate whether to load the intra quantiser matrix.
+* - __u32
+ - ``load_non_intra_quantiser_matrix``
+ - One bit to indicate whether to load the non-intra quantiser matrix.
+* - __u32
+ - ``load_chroma_intra_quantiser_matrix``
+ - One bit to indicate whether to load the chroma intra quantiser matrix,
+only relevant for non-4:2:0 YUV formats.
+* - __u32
+ - ``load_chroma_non_intra_quantiser_matrix``
+ - One bit to indicate whether to load the non-chroma intra quantiser
+matrix, only relevant for non-4:2:0 YUV formats.
+* - __u32
+ - ``intra_quantiser_matrix[64]``
+ - The intra quantiser matrix coefficients, in zigzag
[PATCH v6 4/8] media: platform: Add Cedrus VPU decoder driver
This introduces the Cedrus VPU driver that supports the VPU found in Allwinner SoCs, also known as Video Engine. It is implemented through a v4l2 m2m decoder device and a media device (used for media requests). So far, it only supports MPEG2 decoding. Since this VPU is stateless, synchronization with media requests is required in order to ensure consistency between frame headers that contain metadata about the frame to process and the raw slice data that is used to generate the frame. This driver was made possible thanks to the long-standing effort carried out by the linux-sunxi community in the interest of reverse engineering, documenting and implementing support for Allwinner VPU. Signed-off-by: Paul Kocialkowski --- MAINTAINERS | 7 + drivers/staging/media/Kconfig | 2 + drivers/staging/media/Makefile| 1 + drivers/staging/media/sunxi/Kconfig | 15 + drivers/staging/media/sunxi/Makefile | 1 + drivers/staging/media/sunxi/cedrus/Kconfig| 14 + drivers/staging/media/sunxi/cedrus/Makefile | 3 + drivers/staging/media/sunxi/cedrus/cedrus.c | 419 + drivers/staging/media/sunxi/cedrus/cedrus.h | 166 + .../staging/media/sunxi/cedrus/cedrus_dec.c | 114 .../staging/media/sunxi/cedrus/cedrus_dec.h | 27 + .../staging/media/sunxi/cedrus/cedrus_hw.c| 319 ++ .../staging/media/sunxi/cedrus/cedrus_hw.h| 29 + .../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 240 .../staging/media/sunxi/cedrus/cedrus_regs.h | 235 .../staging/media/sunxi/cedrus/cedrus_video.c | 566 ++ .../staging/media/sunxi/cedrus/cedrus_video.h | 31 + 17 files changed, 2189 insertions(+) create mode 100644 drivers/staging/media/sunxi/Kconfig create mode 100644 drivers/staging/media/sunxi/Makefile create mode 100644 drivers/staging/media/sunxi/cedrus/Kconfig create mode 100644 drivers/staging/media/sunxi/cedrus/Makefile create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus.c create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus.h create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_dec.c create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_dec.h create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_hw.c create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_hw.h create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_regs.h create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_video.c create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_video.h diff --git a/MAINTAINERS b/MAINTAINERS index 89853313c697..342504506a89 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -656,6 +656,13 @@ L: linux-cry...@vger.kernel.org S: Maintained F: drivers/crypto/sunxi-ss/ +ALLWINNER VPU DRIVER +M: Maxime Ripard +M: Paul Kocialkowski +L: linux-media@vger.kernel.org +S: Maintained +F: drivers/staging/media/sunxi/cedrus/ + ALPHA PORT M: Richard Henderson M: Ivan Kokshaysky diff --git a/drivers/staging/media/Kconfig b/drivers/staging/media/Kconfig index db5cf67047ad..b3620a8f2d9f 100644 --- a/drivers/staging/media/Kconfig +++ b/drivers/staging/media/Kconfig @@ -31,6 +31,8 @@ source "drivers/staging/media/mt9t031/Kconfig" source "drivers/staging/media/omap4iss/Kconfig" +source "drivers/staging/media/sunxi/Kconfig" + source "drivers/staging/media/tegra-vde/Kconfig" source "drivers/staging/media/zoran/Kconfig" diff --git a/drivers/staging/media/Makefile b/drivers/staging/media/Makefile index 503fbe47fa58..42948f805548 100644 --- a/drivers/staging/media/Makefile +++ b/drivers/staging/media/Makefile @@ -5,5 +5,6 @@ obj-$(CONFIG_SOC_CAMERA_IMX074) += imx074/ obj-$(CONFIG_SOC_CAMERA_MT9T031) += mt9t031/ obj-$(CONFIG_VIDEO_DM365_VPFE) += davinci_vpfe/ obj-$(CONFIG_VIDEO_OMAP4) += omap4iss/ +obj-$(CONFIG_VIDEO_SUNXI) += sunxi/ obj-$(CONFIG_TEGRA_VDE)+= tegra-vde/ obj-$(CONFIG_VIDEO_ZORAN) += zoran/ diff --git a/drivers/staging/media/sunxi/Kconfig b/drivers/staging/media/sunxi/Kconfig new file mode 100644 index ..c78d92240ceb --- /dev/null +++ b/drivers/staging/media/sunxi/Kconfig @@ -0,0 +1,15 @@ +config VIDEO_SUNXI + bool "Allwinner sunXi family Video Devices" + depends on ARCH_SUNXI || COMPILE_TEST + help + If you have an Allwinner SoC based on the sunXi family, say Y. + + Note that this option doesn't include new drivers in the + kernel: saying N will just cause Kconfig to skip all the + questions about Allwinner media devices. + +if VIDEO_SUNXI + +source "drivers/staging/media/sunxi/cedrus/Kconfig" + +endif diff --git a/drivers/staging/media/sunxi/Makefile b/drivers/staging/media/sunxi/Makefile new file mode 10064
[PATCH v6 0/8] Cedrus driver for the Allwinner Video Engine, using media requests
/9299073/ [1]: https://git.linuxtv.org/hverkuil/media_tree.git/log/?h=reqv16 Paul Kocialkowski (8): media: v4l: Add definitions for MPEG2 slice format and metadata media: v4l: Add definition for Allwinner's MB32-tiled NV12 format dt-bindings: media: Document bindings for the Cedrus VPU driver media: platform: Add Cedrus VPU decoder driver ARM: dts: sun5i: Add Video Engine and reserved memory nodes ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes ARM: dts: sun8i-a33: Add Video Engine and reserved memory nodes ARM: dts: sun8i-h3: Add Video Engine and reserved memory nodes .../devicetree/bindings/media/cedrus.txt | 54 ++ .../media/uapi/v4l/extended-controls.rst | 122 .../media/uapi/v4l/pixfmt-compressed.rst | 5 + .../media/uapi/v4l/pixfmt-reserved.rst| 15 +- MAINTAINERS | 7 + arch/arm/boot/dts/sun5i.dtsi | 26 + arch/arm/boot/dts/sun7i-a20.dtsi | 26 + arch/arm/boot/dts/sun8i-a33.dtsi | 26 + arch/arm/boot/dts/sun8i-h3.dtsi | 25 + drivers/media/v4l2-core/v4l2-ctrls.c | 54 ++ drivers/media/v4l2-core/v4l2-ioctl.c | 2 + drivers/staging/media/Kconfig | 2 + drivers/staging/media/Makefile| 1 + drivers/staging/media/sunxi/Kconfig | 15 + drivers/staging/media/sunxi/Makefile | 1 + drivers/staging/media/sunxi/cedrus/Kconfig| 14 + drivers/staging/media/sunxi/cedrus/Makefile | 3 + drivers/staging/media/sunxi/cedrus/cedrus.c | 419 + drivers/staging/media/sunxi/cedrus/cedrus.h | 166 + .../staging/media/sunxi/cedrus/cedrus_dec.c | 114 .../staging/media/sunxi/cedrus/cedrus_dec.h | 27 + .../staging/media/sunxi/cedrus/cedrus_hw.c| 319 ++ .../staging/media/sunxi/cedrus/cedrus_hw.h| 29 + .../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 240 .../staging/media/sunxi/cedrus/cedrus_regs.h | 235 .../staging/media/sunxi/cedrus/cedrus_video.c | 566 ++ .../staging/media/sunxi/cedrus/cedrus_video.h | 31 + include/media/v4l2-ctrls.h| 18 +- include/uapi/linux/v4l2-controls.h| 43 ++ include/uapi/linux/videodev2.h| 6 + 30 files changed, 2603 insertions(+), 8 deletions(-) create mode 100644 Documentation/devicetree/bindings/media/cedrus.txt create mode 100644 drivers/staging/media/sunxi/Kconfig create mode 100644 drivers/staging/media/sunxi/Makefile create mode 100644 drivers/staging/media/sunxi/cedrus/Kconfig create mode 100644 drivers/staging/media/sunxi/cedrus/Makefile create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus.c create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus.h create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_dec.c create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_dec.h create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_hw.c create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_hw.h create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_regs.h create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_video.c create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_video.h -- 2.18.0
[PATCH v6 2/8] media: v4l: Add definition for Allwinner's MB32-tiled NV12 format
This introduces support for Allwinner's MB32-tiled NV12 format, where
each plane is divided into macroblocks of 32x32 pixels. Hence, the size
of each plane has to be aligned to 32 bytes. The pixels inside each
macroblock are coded as they would be if the macroblock was a single
plane, line after line.
The MB32-tiled NV12 format is used by the video engine on Allwinner
platforms: it is the default format for decoded frames (and the only one
available in the oldest supported platforms).
Signed-off-by: Paul Kocialkowski
---
Documentation/media/uapi/v4l/pixfmt-reserved.rst | 15 ++-
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/videodev2.h | 1 +
3 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
index 38af1472a4b4..9a68b6a787bf 100644
--- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
@@ -243,7 +243,20 @@ please make a proposal on the linux-media mailing list.
It is an opaque intermediate format and the MDP hardware must be
used to convert ``V4L2_PIX_FMT_MT21C`` to ``V4L2_PIX_FMT_NV12M``,
``V4L2_PIX_FMT_YUV420M`` or ``V4L2_PIX_FMT_YVU420``.
-
+* .. _V4L2-PIX-FMT-MB32-NV12:
+
+ - ``V4L2_PIX_FMT_MB32_NV12``
+ - 'MN12'
+ - Two-planar NV12-based format used by the Allwinner video engine
+hardware, with 32x32 tiles for the luminance plane and 32x64 tiles
+for the chrominance plane. Each tile is a linear pixel data
+representation within its own bounds. Each tile follows the previous
+one linearly (as in, from left to right, top to bottom).
+
+The frame dimensions are aligned to match an integer number of
+tiles, resulting in 32-aligned resolutions for the luminance plane
+and 16-aligned resolutions for the chrominance plane (with 2x2
+subsampling).
.. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
index 68e914b83a03..7e1c200de10d 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1331,6 +1331,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_SE401:descr = "GSPCA SE401"; break;
case V4L2_PIX_FMT_S5C_UYVY_JPG: descr = "S5C73MX interleaved
UYVY/JPEG"; break;
case V4L2_PIX_FMT_MT21C:descr = "Mediatek Compressed
Format"; break;
+ case V4L2_PIX_FMT_MB32_NV12:descr = "Allwinner tiled NV12
format"; break;
default:
WARN(1, "Unknown pixelformat 0x%08x\n",
fmt->pixelformat);
if (fmt->description[0])
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index d171361ed9b3..453d27142e31 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -670,6 +670,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_Z16 v4l2_fourcc('Z', '1', '6', ' ') /* Depth data
16-bit */
#define V4L2_PIX_FMT_MT21Cv4l2_fourcc('M', 'T', '2', '1') /* Mediatek
compressed block mode */
#define V4L2_PIX_FMT_INZI v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar
Greyscale 10-bit and Depth 16-bit */
+#define V4L2_PIX_FMT_MB32_NV12 v4l2_fourcc('M', 'N', '1', '2') /* Allwinner
tiled NV12 format */
/* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits
unused */
#define V4L2_PIX_FMT_IPU3_SBGGR10 v4l2_fourcc('i', 'p', '3', 'b') /* IPU3
packed 10-bit BGGR bayer */
--
2.18.0
[PATCH v6 5/8] ARM: dts: sun5i: Add Video Engine and reserved memory nodes
This adds nodes for the Video Engine and the associated reserved memory
for sun5i-based platforms. Up to 96 MiB of memory are dedicated to the
CMA pool.
The VPU can only map the first 256 MiB of DRAM, so the reserved memory
pool has to be located in that area. Following Allwinner's decision in
downstream software, the last 96 MiB of the first 256 MiB of RAM are
reserved for this purpose.
Signed-off-by: Paul Kocialkowski
---
arch/arm/boot/dts/sun5i.dtsi | 26 ++
1 file changed, 26 insertions(+)
diff --git a/arch/arm/boot/dts/sun5i.dtsi b/arch/arm/boot/dts/sun5i.dtsi
index 51dcefc76c12..6a9d6d185ade 100644
--- a/arch/arm/boot/dts/sun5i.dtsi
+++ b/arch/arm/boot/dts/sun5i.dtsi
@@ -108,6 +108,21 @@
};
};
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ /* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+ cma_pool: cma@4a00 {
+ compatible = "shared-dma-pool";
+ size = <0x600>;
+ alloc-ranges = <0x4a00 0x600>;
+ reusable;
+ linux,cma-default;
+ };
+ };
+
soc@1c0 {
compatible = "simple-bus";
#address-cells = <1>;
@@ -295,6 +310,17 @@
};
};
+ video-codec@1c0e000 {
+ compatible = "allwinner,sun5i-a13-video-engine";
+ reg = <0x01c0e000 0x1000>;
+ clocks = < CLK_AHB_VE>, < CLK_VE>,
+< CLK_DRAM_VE>;
+ clock-names = "ahb", "mod", "ram";
+ resets = < RST_VE>;
+ interrupts = <53>;
+ allwinner,sram = <_sram 1>;
+ };
+
mmc0: mmc@1c0f000 {
compatible = "allwinner,sun5i-a13-mmc";
reg = <0x01c0f000 0x1000>;
--
2.18.0
Re: [PATCH] v4l2-ctrls: add v4l2_ctrl_request_hdl_find/put/ctrl_find functions
Hi,
On Wed, 2018-07-04 at 17:38 +0200, Hans Verkuil wrote:
> If a driver needs to find/inspect the controls set in a request then
> it can use these functions.
>
> E.g. to check if a required control is set in a request use this in the
> req_validate() implementation:
>
> int res = -EINVAL;
>
> hdl = v4l2_ctrl_request_hdl_find(req, parent_hdl);
> if (hdl) {
> if (v4l2_ctrl_request_hdl_ctrl_find(hdl, ctrl_id))
> res = 0;
> v4l2_ctrl_request_hdl_put(hdl);
> }
> return res;
Thanks again for pulling this off, that's definitely what I neeeded!
I was able to test the patch and get it to work with sunxi-cedrus, with
one modification highlighted below.
Note that for my use case, I need to have access to the driver's private
data (context) to get the parent hdl. I'm iterating over request objects
and getting the context from a buffer object (checked via
vb2_request_object_is_buffer), a bit like it's done in
vb2_m2m_request_queue.
> Signed-off-by: Hans Verkuil
> ---
> Paul, please test this. This should be what you need.
>
> Regards,
>
> Hans
> ---
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> b/drivers/media/v4l2-core/v4l2-ctrls.c
> index 3ff17c87b3c8..03fd140736fd 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -2976,6 +2976,31 @@ static const struct media_request_object_ops req_ops =
> {
> .release = v4l2_ctrl_request_release,
> };
>
> +struct v4l2_ctrl_handler *v4l2_ctrl_request_hdl_find(struct media_request
> *req,
> + struct v4l2_ctrl_handler *parent)
> +{
> + struct media_request_object *obj;
> +
> + if (WARN_ON(req->state != MEDIA_REQUEST_STATE_VALIDATING &&
> + req->state != MEDIA_REQUEST_STATE_QUEUED))
> + return NULL;
> +
> + obj = media_request_object_find(req, _ops, parent);
> + if (obj)
> + return container_of(obj, struct v4l2_ctrl_handler, req_obj);
> + return NULL;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_ctrl_request_hdl_find);
> +
> +struct v4l2_ctrl *
> +v4l2_ctrl_request_hdl_ctrl_find(struct v4l2_ctrl_handler *hdl, u32 id)
> +{
> + struct v4l2_ctrl_ref *ref = find_ref_lock(hdl, id);
> +
> + return (ref && ref->req == ref) ? ref : NULL;
When req && ref->req == ref, ref->ctrl should be returned instead of
ref.
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [PATCH v3 02/14] drivers: soc: sunxi: Add dedicated compatibles for the A13, A20 and A33
Hi,
On Fri, 2018-05-11 at 12:20 +0200, Maxime Ripard wrote:
> On Thu, May 10, 2018 at 10:05:33PM -0700, Chen-Yu Tsai wrote:
> > On Mon, May 7, 2018 at 5:44 AM, Paul Kocialkowski
> > wrote:
> > > This introduces platform-specific compatibles for the A13, A20 and A33
> > > SRAM driver. No particular adaptation for these platforms is required at
> > > this point, although this might become the case in the future.
> > >
> > > Signed-off-by: Paul Kocialkowski
> > > ---
> > > drivers/soc/sunxi/sunxi_sram.c | 3 +++
> > > 1 file changed, 3 insertions(+)
> > >
> > > diff --git a/drivers/soc/sunxi/sunxi_sram.c
> > > b/drivers/soc/sunxi/sunxi_sram.c
> > > index 74cb81f37bd6..43ebc3bd33f2 100644
> > > --- a/drivers/soc/sunxi/sunxi_sram.c
> > > +++ b/drivers/soc/sunxi/sunxi_sram.c
> > > @@ -315,6 +315,9 @@ static int sunxi_sram_probe(struct platform_device
> > > *pdev)
> > >
> > > static const struct of_device_id sunxi_sram_dt_match[] = {
> > > { .compatible = "allwinner,sun4i-a10-sram-controller" },
> > > + { .compatible = "allwinner,sun5i-a13-sram-controller" },
> > > + { .compatible = "allwinner,sun7i-a20-sram-controller" },
> > > + { .compatible = "allwinner,sun8i-a33-sram-controller" },
> >
> > We should probably name these "system-controller". Maxime?
Would you like me to make that change for v4 of this series, or have it
a separate follow-up patch outside of this series?
> This would make sense yes, but we don't really need to add the A20 one
> to the driver, it's exactly the same than the A10.
Noted, I'll ditch the a20 compatible from the next revison of the
series.
Thanks for the feedback,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [PATCH v3 00/14] Sunxi-Cedrus driver for the Allwinner Video Engine, using media requests
[...] On Mon, 2018-05-07 at 14:44 +0200, Paul Kocialkowski wrote: > Remaining tasks: > * cleaning up registers description and documenting the fields used; > * removing the assigned-clocks property and setting the clock rate > in the driver directly; > * checking the series with checkpatch and fixing warnings; > * documenting the MB32 NV12 format and adding it to v4l_fill_fmtdesc; > * reworking and documenting the MPEG2 header, then adding it to > v4l_fill_fmtdesc; > * checking and fixing the error paths; > * testing on more platforms. Another item for the tasks list that is not yet in this revision: * changing the id for V4L2_CTRL_TYPE_MPEG2_FRAME_HDR; Paul > Cheers! > > [0]: https://patchwork.kernel.org/patch/9299073/ > [1]: https://git.linuxtv.org/hverkuil/media_tree.git/log/?h=reqv13 > > Florent Revest (1): > media: v4l: Add definitions for MPEG2 frame format and header metadata > > Maxime Ripard (4): > drivers: soc: sunxi: Add support for the C1 SRAM region > ARM: sun5i: Add support for the C1 SRAM region with the SRAM > controller > ARM: sun7i-a20: Add support for the C1 SRAM region with the SRAM > controller > ARM: sun8i-a33: Add SRAM controller node and C1 SRAM region > > Paul Kocialkowski (9): > drivers: soc: sunxi: Add dedicated compatibles for the A13, A20 and > A33 > ARM: dts: sun5i: Use dedicated SRAM controller compatible > ARM: dts: sun7i-a20: Use dedicated SRAM controller compatible > media: v4l: Add definition for Allwinner's MB32-tiled NV12 format > dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver > media: platform: Add Sunxi-Cedrus VPU decoder driver > ARM: dts: sun5i: Add Video Engine and reserved memory nodes > ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes > ARM: dts: sun8i-a33: Add Video Engine and reserved memory nodes > > .../devicetree/bindings/media/sunxi-cedrus.txt | 58 +++ > MAINTAINERS| 7 + > arch/arm/boot/dts/sun5i.dtsi | 47 +- > arch/arm/boot/dts/sun7i-a20.dtsi | 47 +- > arch/arm/boot/dts/sun8i-a33.dtsi | 54 +++ > drivers/media/platform/Kconfig | 15 + > drivers/media/platform/Makefile| 1 + > drivers/media/platform/sunxi/cedrus/Makefile | 4 + > drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c | 333 ++ > .../platform/sunxi/cedrus/sunxi_cedrus_common.h| 128 ++ > .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.c | 188 > .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.h | 35 ++ > .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.c | 240 ++ > .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.h | 37 ++ > .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c | 160 +++ > .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h | 33 ++ > .../platform/sunxi/cedrus/sunxi_cedrus_regs.h | 175 +++ > .../platform/sunxi/cedrus/sunxi_cedrus_video.c | 505 > + > .../platform/sunxi/cedrus/sunxi_cedrus_video.h | 31 ++ > drivers/media/v4l2-core/v4l2-ctrls.c | 10 + > drivers/media/v4l2-core/v4l2-ioctl.c | 1 + > drivers/soc/sunxi/sunxi_sram.c | 13 + > include/uapi/linux/v4l2-controls.h | 26 ++ > include/uapi/linux/videodev2.h | 4 + > 24 files changed, 2150 insertions(+), 2 deletions(-) > create mode 100644 Documentation/devicetree/bindings/media/sunxi-cedrus.txt > create mode 100644 drivers/media/platform/sunxi/cedrus/Makefile > create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c > create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h > create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c > create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h > create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c > create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h > create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c > create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h > create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h > create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c > create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h > -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [PATCH v3 08/14] media: v4l: Add definitions for MPEG2 frame format and header metadata
Hi Hans, On Mon, 2018-05-07 at 15:49 +0200, Hans Verkuil wrote: > On 07/05/18 14:44, Paul Kocialkowski wrote: > > From: Florent Revest <florent.rev...@free-electrons.com> > > > > Stateless video decoding engines require both the MPEG slices and > > associated metadata from the video stream in order to decode frames. > > > > This introduces definitions for a new pixel format, describing > > buffers > > with MPEG2 slice data, as well as a control structure for passing > > the > > frame header (metadata) to drivers. Thanks for the review! I should have made it clear that this patch has not seen any improvement between v2 and v3. Cleaning up and documenting the MPEG2 headers is still in the tasks list (presented in the cover letter), as well as the MB32 NV12 format. > > Signed-off-by: Florent Revest <florent.rev...@free-electrons.com> > > Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com> > > --- > > drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++ > > drivers/media/v4l2-core/v4l2-ioctl.c | 1 + > > include/uapi/linux/v4l2-controls.h | 26 > > ++ > > include/uapi/linux/videodev2.h | 3 +++ > > 4 files changed, 40 insertions(+) > > > > diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c > > b/drivers/media/v4l2-core/v4l2-ctrls.c > > index df58a23eb731..cdf860c8e3d8 100644 > > --- a/drivers/media/v4l2-core/v4l2-ctrls.c > > +++ b/drivers/media/v4l2-core/v4l2-ctrls.c > > @@ -826,6 +826,7 @@ const char *v4l2_ctrl_get_name(u32 id) > > case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE: > > return "Vertical MV Search Range"; > > case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER: > > return "Repeat Sequence Header"; > > case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME: re > > turn "Force Key Frame"; > > + case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR: re > > turn "MPEG2 Frame Header"; > > This compound control needs to be documented in the V4l2 spec. > > > > > /* VPX controls */ > > case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS: > > return "VPX Number of Partitions"; > > @@ -1271,6 +1272,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, > > enum v4l2_ctrl_type *type, > > case V4L2_CID_RDS_TX_ALT_FREQS: > > *type = V4L2_CTRL_TYPE_U32; > > break; > > + case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR: > > + *type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR; > > + break; > > default: > > *type = V4L2_CTRL_TYPE_INTEGER; > > break; > > @@ -1591,6 +1595,9 @@ static int std_validate(const struct v4l2_ctrl > > *ctrl, u32 idx, > > return -ERANGE; > > return 0; > > > > + case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR: > > + return 0; > > + > > default: > > return -EINVAL; > > } > > @@ -2165,6 +2172,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct > > v4l2_ctrl_handler *hdl, > > case V4L2_CTRL_TYPE_U32: > > elem_size = sizeof(u32); > > break; > > + case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR: > > + elem_size = sizeof(struct > > v4l2_ctrl_mpeg2_frame_hdr); > > + break; > > default: > > if (type < V4L2_CTRL_COMPOUND_TYPES) > > elem_size = sizeof(s32); > > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c > > b/drivers/media/v4l2-core/v4l2-ioctl.c > > index 561a1fe3160b..38d318c47c55 100644 > > --- a/drivers/media/v4l2-core/v4l2-ioctl.c > > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c > > @@ -1274,6 +1274,7 @@ static void v4l_fill_fmtdesc(struct > > v4l2_fmtdesc *fmt) > > case V4L2_PIX_FMT_VP8: descr = > > "VP8"; break; > > case V4L2_PIX_FMT_VP9: descr = > > "VP9"; break; > > case V4L2_PIX_FMT_HEVC: descr = > > "HEVC"; break; /* aka H.265 */ > > + case V4L2_PIX_FMT_MPEG2_FRAME: descr = > > "MPEG2 Frame"; break; > > Needs to be documented in the spec. > > > case V4L2_PIX_FMT_CPIA1:descr = "GSPCA CPiA > > YUV"; break; > > case V4L2_PIX_FMT_WNVA: descr = > > "WNVA"; break; > > case V4L2_PIX_FMT_SN9C10X: descr = "GSPCA > > SN9C10X"; break; > > diff --git a/inc
[PATCH v3 04/14] ARM: dts: sun7i-a20: Use dedicated SRAM controller compatible
Use the newly-introduced SRAM controller compatible for the A20 instead
of its A10 fashion.
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
arch/arm/boot/dts/sun7i-a20.dtsi | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm/boot/dts/sun7i-a20.dtsi b/arch/arm/boot/dts/sun7i-a20.dtsi
index e529e4ff2174..b179e05408e4 100644
--- a/arch/arm/boot/dts/sun7i-a20.dtsi
+++ b/arch/arm/boot/dts/sun7i-a20.dtsi
@@ -240,7 +240,7 @@
ranges;
sram-controller@1c0 {
- compatible = "allwinner,sun4i-a10-sram-controller";
+ compatible = "allwinner,sun7i-a20-sram-controller";
reg = <0x01c0 0x30>;
#address-cells = <1>;
#size-cells = <1>;
--
2.16.3
[PATCH v3 00/14] Sunxi-Cedrus driver for the Allwinner Video Engine, using media requests
This presents a third iteration of the updated Sunxi-Cedrus driver, that supports the Video Engine found in most Allwinner SoCs, starting with the A10. It was tested on the A13, A20 and A33. The initial version of this driver[0] was originally written and submitted by Florent Revest using a previous version of the request API that is necessary to provide coherency between controls and the buffers they apply to. The driver was adapted to use the latest version of the media request API[1], as submitted by Hand Verkuil. Media request API support is a hard requirement for the Sunxi-Cedrus driver. The driver itself currently only supports MPEG2 and more codecs will be added to the driver eventually. The output frames provided by the Video Engine are in a multi-planar 32x32-tiled YUV format, with a plane for luminance (Y) and a plane for chrominance (UV). A specific format is introduced in the V4L2 API to describe it. This implementation is based on the significant work that was conducted by various members of the linux-sunxi community for understanding and documenting the Video Engine's innards. Changes since v2: * updated to version 13 of the media request API; * integrated various changes from Maxime Ripard; * reworked memory reservation to use CMA, dynamic allocation and allow DMABUF; * removed reserved memory binding since the CMA pool is the default one (and allow ENODEV in the driver, for that use case); * added SRAM controller support for the SRAM region used by the VE; * updated the device-tree bindings the for SRAM region; * added per-platform bindings; * added A13 support; * renamed VE node name and label; * fixed Florent's authorship for the MPEG2 headers; * added a MAINTAINERS entry. Changes since v1: * use the latest version of the request API for Hans Verkuil; * added media controller support and dependency * renamed v4l2 format to the more explicit V4L2_PIX_FMT_MB32_NV12; * reworked bindings documentation; * moved driver to drivers/media/platforms/sunxi/cedrus to pair with incoming CSI support ; * added a workqueue and lists to schedule buffer completion, since it cannot be done in interrupt context; * split mpeg2 support into a setup and a trigger function to avoid race condition; * split video-related ops to a dedicated sunxi_cedrus_video file; * cleaned up the included headers for each file; * used device PFN offset instead of subtracting PHYS_BASE; * used reset_control_reset instead of assert+deassert; * put the device in reset when removing driver; * changed dt bindings to use the last 96 Mib of the first 256 MiB of DRAM; * made it clear in the mpeg frame header structure that forward and backward indexes are used as reference frames for motion vectors; * lots of small cosmetic and consistency changes, including naming harmonization and headers text rework. Remaining tasks: * cleaning up registers description and documenting the fields used; * removing the assigned-clocks property and setting the clock rate in the driver directly; * checking the series with checkpatch and fixing warnings; * documenting the MB32 NV12 format and adding it to v4l_fill_fmtdesc; * reworking and documenting the MPEG2 header, then adding it to v4l_fill_fmtdesc; * checking and fixing the error paths; * testing on more platforms. Cheers! [0]: https://patchwork.kernel.org/patch/9299073/ [1]: https://git.linuxtv.org/hverkuil/media_tree.git/log/?h=reqv13 Florent Revest (1): media: v4l: Add definitions for MPEG2 frame format and header metadata Maxime Ripard (4): drivers: soc: sunxi: Add support for the C1 SRAM region ARM: sun5i: Add support for the C1 SRAM region with the SRAM controller ARM: sun7i-a20: Add support for the C1 SRAM region with the SRAM controller ARM: sun8i-a33: Add SRAM controller node and C1 SRAM region Paul Kocialkowski (9): drivers: soc: sunxi: Add dedicated compatibles for the A13, A20 and A33 ARM: dts: sun5i: Use dedicated SRAM controller compatible ARM: dts: sun7i-a20: Use dedicated SRAM controller compatible media: v4l: Add definition for Allwinner's MB32-tiled NV12 format dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver media: platform: Add Sunxi-Cedrus VPU decoder driver ARM: dts: sun5i: Add Video Engine and reserved memory nodes ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes ARM: dts: sun8i-a33: Add Video Engine and reserved memory nodes .../devicetree/bindings/media/sunxi-cedrus.txt | 58 +++ MAINTAINERS| 7 + arch/arm/boot/dts/sun5i.dtsi | 47 +- arch/arm/boot/dts/sun7i-a20.dtsi | 47 +- arch/arm/boot/dts/sun8i-a33.dtsi | 54 +++ drivers/media/platform/Kconfig | 15 + drivers/media/platform/Makefile| 1 + drivers/media/platform/sunxi/cedrus/Makefile | 4 + drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c | 333
[PATCH v3 12/14] ARM: dts: sun5i: Add Video Engine and reserved memory nodes
This adds nodes for the Video Engine and the associated reserved memory
for sun5i-based platforms. Up to 96 MiB of memory are dedicated to the
CMA pool.
The VPU can only map the first 256 MiB of DRAM, so the reserved memory
pool has to be located in that area. Following Allwinner's decision in
downstream software, the last 96 MiB of the first 256 MiB of RAM are
reserved for this purpose.
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
arch/arm/boot/dts/sun5i.dtsi | 31 +++
1 file changed, 31 insertions(+)
diff --git a/arch/arm/boot/dts/sun5i.dtsi b/arch/arm/boot/dts/sun5i.dtsi
index 6fa9e28edc59..306d265622e2 100644
--- a/arch/arm/boot/dts/sun5i.dtsi
+++ b/arch/arm/boot/dts/sun5i.dtsi
@@ -108,6 +108,21 @@
};
};
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ /* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+ cma_pool: cma@4a00 {
+ compatible = "shared-dma-pool";
+ size = <0x600>;
+ alloc-ranges = <0x4a00 0x600>;
+ reusable;
+ linux,cma-default;
+ };
+ };
+
soc@1c0 {
compatible = "simple-bus";
#address-cells = <1>;
@@ -292,6 +307,22 @@
};
};
+ vpu: video-codec@1c0e000 {
+ compatible = "allwinner,sun5i-a13-video-engine";
+ reg = <0x01c0e000 0x1000>;
+
+ clocks = < CLK_AHB_VE>, < CLK_VE>,
+< CLK_DRAM_VE>;
+ clock-names = "ahb", "mod", "ram";
+
+ assigned-clocks = < CLK_VE>;
+ assigned-clock-rates = <32000>;
+
+ resets = < RST_VE>;
+ interrupts = <53>;
+ allwinner,sram = <_sram 1>;
+ };
+
mmc0: mmc@1c0f000 {
compatible = "allwinner,sun5i-a13-mmc";
reg = <0x01c0f000 0x1000>;
--
2.16.3
[PATCH v3 01/14] drivers: soc: sunxi: Add support for the C1 SRAM region
From: Maxime Ripard <maxime.rip...@bootlin.com>
This introduces support for the SRAM C1 section, that is controlled by
the system controller. This SRAM area can be muxed either to the CPU
or the Video Engine, that needs this area to store various tables (e.g.
the Huffman VLD decoding tables).
This only supports devices with the same layout as the A10 (which also
includes the A13, A20, A33 and other SoCs).
Signed-off-by: Maxime Ripard <maxime.rip...@bootlin.com>
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
drivers/soc/sunxi/sunxi_sram.c | 10 ++
1 file changed, 10 insertions(+)
diff --git a/drivers/soc/sunxi/sunxi_sram.c b/drivers/soc/sunxi/sunxi_sram.c
index 882be5ed7e84..74cb81f37bd6 100644
--- a/drivers/soc/sunxi/sunxi_sram.c
+++ b/drivers/soc/sunxi/sunxi_sram.c
@@ -63,6 +63,12 @@ static struct sunxi_sram_desc sun4i_a10_sram_a3_a4 = {
SUNXI_SRAM_MAP(1, 1, "emac")),
};
+static struct sunxi_sram_desc sun4i_a10_sram_c1 = {
+ .data = SUNXI_SRAM_DATA("C1", 0x0, 0x0, 31,
+ SUNXI_SRAM_MAP(0, 0, "cpu"),
+ SUNXI_SRAM_MAP(0x7fff, 1, "ve")),
+};
+
static struct sunxi_sram_desc sun4i_a10_sram_d = {
.data = SUNXI_SRAM_DATA("D", 0x4, 0x0, 1,
SUNXI_SRAM_MAP(0, 0, "cpu"),
@@ -80,6 +86,10 @@ static const struct of_device_id sunxi_sram_dt_ids[] = {
.compatible = "allwinner,sun4i-a10-sram-a3-a4",
.data = _a10_sram_a3_a4.data,
},
+ {
+ .compatible = "allwinner,sun4i-a10-sram-c1",
+ .data = _a10_sram_c1.data,
+ },
{
.compatible = "allwinner,sun4i-a10-sram-d",
.data = _a10_sram_d.data,
--
2.16.3
[PATCH v3 11/14] media: platform: Add Sunxi-Cedrus VPU decoder driver
This introduces the Sunxi-Cedrus VPU driver that supports the VPU found in Allwinner SoCs, also known as Video Engine. It is implemented through a v4l2 m2m decoder device and a media device (used for media requests). So far, it only supports MPEG2 decoding. Since this VPU is stateless, synchronization with media requests is required in order to ensure consistency between frame headers that contain metadata about the frame to process and the raw slice data that is used to generate the frame. This driver was made possible thanks to the long-standing effort carried out by the linux-sunxi community in the interest of reverse engineering, documenting and implementing support for Allwinner VPU. Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com> --- MAINTAINERS| 7 + drivers/media/platform/Kconfig | 15 + drivers/media/platform/Makefile| 1 + drivers/media/platform/sunxi/cedrus/Makefile | 4 + drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c | 333 ++ .../platform/sunxi/cedrus/sunxi_cedrus_common.h| 128 ++ .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.c | 188 .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.h | 35 ++ .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.c | 240 ++ .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.h | 37 ++ .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c | 160 +++ .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h | 33 ++ .../platform/sunxi/cedrus/sunxi_cedrus_regs.h | 175 +++ .../platform/sunxi/cedrus/sunxi_cedrus_video.c | 505 + .../platform/sunxi/cedrus/sunxi_cedrus_video.h | 31 ++ 15 files changed, 1892 insertions(+) create mode 100644 drivers/media/platform/sunxi/cedrus/Makefile create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h diff --git a/MAINTAINERS b/MAINTAINERS index 79bb02ff812f..489f1dccc810 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -656,6 +656,13 @@ L: linux-cry...@vger.kernel.org S: Maintained F: drivers/crypto/sunxi-ss/ +ALLWINNER VPU DRIVER +M: Maxime Ripard <maxime.rip...@bootlin.com> +M: Paul Kocialkowski <paul.kocialkow...@bootlin.com> +L: linux-media@vger.kernel.org +S: Maintained +F: drivers/media/platform/sunxi/cedrus/ + ALPHA PORT M: Richard Henderson <r...@twiddle.net> M: Ivan Kokshaysky <i...@jurassic.park.msu.ru> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig index 5af07b620094..72d37cd2f7a2 100644 --- a/drivers/media/platform/Kconfig +++ b/drivers/media/platform/Kconfig @@ -476,6 +476,21 @@ config VIDEO_TI_VPE Support for the TI VPE(Video Processing Engine) block found on DRA7XX SoC. +config VIDEO_SUNXI_CEDRUS + tristate "Sunxi-Cedrus VPU driver" + depends on VIDEO_DEV && VIDEO_V4L2 && MEDIA_CONTROLLER + depends on ARCH_SUNXI + depends on HAS_DMA + select VIDEOBUF2_DMA_CONTIG + select MEDIA_REQUEST_API + select V4L2_MEM2MEM_DEV + ---help--- + Support for the VPU found in Allwinner SoCs, also known as the Cedar + video engine. + + To compile this driver as a module, choose M here: the module + will be called sunxi-cedrus. + config VIDEO_TI_VPE_DEBUG bool "VPE debug messages" depends on VIDEO_TI_VPE diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile index 932515df4477..444b995424a5 100644 --- a/drivers/media/platform/Makefile +++ b/drivers/media/platform/Makefile @@ -69,6 +69,7 @@ obj-$(CONFIG_VIDEO_ROCKCHIP_RGA) += rockchip/rga/ obj-y += omap/ obj-$(CONFIG_VIDEO_AM437X_VPFE)+= am437x/ +obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunxi/cedrus/ obj-$(CONFIG_VIDEO_XILINX) += xilinx/ diff --git a/drivers/media/platform/sunxi/cedrus/Makefile b/drivers/media/platform/sunxi/cedrus/Makefile new file mode 100644 index ..98f30df626a9 --- /dev/null +++ b/drivers/media/platform/sunxi/cedrus/Makefile @@ -0,0 +1,4 @@ +obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunx
[PATCH v3 13/14] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes
This adds nodes for the Video Engine and the associated reserved memory
for the A20. Up to 96 MiB of memory are dedicated to the CMA pool.
The VPU can only map the first 256 MiB of DRAM, so the reserved memory
pool has to be located in that area. Following Allwinner's decision in
downstream software, the last 96 MiB of the first 256 MiB of RAM are
reserved for this purpose.
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
arch/arm/boot/dts/sun7i-a20.dtsi | 31 +++
1 file changed, 31 insertions(+)
diff --git a/arch/arm/boot/dts/sun7i-a20.dtsi b/arch/arm/boot/dts/sun7i-a20.dtsi
index 9bb6c35fb850..5fff469b 100644
--- a/arch/arm/boot/dts/sun7i-a20.dtsi
+++ b/arch/arm/boot/dts/sun7i-a20.dtsi
@@ -161,6 +161,21 @@
reg = <0x4000 0x8000>;
};
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ /* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+ cma_pool: cma@4a00 {
+ compatible = "shared-dma-pool";
+ size = <0x600>;
+ alloc-ranges = <0x4a00 0x600>;
+ reusable;
+ linux,cma-default;
+ };
+ };
+
timer {
compatible = "arm,armv7-timer";
interrupts = ,
@@ -463,6 +478,22 @@
};
};
+ vpu: video-codec@1c0e000 {
+ compatible = "allwinner,sun7i-a20-video-engine";
+ reg = <0x01c0e000 0x1000>;
+
+ clocks = < CLK_AHB_VE>, < CLK_VE>,
+< CLK_DRAM_VE>;
+ clock-names = "ahb", "mod", "ram";
+
+ assigned-clocks = < CLK_VE>;
+ assigned-clock-rates = <32000>;
+
+ resets = < RST_VE>;
+ interrupts = ;
+ allwinner,sram = <_sram 1>;
+ };
+
mmc0: mmc@1c0f000 {
compatible = "allwinner,sun7i-a20-mmc";
reg = <0x01c0f000 0x1000>;
--
2.16.3
[PATCH v3 02/14] drivers: soc: sunxi: Add dedicated compatibles for the A13, A20 and A33
This introduces platform-specific compatibles for the A13, A20 and A33
SRAM driver. No particular adaptation for these platforms is required at
this point, although this might become the case in the future.
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
drivers/soc/sunxi/sunxi_sram.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/soc/sunxi/sunxi_sram.c b/drivers/soc/sunxi/sunxi_sram.c
index 74cb81f37bd6..43ebc3bd33f2 100644
--- a/drivers/soc/sunxi/sunxi_sram.c
+++ b/drivers/soc/sunxi/sunxi_sram.c
@@ -315,6 +315,9 @@ static int sunxi_sram_probe(struct platform_device *pdev)
static const struct of_device_id sunxi_sram_dt_match[] = {
{ .compatible = "allwinner,sun4i-a10-sram-controller" },
+ { .compatible = "allwinner,sun5i-a13-sram-controller" },
+ { .compatible = "allwinner,sun7i-a20-sram-controller" },
+ { .compatible = "allwinner,sun8i-a33-sram-controller" },
{ .compatible = "allwinner,sun50i-a64-sram-controller" },
{ },
};
--
2.16.3
[PATCH v3 14/14] ARM: dts: sun8i-a33: Add Video Engine and reserved memory nodes
This adds nodes for the Video Engine and the associated reserved memory
for the A33. Up to 96 MiB of memory are dedicated to the CMA pool.
The VPU can only map the first 256 MiB of DRAM, so the reserved memory
pool has to be located in that area. Following Allwinner's decision in
downstream software, the last 96 MiB of the first 256 MiB of RAM are
reserved for this purpose.
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
arch/arm/boot/dts/sun8i-a33.dtsi | 32
1 file changed, 32 insertions(+)
diff --git a/arch/arm/boot/dts/sun8i-a33.dtsi b/arch/arm/boot/dts/sun8i-a33.dtsi
index 97a976b3b1ef..5308a49601f6 100644
--- a/arch/arm/boot/dts/sun8i-a33.dtsi
+++ b/arch/arm/boot/dts/sun8i-a33.dtsi
@@ -181,6 +181,21 @@
reg = <0x4000 0x8000>;
};
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ /* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+ cma_pool: cma@4a00 {
+ compatible = "shared-dma-pool";
+ size = <0x600>;
+ alloc-ranges = <0x4a00 0x600>;
+ reusable;
+ linux,cma-default;
+ };
+ };
+
sound: sound {
compatible = "simple-audio-card";
simple-audio-card,name = "sun8i-a33-audio";
@@ -262,6 +277,23 @@
};
};
+ vpu: video-codec@01c0e000 {
+ compatible = "allwinner,sun8i-a33-video-engine";
+ reg = <0x01c0e000 0x1000>;
+
+ clocks = < CLK_BUS_VE>, < CLK_VE>,
+< CLK_DRAM_VE>;
+ clock-names = "ahb", "mod", "ram";
+
+ assigned-clocks = < CLK_VE>;
+ assigned-clock-rates = <32000>;
+
+ resets = < RST_BUS_VE>;
+
+ interrupts = ;
+ allwinner,sram = <_sram 1>;
+ };
+
crypto: crypto-engine@1c15000 {
compatible = "allwinner,sun4i-a10-crypto";
reg = <0x01c15000 0x1000>;
--
2.16.3
[PATCH v3 03/14] ARM: dts: sun5i: Use dedicated SRAM controller compatible
Use the newly-introduced SRAM controller compatible for the A13 and
other sun5i platforms instead of its A10 fashion.
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
arch/arm/boot/dts/sun5i.dtsi | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm/boot/dts/sun5i.dtsi b/arch/arm/boot/dts/sun5i.dtsi
index 07f2248ed5f8..72433f38b4e4 100644
--- a/arch/arm/boot/dts/sun5i.dtsi
+++ b/arch/arm/boot/dts/sun5i.dtsi
@@ -115,7 +115,7 @@
ranges;
sram-controller@1c0 {
- compatible = "allwinner,sun4i-a10-sram-controller";
+ compatible = "allwinner,sun5i-a13-sram-controller";
reg = <0x01c0 0x30>;
#address-cells = <1>;
#size-cells = <1>;
--
2.16.3
[PATCH v3 05/14] ARM: sun5i: Add support for the C1 SRAM region with the SRAM controller
From: Maxime Ripard <maxime.rip...@bootlin.com>
This adds support for the C1 SRAM region (to be used with the SRAM
controller driver) for sun5i-based platforms. The region is shared
between the Video Engine and the CPU.
Signed-off-by: Maxime Ripard <maxime.rip...@bootlin.com>
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
arch/arm/boot/dts/sun5i.dtsi | 14 ++
1 file changed, 14 insertions(+)
diff --git a/arch/arm/boot/dts/sun5i.dtsi b/arch/arm/boot/dts/sun5i.dtsi
index 72433f38b4e4..6fa9e28edc59 100644
--- a/arch/arm/boot/dts/sun5i.dtsi
+++ b/arch/arm/boot/dts/sun5i.dtsi
@@ -135,6 +135,20 @@
status = "disabled";
};
+ sram_c: sram@1d0 {
+ compatible = "mmio-sram";
+ reg = <0x01d0 0xd0>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges = <0 0x01d0 0xd0>;
+
+ ve_sram: sram-section@0 {
+ compatible =
"allwinner,sun5i-a13-sram-c1",
+
"allwinner,sun4i-a10-sram-c1";
+ reg = <0x00 0x8>;
+ };
+ };
+
sram_d: sram@1 {
compatible = "mmio-sram";
reg = <0x0001 0x1000>;
--
2.16.3
[PATCH v3 09/14] media: v4l: Add definition for Allwinner's MB32-tiled NV12 format
This introduces support for Allwinner's MB32-tiled NV12 format, where
each plane is divided into macroblocks of 32x32 pixels. Hence, the size
of each plane has to be aligned to 32 bytes. The pixels inside each
macroblock are coded as they would be if the macroblock was a single
plane, line after line.
The MB32-tiled NV12 format is used by the video engine on Allwinner
platforms: it is the default format for decoded frames (and the only one
available in the oldest supported platforms).
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
include/uapi/linux/videodev2.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index d8f9b59d90d7..242a6bfa1440 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -670,6 +670,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_Z16 v4l2_fourcc('Z', '1', '6', ' ') /* Depth data
16-bit */
#define V4L2_PIX_FMT_MT21Cv4l2_fourcc('M', 'T', '2', '1') /* Mediatek
compressed block mode */
#define V4L2_PIX_FMT_INZI v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar
Greyscale 10-bit and Depth 16-bit */
+#define V4L2_PIX_FMT_MB32_NV12 v4l2_fourcc('M', 'N', '1', '2') /* Allwinner
NV12 in 32x32 macroblocks */
/* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits
unused */
#define V4L2_PIX_FMT_IPU3_SBGGR10 v4l2_fourcc('i', 'p', '3', 'b') /* IPU3
packed 10-bit BGGR bayer */
--
2.16.3
[PATCH v3 07/14] ARM: sun8i-a33: Add SRAM controller node and C1 SRAM region
From: Maxime Ripard <maxime.rip...@bootlin.com>
This adds a SRAM controller node for the A33, with support for the C1
SRAM region that is shared between the Video Engine and the CPU.
Signed-off-by: Maxime Ripard <maxime.rip...@bootlin.com>
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
arch/arm/boot/dts/sun8i-a33.dtsi | 22 ++
1 file changed, 22 insertions(+)
diff --git a/arch/arm/boot/dts/sun8i-a33.dtsi b/arch/arm/boot/dts/sun8i-a33.dtsi
index a21f2ed07a52..97a976b3b1ef 100644
--- a/arch/arm/boot/dts/sun8i-a33.dtsi
+++ b/arch/arm/boot/dts/sun8i-a33.dtsi
@@ -204,6 +204,28 @@
};
soc@1c0 {
+ sram-controller@1c0 {
+ compatible = "allwinner,sun8i-a33-sram-controller";
+ reg = <0x01c0 0x30>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ sram_c: sram@1d0 {
+ compatible = "mmio-sram";
+ reg = <0x01d0 0xd0>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges = <0 0x01d0 0xd0>;
+
+ ve_sram: sram-section@0 {
+ compatible =
"allwinner,sun8i-a33-sram-c1",
+
"allwinner,sun4i-a10-sram-c1";
+ reg = <0x00 0x8>;
+ };
+ };
+ };
+
tcon0: lcd-controller@1c0c000 {
compatible = "allwinner,sun8i-a33-tcon";
reg = <0x01c0c000 0x1000>;
--
2.16.3
[PATCH v3 10/14] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver
This adds a device-tree binding document that specifies the properties
used by the Sunxi-Cedurs VPU driver, as well as examples.
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
.../devicetree/bindings/media/sunxi-cedrus.txt | 58 ++
1 file changed, 58 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/sunxi-cedrus.txt
diff --git a/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
new file mode 100644
index ..4c3f2b596ded
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
@@ -0,0 +1,58 @@
+Device-tree bindings for the VPU found in Allwinner SoCs, referred to as the
+Video Engine (VE) in Allwinner literature.
+
+The VPU can only access the first 256 MiB of DRAM, that are DMA-mapped starting
+from the DRAM base. This requires specific memory allocation and handling.
+
+Required properties:
+- compatible : must be one of the following compatibles:
+ - "allwinner,sun4i-a10-video-engine"
+ - "allwinner,sun5i-a13-video-engine"
+ - "allwinner,sun7i-a20-video-engine"
+ - "allwinner,sun8i-a33-video-engine"
+- reg : register base and length of VE;
+- clocks : list of clock specifiers, corresponding to entries in
+ the clock-names property;
+- clock-names : should contain "ahb", "mod" and "ram" entries;
+- assigned-clocks : list of clocks assigned to the VE;
+- assigned-clocks-rates: list of clock rates for the clocks assigned
to the VE;
+- resets : phandle for reset;
+- interrupts : VE interrupt number;
+- allwinner,sram : SRAM region to use with the VE.
+
+Optional properties:
+- memory-region: CMA pool to use for buffers allocation
instead of the
+ default CMA pool.
+
+Example:
+
+reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ /* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+ cma_pool: cma@4a00 {
+ compatible = "shared-dma-pool";
+ size = <0x600>;
+ alloc-ranges = <0x4a00 0x600>;
+ reusable;
+ linux,cma-default;
+ };
+};
+
+video-codec@1c0e000 {
+ compatible = "allwinner,sun7i-a20-video-engine";
+ reg = <0x01c0e000 0x1000>;
+
+ clocks = < CLK_AHB_VE>, < CLK_VE>,
+< CLK_DRAM_VE>;
+ clock-names = "ahb", "mod", "ram";
+
+ assigned-clocks = < CLK_VE>;
+ assigned-clock-rates = <32000>;
+
+ resets = < RST_VE>;
+ interrupts = ;
+ allwinner,sram = <_sram 1>;
+};
--
2.16.3
[PATCH v3 06/14] ARM: sun7i-a20: Add support for the C1 SRAM region with the SRAM controller
From: Maxime Ripard <maxime.rip...@bootlin.com>
This adds support for the C1 SRAM region (to be used with the SRAM
controller driver) for the A20 platform. The region is shared
between the Video Engine and the CPU.
Signed-off-by: Maxime Ripard <maxime.rip...@bootlin.com>
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
arch/arm/boot/dts/sun7i-a20.dtsi | 14 ++
1 file changed, 14 insertions(+)
diff --git a/arch/arm/boot/dts/sun7i-a20.dtsi b/arch/arm/boot/dts/sun7i-a20.dtsi
index b179e05408e4..9bb6c35fb850 100644
--- a/arch/arm/boot/dts/sun7i-a20.dtsi
+++ b/arch/arm/boot/dts/sun7i-a20.dtsi
@@ -273,6 +273,20 @@
status = "disabled";
};
};
+
+ sram_c: sram@1d0 {
+ compatible = "mmio-sram";
+ reg = <0x01d0 0xd0>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges = <0 0x01d0 0xd0>;
+
+ ve_sram: sram-section@0 {
+ compatible =
"allwinner,sun7i-a20-sram-c1",
+
"allwinner,sun4i-a10-sram-c1";
+ reg = <0x00 0x8>;
+ };
+ };
};
nmi_intc: interrupt-controller@1c00030 {
--
2.16.3
[PATCH v3 08/14] media: v4l: Add definitions for MPEG2 frame format and header metadata
From: Florent Revest <florent.rev...@free-electrons.com>
Stateless video decoding engines require both the MPEG slices and
associated metadata from the video stream in order to decode frames.
This introduces definitions for a new pixel format, describing buffers
with MPEG2 slice data, as well as a control structure for passing the
frame header (metadata) to drivers.
Signed-off-by: Florent Revest <florent.rev...@free-electrons.com>
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/v4l2-controls.h | 26 ++
include/uapi/linux/videodev2.h | 3 +++
4 files changed, 40 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
b/drivers/media/v4l2-core/v4l2-ctrls.c
index df58a23eb731..cdf860c8e3d8 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -826,6 +826,7 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE: return
"Vertical MV Search Range";
case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER: return "Repeat
Sequence Header";
case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME: return "Force
Key Frame";
+ case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR: return "MPEG2
Frame Header";
/* VPX controls */
case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:return "VPX
Number of Partitions";
@@ -1271,6 +1272,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
v4l2_ctrl_type *type,
case V4L2_CID_RDS_TX_ALT_FREQS:
*type = V4L2_CTRL_TYPE_U32;
break;
+ case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
+ *type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
+ break;
default:
*type = V4L2_CTRL_TYPE_INTEGER;
break;
@@ -1591,6 +1595,9 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32
idx,
return -ERANGE;
return 0;
+ case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
+ return 0;
+
default:
return -EINVAL;
}
@@ -2165,6 +2172,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct
v4l2_ctrl_handler *hdl,
case V4L2_CTRL_TYPE_U32:
elem_size = sizeof(u32);
break;
+ case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
+ elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
+ break;
default:
if (type < V4L2_CTRL_COMPOUND_TYPES)
elem_size = sizeof(s32);
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
index 561a1fe3160b..38d318c47c55 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1274,6 +1274,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_VP8: descr = "VP8"; break;
case V4L2_PIX_FMT_VP9: descr = "VP9"; break;
case V4L2_PIX_FMT_HEVC: descr = "HEVC"; break; /* aka
H.265 */
+ case V4L2_PIX_FMT_MPEG2_FRAME: descr = "MPEG2 Frame"; break;
case V4L2_PIX_FMT_CPIA1:descr = "GSPCA CPiA YUV"; break;
case V4L2_PIX_FMT_WNVA: descr = "WNVA"; break;
case V4L2_PIX_FMT_SN9C10X: descr = "GSPCA SN9C10X"; break;
diff --git a/include/uapi/linux/v4l2-controls.h
b/include/uapi/linux/v4l2-controls.h
index 8d473c979b61..23da8bfa7e6f 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
};
#define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL (V4L2_CID_MPEG_BASE+407)
+#define V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR (V4L2_CID_MPEG_BASE+450)
+
/* Control IDs for VP8 streams
* Although VP8 is not part of MPEG we add these controls to the MPEG class
* as that class is already handling other video compression standards
@@ -1076,4 +1078,28 @@ enum v4l2_detect_md_mode {
#define V4L2_CID_DETECT_MD_THRESHOLD_GRID (V4L2_CID_DETECT_CLASS_BASE + 3)
#define V4L2_CID_DETECT_MD_REGION_GRID (V4L2_CID_DETECT_CLASS_BASE + 4)
+struct v4l2_ctrl_mpeg2_frame_hdr {
+ __u32 slice_len;
+ __u32 slice_pos;
+ enum { MPEG1, MPEG2 } type;
+
+ __u16 width;
+ __u16 height;
+
+ enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
+ __u8 f_code[2][2];
+
+ __u8 intra_dc_precision;
+ __u8 picture_structure;
+ __u8 top_field_first;
+ __u8 frame_pred_frame_dct;
+ __u8 concealment_motion_vectors;
+ __u8 q_scale_type;
+ __u8 intra_vlc_format;
+ __u8
Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes
On Fri, 2018-05-04 at 15:40 +0200, Maxime Ripard wrote: > On Fri, May 04, 2018 at 02:04:38PM +0200, Paul Kocialkowski wrote: > > On Fri, 2018-05-04 at 11:15 +0200, Maxime Ripard wrote: > > > On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote: > > > > > > > > + reg = <0x01c0e000 0x1000>; > > > > > > > > + memory-region = <_memory>; > > > > > > > > > > > > > > Since you made the CMA region the default one, you don't > > > > > > > need > > > > > > > to > > > > > > > tie > > > > > > > it to that device in particular (and you can drop it being > > > > > > > mandatory > > > > > > > from your binding as well). > > > > > > > > > > > > What if another driver (or the system) claims memory from > > > > > > that > > > > > > zone > > > > > > and > > > > > > that the reserved memory ends up not being available for the > > > > > > VPU > > > > > > anymore? > > > > > > > > > > > > Acccording to the reserved-memory documentation, the > > > > > > reusable > > > > > > property > > > > > > (that we need for dmabuf) puts a limitation that the device > > > > > > driver > > > > > > owning the region must be able to reclaim it back. > > > > > > > > > > > > How does that work out if the CMA region is not tied to a > > > > > > driver > > > > > > in > > > > > > particular? > > > > > > > > > > I'm not sure to get what you're saying. You have the property > > > > > linux,cma-default in your reserved region, so the behaviour > > > > > you > > > > > described is what you explicitly asked for. > > > > > > > > My point is that I don't see how the driver can claim back (part > > > > of) > > > > the > > > > reserved area if the area is not explicitly attached to it. > > > > > > > > Or is that mechanism made in a way that all drivers wishing to > > > > use > > > > the > > > > reserved memory area can claim it back from the system, but > > > > there is > > > > no > > > > priority (other than first-come first-served) for which drivers > > > > claims > > > > it back in case two want to use the same reserved region (in a > > > > scenario > > > > where there isn't enough memory to allow both drivers)? > > > > > > This is indeed what happens. Reusable is to let the system use the > > > reserved memory for things like caches that can easily be dropped > > > when > > > a driver wants to use the memory in that reserved area. Once that > > > memory has been allocated, there's no claiming back, unless that > > > memory segment was freed of course. > > > > Thanks for the clarification. So in our case, perhaps the best fit > > would > > be to make that area the default CMA pool so that we can be ensured > > that > > the whole 96 MiB is available for the VPU and that no other consumer > > of > > CMA will use it? > > The best fit for what use case ? We already discussed this, and I > don't see any point in having two separate CMA regions. If you have a > reasonably sized region that will accomodate for both the VPU and > display engine, why would we want to split them? The use case I have in mind is boilerplate use of the VPU with the display engine, say with DMAbuf. It wasn't exactly clear in my memory whether we had decided that the CMA pool we use for the VPU should also be used for other CMA consumers (I realize that this is what we've been doing all along by having linux,cma-default; though). The fact that the memory region will accomodate for both the display engine and the VPU is not straightforward IMO and I think this has to be an explicit choice that we take. I was under the impression that we chose the 96 MiB because that's what the Allwinner reference code does. Does the reference code also use this pool for display? I liked the idea of having 96 MiB fully reserved to the VPU because it allows us to provide a limit on the use case, such as "this guarantees N buffers for resolution foo in format bar". If the display engine also uses it, then the limit also depends on how many GEM buffers are allocated (unless I'm missing something). > Or did you have any experience of running out of buffers? Not yet, I haven't. I have no objection with making the reserved region the default CMA pool and have other consumers use it too. Cheers, -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes
Hi, On Fri, 2018-05-04 at 11:15 +0200, Maxime Ripard wrote: > On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote: > > > > > > + reg = <0x01c0e000 0x1000>; > > > > > > + memory-region = <_memory>; > > > > > > > > > > Since you made the CMA region the default one, you don't need > > > > > to > > > > > tie > > > > > it to that device in particular (and you can drop it being > > > > > mandatory > > > > > from your binding as well). > > > > > > > > What if another driver (or the system) claims memory from that > > > > zone > > > > and > > > > that the reserved memory ends up not being available for the VPU > > > > anymore? > > > > > > > > Acccording to the reserved-memory documentation, the reusable > > > > property > > > > (that we need for dmabuf) puts a limitation that the device > > > > driver > > > > owning the region must be able to reclaim it back. > > > > > > > > How does that work out if the CMA region is not tied to a driver > > > > in > > > > particular? > > > > > > I'm not sure to get what you're saying. You have the property > > > linux,cma-default in your reserved region, so the behaviour you > > > described is what you explicitly asked for. > > > > My point is that I don't see how the driver can claim back (part of) > > the > > reserved area if the area is not explicitly attached to it. > > > > Or is that mechanism made in a way that all drivers wishing to use > > the > > reserved memory area can claim it back from the system, but there is > > no > > priority (other than first-come first-served) for which drivers > > claims > > it back in case two want to use the same reserved region (in a > > scenario > > where there isn't enough memory to allow both drivers)? > > This is indeed what happens. Reusable is to let the system use the > reserved memory for things like caches that can easily be dropped when > a driver wants to use the memory in that reserved area. Once that > memory has been allocated, there's no claiming back, unless that > memory segment was freed of course. Thanks for the clarification. So in our case, perhaps the best fit would be to make that area the default CMA pool so that we can be ensured that the whole 96 MiB is available for the VPU and that no other consumer of CMA will use it? Cheers, -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes
On Fri, 2018-05-04 at 10:47 +0200, Paul Kocialkowski wrote: > > > > Don't you also need to map the SRAM on the A20? > > > > > > That's a good point, there is currently no syscon handle for A20 > > > (and > > > also A13). Maybe SRAM is muxed to the VE by default so it "just > > > works"? I just checked on the manual and it appears that SRAM Area C1 is muxed to the VE at reset, so we can probably keep things as-is until the SRAM driver is ready to handle explicitly muxing that area to the VE. -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata
On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> Stateless video decoding engines require both the MPEG slices and
> associated metadata from the video stream in order to decode frames.
>
> This introduces definitions for a new pixel format, describing buffers
> with MPEG2 slice data, as well as a control structure for passing the
> frame header (metadata) to drivers.
While working on this, I came accross Hugues Fruchet's series that also
adds similar definitions for parsed MPEG2 metadata:
https://patchwork.kernel.org/patch/9704707/
Since that version made it to a v6, I will take the time to read the
discussion and see what needs to be changed in my proposal, so that we
can avoid discussing the same points over a year later.
This will most likely not make it to the next revision of the driver
series, so I will keep the format/controls definitions in their v2 state
(despite all the useful comments received) and take the time to properly
rework things in a future revision.
Cheers,
Paul
> Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
> Signed-off-by: Florent Revest <florent.rev...@free-electrons.com>
> ---
> drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> include/uapi/linux/v4l2-controls.h | 26 ++
> include/uapi/linux/videodev2.h | 3 +++
> 4 files changed, 40 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> b/drivers/media/v4l2-core/v4l2-ctrls.c
> index ba05a8b9a095..fcdc12b9a9e0 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE: re
> turn "Vertical MV Search Range";
> case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER: re
> turn "Repeat Sequence Header";
> case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME: retu
> rn "Force Key Frame";
> + case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR: retu
> rn "MPEG2 Frame Header";
>
> /* VPX controls */
> case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:r
> eturn "VPX Number of Partitions";
> @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name,
> enum v4l2_ctrl_type *type,
> case V4L2_CID_RDS_TX_ALT_FREQS:
> *type = V4L2_CTRL_TYPE_U32;
> break;
> + case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> + *type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> + break;
> default:
> *type = V4L2_CTRL_TYPE_INTEGER;
> break;
> @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl
> *ctrl, u32 idx,
> return -ERANGE;
> return 0;
>
> + case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> + return 0;
> +
> default:
> return -EINVAL;
> }
> @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct
> v4l2_ctrl_handler *hdl,
> case V4L2_CTRL_TYPE_U32:
> elem_size = sizeof(u32);
> break;
> + case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> + elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
> + break;
> default:
> if (type < V4L2_CTRL_COMPOUND_TYPES)
> elem_size = sizeof(s32);
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 468c3c65362d..8070203da5d2 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc
> *fmt)
> case V4L2_PIX_FMT_VC1_ANNEX_L: descr = "VC-1
> (SMPTE 412M Annex L)"; break;
> case V4L2_PIX_FMT_VP8: descr = "VP8";
> break;
> case V4L2_PIX_FMT_VP9: descr = "VP9";
> break;
> + case V4L2_PIX_FMT_MPEG2_FRAME: descr = "MPEG2
> Frame"; break;
> case V4L2_PIX_FMT_CPIA1:descr = "GSPCA CPiA
> YUV"; break;
> case V4L2_PIX_FMT_WNVA: descr =
> "WNVA"; break;
> case V4L2_PIX_FMT_SN9C10X: descr = "GSPCA
> SN9C10X"; break;
> diff --git a/include/uapi/linux/v4l2-controls.h
> b/include/uapi/linux/v4l2-controls.h
> index cbbb750d87d1..8431b2a540c7 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
> };
> #
Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes
Hi,
On Fri, 2018-05-04 at 10:40 +0200, Maxime Ripard wrote:
> On Fri, May 04, 2018 at 09:49:16AM +0200, Paul Kocialkowski wrote:
> > > > + reserved-memory {
> > > > + #address-cells = <1>;
> > > > + #size-cells = <1>;
> > > > + ranges;
> > > > +
> > > > + /* Address must be kept in the lower 256 MiBs
> > > > of
> > > > DRAM for VE. */
> > > > + ve_memory: cma@4a00 {
> > > > + compatible = "shared-dma-pool";
> > > > + reg = <0x4a00 0x600>;
> > > > + no-map;
> > >
> > > I'm not sure why no-map is needed.
> >
> > In fact, having no-map here would lead to reserving the area as
> > cache-
> > coherent instead of contiguous and thus prevented dmabuf support.
> > Replacing it by "resuable" allows proper CMA reservation.
> >
> > > And I guess we could use alloc-ranges to make sure the region is
> > > in
> > > the proper memory range, instead of hardcoding it.
> >
> > As far as I could understand from the documentation, "alloc-ranges"
> > is
> > used for dynamic allocation while only "reg" is used for static
> > allocation. We are currently going with static allocation and thus
> > reserve the whole 96 MiB. Is using dynamic allocation instead
> > desirable
> > here?
>
> I guess we could turn the question backward. Why do we need a static
> allocation? This isn't a buffer that is always allocated on the same
> area, but rather that we have a range available. So our constraint is
> on the range, nothing else.
That makes sense, I will give it a shot with a range then.
> > > > + reg = <0x01c0e000 0x1000>;
> > > > + memory-region = <_memory>;
> > >
> > > Since you made the CMA region the default one, you don't need to
> > > tie
> > > it to that device in particular (and you can drop it being
> > > mandatory
> > > from your binding as well).
> >
> > What if another driver (or the system) claims memory from that zone
> > and
> > that the reserved memory ends up not being available for the VPU
> > anymore?
> >
> > Acccording to the reserved-memory documentation, the reusable
> > property
> > (that we need for dmabuf) puts a limitation that the device driver
> > owning the region must be able to reclaim it back.
> >
> > How does that work out if the CMA region is not tied to a driver in
> > particular?
>
> I'm not sure to get what you're saying. You have the property
> linux,cma-default in your reserved region, so the behaviour you
> described is what you explicitly asked for.
My point is that I don't see how the driver can claim back (part of) the
reserved area if the area is not explicitly attached to it.
Or is that mechanism made in a way that all drivers wishing to use the
reserved memory area can claim it back from the system, but there is no
priority (other than first-come first-served) for which drivers claims
it back in case two want to use the same reserved region (in a scenario
where there isn't enough memory to allow both drivers)?
> > > > +
> > > > + clocks = < CLK_AHB_VE>, <
> > > > CLK_VE>,
> > > > +< CLK_DRAM_VE>;
> > > > + clock-names = "ahb", "mod", "ram";
> > > > +
> > > > + assigned-clocks = < CLK_VE>;
> > > > + assigned-clock-rates = <32000>;
> > >
> > > This should be set from within the driver. If it's something that
> > > you
> > > absolutely needed for the device to operate, you have no guarantee
> > > that the clock rate won't change at any point in time after the
> > > device
> > > probe, so that's not a proper solution.
> > >
> > > And if it's not needed and can be adjusted depending on the
> > > framerate/codec/resolution, then it shouldn't be in the DT either.
> >
> > Yes, that makes sense.
> >
> > > Don't you also need to map the SRAM on the A20?
> >
> > That's a good point, there is currently no syscon handle for A20
> > (and
> > also A13). Maybe SRAM is muxed to the VE by default so it "just
> > works"?
> >
> > I'll investigate on this side, also keeping in mind that the actual
> > solution is to use the SRAM controller driver (but that won't make
> > it to
> > v3).
>
> The SRAM driver is available on the A20, so you should really use that
> instead of a syscon.
The SRAM driver is indeed available for the A20, but still lacks support
for the VE in particular as far as I can see.
Cheers,
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata
Hi,
On Fri, 2018-04-20 at 09:51 +, Tomasz Figa wrote:
> Hi Paul,
>
> On Fri, Apr 20, 2018 at 12:46 AM Paul Kocialkowski <
> paul.kocialkow...@bootlin.com> wrote:
> [snip]
> > +struct v4l2_ctrl_mpeg2_frame_hdr {
> > + __u32 slice_len;
> > + __u32 slice_pos;
> > + enum { MPEG1, MPEG2 } type;
>
> Is enum suitable for UAPI?
As it turns out, it's not :)
> > +
> > + __u16 width;
> > + __u16 height;
> > +
> > + enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
>
> Ditto.
>
> > + __u8 f_code[2][2];
> > +
> > + __u8 intra_dc_precision;
> > + __u8 picture_structure;
> > + __u8 top_field_first;
> > + __u8 frame_pred_frame_dct;
> > + __u8 concealment_motion_vectors;
> > + __u8 q_scale_type;
> > + __u8 intra_vlc_format;
> > + __u8 alternate_scan;
> > +
> > + __u8 backward_ref_index;
> > + __u8 forward_ref_index;
> > +};
> > +
> > #endif
> > diff --git a/include/uapi/linux/videodev2.h
>
> b/include/uapi/linux/videodev2.h
> > index 31b5728b56e9..4b8336f7bcf0 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -635,6 +635,7 @@ struct v4l2_pix_format {
> > #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L')
> > /*
>
> SMPTE 421M Annex L compliant stream */
> > #define V4L2_PIX_FMT_VP8 v4l2_fourcc('V', 'P', '8', '0') /*
> > VP8 */
> > #define V4L2_PIX_FMT_VP9 v4l2_fourcc('V', 'P', '9', '0') /*
> > VP9 */
> > +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /*
>
> MPEG2 frame */
>
> > /* Vendor-specific formats */
> > #define V4L2_PIX_FMT_CPIA1v4l2_fourcc('C', 'P', 'I', 'A') /*
> > cpia1
>
> YUV */
> > @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
> > __u8 __user *p_u8;
> > __u16 __user *p_u16;
> > __u32 __user *p_u32;
> > + struct v4l2_ctrl_mpeg2_frame_hdr __user
>
> *p_mpeg2_frame_hdr;
> > void __user *ptr;
> > };
> > } __attribute__ ((packed));
> > @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
> > V4L2_CTRL_TYPE_U8= 0x0100,
> > V4L2_CTRL_TYPE_U16 = 0x0101,
> > V4L2_CTRL_TYPE_U32 = 0x0102,
> > + V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
>
> Why 0x0109?
Good catch. I see no reason in particular, so I'll probably make it
0x0103 eventually.
Cheers and thanks for the review!
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata
Hi,
On Fri, 2018-04-20 at 15:57 +0200, Hans Verkuil wrote:
> On 04/19/18 17:45, Paul Kocialkowski wrote:
> > Stateless video decoding engines require both the MPEG slices and
> > associated metadata from the video stream in order to decode frames.
> >
> > This introduces definitions for a new pixel format, describing
> > buffers
> > with MPEG2 slice data, as well as a control structure for passing
> > the
> > frame header (metadata) to drivers.
> >
> > Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
> > Signed-off-by: Florent Revest <florent.rev...@free-electrons.com>
> > ---
> > drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++
> > drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> > include/uapi/linux/v4l2-controls.h | 26
> > ++
> > include/uapi/linux/videodev2.h | 3 +++
> > 4 files changed, 40 insertions(+)
> >
> > diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> > b/drivers/media/v4l2-core/v4l2-ctrls.c
> > index ba05a8b9a095..fcdc12b9a9e0 100644
> > --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> > +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> > @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> > case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:
> > return "Vertical MV Search Range";
> > case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:
> > return "Repeat Sequence Header";
> > case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME: re
> > turn "Force Key Frame";
> > + case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR: re
> > turn "MPEG2 Frame Header";
> >
> > /* VPX controls */
> > case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:
> > return "VPX Number of Partitions";
> > @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name,
> > enum v4l2_ctrl_type *type,
> > case V4L2_CID_RDS_TX_ALT_FREQS:
> > *type = V4L2_CTRL_TYPE_U32;
> > break;
> > + case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> > + *type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> > + break;
> > default:
> > *type = V4L2_CTRL_TYPE_INTEGER;
> > break;
> > @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl
> > *ctrl, u32 idx,
> > return -ERANGE;
> > return 0;
> >
> > + case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> > + return 0;
> > +
> > default:
> > return -EINVAL;
> > }
> > @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct
> > v4l2_ctrl_handler *hdl,
> > case V4L2_CTRL_TYPE_U32:
> > elem_size = sizeof(u32);
> > break;
> > + case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> > + elem_size = sizeof(struct
> > v4l2_ctrl_mpeg2_frame_hdr);
> > + break;
> > default:
> > if (type < V4L2_CTRL_COMPOUND_TYPES)
> > elem_size = sizeof(s32);
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> > b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index 468c3c65362d..8070203da5d2 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct
> > v4l2_fmtdesc *fmt)
> > case V4L2_PIX_FMT_VC1_ANNEX_L: descr = "VC-1
> > (SMPTE 412M Annex L)"; break;
> > case V4L2_PIX_FMT_VP8: descr =
> > "VP8"; break;
> > case V4L2_PIX_FMT_VP9: descr =
> > "VP9"; break;
> > + case V4L2_PIX_FMT_MPEG2_FRAME: descr =
> > "MPEG2 Frame"; break;
> > case V4L2_PIX_FMT_CPIA1:descr = "GSPCA CPiA
> > YUV"; break;
> > case V4L2_PIX_FMT_WNVA: descr =
> > "WNVA"; break;
> > case V4L2_PIX_FMT_SN9C10X: descr = "GSPCA
> > SN9C10X"; break;
> > diff --git a/include/uapi/linux/v4l2-controls.h
> > b/include/uapi/linux/v4l2-controls.h
> > index cbbb750d87d1..8431b2a540c7 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
> > };
> > #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL (V4L2_CID_MPE
> > G_BASE+407)
> >
> > +#define
> > V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_H
Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata
Hi, On Tue, 2018-04-24 at 12:01 +0300, Sakari Ailus wrote: > Hi Paul, > > On Thu, Apr 19, 2018 at 05:45:31PM +0200, Paul Kocialkowski wrote: > > Stateless video decoding engines require both the MPEG slices and > > associated metadata from the video stream in order to decode frames. > > > > This introduces definitions for a new pixel format, describing > > buffers > > with MPEG2 slice data, as well as a control structure for passing > > the > > frame header (metadata) to drivers. > > What's the typical relationship between MPEG2 slice data and the > header? > > Are the two always used together, do they originate from the same > source, for instance? I have to admit I'm not very familiar with > MPEG... Yes, the header is closely related to the slice data, as it expresses the metadata required for properly decoding the slice data. Both are extracted from the MPEG bitstream and there is dedicated metadata for each new set of slices that is decoded to a single frame. > > Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com> > > Signed-off-by: Florent Revest <florent.rev...@free-electrons.com> > > --- > > drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++ > > drivers/media/v4l2-core/v4l2-ioctl.c | 1 + > > include/uapi/linux/v4l2-controls.h | 26 > > ++ > > include/uapi/linux/videodev2.h | 3 +++ > > 4 files changed, 40 insertions(+) > > > > diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c > > b/drivers/media/v4l2-core/v4l2-ctrls.c > > index ba05a8b9a095..fcdc12b9a9e0 100644 > > --- a/drivers/media/v4l2-core/v4l2-ctrls.c > > +++ b/drivers/media/v4l2-core/v4l2-ctrls.c > > @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id) > > case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE: > > return "Vertical MV Search Range"; > > case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER: > > return "Repeat Sequence Header"; > > case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME: re > > turn "Force Key Frame"; > > + case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR: re > > turn "MPEG2 Frame Header"; > > > > /* VPX controls */ > > case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS: > > return "VPX Number of Partitions"; > > @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, > > enum v4l2_ctrl_type *type, > > case V4L2_CID_RDS_TX_ALT_FREQS: > > *type = V4L2_CTRL_TYPE_U32; > > break; > > + case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR: > > + *type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR; > > + break; > > default: > > *type = V4L2_CTRL_TYPE_INTEGER; > > break; > > @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl > > *ctrl, u32 idx, > > return -ERANGE; > > return 0; > > > > + case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR: > > + return 0; > > + > > default: > > return -EINVAL; > > } > > @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct > > v4l2_ctrl_handler *hdl, > > case V4L2_CTRL_TYPE_U32: > > elem_size = sizeof(u32); > > break; > > + case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR: > > + elem_size = sizeof(struct > > v4l2_ctrl_mpeg2_frame_hdr); > > + break; > > default: > > if (type < V4L2_CTRL_COMPOUND_TYPES) > > elem_size = sizeof(s32); > > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c > > b/drivers/media/v4l2-core/v4l2-ioctl.c > > index 468c3c65362d..8070203da5d2 100644 > > --- a/drivers/media/v4l2-core/v4l2-ioctl.c > > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c > > @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct > > v4l2_fmtdesc *fmt) > > case V4L2_PIX_FMT_VC1_ANNEX_L: descr = "VC-1 > > (SMPTE 412M Annex L)"; break; > > case V4L2_PIX_FMT_VP8: descr = > > "VP8"; break; > > case V4L2_PIX_FMT_VP9: descr = > > "VP9"; break; > > + case V4L2_PIX_FMT_MPEG2_FRAME: descr = > > "MPEG2 Frame"; break; > > case V4L2_PIX_FMT_CPIA1:descr = "GSPCA CPiA > > YUV"; break; > > case V4L2_PIX_FMT_WNVA: descr = > > "WNVA"; break; > > case V4L2_PIX_FMT_SN9C10X: descr
Re: [PATCH v2 03/10] videobuf2-core: Add helper to get buffer private data from media request
Hi,
On Thu, 2018-04-19 at 17:41 +0200, Paul Kocialkowski wrote:
> When calling media operation driver callbacks related to media
> requests,
> only a pointer to the request itself is provided, which is
> insufficient
> to retrieve the driver's context. Since the driver context is usually
> set as vb2 queue private data and given that the core can determine
> which objects attached to the request are buffers, it is possible to
> extract the associated private data for the first buffer found.
>
> This is required in order to access the current m2m context from m2m
> drivers' private data in the context of media request operation
> callbacks. More specifically, this allows scheduling m2m device runs
> from the newly-introduced request complete operation.
This patch too will be dropped since it's no longer useful with the
latest version of the request API.
> Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
> ---
> drivers/media/common/videobuf2/videobuf2-core.c | 15 +++
> include/media/videobuf2-core.h | 1 +
> 2 files changed, 16 insertions(+)
>
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> b/drivers/media/common/videobuf2/videobuf2-core.c
> index 13c9d9e243dd..6fa46bfc620f 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -1351,6 +1351,21 @@ bool vb2_core_request_has_buffers(struct
> media_request *req)
> }
> EXPORT_SYMBOL_GPL(vb2_core_request_has_buffers);
>
> +void *vb2_core_request_find_buffer_priv(struct media_request *req)
> +{
> + struct media_request_object *obj;
> + struct vb2_buffer *vb;
> +
> + obj = media_request_object_find(req, _core_req_ops,
> NULL);
> + if (!obj)
> + return NULL;
> +
> + vb = container_of(obj, struct vb2_buffer, req_obj);
> +
> + return vb2_get_drv_priv(vb->vb2_queue);
> +}
> +EXPORT_SYMBOL_GPL(vb2_core_request_find_buffer_priv);
> +
> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
> void *pb,
>struct media_request *req)
> {
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-
> core.h
> index 032bd1bec555..65c0cf6afb55 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -1153,4 +1153,5 @@ int vb2_verify_memory_type(struct vb2_queue *q,
> enum vb2_memory memory, unsigned int type);
>
> bool vb2_core_request_has_buffers(struct media_request *req);
> +void *vb2_core_request_find_buffer_priv(struct media_request *req);
> #endif /* _MEDIA_VIDEOBUF2_CORE_H */
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [PATCH v2 02/10] media-request: Add a request complete operation to allow m2m scheduling
Hi,
On Thu, 2018-04-19 at 17:41 +0200, Paul Kocialkowski wrote:
> When using the request API in the context of a m2m driver, the
> operations that come with a m2m run scheduling call in their
> (m2m-specific) ioctl handler are delayed until the request is queued
> (for instance, this includes queuing buffers and streamon).
>
> Thus, the m2m run scheduling calls are not called in due time since
> the
> request AP's internal plumbing will (rightfully) use the relevant core
> functions directly instead of the ioctl handler.
>
> This ends up in a situation where nothing happens if there is no
> run-scheduling ioctl called after queuing the request.
>
> In order to circumvent the issue, a new media operation is introduced,
> called at the time of handling the media request queue ioctl. It gives
> m2m drivers a chance to schedule a m2m device run at that time.
>
> The existing req_queue operation cannot be used for this purpose,
> since
> it is called with the request queue mutex held, that is eventually
> needed
> in the device_run call to apply relevant controls.
This patch will be dropped since it's no longer useful with the latest
version of the request API.
> Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
> ---
> drivers/media/media-request.c | 3 +++
> include/media/media-device.h | 2 ++
> 2 files changed, 5 insertions(+)
>
> diff --git a/drivers/media/media-request.c b/drivers/media/media-
> request.c
> index 415f7e31019d..28ac5ccfe6a2 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -157,6 +157,9 @@ static long media_request_ioctl_queue(struct
> media_request *req)
> media_request_get(req);
> }
>
> + if (mdev->ops->req_complete)
> + mdev->ops->req_complete(req);
> +
> return ret;
> }
>
> diff --git a/include/media/media-device.h b/include/media/media-
> device.h
> index 07e323c57202..c7dcf2079cc9 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -55,6 +55,7 @@ struct media_entity_notify {
> * @req_alloc: Allocate a request
> * @req_free: Free a request
> * @req_queue: Queue a request
> + * @req_complete: Complete a request
> */
> struct media_device_ops {
> int (*link_notify)(struct media_link *link, u32 flags,
> @@ -62,6 +63,7 @@ struct media_device_ops {
> struct media_request *(*req_alloc)(struct media_device
> *mdev);
> void (*req_free)(struct media_request *req);
> int (*req_queue)(struct media_request *req);
> + void (*req_complete)(struct media_request *req);
> };
>
> /**
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [PATCH v2 06/10] media: v4l: Add definition for Allwinner's MB32-tiled NV12 format
Hi,
On Fri, 2018-04-20 at 15:59 +0200, Hans Verkuil wrote:
> On 04/19/18 17:45, Paul Kocialkowski wrote:
> > This introduces support for Allwinner's MB32-tiled NV12 format,
> > where
> > each plane is divided into macroblocks of 32x32 pixels. Hence, the
> > size
> > of each plane has to be aligned to 32 bytes. The pixels inside each
> > macroblock are coded as they would be if the macroblock was a single
> > plane, line after line.
> >
> > The MB32-tiled NV12 format is used by the video engine on Allwinner
> > platforms: it is the default format for decoded frames (and the only
> > one
> > available in the oldest supported platforms).
> >
> > Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
> > ---
> > include/uapi/linux/videodev2.h | 1 +
> > 1 file changed, 1 insertion(+)
> >
> > diff --git a/include/uapi/linux/videodev2.h
> > b/include/uapi/linux/videodev2.h
> > index 4b8336f7bcf0..43993a116e2b 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -669,6 +669,7 @@ struct v4l2_pix_format {
> > #define V4L2_PIX_FMT_Z16 v4l2_fourcc('Z', '1', '6', ' ') /*
> > Depth data 16-bit */
> > #define V4L2_PIX_FMT_MT21Cv4l2_fourcc('M', 'T', '2', '1') /*
> > Mediatek compressed block mode */
> > #define V4L2_PIX_FMT_INZI v4l2_fourcc('I', 'N', 'Z', 'I') /*
> > Intel Planar Greyscale 10-bit and Depth 16-bit */
> > +#define V4L2_PIX_FMT_MB32_NV12 v4l2_fourcc('M', 'N', '1', '2') /*
> > Allwinner NV12 in 32x32 macroblocks */
> >
> > /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6
> > bits unused */
> > #define V4L2_PIX_FMT_IPU3_SBGGR10 v4l2_fourcc('i', 'p', '3',
> > 'b') /* IPU3 packed 10-bit BGGR bayer */
> >
>
> Add an entry for this to v4l_fill_fmtdesc() in v4l2-ioctl.c.
>
> It also needs to be documented in the spec.
Noted, I will look in that direction for the future versions.
Cheers,
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [PATCH v2 07/10] media: platform: Add Sunxi-Cedrus VPU decoder driver
Hi, On Tue, 2018-04-24 at 12:13 +0300, Sakari Ailus wrote: > Hi Paul > > On Thu, Apr 19, 2018 at 05:45:33PM +0200, Paul Kocialkowski wrote: > > This introduces the Sunxi-Cedrus VPU driver that supports the VPU > > found > > in Allwinner SoCs, also known as Video Engine. It is implemented > > through > > a v4l2 m2m decoder device and a media device (used for media > > requests). > > So far, it only supports MPEG2 decoding. > > > > Since this VPU is stateless, synchronization with media requests is > > required in order to ensure consistency between frame headers that > > contain metadata about the frame to process and the raw slice data > > that > > is used to generate the frame. > > > > This driver was made possible thanks to the long-standing effort > > carried out by the linux-sunxi community in the interest of reverse > > engineering, documenting and implementing support for Allwinner VPU. > > No code review yet, but DT bindings precede the driver. Please also > add the appropriate MAINTAINERS entries. Thanks for the indication, will do in the next version(s). Cheers, -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver
Hi,
On Thu, 2018-04-26 at 22:04 -0500, Rob Herring wrote:
> On Fri, Apr 20, 2018 at 09:22:20AM +0200, Paul Kocialkowski wrote:
> > Hi and thanks for the review,
> >
> > On Fri, 2018-04-20 at 01:31 +, Tomasz Figa wrote:
> > > Hi Paul, Philipp,
> > >
> > > On Fri, Apr 20, 2018 at 1:04 AM Philipp Zabel <p.zabel@pengutronix
> > > .de>
> > > wrote:
> > >
> > > > Hi Paul,
> > > > On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> > > > > This adds a device-tree binding document that specifies the
> > > > > properties
> > > > > used by the Sunxi-Cedurs VPU driver, as well as examples.
> > > > >
> > > > > Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.co
> > > > > m>
> > > > > ---
> > > > > .../devicetree/bindings/media/sunxi-cedrus.txt | 50
> > >
> > > ++
> > > > > 1 file changed, 50 insertions(+)
> > > > > create mode 100644
> > >
> > > Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > >
> > > > > diff --git a/Documentation/devicetree/bindings/media/sunxi-
> > > > > cedrus.txt
> > >
> > > b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > > new file mode 100644
> > > > > index ..71ad3f9c3352
> > > > > --- /dev/null
> > > > > +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > > @@ -0,0 +1,50 @@
> > > > > +Device-tree bindings for the VPU found in Allwinner SoCs,
> > > > > referred to
> > >
> > > as the
> > > > > +Video Engine (VE) in Allwinner literature.
> > > > > +
> > > > > +The VPU can only access the first 256 MiB of DRAM, that are
> > > > > DMA-
> > > > > mapped
> > >
> > > starting
> > > > > +from the DRAM base. This requires specific memory allocation
> > > > > and
> > >
> > > handling.
> > >
> > > And no IOMMU? Brings back memories.
> >
> > Exactly, no IOMMU so we don't have much choice but cope with that
> > hardware limitation...
> >
> > > > > +
> > > > > +Required properties:
> > > > > +- compatible : "allwinner,sun4i-a10-video-engine";
> > > > > +- memory-region : DMA pool for buffers allocation;
> > > > > +- clocks : list of clock specifiers,
> > > > > corresponding to
> > >
> > > entries in
> > > > > + the clock-names property;
> > > > > +- clock-names: should contain "ahb", "mod"
> > > > > and
> > > > > "ram"
> > >
> > > entries;
> > > > > +- assigned-clocks : list of clocks assigned to the VE;
> > > > > +- assigned-clocks-rates : list of clock rates for the clocks
> > > > > assigned
> > >
> > > to the VE;
> > > > > +- resets : phandle for reset;
> > > > > +- interrupts : should contain VE interrupt number;
> > > > > +- reg: should contain register base
> > > > > and
> > > > > length
> > >
> > > of VE.
> > > > > +
> > > > > +Example:
> > > > > +
> > > > > +reserved-memory {
> > > > > + #address-cells = <1>;
> > > > > + #size-cells = <1>;
> > > > > + ranges;
> > > > > +
> > > > > + /* Address must be kept in the lower 256 MiBs of DRAM
> > > > > for
> > > > > VE. */
> > > > > + ve_memory: cma@4a00 {
> > > > > + compatible = "shared-dma-pool";
> > > > > + reg = <0x4a00 0x600>;
> > > > > + no-map;
> > > > > + linux,cma-default;
> > > > > + };
> > > > > +};
> > > > > +
> > > > > +video-engine@1c0e000 {
> > > >
> > > > This is not really required by any specification, and not as
> > > > common
> > > > as
> > > > gpu@..., but could this reasonably b
Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes
Hi,
On Fri, 2018-04-20 at 09:39 +0200, Maxime Ripard wrote:
> On Thu, Apr 19, 2018 at 05:45:35PM +0200, Paul Kocialkowski wrote:
> > This adds nodes for the Video Engine and the associated reserved
> > memory
> > for the Allwinner A20. Up to 96 MiB of memory are dedicated to the
> > VPU.
> >
> > The VPU can only map the first 256 MiB of DRAM, so the reserved
> > memory
> > pool has to be located in that area. Following Allwinner's decision
> > in
> > downstream software, the last 96 MiB of the first 256 MiB of RAM are
> > reserved for this purpose.
> >
> > Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
> > ---
> > arch/arm/boot/dts/sun7i-a20.dtsi | 31
> > +++
> > 1 file changed, 31 insertions(+)
> >
> > diff --git a/arch/arm/boot/dts/sun7i-a20.dtsi
> > b/arch/arm/boot/dts/sun7i-a20.dtsi
> > index bd0cd3204273..cb6d82065dcf 100644
> > --- a/arch/arm/boot/dts/sun7i-a20.dtsi
> > +++ b/arch/arm/boot/dts/sun7i-a20.dtsi
> > @@ -163,6 +163,20 @@
> > reg = <0x4000 0x8000>;
> > };
> >
> > + reserved-memory {
> > + #address-cells = <1>;
> > + #size-cells = <1>;
> > + ranges;
> > +
> > + /* Address must be kept in the lower 256 MiBs of
> > DRAM for VE. */
> > + ve_memory: cma@4a00 {
> > + compatible = "shared-dma-pool";
> > + reg = <0x4a00 0x600>;
> > + no-map;
>
> I'm not sure why no-map is needed.
In fact, having no-map here would lead to reserving the area as cache-
coherent instead of contiguous and thus prevented dmabuf support.
Replacing it by "resuable" allows proper CMA reservation.
> And I guess we could use alloc-ranges to make sure the region is in
> the proper memory range, instead of hardcoding it.
As far as I could understand from the documentation, "alloc-ranges" is
used for dynamic allocation while only "reg" is used for static
allocation. We are currently going with static allocation and thus
reserve the whole 96 MiB. Is using dynamic allocation instead desirable
here?
> > + linux,cma-default;
> > + };
> > + };
> > +
> > timer {
> > compatible = "arm,armv7-timer";
> > interrupts = > IRQ_TYPE_LEVEL_LOW)>,
> > @@ -451,6 +465,23 @@
> > };
> > };
> >
> > + ve: video-engine@1c0e000 {
> > + compatible = "allwinner,sun4i-a10-video-
> > engine";
>
> We should have an A20-specific compatible here, at least, so that we
> can deal with any quirk we might find down the road.
Yes that's a very good point.
> > + reg = <0x01c0e000 0x1000>;
> > + memory-region = <_memory>;
>
> Since you made the CMA region the default one, you don't need to tie
> it to that device in particular (and you can drop it being mandatory
> from your binding as well).
What if another driver (or the system) claims memory from that zone and
that the reserved memory ends up not being available for the VPU
anymore?
Acccording to the reserved-memory documentation, the reusable property
(that we need for dmabuf) puts a limitation that the device driver
owning the region must be able to reclaim it back.
How does that work out if the CMA region is not tied to a driver in
particular?
> > +
> > + clocks = < CLK_AHB_VE>, < CLK_VE>,
> > +< CLK_DRAM_VE>;
> > + clock-names = "ahb", "mod", "ram";
> > +
> > + assigned-clocks = < CLK_VE>;
> > + assigned-clock-rates = <32000>;
>
> This should be set from within the driver. If it's something that you
> absolutely needed for the device to operate, you have no guarantee
> that the clock rate won't change at any point in time after the device
> probe, so that's not a proper solution.
>
> And if it's not needed and can be adjusted depending on the
> framerate/codec/resolution, then it shouldn't be in the DT either.
Yes, that makes sense.
> Don't you also need to map the SRAM on the A20?
That's a good point, there is currently no syscon handle for A20 (and
also A13). Maybe SRAM is muxed to the VE by default so it "just works"?
I'll investigate on this side, also keeping in mind that the actual
solution is to use the SRAM controller driver (but that won't make it to
v3).
Cheers and thanks for the review!
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver
Hi and thanks for the review,
On Fri, 2018-04-20 at 01:31 +, Tomasz Figa wrote:
> Hi Paul, Philipp,
>
> On Fri, Apr 20, 2018 at 1:04 AM Philipp Zabel <p.za...@pengutronix.de>
> wrote:
>
> > Hi Paul,
> > On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> > > This adds a device-tree binding document that specifies the
> > > properties
> > > used by the Sunxi-Cedurs VPU driver, as well as examples.
> > >
> > > Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
> > > ---
> > > .../devicetree/bindings/media/sunxi-cedrus.txt | 50
>
> ++
> > > 1 file changed, 50 insertions(+)
> > > create mode 100644
>
> Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > >
> > > diff --git a/Documentation/devicetree/bindings/media/sunxi-
> > > cedrus.txt
>
> b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > new file mode 100644
> > > index ..71ad3f9c3352
> > > --- /dev/null
> > > +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > @@ -0,0 +1,50 @@
> > > +Device-tree bindings for the VPU found in Allwinner SoCs,
> > > referred to
>
> as the
> > > +Video Engine (VE) in Allwinner literature.
> > > +
> > > +The VPU can only access the first 256 MiB of DRAM, that are DMA-
> > > mapped
>
> starting
> > > +from the DRAM base. This requires specific memory allocation and
>
> handling.
>
> And no IOMMU? Brings back memories.
Exactly, no IOMMU so we don't have much choice but cope with that
hardware limitation...
> > > +
> > > +Required properties:
> > > +- compatible : "allwinner,sun4i-a10-video-engine";
> > > +- memory-region : DMA pool for buffers allocation;
> > > +- clocks : list of clock specifiers, corresponding to
>
> entries in
> > > + the clock-names property;
> > > +- clock-names: should contain "ahb", "mod" and
> > > "ram"
>
> entries;
> > > +- assigned-clocks : list of clocks assigned to the VE;
> > > +- assigned-clocks-rates : list of clock rates for the clocks
> > > assigned
>
> to the VE;
> > > +- resets : phandle for reset;
> > > +- interrupts : should contain VE interrupt number;
> > > +- reg: should contain register base and
> > > length
>
> of VE.
> > > +
> > > +Example:
> > > +
> > > +reserved-memory {
> > > + #address-cells = <1>;
> > > + #size-cells = <1>;
> > > + ranges;
> > > +
> > > + /* Address must be kept in the lower 256 MiBs of DRAM for
> > > VE. */
> > > + ve_memory: cma@4a00 {
> > > + compatible = "shared-dma-pool";
> > > + reg = <0x4a00 0x600>;
> > > + no-map;
> > > + linux,cma-default;
> > > + };
> > > +};
> > > +
> > > +video-engine@1c0e000 {
> > This is not really required by any specification, and not as common
> > as
> > gpu@..., but could this reasonably be called "vpu@1c0e000" to follow
> > somewhat-common practice?
>
> AFAIR the name is supposed to be somewhat readable for someone that
> doesn't know the hardware. To me, "video-engine" sounds more obvious
> than "vpu", but we actually use "codec" already, in case of MFC and
> JPEG codec on Exynos. If encode/decode is the only functionality of
> this block, I'd personally go with "codec". If it can do other things,
> e.g. scaling/rotation without encode/decode, I'd probably call it
> "video-processor".
I agree that the term VPU is more commonly associated with video
decoding, while video engine could mean a number of things.
The reason I went with "video-engine" here (while still presenting the
driver as a VPU driver) is that Video Engine is the term used in
Allwinner's litterature. Other nodes in Allwinner device-trees generally
stick to these terms (for instance, we have "display-engine" nodes).
This also makes it easier to find the matching parts in the
documentation.
Cheers,
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
[PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata
Stateless video decoding engines require both the MPEG slices and
associated metadata from the video stream in order to decode frames.
This introduces definitions for a new pixel format, describing buffers
with MPEG2 slice data, as well as a control structure for passing the
frame header (metadata) to drivers.
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
Signed-off-by: Florent Revest <florent.rev...@free-electrons.com>
---
drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/v4l2-controls.h | 26 ++
include/uapi/linux/videodev2.h | 3 +++
4 files changed, 40 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
b/drivers/media/v4l2-core/v4l2-ctrls.c
index ba05a8b9a095..fcdc12b9a9e0 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE: return
"Vertical MV Search Range";
case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER: return "Repeat
Sequence Header";
case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME: return "Force
Key Frame";
+ case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR: return "MPEG2
Frame Header";
/* VPX controls */
case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:return "VPX
Number of Partitions";
@@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
v4l2_ctrl_type *type,
case V4L2_CID_RDS_TX_ALT_FREQS:
*type = V4L2_CTRL_TYPE_U32;
break;
+ case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
+ *type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
+ break;
default:
*type = V4L2_CTRL_TYPE_INTEGER;
break;
@@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32
idx,
return -ERANGE;
return 0;
+ case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
+ return 0;
+
default:
return -EINVAL;
}
@@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct
v4l2_ctrl_handler *hdl,
case V4L2_CTRL_TYPE_U32:
elem_size = sizeof(u32);
break;
+ case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
+ elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
+ break;
default:
if (type < V4L2_CTRL_COMPOUND_TYPES)
elem_size = sizeof(s32);
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
index 468c3c65362d..8070203da5d2 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_VC1_ANNEX_L: descr = "VC-1 (SMPTE 412M Annex
L)"; break;
case V4L2_PIX_FMT_VP8: descr = "VP8"; break;
case V4L2_PIX_FMT_VP9: descr = "VP9"; break;
+ case V4L2_PIX_FMT_MPEG2_FRAME: descr = "MPEG2 Frame"; break;
case V4L2_PIX_FMT_CPIA1:descr = "GSPCA CPiA YUV"; break;
case V4L2_PIX_FMT_WNVA: descr = "WNVA"; break;
case V4L2_PIX_FMT_SN9C10X: descr = "GSPCA SN9C10X"; break;
diff --git a/include/uapi/linux/v4l2-controls.h
b/include/uapi/linux/v4l2-controls.h
index cbbb750d87d1..8431b2a540c7 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
};
#define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL (V4L2_CID_MPEG_BASE+407)
+#define V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR (V4L2_CID_MPEG_BASE+450)
+
/* Control IDs for VP8 streams
* Although VP8 is not part of MPEG we add these controls to the MPEG class
* as that class is already handling other video compression standards
@@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
#define V4L2_CID_DETECT_MD_THRESHOLD_GRID (V4L2_CID_DETECT_CLASS_BASE + 3)
#define V4L2_CID_DETECT_MD_REGION_GRID (V4L2_CID_DETECT_CLASS_BASE + 4)
+struct v4l2_ctrl_mpeg2_frame_hdr {
+ __u32 slice_len;
+ __u32 slice_pos;
+ enum { MPEG1, MPEG2 } type;
+
+ __u16 width;
+ __u16 height;
+
+ enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
+ __u8 f_code[2][2];
+
+ __u8 intra_dc_precision;
+ __u8 picture_structure;
+ __u8 top_field_first;
+ __u8 frame_pred_frame_dct;
+ __u8 concealment_motion_vectors;
+ __u8 q_scale_type;
+ __u8 intra_vlc_format;
+ __u8 alternate_scan;
+
+ __u8 backward_ref_index;
+
[PATCH v2 06/10] media: v4l: Add definition for Allwinner's MB32-tiled NV12 format
This introduces support for Allwinner's MB32-tiled NV12 format, where
each plane is divided into macroblocks of 32x32 pixels. Hence, the size
of each plane has to be aligned to 32 bytes. The pixels inside each
macroblock are coded as they would be if the macroblock was a single
plane, line after line.
The MB32-tiled NV12 format is used by the video engine on Allwinner
platforms: it is the default format for decoded frames (and the only one
available in the oldest supported platforms).
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
include/uapi/linux/videodev2.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 4b8336f7bcf0..43993a116e2b 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -669,6 +669,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_Z16 v4l2_fourcc('Z', '1', '6', ' ') /* Depth data
16-bit */
#define V4L2_PIX_FMT_MT21Cv4l2_fourcc('M', 'T', '2', '1') /* Mediatek
compressed block mode */
#define V4L2_PIX_FMT_INZI v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar
Greyscale 10-bit and Depth 16-bit */
+#define V4L2_PIX_FMT_MB32_NV12 v4l2_fourcc('M', 'N', '1', '2') /* Allwinner
NV12 in 32x32 macroblocks */
/* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits
unused */
#define V4L2_PIX_FMT_IPU3_SBGGR10 v4l2_fourcc('i', 'p', '3', 'b') /* IPU3
packed 10-bit BGGR bayer */
--
2.16.3
[PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes
This adds nodes for the Video Engine and the associated reserved memory
for the Allwinner A20. Up to 96 MiB of memory are dedicated to the VPU.
The VPU can only map the first 256 MiB of DRAM, so the reserved memory
pool has to be located in that area. Following Allwinner's decision in
downstream software, the last 96 MiB of the first 256 MiB of RAM are
reserved for this purpose.
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
arch/arm/boot/dts/sun7i-a20.dtsi | 31 +++
1 file changed, 31 insertions(+)
diff --git a/arch/arm/boot/dts/sun7i-a20.dtsi b/arch/arm/boot/dts/sun7i-a20.dtsi
index bd0cd3204273..cb6d82065dcf 100644
--- a/arch/arm/boot/dts/sun7i-a20.dtsi
+++ b/arch/arm/boot/dts/sun7i-a20.dtsi
@@ -163,6 +163,20 @@
reg = <0x4000 0x8000>;
};
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ /* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+ ve_memory: cma@4a00 {
+ compatible = "shared-dma-pool";
+ reg = <0x4a00 0x600>;
+ no-map;
+ linux,cma-default;
+ };
+ };
+
timer {
compatible = "arm,armv7-timer";
interrupts = ,
@@ -451,6 +465,23 @@
};
};
+ ve: video-engine@1c0e000 {
+ compatible = "allwinner,sun4i-a10-video-engine";
+ reg = <0x01c0e000 0x1000>;
+ memory-region = <_memory>;
+
+ clocks = < CLK_AHB_VE>, < CLK_VE>,
+< CLK_DRAM_VE>;
+ clock-names = "ahb", "mod", "ram";
+
+ assigned-clocks = < CLK_VE>;
+ assigned-clock-rates = <32000>;
+
+ resets = < RST_VE>;
+
+ interrupts = ;
+ };
+
mmc0: mmc@1c0f000 {
compatible = "allwinner,sun7i-a20-mmc";
reg = <0x01c0f000 0x1000>;
--
2.16.3
[PATCH v2 10/10] ARM: dts: sun8i-a33: Add Video Engine and reserved memory nodes
This adds nodes for the Video Engine and the associated reserved memory
for the Allwinner A33. Up to 96 MiB of memory are dedicated to the VPU.
The VPU can only map the first 256 MiB of DRAM, so the reserved memory
pool has to be located in that area. Following Allwinner's decision in
downstream software, the last 96 MiB of the first 256 MiB of RAM are
reserved for this purpose.
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
arch/arm/boot/dts/sun8i-a33.dtsi | 38 ++
1 file changed, 38 insertions(+)
diff --git a/arch/arm/boot/dts/sun8i-a33.dtsi b/arch/arm/boot/dts/sun8i-a33.dtsi
index a21f2ed07a52..308b532aee1d 100644
--- a/arch/arm/boot/dts/sun8i-a33.dtsi
+++ b/arch/arm/boot/dts/sun8i-a33.dtsi
@@ -181,6 +181,20 @@
reg = <0x4000 0x8000>;
};
+ reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ /* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+ ve_memory: cma@4a00 {
+ compatible = "shared-dma-pool";
+ reg = <0x4a00 0x600>;
+ no-map;
+ linux,cma-default;
+ };
+ };
+
sound: sound {
compatible = "simple-audio-card";
simple-audio-card,name = "sun8i-a33-audio";
@@ -204,6 +218,11 @@
};
soc@1c0 {
+ syscon: system-controller@01c0 {
+ compatible = "allwinner,sun8i-a33-syscon", "syscon";
+ reg = <0x01c0 0x1000>;
+ };
+
tcon0: lcd-controller@1c0c000 {
compatible = "allwinner,sun8i-a33-tcon";
reg = <0x01c0c000 0x1000>;
@@ -240,6 +259,25 @@
};
};
+ ve: video-engine@01c0e000 {
+ compatible = "allwinner,sun4i-a10-video-engine";
+ memory-region = <_memory>;
+ syscon = <>;
+
+ clocks = < CLK_BUS_VE>, < CLK_VE>,
+< CLK_DRAM_VE>;
+ clock-names = "ahb", "mod", "ram";
+
+ assigned-clocks = < CLK_VE>;
+ assigned-clock-rates = <32000>;
+
+ resets = < RST_BUS_VE>;
+
+ interrupts = ;
+
+ reg = <0x01c0e000 0x1000>;
+ };
+
crypto: crypto-engine@1c15000 {
compatible = "allwinner,sun4i-a10-crypto";
reg = <0x01c15000 0x1000>;
--
2.16.3
[PATCH v2 07/10] media: platform: Add Sunxi-Cedrus VPU decoder driver
This introduces the Sunxi-Cedrus VPU driver that supports the VPU found in Allwinner SoCs, also known as Video Engine. It is implemented through a v4l2 m2m decoder device and a media device (used for media requests). So far, it only supports MPEG2 decoding. Since this VPU is stateless, synchronization with media requests is required in order to ensure consistency between frame headers that contain metadata about the frame to process and the raw slice data that is used to generate the frame. This driver was made possible thanks to the long-standing effort carried out by the linux-sunxi community in the interest of reverse engineering, documenting and implementing support for Allwinner VPU. Signed-off-by: Florent Revest <florent.rev...@free-electrons.com> Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com> --- drivers/media/platform/Kconfig | 15 + drivers/media/platform/Makefile| 1 + drivers/media/platform/sunxi/cedrus/Makefile | 4 + drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c | 292 .../platform/sunxi/cedrus/sunxi_cedrus_common.h| 105 + .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.c | 228 ++ .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.h | 36 ++ .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.c | 201 + .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.h | 29 ++ .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c | 157 +++ .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h | 33 ++ .../platform/sunxi/cedrus/sunxi_cedrus_regs.h | 172 +++ .../platform/sunxi/cedrus/sunxi_cedrus_video.c | 497 + .../platform/sunxi/cedrus/sunxi_cedrus_video.h | 31 ++ 14 files changed, 1801 insertions(+) create mode 100644 drivers/media/platform/sunxi/cedrus/Makefile create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig index 614fbef08ddc..77d89e84ed3b 100644 --- a/drivers/media/platform/Kconfig +++ b/drivers/media/platform/Kconfig @@ -488,6 +488,21 @@ config VIDEO_TI_VPE Support for the TI VPE(Video Processing Engine) block found on DRA7XX SoC. +config VIDEO_SUNXI_CEDRUS + tristate "Sunxi-Cedrus VPU driver" + depends on VIDEO_DEV && VIDEO_V4L2 && MEDIA_CONTROLLER + depends on ARCH_SUNXI + depends on HAS_DMA + select VIDEOBUF2_DMA_CONTIG + select MEDIA_REQUEST_API + select V4L2_MEM2MEM_DEV + ---help--- + Support for the VPU found in Allwinner SoCs, also known as the Cedar + video engine. + + To compile this driver as a module, choose M here: the module + will be called sunxi-cedrus. + config VIDEO_TI_VPE_DEBUG bool "VPE debug messages" depends on VIDEO_TI_VPE diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile index 7f3080437be6..a2be170f6dff 100644 --- a/drivers/media/platform/Makefile +++ b/drivers/media/platform/Makefile @@ -72,6 +72,7 @@ obj-$(CONFIG_VIDEO_ROCKCHIP_RGA) += rockchip/rga/ obj-y += omap/ obj-$(CONFIG_VIDEO_AM437X_VPFE)+= am437x/ +obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunxi/cedrus/ obj-$(CONFIG_VIDEO_XILINX) += xilinx/ diff --git a/drivers/media/platform/sunxi/cedrus/Makefile b/drivers/media/platform/sunxi/cedrus/Makefile new file mode 100644 index ..98f30df626a9 --- /dev/null +++ b/drivers/media/platform/sunxi/cedrus/Makefile @@ -0,0 +1,4 @@ +obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunxi-cedrus.o + +sunxi-cedrus-y = sunxi_cedrus.o sunxi_cedrus_video.o sunxi_cedrus_hw.o \ +sunxi_cedrus_dec.o sunxi_cedrus_mpeg2.o diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c new file mode 100644 index ..364799c00703 --- /dev/null +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c @@ -0,0 +1,292 @@ +/* + * Sunxi-Cedrus VPU driver + * + * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkow...@bootlin.com> + * Copyright (C) 2016 Florent Revest <florent.rev
[PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver
This adds a device-tree binding document that specifies the properties
used by the Sunxi-Cedurs VPU driver, as well as examples.
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
.../devicetree/bindings/media/sunxi-cedrus.txt | 50 ++
1 file changed, 50 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/sunxi-cedrus.txt
diff --git a/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
new file mode 100644
index ..71ad3f9c3352
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
@@ -0,0 +1,50 @@
+Device-tree bindings for the VPU found in Allwinner SoCs, referred to as the
+Video Engine (VE) in Allwinner literature.
+
+The VPU can only access the first 256 MiB of DRAM, that are DMA-mapped starting
+from the DRAM base. This requires specific memory allocation and handling.
+
+Required properties:
+- compatible : "allwinner,sun4i-a10-video-engine";
+- memory-region : DMA pool for buffers allocation;
+- clocks : list of clock specifiers, corresponding to entries in
+ the clock-names property;
+- clock-names : should contain "ahb", "mod" and "ram" entries;
+- assigned-clocks : list of clocks assigned to the VE;
+- assigned-clocks-rates : list of clock rates for the clocks assigned to the
VE;
+- resets : phandle for reset;
+- interrupts : should contain VE interrupt number;
+- reg : should contain register base and length of VE.
+
+Example:
+
+reserved-memory {
+ #address-cells = <1>;
+ #size-cells = <1>;
+ ranges;
+
+ /* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+ ve_memory: cma@4a00 {
+ compatible = "shared-dma-pool";
+ reg = <0x4a00 0x600>;
+ no-map;
+ linux,cma-default;
+ };
+};
+
+video-engine@1c0e000 {
+ compatible = "allwinner,sun4i-a10-video-engine";
+ reg = <0x01c0e000 0x1000>;
+ memory-region = <_memory>;
+
+ clocks = < CLK_AHB_VE>, < CLK_VE>,
+< CLK_DRAM_VE>;
+ clock-names = "ahb", "mod", "ram";
+
+ assigned-clocks = < CLK_VE>;
+ assigned-clock-rates = <32000>;
+
+ resets = < RST_VE>;
+
+ interrupts = ;
+};
--
2.16.3
[PATCH v2 04/10] media: vim2m: Implement media request complete op to schedule m2m run
This adds an implementation of the media request complete operation for
the vim2m driver, that ensures that the driver will try to schedule a
m2m run whenever a request was completed. Without this operation, no m2m
device run will be scheduled in many scenarios.
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
drivers/media/platform/vim2m.c | 12
1 file changed, 12 insertions(+)
diff --git a/drivers/media/platform/vim2m.c b/drivers/media/platform/vim2m.c
index 2dcf0ea85705..08c4c5566614 100644
--- a/drivers/media/platform/vim2m.c
+++ b/drivers/media/platform/vim2m.c
@@ -453,6 +453,17 @@ static void device_isr(struct timer_list *t)
schedule_work(_dev->work_run);
}
+static void request_complete(struct media_request *req)
+{
+ struct vim2m_ctx *curr_ctx;
+
+ curr_ctx = (struct vim2m_ctx *) vb2_core_request_find_buffer_priv(req);
+ if (curr_ctx == NULL)
+ return;
+
+ v4l2_m2m_try_schedule(curr_ctx->fh.m2m_ctx);
+}
+
/*
* video ioctls
*/
@@ -1025,6 +1036,7 @@ static const struct v4l2_m2m_ops m2m_ops = {
static const struct media_device_ops m2m_media_ops = {
.req_queue = vb2_request_queue,
+ .req_complete = request_complete,
};
static int vim2m_probe(struct platform_device *pdev)
--
2.16.3
[PATCH v2 01/10] media: v4l2-ctrls: Add missing v4l2 ctrl unlock
This adds a missing v4l2_ctrl_unlock call that is required to avoid
deadlocks.
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
drivers/media/v4l2-core/v4l2-ctrls.c | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
b/drivers/media/v4l2-core/v4l2-ctrls.c
index f67e9f5531fa..ba05a8b9a095 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -3614,10 +3614,12 @@ void v4l2_ctrl_request_complete(struct media_request
*req,
continue;
v4l2_ctrl_lock(ctrl);
+
if (ref->req)
ptr_to_ptr(ctrl, ref->req->p_req, ref->p_req);
else
ptr_to_ptr(ctrl, ctrl->p_cur, ref->p_req);
+
v4l2_ctrl_unlock(ctrl);
}
@@ -3677,8 +3679,11 @@ void v4l2_ctrl_request_setup(struct media_request *req,
}
}
}
- if (!have_new_data)
+
+ if (!have_new_data) {
+ v4l2_ctrl_unlock(master);
continue;
+ }
for (i = 0; i < master->ncontrols; i++) {
if (master->cluster[i]) {
--
2.16.3
[PATCH v2 02/10] media-request: Add a request complete operation to allow m2m scheduling
When using the request API in the context of a m2m driver, the
operations that come with a m2m run scheduling call in their
(m2m-specific) ioctl handler are delayed until the request is queued
(for instance, this includes queuing buffers and streamon).
Thus, the m2m run scheduling calls are not called in due time since the
request AP's internal plumbing will (rightfully) use the relevant core
functions directly instead of the ioctl handler.
This ends up in a situation where nothing happens if there is no
run-scheduling ioctl called after queuing the request.
In order to circumvent the issue, a new media operation is introduced,
called at the time of handling the media request queue ioctl. It gives
m2m drivers a chance to schedule a m2m device run at that time.
The existing req_queue operation cannot be used for this purpose, since
it is called with the request queue mutex held, that is eventually needed
in the device_run call to apply relevant controls.
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
drivers/media/media-request.c | 3 +++
include/media/media-device.h | 2 ++
2 files changed, 5 insertions(+)
diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
index 415f7e31019d..28ac5ccfe6a2 100644
--- a/drivers/media/media-request.c
+++ b/drivers/media/media-request.c
@@ -157,6 +157,9 @@ static long media_request_ioctl_queue(struct media_request
*req)
media_request_get(req);
}
+ if (mdev->ops->req_complete)
+ mdev->ops->req_complete(req);
+
return ret;
}
diff --git a/include/media/media-device.h b/include/media/media-device.h
index 07e323c57202..c7dcf2079cc9 100644
--- a/include/media/media-device.h
+++ b/include/media/media-device.h
@@ -55,6 +55,7 @@ struct media_entity_notify {
* @req_alloc: Allocate a request
* @req_free: Free a request
* @req_queue: Queue a request
+ * @req_complete: Complete a request
*/
struct media_device_ops {
int (*link_notify)(struct media_link *link, u32 flags,
@@ -62,6 +63,7 @@ struct media_device_ops {
struct media_request *(*req_alloc)(struct media_device *mdev);
void (*req_free)(struct media_request *req);
int (*req_queue)(struct media_request *req);
+ void (*req_complete)(struct media_request *req);
};
/**
--
2.16.3
[PATCH v2 00/10] Sunxi-Cedrus driver for the Allwinner Video Engine, using media requests
This presents a second iteration of the updated Sunxi-Cedrus driver, that supports the Video Engine found in most Allwinner SoCs, starting with the A10. It was tested on both the A20 and the A33. The initial version of this driver[0] was originally written and submitted by Florent Revest using a previous version of the request API that is necessary to provide coherency between controls and the buffers they apply to. The driver was adapted to use the latest version of the media request API[1], as submitted by Hand Verkuil. Media request API support is a hard requirement for the Sunxi-Cedrus driver. This series also contains fixes for issues encountered with the current version of the request API. If accepted, these should eventually be squashed into the request API series. The driver itself currently only supports MPEG2 and more codecs will be added to the driver eventually. The output frames provided by the Video Engine are in a multi-planar 32x32-tiled YUV format, with a plane for luminance (Y) and a plane for chrominance (UV). A specific format is introduced in the V4L2 API to describe it. This implementation is based on the significant work that was conducted by various members of the linux-sunxi community for understanding and documenting the Video Engine's innards. Changes since v1: * use the latest version of the request API for Hans Verkuil; * added media controller support and dependency * renamed v4l2 format to the more explicit V4L2_PIX_FMT_MB32_NV12; * reworked bindings documentation; * moved driver to drivers/media/platforms/sunxi/cedrus to pair with incoming CSI support ; * added a workqueue and lists to schedule buffer completion, since it cannot be done in interrupt context; * split mpeg2 support into a setup and a trigger function to avoid race condition; * split video-related ops to a dedicated sunxi_cedrus_video file; * cleaned up the included headers for each file; * used device PFN offset instead of subtracting PHYS_BASE; * used reset_control_reset instead of assert+deassert; * put the device in reset when removing driver; * changed dt bindings to use the last 96 Mib of the first 256 MiB of DRAM; * made it clear in the mpeg frame header structure that forward and backward indexes are used as reference frames for motion vectors; * lots of small cosmetic and consistency changes, including naming harmonization and headers text rework. Remaining tasks: * using the dedicated SRAM controller driver; * cleaning up registers description and documenting the fields used; * removing the assigned-clocks property and setting the clock rate in the driver directly; * testing on more platforms. Cheers! Paul Kocialkowski (10): media: v4l2-ctrls: Add missing v4l2 ctrl unlock media-request: Add a request complete operation to allow m2m scheduling videobuf2-core: Add helper to get buffer private data from media request media: vim2m: Implement media request complete op to schedule m2m run media: v4l: Add definitions for MPEG2 frame format and header metadata media: v4l: Add definition for Allwinner's MB32-tiled NV12 format media: platform: Add Sunxi-Cedrus VPU decoder driver dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes ARM: dts: sun8i-a33: Add Video Engine and reserved memory nodes .../devicetree/bindings/media/sunxi-cedrus.txt | 50 +++ arch/arm/boot/dts/sun7i-a20.dtsi | 31 ++ arch/arm/boot/dts/sun8i-a33.dtsi | 38 ++ drivers/media/common/videobuf2/videobuf2-core.c| 15 + drivers/media/media-request.c | 3 + drivers/media/platform/Kconfig | 15 + drivers/media/platform/Makefile| 1 + drivers/media/platform/sunxi/cedrus/Makefile | 4 + drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c | 292 .../platform/sunxi/cedrus/sunxi_cedrus_common.h| 105 + .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.c | 228 ++ .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.h | 36 ++ .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.c | 201 + .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.h | 29 ++ .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c | 157 +++ .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h | 33 ++ .../platform/sunxi/cedrus/sunxi_cedrus_regs.h | 172 +++ .../platform/sunxi/cedrus/sunxi_cedrus_video.c | 497 + .../platform/sunxi/cedrus/sunxi_cedrus_video.h | 31 ++ drivers/media/platform/vim2m.c | 12 + drivers/media/v4l2-core/v4l2-ctrls.c | 17 +- drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/media/media-device.h | 2 + include/media/videobuf2-core.h | 1 + include/uapi/linux/v4l2-controls.h | 26 ++ include/uapi/linux/videodev2.h
[PATCH v2 03/10] videobuf2-core: Add helper to get buffer private data from media request
When calling media operation driver callbacks related to media requests,
only a pointer to the request itself is provided, which is insufficient
to retrieve the driver's context. Since the driver context is usually
set as vb2 queue private data and given that the core can determine
which objects attached to the request are buffers, it is possible to
extract the associated private data for the first buffer found.
This is required in order to access the current m2m context from m2m
drivers' private data in the context of media request operation
callbacks. More specifically, this allows scheduling m2m device runs
from the newly-introduced request complete operation.
Signed-off-by: Paul Kocialkowski <paul.kocialkow...@bootlin.com>
---
drivers/media/common/videobuf2/videobuf2-core.c | 15 +++
include/media/videobuf2-core.h | 1 +
2 files changed, 16 insertions(+)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
b/drivers/media/common/videobuf2/videobuf2-core.c
index 13c9d9e243dd..6fa46bfc620f 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -1351,6 +1351,21 @@ bool vb2_core_request_has_buffers(struct media_request
*req)
}
EXPORT_SYMBOL_GPL(vb2_core_request_has_buffers);
+void *vb2_core_request_find_buffer_priv(struct media_request *req)
+{
+ struct media_request_object *obj;
+ struct vb2_buffer *vb;
+
+ obj = media_request_object_find(req, _core_req_ops, NULL);
+ if (!obj)
+ return NULL;
+
+ vb = container_of(obj, struct vb2_buffer, req_obj);
+
+ return vb2_get_drv_priv(vb->vb2_queue);
+}
+EXPORT_SYMBOL_GPL(vb2_core_request_find_buffer_priv);
+
int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb,
struct media_request *req)
{
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 032bd1bec555..65c0cf6afb55 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -1153,4 +1153,5 @@ int vb2_verify_memory_type(struct vb2_queue *q,
enum vb2_memory memory, unsigned int type);
bool vb2_core_request_has_buffers(struct media_request *req);
+void *vb2_core_request_find_buffer_priv(struct media_request *req);
#endif /* _MEDIA_VIDEOBUF2_CORE_H */
--
2.16.3
Re: [PATCH 5/9] media: platform: Add Sunxi Cedrus decoder driver
Hi and thanks for the review,
On Fri, 2018-03-09 at 14:57 +0100, Maxime Ripard wrote:
> On Fri, Mar 09, 2018 at 11:14:41AM +0100, Paul Kocialkowski wrote:
> > +/*
> > + * mem2mem callbacks
> > + */
> > +
> > +void job_abort(void *priv)
> > +{}
>
> Is that still needed?
v2 contains a proper implementation of job abortion, so yes :)
> > +/*
> > + * device_run() - prepares and starts processing
> > + */
> > +void device_run(void *priv)
> > +{
>
> This function (and the one above) should probably made static. Or at
> least if you can't, they should have a much more specific name in
> order not to conflict with anything from the core.
Agreed, will fix in v2.
> > + /*
> > +* The VPU is only able to handle bus addresses so we have
> > to subtract
> > +* the RAM offset to the physcal addresses
> > +*/
> > + in_buf -= PHYS_OFFSET;
> > + out_luma -= PHYS_OFFSET;
> > + out_chroma -= PHYS_OFFSET;
>
> You should take care of that by putting it in the dma_pfn_offset field
> of the struct device (at least before we come up with something
> better).
>
> You'll then be able to use the dma_addr_t directly without modifying
> it.
Ditto.
> > + vpu->syscon = syscon_regmap_lookup_by_phandle(vpu->dev-
> > >of_node,
> > + "syscon");
> > + if (IS_ERR(vpu->syscon)) {
> > + vpu->syscon = NULL;
> > + } else {
> > + regmap_write_bits(vpu->syscon,
> > SYSCON_SRAM_CTRL_REG0,
> > + SYSCON_SRAM_C1_MAP_VE,
> > + SYSCON_SRAM_C1_MAP_VE);
> > + }
>
> This should be using our SRAM controller driver (and API), see
> Documentation/devicetree/bindings/sram/sunxi-sram.txt
> include/linux/soc/sunxi/sunxi_sram.h
This will require adding support for the VE (and the A33 along the way)
in the SRAM driver, so a dedicated patch series will be sent in this
direction eventually.
> > + ret = clk_prepare_enable(vpu->ahb_clk);
> > + if (ret) {
> > + dev_err(vpu->dev, "could not enable ahb clock\n");
> > + return -EFAULT;
> > + }
> > + ret = clk_prepare_enable(vpu->mod_clk);
> > + if (ret) {
> > + clk_disable_unprepare(vpu->ahb_clk);
> > + dev_err(vpu->dev, "could not enable mod clock\n");
> > + return -EFAULT;
> > + }
> > + ret = clk_prepare_enable(vpu->ram_clk);
> > + if (ret) {
> > + clk_disable_unprepare(vpu->mod_clk);
> > + clk_disable_unprepare(vpu->ahb_clk);
> > + dev_err(vpu->dev, "could not enable ram clock\n");
> > + return -EFAULT;
> > + }
>
> Ideally, this should be using runtime_pm to manage the device power
> state, and disable it when not used.
>
> > + reset_control_assert(vpu->rstc);
> > + reset_control_deassert(vpu->rstc);
>
> You can use reset_control_reset here
Will do in v2.
> > + return 0;
> > +}
> > +
> > +void sunxi_cedrus_hw_remove(struct sunxi_cedrus_dev *vpu)
> > +{
> > + clk_disable_unprepare(vpu->ram_clk);
> > + clk_disable_unprepare(vpu->mod_clk);
> > + clk_disable_unprepare(vpu->ahb_clk);
>
> The device is not put back into reset here
Good catch!
Cheers,
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [linux-sunxi] [PATCH 5/9] media: platform: Add Sunxi Cedrus decoder driver
Hi, On Mon, 2018-03-12 at 17:15 +, Joonas Kylmälä wrote: > Paul Kocialkowski: > > diff --git a/drivers/media/platform/sunxi-cedrus/sunxi_cedrus_regs.h > > b/drivers/media/platform/sunxi-cedrus/sunxi_cedrus_regs.h > > new file mode 100644 > > index ..7384daa94737 > > --- /dev/null > > +++ b/drivers/media/platform/sunxi-cedrus/sunxi_cedrus_regs.h > > @@ -0,0 +1,170 @@ > > +/* > > + * Sunxi Cedrus codec driver > > + * > > + * Copyright (C) 2016 Florent Revest > > + * Florent Revest <florent.rev...@free-electrons.com> > > + * > > + * Based on Cedrus > > + * > > + * Copyright (c) 2013 Jens Kuske <jensku...@gmail.com> > > + * > > + * This software is licensed under the terms of the GNU General > > Public > > + * License version 2, as published by the Free Software Foundation, > > and > > + * may be copied, distributed, and modified under those terms. > > + * > > + * This program is distributed in the hope that it will be useful, > > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > + * GNU General Public License for more details. > > + */ > > + > > +#ifndef SUNXI_CEDRUS_REGS_H > > +#define SUNXI_CEDRUS_REGS_H > > + > > +/* > > + * For more information consult http://linux-sunxi.org/VE_Register_ > > guide > > + */ > > + > > +/* Special registers values */ > > + > > +/* VE_CTRL: > > + * The first 3 bits indicate the engine (0 for MPEG, 1 for H264, b > > for AVC...) > > + * The 16th and 17th bits indicate the memory type (3 for DDR3 32 > > bits) > > + * The 20th bit is unknown but needed > > + */ > > +#define VE_CTRL_MPEG 0x13 > > +#define VE_CTRL_H264 0x130001 > > +#define VE_CTRL_AVC0x13000b > > +#define VE_CTRL_REINIT 0x130007 > > + > > +/* VE_MPEG_CTRL: > > + * The bit 3 (0x8) is used to enable IRQs > > + * The other bits are unknown but needed > > + */ > > +#define VE_MPEG_CTRL_MPEG2 0x81b8 > > +#define VE_MPEG_CTRL_MPEG4 (0x80084118 | BIT(7)) > > +#define VE_MPEG_CTRL_MPEG4_P (VE_MPEG_CTRL_MPEG4 | BIT(12)) > > + > > +/* VE_MPEG_VLD_ADDR: > > + * The bits 27 to 4 are used for the address > > + * The bits 31 to 28 (0x7) are used to select the MPEG or JPEG > > engine > > + */ > > +#define VE_MPEG_VLD_ADDR_VAL(x)((x & 0x0ff0) | (x >> > > 28) | (0x7 << 28)) > > + > > +/* VE_MPEG_TRIGGER: > > + * The first three bits are used to trigger the engine > > + * The bits 24 to 26 are used to select the input format (1 for > > MPEG1, 2 for > > Trailing whitespace. Will fix in v2, thanks! Cheers, -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [linux-sunxi] [PATCH 5/9] media: platform: Add Sunxi Cedrus decoder driver
Hi, On Mon, 2018-03-12 at 20:29 +, Joonas Kylmälä wrote: > Paul Kocialkowski: > > diff --git a/drivers/media/platform/sunxi-cedrus/sunxi_cedrus.c > > b/drivers/media/platform/sunxi-cedrus/sunxi_cedrus.c > > new file mode 100644 > > index ..88624035e0e3 > > --- /dev/null > > +++ b/drivers/media/platform/sunxi-cedrus/sunxi_cedrus.c > > @@ -0,0 +1,313 @@ > > +/* > > + * Sunxi Cedrus codec driver > > + * > > + * Copyright (C) 2016 Florent Revest > > + * Florent Revest <florent.rev...@free-electrons.com> > > + * > > + * Based on vim2m > > + * > > + * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd. > > + * Pawel Osciak, <pa...@osciak.com> > > + * Marek Szyprowski, <m.szyprow...@samsung.com> > > + * > > + * This software is licensed under the terms of the GNU General > > Public > > + * License version 2, as published by the Free Software Foundation, > > and > > + * may be copied, distributed, and modified under those terms. > > + * > > + * This program is distributed in the hope that it will be useful, > > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > + * GNU General Public License for more details. > > + */ > > + > > +#include "sunxi_cedrus_common.h" > > + > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > + > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > I think that the definitions > > #include > #include > #include > #include > #include > #include > > are not used directly in the sunxi_cedrus.c file. Therefore they > should be removed. Thanks for the review, this will be done in v2. Cheers, -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [PATCH 6/9] sunxi-cedrus: Add device tree binding document
Hi,
On Sun, 2018-03-18 at 07:48 -0500, Rob Herring wrote:
> On Fri, Mar 09, 2018 at 11:14:42AM +0100, Paul Kocialkowski wrote:
> > From: Florent Revest <florent.rev...@free-electrons.com>
>
> "device tree binding document" can all be summarized with the subject
> prefix "dt-bindings: media: ".
Will do in v2, thanks.
> Also, email should be updated to @bootlin.com?
I will keep the address @free-electrons.com (since there is no matching
@bootlin.com address). Although that address was broken at the time of
sending v1, it should be a valid redirect nowadays.
> >
> > Device Tree bindings for the Allwinner's video engine
> >
> > Signed-off-by: Florent Revest <florent.rev...@free-electrons.com>
> > ---
> > .../devicetree/bindings/media/sunxi-cedrus.txt | 44
> > ++
> > 1 file changed, 44 insertions(+)
> > create mode 100644 Documentation/devicetree/bindings/media/sunxi-
> > cedrus.txt
> >
> > diff --git a/Documentation/devicetree/bindings/media/sunxi-
> > cedrus.txt b/Documentation/devicetree/bindings/media/sunxi-
> > cedrus.txt
> > new file mode 100644
> > index ..138581113c49
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > @@ -0,0 +1,44 @@
> > +Device-Tree bindings for SUNXI video engine found in sunXi SoC
> > family
> > +
> > +Required properties:
> > +- compatible : "allwinner,sun4i-a10-video-engine";
> > +- memory-region : DMA pool for buffers allocation;
>
> Why do you need this linkage? Many drivers use CMA and don't need
> this.
The VPU can only access the first 256 MiB of DRAM, that are DMA-mapped
starting from the DRAM base. This requires specific memory allocation
and handling. I'll add the information in v2.
> > +- clocks : list of clock specifiers, corresponding to
> > + entries in clock-names property;
> > +- clock-names : should contain "ahb", "mod" and "ram"
> > entries;
> > +- resets : phandle for reset;
> > +- interrupts : should contain VE interrupt number;
> > +- reg : should contain register base and length
> > of VE.
> > +
> > +Example:
> > +
> > +reserved-memory {
> > + #address-cells = <1>;
> > + #size-cells = <1>;
> > + ranges;
> > +
> > + ve_reserved: cma {
> > + compatible = "shared-dma-pool";
> > + reg = <0x43d0 0x900>;
> > + no-map;
> > + linux,cma-default;
> > + };
> > +};
> > +
> > +video-engine {
> > + compatible = "allwinner,sun4i-a10-video-engine";
> > + memory-region = <_reserved>;
> > +
> > + clocks = <_gates 32>, < CLK_VE>,
> > +<_gates 0>;
> > + clock-names = "ahb", "mod", "ram";
> > +
> > + assigned-clocks = < CLK_VE>;
> > + assigned-clock-rates = <32000>;
>
> Not documented.
Will do in v2.
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [RFCv11 PATCH 00/29] Request API
Hi, On Wed, 2018-04-18 at 02:06 +, Alexandre Courbot wrote: > On Tue, Apr 17, 2018 at 8:41 PM Paul Kocialkowski < > paul.kocialkow...@bootlin.com> wrote: > On Tue, 2018-04-17 at 06:17 +, Alexandre Courbot wrote: > > > On Tue, Apr 17, 2018 at 3:12 PM Hans Verkuil <hverk...@xs4all.nl> > > > wrote: > > > > > > > On 04/17/2018 06:33 AM, Alexandre Courbot wrote: > > > > > On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all. > > > > > nl> > > > > > wrote: > > > > > > > > > > > From: Hans Verkuil <hans.verk...@cisco.com> > > > > > > Hi all, > > > > > > This is a cleaned up version of the v10 series (never posted > > > > > > to > > > > > > the list since it was messy). > > > > > > > > > > Hi Hans, > > > > > > > > > > It took me a while to test and review this, but finally have > > > > > been > > > > > able > > > > > > to > > > > > do it. > > > > > > > > > > First the result of the test: I have tried porting my dummy > > > > > vim2m > > > > > test > > > > > program > > > > > (https://gist.github.com/Gnurou/34c35f1f8e278dad454b51578d239a > > > > > 42 > > > > > for > > > > > reference), > > > > > and am getting a hang when trying to queue the second OUTPUT > > > > > buffer > > > > > > (right > > > > > after > > > > > queuing the first request). If I move the calls the > > > > > VIDIOC_STREAMON > > > > > > after > > > > > the > > > > > requests are queued, the hang seems to happen at that moment. > > > > > Probably a > > > > > deadlock, haven't looked in detail yet. > > > > > > > > > > I have a few other comments, will follow up per-patch. > > > > > > > > > > > > > I had a similar/same (?) report about this from Paul: > > > > https://www.mail-archive.com/linux-media@vger.kernel.org/msg1291 > > > > 77.h > > > > tml > > > > > > I saw this and tried to move the call to STREAMON to after the > > > requests are queued in my example program, but it then hanged > > > there. > > > So there is probably something more intricate taking place. > > I figured out the issue (but forgot to report back to the list): > > Hans' > > version of the request API doesn't set the POLLIN bit but POLLPRI > > instead, so you need to select for expect_fds instead of read_fds in > > the > > select call. That's pretty much all there is to it. > > I am not using select() but poll() in my test program (see the gist > link > above) and have set POLLPRI as the event to poll for. I may be missing > something but this looks correct to me? You're right, I overlooked your email and assumed you were hitting the same issue that I had at first. Anyway, I also had a similar issue when calling the STREAMON ioctl *before* enqueuing the request. What happens here is that v4l2_m2m_streamon (called from the ioctl) will also try to schedule a device run (v4l2_m2m_try_schedule). When the ioctl is called and the request was not queued yet, the lack of buffers will delay the streamon call. Then, when the request is later submitted, vb2_streamon is called with the queue directly, and there is no m2m-specific provision there, so no device run is scheduled and nothing happens. If the STREAMON ioctl happens after queuing the request, then things should be fine (but that's definitely not what we want userspace to be doing) since the vb2_streamon call from the ioctl handler will take effect and v4l2_m2m_try_schedule will be called. The way that I have solved this with the Sunxi-Cedrus driver is to add a req_complete callback function pointer (holding a call to v4l2_m2m_try_schedule) in media_device_ops and call it (if available) from media_request_ioctl_queue. I initially put this in req_queue directly, but since it is wrapped by the request queue mutex lock and provided that device_run needs to access the request queue, we need an extra op called out of this lock, before completing the ioctl handler. I will be sending v2 of my driver with preliminary patches to fix this (perhaps I should also fix vim2m that way along the way). Cheers, -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [RFCv11 PATCH 27/29] vim2m: support requests
Hi,
On Mon, 2018-04-09 at 16:20 +0200, Hans Verkuil wrote:
> From: Hans Verkuil <hans.verk...@cisco.com>
>
> Add support for requests to vim2m.
Please find a nit below.
> Signed-off-by: Hans Verkuil <hans.verk...@cisco.com>
> ---
> drivers/media/platform/vim2m.c | 25 +
> 1 file changed, 25 insertions(+)
>
> diff --git a/drivers/media/platform/vim2m.c
> b/drivers/media/platform/vim2m.c
> index 9b18b32c255d..2dcf0ea85705 100644
> --- a/drivers/media/platform/vim2m.c
> +++ b/drivers/media/platform/vim2m.c
> @@ -387,8 +387,26 @@ static void device_run(void *priv)
> src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
> dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
>
> + /* Apply request if needed */
This comment suggests that this is where request submission is handled.
I suggest making it clear that this the place where the *controls*
attached to the request are applied, instead.
> + if (src_buf->vb2_buf.req_obj.req)
> + v4l2_ctrl_request_setup(src_buf->vb2_buf.req_obj.req,
> + >hdl);
> + if (dst_buf->vb2_buf.req_obj.req &&
> + dst_buf->vb2_buf.req_obj.req != src_buf-
> >vb2_buf.req_obj.req)
> + v4l2_ctrl_request_setup(dst_buf->vb2_buf.req_obj.req,
> + >hdl);
> +
> device_process(ctx, src_buf, dst_buf);
>
> + /* Complete request if needed */
Ditto here.
> + if (src_buf->vb2_buf.req_obj.req)
> + v4l2_ctrl_request_complete(src_buf-
> >vb2_buf.req_obj.req,
> + >hdl);
> + if (dst_buf->vb2_buf.req_obj.req &&
> + dst_buf->vb2_buf.req_obj.req != src_buf-
> >vb2_buf.req_obj.req)
> + v4l2_ctrl_request_complete(dst_buf-
> >vb2_buf.req_obj.req,
> + >hdl);
> +
> /* Run a timer, which simulates a hardware irq */
> schedule_irq(dev, ctx->transtime);
> }
> @@ -823,6 +841,8 @@ static void vim2m_stop_streaming(struct vb2_queue
> *q)
> vbuf = v4l2_m2m_dst_buf_remove(ctx-
> >fh.m2m_ctx);
> if (vbuf == NULL)
> return;
> + v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req,
> +>hdl);
> spin_lock_irqsave(>dev->irqlock, flags);
> v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
> spin_unlock_irqrestore(>dev->irqlock, flags);
> @@ -1003,6 +1023,10 @@ static const struct v4l2_m2m_ops m2m_ops = {
> .job_abort = job_abort,
> };
>
> +static const struct media_device_ops m2m_media_ops = {
> + .req_queue = vb2_request_queue,
> +};
> +
> static int vim2m_probe(struct platform_device *pdev)
> {
> struct vim2m_dev *dev;
> @@ -1027,6 +1051,7 @@ static int vim2m_probe(struct platform_device
> *pdev)
> dev->mdev.dev = >dev;
> strlcpy(dev->mdev.model, "vim2m", sizeof(dev->mdev.model));
> media_device_init(>mdev);
> + dev->mdev.ops = _media_ops;
> dev->v4l2_dev.mdev = >mdev;
> dev->pad[0].flags = MEDIA_PAD_FL_SINK;
> dev->pad[1].flags = MEDIA_PAD_FL_SOURCE;
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [RFCv11 PATCH 00/29] Request API
Hi, On Tue, 2018-04-17 at 06:17 +, Alexandre Courbot wrote: > On Tue, Apr 17, 2018 at 3:12 PM Hans Verkuil <hverk...@xs4all.nl> > wrote: > > > On 04/17/2018 06:33 AM, Alexandre Courbot wrote: > > > On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverk...@xs4all.nl> > > > wrote: > > > > > > > From: Hans Verkuil <hans.verk...@cisco.com> > > > > Hi all, > > > > This is a cleaned up version of the v10 series (never posted to > > > > the list since it was messy). > > > > > > Hi Hans, > > > > > > It took me a while to test and review this, but finally have been > > > able > > to > > > do it. > > > > > > First the result of the test: I have tried porting my dummy vim2m > > > test > > > program > > > (https://gist.github.com/Gnurou/34c35f1f8e278dad454b51578d239a42 > > > for > > > reference), > > > and am getting a hang when trying to queue the second OUTPUT > > > buffer > > (right > > > after > > > queuing the first request). If I move the calls the > > > VIDIOC_STREAMON > > after > > > the > > > requests are queued, the hang seems to happen at that moment. > > > Probably a > > > deadlock, haven't looked in detail yet. > > > > > > I have a few other comments, will follow up per-patch. > > > > > I had a similar/same (?) report about this from Paul: > > https://www.mail-archive.com/linux-media@vger.kernel.org/msg129177.h > > tml > > I saw this and tried to move the call to STREAMON to after the > requests are queued in my example program, but it then hanged there. > So there is probably something more intricate taking place. I figured out the issue (but forgot to report back to the list): Hans' version of the request API doesn't set the POLLIN bit but POLLPRI instead, so you need to select for expect_fds instead of read_fds in the select call. That's pretty much all there is to it. Hope this helps, -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [RFCv11 PATCH 26/29] vim2m: use workqueue
Hi,
On Mon, 2018-04-09 at 16:20 +0200, Hans Verkuil wrote:
> From: Hans Verkuil <hans.verk...@cisco.com>
>
> v4l2_ctrl uses mutexes, so we can't setup a ctrl_handler in
> interrupt context. Switch to a workqueue instead.
See one comment below.
> Signed-off-by: Hans Verkuil <hans.verk...@cisco.com>
> ---
> drivers/media/platform/vim2m.c | 15 +--
> 1 file changed, 13 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/media/platform/vim2m.c
> b/drivers/media/platform/vim2m.c
> index ef970434af13..9b18b32c255d 100644
> --- a/drivers/media/platform/vim2m.c
> +++ b/drivers/media/platform/vim2m.c
> @@ -150,6 +150,7 @@ struct vim2m_dev {
> spinlock_t irqlock;
>
> struct timer_list timer;
> + struct work_struct work_run;
Wouldn't it make more sense to move this to vim2m_ctx instead (since
this is heavily m2m-specific)?
> struct v4l2_m2m_dev *m2m_dev;
> };
> @@ -392,9 +393,10 @@ static void device_run(void *priv)
> schedule_irq(dev, ctx->transtime);
> }
>
> -static void device_isr(struct timer_list *t)
> +static void device_work(struct work_struct *w)
> {
> - struct vim2m_dev *vim2m_dev = from_timer(vim2m_dev, t,
> timer);
> + struct vim2m_dev *vim2m_dev =
> + container_of(w, struct vim2m_dev, work_run);
> struct vim2m_ctx *curr_ctx;
> struct vb2_v4l2_buffer *src_vb, *dst_vb;
> unsigned long flags;
> @@ -426,6 +428,13 @@ static void device_isr(struct timer_list *t)
> }
> }
>
> +static void device_isr(struct timer_list *t)
> +{
> + struct vim2m_dev *vim2m_dev = from_timer(vim2m_dev, t,
> timer);
> +
> + schedule_work(_dev->work_run);
> +}
> +
> /*
> * video ioctls
> */
> @@ -806,6 +815,7 @@ static void vim2m_stop_streaming(struct vb2_queue
> *q)
> struct vb2_v4l2_buffer *vbuf;
> unsigned long flags;
>
> + flush_scheduled_work();
> for (;;) {
> if (V4L2_TYPE_IS_OUTPUT(q->type))
> vbuf = v4l2_m2m_src_buf_remove(ctx-
> >fh.m2m_ctx);
> @@ -1011,6 +1021,7 @@ static int vim2m_probe(struct platform_device
> *pdev)
> vfd = >vfd;
> vfd->lock = >dev_mutex;
> vfd->v4l2_dev = >v4l2_dev;
> + INIT_WORK(>work_run, device_work);
>
> #ifdef CONFIG_MEDIA_CONTROLLER
> dev->mdev.dev = >dev;
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [RFCv11 PATCH 27/29] vim2m: support requests
Hi,
On Mon, 2018-04-09 at 16:20 +0200, Hans Verkuil wrote:
> From: Hans Verkuil <hans.verk...@cisco.com>
>
> Add support for requests to vim2m.
Depending on where STREAMON happens in the sequence, there is a
possibility that v4l2_m2m_try_schedule is never called and thus that
device_run never runs when submitting the request.
More specifically, the m2m fashion of the STREAMON ioctl handler will
normally call v4l2_m2m_try_schedule. Hence, there is no issue if it's
called after submitting the request. On the other hand, if the request
was not submitted when calling STREAMON (which is a valid use case),
buffers are not available and v4l2_m2m_try_schedule won't schedule
anything. Once the request is submitted, the internal plumbing will
detect that STREAMON was requested but did not take effect, so it will
call vb2_streamon. And of course, vb2_streamon has no provision for
trying to schedule a m2m device run.
One way to fix this is to call v4l2_m2m_try_schedule from a workqueue
triggered in the driver's start_streaming qop. The same is also probably
necessary in buf_queue and buf_prepare since those qops are not called
from the m2m ioctl handler either.
What do you think?
> Signed-off-by: Hans Verkuil <hans.verk...@cisco.com>
> ---
> drivers/media/platform/vim2m.c | 25 +
> 1 file changed, 25 insertions(+)
>
> diff --git a/drivers/media/platform/vim2m.c
> b/drivers/media/platform/vim2m.c
> index 9b18b32c255d..2dcf0ea85705 100644
> --- a/drivers/media/platform/vim2m.c
> +++ b/drivers/media/platform/vim2m.c
> @@ -387,8 +387,26 @@ static void device_run(void *priv)
> src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
> dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
>
> + /* Apply request if needed */
> + if (src_buf->vb2_buf.req_obj.req)
> + v4l2_ctrl_request_setup(src_buf->vb2_buf.req_obj.req,
> + >hdl);
> + if (dst_buf->vb2_buf.req_obj.req &&
> + dst_buf->vb2_buf.req_obj.req != src_buf-
> >vb2_buf.req_obj.req)
> + v4l2_ctrl_request_setup(dst_buf->vb2_buf.req_obj.req,
> + >hdl);
> +
> device_process(ctx, src_buf, dst_buf);
>
> + /* Complete request if needed */
> + if (src_buf->vb2_buf.req_obj.req)
> + v4l2_ctrl_request_complete(src_buf-
> >vb2_buf.req_obj.req,
> + >hdl);
> + if (dst_buf->vb2_buf.req_obj.req &&
> + dst_buf->vb2_buf.req_obj.req != src_buf-
> >vb2_buf.req_obj.req)
> + v4l2_ctrl_request_complete(dst_buf-
> >vb2_buf.req_obj.req,
> + >hdl);
> +
> /* Run a timer, which simulates a hardware irq */
> schedule_irq(dev, ctx->transtime);
> }
> @@ -823,6 +841,8 @@ static void vim2m_stop_streaming(struct vb2_queue
> *q)
> vbuf = v4l2_m2m_dst_buf_remove(ctx-
> >fh.m2m_ctx);
> if (vbuf == NULL)
> return;
> + v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req,
> +>hdl);
> spin_lock_irqsave(>dev->irqlock, flags);
> v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
> spin_unlock_irqrestore(>dev->irqlock, flags);
> @@ -1003,6 +1023,10 @@ static const struct v4l2_m2m_ops m2m_ops = {
> .job_abort = job_abort,
> };
>
> +static const struct media_device_ops m2m_media_ops = {
> + .req_queue = vb2_request_queue,
> +};
> +
> static int vim2m_probe(struct platform_device *pdev)
> {
> struct vim2m_dev *dev;
> @@ -1027,6 +1051,7 @@ static int vim2m_probe(struct platform_device
> *pdev)
> dev->mdev.dev = >dev;
> strlcpy(dev->mdev.model, "vim2m", sizeof(dev->mdev.model));
> media_device_init(>mdev);
> + dev->mdev.ops = _media_ops;
> dev->v4l2_dev.mdev = >mdev;
> dev->pad[0].flags = MEDIA_PAD_FL_SINK;
> dev->pad[1].flags = MEDIA_PAD_FL_SOURCE;
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [RFCv11 PATCH 14/29] v4l2-ctrls: add core request support
= media_request_object_find(req, _ops, main_hdl);
> + if (!obj)
> + return;
> + if (obj->completed) {
> + media_request_object_put(obj);
> + return;
> + }
> + hdl = container_of(obj, struct v4l2_ctrl_handler, req_obj);
> +
> + mutex_lock(hdl->lock);
> +
> + list_for_each_entry(ref, >ctrl_refs, node)
> + ref->done = false;
> +
> + list_for_each_entry(ref, >ctrl_refs, node) {
> + struct v4l2_ctrl *ctrl = ref->ctrl;
> + struct v4l2_ctrl *master = ctrl->cluster[0];
> + bool have_new_data = false;
> + int i;
> +
> + /* Skip if this control was already handled by a
> cluster. */
> + /* Skip button controls and read-only controls. */
> + if (ref->done || ctrl->type == V4L2_CTRL_TYPE_BUTTON
> ||
> + (ctrl->flags & V4L2_CTRL_FLAG_READ_ONLY))
> + continue;
> +
> + v4l2_ctrl_lock(master);
> + for (i = 0; i < master->ncontrols; i++) {
> + if (master->cluster[i]) {
> + struct v4l2_ctrl_ref *r =
> + find_ref(hdl, master-
> >cluster[i]->id);
> +
> + if (r->req && r == r->req) {
> + have_new_data = true;
> + break;
> + }
> + }
> + }
> + if (!have_new_data)
The v4l2 control lock has not been unlocked here, so a call to
v4l2_ctrl_unlock is required before continue.
> + continue;
> +
> + for (i = 0; i < master->ncontrols; i++) {
> + if (master->cluster[i]) {
> + struct v4l2_ctrl_ref *r =
> + find_ref(hdl, master-
> >cluster[i]->id);
> +
> + req_to_new(r);
> + master->cluster[i]->is_new = 1;
> + r->done = true;
> + }
> + }
> + /*
> + * For volatile autoclusters that are currently in
> auto mode
> + * we need to discover if it will be set to manual
> mode.
> + * If so, then we have to copy the current volatile
> values
> + * first since those will become the new manual
> values (which
> + * may be overwritten by explicit new values from
> this set
> + * of controls).
> + */
> + if (master->is_auto && master->has_volatiles &&
> + !is_cur_manual(master)) {
> + s32 new_auto_val = *master->p_new.p_s32;
> +
> + /*
> + * If the new value == the manual value, then
> copy
> + * the current volatile values.
> + */
> + if (new_auto_val == master-
> >manual_mode_value)
> + update_from_auto_cluster(master);
> + }
> +
> + try_or_set_cluster(NULL, master, true, 0);
> +
> + v4l2_ctrl_unlock(master);
> + }
> +
> + mutex_unlock(hdl->lock);
> + media_request_object_put(obj);
> +}
> +EXPORT_SYMBOL(v4l2_ctrl_request_setup);
> +
> void v4l2_ctrl_notify(struct v4l2_ctrl *ctrl, v4l2_ctrl_notify_fnc
> notify, void *priv)
> {
> if (ctrl == NULL)
> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
> index 89a985607126..9499846aa1d1 100644
> --- a/include/media/v4l2-ctrls.h
> +++ b/include/media/v4l2-ctrls.h
> @@ -250,6 +250,10 @@ struct v4l2_ctrl {
> * ``prepare_ext_ctrls`` function at ``v4l2-ctrl.c``.
> * @from_other_dev: If true, then @ctrl was defined in another
> * device then the v4l2_ctrl_handler.
> + * @done:If true, then this control reference is part of a
> + * control cluster that was already set while applying
> + * the controls in this media request object.
> + * @req: If set, this refers to another request that sets this
> control.
> * @p_req: The request value. Only used if the control handler
> * is bound to a media request.
> *
> @@ -263,6 +267,8 @@ struct v4l2_ctrl_ref {
> struct v4l2_ctrl *ctrl;
> struct v4l2_ctrl_helper *helper;
> bool from_other_dev;
> + bool done;
> + struct v4l2_ctrl_ref *req;
> union v4l2_ctrl_ptr p_req;
> };
>
> @@ -287,6 +293,15 @@ struct v4l2_ctrl_ref {
> * @notify_priv: Passed as argument to the v4l2_ctrl notify callback.
> * @nr_of_buckets: Total number of buckets in the array.
> * @error: The error code of the first failed control
> addition.
> + * @request_is_queued: True if the request was queued.
> + * @requests:List to keep track of open control handler
> request objects.
> + * For the parent control handler (@req_obj.req ==
> NULL) this
> + * is the list header. When the parent control handler
> is
> + * removed, it has to unbind and put all these
> requests since
> + * they refer to the parent.
> + * @requests_queued: List of the queued requests. This determines the
> order
> + * in which these controls are applied. Once the
> request is
> + * completed it is removed from this list.
> * @req_obj: The media_request_object, used to link
> into a
> *media_request.
> */
> @@ -301,6 +316,9 @@ struct v4l2_ctrl_handler {
> void *notify_priv;
> u16 nr_of_buckets;
> int error;
> + bool request_is_queued;
> + struct list_head requests;
> + struct list_head requests_queued;
> struct media_request_object req_obj;
> };
>
> @@ -1059,6 +1077,11 @@ int v4l2_ctrl_subscribe_event(struct v4l2_fh
> *fh,
> */
> __poll_t v4l2_ctrl_poll(struct file *file, struct poll_table_struct
> *wait);
>
> +void v4l2_ctrl_request_setup(struct media_request *req,
> + struct v4l2_ctrl_handler *hdl);
> +void v4l2_ctrl_request_complete(struct media_request *req,
> + struct v4l2_ctrl_handler *hdl);
> +
> /* Helpers for ioctl_ops */
>
> /**
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [RFCv4,19/21] media: vim2m: add request support
Hi,
On Tue, 2018-03-13 at 19:24 +0900, Alexandre Courbot wrote:
> On Fri, Mar 9, 2018 at 11:35 PM, Paul Kocialkowski
> <paul.kocialkow...@bootlin.com> wrote:
> > Hi,
> >
> > On Thu, 2018-03-08 at 22:48 +0900, Alexandre Courbot wrote:
> > > Hi Paul!
> > >
> > > Thanks a lot for taking the time to try this! I am also working on
> > > getting it to work with an actual driver, but you apparently found
> > > rough edges that I missed.
> > >
> > > On Thu, Mar 8, 2018 at 1:37 AM, Paul Kocialkowski
> > > <paul.kocialkow...@bootlin.com> wrote:
> > > > On Tue, 2018-02-20 at 13:44 +0900, Alexandre Courbot wrote:
[...]
> > > > > +static int vim2m_request_submit(struct media_request *req,
> > > > > + struct media_request_entity_data
> > > > > *_data)
> > > > > +{
> > > > > + struct v4l2_request_entity_data *data;
> > > > > +
> > > > > + data = to_v4l2_entity_data(_data);
> > > >
> > > > We need to call v4l2_m2m_try_schedule here so that m2m
> > > > scheduling
> > > > can
> > > > happen when only 2 buffers were queued and no other action was
> > > > taken
> > > > from usespace. In that scenario, m2m scheduling currently
> > > > doesn't
> > > > happen.
> > >
> > > I don't think I understand the sequence of events that results in
> > > v4l2_m2m_try_schedule() not being called. Do you mean something
> > > like:
> > >
> > > *
> > > * QBUF on output queue with request set
> > > * QBUF on capture queue
> > > * SUBMIT_REQUEST
> > >
> > > ?
> > >
> > > The call to vb2_request_submit() right after should trigger
> > > v4l2_m2m_try_schedule(), since the buffers associated to the
> > > request
> > > will enter the vb2 queue and be passed to the m2m framework, which
> > > will then call v4l2_m2m_try_schedule(). Or maybe you are thinking
> > > about a different sequence of events?
> >
> > This is indeed the sequence of events that I'm seeing, but the
> > scheduling call simply did not happen on vb2_request_submit. I
> > suppose I will need to investigate some more to find out exactly
> > why.
> >
> > IIRC, the m2m qbuf function is called (and fails to schedule) when
> > the
> > ioctl happens, not when the task is submitted.
> >
> > This issue is seen with vim2m as well as the rencently-submitted
> > sunxi-
> > cedrus driver (with the in-driver calls to v4l2_m2m_try_schedule
> > removed, obviously). If needs be, I could provide a standalone test
> > program to reproduce it.
>
> If you have a standalone program that can reproduce this on vim2m,
> then I would like to see it indeed, if only to understand what I have
> missed.
You can find the test file for this use case at:
https://gist.github.com/paulkocialkowski/4cfa350e1bbe8e3bf714480bba83ea72
Cheers!
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [RFCv4,19/21] media: vim2m: add request support
Hi, On Mon, 2018-03-12 at 17:15 +0900, Tomasz Figa wrote: > Hi Paul, Dmitry, > > On Mon, Mar 12, 2018 at 5:10 PM, Paul Kocialkowski > <paul.kocialkow...@bootlin.com> wrote: > > Hi, > > > > On Sun, 2018-03-11 at 22:42 +0300, Dmitry Osipenko wrote: > > > Hello, > > > > > > On 07.03.2018 19:37, Paul Kocialkowski wrote: > > > > Hi, > > > > > > > > First off, I'd like to take the occasion to say thank-you for > > > > your > > > > work. > > > > This is a major piece of plumbing that is required for me to add > > > > support > > > > for the Allwinner CedarX VPU hardware in upstream Linux. Other > > > > drivers, > > > > such as tegra-vde (that was recently merged in staging) are also > > > > badly > > > > in need of this API. > > > > > > Certainly it would be good to have a common UAPI. Yet I haven't > > > got my > > > hands on > > > trying to implement the V4L interface for the tegra-vde driver, > > > but > > > I've taken a > > > look at Cedrus driver and for now I've one question: > > > > > > Would it be possible (or maybe already is) to have a single IOCTL > > > that > > > takes input/output buffers with codec parameters, processes the > > > request(s) and returns to userspace when everything is done? > > > Having 5 > > > context switches for a single frame decode (like Cedrus VAAPI > > > driver > > > does) looks like a bit of overhead. > > > > The V4L2 interface exposes ioctls for differents actions and I don't > > think there's a combined ioctl for this. The request API was > > introduced > > precisely because we need to have consistency between the various > > ioctls > > needed for each frame. Maybe one single (atomic) ioctl would have > > worked > > too, but that's apparently not how the V4L2 API was designed. > > > > I don't think there is any particular overhead caused by having n > > ioctls > > instead of a single one. At least that would be very surprising > > IMHO. > > Well, there is small syscall overhead, which normally shouldn't be > very painful, although with all the speculative execution hardening, > can't be sure of anything anymore. :) Oh, my mistake then, I had it in mind that it is not really something noticeable. Hopefully, it won't be a limiting factor in our cases. > Hans and Alex can correct me if I'm wrong, but I believe there is a > more atomic-like API being planned, which would only need one IOCTL to > do everything. However, that would be a more serious change to the > V4L2 interfaces, so should be decoupled from Request API itself. > > Best regards, > Tomasz -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
Re: [RFCv4,19/21] media: vim2m: add request support
Hi,
On Sun, 2018-03-11 at 22:42 +0300, Dmitry Osipenko wrote:
> Hello,
>
> On 07.03.2018 19:37, Paul Kocialkowski wrote:
> > Hi,
> >
> > First off, I'd like to take the occasion to say thank-you for your
> > work.
> > This is a major piece of plumbing that is required for me to add
> > support
> > for the Allwinner CedarX VPU hardware in upstream Linux. Other
> > drivers,
> > such as tegra-vde (that was recently merged in staging) are also
> > badly
> > in need of this API.
>
> Certainly it would be good to have a common UAPI. Yet I haven't got my
> hands on
> trying to implement the V4L interface for the tegra-vde driver, but
> I've taken a
> look at Cedrus driver and for now I've one question:
>
> Would it be possible (or maybe already is) to have a single IOCTL that
> takes input/output buffers with codec parameters, processes the
> request(s) and returns to userspace when everything is done? Having 5
> context switches for a single frame decode (like Cedrus VAAPI driver
> does) looks like a bit of overhead.
The V4L2 interface exposes ioctls for differents actions and I don't
think there's a combined ioctl for this. The request API was introduced
precisely because we need to have consistency between the various ioctls
needed for each frame. Maybe one single (atomic) ioctl would have worked
too, but that's apparently not how the V4L2 API was designed.
I don't think there is any particular overhead caused by having n ioctls
instead of a single one. At least that would be very surprising IMHO.
> > I have a few comments based on my experience integrating this
> > request API with the Cedrus VPU driver (and the associated libva
> > backend), that also concern the vim2m driver.
> >
> > On Tue, 2018-02-20 at 13:44 +0900, Alexandre Courbot wrote:
> > > Set the necessary ops for supporting requests in vim2m.
> > >
> > > Signed-off-by: Alexandre Courbot <acour...@chromium.org>
> > > ---
> > > drivers/media/platform/Kconfig | 1 +
> > > drivers/media/platform/vim2m.c | 75
> > > ++
> > > 2 files changed, 76 insertions(+)
> > >
> > > diff --git a/drivers/media/platform/Kconfig
> > > b/drivers/media/platform/Kconfig
> > > index 614fbef08ddc..09be0b5f9afe 100644
> > > --- a/drivers/media/platform/Kconfig
> > > +++ b/drivers/media/platform/Kconfig
> >
> > [...]
> >
> > > +static int vim2m_request_submit(struct media_request *req,
> > > + struct media_request_entity_data
> > > *_data)
> > > +{
> > > + struct v4l2_request_entity_data *data;
> > > +
> > > + data = to_v4l2_entity_data(_data);
> >
> > We need to call v4l2_m2m_try_schedule here so that m2m scheduling
> > can
> > happen when only 2 buffers were queued and no other action was taken
> > from usespace. In that scenario, m2m scheduling currently doesn't
> > happen.
> >
> > However, this requires access to the m2m context, which is not easy
> > to
> > get from req or _data. I'm not sure that some container_of magic
> > would
> > even do the trick here.
> >
> > > + return vb2_request_submit(data);
> >
> > vb2_request_submit does not lock the associated request mutex
> > although
> > it accesses the associated queued buffers list, which I believe this
> > mutex is supposed to protect.
> >
> > We could either wrap this call with media_request_lock(req) and
> > media_request_unlock(req) or have the lock in the function itself,
> > which
> > would require passing it the req pointer.
> >
> > The latter would probably be safer for future use of the function.
> >
> > > +}
> > > +
> > > +static const struct media_request_entity_ops
> > > vim2m_request_entity_ops
> > > = {
> > > + .data_alloc = vim2m_entity_data_alloc,
> > > + .data_free = v4l2_request_entity_data_free,
> > > + .submit = vim2m_request_submit,
> > > +};
> > > +
> > > /*
> > > * File operations
> > > */
> > > @@ -900,6 +967,9 @@ static int vim2m_open(struct file *file)
> > > ctx->dev = dev;
> > > hdl = >hdl;
> > > v4l2_ctrl_handler_init(hdl, 4);
> > > + v4l2_request_entity_init(>req_entity,
> > > _request_entity_ops,
> > > + >dev->vfd);
> > > + ctx->fh.entity = >req_entity.base;
> > > v4l2_ctrl_new_std(hdl, _ctrl_ops, V4L2_CID_HFLIP,
> > >
Re: [RFCv4,19/21] media: vim2m: add request support
Hi,
On Thu, 2018-03-08 at 22:48 +0900, Alexandre Courbot wrote:
> Hi Paul!
>
> Thanks a lot for taking the time to try this! I am also working on
> getting it to work with an actual driver, but you apparently found
> rough edges that I missed.
>
> On Thu, Mar 8, 2018 at 1:37 AM, Paul Kocialkowski
> <paul.kocialkow...@bootlin.com> wrote:
> > Hi,
> >
> > First off, I'd like to take the occasion to say thank-you for your
> > work.
> > This is a major piece of plumbing that is required for me to add
> > support
> > for the Allwinner CedarX VPU hardware in upstream Linux. Other
> > drivers,
> > such as tegra-vde (that was recently merged in staging) are also
> > badly
> > in need of this API.
> >
> > I have a few comments based on my experience integrating this
> > request
> > API with the Cedrus VPU driver (and the associated libva backend),
> > that
> > also concern the vim2m driver.
> >
> > On Tue, 2018-02-20 at 13:44 +0900, Alexandre Courbot wrote:
> > > Set the necessary ops for supporting requests in vim2m.
> > >
> > > Signed-off-by: Alexandre Courbot <acour...@chromium.org>
> > > ---
> > > drivers/media/platform/Kconfig | 1 +
> > > drivers/media/platform/vim2m.c | 75
> > > ++
> > > 2 files changed, 76 insertions(+)
> > >
> > > diff --git a/drivers/media/platform/Kconfig
> > > b/drivers/media/platform/Kconfig
> > > index 614fbef08ddc..09be0b5f9afe 100644
> > > --- a/drivers/media/platform/Kconfig
> > > +++ b/drivers/media/platform/Kconfig
> >
> > [...]
> >
> > > +static int vim2m_request_submit(struct media_request *req,
> > > + struct media_request_entity_data
> > > *_data)
> > > +{
> > > + struct v4l2_request_entity_data *data;
> > > +
> > > + data = to_v4l2_entity_data(_data);
> >
> > We need to call v4l2_m2m_try_schedule here so that m2m scheduling
> > can
> > happen when only 2 buffers were queued and no other action was taken
> > from usespace. In that scenario, m2m scheduling currently doesn't
> > happen.
>
> I don't think I understand the sequence of events that results in
> v4l2_m2m_try_schedule() not being called. Do you mean something like:
>
> *
> * QBUF on output queue with request set
> * QBUF on capture queue
> * SUBMIT_REQUEST
>
> ?
>
> The call to vb2_request_submit() right after should trigger
> v4l2_m2m_try_schedule(), since the buffers associated to the request
> will enter the vb2 queue and be passed to the m2m framework, which
> will then call v4l2_m2m_try_schedule(). Or maybe you are thinking
> about a different sequence of events?
This is indeed the sequence of events that I'm seeing, but the
scheduling call simply did not happen on vb2_request_submit. I suppose I will
need to investigate some more to find out exactly why.
IIRC, the m2m qbuf function is called (and fails to schedule) when the
ioctl happens, not when the task is submitted.
This issue is seen with vim2m as well as the rencently-submitted sunxi-
cedrus driver (with the in-driver calls to v4l2_m2m_try_schedule
removed, obviously). If needs be, I could provide a standalone test
program to reproduce it.
> > However, this requires access to the m2m context, which is not easy
> > to
> > get from req or _data. I'm not sure that some container_of magic
> > would
> > even do the trick here.
>
> data_->entity will give you a pointer to the media_request_entity,
> which is part of vim2m_ctx. You can thus get the m2m context by doing
> container_of(data_->entity, struct vim2m_ctx, req_entity). See
> vim2m_entity_data_alloc() for an example.
Excellent, that's exactly what I was looking for! Thanks a lot and see
the related patch I submitted earlier today.
> > > + return vb2_request_submit(data);
> >
> > vb2_request_submit does not lock the associated request mutex
> > although
> > it accesses the associated queued buffers list, which I believe this
> > mutex is supposed to protect.
>
> After a request is submitted, the data protected by the mutex can only
> be accessed by the driver when it processes the request. It cannot be
> modified concurrently, so I think we are safe here.
Fait enough, that should be enough then. I've dropped this change.
Cheers,
Paul
> I am also wondering whether the ioctl locking doesn't make the request
> locking redundant. Request information can only be modified and
> accessed through ioctls until it is submitted, and after that there
> are no conc
Re: [PATCH 5/9] media: platform: Add Sunxi Cedrus decoder driver
Hi,
On Fri, 2018-03-09 at 14:57 +0100, Maxime Ripard wrote:
> Hi,
>
> On Fri, Mar 09, 2018 at 11:14:41AM +0100, Paul Kocialkowski wrote:
> > +/*
> > + * mem2mem callbacks
> > + */
> > +
> > +void job_abort(void *priv)
> > +{}
>
> Is that still needed?
It looks like we need a dummy callback here, the v4l2_m2m_init function
puts a hard requirement on it.
The feature is definitely not used for now, but maybe this could be
hooked to aborting the matching request? It was probably designed for
the case where the driver handles a queue of jobs on its own (that's how
it's used in vim2m) and such an internal queue is perhaps irrelevant
when using the request API (and fences later).
> > +/*
> > + * device_run() - prepares and starts processing
> > + */
> > +void device_run(void *priv)
> > +{
>
> This function (and the one above) should probably made static. Or at
> least if you can't, they should have a much more specific name in
> order not to conflict with anything from the core.
Good point, let's go for static. Since these are passed as function
pointers, it shouldn't be a problem.
> > + /*
> > +* The VPU is only able to handle bus addresses so we have
> > to subtract
> > +* the RAM offset to the physcal addresses
> > +*/
> > + in_buf -= PHYS_OFFSET;
> > + out_luma -= PHYS_OFFSET;
> > + out_chroma -= PHYS_OFFSET;
>
> You should take care of that by putting it in the dma_pfn_offset field
> of the struct device (at least before we come up with something
> better).
>
> You'll then be able to use the dma_addr_t directly without modifying
> it.
Definitely!
> > + vpu->syscon = syscon_regmap_lookup_by_phandle(vpu->dev-
> > >of_node,
> > + "syscon");
> > + if (IS_ERR(vpu->syscon)) {
> > + vpu->syscon = NULL;
> > + } else {
> > + regmap_write_bits(vpu->syscon,
> > SYSCON_SRAM_CTRL_REG0,
> > + SYSCON_SRAM_C1_MAP_VE,
> > + SYSCON_SRAM_C1_MAP_VE);
> > + }
>
> This should be using our SRAM controller driver (and API), see
> Documentation/devicetree/bindings/sram/sunxi-sram.txt
> include/linux/soc/sunxi/sunxi_sram.h
I'll look into that.
> > + ret = clk_prepare_enable(vpu->ahb_clk);
> > + if (ret) {
> > + dev_err(vpu->dev, "could not enable ahb clock\n");
> > + return -EFAULT;
> > + }
> > + ret = clk_prepare_enable(vpu->mod_clk);
> > + if (ret) {
> > + clk_disable_unprepare(vpu->ahb_clk);
> > + dev_err(vpu->dev, "could not enable mod clock\n");
> > + return -EFAULT;
> > + }
> > + ret = clk_prepare_enable(vpu->ram_clk);
> > + if (ret) {
> > + clk_disable_unprepare(vpu->mod_clk);
> > + clk_disable_unprepare(vpu->ahb_clk);
> > + dev_err(vpu->dev, "could not enable ram clock\n");
> > + return -EFAULT;
> > + }
>
> Ideally, this should be using runtime_pm to manage the device power
> state, and disable it when not used.
I'll add that to my tasks list. I suppose we shouldn't make this a
priority for now, but this is definitely good to have.
> > + reset_control_assert(vpu->rstc);
> > + reset_control_deassert(vpu->rstc);
>
> You can use reset_control_reset here
Noted!
> > + return 0;
> > +}
> > +
> > +void sunxi_cedrus_hw_remove(struct sunxi_cedrus_dev *vpu)
> > +{
> > + clk_disable_unprepare(vpu->ram_clk);
> > + clk_disable_unprepare(vpu->mod_clk);
> > + clk_disable_unprepare(vpu->ahb_clk);
>
> The device is not put back into reset here
Good catch, thanks!
Cheers,
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [linux-sunxi] [PATCH 6/9] sunxi-cedrus: Add device tree binding document
Hi,
Thanks for the review!
On Fri, 2018-03-09 at 15:38 +0200, Priit Laes wrote:
> On Fri, Mar 09, 2018 at 11:14:42AM +0100, Paul Kocialkowski wrote:
> > From: Florent Revest <florent.rev...@free-electrons.com>
> >
> > Device Tree bindings for the Allwinner's video engine
> >
> > Signed-off-by: Florent Revest <florent.rev...@free-electrons.com>
> > ---
> > .../devicetree/bindings/media/sunxi-cedrus.txt | 44
> > ++
> > 1 file changed, 44 insertions(+)
> > create mode 100644 Documentation/devicetree/bindings/media/sunxi-
> > cedrus.txt
> >
> > diff --git a/Documentation/devicetree/bindings/media/sunxi-
> > cedrus.txt b/Documentation/devicetree/bindings/media/sunxi-
> > cedrus.txt
> > new file mode 100644
> > index ..138581113c49
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > @@ -0,0 +1,44 @@
> > +Device-Tree bindings for SUNXI video engine found in sunXi SoC
> > family
> > +
> > +Required properties:
> > +- compatible : "allwinner,sun4i-a10-video-engine";
> > +- memory-region : DMA pool for buffers allocation;
> > +- clocks : list of clock specifiers, corresponding to
> > + entries in clock-names property;
> > +- clock-names : should contain "ahb", "mod" and "ram"
> > entries;
> > +- resets : phandle for reset;
> > +- interrupts : should contain VE interrupt number;
> > +- reg : should contain register base and length
> > of VE.
> > +
> > +Example:
> > +
> > +reserved-memory {
> > + #address-cells = <1>;
> > + #size-cells = <1>;
> > + ranges;
> > +
> > + ve_reserved: cma {
> > + compatible = "shared-dma-pool";
> > + reg = <0x43d0 0x900>;
> > + no-map;
> > + linux,cma-default;
> > + };
> > +};
> > +
> > +video-engine {
> > + compatible = "allwinner,sun4i-a10-video-engine";
> > + memory-region = <_reserved>;
> > +
> > + clocks = <_gates 32>, < CLK_VE>,
> > +<_gates 0>;
>
> This should be updated to sunxi-ng clocks:
>
> clocks = < CLK_BUS_VE>, < CLK_VE>, < CLK_DRAM_VE>;
I will definitely keep that in mind and make the change for the next
revision, thanks!
> > + clock-names = "ahb", "mod", "ram";
> > +
> > + assigned-clocks = < CLK_VE>;
> > + assigned-clock-rates = <32000>;
> > +
> > + resets = < RST_VE>;
> > +
> > + interrupts = <53>;
> > +
> > + reg = <0x01c0e000 4096>;
> > +};
> > --
> > 2.16.2
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "linux-sunxi" group.
> > To unsubscribe from this group and stop receiving emails from it,
> > send an email to linux-sunxi+unsubscr...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
signature.asc
Description: This is a digitally signed message part
Re: [PATCH 0/9] Sunxi-Cedrus driver for the Allwinner Video Engine, using the V4L2 request API
On Fri, 2018-03-09 at 11:09 +0100, Paul Kocialkowski wrote: > This presents a newer version of the Sunxi-Cedrus driver, that > supports > the Video Engine found in most Allwinner SoCs, starting with the A10. I had to send this series in two parts (and tried to keep them under the original thread) after the mail server rejected one of the emails mid- series, causing git-send-email to stop. Sorry for the mess. > The first version of this driver[0] was originally written and > submitted > by Florent Revest using a previous version of the request API, that is > necessary to provide coherency between controls and the buffers they > apply > to. The driver was since adapted to use the latest version of the > request > API[1], as submitted by Alexandre Courbot. It is a hard requirement > for > this driver. > > This series also contains fixes for issues encountered with the > current > version of the request API. If accepted, these should eventually be > squashed into the request API series. > > The driver itself currently only supports MPEG2 and more codecs will > be > added to the driver eventually. The output frames provided by the > Video Engine are in a multi-planar 32x32-tiled YUV format, with a > plane > for luminance (Y) and a plane for chrominance (UV). A specific format > is > introduced in the V4L2 API to describe it. > > This implementation is based on the significant work that was > conducted > by various members of the linux-sunxi community for understanding and > documenting the Video Engine's innards. > > [0]: https://lkml.org/lkml/2016/8/25/246 > [1]: https://lkml.org/lkml/2018/2/19/872 > > Florent Revest (5): > v4l: Add sunxi Video Engine pixel format > v4l: Add MPEG2 low-level decoder API control > media: platform: Add Sunxi Cedrus decoder driver > sunxi-cedrus: Add device tree binding document > ARM: dts: sun5i: Use video-engine node > > Icenowy Zheng (1): > ARM: dts: sun8i: add video engine support for A33 > > Paul Kocialkowski (2): > media: vim2m: Try to schedule a m2m device run on request submission > media: videobuf2-v4l2: Copy planes when needed in request qbuf > > Thomas van Kleef (1): > ARM: dts: sun7i: Add video engine support for the A20 > > .../devicetree/bindings/media/sunxi-cedrus.txt | 44 ++ > arch/arm/boot/dts/sun5i-a13.dtsi | 30 ++ > arch/arm/boot/dts/sun7i-a20.dtsi | 47 ++ > arch/arm/boot/dts/sun8i-a33.dtsi | 39 ++ > drivers/media/common/videobuf2/videobuf2-v4l2.c| 19 + > drivers/media/platform/Kconfig | 14 + > drivers/media/platform/Makefile| 1 + > drivers/media/platform/sunxi-cedrus/Makefile | 4 + > drivers/media/platform/sunxi-cedrus/sunxi_cedrus.c | 313 > .../platform/sunxi-cedrus/sunxi_cedrus_common.h| 106 > .../media/platform/sunxi-cedrus/sunxi_cedrus_dec.c | 568 > + > .../media/platform/sunxi-cedrus/sunxi_cedrus_dec.h | 33 ++ > .../media/platform/sunxi-cedrus/sunxi_cedrus_hw.c | 185 +++ > .../media/platform/sunxi-cedrus/sunxi_cedrus_hw.h | 36 ++ > .../platform/sunxi-cedrus/sunxi_cedrus_mpeg2.c | 152 ++ > .../platform/sunxi-cedrus/sunxi_cedrus_regs.h | 170 ++ > drivers/media/platform/vim2m.c | 13 +- > drivers/media/v4l2-core/v4l2-ctrls.c | 15 + > drivers/media/v4l2-core/v4l2-ioctl.c | 1 + > include/uapi/linux/v4l2-controls.h | 26 + > include/uapi/linux/videodev2.h | 6 + > 21 files changed, 1821 insertions(+), 1 deletion(-) > create mode 100644 Documentation/devicetree/bindings/media/sunxi- > cedrus.txt > create mode 100644 drivers/media/platform/sunxi-cedrus/Makefile > create mode 100644 drivers/media/platform/sunxi-cedrus/sunxi_cedrus.c > create mode 100644 drivers/media/platform/sunxi- > cedrus/sunxi_cedrus_common.h > create mode 100644 drivers/media/platform/sunxi- > cedrus/sunxi_cedrus_dec.c > create mode 100644 drivers/media/platform/sunxi- > cedrus/sunxi_cedrus_dec.h > create mode 100644 drivers/media/platform/sunxi- > cedrus/sunxi_cedrus_hw.c > create mode 100644 drivers/media/platform/sunxi- > cedrus/sunxi_cedrus_hw.h > create mode 100644 drivers/media/platform/sunxi- > cedrus/sunxi_cedrus_mpeg2.c > create mode 100644 drivers/media/platform/sunxi- > cedrus/sunxi_cedrus_regs.h > -- Paul Kocialkowski, Bootlin (formerly Free Electrons) Embedded Linux and kernel engineering https://bootlin.com signature.asc Description: This is a digitally signed message part
