Hi all,
While working on extending v4l2-compliance with cropping/selection test cases
I decided to add support for that to vivi as well (this would give applications
a good test driver to work with).
However, I ran into problems how this should be implemented for V4L2 devices
(we are not talking about complex media controller devices where the video
pipelines are setup manually).
There are two problems, one related to ENUM_FRAMESIZES and one to S_FMT.
The ENUM_FRAMESIZES issue is simple: if you have a sensor that has several
possible frame sizes, and that can crop, compose and/or scale, then you need
to be able to set the frame size. Currently this is decided by S_FMT which
maps the format size to the closest valid frame size. This however makes
it impossible to e.g. scale up a frame, or compose the image into a larger
buffer.
For video receivers this issue doesn't exist: there the size of the incoming
video is decided by S_STD or S_DV_TIMINGS, but no equivalent exists for sensors.
I propose that a new selection target is added: V4L2_SEL_TGT_FRAMESIZE.
However, this leads to another problem: the current S_FMT behavior is that
it implicitly sets the framesize. That behavior we will have to keep, otherwise
applications will start to behave differently.
I have an idea on how to solve that, but the solution is related to the second
problem I found:
When you set a new format size, then the compose rectangle must be set to the
new format size as well since that has always been the behavior in the past
that applications have come to expect.
But this makes certain operations impossible to execute: if a driver can't
scale, then you can never select a new format size larger than the current
(possibly cropped) frame size, even though you would want to compose the
unscaled image into such a larger buffer.
So what is the behavior that I would expect from drivers?
1) After calling S_STD, S_DV_TIMINGS or S_SELECTION(V4L2_SEL_TGT_FRAMESIZE)
the cropping, composing and format parameters are all adjusted to support the
new input video size (typically they are all set to the new size).
2) After calling S_CROP/S_SELECTION(CROP) the compose and format parameters are
all
adjusted to support the new crop rectangle.
3) After calling S_SEL(COMPOSE) the format parameters are adjusted.
4) Calling S_FMT validates the format parameters to support the compose
rectangle.
However, today step 4 does something different: the compose rectangle will be
adjusted to the format size (and in the case of a sensor supporting different
framesizes the whole pipeline will be adjusted).
The only way I see that would solve this (although it isn't perfect) is to
change the behavior of S_FMT only if the selection API was used before by the
filehandle. The core can keep easily keep track of that. When the application
calls S_FMT and no selection API was used in the past by that filehandle, then
the core will call first S_SELECTION(V4L2_SEL_TGT_FRAMESIZE). If that returns
-EINVAL, then it will call S_SELECTION(V4L2_SEL_TGT_COMPOSE). Finally it will
call S_FMT. Note that a similar sequence is needed for the display case.
This means that a driver supporting the selection API can implement the logical
behavior and the core will implement the historically-required illogical part.
So the fix for this would be to add a new selection target and add compatibility
code to the v4l2-core.
With that in place I can easily add crop/compose support to vivi.
One area of uncertainty is how drivers currently implement S_FMT: do they reset
any cropping done before? They should keep the crop rectangle according to the
spec (well, it is implied there). Guennadi, what does soc_camera do?
Sylwester, I am also looking at exynos4-is/fimc-lite.c. I do see that setting
the compose rectangle will adjust it to the format size instead of the other
way around, but I can't tell if setting the format size will also adjust the
compose rectangle if that is now out-of-bounds of the new format size.
Comments? Questions?
Regards,
Hans
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
