[FFmpeg-devel] [PATCH v4] avfilter: add libdewobble filter
All of the existing filters for video stabilization use an affine model
(or a limited version of it) to represent the movement of the camera. When
used with cameras with a very wide field of view and/or where the camera
shaking is severe, the corrections result in significant geometric
distortion ("wobbling").
Dewobble (https://git.sr.ht/~hedgepigdaniel/dewobble) is a library built
to solve this problem. It requires knowledge of the projection used by
the input camera, and it performs stabilization using a homography
model, which is limited to include only changes in camera orientation.
Additionally, it can perform projection change by specifying a different
projection for the output camera. This is more efficient and results in
less loss of information than using separate filters to perform
stabilization and projection change.
The libdewobble filter is a wrapper for Dewobble. Dewobble supports
input and output in OpenCL buffers containing NV12 frames. Hence, the
filter has the same limitations. Currently
all of the options of Dewobble are supported. Of the two types of filter
available in Dewobble (FilterSync and FilterThreaded), FilterThreaded is
used. The API is synchronous, but the transformations are done in a
separate thread. The purpose of this is to isolate the global per thread
OpenCL context used by OpenCV, which Dewobble uses internally. This
prevents dewobble_opencl from interfering with any other usage of OpenCV
from within FFmpeg.
Signed-off-by: Daniel Playfair Cal
---
Changelog v4:
- correct rename in documentation
- update examples to include hwupload/hwdownload, now that name does
not contain _opencl postfix.
---
Changelog|1 +
LICENSE.md |2 +-
configure|4 +
doc/filters.texi | 149
libavfilter/Makefile |1 +
libavfilter/allfilters.c |1 +
libavfilter/vf_libdewobble.c | 1273 ++
7 files changed, 1430 insertions(+), 1 deletion(-)
create mode 100644 libavfilter/vf_libdewobble.c
diff --git a/Changelog b/Changelog
index 20d71bed88..b4e0a4f4f9 100644
--- a/Changelog
+++ b/Changelog
@@ -12,6 +12,7 @@ version :
- audio and video segment filters
- Apple Graphics (SMC) encoder
- hsvkey and hsvhold video filters
+- libdewobble video filter
version 4.4:
diff --git a/LICENSE.md b/LICENSE.md
index 613070e1b6..dfdf010d8e 100644
--- a/LICENSE.md
+++ b/LICENSE.md
@@ -112,7 +112,7 @@ The VMAF, mbedTLS, RK MPI, OpenCORE and VisualOn libraries
are under the Apache
version 3 of those licenses. So to combine these libraries with FFmpeg, the
license version needs to be upgraded by passing `--enable-version3` to
configure.
-The smbclient library is under the GPL v3, to combine it with FFmpeg,
+The dewobble and smbclient libraries are under the GPL v3, to combine them
with FFmpeg,
the options `--enable-gpl` and `--enable-version3` have to be passed to
configure to upgrade FFmpeg to the GPL v3.
diff --git a/configure b/configure
index 9249254b70..00f3e8deea 100755
--- a/configure
+++ b/configure
@@ -230,6 +230,7 @@ External library support:
--enable-libdavs2enable AVS2 decoding via libdavs2 [no]
--enable-libdc1394 enable IIDC-1394 grabbing using libdc1394
and libraw1394 [no]
+ --enable-libdewobble enable video stabilization via libdewobble [no]
--enable-libfdk-aac enable AAC de/encoding via libfdk-aac [no]
--enable-libfliteenable flite (voice synthesis) support via libflite
[no]
--enable-libfontconfig enable libfontconfig, useful for drawtext filter
[no]
@@ -1781,6 +1782,7 @@ EXTERNAL_LIBRARY_VERSION3_LIST="
"
EXTERNAL_LIBRARY_GPLV3_LIST="
+libdewobble
libsmbclient
"
@@ -3606,6 +3608,7 @@ interlace_filter_deps="gpl"
kerndeint_filter_deps="gpl"
ladspa_filter_deps="ladspa libdl"
lensfun_filter_deps="liblensfun version3"
+libdewobble_filter_deps="libdewobble opencl"
lv2_filter_deps="lv2"
mcdeint_filter_deps="avcodec gpl"
metadata_filter_deps="avformat"
@@ -6406,6 +6409,7 @@ enabled libcodec2 && require libcodec2
codec2/codec2.h codec2_create -lc
enabled libdav1d && require_pkg_config libdav1d "dav1d >= 0.5.0"
"dav1d/dav1d.h" dav1d_version
enabled libdavs2 && require_pkg_config libdavs2 "davs2 >= 1.6.0"
davs2.h davs2_decoder_open
enabled libdc1394 && require_pkg_config libdc1394 libdc1394-2
dc1394/dc1394.h dc1394_new
+enabled libdewobble && require_pkg_config libdewobble dewobble
dewobble/filter.h dewobble_filter_create_threaded
enabled libdrm&& require_pkg_config libdrm libdrm xf86drm.h
drmGetVersion
enabled libfdk_aac&& { check_pkg_config libfdk_aac fdk-aac
"fdk-aac/aace
[FFmpeg-devel] [PATCH v3] avfilter: add libdewobble filter
All of the existing filters for video stabilization use an affine model
(or a limited version of it) to represent the movement of the camera. When
used with cameras with a very wide field of view and/or where the camera
shaking is severe, the corrections result in significant geometric
distortion ("wobbling").
Dewobble (https://git.sr.ht/~hedgepigdaniel/dewobble) is a library built
to solve this problem. It requires knowledge of the projection used by
the input camera, and it performs stabilization using a homography
model, which is limited to include only changes in camera orientation.
Additionally, it can perform projection change by specifying a different
projection for the output camera. This is more efficient and results in
less loss of information than using separate filters to perform
stabilization and projection change.
The libdewobble filter is a wrapper for Dewobble. Dewobble supports
input and output in OpenCL buffers containing NV12 frames. Hence, the
filter has the same limitations. Currently
all of the options of Dewobble are supported. Of the two types of filter
available in Dewobble (FilterSync and FilterThreaded), FilterThreaded is
used. The API is synchronous, but the transformations are done in a
separate thread. The purpose of this is to isolate the global per thread
OpenCL context used by OpenCV, which Dewobble uses internally. This
prevents dewobble_opencl from interfering with any other usage of OpenCV
from within FFmpeg.
Signed-off-by: Daniel Playfair Cal
---
Changelog v3:
- rename filter to "libdewobble"
- capitalize first word of filter description
---
Changelog|1 +
LICENSE.md |2 +-
configure|4 +
doc/filters.texi | 149
libavfilter/Makefile |1 +
libavfilter/allfilters.c |1 +
libavfilter/vf_libdewobble.c | 1273 ++
7 files changed, 1430 insertions(+), 1 deletion(-)
create mode 100644 libavfilter/vf_libdewobble.c
diff --git a/Changelog b/Changelog
index 20d71bed88..b4e0a4f4f9 100644
--- a/Changelog
+++ b/Changelog
@@ -12,6 +12,7 @@ version :
- audio and video segment filters
- Apple Graphics (SMC) encoder
- hsvkey and hsvhold video filters
+- libdewobble video filter
version 4.4:
diff --git a/LICENSE.md b/LICENSE.md
index 613070e1b6..dfdf010d8e 100644
--- a/LICENSE.md
+++ b/LICENSE.md
@@ -112,7 +112,7 @@ The VMAF, mbedTLS, RK MPI, OpenCORE and VisualOn libraries
are under the Apache
version 3 of those licenses. So to combine these libraries with FFmpeg, the
license version needs to be upgraded by passing `--enable-version3` to
configure.
-The smbclient library is under the GPL v3, to combine it with FFmpeg,
+The dewobble and smbclient libraries are under the GPL v3, to combine them
with FFmpeg,
the options `--enable-gpl` and `--enable-version3` have to be passed to
configure to upgrade FFmpeg to the GPL v3.
diff --git a/configure b/configure
index 9249254b70..00f3e8deea 100755
--- a/configure
+++ b/configure
@@ -230,6 +230,7 @@ External library support:
--enable-libdavs2enable AVS2 decoding via libdavs2 [no]
--enable-libdc1394 enable IIDC-1394 grabbing using libdc1394
and libraw1394 [no]
+ --enable-libdewobble enable video stabilization via libdewobble [no]
--enable-libfdk-aac enable AAC de/encoding via libfdk-aac [no]
--enable-libfliteenable flite (voice synthesis) support via libflite
[no]
--enable-libfontconfig enable libfontconfig, useful for drawtext filter
[no]
@@ -1781,6 +1782,7 @@ EXTERNAL_LIBRARY_VERSION3_LIST="
"
EXTERNAL_LIBRARY_GPLV3_LIST="
+libdewobble
libsmbclient
"
@@ -3606,6 +3608,7 @@ interlace_filter_deps="gpl"
kerndeint_filter_deps="gpl"
ladspa_filter_deps="ladspa libdl"
lensfun_filter_deps="liblensfun version3"
+libdewobble_filter_deps="libdewobble opencl"
lv2_filter_deps="lv2"
mcdeint_filter_deps="avcodec gpl"
metadata_filter_deps="avformat"
@@ -6406,6 +6409,7 @@ enabled libcodec2 && require libcodec2
codec2/codec2.h codec2_create -lc
enabled libdav1d && require_pkg_config libdav1d "dav1d >= 0.5.0"
"dav1d/dav1d.h" dav1d_version
enabled libdavs2 && require_pkg_config libdavs2 "davs2 >= 1.6.0"
davs2.h davs2_decoder_open
enabled libdc1394 && require_pkg_config libdc1394 libdc1394-2
dc1394/dc1394.h dc1394_new
+enabled libdewobble && require_pkg_config libdewobble dewobble
dewobble/filter.h dewobble_filter_create_threaded
enabled libdrm&& require_pkg_config libdrm libdrm xf86drm.h
drmGetVersion
enabled libfdk_aac&& { check_pkg_config libfdk_aac fdk-aac
"fdk-aac/aacenc_lib.h" aacEncOpen ||
Re: [FFmpeg-devel] [PATCH v2] avfilter: add libdewobble_opencl filter
On Tue, Aug 24, 2021 at 11:42 PM Mapul Bhola wrote: > I agree with Mahol here. It's good to make sure all the code in FFmpeg meets > a certain quality. > I thought there were OpenCV filters in ffmpeg already? I'm more than happy to address any issues of code quality in this filter to go into FFmpeg - please let me know if you can see something specific that I haven't addressed. I don't think using OpenCV is a quality issue. OpenCV is an established and well maintained library, and the specific algorithms used from it in Dewobble are well used and tested. Within Dewobble there are some functionalities that are implemented natively and others that use OpenCV implementations. My choices have depended on how much specialisation is needed and the relative difficulty of implementing the algorithms. For example, I wrote my own OpenCL kernels to build the final map for warping and for colour conversion. But I didn't write my own implementation of warping/interpolation, Shi-Tomasi corner detection, Lucas-Kanade optical flow, RANSAC, etc. There is no point IMO making such a huge effort to recreate what is already there in OpenCV, unless there is a good reason to think the result would be better. There are also alternative algorithms for video stabilization, many of which are also implemented in OpenCV. So it's easy to experiment with different methods without having to implement complex computer vision algorithms each time. For those parts that don't change so much, which are customised more, or which could be done better than by using OpenCV, I will probably slowly implement algorithms in Dewobble (and any help is welcome). And yes, there is an existing filter "ocv" (vf_libopencv.c) which wraps a very specific set of functionalities in OpenCV, from its image filtering category. These functionalities are unrelated to this filter or to Dewobble. There is also a filter "deshake_opencl" which doesn't depend on OpenCV but contains code copied from some of it's OpenCL kernels. > And if you are the writer of this plugin as well, you should consider > relicensing it to LGPL for its usage in ffmpeg. I considered this but I've decided to license Dewobble as GPL. I realise this prevents it from being used in closed source or permissively licensed libraries that links to it, but it can still be usable for end users of FFmpeg and within other GPL software. FFmpeg already has build infrastructure to support this, as well as multiple existing filters which are also licensed under GPL. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v2] avfilter: add libdewobble_opencl filter
On Tue, Aug 24, 2021 at 3:09 AM Paul B Mahol wrote:
> library is named dewobble, thus filter should be libdewobble.
Lynne suggested "libdewobble_opencl". I can rename it to "libdewobble"
if you prefer, but will everyone be happy with that?
Perhaps it will be easier if you explain the convention and your
reasoning for the name. All the other filters which work with OpenCL
hardware frames are postfixed with "_opencl". All the other filters
which wrap external libraries are not prefixed with "lib" (with the
exception of "libvmaf"). Why is this filter different in both senses?
> no, libdewobble video filter
OK, I will update this to match the name of the filter, whatever that is.
>> +
>> +/**
>> + * Camera properties, mirroring those present in libdewobble's camera
>> object.
>> + */
>> +typedef struct Camera {
>> +/**
>> + * Camera projection model, e.g. `DEWOBBLE_PROJECTION_RECTILINEAR`
>> + */
>> +int model;
>> +
>> +/**
>> + * Camera diagonal field of view in degrees
>> + */
>> +double diagonal_fov;
>> +
>> +/**
>> + * Width in pixels
>> + */
>> +int width;
>> +
>> +/**
>> + * Height in pixels
>> + */
>> +int height;
>> +
>> +/**
>> + * Horizonal coordinate of focal point in pixels
>> + */
>> +double focal_point_x;
>> +
>> +/**
>> + * Vertical coordinate of focal point in pixels
>> + */
>> +double focal_point_y;
>> +} Camera;
>> +
>> +/**
>> + * Motion stabilization algorithm, mirroring those available in libdewobble.
>> + */
>> +typedef enum StabilizationAlgorithm {
>> +
>> +/**
>> + * Do not apply stabilization
>> + */
>> +STABILIZATION_ALGORITHM_ORIGINAL,
>> +
>> +/**
>> + * Keep the camera orientation fixed at its orientation in the first
>> frame
>> + */
>> +STABILIZATION_ALGORITHM_FIXED,
>> +
>> +/**
>> + * Smooth camera orientation with a Savitsky-Golay filter
>> + */
>> +STABILIZATION_ALGORITHM_SMOOTH,
>> +
>> +/**
>> + * Number of stabilization algorithms
>> + */
>> +NB_STABILIZATION_ALGORITHMS,
>> +
>> +} StabilizationAlgorithm;
>> +
>
>
> Huh? Why this and bellow similar stuff are not part of library?
Because the FFmpeg filter options work by writing values to offsets in
memory, but this is not supported in libdewobble. So there are
intermediate structs/enums in the filter context where options are
written to, which are later passed to the libdewobble API in the
appropriate way. e.g. see the use of dewobble_filter_config_create()
and stabilization_algorithm.
> Apply
Ok, will fix.
> I really dislike GPL3 and use of opencv.
> I would prefer pure C solution.
You are welcome to use the same ideas and write a native C filter with
LGPL license if you want. But there will be alot more code to write.
OpenCV provides sparse optical flow, RANSAC, warping with 5
interpolation algorithms, text/shape drawing utilities, etc. And most
of these with OpenCL acceleration or optimized multithreaded CPU
implementations.
Use of OpenCV is an implementation detail of libdewobble. OpenCV is
not used directly in the filter, and no OpenCV objects are passed to
or received from libdewobble.
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2] avfilter: add libdewobble_opencl filter
All of the existing filters for video stabilization use an affine model
(or a limited version of it) to represent the movement of the camera. When
used with cameras with a very wide field of view and/or where the camera
shaking is severe, the corrections result in significant geometric
distortion ("wobbling").
Dewobble (https://git.sr.ht/~hedgepigdaniel/dewobble) is a library built
to solve this problem. It requires knowledge of the projection used by
the input camera, and it performs stabilization using a homography
model, which is limited to include only changes in camera orientation.
Additionally, it can perform projection change by specifying a different
projection for the output camera. This is more efficient and results in
less loss of information than using separate filters to perform
stabilization and projection change.
The dewobble_opencl filter is a wrapper for Dewobble. Dewobble supports
input and output in OpenCL buffers containing NV12 frames. Hence, the
filter is named dewobble_opencl and has the same limitations. Currently
all of the options of Dewobble are supported. Of the two types of filter
available in Dewobble (FilterSync and FilterThreaded), FilterThreaded is
used. The API is synchronous, but the transformations are done in a
separate thread. The purpose of this is to isolate the global per thread
OpenCL context used by OpenCV, which Dewobble uses internally. This
prevents dewobble_opencl from interfering with any other usage of OpenCV
from within FFmpeg.
Signed-off-by: Daniel Playfair Cal
---
Changelog v2:
- style improvements
- rename from "dewobble_opencl" to "libdewobble_opencl"
I'm still confused as to why this filter should be prefixed with lib but
not others that wrap external libraries like lensfun, vidstabtransform,
vidstabdetect, etc. In any case, I've renamed as you requested.
I've addressed all the comments about style as well as doing a general
cleanup. Function arguments are arranged in a different way which
doesn't result in so many new lines.
---
Changelog |1 +
LICENSE.md |2 +-
configure |4 +
doc/filters.texi| 149
libavfilter/Makefile|1 +
libavfilter/allfilters.c|1 +
libavfilter/version.h |2 +-
libavfilter/vf_libdewobble_opencl.c | 1273 +++
8 files changed, 1431 insertions(+), 2 deletions(-)
create mode 100644 libavfilter/vf_libdewobble_opencl.c
diff --git a/Changelog b/Changelog
index 5a5b50eb66..a8d71ab4ee 100644
--- a/Changelog
+++ b/Changelog
@@ -11,6 +11,7 @@ version :
- afwtdn audio filter
- audio and video segment filters
- Apple Graphics (SMC) encoder
+- Dewobble filter
version 4.4:
diff --git a/LICENSE.md b/LICENSE.md
index 613070e1b6..dfdf010d8e 100644
--- a/LICENSE.md
+++ b/LICENSE.md
@@ -112,7 +112,7 @@ The VMAF, mbedTLS, RK MPI, OpenCORE and VisualOn libraries
are under the Apache
version 3 of those licenses. So to combine these libraries with FFmpeg, the
license version needs to be upgraded by passing `--enable-version3` to
configure.
-The smbclient library is under the GPL v3, to combine it with FFmpeg,
+The dewobble and smbclient libraries are under the GPL v3, to combine them
with FFmpeg,
the options `--enable-gpl` and `--enable-version3` have to be passed to
configure to upgrade FFmpeg to the GPL v3.
diff --git a/configure b/configure
index 9249254b70..60b3d3dbea 100755
--- a/configure
+++ b/configure
@@ -230,6 +230,7 @@ External library support:
--enable-libdavs2enable AVS2 decoding via libdavs2 [no]
--enable-libdc1394 enable IIDC-1394 grabbing using libdc1394
and libraw1394 [no]
+ --enable-libdewobble enable video stabilization via libdewobble [no]
--enable-libfdk-aac enable AAC de/encoding via libfdk-aac [no]
--enable-libfliteenable flite (voice synthesis) support via libflite
[no]
--enable-libfontconfig enable libfontconfig, useful for drawtext filter
[no]
@@ -1781,6 +1782,7 @@ EXTERNAL_LIBRARY_VERSION3_LIST="
"
EXTERNAL_LIBRARY_GPLV3_LIST="
+libdewobble
libsmbclient
"
@@ -3606,6 +3608,7 @@ interlace_filter_deps="gpl"
kerndeint_filter_deps="gpl"
ladspa_filter_deps="ladspa libdl"
lensfun_filter_deps="liblensfun version3"
+libdewobble_opencl_filter_deps="libdewobble opencl"
lv2_filter_deps="lv2"
mcdeint_filter_deps="avcodec gpl"
metadata_filter_deps="avformat"
@@ -6406,6 +6409,7 @@ enabled libcodec2 && require libcodec2
codec2/codec2.h codec2_create -lc
enabled libdav1d && require_pkg_config libdav1d "dav1d >= 0.5.0"
"dav1d/dav1d.h" dav1d_version
enabled libdavs2 && require_pkg_config libd
Re: [FFmpeg-devel] [PATCH] avfilter: add dewobble_opencl filter
Thanks for the feedback. On Sun Aug 15 14:11:27 EEST 2021 Lynne wrote: > > 15 Aug 2021, 11:13 by daniel.playfair.cal at gmail.com: > > > On Sun, Aug 15, 2021 at 6:18 PM Paul B Mahol wrote: > > > >> Non native filters can not be named like this. > > All library wrappers must be prefixed with 'lib', so 'libdewobble_opencl'. Ok, I can do that, but I'm a little confused though what the convention is. Other comparable filters which wrap an external library such as lensfun and vidstabdetect/vidstabtransform are not named with a lib prefix. Is this a recent change in policy, or are these filters not comparable for some reason? > Why are there mandatory GPU memcpys on both the input and output? > Can't the library be made to work with images rather than buffers? The images do eventually need to be converted to buffers because that is all that OpenCV supports. Having said that, there are probably some optimizations that could be done if Dewobble had access to the original images. For example, the NV12 -> BGR conversion could probably be combined into the same kernel with the image -> buffer conversion (and vice versa). However there are a few complicating/limiting factors: - OpenCL frames that were mapped from VA-API frames still occupy the very limited VA-API frame pool, so it's necessary to avoid pulling too many of them without first freeing those that are already held - libdewobble requires keeping a queue of frames in memory in some form (how many depends on the options) - libdewobble currently directly uses the luma portion of the input frames, including the luma portion of previously pushed frames - dewobble_filter_push_frame (for the threaded version of DewobbleFilter as is used here) doesn't currently block on anything more than pushing the frame onto a queue. Doing so (i.e. to avoid keeping a reference to the input image) would require more message passing between the main thread and the worker thread (where it's safe to use OpenCV without disturbing any global context). So TL;DR, I agree that it's a good idea to pass the images directly to Dewobble, and I think it probably possible to avoid some/all of the copying depending on the options. However it would be a significant change, so I'd prefer to add this ability to libdewobble at a later time, by adding a new input/output format on top of the existing NV12 OpenCL buffer. At that point I'll be able to submit another patch to make use of it and avoid these copies in the FFmpeg filter. > > /// Motion stabilization algorithm, mirroring those available in > > libdewobble. > Can't it just use the defines from libdewobble instead? There are no defines in libdewobble for motion stabilization algorithms. The stabilizers are each separate classes with separate constructors and varying parameters. > Finally, coding style, follow the coding style. No brackets around > 1-line statements, > function arguments do not go on one line each, for (int i) is > supported, and don't mix > C++ and C-style comments, just use C, since that's what we use most > often, we don't > strictly enforce the 80-char line guideline if the code looks like a > mess after newlines. Ok. I ran `indent -i4 -kr -nut` as is suggested on https://ffmpeg.org/developer.html - can you suggest alternative options that would fit with your requirements? Otherwise I'll change to C style comments and fumble with `indent` or `clang-format` options until they do what you suggest. > function arguments do not go on one line each This one in particular is not clear to me. If not on one line each, then where? They are not currently all on separate lines, but putting them all on one line would result in some unreasonably long lines. On Sun, Aug 15, 2021 at 7:13 PM Daniel Playfair Cal wrote: > > On Sun, Aug 15, 2021 at 6:18 PM Paul B Mahol wrote: > > > Non native filters can not be named like this. > > OK, how would you suggest to name it - just "dewobble"? > > My thinking was that since the user must ensure that the input/output > is OpenCL hardware frames (e.g. using hwupload/hwmap), the "_opencl" > postfix in the name would serve a hint that this is necessary (as it > is for all the native OpenCL filters). ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] avfilter: add dewobble_opencl filter
On Sun, Aug 15, 2021 at 6:18 PM Paul B Mahol wrote: > Non native filters can not be named like this. OK, how would you suggest to name it - just "dewobble"? My thinking was that since the user must ensure that the input/output is OpenCL hardware frames (e.g. using hwupload/hwmap), the "_opencl" postfix in the name would serve a hint that this is necessary (as it is for all the native OpenCL filters). ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] avfilter: add dewobble_opencl filter
All of the existing filters for video stabilization use an affine model
(or a limited version of it) to represent the movement of the camera. When
used with cameras with a very wide field of view and/or where the camera
shaking is severe, the corrections result in significant geometric
distortion ("wobbling").
Dewobble (https://git.sr.ht/~hedgepigdaniel/dewobble) is a library built
to solve this problem. It requires knowledge of the projection used by
the input camera, and it performs stabilization using a homography
model, which is limited to include only changes in camera orientation.
Additionally, it can perform projection change by specifying a different
projection for the output camera. This is more efficient and results in
less loss of information than using separate filters to perform
stabilization and projection change.
The dewobble_opencl filter is a wrapper for Dewobble. Dewobble supports
input and output in OpenCL buffers containing NV12 frames. Hence, the
filter is named dewobble_opencl and has the same limitations. Currently
all of the options of Dewobble are supported. Of the two types of filter
available in Dewobble (FilterSync and FilterThreaded), FilterThreaded is
used. The API is synchronous, but the transformations are done in a
separate thread. The purpose of this is to isolate the global per thread
OpenCL context used by OpenCV, which Dewobble uses internally. This
prevents dewobble_opencl from interfering with any other usage of OpenCV
from within FFmpeg.
Signed-off-by: Daniel Playfair Cal
---
Changelog|1 +
LICENSE.md |2 +-
configure|4 +
doc/filters.texi | 149
libavfilter/Makefile |1 +
libavfilter/allfilters.c |1 +
libavfilter/version.h|4 +-
libavfilter/vf_dewobble_opencl.c | 1256 ++
8 files changed, 1415 insertions(+), 3 deletions(-)
create mode 100644 libavfilter/vf_dewobble_opencl.c
diff --git a/Changelog b/Changelog
index 1037688682..31f7dbdfe9 100644
--- a/Changelog
+++ b/Changelog
@@ -9,6 +9,7 @@ version :
- Argonaut Games CVG muxer
- Concatf protocol
- afwtdn audio filter
+- Dewobble filter
version 4.4:
diff --git a/LICENSE.md b/LICENSE.md
index 613070e1b6..dfdf010d8e 100644
--- a/LICENSE.md
+++ b/LICENSE.md
@@ -112,7 +112,7 @@ The VMAF, mbedTLS, RK MPI, OpenCORE and VisualOn libraries
are under the Apache
version 3 of those licenses. So to combine these libraries with FFmpeg, the
license version needs to be upgraded by passing `--enable-version3` to
configure.
-The smbclient library is under the GPL v3, to combine it with FFmpeg,
+The dewobble and smbclient libraries are under the GPL v3, to combine them
with FFmpeg,
the options `--enable-gpl` and `--enable-version3` have to be passed to
configure to upgrade FFmpeg to the GPL v3.
diff --git a/configure b/configure
index 82639ce057..aa136b16e5 100755
--- a/configure
+++ b/configure
@@ -230,6 +230,7 @@ External library support:
--enable-libdavs2enable AVS2 decoding via libdavs2 [no]
--enable-libdc1394 enable IIDC-1394 grabbing using libdc1394
and libraw1394 [no]
+ --enable-libdewobble enable video stabilization via libdewobble [no]
--enable-libfdk-aac enable AAC de/encoding via libfdk-aac [no]
--enable-libfliteenable flite (voice synthesis) support via libflite
[no]
--enable-libfontconfig enable libfontconfig, useful for drawtext filter
[no]
@@ -1781,6 +1782,7 @@ EXTERNAL_LIBRARY_VERSION3_LIST="
"
EXTERNAL_LIBRARY_GPLV3_LIST="
+libdewobble
libsmbclient
"
@@ -3582,6 +3584,7 @@ denoise_vaapi_filter_deps="vaapi"
derain_filter_select="dnn"
deshake_filter_select="pixelutils"
deshake_opencl_filter_deps="opencl"
+dewobble_opencl_filter_deps="libdewobble opencl"
dilation_opencl_filter_deps="opencl"
dnn_classify_filter_select="dnn"
dnn_detect_filter_select="dnn"
@@ -6410,6 +6413,7 @@ enabled libcodec2 && require libcodec2
codec2/codec2.h codec2_create -lc
enabled libdav1d && require_pkg_config libdav1d "dav1d >= 0.5.0"
"dav1d/dav1d.h" dav1d_version
enabled libdavs2 && require_pkg_config libdavs2 "davs2 >= 1.6.0"
davs2.h davs2_decoder_open
enabled libdc1394 && require_pkg_config libdc1394 libdc1394-2
dc1394/dc1394.h dc1394_new
+enabled libdewobble && require_pkg_config libdewobble dewobble
dewobble/filter.h dewobble_filter_create_threaded
enabled libdrm&& require_pkg_config libdrm libdrm xf86drm.h
drmGetVersion
enabled libfdk_aac&& { check_pkg_config libfdk_aac fdk-aac
"fdk-aac/aacenc_lib.h" aacEncOpen ||
{ require lib
Re: [FFmpeg-devel] [PATCH 4/4] avfilter/vf_v360: refactor (i)flat_range for fisheye
> Ok, dfov for width != height in (d)fisheye have been fixed. Great, thanks :) > The aspect ratio one breaks handling in case input is in equirectangular > format. That's not obvious to me - can you provide an example filtergraph where it breaks, and explain why it's wrong? > Dunno what to do with rest of patches. The most important one I think is the visibility test for fisheye input - that is the only one that can't be worked around. Going back to your original example: `v360=input=e:output=fisheye:h_fov=180:v_fov=180,v360=input=fisheye:output=e:ih_fov=180:iv_fov=180` Now that you've added the ability to specify fov settings for the equirectangular projection, you can apply my patch and still have the output you expect, for example: v360=input=e:output=fisheye:h_fov=180:v_fov=180,v360=input=fisheye:output=e:ih_fov=180:iv_fov=180:h_fov=180 The use of h_fov on the output prevents anything outside of 180 horizontal field of view from being mapped (because its outside the bounds of the output image). At the same time, its possible to map an entire fisheye image including the corners, for example in my use case of converting fisheye -> flat. On Sun, Apr 11, 2021 at 6:23 PM Paul B Mahol wrote: > > Ok, dfov for width != height in (d)fisheye have been fixed. > > Dunno what to do with rest of patches. > > The aspect ratio one breaks handling in case input is in equirectangular > format. > > On Sun, Apr 11, 2021 at 6:48 AM Daniel Playfair Cal > wrote: >> >> > AFAIK, the h/v/d fov works fine with fisheye in/out. I used synthetic >> > fisheye images from paul bourke site. >> >> > And diagonal fov from w/h either works with both in and out or not at all. >> >> That doesn't seem correct. If an image with an equidistant projection >> is of dimensions WxH and has the focal point at the center, then the >> fields of view are related as follows for some value of f: >> horizontal: f*W >> vertical: f*H >> diagonal: f*sqrt( W^2 + H^2 ) >> >> In the example I gave, v360 chooses horizontal/vertical fields of view >> that are incompatible with the diagonal field of view provided. >> Therefore (assuming 1:1 pixel aspect ratio), since >> sqrt(116.66^2+87.5^2) = 145.83, `input=fisheye:id_fov=145.83` should >> behave the same as `input=fisheye:ih_fov=116.66:iv_fov=87.50` in terms >> of the focal length. It doesn't, so I think this is a bug. >> >> > So you want to not discard pixels out of circle defined by fov? That >> > generally does not make sense to me as that may not gonna have actual >> > pixels that belong to output. >> >> What circle/FoV are you referring to? The FoV depends on which points >> on the image you choose to compare (for example the >> horizontal/vertical/diagonal FoVs are all different). >> Sometimes/usually there is no circle which describes which points in >> the image can be correctly mapped. >> >> If a pixel in an image lies at a point where the projection of that >> image is not defined, then yes it should be discarded. I think this is >> correct in my patches. >> >> > Make sure that your files have correct projection, fisheye in v360 is >> > strict equidistant mapping, and may not be what your input is actually. >> >> Yes, I agree we shouldn't assume anything based on the >> content/appearance of the images from my GoPro. But it's possible to >> make a rectangular image of 1920x1440 pixels with an equidistant >> fisheye projection where the horizontal, vertical, and diagonal FoV is >> 116.66, 87.5, and 145.83. All the pixels in that image are defined in >> the equidistant projection, so I think it should be possible for v360 >> to map all the pixels to an appropriate output. If my image is not a >> real equidistant fisheye projection, or if my FoV measurements are >> wrong, then the chessboard won't have the right shape in the output. >> This is not the problem I am trying to solve here. >> >> On Sun, Apr 11, 2021 at 9:37 AM Paul B Mahol wrote: >> > >> > >> > >> > On Mon, Mar 22, 2021 at 1:35 PM Daniel Playfair Cal >> > wrote: >> >> >> >> > I disagree, if I use 180 hfov and 180 vfov it should not have extra >> >> > areas but only half of previous input. >> >> >> >> Not sure I follow - the ih_fov and vh_fov refer to the input (i.e. the >> >> fisheye image). If you wanted to restrict the FoV of the output, surely >> >> the way to do that would be to implement and use the FoV settings for the >> >> equirectangular projection?. It
Re: [FFmpeg-devel] [PATCH 4/4] avfilter/vf_v360: refactor (i)flat_range for fisheye
> AFAIK, the h/v/d fov works fine with fisheye in/out. I used synthetic fisheye > images from paul bourke site. > And diagonal fov from w/h either works with both in and out or not at all. That doesn't seem correct. If an image with an equidistant projection is of dimensions WxH and has the focal point at the center, then the fields of view are related as follows for some value of f: horizontal: f*W vertical: f*H diagonal: f*sqrt( W^2 + H^2 ) In the example I gave, v360 chooses horizontal/vertical fields of view that are incompatible with the diagonal field of view provided. Therefore (assuming 1:1 pixel aspect ratio), since sqrt(116.66^2+87.5^2) = 145.83, `input=fisheye:id_fov=145.83` should behave the same as `input=fisheye:ih_fov=116.66:iv_fov=87.50` in terms of the focal length. It doesn't, so I think this is a bug. > So you want to not discard pixels out of circle defined by fov? That > generally does not make sense to me as that may not gonna have actual pixels > that belong to output. What circle/FoV are you referring to? The FoV depends on which points on the image you choose to compare (for example the horizontal/vertical/diagonal FoVs are all different). Sometimes/usually there is no circle which describes which points in the image can be correctly mapped. If a pixel in an image lies at a point where the projection of that image is not defined, then yes it should be discarded. I think this is correct in my patches. > Make sure that your files have correct projection, fisheye in v360 is strict > equidistant mapping, and may not be what your input is actually. Yes, I agree we shouldn't assume anything based on the content/appearance of the images from my GoPro. But it's possible to make a rectangular image of 1920x1440 pixels with an equidistant fisheye projection where the horizontal, vertical, and diagonal FoV is 116.66, 87.5, and 145.83. All the pixels in that image are defined in the equidistant projection, so I think it should be possible for v360 to map all the pixels to an appropriate output. If my image is not a real equidistant fisheye projection, or if my FoV measurements are wrong, then the chessboard won't have the right shape in the output. This is not the problem I am trying to solve here. On Sun, Apr 11, 2021 at 9:37 AM Paul B Mahol wrote: > > > > On Mon, Mar 22, 2021 at 1:35 PM Daniel Playfair Cal > wrote: >> >> > I disagree, if I use 180 hfov and 180 vfov it should not have extra areas >> > but only half of previous input. >> >> Not sure I follow - the ih_fov and vh_fov refer to the input (i.e. the >> fisheye image). If you wanted to restrict the FoV of the output, surely the >> way to do that would be to implement and use the FoV settings for the >> equirectangular projection?. It doesn't seem right that the code for the >> input projection is responsible for deciding what appears in the output. My >> understanding was that the FoV settings simply describe the focal length of >> the input or output camera so that points in the images can me mapped >> to/from 3d coordinates. >> >> To give you an idea of what I am trying to fix, here is an example input: >> https://photos.app.goo.gl/o51NfY6aqWn3unPG6 >> This is a 1920x1440 image taken on a GoPro Hero 5 black with the 4:3 Wide >> FoV setting and stabilisation disabled. >> >> The following filtergraph demonstrates the issues: >> 'v360=input=fisheye:ih_fov=116.66:iv_fov=87.50:output=flat:d_fov=145.8' >> 1. the dfov_from_hfov issue is worked around by the use of ih_fov and >> iv_fov instead of id_fov, although you can try with id_fov=145.8 to see that >> problem too > > > AFAIK, the h/v/d fov works fine with fisheye in/out. I used synthetic fisheye > images from paul bourke site. > And diagonal fov from w/h either works with both in and out or not at all. > >> >> 2. by default the output has double the aspect ratio of the input, even >> though the fisheye -> rectilinear transformation doesn't change the aspect >> ratio (assuming the entire input image is included as it is in this example) >> 3. much of the input is not visible in the output even though there is a >> mapping between the chosen projections (changed in the visibility test patch) >> >> 3 in particular I don't think can be solved by changing the settings - the >> input field of view needs to match the FoV of the input camera, otherwise >> the mapping is wrong. But it seems there is no other way to include the >> entire input from a fisheye image. > > > So you want to not discard pixels out of circle defined by fov? That > generally does not make sense to me as that may not gonna have actual pixels > that
Re: [FFmpeg-devel] [PATCH 4/4] avfilter/vf_v360: refactor (i)flat_range for fisheye
Do you agree with my definition or not? And which code are you referring to - the master branch or my patches? I'd like to get these patches to a point where they can be applied, but it's going to be difficult if we can't agree on the goal. On Tue, Mar 23, 2021 at 5:15 PM Paul B Mahol wrote: > > > On Tue, Mar 23, 2021 at 5:00 AM Daniel Playfair Cal < > daniel.playfair@gmail.com> wrote: > >> What exactly is your definition of fisheye? >> > > Take look at source code. I do not see how your definition matches one in > source code. > > >> >> The definition I'm working with is the equidistant fisheye projection as >> described here: https://wiki.panotools.org/Fisheye_Projection, i.e. r = >> f * theta >> >> That mapping works for any theta, and you can have a circular image with >> a field of view of up to 360 degrees before anything is repeated and the >> inverse mapping is ambiguous. Hence my assumption that a rectangular output >> image with a 180 horizontal/vertical field of view should still contain >> areas near the corners where theta > 90 (because the diagonal FoV is > >> 180), and these should still be mapped from such an image to a >> equirectangular projection. >> >> Do you prefer for some reason to limit the fisheye projection to 180 >> degrees on any axis, i.e. have the constraint that theta <= 90? If that's >> the case I could patch xyz_to_fisheye and fisheye_to_xyz so that such areas >> are marked as invisible? That causes your example filtergraph to work as >> before. >> > >> On Tue, Mar 23, 2021 at 3:46 AM Paul B Mahol wrote: >> >>> >>> >>> On Mon, Mar 22, 2021 at 1:35 PM Daniel Playfair Cal < >>> daniel.playfair@gmail.com> wrote: >>> >>>> > I disagree, if I use 180 hfov and 180 vfov it should not have extra >>>> areas but only half of previous input. >>>> >>>> Not sure I follow - the ih_fov and vh_fov refer to the input (i.e. the >>>> fisheye image). If you wanted to restrict the FoV of the output, surely the >>>> way to do that would be to implement and use the FoV settings for the >>>> equirectangular projection?. It doesn't seem right that the code for the >>>> input projection is responsible for deciding what appears in the output. My >>>> understanding was that the FoV settings simply describe the focal length of >>>> the input or output camera so that points in the images can me mapped >>>> to/from 3d coordinates. >>>> >>>> >>> Take any equirectangular input and convert it to fisheye and than back >>> to equirectangular. >>> Or just take pure fisheye input with 180 h & v fov and convert it to >>> equirectangular. There is plenty of such video content on esa website. >>> >>> To give you an idea of what I am trying to fix, here is an example >>>> input: https://photos.app.goo.gl/o51NfY6aqWn3unPG6 >>>> This is a 1920x1440 image taken on a GoPro Hero 5 black with the 4:3 >>>> Wide FoV setting and stabilisation disabled. >>>> >>>> >>> That is flat take of something else. Not real fisheye input. >>> >>> >>>> The following filtergraph demonstrates the issues: >>>> 'v360=input=fisheye:ih_fov=116.66:iv_fov=87.50:output=flat:d_fov=145.8' >>>> 1. the dfov_from_hfov issue is worked around by the use of ih_fov and >>>> iv_fov instead of id_fov, although you can try with id_fov=145.8 to see >>>> that problem too >>>> 2. by default the output has double the aspect ratio of the input, >>>> even though the fisheye -> rectilinear transformation doesn't change the >>>> aspect ratio (assuming the entire input image is included as it is in this >>>> example) >>>> 3. much of the input is not visible in the output even though there is >>>> a mapping between the chosen projections (changed in the visibility test >>>> patch) >>>> >>>> 3 in particular I don't think can be solved by changing the settings - >>>> the input field of view needs to match the FoV of the input camera, >>>> otherwise the mapping is wrong. But it seems there is no other way to >>>> include the entire input from a fisheye image. >>>> >>>> On Mon, Mar 22, 2021 at 5:59 PM Paul B Mahol wrote: >>>> >>>>> >>>>> >>>>> On Mon, Mar 22, 2021 at 5:09 AM Daniel Playfa
Re: [FFmpeg-devel] [PATCH 4/4] avfilter/vf_v360: refactor (i)flat_range for fisheye
What exactly is your definition of fisheye? The definition I'm working with is the equidistant fisheye projection as described here: https://wiki.panotools.org/Fisheye_Projection, i.e. r = f * theta That mapping works for any theta, and you can have a circular image with a field of view of up to 360 degrees before anything is repeated and the inverse mapping is ambiguous. Hence my assumption that a rectangular output image with a 180 horizontal/vertical field of view should still contain areas near the corners where theta > 90 (because the diagonal FoV is > 180), and these should still be mapped from such an image to a equirectangular projection. Do you prefer for some reason to limit the fisheye projection to 180 degrees on any axis, i.e. have the constraint that theta <= 90? If that's the case I could patch xyz_to_fisheye and fisheye_to_xyz so that such areas are marked as invisible? That causes your example filtergraph to work as before. On Tue, Mar 23, 2021 at 3:46 AM Paul B Mahol wrote: > > > On Mon, Mar 22, 2021 at 1:35 PM Daniel Playfair Cal < > daniel.playfair@gmail.com> wrote: > >> > I disagree, if I use 180 hfov and 180 vfov it should not have extra >> areas but only half of previous input. >> >> Not sure I follow - the ih_fov and vh_fov refer to the input (i.e. the >> fisheye image). If you wanted to restrict the FoV of the output, surely the >> way to do that would be to implement and use the FoV settings for the >> equirectangular projection?. It doesn't seem right that the code for the >> input projection is responsible for deciding what appears in the output. My >> understanding was that the FoV settings simply describe the focal length of >> the input or output camera so that points in the images can me mapped >> to/from 3d coordinates. >> >> > Take any equirectangular input and convert it to fisheye and than back to > equirectangular. > Or just take pure fisheye input with 180 h & v fov and convert it to > equirectangular. There is plenty of such video content on esa website. > > To give you an idea of what I am trying to fix, here is an example input: >> https://photos.app.goo.gl/o51NfY6aqWn3unPG6 >> This is a 1920x1440 image taken on a GoPro Hero 5 black with the 4:3 Wide >> FoV setting and stabilisation disabled. >> >> > That is flat take of something else. Not real fisheye input. > > >> The following filtergraph demonstrates the issues: >> 'v360=input=fisheye:ih_fov=116.66:iv_fov=87.50:output=flat:d_fov=145.8' >> 1. the dfov_from_hfov issue is worked around by the use of ih_fov and >> iv_fov instead of id_fov, although you can try with id_fov=145.8 to see >> that problem too >> 2. by default the output has double the aspect ratio of the input, even >> though the fisheye -> rectilinear transformation doesn't change the aspect >> ratio (assuming the entire input image is included as it is in this example) >> 3. much of the input is not visible in the output even though there is a >> mapping between the chosen projections (changed in the visibility test >> patch) >> >> 3 in particular I don't think can be solved by changing the settings - >> the input field of view needs to match the FoV of the input camera, >> otherwise the mapping is wrong. But it seems there is no other way to >> include the entire input from a fisheye image. >> >> On Mon, Mar 22, 2021 at 5:59 PM Paul B Mahol wrote: >> >>> >>> >>> On Mon, Mar 22, 2021 at 5:09 AM Daniel Playfair Cal < >>> daniel.playfair@gmail.com> wrote: >>> >>>> I've tried that filtergraph and a few other similar ones and I'm not >>>> sure what you mean - what exactly is the regression? >>>> >>>> I tried it on this image with an equirectangular projection: >>>> https://wiki.panotools.org/images/0/01/Big_ben_equirectangular.jpg >>>> >>>> The only difference I can see is that there are less unmapped areas in >>>> the output with the patches, because the final mapping from the output >>>> equirectangular image to the intermediate fisheye image no longer fails to >>>> map some areas which are present in the fisheye image. I would describe >>>> this as an improvement? >>>> >>> >>> I disagree, if I use 180 hfov and 180 vfov it should not have extra >>> areas but only half of previous input. >>> >>> >>>> >>>> On Mon, Mar 22, 2021 at 3:30 AM Paul B Mahol wrote: >>>> >>>>> Sorry, but I cannot apply this set as is, It makes at
Re: [FFmpeg-devel] [PATCH 4/4] avfilter/vf_v360: refactor (i)flat_range for fisheye
> I disagree, if I use 180 hfov and 180 vfov it should not have extra areas
but only half of previous input.
Not sure I follow - the ih_fov and vh_fov refer to the input (i.e. the
fisheye image). If you wanted to restrict the FoV of the output, surely the
way to do that would be to implement and use the FoV settings for the
equirectangular projection?. It doesn't seem right that the code for the
input projection is responsible for deciding what appears in the output. My
understanding was that the FoV settings simply describe the focal length of
the input or output camera so that points in the images can me mapped
to/from 3d coordinates.
To give you an idea of what I am trying to fix, here is an example input:
https://photos.app.goo.gl/o51NfY6aqWn3unPG6
This is a 1920x1440 image taken on a GoPro Hero 5 black with the 4:3 Wide
FoV setting and stabilisation disabled.
The following filtergraph demonstrates the issues:
'v360=input=fisheye:ih_fov=116.66:iv_fov=87.50:output=flat:d_fov=145.8'
1. the dfov_from_hfov issue is worked around by the use of ih_fov and
iv_fov instead of id_fov, although you can try with id_fov=145.8 to see
that problem too
2. by default the output has double the aspect ratio of the input, even
though the fisheye -> rectilinear transformation doesn't change the aspect
ratio (assuming the entire input image is included as it is in this example)
3. much of the input is not visible in the output even though there is a
mapping between the chosen projections (changed in the visibility test
patch)
3 in particular I don't think can be solved by changing the settings - the
input field of view needs to match the FoV of the input camera, otherwise
the mapping is wrong. But it seems there is no other way to include the
entire input from a fisheye image.
On Mon, Mar 22, 2021 at 5:59 PM Paul B Mahol wrote:
>
>
> On Mon, Mar 22, 2021 at 5:09 AM Daniel Playfair Cal <
> daniel.playfair@gmail.com> wrote:
>
>> I've tried that filtergraph and a few other similar ones and I'm not sure
>> what you mean - what exactly is the regression?
>>
>> I tried it on this image with an equirectangular projection:
>> https://wiki.panotools.org/images/0/01/Big_ben_equirectangular.jpg
>>
>> The only difference I can see is that there are less unmapped areas in
>> the output with the patches, because the final mapping from the output
>> equirectangular image to the intermediate fisheye image no longer fails to
>> map some areas which are present in the fisheye image. I would describe
>> this as an improvement?
>>
>
> I disagree, if I use 180 hfov and 180 vfov it should not have extra areas
> but only half of previous input.
>
>
>>
>> On Mon, Mar 22, 2021 at 3:30 AM Paul B Mahol wrote:
>>
>>> Sorry, but I cannot apply this set as is, It makes at least one serious
>>> regression.
>>>
>>> For example try this filtergraph:
>>>
>>>
>>> v360=input=e:output=fisheye:h_fov=180:v_fov=180,v360=input=fisheye:output=e:ih_fov=180:iv_fov=180
>>>
>>> On Sun, Mar 21, 2021 at 1:45 PM Daniel Playfair Cal <
>>> daniel.playfair@gmail.com> wrote:
>>>
>>>> This changes the iflat_range and flat_range values for the fisheye
>>>> projection to match their meaning for the flat/rectilinear projection.
>>>> That is, the range is between the two x or two y coordinates of the
>>>> outermost points above/below or left/right of the center, in the
>>>> flat/rectilinear projection.
>>>>
>>>> Signed-off-by: Daniel Playfair Cal
>>>> ---
>>>> libavfilter/vf_v360.c | 19 +--
>>>> 1 file changed, 9 insertions(+), 10 deletions(-)
>>>>
>>>> diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c
>>>> index 68bb2f7b0f..3158451963 100644
>>>> --- a/libavfilter/vf_v360.c
>>>> +++ b/libavfilter/vf_v360.c
>>>> @@ -2807,9 +2807,8 @@ static int prepare_fisheye_out(AVFilterContext
>>>> *ctx)
>>>> {
>>>> V360Context *s = ctx->priv;
>>>>
>>>> -s->flat_range[0] = s->h_fov / 180.f;
>>>> -s->flat_range[1] = s->v_fov / 180.f;
>>>> -
>>>> +s->flat_range[0] = 0.5f * s->h_fov * M_PI / 180.f;
>>>> +s->flat_range[1] = 0.5f * s->v_fov * M_PI / 180.f;
>>>> return 0;
>>>> }
>>>>
>>>> @@ -2827,8 +2826,8 @@ static int fisheye_to_xyz(const V360Context *s,
>>>>int i, int j, int width, int height,
>>>>float *
Re: [FFmpeg-devel] [PATCH 4/4] avfilter/vf_v360: refactor (i)flat_range for fisheye
I've tried that filtergraph and a few other similar ones and I'm not sure
what you mean - what exactly is the regression?
I tried it on this image with an equirectangular projection:
https://wiki.panotools.org/images/0/01/Big_ben_equirectangular.jpg
The only difference I can see is that there are less unmapped areas in the
output with the patches, because the final mapping from the output
equirectangular image to the intermediate fisheye image no longer fails to
map some areas which are present in the fisheye image. I would describe
this as an improvement?
On Mon, Mar 22, 2021 at 3:30 AM Paul B Mahol wrote:
> Sorry, but I cannot apply this set as is, It makes at least one serious
> regression.
>
> For example try this filtergraph:
>
>
> v360=input=e:output=fisheye:h_fov=180:v_fov=180,v360=input=fisheye:output=e:ih_fov=180:iv_fov=180
>
> On Sun, Mar 21, 2021 at 1:45 PM Daniel Playfair Cal <
> daniel.playfair@gmail.com> wrote:
>
>> This changes the iflat_range and flat_range values for the fisheye
>> projection to match their meaning for the flat/rectilinear projection.
>> That is, the range is between the two x or two y coordinates of the
>> outermost points above/below or left/right of the center, in the
>> flat/rectilinear projection.
>>
>> Signed-off-by: Daniel Playfair Cal
>> ---
>> libavfilter/vf_v360.c | 19 +--
>> 1 file changed, 9 insertions(+), 10 deletions(-)
>>
>> diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c
>> index 68bb2f7b0f..3158451963 100644
>> --- a/libavfilter/vf_v360.c
>> +++ b/libavfilter/vf_v360.c
>> @@ -2807,9 +2807,8 @@ static int prepare_fisheye_out(AVFilterContext *ctx)
>> {
>> V360Context *s = ctx->priv;
>>
>> -s->flat_range[0] = s->h_fov / 180.f;
>> -s->flat_range[1] = s->v_fov / 180.f;
>> -
>> +s->flat_range[0] = 0.5f * s->h_fov * M_PI / 180.f;
>> +s->flat_range[1] = 0.5f * s->v_fov * M_PI / 180.f;
>> return 0;
>> }
>>
>> @@ -2827,8 +2826,8 @@ static int fisheye_to_xyz(const V360Context *s,
>>int i, int j, int width, int height,
>>float *vec)
>> {
>> -const float uf = s->flat_range[0] * ((2.f * i) / width - 1.f);
>> -const float vf = s->flat_range[1] * ((2.f * j + 1.f) / height - 1.f);
>> +const float uf = 2.f * s->flat_range[0] / M_PI * ((2.f * i) / width
>> - 1.f);
>> +const float vf = 2.f * s->flat_range[1] / M_PI * ((2.f * j + 1.f) /
>> height - 1.f);
>>
>> const float phi = atan2f(vf, uf);
>> const float theta = M_PI_2 * (1.f - hypotf(uf, vf));
>> @@ -2858,8 +2857,8 @@ static int prepare_fisheye_in(AVFilterContext *ctx)
>> {
>> V360Context *s = ctx->priv;
>>
>> -s->iflat_range[0] = s->ih_fov / 180.f;
>> -s->iflat_range[1] = s->iv_fov / 180.f;
>> +s->iflat_range[0] = 0.5f * s->ih_fov * M_PI / 180.f;
>> +s->iflat_range[1] = 0.5f * s->iv_fov * M_PI / 180.f;
>>
>> return 0;
>> }
>> @@ -2882,10 +2881,10 @@ static int xyz_to_fisheye(const V360Context *s,
>> {
>> const float h = hypotf(vec[0], vec[1]);
>> const float lh = h > 0.f ? h : 1.f;
>> -const float phi = atan2f(h, vec[2]) / M_PI;
>> +const float phi = atan2f(h, vec[2]);
>>
>> -float uf = vec[0] / lh * phi / s->iflat_range[0];
>> -float vf = vec[1] / lh * phi / s->iflat_range[1];
>> +float uf = 0.5f * vec[0] / lh * phi / s->iflat_range[0];
>> +float vf = 0.5f * vec[1] / lh * phi / s->iflat_range[1];
>>
>> const int visible = -0.5f < uf && uf < 0.5f && -0.5f < vf && vf <
>> 0.5f;
>> int ui, vi;
>> --
>> 2.31.0
>>
>> ___
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
>
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] vf_v360: fix (i)flat_range for fisheye projection
Sure, I've done so. I'm pretty new to sending patches by email so sorry if
it's a bit messy!
On Sun, Mar 21, 2021 at 8:14 PM Paul B Mahol wrote:
> Please make log message consistent with other commits to this file.
>
> Also make set of patches to be applied all at once instead each single one.
>
> On Fri, Mar 19, 2021 at 1:08 PM Daniel Playfair Cal <
> daniel.playfair@gmail.com> wrote:
>
>> This changes the iflat_range and flat_range values for the fisheye
>> projection so that they indicate the maximum angle between an observed
>> point and the center (direction the camera is facing). This matches the
>> meaning of those variables in the flat projection.
>>
>> Signed-off-by: Daniel Playfair Cal
>> ---
>>
>> Sorry for the previous identical patch, this version really does remove
>> the use of double literals.
>>
>> libavfilter/vf_v360.c | 19 +--
>> 1 file changed, 9 insertions(+), 10 deletions(-)
>>
>> diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c
>> index 94473cd5b3..7535612d34 100644
>> --- a/libavfilter/vf_v360.c
>> +++ b/libavfilter/vf_v360.c
>> @@ -2807,9 +2807,8 @@ static int prepare_fisheye_out(AVFilterContext *ctx)
>> {
>> V360Context *s = ctx->priv;
>>
>> -s->flat_range[0] = s->h_fov / 180.f;
>> -s->flat_range[1] = s->v_fov / 180.f;
>> -
>> +s->flat_range[0] = 0.5f * s->h_fov * M_PI / 180.f;
>> +s->flat_range[1] = 0.5f * s->v_fov * M_PI / 180.f;
>> return 0;
>> }
>>
>> @@ -2827,8 +2826,8 @@ static int fisheye_to_xyz(const V360Context *s,
>>int i, int j, int width, int height,
>>float *vec)
>> {
>> -const float uf = s->flat_range[0] * ((2.f * i) / width - 1.f);
>> -const float vf = s->flat_range[1] * ((2.f * j + 1.f) / height - 1.f);
>> +const float uf = 2.f * s->flat_range[0] / M_PI * ((2.f * i) / width
>> - 1.f);
>> +const float vf = 2.f * s->flat_range[1] / M_PI * ((2.f * j + 1.f) /
>> height - 1.f);
>>
>> const float phi = atan2f(vf, uf);
>> const float theta = M_PI_2 * (1.f - hypotf(uf, vf));
>> @@ -2858,8 +2857,8 @@ static int prepare_fisheye_in(AVFilterContext *ctx)
>> {
>> V360Context *s = ctx->priv;
>>
>> -s->iflat_range[0] = s->ih_fov / 180.f;
>> -s->iflat_range[1] = s->iv_fov / 180.f;
>> +s->iflat_range[0] = 0.5f * s->ih_fov * M_PI / 180.f;
>> +s->iflat_range[1] = 0.5f * s->iv_fov * M_PI / 180.f;
>>
>> return 0;
>> }
>> @@ -2882,10 +2881,10 @@ static int xyz_to_fisheye(const V360Context *s,
>> {
>> const float h = hypotf(vec[0], vec[1]);
>> const float lh = h > 0.f ? h : 1.f;
>> -const float phi = atan2f(h, vec[2]) / M_PI;
>> +const float phi = atan2f(h, vec[2]);
>>
>> -float uf = vec[0] / lh * phi / s->iflat_range[0];
>> -float vf = vec[1] / lh * phi / s->iflat_range[1];
>> +float uf = 0.5f * vec[0] / lh * phi / s->iflat_range[0];
>> +float vf = 0.5f * vec[1] / lh * phi / s->iflat_range[1];
>>
>> const int visible = hypotf(uf, vf) <= 0.5f;
>> int ui, vi;
>> --
>> 2.31.0
>>
>> ___
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
>
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 2/4] avfilter/vf_v360: stop doubling width for fisheye
This resulted in the default aspect ratio being doubled relative to most input formats like flat/rectilinear. After this patch the default aspect ratio is the same as a rectilinear input. Signed-off-by: Daniel Playfair Cal --- libavfilter/vf_v360.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c index 425f04da94..be886e9bb1 100644 --- a/libavfilter/vf_v360.c +++ b/libavfilter/vf_v360.c @@ -4365,7 +4365,7 @@ static int config_output(AVFilterLink *outlink) case FISHEYE: s->in_transform = xyz_to_fisheye; err = prepare_fisheye_in(ctx); -wf = w * 2; +wf = w; hf = h; break; case PANNINI: @@ -4513,7 +4513,7 @@ static int config_output(AVFilterLink *outlink) case FISHEYE: s->out_transform = fisheye_to_xyz; prepare_out = prepare_fisheye_out; -w = lrintf(wf * 0.5f); +w = lrintf(wf); h = lrintf(hf); break; case PANNINI: -- 2.31.0 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 3/4] avfilter/vf_v360: fix visibility test for fisheye
Previously the visibility test referred to a circle in the input. This changes it so that it refers accurately to the entire area in the input. Signed-off-by: Daniel Playfair Cal --- libavfilter/vf_v360.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c index be886e9bb1..68bb2f7b0f 100644 --- a/libavfilter/vf_v360.c +++ b/libavfilter/vf_v360.c @@ -2887,7 +2887,7 @@ static int xyz_to_fisheye(const V360Context *s, float uf = vec[0] / lh * phi / s->iflat_range[0]; float vf = vec[1] / lh * phi / s->iflat_range[1]; -const int visible = hypotf(uf, vf) <= 0.5f; +const int visible = -0.5f < uf && uf < 0.5f && -0.5f < vf && vf < 0.5f; int ui, vi; uf = (uf + 0.5f) * width; -- 2.31.0 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 4/4] avfilter/vf_v360: refactor (i)flat_range for fisheye
This changes the iflat_range and flat_range values for the fisheye
projection to match their meaning for the flat/rectilinear projection.
That is, the range is between the two x or two y coordinates of the
outermost points above/below or left/right of the center, in the
flat/rectilinear projection.
Signed-off-by: Daniel Playfair Cal
---
libavfilter/vf_v360.c | 19 +--
1 file changed, 9 insertions(+), 10 deletions(-)
diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c
index 68bb2f7b0f..3158451963 100644
--- a/libavfilter/vf_v360.c
+++ b/libavfilter/vf_v360.c
@@ -2807,9 +2807,8 @@ static int prepare_fisheye_out(AVFilterContext *ctx)
{
V360Context *s = ctx->priv;
-s->flat_range[0] = s->h_fov / 180.f;
-s->flat_range[1] = s->v_fov / 180.f;
-
+s->flat_range[0] = 0.5f * s->h_fov * M_PI / 180.f;
+s->flat_range[1] = 0.5f * s->v_fov * M_PI / 180.f;
return 0;
}
@@ -2827,8 +2826,8 @@ static int fisheye_to_xyz(const V360Context *s,
int i, int j, int width, int height,
float *vec)
{
-const float uf = s->flat_range[0] * ((2.f * i) / width - 1.f);
-const float vf = s->flat_range[1] * ((2.f * j + 1.f) / height - 1.f);
+const float uf = 2.f * s->flat_range[0] / M_PI * ((2.f * i) / width -
1.f);
+const float vf = 2.f * s->flat_range[1] / M_PI * ((2.f * j + 1.f) / height
- 1.f);
const float phi = atan2f(vf, uf);
const float theta = M_PI_2 * (1.f - hypotf(uf, vf));
@@ -2858,8 +2857,8 @@ static int prepare_fisheye_in(AVFilterContext *ctx)
{
V360Context *s = ctx->priv;
-s->iflat_range[0] = s->ih_fov / 180.f;
-s->iflat_range[1] = s->iv_fov / 180.f;
+s->iflat_range[0] = 0.5f * s->ih_fov * M_PI / 180.f;
+s->iflat_range[1] = 0.5f * s->iv_fov * M_PI / 180.f;
return 0;
}
@@ -2882,10 +2881,10 @@ static int xyz_to_fisheye(const V360Context *s,
{
const float h = hypotf(vec[0], vec[1]);
const float lh = h > 0.f ? h : 1.f;
-const float phi = atan2f(h, vec[2]) / M_PI;
+const float phi = atan2f(h, vec[2]);
-float uf = vec[0] / lh * phi / s->iflat_range[0];
-float vf = vec[1] / lh * phi / s->iflat_range[1];
+float uf = 0.5f * vec[0] / lh * phi / s->iflat_range[0];
+float vf = 0.5f * vec[1] / lh * phi / s->iflat_range[1];
const int visible = -0.5f < uf && uf < 0.5f && -0.5f < vf && vf < 0.5f;
int ui, vi;
--
2.31.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH 1/4] avfilter/vf_v360: fix fov_from_hfov for fisheye
This was previously incorrect, so that passing only id_fov or d_fov
resulted in incorrect transformation.
Signed-off-by: Daniel Playfair Cal
---
libavfilter/vf_v360.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c
index 94473cd5b3..425f04da94 100644
--- a/libavfilter/vf_v360.c
+++ b/libavfilter/vf_v360.c
@@ -4045,10 +4045,10 @@ static void fov_from_dfov(int format, float d_fov,
float w, float h, float *h_fo
break;
case FISHEYE:
{
-const float d = 0.5f * hypotf(w, h);
+const float d = hypotf(w, h);
-*h_fov = d / w * d_fov;
-*v_fov = d / h * d_fov;
+*h_fov = w / d * d_fov;
+*v_fov = h / d * d_fov;
}
break;
case FLAT:
--
2.31.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] vf_v360: fix (i)flat_range for fisheye projection
This changes the iflat_range and flat_range values for the fisheye
projection so that they indicate the maximum angle between an observed
point and the center (direction the camera is facing). This matches the
meaning of those variables in the flat projection.
Signed-off-by: Daniel Playfair Cal
---
Sorry for the previous identical patch, this version really does remove
the use of double literals.
libavfilter/vf_v360.c | 19 +--
1 file changed, 9 insertions(+), 10 deletions(-)
diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c
index 94473cd5b3..7535612d34 100644
--- a/libavfilter/vf_v360.c
+++ b/libavfilter/vf_v360.c
@@ -2807,9 +2807,8 @@ static int prepare_fisheye_out(AVFilterContext *ctx)
{
V360Context *s = ctx->priv;
-s->flat_range[0] = s->h_fov / 180.f;
-s->flat_range[1] = s->v_fov / 180.f;
-
+s->flat_range[0] = 0.5f * s->h_fov * M_PI / 180.f;
+s->flat_range[1] = 0.5f * s->v_fov * M_PI / 180.f;
return 0;
}
@@ -2827,8 +2826,8 @@ static int fisheye_to_xyz(const V360Context *s,
int i, int j, int width, int height,
float *vec)
{
-const float uf = s->flat_range[0] * ((2.f * i) / width - 1.f);
-const float vf = s->flat_range[1] * ((2.f * j + 1.f) / height - 1.f);
+const float uf = 2.f * s->flat_range[0] / M_PI * ((2.f * i) / width -
1.f);
+const float vf = 2.f * s->flat_range[1] / M_PI * ((2.f * j + 1.f) / height
- 1.f);
const float phi = atan2f(vf, uf);
const float theta = M_PI_2 * (1.f - hypotf(uf, vf));
@@ -2858,8 +2857,8 @@ static int prepare_fisheye_in(AVFilterContext *ctx)
{
V360Context *s = ctx->priv;
-s->iflat_range[0] = s->ih_fov / 180.f;
-s->iflat_range[1] = s->iv_fov / 180.f;
+s->iflat_range[0] = 0.5f * s->ih_fov * M_PI / 180.f;
+s->iflat_range[1] = 0.5f * s->iv_fov * M_PI / 180.f;
return 0;
}
@@ -2882,10 +2881,10 @@ static int xyz_to_fisheye(const V360Context *s,
{
const float h = hypotf(vec[0], vec[1]);
const float lh = h > 0.f ? h : 1.f;
-const float phi = atan2f(h, vec[2]) / M_PI;
+const float phi = atan2f(h, vec[2]);
-float uf = vec[0] / lh * phi / s->iflat_range[0];
-float vf = vec[1] / lh * phi / s->iflat_range[1];
+float uf = 0.5f * vec[0] / lh * phi / s->iflat_range[0];
+float vf = 0.5f * vec[1] / lh * phi / s->iflat_range[1];
const int visible = hypotf(uf, vf) <= 0.5f;
int ui, vi;
--
2.31.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] vf_v360: fix (i)flat_range for fisheye projection
This changes the iflat_range and flat_range values for the fisheye
projection so that they indicate the maximum angle between an observed
point and the center (direction the camera is facing). This matches the
meaning of those variables in the flat projection.
Signed-off-by: Daniel Playfair Cal
---
This version avoids the use of double literals.
libavfilter/vf_v360.c | 19 +--
1 file changed, 9 insertions(+), 10 deletions(-)
diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c
index 94473cd5b3..d4f0836937 100644
--- a/libavfilter/vf_v360.c
+++ b/libavfilter/vf_v360.c
@@ -2807,9 +2807,8 @@ static int prepare_fisheye_out(AVFilterContext *ctx)
{
V360Context *s = ctx->priv;
-s->flat_range[0] = s->h_fov / 180.f;
-s->flat_range[1] = s->v_fov / 180.f;
-
+s->flat_range[0] = 0.5 * s->h_fov * M_PI / 180.f;
+s->flat_range[1] = 0.5 * s->v_fov * M_PI / 180.f;
return 0;
}
@@ -2827,8 +2826,8 @@ static int fisheye_to_xyz(const V360Context *s,
int i, int j, int width, int height,
float *vec)
{
-const float uf = s->flat_range[0] * ((2.f * i) / width - 1.f);
-const float vf = s->flat_range[1] * ((2.f * j + 1.f) / height - 1.f);
+const float uf = 2.f * s->flat_range[0] / M_PI * ((2.f * i) / width -
1.f);
+const float vf = 2.f * s->flat_range[1] / M_PI * ((2.f * j + 1.f) / height
- 1.f);
const float phi = atan2f(vf, uf);
const float theta = M_PI_2 * (1.f - hypotf(uf, vf));
@@ -2858,8 +2857,8 @@ static int prepare_fisheye_in(AVFilterContext *ctx)
{
V360Context *s = ctx->priv;
-s->iflat_range[0] = s->ih_fov / 180.f;
-s->iflat_range[1] = s->iv_fov / 180.f;
+s->iflat_range[0] = 0.5 * s->ih_fov * M_PI / 180.f;
+s->iflat_range[1] = 0.5 * s->iv_fov * M_PI / 180.f;
return 0;
}
@@ -2882,10 +2881,10 @@ static int xyz_to_fisheye(const V360Context *s,
{
const float h = hypotf(vec[0], vec[1]);
const float lh = h > 0.f ? h : 1.f;
-const float phi = atan2f(h, vec[2]) / M_PI;
+const float phi = atan2f(h, vec[2]);
-float uf = vec[0] / lh * phi / s->iflat_range[0];
-float vf = vec[1] / lh * phi / s->iflat_range[1];
+float uf = 0.5f * vec[0] / lh * phi / s->iflat_range[0];
+float vf = 0.5f * vec[1] / lh * phi / s->iflat_range[1];
const int visible = hypotf(uf, vf) <= 0.5f;
int ui, vi;
--
2.31.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH v2] vf_v360: fix visibility test for fisheye projection
Previously the visibility test referred to a circle in the input. This changes it so that it refers accurately to the entire area in the input. Signed-off-by: Daniel Playfair Cal --- This version avoids using double literals libavfilter/vf_v360.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c index 94473cd5b3..3ac2b914eb 100644 --- a/libavfilter/vf_v360.c +++ b/libavfilter/vf_v360.c @@ -2887,7 +2887,7 @@ static int xyz_to_fisheye(const V360Context *s, float uf = vec[0] / lh * phi / s->iflat_range[0]; float vf = vec[1] / lh * phi / s->iflat_range[1]; -const int visible = hypotf(uf, vf) <= 0.5f; +const int visible = -0.5f < uf && uf < 0.5f && -0.5f < vf && vf < 0.5f; int ui, vi; uf = (uf + 0.5f) * width; -- 2.31.0 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] vf_v360: fix fov_from_hfov for fisheye projection
This was previously incorrect, so that passing only id_fov or d_fov
resulted in incorrect transformation.
Signed-off-by: Daniel Playfair Cal
---
libavfilter/vf_v360.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c
index 94473cd5b3..425f04da94 100644
--- a/libavfilter/vf_v360.c
+++ b/libavfilter/vf_v360.c
@@ -4045,10 +4045,10 @@ static void fov_from_dfov(int format, float d_fov,
float w, float h, float *h_fo
break;
case FISHEYE:
{
-const float d = 0.5f * hypotf(w, h);
+const float d = hypotf(w, h);
-*h_fov = d / w * d_fov;
-*v_fov = d / h * d_fov;
+*h_fov = w / d * d_fov;
+*v_fov = h / d * d_fov;
}
break;
case FLAT:
--
2.31.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] vf_v360: fix fov_from_hfov for fisheye projection
This was previously incorrect, so that passing only id_fov or d_fov
resulted in incorrect transformation.
Signed-off-by: Daniel Playfair Cal
---
libavfilter/vf_v360.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c
index 94473cd5b3..425f04da94 100644
--- a/libavfilter/vf_v360.c
+++ b/libavfilter/vf_v360.c
@@ -4045,10 +4045,10 @@ static void fov_from_dfov(int format, float d_fov,
float w, float h, float *h_fo
break;
case FISHEYE:
{
-const float d = 0.5f * hypotf(w, h);
+const float d = hypotf(w, h);
-*h_fov = d / w * d_fov;
-*v_fov = d / h * d_fov;
+*h_fov = w / d * d_fov;
+*v_fov = h / d * d_fov;
}
break;
case FLAT:
--
2.31.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] vf_v360: fix (i)flat_range for fisheye projection
This changes the iflat_range and flat_range values for the fisheye
projection so that they indicate the maximum angle between an observed
point and the center (direction the camera is facing). This matches the
meaning of those variables in the flat projection.
Signed-off-by: Daniel Playfair Cal
---
libavfilter/vf_v360.c | 19 +--
1 file changed, 9 insertions(+), 10 deletions(-)
diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c
index 94473cd5b3..d4f0836937 100644
--- a/libavfilter/vf_v360.c
+++ b/libavfilter/vf_v360.c
@@ -2807,9 +2807,8 @@ static int prepare_fisheye_out(AVFilterContext *ctx)
{
V360Context *s = ctx->priv;
-s->flat_range[0] = s->h_fov / 180.f;
-s->flat_range[1] = s->v_fov / 180.f;
-
+s->flat_range[0] = 0.5 * s->h_fov * M_PI / 180.f;
+s->flat_range[1] = 0.5 * s->v_fov * M_PI / 180.f;
return 0;
}
@@ -2827,8 +2826,8 @@ static int fisheye_to_xyz(const V360Context *s,
int i, int j, int width, int height,
float *vec)
{
-const float uf = s->flat_range[0] * ((2.f * i) / width - 1.f);
-const float vf = s->flat_range[1] * ((2.f * j + 1.f) / height - 1.f);
+const float uf = 2.f * s->flat_range[0] / M_PI * ((2.f * i) / width -
1.f);
+const float vf = 2.f * s->flat_range[1] / M_PI * ((2.f * j + 1.f) / height
- 1.f);
const float phi = atan2f(vf, uf);
const float theta = M_PI_2 * (1.f - hypotf(uf, vf));
@@ -2858,8 +2857,8 @@ static int prepare_fisheye_in(AVFilterContext *ctx)
{
V360Context *s = ctx->priv;
-s->iflat_range[0] = s->ih_fov / 180.f;
-s->iflat_range[1] = s->iv_fov / 180.f;
+s->iflat_range[0] = 0.5 * s->ih_fov * M_PI / 180.f;
+s->iflat_range[1] = 0.5 * s->iv_fov * M_PI / 180.f;
return 0;
}
@@ -2882,10 +2881,10 @@ static int xyz_to_fisheye(const V360Context *s,
{
const float h = hypotf(vec[0], vec[1]);
const float lh = h > 0.f ? h : 1.f;
-const float phi = atan2f(h, vec[2]) / M_PI;
+const float phi = atan2f(h, vec[2]);
-float uf = vec[0] / lh * phi / s->iflat_range[0];
-float vf = vec[1] / lh * phi / s->iflat_range[1];
+float uf = 0.5f * vec[0] / lh * phi / s->iflat_range[0];
+float vf = 0.5f * vec[1] / lh * phi / s->iflat_range[1];
const int visible = hypotf(uf, vf) <= 0.5f;
int ui, vi;
--
2.31.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] vf_v360: stop doubling width for fisheye projection
This resulted in the default aspect ratio being doubled relative to most input formats like flat/rectilinear. After this patch the default aspect ratio is the same as a rectilinear input. Signed-off-by: Daniel Playfair Cal --- libavfilter/vf_v360.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c index 94473cd5b3..3d01df4cf3 100644 --- a/libavfilter/vf_v360.c +++ b/libavfilter/vf_v360.c @@ -4365,7 +4365,7 @@ static int config_output(AVFilterLink *outlink) case FISHEYE: s->in_transform = xyz_to_fisheye; err = prepare_fisheye_in(ctx); -wf = w * 2; +wf = w; hf = h; break; case PANNINI: @@ -4513,7 +4513,7 @@ static int config_output(AVFilterLink *outlink) case FISHEYE: s->out_transform = fisheye_to_xyz; prepare_out = prepare_fisheye_out; -w = lrintf(wf * 0.5f); +w = lrintf(wf); h = lrintf(hf); break; case PANNINI: -- 2.31.0 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] vf_v360: fix visibility test for fisheye projection
Previously the visibility test referred to a circle in the input. This changes it so that it refers accurately to the entire area in the input. Signed-off-by: Daniel Playfair Cal --- libavfilter/vf_v360.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c index 94473cd5b3..602e0e2d72 100644 --- a/libavfilter/vf_v360.c +++ b/libavfilter/vf_v360.c @@ -2887,7 +2887,7 @@ static int xyz_to_fisheye(const V360Context *s, float uf = vec[0] / lh * phi / s->iflat_range[0]; float vf = vec[1] / lh * phi / s->iflat_range[1]; -const int visible = hypotf(uf, vf) <= 0.5f; +const int visible = -0.5 < uf && uf < 0.5 && -0.5 < vf && vf < 0.5; int ui, vi; uf = (uf + 0.5f) * width; -- 2.31.0 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] hwcontext_opencl: include header file in HEADERS
This matches the inclusion of the other hwcontext_.h headers. The skipping of the header depending on build flags is already present. Signed-off-by: Daniel Playfair Cal: --- libavutil/Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/libavutil/Makefile b/libavutil/Makefile index a2dae8e89a..8feb029a3a 100644 --- a/libavutil/Makefile +++ b/libavutil/Makefile @@ -40,6 +40,7 @@ HEADERS = adler32.h \ hwcontext_dxva2.h \ hwcontext_qsv.h \ hwcontext_mediacodec.h\ + hwcontext_opencl.h\ hwcontext_vaapi.h \ hwcontext_videotoolbox.h \ hwcontext_vdpau.h \ -- 2.25.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] Add scale parameter to lensfun filter
Ah cool, thanks! On Fri., 29 Mar. 2019, 10:06 am Paul B Mahol, wrote: > On 3/28/19, Daniel Playfair Cal wrote: > > Hi, > > > > Is anyone able to take a look at this? > > > > I'd appreciate it. It's my first time posting a patch to ffmpeg (or any > > mailing list) so please let me know if I've done anything wrong :) > > > > Daniel > > This have been already applied. > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH] Add scale parameter to lensfun filter
Hi,
Is anyone able to take a look at this?
I'd appreciate it. It's my first time posting a patch to ffmpeg (or any
mailing list) so please let me know if I've done anything wrong :)
Daniel
On Mon, Mar 25, 2019 at 1:07 PM wrote:
> From: Daniel Playfair Cal
>
> The lensfun filter wraps the lensfun library which performs
> transformations on videos to correct for lens distortion. Often this
> results in areas in the input being mapped to areas that fall outside
> the boundaries of the output. The library has a parameter called scale
> which is a scale factor applied to the output video. By decreasing it it
> is possible to regain the areas of the video which would otherwise have
> been lost. There is a special value of 0 which indicates that the
> library should automatically determine a scale factor that results in
> the output frame being filled (i.e. little or no black/unmapped areas).
>
> This patch adds a corresponding scale option to the lensfun filter which
> is passed through to the library. The existing behaviour of using the
> automatic value of 0 is retained as the default behaviour, while other
> values will be passed through to the library.
>
> Signed-off-by: Daniel Playfair Cal
> ---
> doc/filters.texi | 9 +
> libavfilter/vf_lensfun.c | 4 +++-
> 2 files changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 4ffb392a7f..c04fe3a4b6 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -11417,6 +11417,15 @@ focus distance is only used for vignetting and
> only slightly affects the
> vignetting correction process. If unknown, leave it at the default value
> (which
> is 1000).
>
> +@item scale
> +The scale factor which is applied after transformation. After correction
> the
> +video is no longer necessarily rectangular. This parameter controls how
> much of
> +the resulting image is visible. The value 0 means that a value will be
> chosen
> +automatically such that there is little or no unmapped area in the output
> +image. 1.0 means that no additional scaling is done. Lower values may
> result
> +in more of the corrected image being visible, while higher values may
> avoid
> +unmapped areas in the output.
> +
> @item target_geometry
> The target geometry of the output image/video. The following values are
> valid
> options:
> diff --git a/libavfilter/vf_lensfun.c b/libavfilter/vf_lensfun.c
> index 901cd9ff90..3b723dd2d0 100644
> --- a/libavfilter/vf_lensfun.c
> +++ b/libavfilter/vf_lensfun.c
> @@ -79,6 +79,7 @@ typedef struct LensfunContext {
> float focal_length;
> float aperture;
> float focus_distance;
> +float scale;
> int target_geometry;
> int reverse;
> int interpolation_type;
> @@ -108,6 +109,7 @@ static const AVOption lensfun_options[] = {
> { "focal_length", "focal length of video (zoom; constant for the
> duration of the use of this filter)", OFFSET(focal_length),
> AV_OPT_TYPE_FLOAT, {.dbl=18}, 0.0, DBL_MAX, FLAGS },
> { "aperture", "aperture (constant for the duration of the use of this
> filter)", OFFSET(aperture), AV_OPT_TYPE_FLOAT, {.dbl=3.5}, 0.0, DBL_MAX,
> FLAGS },
> { "focus_distance", "focus distance (constant for the duration of the
> use of this filter)", OFFSET(focus_distance), AV_OPT_TYPE_FLOAT,
> {.dbl=1000.0f}, 0.0, DBL_MAX, FLAGS },
> +{ "scale", "scale factor applied after corrections (0.0 means
> automatic scaling)", OFFSET(scale), AV_OPT_TYPE_FLOAT, {.dbl=0.0}, 0.0,
> DBL_MAX, FLAGS },
> { "target_geometry", "target geometry of the lens correction (only
> when geometry correction is enabled)", OFFSET(target_geometry),
> AV_OPT_TYPE_INT, {.i64=LF_RECTILINEAR}, 0, INT_MAX, FLAGS, "lens_geometry"
> },
> { "rectilinear", "rectilinear lens (default)", 0,
> AV_OPT_TYPE_CONST, {.i64=LF_RECTILINEAR}, 0, 0, FLAGS, "lens_geometry" },
> { "fisheye", "fisheye lens", 0, AV_OPT_TYPE_CONST,
> {.i64=LF_FISHEYE}, 0, 0, FLAGS, "lens_geometry" },
> @@ -228,7 +230,7 @@ static int config_props(AVFilterLink *inlink)
> lensfun->focal_length,
> lensfun->aperture,
> lensfun->focus_distance,
> - 0.0,
> + lensfun->scale,
> lensfun->target_geometry,
> lensfun_mode,
> lensfun->reverse);
> --
> 2.21.0
>
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] [PATCH] Add scale parameter to lensfun filter
From: Daniel Playfair Cal
The lensfun filter wraps the lensfun library which performs
transformations on videos to correct for lens distortion. Often this
results in areas in the input being mapped to areas that fall outside
the boundaries of the output. The library has a parameter called scale
which is a scale factor applied to the output video. By decreasing it it
is possible to regain the areas of the video which would otherwise have
been lost. There is a special value of 0 which indicates that the
library should automatically determine a scale factor that results in
the output frame being filled (i.e. little or no black/unmapped areas).
This patch adds a corresponding scale option to the lensfun filter which
is passed through to the library. The existing behaviour of using the
automatic value of 0 is retained as the default behaviour, while other
values will be passed through to the library.
Signed-off-by: Daniel Playfair Cal
---
doc/filters.texi | 9 +
libavfilter/vf_lensfun.c | 4 +++-
2 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index 4ffb392a7f..c04fe3a4b6 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -11417,6 +11417,15 @@ focus distance is only used for vignetting and only
slightly affects the
vignetting correction process. If unknown, leave it at the default value (which
is 1000).
+@item scale
+The scale factor which is applied after transformation. After correction the
+video is no longer necessarily rectangular. This parameter controls how much of
+the resulting image is visible. The value 0 means that a value will be chosen
+automatically such that there is little or no unmapped area in the output
+image. 1.0 means that no additional scaling is done. Lower values may result
+in more of the corrected image being visible, while higher values may avoid
+unmapped areas in the output.
+
@item target_geometry
The target geometry of the output image/video. The following values are valid
options:
diff --git a/libavfilter/vf_lensfun.c b/libavfilter/vf_lensfun.c
index 901cd9ff90..3b723dd2d0 100644
--- a/libavfilter/vf_lensfun.c
+++ b/libavfilter/vf_lensfun.c
@@ -79,6 +79,7 @@ typedef struct LensfunContext {
float focal_length;
float aperture;
float focus_distance;
+float scale;
int target_geometry;
int reverse;
int interpolation_type;
@@ -108,6 +109,7 @@ static const AVOption lensfun_options[] = {
{ "focal_length", "focal length of video (zoom; constant for the duration
of the use of this filter)", OFFSET(focal_length), AV_OPT_TYPE_FLOAT,
{.dbl=18}, 0.0, DBL_MAX, FLAGS },
{ "aperture", "aperture (constant for the duration of the use of this
filter)", OFFSET(aperture), AV_OPT_TYPE_FLOAT, {.dbl=3.5}, 0.0, DBL_MAX, FLAGS
},
{ "focus_distance", "focus distance (constant for the duration of the use
of this filter)", OFFSET(focus_distance), AV_OPT_TYPE_FLOAT, {.dbl=1000.0f},
0.0, DBL_MAX, FLAGS },
+{ "scale", "scale factor applied after corrections (0.0 means automatic
scaling)", OFFSET(scale), AV_OPT_TYPE_FLOAT, {.dbl=0.0}, 0.0, DBL_MAX, FLAGS },
{ "target_geometry", "target geometry of the lens correction (only when
geometry correction is enabled)", OFFSET(target_geometry), AV_OPT_TYPE_INT,
{.i64=LF_RECTILINEAR}, 0, INT_MAX, FLAGS, "lens_geometry" },
{ "rectilinear", "rectilinear lens (default)", 0, AV_OPT_TYPE_CONST,
{.i64=LF_RECTILINEAR}, 0, 0, FLAGS, "lens_geometry" },
{ "fisheye", "fisheye lens", 0, AV_OPT_TYPE_CONST, {.i64=LF_FISHEYE},
0, 0, FLAGS, "lens_geometry" },
@@ -228,7 +230,7 @@ static int config_props(AVFilterLink *inlink)
lensfun->focal_length,
lensfun->aperture,
lensfun->focus_distance,
- 0.0,
+ lensfun->scale,
lensfun->target_geometry,
lensfun_mode,
lensfun->reverse);
--
2.21.0
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
