Re: [FFmpeg-devel] [PATCH v2] Change cube faces order to match Youtube's
On 4/1/18, Hazem Ashmawy wrote: > Please find another implementation for bilinear > https://github.com/HazemSamir/FFmpeg/commit/6a62f6db30bed37323d70eb7d66d6c33fa8f00c3 > > Please let me know your thoughts. > Should be fine if it works. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH v2] Change cube faces order to match Youtube's
Please find another implementation for bilinear https://github.com/HazemSamir/FFmpeg/commit/6a62f6db30bed37323d70eb7d66d6c33fa8f00c3 Please let me know your thoughts. On 3/14/18, Paul B Mahol wrote: > On 3/14/18, Hazem Ashmawy wrote: >> So I'm still looking into using bilinear instead of nearest. Most of >> the online discussions are in the context of computer graphics and >> using things like openGL shaders. >> >> One solution I found and may try to implement is to add padding pixels >> around each face tile in the input frame. This should help in handling >> sampling near cube edges and corners. However, I'm not sure about the >> complexity of this or its effect on the performance. WDYT? > > Either that or doing it hard way, by picking different pixels for each face. > Anyway that (padding) solution doesn't sound that bad, and is much simpler > than > above mentioned alternative. > >> >> Meanwhile this is a hacky/lazy way to use bilinear, but I can barely >> notice any improvements >> over using nearest by my bare eyes, are there any systematic way to >> evaluate the quality? > > Not realy. One could compare with another projection conversion utility, > there are at least two freely available AFAIK. > >> https://github.com/HazemSamir/FFmpeg/commit/fcd82b9782b5f32072d9b0516b275fe3d4f4a163 >> >> On 3/12/18, Paul B Mahol wrote: >>> On 3/12/18, Hazem Ashmawy wrote: On 3/12/18, Paul B Mahol wrote: > On 3/12/18, Hazem Ashmawy wrote: >> Sorry about that! Here is github branch >> >> https://github.com/FFmpeg/FFmpeg/compare/master...HazemSamir:panorama_youtube > > Good, now can you look at how to use bilinear instead of nearest > interpolation > for cubemap to equirectangular conversion. > > Nearest is extremly ugly. > Sure, will look into that. Just to take that into my consideration, why didn't you use it at the first place? Were there any obstacles? >>> >>> It was non-trivial to do. >>> ___ >>> ffmpeg-devel mailing list >>> ffmpeg-devel@ffmpeg.org >>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>> >> ___ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH v2] Change cube faces order to match Youtube's
On 3/14/18, Hazem Ashmawy wrote: > So I'm still looking into using bilinear instead of nearest. Most of > the online discussions are in the context of computer graphics and > using things like openGL shaders. > > One solution I found and may try to implement is to add padding pixels > around each face tile in the input frame. This should help in handling > sampling near cube edges and corners. However, I'm not sure about the > complexity of this or its effect on the performance. WDYT? Either that or doing it hard way, by picking different pixels for each face. Anyway that (padding) solution doesn't sound that bad, and is much simpler than above mentioned alternative. > > Meanwhile this is a hacky/lazy way to use bilinear, but I can barely > notice any improvements > over using nearest by my bare eyes, are there any systematic way to > evaluate the quality? Not realy. One could compare with another projection conversion utility, there are at least two freely available AFAIK. > https://github.com/HazemSamir/FFmpeg/commit/fcd82b9782b5f32072d9b0516b275fe3d4f4a163 > > On 3/12/18, Paul B Mahol wrote: >> On 3/12/18, Hazem Ashmawy wrote: >>> On 3/12/18, Paul B Mahol wrote: On 3/12/18, Hazem Ashmawy wrote: > Sorry about that! Here is github branch > > https://github.com/FFmpeg/FFmpeg/compare/master...HazemSamir:panorama_youtube Good, now can you look at how to use bilinear instead of nearest interpolation for cubemap to equirectangular conversion. Nearest is extremly ugly. >>> Sure, will look into that. Just to take that into my consideration, >>> why didn't you use it at the first place? Were there any obstacles? >> >> It was non-trivial to do. >> ___ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH v2] Change cube faces order to match Youtube's
So I'm still looking into using bilinear instead of nearest. Most of the online discussions are in the context of computer graphics and using things like openGL shaders. One solution I found and may try to implement is to add padding pixels around each face tile in the input frame. This should help in handling sampling near cube edges and corners. However, I'm not sure about the complexity of this or its effect on the performance. WDYT? Meanwhile this is a hacky/lazy way to use bilinear, but I can barely notice any improvements over using nearest by my bare eyes, are there any systematic way to evaluate the quality? https://github.com/HazemSamir/FFmpeg/commit/fcd82b9782b5f32072d9b0516b275fe3d4f4a163 On 3/12/18, Paul B Mahol wrote: > On 3/12/18, Hazem Ashmawy wrote: >> On 3/12/18, Paul B Mahol wrote: >>> On 3/12/18, Hazem Ashmawy wrote: Sorry about that! Here is github branch https://github.com/FFmpeg/FFmpeg/compare/master...HazemSamir:panorama_youtube >>> >>> Good, now can you look at how to use bilinear instead of nearest >>> interpolation >>> for cubemap to equirectangular conversion. >>> >>> Nearest is extremly ugly. >>> >> Sure, will look into that. Just to take that into my consideration, >> why didn't you use it at the first place? Were there any obstacles? > > It was non-trivial to do. > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH v2] Change cube faces order to match Youtube's
On Mon, 12 Mar 2018 17:05:30 +0100 Paul B Mahol wrote: > On 3/12/18, Nicolas George wrote: > > Paul B Mahol (2018-03-12): > >> > What are these avfilter limitations you speak of? > > > >> Sure. > > > > Is "sure" supposed to be an answer to that question? > > Look at thread for stereo3d video filter patch. Right, it can't know the output format until it sees an AVFrame with the side data. But in libavfilter it needs to report the output format before any frame is input. Fixing it requires either adding side data to the input/output filter pad configuration, or dynamic reconfiguration at runtime. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH v2] Change cube faces order to match Youtube's
On 3/12/18, Hazem Ashmawy wrote: > On 3/12/18, Paul B Mahol wrote: >> On 3/12/18, Hazem Ashmawy wrote: >>> Sorry about that! Here is github branch >>> >>> https://github.com/FFmpeg/FFmpeg/compare/master...HazemSamir:panorama_youtube >> >> Good, now can you look at how to use bilinear instead of nearest >> interpolation >> for cubemap to equirectangular conversion. >> >> Nearest is extremly ugly. >> > Sure, will look into that. Just to take that into my consideration, > why didn't you use it at the first place? Were there any obstacles? It was non-trivial to do. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH v2] Change cube faces order to match Youtube's
On 3/12/18, Paul B Mahol wrote: > On 3/12/18, Hazem Ashmawy wrote: >> Sorry about that! Here is github branch >> >> https://github.com/FFmpeg/FFmpeg/compare/master...HazemSamir:panorama_youtube > > Good, now can you look at how to use bilinear instead of nearest > interpolation > for cubemap to equirectangular conversion. > > Nearest is extremly ugly. > Sure, will look into that. Just to take that into my consideration, why didn't you use it at the first place? Were there any obstacles? >> >> On 3/12/18, Paul B Mahol wrote: >>> On 3/12/18, wm4 wrote: On Mon, 12 Mar 2018 11:26:26 +0100 Paul B Mahol wrote: > On 3/12/18, Hazem Ashmawy wrote: > > Specifications can be found here: > > https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md > > > > Signed-off-by: Hazem Ashmawy > > --- > > libavfilter/vf_panorama.c | 32 > > 1 file changed, 16 insertions(+), 16 deletions(-) > > > > Please post whole patch, not just recent changes. > > Or just link to github branch. Also, in comment on this whole issue: this is why it should use the libavutil spherical metadata, because then we have something rigorously defined. Just saying. >>> >>> I already wrote that can not be used, similar to stereo3d frame side >>> data for another filter. >>> ___ >>> ffmpeg-devel mailing list >>> ffmpeg-devel@ffmpeg.org >>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>> >> ___ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH v2] Change cube faces order to match Youtube's
On 3/12/18, Nicolas George wrote: > Paul B Mahol (2018-03-12): >> > What are these avfilter limitations you speak of? > >> Sure. > > Is "sure" supposed to be an answer to that question? Look at thread for stereo3d video filter patch. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH v2] Change cube faces order to match Youtube's
Paul B Mahol (2018-03-12): > > What are these avfilter limitations you speak of? > Sure. Is "sure" supposed to be an answer to that question? Regards, -- Nicolas George signature.asc Description: Digital signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH v2] Change cube faces order to match Youtube's
On 3/12/18, James Almer wrote: > On 3/12/2018 7:48 AM, Paul B Mahol wrote: >> On 3/12/18, wm4 wrote: >>> On Mon, 12 Mar 2018 11:26:26 +0100 >>> Paul B Mahol wrote: >>> On 3/12/18, Hazem Ashmawy wrote: > Specifications can be found here: > https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md > > Signed-off-by: Hazem Ashmawy > --- > libavfilter/vf_panorama.c | 32 > 1 file changed, 16 insertions(+), 16 deletions(-) > Please post whole patch, not just recent changes. Or just link to github branch. >>> >>> Also, in comment on this whole issue: this is why it should use the >>> libavutil spherical metadata, because then we have something rigorously >>> defined. Just saying. >> >> I already wrote that can not be used, similar to stereo3d frame side >> data for another filter. > > What are these avfilter limitations you speak of? > > The API for that matter follows the Google spec to the letter for > Cubemap and Equirectangular projections. If they came up with some new > projection definition not currently in the spec then they should be > asked to make it official in said document, so we can implement it. Case > in point the contents of this "ytmp" box in recent spherical mp4 streams > they seem to be serving. > > In any case, please add a big TODO message to make sure this filter and > the framework are eventually adapted to use the frame side data when > available. Sure. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH v2] Change cube faces order to match Youtube's
On 3/12/2018 7:48 AM, Paul B Mahol wrote: > On 3/12/18, wm4 wrote: >> On Mon, 12 Mar 2018 11:26:26 +0100 >> Paul B Mahol wrote: >> >>> On 3/12/18, Hazem Ashmawy wrote: Specifications can be found here: https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md Signed-off-by: Hazem Ashmawy --- libavfilter/vf_panorama.c | 32 1 file changed, 16 insertions(+), 16 deletions(-) >>> >>> Please post whole patch, not just recent changes. >>> >>> Or just link to github branch. >> >> Also, in comment on this whole issue: this is why it should use the >> libavutil spherical metadata, because then we have something rigorously >> defined. Just saying. > > I already wrote that can not be used, similar to stereo3d frame side > data for another filter. What are these avfilter limitations you speak of? The API for that matter follows the Google spec to the letter for Cubemap and Equirectangular projections. If they came up with some new projection definition not currently in the spec then they should be asked to make it official in said document, so we can implement it. Case in point the contents of this "ytmp" box in recent spherical mp4 streams they seem to be serving. In any case, please add a big TODO message to make sure this filter and the framework are eventually adapted to use the frame side data when available. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH v2] Change cube faces order to match Youtube's
On 3/12/18, Hazem Ashmawy wrote: > Sorry about that! Here is github branch > > https://github.com/FFmpeg/FFmpeg/compare/master...HazemSamir:panorama_youtube Good, now can you look at how to use bilinear instead of nearest interpolation for cubemap to equirectangular conversion. Nearest is extremly ugly. > > On 3/12/18, Paul B Mahol wrote: >> On 3/12/18, wm4 wrote: >>> On Mon, 12 Mar 2018 11:26:26 +0100 >>> Paul B Mahol wrote: >>> On 3/12/18, Hazem Ashmawy wrote: > Specifications can be found here: > https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md > > Signed-off-by: Hazem Ashmawy > --- > libavfilter/vf_panorama.c | 32 > 1 file changed, 16 insertions(+), 16 deletions(-) > Please post whole patch, not just recent changes. Or just link to github branch. >>> >>> Also, in comment on this whole issue: this is why it should use the >>> libavutil spherical metadata, because then we have something rigorously >>> defined. Just saying. >> >> I already wrote that can not be used, similar to stereo3d frame side >> data for another filter. >> ___ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH v2] Change cube faces order to match Youtube's
Sorry about that! Here is github branch https://github.com/FFmpeg/FFmpeg/compare/master...HazemSamir:panorama_youtube On 3/12/18, Paul B Mahol wrote: > On 3/12/18, wm4 wrote: >> On Mon, 12 Mar 2018 11:26:26 +0100 >> Paul B Mahol wrote: >> >>> On 3/12/18, Hazem Ashmawy wrote: >>> > Specifications can be found here: >>> > https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md >>> > >>> > Signed-off-by: Hazem Ashmawy >>> > --- >>> > libavfilter/vf_panorama.c | 32 >>> > 1 file changed, 16 insertions(+), 16 deletions(-) >>> > >>> >>> Please post whole patch, not just recent changes. >>> >>> Or just link to github branch. >> >> Also, in comment on this whole issue: this is why it should use the >> libavutil spherical metadata, because then we have something rigorously >> defined. Just saying. > > I already wrote that can not be used, similar to stereo3d frame side > data for another filter. > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH v2] Change cube faces order to match Youtube's
On 3/12/18, wm4 wrote: > On Mon, 12 Mar 2018 11:26:26 +0100 > Paul B Mahol wrote: > >> On 3/12/18, Hazem Ashmawy wrote: >> > Specifications can be found here: >> > https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md >> > >> > Signed-off-by: Hazem Ashmawy >> > --- >> > libavfilter/vf_panorama.c | 32 >> > 1 file changed, 16 insertions(+), 16 deletions(-) >> > >> >> Please post whole patch, not just recent changes. >> >> Or just link to github branch. > > Also, in comment on this whole issue: this is why it should use the > libavutil spherical metadata, because then we have something rigorously > defined. Just saying. I already wrote that can not be used, similar to stereo3d frame side data for another filter. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH v2] Change cube faces order to match Youtube's
On Mon, 12 Mar 2018 11:26:26 +0100 Paul B Mahol wrote: > On 3/12/18, Hazem Ashmawy wrote: > > Specifications can be found here: > > https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md > > > > Signed-off-by: Hazem Ashmawy > > --- > > libavfilter/vf_panorama.c | 32 > > 1 file changed, 16 insertions(+), 16 deletions(-) > > > > Please post whole patch, not just recent changes. > > Or just link to github branch. Also, in comment on this whole issue: this is why it should use the libavutil spherical metadata, because then we have something rigorously defined. Just saying. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH v2] Change cube faces order to match Youtube's
On 3/12/18, Hazem Ashmawy wrote: > Specifications can be found here: > https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md > > Signed-off-by: Hazem Ashmawy > --- > libavfilter/vf_panorama.c | 32 > 1 file changed, 16 insertions(+), 16 deletions(-) > Please post whole patch, not just recent changes. Or just link to github branch. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel