On Sat, Jul 2, 2022 at 12:35 PM Jan Ekström <jee...@gmail.com> wrote: > > On Sat, Jul 2, 2022 at 7:32 PM Vignesh Venkatasubramanian > <vigneshv-at-google....@ffmpeg.org> wrote: > > > > On Sat, Jul 2, 2022 at 2:35 AM Anton Khirnov <an...@khirnov.net> wrote: > > > > > > Quoting Vignesh Venkatasubramanian (2022-06-30 23:04:34) > > > > Parse the alpha channel for still AVIF images and expose it as a > > > > separate track. This is the simplest way of supporting AVIF alpha > > > > channel in a codec independent manner (similar to how ffmpeg > > > > supports animated AVIF with alpha channel). > > > > > > > > One can use the alphamerge filter to get a transparent image with > > > > a single command. For example: > > > > > > > > ffmpeg -i image_with_alpha.avif -filter_complex alphamerge > > > > image_with_alpha.png > > > > > > > > Signed-off-by: Vignesh Venkatasubramanian <vigne...@google.com> > > > > --- > > > > libavformat/isom.h | 1 + > > > > libavformat/mov.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++ > > > > 2 files changed, 67 insertions(+) > > > > > > I am against this patch, this is a digusting hack. > > > > > > These are not two streams, it is a single stream and should be exposed > > > as such. > > > > > > > Yes, while it is a hack, it is also the simplest way of supporting > > AVIF images with alpha. Since AVIF alpha images require two separate > > decoders, i could not think of any other solution that does not > > involve modifying the underlying av1 codec itself. > > > > If there are any alternative solutions where we can expose a single > > stream that needs two codecs, then i am open to implementing that. > > > > Otherwise, this solution improves the current situation where there is > > no way for the user to extract the alpha channel (albeit in a hacky > > way). > > I have discussed this on IRC,
Thank you for discussing this! > and as far as I can see we have multiple > cases where an additional image has to be decoded by a separate > decoder context to get alpha, namely: > > - vp8/9 (currently handled by AV_PKT_DATA_MATROSKA_BLOCKADDITIONAL) > - HEIF-like formats (such as AVIF), which are codec agnostic (you can > put JPEG, HEVC, AV1 etc into HEIF) > > This means the logic (preferably) should not be codec-specific, and > would allow to implement at least two separate > > My proposal - which Anton didn't really like - was to add a side data > entry AV_PKT_DATA_AUXILIARY_IMAGE , which could have type (ALPHA, > DEPTH etc) as well as the relevant packet(s) - as well as if any of > specifications define codec initialization data, that. Then > libavcodec/decode or so could initialize a secondary decoder context > for alpha images and add the third plane if parameters match what's > required. > My only concern about this solution is that, while the demuxer can easily put the alpha data in the side data, we will then have to modify the decoder (in case of HEIF, we will have to change each underlying codec like av1, hevc, jpeg, etc) to support handling of this side data. I was just hoping it would be nicer if we can have something simpler without having to change every individual libavcodec implementation. ffmpeg already has a way to cleanly initialize codecs and pass data to them, so it would be nicer if we can leverage that somehow instead of making each libavcodec implementation handle this side data type. For example, the proposed solution is what we do for the VP8 alpha demuxing and there is already some fragmentation. The libvpx based vp8 decoder supports alpha channel decoding while the native ffmpeg vp8 decoder does not support it. So, a user has to explicitly pass -c:v libvpx-vp8 to be able to decode files with alpha channel properly. > This would also make remux possible, since containers that do not > support such auxiliary images would ignore the alpha plane, and those > that do would properly pass it through. > The remux is possible in the multi-stream approach/hack as well. But I do see your point. > As for encoding, not fully sure how it should be integrated, if any > encoders actually at this moment do proper alpha coding, or do all API > clients have to separately encode with one context the primary image, > and the alpha with another? I am not sure about other codecs, but in the case of AVIF/AV1, the encoder does not understand/support alpha channels. The only way to do it is to use two separate encoders. > That said, the multi-stream implementation > is already merged on the muxer side, so that can be utilized in the > mean time, even though the auxiliary images are not technically a > separate stream within HEIF-likes. > > Jan > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". -- Vignesh _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
