PR #24927 opened by HansKristian-Work URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24927 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24927.patch
# Summary of changes This adds encoding and decoding support of a video codec I developed last year https://themaister.net/blog/2025/06/16/i-designed-my-own-ridiculously-fast-game-streaming-video-codec-pyrowave/. It is currently shipping in Steam (https://steamcommunity.com/groups/homestream/discussions/0/564794422009744473), and there might be interest in having the codec be available in FFmpeg. Its primary aim is a super low-latency codec for local network game streaming, but it may have some fringe uses as a super fast mezzanine codec. I don't know how this codec stacks up against APV and ProRes, and I don't intent to replace any of them. It is not a low-latency codec in the sense that the image is sliced up to facilitate cable adaptation. Game streaming only operates on full frames. It's low-latency in the sense that the delay from GPU frame is signaled to compute, the bitstream reaches system memory in a fraction of a millisecond. The implementation uses my C pyrowave library, same as Steam: https://github.com/Themaister/pyrowave. Internally the codec uses Vulkan to accelerate encode and decode. The API is designed with the possibility of directly plugging into the Vulkan HW device in FFmpeg which would allow zero-copy encode and decode, but that is left as a TODO here. I'm not familiar enough with the Vulkan codebase here to do such work right now and it would only make sense to do so after further discussion. The basic features of the codec are: - Typical intra-only wavelet coding with ye olde CDF 9/7. - Rethinks the block coding to be as fast as possible for GPU encode and decode. It's mostly an adaptation of my 2014 master thesis. - Extremely fast. Ignoring any overhead for schlepping buffers across the PCI-e bus and memcpy-ing staging buffers, 10000 FPS encode for 1080p is achievable. 5k FPS encode for 4K should be achievable on high-end desktop GPUs. - Exact rate control. Ask for 200 mbit and you get exactly that. The target byte budget will not be exceeded. Usually hits ~32 bytes under the target or something like that. Intended for real-time game streaming where low end-to-end latency and predictable bit-rate behavior is of utmost importance. The implementation does a global RD-optimization with some very basic psychovisual considerations (CSF based) which only works due to the trivial nature of the block coder. - Robust to packet loss. Lost packets translate to 0.0 wavelet coeffs, which just means a spurious blur somewhere in the frame. Error correction details are out of scope here. The closest codec in FFmpeg already that I know of is likely the VC2 low-delay codec. This implementation is order(s) of magnitude faster and has much better quality at typical compression ratios used by this codec, at least in my rough testing. For MKV support I assigned the PWV1 fourcc. I'm not sure if this needs to be done in a more official way. There is also a basic raw muxer and demuxer. Given this is a "one dude cooked up something" kind of effort, it's understandable if this cannot or won't be upstreamed, but at least I tried making the effort. In case it's important, no AI was used here or during development of the codec. Also, this is the first significant contribution I've done to FFmpeg in like 15 years or so. I tried to stumble my way through the build system setup and hopefully got things mostly right. Keeping it WIP for now since I'm mostly looking for discussion if this is worth adopting in FFmpeg. >From 960588ef071d0fec57c55779d31606b8bd48ac40 Mon Sep 17 00:00:00 2001 From: Hans-Kristian Arntzen <[email protected]> Date: Wed, 7 Oct 2026 20:47:58 +0200 Subject: [PATCH 1/3] avcodec: Add definitions for PyroWave codec. Also adds RIFF format PWV1. Signed-off-by: Hans-Kristian Arntzen <[email protected]> --- libavcodec/codec_desc.c | 7 +++++++ libavcodec/codec_id.h | 1 + libavformat/riff.c | 1 + 3 files changed, 9 insertions(+) diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c index c02b48ac01..3e264ee0a0 100644 --- a/libavcodec/codec_desc.c +++ b/libavcodec/codec_desc.c @@ -2001,6 +2001,13 @@ static const AVCodecDescriptor codec_descriptors[] = { .profiles = NULL_IF_CONFIG_SMALL(ff_astc_profiles), .mime_types= MT("image/astc"), }, + { + .id = AV_CODEC_ID_PYROWAVE, + .type = AVMEDIA_TYPE_VIDEO, + .name = "pyrowave", + .long_name = NULL_IF_CONFIG_SMALL("PyroWave"), + .props = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSY, + }, /* various PCM "codecs" */ { diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h index 0b388594b9..06def90355 100644 --- a/libavcodec/codec_id.h +++ b/libavcodec/codec_id.h @@ -325,6 +325,7 @@ enum AVCodecID { AV_CODEC_ID_JPEGXS, AV_CODEC_ID_WEBP_ANIM, AV_CODEC_ID_ASTC, + AV_CODEC_ID_PYROWAVE, /* various PCM "codecs" */ AV_CODEC_ID_FIRST_AUDIO = 0x10000, ///< A dummy id pointing at the start of audio codecs diff --git a/libavformat/riff.c b/libavformat/riff.c index f60ef8aedf..b643067cd4 100644 --- a/libavformat/riff.c +++ b/libavformat/riff.c @@ -513,6 +513,7 @@ const AVCodecTag ff_codec_bmp_tags[] = { { AV_CODEC_ID_LEAD, MKTAG('L', 'E', 'A', 'D') }, { AV_CODEC_ID_EVC, MKTAG('e', 'v', 'c', '1') }, { AV_CODEC_ID_RV60, MKTAG('R', 'V', '6', '0') }, + { AV_CODEC_ID_PYROWAVE, MKTAG('P', 'W', 'V', '1') }, { AV_CODEC_ID_NONE, 0 } }; -- 2.52.0 >From bebad068aeaafa0b73acc191b8f4fdcb5e144fba Mon Sep 17 00:00:00 2001 From: Hans-Kristian Arntzen <[email protected]> Date: Wed, 7 Oct 2026 20:49:17 +0200 Subject: [PATCH 2/3] avformat: Add raw muxer and demuxer for PyroWave. Signed-off-by: Hans-Kristian Arntzen <[email protected]> --- libavformat/Makefile | 2 + libavformat/allformats.c | 2 + libavformat/pyrowave_common.h | 102 ++++++++++++++++++++++ libavformat/pyrowave_dec.c | 159 ++++++++++++++++++++++++++++++++++ libavformat/pyrowave_enc.c | 121 ++++++++++++++++++++++++++ 5 files changed, 386 insertions(+) create mode 100644 libavformat/pyrowave_common.h create mode 100644 libavformat/pyrowave_dec.c create mode 100644 libavformat/pyrowave_enc.c diff --git a/libavformat/Makefile b/libavformat/Makefile index 24066050a5..35748caf52 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -517,6 +517,8 @@ OBJS-$(CONFIG_PMP_DEMUXER) += pmpdec.o OBJS-$(CONFIG_PP_BNK_DEMUXER) += pp_bnk.o OBJS-$(CONFIG_PVA_DEMUXER) += pva.o OBJS-$(CONFIG_PVF_DEMUXER) += pvfdec.o pcm.o +OBJS-$(CONFIG_PYROWAVE_DEMUXER) += pyrowave_dec.o +OBJS-$(CONFIG_PYROWAVE_MUXER) += pyrowave_enc.o OBJS-$(CONFIG_QCP_DEMUXER) += qcp.o OBJS-$(CONFIG_QOA_DEMUXER) += qoadec.o OBJS-$(CONFIG_R3D_DEMUXER) += r3d.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index a79b1e16c0..71de50107e 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -400,6 +400,8 @@ extern const FFInputFormat ff_pp_bnk_demuxer; extern const FFOutputFormat ff_psp_muxer; extern const FFInputFormat ff_pva_demuxer; extern const FFInputFormat ff_pvf_demuxer; +extern const FFInputFormat ff_pyrowave_demuxer; +extern const FFOutputFormat ff_pyrowave_muxer; extern const FFInputFormat ff_qcp_demuxer; extern const FFInputFormat ff_qoa_demuxer; extern const FFInputFormat ff_r3d_demuxer; diff --git a/libavformat/pyrowave_common.h b/libavformat/pyrowave_common.h new file mode 100644 index 0000000000..80f16a1da9 --- /dev/null +++ b/libavformat/pyrowave_common.h @@ -0,0 +1,102 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef PYROWAVE_COMMON_H +#define PYROWAVE_COMMON_H + +#include <stdint.h> +#include <libavutil/pixfmt.h> +#include <vulkan/vulkan.h> +#include <pyrowave.h> + +struct PyroWaveBitstreamSequenceHeader +{ + uint32_t width_minus_1 : 14; + uint32_t height_minus_1 : 14; + uint32_t sequence : 3; + uint32_t extended : 1; + uint32_t total_blocks : 24; + uint32_t code : 2; + uint32_t chroma_resolution : 1; + uint32_t color_primaries : 1; + uint32_t transfer_function : 1; + uint32_t ycbcr_transform : 1; + uint32_t ycbcr_range : 1; + uint32_t chroma_siting : 1; +}; + +struct PWV1Header +{ + uint32_t magic; + struct PyroWaveBitstreamSequenceHeader pyro; + uint32_t frame_rate_num; + uint32_t frame_rate_den; + uint8_t reference_bit_depth; + uint8_t header_version; // Must be 1. + uint8_t padding[2]; // Reserved, must be 0. +}; + +enum { + CHROMA_RESOLUTION_420 = 0, + CHROMA_RESOLUTION_444 = 1 +}; + +enum { + CHROMA_SITING_CENTER = 0, + CHROMA_SITING_LEFT = 1 +}; + +enum { + YCBCR_RANGE_FULL = 0, + YCBCR_RANGE_LIMITED = 1 +}; + +enum { + COLOR_PRIMARIES_SRGB = 0, // VK_COLOR_SPACE_SRGB_NONLINEAR_KHR + COLOR_PRIMARIES_BT2020 = 1 // VK_COLOR_SPACE_HDR10_ST2084 +}; + +enum { + YCBCR_TRANSFORM_BT709 = 0, + YCBCR_TRANSFORM_BT2020 = 1 +}; + +enum { + TRANSFER_FUNCTION_SRGB = 0, // VK_COLOR_SPACE_SRGB_NONLINEAR_KHR + TRANSFER_FUNCTION_PQ = 1 // VK_COLOR_SPACE_HDR10_ST2084 +}; + +struct pyrowave_format_mapping { + int chroma_resolution; + int depth; + enum AVPixelFormat pix_fmt; + pyrowave_cpu_buffer_format pyro_cpu_format; + int subsampled; +}; + +static const struct pyrowave_format_mapping pwv1_format_mapping[] = { + { CHROMA_RESOLUTION_420, 8, AV_PIX_FMT_YUV420P, PYROWAVE_CPU_BUFFER_FORMAT_YUV420P, 1 }, + { CHROMA_RESOLUTION_420, 10, AV_PIX_FMT_YUV420P10, PYROWAVE_CPU_BUFFER_FORMAT_YUV420P10, 1 }, + { CHROMA_RESOLUTION_420, 16, AV_PIX_FMT_YUV420P16, PYROWAVE_CPU_BUFFER_FORMAT_YUV420P16, 1 }, + { CHROMA_RESOLUTION_444, 8, AV_PIX_FMT_YUV444P, PYROWAVE_CPU_BUFFER_FORMAT_YUV444P, 0 }, + { CHROMA_RESOLUTION_444, 10, AV_PIX_FMT_YUV444P10, PYROWAVE_CPU_BUFFER_FORMAT_YUV444P10, 0 }, + { CHROMA_RESOLUTION_444, 16, AV_PIX_FMT_YUV444P16, PYROWAVE_CPU_BUFFER_FORMAT_YUV444P16, 0 }, +}; + +#endif + diff --git a/libavformat/pyrowave_dec.c b/libavformat/pyrowave_dec.c new file mode 100644 index 0000000000..48ec3b64d6 --- /dev/null +++ b/libavformat/pyrowave_dec.c @@ -0,0 +1,159 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/opt.h" +#include "avformat.h" +#include "avio_internal.h" +#include "demux.h" +#include "internal.h" +#include "pyrowave_common.h" + +typedef struct PyroWaveDemuxerContext +{ + int64_t pts; +} PyroWaveDemuxerContext; + +static int sanitze_pwv1_header(const struct PWV1Header *p) +{ + // Only expect to run correctly on little-endian systems. + // Vulkan endianness needs to match host, and libpyrowave + // has not been implemented with big-endian in mind at this time. + return p->magic == MKTAG('P', 'W', 'V', '1') && + p->pyro.sequence == 0 && p->header_version == 1 && + p->pyro.code == 0 && p->pyro.extended == 0 && + p->pyro.total_blocks == 0 && p->padding[0] == 0 && p->padding[1] == 0; +} + +static int pyrowave_probe(const AVProbeData *p) +{ + struct PWV1Header pwv1; + + if (p->buf_size < 4 + sizeof(struct PWV1Header)) { + // Too small to fit magic and header. + return 0; + } + + if (memcmp(p->buf, "PWV1", 4) != 0) { + return 0; + } + + memcpy(&pwv1, p->buf + 4, sizeof(struct PWV1Header)); + + if (!sanitze_pwv1_header(&pwv1)) + return 0; + + return AVPROBE_SCORE_MAX; +} + +static int pyrowave_read_header(AVFormatContext *s) +{ + PyroWaveDemuxerContext *ctx = s->priv_data; + enum AVPixelFormat pix_fmt; + AVIOContext *pb = s->pb; + struct PWV1Header pwv1; + AVStream *st; + + if (avio_read(pb, (void *)&pwv1, sizeof(pwv1)) != sizeof(pwv1)) { + av_log(s, AV_LOG_ERROR, "Failed to read PWV1 header.\n"); + return AVERROR(EINVAL); + } + + if (!sanitze_pwv1_header(&pwv1)) { + av_log(s, AV_LOG_ERROR, "PWV1 header contains invalid values.\n"); + return AVERROR(EINVAL); + } + + pix_fmt = AV_PIX_FMT_NONE; + for (int i = 0; i < FF_ARRAY_ELEMS(pwv1_format_mapping); i++) { + if (pwv1_format_mapping[i].chroma_resolution == pwv1.pyro.chroma_resolution && + pwv1_format_mapping[i].depth == pwv1.reference_bit_depth) { + pix_fmt = pwv1_format_mapping[i].pix_fmt; + break; + } + } + + if (pix_fmt == AV_PIX_FMT_NONE) { + av_log(s, AV_LOG_ERROR, "PWV1 header contains unexpected reference bit depth of %d.\n", + pwv1.reference_bit_depth); + return AVERROR(EINVAL); + } + + st = avformat_new_stream(s, NULL); + if (!st) + return AVERROR(ENOMEM); + + st->codecpar->width = pwv1.pyro.width_minus_1 + 1; + st->codecpar->height = pwv1.pyro.height_minus_1 + 1; + + int raten, rated; + av_reduce(&raten, &rated, pwv1.frame_rate_num, pwv1.frame_rate_den, (1UL << 31) - 1); + avpriv_set_pts_info(st, 64, rated, raten); + st->avg_frame_rate = av_inv_q(st->time_base); + st->codecpar->format = pix_fmt; + st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; + st->codecpar->codec_id = AV_CODEC_ID_PYROWAVE; + st->sample_aspect_ratio = (AVRational){1, 1}; + st->codecpar->chroma_location = + pwv1.pyro.chroma_siting == CHROMA_SITING_CENTER ? AVCHROMA_LOC_CENTER : AVCHROMA_LOC_LEFT; + st->codecpar->color_range = + pwv1.pyro.ycbcr_range == YCBCR_RANGE_FULL ? AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG; + // Leave sRGB vague. Common game captures include both gamma 2.2 or sRGB piecewise, and don't + // want to give too much confidence to a decoder which exact curve is assumed. + st->codecpar->color_trc = + pwv1.pyro.transfer_function == TRANSFER_FUNCTION_PQ ? AVCOL_TRC_SMPTE2084 : AVCOL_TRC_UNSPECIFIED; + st->codecpar->color_primaries = + pwv1.pyro.color_primaries == COLOR_PRIMARIES_BT2020 ? AVCOL_PRI_BT2020 : AVCOL_PRI_BT709; + st->codecpar->color_space = + pwv1.pyro.ycbcr_transform == YCBCR_TRANSFORM_BT2020 ? AVCOL_SPC_BT2020_NCL : AVCOL_SPC_BT709; + + ctx->pts = 0; + + return 0; +} + +static int pyrowave_read_packet(AVFormatContext *s, AVPacket *pkt) +{ + PyroWaveDemuxerContext *ctx = s->priv_data; + AVIOContext *pb = s->pb; + uint32_t packet_size; + int ret; + + packet_size = avio_rl32(pb); + + ret = av_get_packet(s->pb, pkt, packet_size); + if (ret < 0) + return ret; + else if (ret != packet_size) + return pb->eof_reached ? AVERROR_EOF : AVERROR_INVALIDDATA; + + pkt->stream_index = 0; + pkt->pts = ctx->pts++; + pkt->dts = pkt->pts; + pkt->duration = 1; + return 0; +} + +const FFInputFormat ff_pyrowave_demuxer = { + .p.name = "pyrowave", + .p.long_name = NULL_IF_CONFIG_SMALL("pyrowave raw bitstream"), + .p.extensions = "pwv1", + .priv_data_size = sizeof(PyroWaveDemuxerContext), + .read_probe = pyrowave_probe, + .read_header = pyrowave_read_header, + .read_packet = pyrowave_read_packet, +}; diff --git a/libavformat/pyrowave_enc.c b/libavformat/pyrowave_enc.c new file mode 100644 index 0000000000..003b709863 --- /dev/null +++ b/libavformat/pyrowave_enc.c @@ -0,0 +1,121 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "avformat.h" +#include "mux.h" +#include "pyrowave_common.h" +#include "libavutil/pixdesc.h" + +static int pyrowave_init(AVFormatContext *s) +{ + if (s->streams[0]->codecpar->codec_id != AV_CODEC_ID_PYROWAVE) { + av_log(s, AV_LOG_ERROR, "ERROR: Codec not supported.\n"); + return AVERROR_INVALIDDATA; + } + return 0; +} + +static int pyrowave_write_header(AVFormatContext *s) +{ + const AVPixFmtDescriptor *pix_desc; + struct PWV1Header header = {}; + AVIOContext *pb = s->pb; + AVStream *st; + + st = s->streams[0]; + + pix_desc = av_pix_fmt_desc_get(st->codecpar->format); + if (pix_desc->nb_components != 3) { + av_log(s, AV_LOG_ERROR, "Only three component formats are supported for encode.\n"); + return AVERROR(EINVAL); + } + + if (pix_desc->log2_chroma_w != pix_desc->log2_chroma_h) { + av_log(s, AV_LOG_ERROR, "4:2:2 is not supported.\n"); + return AVERROR(EINVAL); + } + + if (pix_desc->log2_chroma_w > 1) { + av_log(s, AV_LOG_ERROR, "Only 4:4:4 and 4:2:0 is supported.\n"); + return AVERROR(EINVAL); + } + + header.magic = MKTAG('P', 'W', 'V', '1'); + header.reference_bit_depth = pix_desc->comp[0].depth; + header.frame_rate_num = st->codecpar->framerate.num; + header.frame_rate_den = st->codecpar->framerate.den; + header.header_version = 1; + header.pyro.chroma_resolution = pix_desc->log2_chroma_w == 1 ? + CHROMA_RESOLUTION_420 : CHROMA_RESOLUTION_444; + header.pyro.width_minus_1 = st->codecpar->width - 1; + header.pyro.height_minus_1 = st->codecpar->height - 1; + + // Default to + header.pyro.color_primaries = st->codecpar->color_primaries == AVCOL_PRI_BT2020 ? + COLOR_PRIMARIES_BT2020 : COLOR_PRIMARIES_SRGB; + header.pyro.transfer_function = st->codecpar->color_trc == AVCOL_TRC_SMPTE2084 ? + TRANSFER_FUNCTION_PQ : TRANSFER_FUNCTION_SRGB; + header.pyro.ycbcr_range = st->codecpar->color_range == AVCOL_RANGE_JPEG ? + YCBCR_RANGE_FULL : YCBCR_RANGE_LIMITED; + header.pyro.ycbcr_transform = st->codecpar->color_space == AVCOL_SPC_BT2020_NCL ? + YCBCR_TRANSFORM_BT2020 : YCBCR_TRANSFORM_BT709; + + // Make some default assumptions if chroma siting is not specified. + if (st->codecpar->color_range == AVCOL_RANGE_JPEG && st->codecpar->chroma_location == AVCHROMA_LOC_UNSPECIFIED) { + header.pyro.chroma_siting = CHROMA_SITING_CENTER; + av_log(s, AV_LOG_WARNING, "Chroma siting is not set, assuming CENTER due to JPEG range.\n"); + } else if (st->codecpar->color_range == AVCOL_RANGE_MPEG && st->codecpar->chroma_location == AVCHROMA_LOC_UNSPECIFIED) { + header.pyro.chroma_siting = CHROMA_SITING_LEFT; + av_log(s, AV_LOG_WARNING, "Chroma siting is not set, guessing LEFT due to MPEG range.\n"); + } else if (st->codecpar->chroma_location == AVCHROMA_LOC_CENTER) { + header.pyro.chroma_siting = CHROMA_SITING_CENTER; + } else if (st->codecpar->chroma_location == AVCHROMA_LOC_LEFT) { + header.pyro.chroma_siting = CHROMA_SITING_LEFT; + } else if (st->codecpar->color_range == AVCOL_RANGE_JPEG) { + av_log(s, AV_LOG_WARNING, "Unsupported chroma siting, assuming CENTER.\n"); + header.pyro.chroma_siting = CHROMA_SITING_CENTER; + } else { + av_log(s, AV_LOG_WARNING, "Unsupported chroma siting, assuming LEFT.\n"); + header.pyro.chroma_siting = CHROMA_SITING_LEFT; + } + + avio_write(pb, (const void *)&header, sizeof(header)); + return 0; +} + +static int pyrowave_write_packet(AVFormatContext *s, AVPacket *pkt) +{ + AVIOContext *pb = s->pb; + avio_wl32(pb, pkt->size); + avio_write(pb, pkt->data, pkt->size); + return 0; +} + +const FFOutputFormat ff_pyrowave_muxer = { + .p.name = "pyrowave", + .p.long_name = NULL_IF_CONFIG_SMALL("pyrowave raw bitstream"), + .p.extensions = "pwv1", + .p.audio_codec = AV_CODEC_ID_NONE, + .p.video_codec = AV_CODEC_ID_PYROWAVE, + .p.subtitle_codec = AV_CODEC_ID_NONE, + .p.flags = AVFMT_NOTIMESTAMPS, + .flags_internal = FF_OFMT_FLAG_MAX_ONE_OF_EACH, + .init = pyrowave_init, + .write_header = pyrowave_write_header, + .write_packet = pyrowave_write_packet, +}; -- 2.52.0 >From d915a65427f649b7d03f7e7c54b9f6dc61c29b97 Mon Sep 17 00:00:00 2001 From: Hans-Kristian Arntzen <[email protected]> Date: Wed, 7 Oct 2026 20:50:15 +0200 Subject: [PATCH 3/3] avcodec: Add libpyrowave encode and decode. This is the simpler path that abstracts away the Vulkan implementation behind the API boundary. This is convenient for simple CPU upload and readback paths where the source image might not be on GPU already. The implementation is effectively fully bound on either CPU side memcpy(), or PCI-e bus transfer queue operations. A sufficiently powerful GPU will twiddle its thumbs at well over 1000 FPS. There is room for improvement here to add direct Vulkan HW device integration which eliminates these bottlenecks. The pyrowave API is designed to allow deeper integration with existing HW devices, similar to the FFmpeg API already. This could be implemented as a follow-up similar to how FFV1 encoder and decoder optionally support a Vulkan HW path if enabled. Signed-off-by: Hans-Kristian Arntzen <[email protected]> --- Changelog | 1 + configure | 10 ++ doc/general_contents.texi | 7 + libavcodec/Makefile | 2 + libavcodec/allcodecs.c | 2 + libavcodec/libpyrowave_dec.c | 294 +++++++++++++++++++++++++++++++++++ libavcodec/libpyrowave_enc.c | 276 ++++++++++++++++++++++++++++++++ 7 files changed, 592 insertions(+) create mode 100644 libavcodec/libpyrowave_dec.c create mode 100644 libavcodec/libpyrowave_enc.c diff --git a/Changelog b/Changelog index 876497f2e1..8d0bcd622e 100644 --- a/Changelog +++ b/Changelog @@ -16,6 +16,7 @@ version <next>: - DSD (dsd_msbf) encoder - Rhetorex ADPCM decoder - ADPCM IMA Citrix decoder +- PyroWave Vulkan encoder and decoder via libpyrowave version 9.0: diff --git a/configure b/configure index 08ff6892fb..420ea7c532 100755 --- a/configure +++ b/configure @@ -266,6 +266,7 @@ External library support: --enable-libopus enable Opus de/encoding via libopus [no] --enable-libplacebo enable libplacebo library [no] --enable-libpulse enable Pulseaudio input via libpulse [no] + --enable-libpyrowave enable pyrowave library [no] --enable-libqrencode enable QR encode generation via libqrencode [no] --enable-libquirc enable QR decoding via libquirc [no] --enable-librabbitmq enable RabbitMQ library [no] @@ -2100,6 +2101,7 @@ EXTERNAL_LIBRARY_LIST=" libopus libplacebo libpulse + libpyrowave libqrencode libquirc librabbitmq @@ -3253,6 +3255,10 @@ prores_aw_encoder_select="fdctdsp" prores_ks_encoder_select="fdctdsp" prores_ks_vulkan_encoder_select="vulkan spirv_compiler vulkan_seg_gather" prores_raw_decoder_select="blockdsp idctdsp" +libpyrowave_encoder_deps="libpyrowave" +libpyrowave_decoder_deps="libpyrowave" +libpyrowave_encoder_select="vulkan" +libpyrowave_decoder_select="vulkan" qcelp_decoder_select="lsp" qdm2_decoder_select="mpegaudiodsp" ra_144_decoder_select="audiodsp" @@ -3838,6 +3844,8 @@ libopenmpt_demuxer_deps="libopenmpt" libopus_decoder_deps="libopus" libopus_encoder_deps="libopus" libopus_encoder_select="audio_frame_queue" +libpyrowave_encoder_deps="libpyrowave vulkan" +libpyrowave_decoder_deps="libpyrowave vulkan" librav1e_encoder_deps="librav1e" librsvg_decoder_deps="librsvg" libshine_encoder_deps="libshine" @@ -7801,6 +7809,8 @@ enabled amf && check_cpp_condition amf "AMF/core/Version.h" \ "(AMF_VERSION_MAJOR << 48 | AMF_VERSION_MINOR << 32 | AMF_VERSION_RELEASE << 16 | AMF_VERSION_BUILD_NUM) >= 0x1000500020000" +enabled libpyrowave && enabled vulkan && require_pkg_config libpyrowave pyrowave-shared "pyrowave.h" pyrowave_create_device + # Funny iconv installations are not unusual, so check it after all flags have been set if enabled libc_iconv; then check_func_headers iconv.h iconv diff --git a/doc/general_contents.texi b/doc/general_contents.texi index 597314ac04..05a4772fe8 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -482,6 +482,13 @@ Go to @url{http://sourceforge.net/projects/zapping/} and follow the instructions installing the library. Then pass @code{--enable-libzvbi} to configure to enable it. +@section PyroWave + +FFmpeg can use the pyrowave-shared library for encoding and decoding of PyroWave. +The library requires a GPU capable of supporting Vulkan. +Got to @url{http://github.com/Themaister/Pyrowave} and follow the instructions for +installing the library. Then pass @code{--enable-libpyrowave} to configure to enable it. + @chapter Supported File Formats, Codecs or Features You can use the @code{-formats} and @code{-codecs} options to have an exhaustive list. diff --git a/libavcodec/Makefile b/libavcodec/Makefile index ece43a46d6..ff93a1d9d8 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1217,6 +1217,8 @@ OBJS-$(CONFIG_LIBOPUS_DECODER) += libopusdec.o libopus.o \ vorbis_data.o OBJS-$(CONFIG_LIBOPUS_ENCODER) += libopusenc.o libopus.o \ vorbis_data.o +OBJS-$(CONFIG_LIBPYROWAVE_DECODER) += libpyrowave_dec.o +OBJS-$(CONFIG_LIBPYROWAVE_ENCODER) += libpyrowave_enc.o OBJS-$(CONFIG_LIBRAV1E_ENCODER) += librav1e.o OBJS-$(CONFIG_LIBRSVG_DECODER) += librsvgdec.o OBJS-$(CONFIG_LIBSHINE_ENCODER) += libshine.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 9c3d62f058..d899cd2d5e 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -805,6 +805,8 @@ extern const FFCodec ff_libopencore_amrwb_decoder; extern const FFCodec ff_libopenjpeg_encoder; extern const FFCodec ff_libopus_encoder; extern const FFCodec ff_libopus_decoder; +extern const FFCodec ff_libpyrowave_decoder; +extern const FFCodec ff_libpyrowave_encoder; extern const FFCodec ff_librav1e_encoder; extern const FFCodec ff_librsvg_decoder; extern const FFCodec ff_libshine_encoder; diff --git a/libavcodec/libpyrowave_dec.c b/libavcodec/libpyrowave_dec.c new file mode 100644 index 0000000000..a58753f991 --- /dev/null +++ b/libavcodec/libpyrowave_dec.c @@ -0,0 +1,294 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "avcodec.h" +#include "codec_internal.h" +#include <vulkan/vulkan.h> +#include <pyrowave.h> +#include "libavutil/pixdesc.h" +#include "libavformat/pyrowave_common.h" +#include "libavutil/opt.h" + +typedef struct PyroWaveDecodeContext +{ + AVClass *class; + pyrowave_device device; + pyrowave_decoder decoder; + const struct pyrowave_format_mapping *format; + int doublebuffer; + int flushing; + int64_t frame_count; + int64_t pending_pts; +} PyroWaveDecodeContext; + +static void parse_extradata(AVCodecContext *avctx) +{ + const struct PWV1Header *header = (const void *)avctx->extradata; + if (avctx->extradata_size < sizeof(struct PWV1Header)) + return; + + if (header->magic != MKTAG('P', 'W', 'V', '1')) + return; + + for (int i = 0; i < FF_ARRAY_ELEMS(pwv1_format_mapping); i++) { + if (pwv1_format_mapping[i].chroma_resolution == header->pyro.chroma_resolution && + pwv1_format_mapping[i].depth == header->reference_bit_depth) { + avctx->pix_fmt = pwv1_format_mapping[i].pix_fmt; + break; + } + } + + avctx->chroma_sample_location = + header->pyro.chroma_siting == CHROMA_SITING_CENTER ? AVCHROMA_LOC_CENTER : AVCHROMA_LOC_LEFT; + avctx->color_range = + header->pyro.ycbcr_range == YCBCR_RANGE_FULL ? AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG; + // Leave sRGB vague. Common game captures include both gamma 2.2 or sRGB piecewise, and don't + // want to give too much confidence to a decoder which exact curve is assumed. + avctx->color_trc = + header->pyro.transfer_function == TRANSFER_FUNCTION_PQ ? AVCOL_TRC_SMPTE2084 : AVCOL_TRC_UNSPECIFIED; + avctx->color_primaries = + header->pyro.color_primaries == COLOR_PRIMARIES_BT2020 ? AVCOL_PRI_BT2020 : AVCOL_PRI_BT709; + avctx->colorspace = + header->pyro.ycbcr_transform == YCBCR_TRANSFORM_BT2020 ? AVCOL_SPC_BT2020_NCL : AVCOL_SPC_BT709; +} + +static void pwv_flush_decoder(AVCodecContext *avctx) +{ + PyroWaveDecodeContext *context = avctx->priv_data; + // Resets the decoder to initial state. For seeking this is important since the bitstream seq counter + // may jump unexpectedly. + pyrowave_decoder_clear(context->decoder); +} + +static av_cold int pwv_init_decoder(AVCodecContext *avctx) +{ + PyroWaveDecodeContext *context = avctx->priv_data; + pyrowave_decoder_create_info create_info = {}; + const AVPixFmtDescriptor *pix_desc; + pyrowave_result result; + + parse_extradata(avctx); + + if (avctx->pix_fmt == AV_PIX_FMT_NONE) { + // TODO: We can defer to decoder time and pick yuv420p or yuv444p based on bitstream chroma, but we won't + // know if we should actually be doing 10 or 16-bit decode, or which video usability configuration to use, + // so it's kinda useless on its own. + av_log(avctx, AV_LOG_ERROR, "Cannot determine correct CPU decode format. Missing the global PWV1Header. " + "Make sure to request global headers (e.g. MKV) or use the raw muxer.\n"); + return AVERROR(EINVAL); + } + + pix_desc = av_pix_fmt_desc_get(avctx->pix_fmt); + if (!pix_desc || pix_desc->nb_components != 3) { + av_log(avctx, AV_LOG_ERROR, "Only three component formats are supported for decode.\n"); + return AVERROR(EINVAL); + } + + if (pix_desc->log2_chroma_w != pix_desc->log2_chroma_h) { + av_log(avctx, AV_LOG_ERROR, "4:2:2 is not supported.\n"); + return AVERROR(EINVAL); + } + + if (pix_desc->log2_chroma_w > 1) { + av_log(avctx, AV_LOG_ERROR, "Only 4:4:4 and 4:2:0 is supported.\n"); + return AVERROR(EINVAL); + } + + context->format = NULL; + + while (context->format == NULL) { + for (int i = 0; i < FF_ARRAY_ELEMS(pwv1_format_mapping); i++) { + if (pwv1_format_mapping[i].pix_fmt == avctx->pix_fmt) { + context->format = &pwv1_format_mapping[i]; + break; + } + } + + if (!context->format) { + av_log(avctx, AV_LOG_WARNING, "Unrecognized format %d for decode, falling back.\n", avctx->pix_fmt); + avctx->pix_fmt = pix_desc->log2_chroma_w == 1 ? AV_PIX_FMT_YUV420P : AV_PIX_FMT_YUV444P; + } + } + + result = pyrowave_create_default_device(&context->device); + if (result == PYROWAVE_ERROR_NO_VULKAN) { + av_log(avctx, AV_LOG_ERROR, "No available Vulkan implementation for PyroWave.\n"); + return AVERROR_DECODER_NOT_FOUND; + } + + if (result != PYROWAVE_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to create Vulkan device.\n"); + return AVERROR_DECODER_NOT_FOUND; + } + + create_info.device = context->device; + create_info.width = avctx->width; + create_info.height = avctx->height; + create_info.chroma = context->format->subsampled ? PYROWAVE_CHROMA_SUBSAMPLING_420 : PYROWAVE_CHROMA_SUBSAMPLING_444; + + result = pyrowave_decoder_create(&create_info, &context->decoder); + if (result != PYROWAVE_SUCCESS) { + pyrowave_device_destroy(context->device); + context->device = NULL; + return AVERROR_DECODER_NOT_FOUND; + } + + return 0; +} + +static av_cold int pwv_close_decoder(AVCodecContext *avctx) +{ + PyroWaveDecodeContext *context = avctx->priv_data; + + if (context->decoder) { + pyrowave_decoder_destroy(context->decoder); + context->decoder = NULL; + } + + if (context->device) { + pyrowave_device_destroy(context->device); + context->device = NULL; + } + + return 0; +} + +static int pwv_decode(AVCodecContext *avctx, AVFrame *frame, + int *got_frame, AVPacket *avpkt) +{ + PyroWaveDecodeContext *context = avctx->priv_data; + pyrowave_result result = PYROWAVE_SUCCESS; + pyrowave_cpu_buffer cpu_buffers = {}; + + *got_frame = 0; + + if (!context->doublebuffer && !avpkt) + return AVERROR(EINVAL); + + if (context->flushing == 2) + return 0; + + if (avpkt && avpkt->size) { + result = pyrowave_decoder_push_packet(context->decoder, avpkt->data, avpkt->size); + if (result != PYROWAVE_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to push packet. Corrupt packet?\n"); + return AVERROR(EINVAL); + } + + // Partial packets. + if (!pyrowave_decoder_decode_is_ready(context->decoder, false)) + return 0; + } else if (!context->flushing) { + context->flushing = 1; + } + + // TODO: Is this really the correct way to do this? + av_frame_unref(frame); + frame->width = avctx->width; + frame->height = avctx->height; + frame->format = avctx->pix_fmt; + if (av_frame_get_buffer(frame, 0) < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to allocate frame buffer.\n"); + return AVERROR(ENOMEM); + } + + cpu_buffers.width = avctx->width; + cpu_buffers.height = avctx->height; + cpu_buffers.format = context->format->pyro_cpu_format; + + for (int i = 0; i < 3; i++) { + if (frame->linesize[i] < 0) { + av_log(avctx, AV_LOG_ERROR, "Negative linesize not currently supported.\n"); + return AVERROR(EINVAL); + } + cpu_buffers.data[i] = frame->data[i]; + cpu_buffers.row_stride_in_bytes[i] = frame->linesize[i]; + + // Assume it's safe to write in the padding area. + cpu_buffers.plane_size_in_bytes[i] = frame->linesize[i] * avctx->height; + if (i && context->format->subsampled) + cpu_buffers.plane_size_in_bytes[i] /= 2; + } + + if (!context->doublebuffer) { + result = pyrowave_decoder_decode_cpu_buffer_synchronous(context->decoder, &cpu_buffers); + *got_frame = result == PYROWAVE_SUCCESS; + frame->pts = avpkt->pts; + } else { + if (avpkt && avpkt->size) { + result = pyrowave_decoder_decode_cpu_buffer_async(context->decoder, &cpu_buffers, + context->frame_count % 2); + } + + if (result == PYROWAVE_SUCCESS && context->frame_count) { + // A more extreme API would import host pointers from AVFrames, but that's getting rather silly. + // Probably better to leave to a Vulkan HW device integration path. + result = pyrowave_decoder_decode_cpu_buffer_complete(context->decoder, &cpu_buffers, + (context->frame_count - 1) % 2); + *got_frame = result == PYROWAVE_SUCCESS; + frame->pts = context->pending_pts; + } else { + // There's probably something smarter we can do here on first frame. + // We just need the strides and plane sizes to match to kick off the async decode, but that's it. + av_frame_unref(frame); + } + + if (avpkt) + context->pending_pts = avpkt->pts; + } + + if (result != PYROWAVE_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to decode frame.\n"); + return AVERROR(EINVAL); + } + + if (context->flushing) + context->flushing++; + + context->frame_count++; + return 0; +} + +#define OFFSET(x) offsetof(PyroWaveDecodeContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM +static const AVOption options[] = { + { "doublebuffer", "Double buffer download with decode", OFFSET(doublebuffer), AV_OPT_TYPE_BOOL, { .i64 = -1 }, -1, 1, VD }, + { NULL }, +}; + +static const AVClass libpyrowave_class = { + .class_name = "libpyrowave decoder", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFCodec ff_libpyrowave_decoder = { + .p.name = "libpyrowave", + CODEC_LONG_NAME("PyroWave"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_PYROWAVE, + .p.capabilities = AV_CODEC_CAP_DELAY, + .priv_data_size = sizeof(PyroWaveDecodeContext), + .init = pwv_init_decoder, + .close = pwv_close_decoder, + .flush = pwv_flush_decoder, + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, + FF_CODEC_DECODE_CB(pwv_decode), + .p.priv_class = &libpyrowave_class, +}; diff --git a/libavcodec/libpyrowave_enc.c b/libavcodec/libpyrowave_enc.c new file mode 100644 index 0000000000..b7eb487a39 --- /dev/null +++ b/libavcodec/libpyrowave_enc.c @@ -0,0 +1,276 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "avcodec.h" +#include "codec_internal.h" +#include "encode.h" +#include <vulkan/vulkan.h> +#include <pyrowave.h> +#include "libavutil/opt.h" +#include "libavutil/mem.h" +#include "libavformat/pyrowave_common.h" + +typedef struct PyroWaveEncodeContext +{ + AVClass *class; + pyrowave_device device; + pyrowave_encoder encoder; + int64_t target_payload_size; + const struct pyrowave_format_mapping *format; + int doublebuffer; + int flushing; + int64_t frame_count; + int64_t pending_pts; +} PyroWaveEncodeContext; + +static void set_extradata(AVCodecContext *avctx, const struct pyrowave_format_mapping *format) +{ + struct PWV1Header *header = av_mallocz(sizeof(*header)); + avctx->extradata = (void *)header; + avctx->extradata_size = sizeof(*header); + + // Frame rate is ignored (0). Container decides. + header->magic = MKTAG('P', 'W', 'V', '1'); + header->reference_bit_depth = format->depth; + header->header_version = 1; + header->pyro.chroma_resolution = format->subsampled ? CHROMA_RESOLUTION_420 : CHROMA_RESOLUTION_444; + header->pyro.width_minus_1 = avctx->width - 1; + header->pyro.height_minus_1 = avctx->height - 1; + + header->pyro.color_primaries = avctx->color_primaries == AVCOL_PRI_BT2020 ? + COLOR_PRIMARIES_BT2020 : COLOR_PRIMARIES_SRGB; + header->pyro.transfer_function = avctx->color_trc == AVCOL_TRC_SMPTE2084 ? + TRANSFER_FUNCTION_PQ : TRANSFER_FUNCTION_SRGB; + header->pyro.ycbcr_range = avctx->color_range == AVCOL_RANGE_JPEG ? + YCBCR_RANGE_FULL : YCBCR_RANGE_LIMITED; + header->pyro.ycbcr_transform = avctx->colorspace == AVCOL_SPC_BT2020_NCL ? + YCBCR_TRANSFORM_BT2020 : YCBCR_TRANSFORM_BT709; + + // Make some default assumptions if chroma siting is not specified. + if (avctx->color_range == AVCOL_RANGE_JPEG && avctx->chroma_sample_location == AVCHROMA_LOC_UNSPECIFIED) { + header->pyro.chroma_siting = CHROMA_SITING_CENTER; + av_log(avctx, AV_LOG_WARNING, "Chroma siting is not set, assuming CENTER due to JPEG range.\n"); + } else if (avctx->color_range == AVCOL_RANGE_MPEG && avctx->chroma_sample_location == AVCHROMA_LOC_UNSPECIFIED) { + header->pyro.chroma_siting = CHROMA_SITING_LEFT; + av_log(avctx, AV_LOG_WARNING, "Chroma siting is not set, guessing LEFT due to MPEG range.\n"); + } else if (avctx->chroma_sample_location == AVCHROMA_LOC_CENTER) { + header->pyro.chroma_siting = CHROMA_SITING_CENTER; + } else if (avctx->chroma_sample_location == AVCHROMA_LOC_LEFT) { + header->pyro.chroma_siting = CHROMA_SITING_LEFT; + } else if (avctx->color_range == AVCOL_RANGE_JPEG) { + av_log(avctx, AV_LOG_WARNING, "Unsupported chroma siting, assuming CENTER.\n"); + header->pyro.chroma_siting = CHROMA_SITING_CENTER; + } else { + av_log(avctx, AV_LOG_WARNING, "Unsupported chroma siting, assuming LEFT.\n"); + header->pyro.chroma_siting = CHROMA_SITING_LEFT; + } +} + +static av_cold int pwv_encode_init(AVCodecContext *avctx) +{ + PyroWaveEncodeContext *context = avctx->priv_data; + pyrowave_encoder_create_info create_info = {}; + pyrowave_result result; + + result = pyrowave_create_default_device(&context->device); + + if (result == PYROWAVE_ERROR_NO_VULKAN) { + av_log(avctx, AV_LOG_ERROR, "No available Vulkan implementation for PyroWave.\n"); + return AVERROR_ENCODER_NOT_FOUND; + } + + if (result != PYROWAVE_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to create Vulkan device.\n"); + return AVERROR_ENCODER_NOT_FOUND; + } + + context->format = NULL; + for (int i = 0; i < FF_ARRAY_ELEMS(pwv1_format_mapping); i++) { + if (pwv1_format_mapping[i].pix_fmt == avctx->pix_fmt) { + context->format = &pwv1_format_mapping[i]; + break; + } + } + + if (!context->format) { + // Shouldn't really happen, but be safe? + av_log(avctx, AV_LOG_ERROR, "Unexpected format %d\n", avctx->pix_fmt); + return AVERROR(EINVAL); + } + + // For e.g. MKV where we want to store the PWV1Header in global data. + if (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER) + set_extradata(avctx, context->format); + + create_info.width = avctx->width; + create_info.height = avctx->height; + create_info.chroma = context->format->subsampled ? PYROWAVE_CHROMA_SUBSAMPLING_420 : PYROWAVE_CHROMA_SUBSAMPLING_444; + create_info.device = context->device; + + result = pyrowave_encoder_create(&create_info, &context->encoder); + if (result != PYROWAVE_SUCCESS) { + pyrowave_device_destroy(context->device); + context->device = NULL; + return AVERROR_ENCODER_NOT_FOUND; + } + + context->target_payload_size = av_rescale_rnd(avctx->bit_rate, + avctx->framerate.den, avctx->framerate.num * 8, AV_ROUND_UP); + context->target_payload_size -= context->target_payload_size & 3; + + return 0; +} + +static int pwv_encode_close(AVCodecContext *avctx) +{ + PyroWaveEncodeContext *context = avctx->priv_data; + + if (context->encoder) { + pyrowave_encoder_destroy(context->encoder); + context->encoder = NULL; + } + + if (context->device) { + pyrowave_device_destroy(context->device); + context->device = NULL; + } + + return 0; +} + +static int pwv_encode(AVCodecContext *avctx, AVPacket *pkt, + const AVFrame *frame, int *got_packet) +{ + PyroWaveEncodeContext *context = avctx->priv_data; + pyrowave_result result = PYROWAVE_SUCCESS; + pyrowave_rate_control rate_control = {}; + pyrowave_cpu_buffer cpu_buffers = {}; + pyrowave_packet packet; + size_t num_packets; + int ret; + + *got_packet = 0; + + if (!frame && !context->doublebuffer) + return AVERROR(EINVAL); + + if (context->flushing == 2) { + // Only one frame of buffering. + return 0; + } + + if (!frame && context->doublebuffer && context->flushing == 0) + context->flushing = 1; + + result = PYROWAVE_SUCCESS; + + if (!context->flushing) { + cpu_buffers.format = context->format->pyro_cpu_format; + cpu_buffers.width = avctx->width; + cpu_buffers.height = avctx->height; + + for (int i = 0; i < 3; i++) { + if (frame->linesize[i] < 0) { + av_log(avctx, AV_LOG_ERROR, "Negative linesize not supported.\n"); + return AVERROR(EINVAL); + } + cpu_buffers.data[i] = frame->data[i]; + cpu_buffers.row_stride_in_bytes[i] = frame->linesize[i]; + + cpu_buffers.plane_size_in_bytes[i] = frame->linesize[i] * avctx->height; + if (i && context->format->subsampled) + cpu_buffers.plane_size_in_bytes[i] /= 2; + } + + rate_control.maximum_bitstream_size = context->target_payload_size; + + if (context->doublebuffer) + pyrowave_encoder_set_frame_context(context->encoder, context->frame_count % 2); + result = pyrowave_encoder_encode_cpu(context->encoder, &cpu_buffers, &rate_control); + } + + if (result != PYROWAVE_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to encode frame.\n"); + return AVERROR(EINVAL); + } + + if (!context->doublebuffer || context->frame_count > 0) { + if ((ret = ff_alloc_packet(avctx, pkt, context->target_payload_size)) < 0) + return ret; + + if (context->doublebuffer) + pyrowave_encoder_set_frame_context(context->encoder, (context->frame_count - 1) % 2); + + num_packets = 1; + result = pyrowave_encoder_packetize(context->encoder, &packet, context->target_payload_size, + &num_packets, pkt->data, context->target_payload_size); + + if (result != PYROWAVE_SUCCESS || + num_packets != 1 || packet.offset != 0 || + packet.size > context->target_payload_size) { + av_log(avctx, AV_LOG_ERROR, "Something unexpected happened with packetization.\n"); + return AVERROR_UNKNOWN; + } + + pkt->size = packet.size; + pkt->flags = AV_PKT_FLAG_KEY; + pkt->pts = context->doublebuffer ? context->pending_pts : frame->pts; + *got_packet = 1; + } + + if (frame) + context->pending_pts = frame->pts; + + if (context->flushing) + context->flushing++; + + context->frame_count++; + return 0; +} + +#define OFFSET(x) offsetof(PyroWaveEncodeContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM +static const AVOption options[] = { + { "doublebuffer", "Double buffer upload with encode", OFFSET(doublebuffer), AV_OPT_TYPE_BOOL, { .i64 = -1 }, -1, 1, VE }, + { NULL }, +}; + +static const AVClass libpyrowave_class = { + .class_name = "libpyrowave encoder", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFCodec ff_libpyrowave_encoder = { + .p.name = "libpyrowave", + CODEC_LONG_NAME("raw video"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_PYROWAVE, + .p.capabilities = AV_CODEC_CAP_DELAY, // Only applies to doublebuffer mode. + .priv_data_size = sizeof(PyroWaveEncodeContext), + .init = pwv_encode_init, + .close = pwv_encode_close, + FF_CODEC_ENCODE_CB(pwv_encode), + CODEC_PIXFMTS(AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV444P, + AV_PIX_FMT_YUV420P10, AV_PIX_FMT_YUV420P16, + AV_PIX_FMT_YUV444P10, AV_PIX_FMT_YUV444P16), + .color_ranges = AVCOL_RANGE_JPEG | AVCOL_RANGE_MPEG, + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, + .p.priv_class = &libpyrowave_class, +}; -- 2.52.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
