My motivations for this change do not come from a need or desire to use this feature; after performing this work, it is clear that OpenJPEG's capabilities are far superior and ought be preferred. My motivations come from the recent change of eae01677f47a200e841188322ad7982714594e34, which is itself motivated by attempts to point out "[human] slop" (focused on FFmpeg's) code. The author's change (and claim) does not acknowledge the intentionality of the removed conditional, which is evidenced in its initial inclusion: https://ffmpeg.org/pipermail/ffmpeg-devel/2015-June/174337.html
Again, upon review it is clear that OpenJPEG's capabilities and results far exceed that of FFmpeg's implemented JPEG-2000 encoder; I assume it generally preferred in practice and FFmpeg's implementation isn't meant to be all-encompassing, and as such won't be disappointed to learn these changes aren't welcome. That is of course counter to my point that the original "identical" branched code was intentional and not sloppy, but I digress. In that case, the issue is more one of policy and communicating intent of FFmpeg's implementation than of quality. As for the implementation: it's not perfect, due to my unfamiliarity of the domain and the existing assumptions/limitations of the encoder implementation. Of note is the [forced] use of an extra guard bit, to account for the increased chromatic channel bit depth of MCT (+1), and further (fractional) increases by subsequent DWT. The specification allows variable channel bit depth (signaled by ihdr bps field to 255), but this runs against other assumptions in tile encoding. Increasing the guard bit count accommodates the entire situation, and is common in other encoders as well; for example, OpenJPEG defaults to 2 guard bits (while, however, still producing codestreams that function with FFmpeg's j2k decoder when using only 1 guard bit). I've added some FATE tests for round-trip cases, but this does not well-test the codestream itself. I have verified the results with other decoders and mechanisms, and they seem well. Feedback is much appreciated. Thank you. -- Nathan --- [PATCH] avcodec/j2kenc: add support for MCT (RCT/ICT) -mct option for j2kenc; 0: off (default), 1: on, -1: auto only supported for RGB-like pixel formats (by nature) FATE round-trip tests for various MCT options Signed-off-by: Nathan Schulte <[email protected]> --- Changelog | 2 + doc/encoders.texi | 19 ++++++ libavcodec/j2kenc.c | 71 ++++++++++++++++++++++- tests/fate/jpeg2000.mak | 34 +++++++++++ tests/ref/fate/jpeg2000enc-mct-auto | 8 +++ tests/ref/fate/jpeg2000enc-mct-off | 8 +++ tests/ref/fate/jpeg2000enc-mct-on-rgb-ict | 8 +++ tests/ref/fate/jpeg2000enc-mct-on-rgb-rct | 8 +++ 8 files changed, 157 insertions(+), 1 deletion(-) create mode 100644 tests/ref/fate/jpeg2000enc-mct-auto create mode 100644 tests/ref/fate/jpeg2000enc-mct-off create mode 100644 tests/ref/fate/jpeg2000enc-mct-on-rgb-ict create mode 100644 tests/ref/fate/jpeg2000enc-mct-on-rgb-rct diff --git a/Changelog b/Changelog index 32e7c8bfe7..a9db12c321 100644 --- a/Changelog +++ b/Changelog @@ -19,6 +19,8 @@ version <next>: - Panasonic RW2 decoder - xHE-AAC encoding via libfdk-xhe-aac - jpegapp bitstream filter +- JPEG-2000 encoder MCT (RCT/ICT) support for RGB-like pixel formats with new + option -mct (default off) version 9.0: diff --git a/doc/encoders.texi b/doc/encoders.texi index 2c173ac916..92626ab507 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -1617,6 +1617,25 @@ Enable this to add SOP marker at the start of each packet. Disabled by default. @item eph @var{boolean} Enable this to add EPH marker at the end of each packet header. Disabled by default. +@item mct @var{integer} +Multiple Component Transformation. +@table @option +@item -1 +Automatically enable MCT for RGB-like pixel formats with at least three components. +@item 0 +Disable MCT (default). +@item 1 +Enable MCT. Applies the reversible colour transform (RCT) when using the +5/3 DWT (@option{pred=dwt53}) or the irreversible colour transform (ICT) +when using the 9/7 DWT. Only valid for RGB-like pixel formats with at least +three components (and without chroma subsampling). Alpha (or any extra components) +is left untouched. +@end table + +When enabled, the encoder both applies the transform and signals it in the +COD marker. Enabling MCT is recommended for RGB content as it usually +improves compression. + @item prog @var{integer} Sets the progression order to be used by the encoder. Possible values are: diff --git a/libavcodec/j2kenc.c b/libavcodec/j2kenc.c index da777580a4..e08799ec96 100644 --- a/libavcodec/j2kenc.c +++ b/libavcodec/j2kenc.c @@ -145,6 +145,7 @@ typedef struct { int prog; int nlayers; char *lr_str; + int mct; //< multiple component transform (RGB-like): -1: auto, 0: off (default), 1: on } Jpeg2000EncoderContext; @@ -279,7 +280,7 @@ static int put_cod(Jpeg2000EncoderContext *s) // SGcod bytestream_put_byte(&s->buf, s->prog); // progression level bytestream_put_be16(&s->buf, s->nlayers); // num of layers - bytestream_put_byte(&s->buf, 0); // unspecified + bytestream_put_byte(&s->buf, codsty->mct); // multiple component transformation // SPcod bytestream_put_byte(&s->buf, codsty->nreslevels - 1); // num of decomp. levels bytestream_put_byte(&s->buf, codsty->log2_cblk_width-2); // cblk width @@ -435,6 +436,51 @@ static int init_tiles(Jpeg2000EncoderContext *s) return 0; } +static void mct_encode_rct(int *y, int *u, int *v, int n) +{ + int i; + for (i = 0; i < n; i++) { + int R = y[i], G = u[i], B = v[i]; + y[i] = (R + (2 * G) + B) >> 2; + u[i] = B - G; + v[i] = R - G; + } +} + +static void mct_encode_ict(int *y, int *u, int *v, int n) +{ + int i; + for (i = 0; i < n; i++) { + double R = y[i], G = u[i], B = v[i]; + y[i] = lrint( 0.29900 * R + 0.58700 * G + 0.11400 * B); + u[i] = lrint(-0.16875 * R - 0.33126 * G + 0.50000 * B); + v[i] = lrint( 0.50000 * R - 0.41869 * G - 0.08131 * B); + } +} + +static void apply_mct(Jpeg2000EncoderContext *s) +{ + int tileno; + + if (!s->codsty.mct) + return; + + for (tileno = 0; tileno < s->numXtiles * s->numYtiles; tileno++) { + Jpeg2000Tile *tile = s->tile + tileno; + Jpeg2000Component *c0 = tile->comp + 0; + Jpeg2000Component *c1 = tile->comp + 1; + Jpeg2000Component *c2 = tile->comp + 2; + int w = c0->coord[0][1] - c0->coord[0][0]; + int h = c0->coord[1][1] - c0->coord[1][0]; + int n = w * h; + + if (s->codsty.transform == FF_DWT53) + mct_encode_rct(c0->i_data, c1->i_data, c2->i_data, n); + else + mct_encode_ict(c0->i_data, c1->i_data, c2->i_data, n); + } +} + #define COPY_FRAME(D, PIXEL) \ static void copy_frame_ ##D(Jpeg2000EncoderContext *s) \ { \ @@ -1491,6 +1537,8 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt, else copy_frame_8(s); + apply_mct(s); + reinit(s); if (s->format == CODEC_JP2) { @@ -1722,6 +1770,26 @@ static av_cold int j2kenc_init(AVCodecContext *avctx) } } + s->codsty.mct = 0; + if (s->mct < 0) { // mct == -1; auto-on for RGB-like + s->codsty.mct = (desc->flags & AV_PIX_FMT_FLAG_RGB) && s->ncomponents >= 3; + } else if (s->mct == 1) { + if (s->ncomponents >= 3) + s->codsty.mct = 1; + else + av_log(avctx, AV_LOG_WARNING, + "MCT requested but fewer than 3 components; disabling\n"); + } + if (s->codsty.mct && (s->chroma_shift[0] || s->chroma_shift[1])) { + av_log(avctx, AV_LOG_ERROR, "MCT is not supported with chroma subsampling; disabling\n"); + s->codsty.mct = 0; + } + if (s->codsty.mct && codsty->transform == FF_DWT53) { + // MCT expands chrominance range by 1 bit, and DWT can add more + // use extra guard bits to compensate + qntsty->nguardbits++; + } + ff_thread_once(&init_static_once, init_luts); init_quantization(s); @@ -1756,6 +1824,7 @@ static const AVOption options[] = { { "dwt53", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = 1 }, INT_MIN, INT_MAX, VE, .unit = "pred" }, { "sop", "SOP marker", OFFSET(sop), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE, }, { "eph", "EPH marker", OFFSET(eph), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE, }, + { "mct", "Multiple component transformation", OFFSET(mct), AV_OPT_TYPE_INT, { .i64 = 0 }, -1, 1, VE, .unit = "mct" }, { "prog", "Progression Order", OFFSET(prog), AV_OPT_TYPE_INT, { .i64 = 0 }, JPEG2000_PGOD_LRCP, JPEG2000_PGOD_CPRL, VE, .unit = "prog" }, { "lrcp", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = JPEG2000_PGOD_LRCP }, 0, 0, VE, .unit = "prog" }, { "rlcp", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = JPEG2000_PGOD_RLCP }, 0, 0, VE, .unit = "prog" }, diff --git a/tests/fate/jpeg2000.mak b/tests/fate/jpeg2000.mak index a99b0c4e0c..9fb7a356f2 100644 --- a/tests/fate/jpeg2000.mak +++ b/tests/fate/jpeg2000.mak @@ -67,3 +67,37 @@ FATE_JPEG2000DEC += $(FATE_JPEG2000DEC-yes) FATE_SAMPLES_FFMPEG-$(call FRAMECRC, IMAGE_J2K_PIPE, JPEG2000) += $(FATE_JPEG2000DEC) fate-jpeg2000dec: $(FATE_JPEG2000DEC) + +# JPEG 2000 encoder MCT tests. +# RGB + irreversible 9/7 transform: lossy round trip +FATE_JPEG2000ENC += fate-jpeg2000enc-mct-on-rgb-ict +fate-jpeg2000enc-mct-on-rgb-ict: CMD = transcode \ + "lavfi" "testsrc2=size=64x64:rate=1:duration=1,format=rgb24" \ + rawvideo "-c:v jpeg2000 -mct 1 -pred dwt97int -frames:v 1" \ + "-pix_fmt rgb24" + +# RGB + reversible 5/3 transform: lossless round trip +FATE_JPEG2000ENC += fate-jpeg2000enc-mct-on-rgb-rct +fate-jpeg2000enc-mct-on-rgb-rct: CMD = transcode \ + "lavfi" "testsrc2=size=64x64:rate=1:duration=1,format=rgb24" \ + rawvideo "-c:v jpeg2000 -mct 1 -pred dwt53 -frames:v 1" \ + "-pix_fmt rgb24" + +# Explicitly disabled MCT must retain lossless round-trip behavior +FATE_JPEG2000ENC += fate-jpeg2000enc-mct-off +fate-jpeg2000enc-mct-off: CMD = transcode \ + "lavfi" "testsrc2=size=64x64:rate=1:duration=1,format=rgb24" \ + rawvideo "-c:v jpeg2000 -mct 0 -pred dwt53 -frames:v 1" \ + "-pix_fmt rgb24" + +# Automatic MCT selection on RGB input; round trip +FATE_JPEG2000ENC += fate-jpeg2000enc-mct-auto +fate-jpeg2000enc-mct-auto: CMD = transcode \ + "lavfi" "testsrc2=size=64x64:rate=1:duration=1,format=rgb24" \ + rawvideo "-c:v jpeg2000 -mct -1 -pred dwt53 -frames:v 1" \ + "-pix_fmt rgb24" + +FATE_JPEG2000ENC-$(call ALLYES, JPEG2000_ENCODER JPEG2000_DECODER RAWVIDEO_ENCODER RAWVIDEO_DECODER LAVFI_INDEV) += $(FATE_JPEG2000ENC) + +FATE_SAMPLES_FFMPEG-$(call ALLYES, JPEG2000_ENCODER JPEG2000_DECODER RAWVIDEO_ENCODER RAWVIDEO_DECODER LAVFI_INDEV) += $(FATE_JPEG2000ENC-yes) +fate-jpeg2000enc: $(FATE_JPEG2000ENC-yes) diff --git a/tests/ref/fate/jpeg2000enc-mct-auto b/tests/ref/fate/jpeg2000enc-mct-auto new file mode 100644 index 0000000000..d7601398b8 --- /dev/null +++ b/tests/ref/fate/jpeg2000enc-mct-auto @@ -0,0 +1,8 @@ +f831eb2c9fb1673962435a257332ced4 *tests/data/fate/jpeg2000enc-mct-auto.rawvideo +6748 tests/data/fate/jpeg2000enc-mct-auto.rawvideo +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 64x64 +#sar 0: 0/1 +0, 0, 0, 1, 12288, 0xcd3a18af diff --git a/tests/ref/fate/jpeg2000enc-mct-off b/tests/ref/fate/jpeg2000enc-mct-off new file mode 100644 index 0000000000..392646eb9d --- /dev/null +++ b/tests/ref/fate/jpeg2000enc-mct-off @@ -0,0 +1,8 @@ +f7f378e60f4496b3f6854affe01af4fc *tests/data/fate/jpeg2000enc-mct-off.rawvideo +6043 tests/data/fate/jpeg2000enc-mct-off.rawvideo +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 64x64 +#sar 0: 0/1 +0, 0, 0, 1, 12288, 0xcd3a18af diff --git a/tests/ref/fate/jpeg2000enc-mct-on-rgb-ict b/tests/ref/fate/jpeg2000enc-mct-on-rgb-ict new file mode 100644 index 0000000000..f55a1e01a2 --- /dev/null +++ b/tests/ref/fate/jpeg2000enc-mct-on-rgb-ict @@ -0,0 +1,8 @@ +ccf2a1580d61f29a8c36d941a06a2944 *tests/data/fate/jpeg2000enc-mct-on-rgb-ict.rawvideo +7499 tests/data/fate/jpeg2000enc-mct-on-rgb-ict.rawvideo +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 64x64 +#sar 0: 0/1 +0, 0, 0, 1, 12288, 0xe11a1b4d diff --git a/tests/ref/fate/jpeg2000enc-mct-on-rgb-rct b/tests/ref/fate/jpeg2000enc-mct-on-rgb-rct new file mode 100644 index 0000000000..6576b592b7 --- /dev/null +++ b/tests/ref/fate/jpeg2000enc-mct-on-rgb-rct @@ -0,0 +1,8 @@ +f831eb2c9fb1673962435a257332ced4 *tests/data/fate/jpeg2000enc-mct-on-rgb-rct.rawvideo +6748 tests/data/fate/jpeg2000enc-mct-on-rgb-rct.rawvideo +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 64x64 +#sar 0: 0/1 +0, 0, 0, 1, 12288, 0xcd3a18af -- 2.55.0 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
