Dear Roman,
Thank you a lot for your comments and raising the important questions.
> How should a set of QPs be specified?
As you probably know, a set of available QP values can be codec specific.
Usually, a QP set is selected to cover medium bit-rate range that is considered
to be the most complex for compressing. We proposed to set up the assessment
process not only for this range but also for low and high QP ranges to cover a
wider range of applications. According to my understanding, concrete QP values
should be specified for candidate codecs in draft-ietf-netvc-testing.
> How should the quality values be calculated?
It is explained in Section 4.1, namely: “To assess the quality of output
(decoded) sequences, two indexes, PSNR [3] and MS-SSIM [3,11] are separately
computed. In the case of the YCbCr color format, PSNR should be calculated for
each color plane whereas MS-SSIM is calculated for luma channel only. In the
case of the RGB color format, both metrics are computed for R, G and B
channels. Thus, for each sequence, 30 RD-points for PSNR (i.e. three RD-curves,
one for each channel) and 10 RD-points for MS-SSIM (i.e. one RD-curve, for luma
channel only) should be calculated in the case of YCbCr. If content is encoded
as RGB, 60 RD-points (30 for PSNR and 30 for MS-SSIM) should be calculated,
i.e. three RD-curves (one for each channel) are computed for PSNR as well as
three RD-curves (one for each channel) for MS-SSIM.” In references [3, 11],
these 2 quality assessment metrics and the ways of how to calculate them are
described in detail.
>-- What does the text “(in a separate document on Internet video codec
>testing)” mean?
> Per “A list of video sequences that should be used for testing as well as the
> 10 QP values for the reference codec are defined in a separate document ",
> what document is that? Is it draft-ietf-netvc-testing?
According to my understanding, it is draft-ietf-netvc-testing.
>What does “codec implementation (for both an encoder and a decoder) should
>cover the worst case of computational complexity, memory bandwidth, and
>physical memory size” mean?
I’d like to thank Adam for his comment. I also think that his formulation
("...should take into consideration the worst-case...") is clearer.
A more detailed explanation on this Section can be found in my response to the
secdir reviewer:
https://mailarchive.ietf.org/arch/msg/secdir/QYx-hl07aec5XhgqkldqgWFG3Is
>Please add additional language that codec should be written in a defensive
>style as they will be processing untrusted input.
Good point. Thank you. I’ll add it.
--
Best regards,
Alexey Filippov
-----Original Message-----
From: Roman Danyliw via Datatracker [mailto:[email protected]]
Sent: Wednesday, June 12, 2019 4:25 AM
To: The IESG <[email protected]>
Cc: [email protected]; Mo Zanaty <[email protected]>;
[email protected]; [email protected]; [email protected]
Subject: Roman Danyliw's Discuss on draft-ietf-netvc-requirements-09: (with
DISCUSS and COMMENT)
Roman Danyliw has entered the following ballot position for
draft-ietf-netvc-requirements-09: Discuss
When responding, please keep the subject line intact and reply to all email
addresses included in the To and CC lines. (Feel free to cut this introductory
paragraph, however.)
Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.
The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-netvc-requirements/
----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------
(1) I worry that the level of detail in the in the Compression Performance
Evaluation (Section 4.1) is insufficient for implementation. Specifically:
(a) Per “Initially, for the codec selected as a reference one (e.g., HEVC or
VP9), a set of 10 QP quantization parameter) values should be specified (in a
separate document on Internet video codec testing) and corresponding quality
values should be calculated.”
-- How should a set of QPs be specified?
--How should the quality values be calculated?
-- What does the text “(in a separate document on Internet video codec
testing)” mean?
(b) Per “A list of video sequences that should be used for testing as well as
the 10 QP values for the reference codec are defined in a separate document ",
what document is that? Is it draft-ietf-netvc-testing?
(2) Per the Security Considerations Section (Section 5)
-- What does “codec implementation (for both an encoder and a decoder) should
cover the worst case of computational complexity, memory bandwidth, and
physical memory size” mean?
-- Please add additional language that codec should be written in a defensive
style as they will be processing untrusted input.
----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------
(1) There is something odd about the document formatting – the title and the
first author last name in the footer is wrapped in “< … >”
(2) It would be helpful to forward reference that acronyms are explained in
Appendix A.
(3) This draft uses the words should and must to prescribe action. Why wasn’t
RFC2119 cited to explain these words?
(4) Section 2.0. A reference to explain “YCbCr 4:2:0” would be helpful
(5) Section 2.1. Per “high encoder complexity” and “decoding complexity”, I
initiate read that as a qualitative measure. However, the text says “up to 10x
and more” so that implies some quantitative measure. What is that?
(6) Section 2.1. Expand QP values on first use
(7) Section 2.x. The language around content doesn’t appear to be consist.
For example:
-- Section 2.1, Internet Video Streaming says “movies, TV-series and shows, and
animation.”
-- Section 2.2, IPTV says “television content”
-- Section 2.5, Screen casting says “business presentations …, animation
(cartoons), gaming content, data visualization, …, virtual desktop
infrastructure (VDI), screen/desktop sharing and collaboration, supervisory
control and data acquisition (SCADA) display, automotive/navigation display,
cloud gaming, factory automation display, wireless display, display wall,
digital operating room (DiOR), etc.
What the difference between Section 2.1’s animation and Section 2.5’s cartoons?
What’s the different between Section 2.1’s “movies, TV series …” and Section
2.2’s “television content”?
(8) Section 2.5. The sentence “Currently, …” is very challenging to parse as
it includes inline “i.e.,” and “etc”.
(9) Section 2.5. Per “powerpoint, word documents”, these are specific
Microsoft products. I recommend using more generic names.
(10) Section 4. I found it confusing that an evaluation methodology was in a
requirements document. I would have expected it in the draft-ietf-netvc-testing
(11) Section 4.1. VP9 needs a reference.
(12) Section 6. I don’t think this entire section is necessary.
(13) Editorial Nits:
-- Section 3. Style nit. s/chapter/section/
-- Section 4.1. Typo. s/computged/computed
_______________________________________________
video-codec mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/video-codec
