Re: [FFmpeg-devel] [PATCH] avformat/mov: fix hang while seek on a kind of fragmented mp4.
I can’t reproduce the OOM. Is it possible the problem relate to a specific clip? Here is my test steps: 1. To ensure that it’s not affected by caches, delete all except the .git dir in my code dir. Then checkout again. 2. To ensure that the latest test clips are used. make fate-rsync SAMPLES=fate-suite/ 3. To ensure that OS environment is clean, run test in docker. Below is one of my test logs: > /ffmpeg_src# valgrind --leak-check=full ./ffmpeg -i issue/dash.mp4 -map 0 > -f null - > > *==8552== Memcheck, a memory error detector* > *==8552== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.* > *==8552== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright > info* > *==8552== Command: ./ffmpeg -i issue/dash.mp4 -map 0 -f null -* > *==8552== * > *ffmpeg version 4.1.git Copyright (c) 2000-2019 the FFmpeg developers* > * built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)* > * configuration: --samples=fate-suite/ --disable-doc --fatal-warnings > --valgrind=VALGRIND* > * libavutil 56. 26.100 / 56. 26.100* > * libavcodec 58. 47.102 / 58. 47.102* > * libavformat58. 26.101 / 58. 26.101* > * libavdevice58. 6.101 / 58. 6.101* > * libavfilter 7. 48.100 / 7. 48.100* > * libswscale 5. 4.100 / 5. 4.100* > * libswresample 3. 4.100 / 3. 4.100* > *Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'issue/dash.mp4':* > * Metadata:* > *major_brand : iso5* > *minor_version : 512* > *compatible_brands: iso6mp41* > *encoder : Lavf58.20.100* > * Duration: 00:00:57.76, start: 0.00, bitrate: 5193 kb/s* > *Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, > smpte170m/bt709/bt709), 1280x720, 5125 kb/s, 25 fps, 25 tbr, 250k tbn, 50 > tbc (default)* > *Metadata:* > * handler_name: VideoHandler* > *Stream #0:1(und): Audio: aac (HE-AAC) (mp4a / 0x6134706D), 44100 Hz, > stereo, fltp, 79 kb/s (default)* > *Metadata:* > * handler_name: SoundHandler* > *Stream mapping:* > * Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))* > * Stream #0:1 -> #0:1 (aac (native) -> pcm_s16le (native))* > *Press [q] to stop, [?] for help* > *Output #0, null, to 'pipe:':ze=N/A time=-577014:32:22.77 bitrate=N/A > speed=N/A* > * Metadata:* > *major_brand : iso5* > *minor_version : 512* > *compatible_brands: iso6mp41* > *encoder : Lavf58.26.101* > *Stream #0:0(und): Video: wrapped_avframe, yuv420p(progressive), > 1280x720, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)* > *Metadata:* > * handler_name: VideoHandler* > * encoder : Lavc58.47.102 wrapped_avframe* > *Stream #0:1(und): Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s > (default)* > *Metadata:* > * handler_name: SoundHandler* > * encoder : Lavc58.47.102 pcm_s16le* > *frame= 1438 fps=5.0 q=-0.0 Lsize=N/A time=00:00:57.68 bitrate=N/A > speed=0.202x* > *video:753kB audio:9904kB subtitle:0kB other streams:0kB global > headers:0kB muxing overhead: unknown* > *==8552== * > *==8552== HEAP SUMMARY:* > *==8552== in use at exit: 0 bytes in 0 blocks* > *==8552== total heap usage: 207,112 allocs, 207,112 frees, 121,288,827 > bytes allocated* > *==8552== * > *==8552== All heap blocks were freed -- no leaks are possible* > *==8552== * > *==8552== For counts of detected and suppressed errors, rerun with: > -v**==8552== > ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)* BTW, below is my dockefile. It show that the installed dependences. And how to configure and test. *FROM ubuntu:18.04* *RUN apt-get update -qq && apt-get dist-upgrade -qy* *RUN apt-get -qy install autoconf automake build-essential cmake libass-dev libfreetype6-dev libtool libvorbis-dev pkg-config texinfo zlib1g-dev \* * libnuma-dev* *RUN apt-get -qy install yasm libx264-dev libx265-dev libfdk-aac-dev* *RUN apt-get -qy install valgrind* *COPY . /ffmpeg_src/* *WORKDIR /ffmpeg_src* *RUN make distclean* *RUN ./configure --samples=fate-suite/ --disable-doc --fatal-warnings --valgrind=VALGRIND* *RUN make -j3 && make fate* *CMD ["/bin/bash"]* Michael Niedermayer 于2019年2月23日周六 上午8:25写道： > On Fri, Feb 22, 2019 at 06:20:40PM +0800, Charles Liu wrote: > > 1. organize fragmented information according to the tracks. > > 2. do NOT skip the last boxes of fragmented info. > > > > ticket #7572 > > > > Signed-off-by: Charles Liu > > --- > > libavformat/isom.h | 10 +- > > libavformat/mov.c | 374 + > > 2 files changed, 181 insertions(+), 203 deletions(-) > > still hitting out of memory and being killed by the kernel > the other issue seems to be fixed though > > thx > > > [...] > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > The educated differ from the uneducated as much as the living from the > dead. -- Aristotle > ___ > ffmpeg-devel mailing list >
Re: [FFmpeg-devel] [PATCH 4/5] avfilter/vf_thumbnail_cuda: Switch to using ffnvcodec
On Fri, 22 Feb 2019 05:46:12 + Soft Works wrote: > > Subject: [FFmpeg-devel] [PATCH 4/5] avfilter/vf_thumbnail_cuda: > > Switch to using ffnvcodec > > > > This change switches the vf_thumbnail_cuda filter from using the > > full cuda sdk to using the ffnvcodec headers and loader. > > > > Most of the change is a direct mapping, but I also switched from > > using texture references to using texture objects. This is supposed > > to be the preferred way of using textures, and the texture object > > API is the one I added to ffnvcodec. > > > > Signed-off-by: Philip Langdale > > --- > > configure| 2 +- > > libavfilter/vf_thumbnail_cuda.c | 147 > > +-- libavfilter/vf_thumbnail_cuda.cu | > > 25 +++--- 3 files changed, 93 insertions(+), 81 deletions(-) > > > -CHECK_CU(cuMemsetD8(s->data, 0, HIST_SIZE * sizeof(int))); > > +CHECK_CU(cu->cuMemsetD8Async(s->data, 0, HIST_SIZE * > > sizeof(int), s->cu_stream)); > Hi Phil, > > the only problem I encountered was the cuMemsetD8Async function > which needs yet to be added to the ffnvcodec headers. > > The other filters are building fine (MSYS2/Win). https://github.com/philipl/nv-codec-headers/commit/dc12b592e5efc195d75363b8aae74db1180c9b0d --phil ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] lavc/libx265: signal CPB properties through side data
This way values such as maxrate/bufsize can be utilized
further down the chain.
---
libavcodec/libx265.c | 8
1 file changed, 8 insertions(+)
diff --git a/libavcodec/libx265.c b/libavcodec/libx265.c
index 98415366da..fe39f45241 100644
--- a/libavcodec/libx265.c
+++ b/libavcodec/libx265.c
@@ -79,6 +79,7 @@ static av_cold int libx265_encode_close(AVCodecContext *avctx)
static av_cold int libx265_encode_init(AVCodecContext *avctx)
{
libx265Context *ctx = avctx->priv_data;
+AVCPBProperties *cpb_props = NULL;
ctx->api =
x265_api_get(av_pix_fmt_desc_get(avctx->pix_fmt)->comp[0].depth);
if (!ctx->api)
@@ -208,6 +209,13 @@ static av_cold int libx265_encode_init(AVCodecContext
*avctx)
ctx->params->rc.vbvBufferSize = avctx->rc_buffer_size / 1000;
ctx->params->rc.vbvMaxBitrate = avctx->rc_max_rate/ 1000;
+cpb_props = ff_add_cpb_side_data(avctx);
+if (!cpb_props)
+return AVERROR(ENOMEM);
+cpb_props->buffer_size = ctx->params->rc.vbvBufferSize * 1000;
+cpb_props->max_bitrate = ctx->params->rc.vbvMaxBitrate * 1000;
+cpb_props->avg_bitrate = ctx->params->rc.bitrate * 1000;
+
if (!(avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER))
ctx->params->bRepeatHeaders = 1;
--
2.20.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] avformat/mov: fix hang while seek on a kind of fragmented mp4.
On Fri, Feb 22, 2019 at 06:20:40PM +0800, Charles Liu wrote: > 1. organize fragmented information according to the tracks. > 2. do NOT skip the last boxes of fragmented info. > > ticket #7572 > > Signed-off-by: Charles Liu > --- > libavformat/isom.h | 10 +- > libavformat/mov.c | 374 + > 2 files changed, 181 insertions(+), 203 deletions(-) still hitting out of memory and being killed by the kernel the other issue seems to be fixed though thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB The educated differ from the uneducated as much as the living from the dead. -- Aristotle signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [RFC PATCH] ffmpeg: explicitly handle timestamp re-initialization
Now each time as the sub2video structure is initialized, the
current time is updated to the first received heartbeat's PTS.
---
Sending this out as an alternative to the other patch, showing how the
sub2video time can be synchronized to the heartbeat explicitly when the
structure is (re-)initialized.
Jan
fftools/ffmpeg.c| 12 +++-
fftools/ffmpeg.h| 3 ++-
fftools/ffmpeg_filter.c | 8 +++-
3 files changed, 16 insertions(+), 7 deletions(-)
diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
index 544f1a1cef..4cba08e9b8 100644
--- a/fftools/ffmpeg.c
+++ b/fftools/ffmpeg.c
@@ -237,7 +237,7 @@ static void sub2video_push_ref(InputStream *ist, int64_t
pts)
}
}
-void sub2video_update(InputStream *ist, AVSubtitle *sub)
+void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub)
{
AVFrame *frame = ist->sub2video.frame;
int8_t *dst;
@@ -254,7 +254,8 @@ void sub2video_update(InputStream *ist, AVSubtitle *sub)
AV_TIME_BASE_Q, ist->st->time_base);
num_rects = sub->num_rects;
} else {
-pts = ist->sub2video.end_pts;
+pts = ist->sub2video.init_timestamps ?
+heartbeat_pts : ist->sub2video.end_pts;
end_pts = INT64_MAX;
num_rects = 0;
}
@@ -269,6 +270,7 @@ void sub2video_update(InputStream *ist, AVSubtitle *sub)
sub2video_copy_rect(dst, dst_linesize, frame->width, frame->height,
sub->rects[i]);
sub2video_push_ref(ist, pts);
ist->sub2video.end_pts = end_pts;
+ist->sub2video.init_timestamps = 0;
}
static void sub2video_heartbeat(InputStream *ist, int64_t pts)
@@ -293,7 +295,7 @@ static void sub2video_heartbeat(InputStream *ist, int64_t
pts)
continue;
if (pts2 >= ist2->sub2video.end_pts ||
(!ist2->sub2video.frame->data[0] && ist2->sub2video.end_pts <
INT64_MAX))
-sub2video_update(ist2, NULL);
+sub2video_update(ist2, pts2 + 1, NULL);
for (j = 0, nb_reqs = 0; j < ist2->nb_filters; j++)
nb_reqs +=
av_buffersrc_get_nb_failed_requests(ist2->filters[j]->filter);
if (nb_reqs)
@@ -307,7 +309,7 @@ static void sub2video_flush(InputStream *ist)
int ret;
if (ist->sub2video.end_pts < INT64_MAX)
-sub2video_update(ist, NULL);
+sub2video_update(ist, INT64_MAX, NULL);
for (i = 0; i < ist->nb_filters; i++) {
ret = av_buffersrc_add_frame(ist->filters[i]->filter, NULL);
if (ret != AVERROR_EOF && ret < 0)
@@ -2514,7 +2516,7 @@ static int transcode_subtitles(InputStream *ist, AVPacket
*pkt, int *got_output,
return ret;
if (ist->sub2video.frame) {
-sub2video_update(ist, );
+sub2video_update(ist, INT64_MIN, );
} else if (ist->nb_filters) {
if (!ist->sub2video.sub_queue)
ist->sub2video.sub_queue = av_fifo_alloc(8 * sizeof(AVSubtitle));
diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
index eb1eaf6363..58b52984c3 100644
--- a/fftools/ffmpeg.h
+++ b/fftools/ffmpeg.h
@@ -349,6 +349,7 @@ typedef struct InputStream {
AVFifoBuffer *sub_queue;///< queue of AVSubtitle* before filter
init
AVFrame *frame;
int w, h;
+unsigned int init_timestamps; ///< marks if sub2video_update should
re-init timestamps
} sub2video;
int dr1;
@@ -646,7 +647,7 @@ int filtergraph_is_simple(FilterGraph *fg);
int init_simple_filtergraph(InputStream *ist, OutputStream *ost);
int init_complex_filtergraph(FilterGraph *fg);
-void sub2video_update(InputStream *ist, AVSubtitle *sub);
+void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle
*sub);
int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *frame);
diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
index 72838de1e2..c0119a4676 100644
--- a/fftools/ffmpeg_filter.c
+++ b/fftools/ffmpeg_filter.c
@@ -740,6 +740,12 @@ static int sub2video_prepare(InputStream *ist, InputFilter
*ifilter)
return AVERROR(ENOMEM);
ist->sub2video.last_pts = INT64_MIN;
ist->sub2video.end_pts = INT64_MIN;
+
+/* sub2video structure has been (re-)initialized.
+ Mark it as such so that the current time
+ can be initialized with the first heartbeat. */
+ist->sub2video.init_timestamps = 1;
+
return 0;
}
@@ -1169,7 +1175,7 @@ int configure_filtergraph(FilterGraph *fg)
while (av_fifo_size(ist->sub2video.sub_queue)) {
AVSubtitle tmp;
av_fifo_generic_read(ist->sub2video.sub_queue, ,
sizeof(tmp), NULL);
-sub2video_update(ist, );
+sub2video_update(ist, INT64_MIN, );
avsubtitle_free();
}
}
--
2.20.1
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] avformat/matroskadec: Check parents remaining length
Sent http://ffmpeg.org/pipermail/ffmpeg-devel/2019-February/240418.html -
which passes fate and fixes the issue with our test clip.
- dale
On Fri, Feb 22, 2019 at 12:31 PM Dale Curtis
wrote:
> +steve who submitted the original patch.
>
> - dale
>
>
> On Thu, Feb 21, 2019 at 2:30 PM Dale Curtis
> wrote:
>
>> One of our test clips is behaving differently after this patch:
>>
>> https://cs.chromium.org/chromium/src/media/test/data/bear-320x240-live.webm
>>
>> The printed log message is:
>> [matroska,webm @ 0x1516c84f4e00] Invalid length 0xff >
>> 0x12f in parent
>>
>> Looking through the code I'm unsure which of the mixed usage
>> "(uint64_t)-1" and "2**56-1" as marker values is correct. Changing the code
>> to:
>> diff --git a/libavformat/matroskadec.c b/libavformat/matroskadec.c
>> index 9b706ab4e0..3015a0b230 100644
>> --- a/libavformat/matroskadec.c
>> +++ b/libavformat/matroskadec.c
>> @@ -1205,7 +1205,7 @@ static int ebml_parse_elem(MatroskaDemuxContext
>> *matroska,
>> MatroskaLevel *level =
>> >levels[matroska->num_levels - 1];
>> AVIOContext *pb = matroska->ctx->pb;
>> int64_t pos = avio_tell(pb);
>> -if (level->length != (uint64_t) -1 &&
>> +if (level->length != 0xffULL &&
>> (pos + length) > (level->start + level->length)) {
>> av_log(matroska->ctx, AV_LOG_ERROR,
>> "Invalid length 0x%"PRIx64" > 0x%"PRIx64" in
>> parent\n",
>>
>> Resolves our issues. Should all other length tests be done the same way?
>>
>> - dale
>>
>> On Sun, Feb 17, 2019 at 12:45 AM Michael Niedermayer
>> wrote:
>>
>>> On Wed, Feb 13, 2019 at 01:41:31PM +0100, Michael Niedermayer wrote:
>>> > Reported-by: Steve Lhomme
>>> > This was found through the Hacker One program on VLC but is not a
>>> security issue in libavformat
>>> > Signed-off-by: Michael Niedermayer
>>> > ---
>>> > libavformat/matroskadec.c | 21 +
>>> > 1 file changed, 21 insertions(+)
>>>
>>> will apply an equivalent variant from steve
>>>
>>> [...]
>>> --
>>> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>>>
>>> Asymptotically faster algorithms should always be preferred if you have
>>> asymptotical amounts of data
>>> ___
>>> ffmpeg-devel mailing list
>>> ffmpeg-devel@ffmpeg.org
>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>
>>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] Fix handling of unknown length case for matroska files.
Unknown length has a special encoding which is not uint64_t(-1).
Signed-off-by: Dale Curtis
---
libavformat/matroskadec.c | 13 -
1 file changed, 8 insertions(+), 5 deletions(-)
From 2bf28a1edb54297f44021771b4c3d847c1f923f4 Mon Sep 17 00:00:00 2001
From: Dale Curtis
Date: Fri, 22 Feb 2019 15:39:25 -0800
Subject: [PATCH] Fix handling of unknown length case for matroska files.
Unknown length has a special encoding which is not uint64_t(-1).
Signed-off-by: Dale Curtis
---
libavformat/matroskadec.c | 13 -
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/libavformat/matroskadec.c b/libavformat/matroskadec.c
index 5aa8a105dc..6d86342016 100644
--- a/libavformat/matroskadec.c
+++ b/libavformat/matroskadec.c
@@ -68,6 +68,9 @@
#include "qtpalette.h"
+// 2^56 - 1.
+#define EBML_UNKNOWN_LEN 0xffULL
+
typedef enum {
EBML_NONE,
EBML_UINT,
@@ -869,7 +872,7 @@ static int ebml_read_length(MatroskaDemuxContext *matroska, AVIOContext *pb,
{
int res = ebml_read_num(matroska, pb, 8, number);
if (res > 0 && *number + 1 == 1ULL << (7 * res))
-*number = 0xffULL;
+*number = EBML_UNKNOWN_LEN;
return res;
}
@@ -1049,7 +1052,7 @@ static int ebml_parse_id(MatroskaDemuxContext *matroska, EbmlSyntax *syntax,
break;
if (!syntax[i].id && id == MATROSKA_ID_CLUSTER &&
matroska->num_levels > 0 &&
-matroska->levels[matroska->num_levels - 1].length == 0xff)
+matroska->levels[matroska->num_levels - 1].length == EBML_UNKNOWN_LEN)
return 0; // we reached the end of an unknown size cluster
if (!syntax[i].id && id != EBML_ID_VOID && id != EBML_ID_CRC32) {
av_log(matroska->ctx, AV_LOG_DEBUG, "Unknown entry 0x%"PRIX32"\n", id);
@@ -1201,7 +1204,7 @@ static int ebml_parse_elem(MatroskaDemuxContext *matroska,
MatroskaLevel *level = >levels[matroska->num_levels - 1];
AVIOContext *pb = matroska->ctx->pb;
int64_t pos = avio_tell(pb);
-if (level->length != (uint64_t) -1 &&
+if (level->length != EBML_UNKNOWN_LEN &&
(pos + length) > (level->start + level->length)) {
av_log(matroska->ctx, AV_LOG_ERROR,
"Invalid length 0x%"PRIx64" > 0x%"PRIx64" in parent\n",
@@ -1610,7 +1613,7 @@ static int matroska_parse_seekhead_entry(MatroskaDemuxContext *matroska,
ret = AVERROR_INVALIDDATA;
} else {
level.start = 0;
-level.length = (uint64_t) -1;
+level.length = EBML_UNKNOWN_LEN;
matroska->levels[matroska->num_levels] = level;
matroska->num_levels++;
matroska->current_id = 0;
@@ -1620,7 +1623,7 @@ static int matroska_parse_seekhead_entry(MatroskaDemuxContext *matroska,
/* remove dummy level */
while (matroska->num_levels) {
uint64_t length = matroska->levels[--matroska->num_levels].length;
-if (length == (uint64_t) -1)
+if (length == EBML_UNKNOWN_LEN)
break;
}
}
--
2.21.0.rc0.258.g878e2cd30e-goog
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 1/2] libavcodec/zmbvenc: block scoring improvements/bug fixes
On Fri, Feb 22, 2019 at 01:09:12PM +0100, Tomas Härdin wrote: > fre 2019-02-22 klockan 00:11 +0100 skrev Michael Niedermayer: > > On Thu, Feb 21, 2019 at 09:26:44PM +, Matthew Fearnley wrote: > > > - To clear up the effects: the change from 'i=1' to 'i=0' was the only > > > change that should make any difference in the output. > > > The rest of the changes were to improve the speed of an inner loop, and to > > > fix two bugs that happily cancelled each other out, but would have broken > > > if the code were changed to emit larger blocks. These should not lead to > > > changes in the blocks chosen or the final compression size. > > > > > > - Regarding the worse compression: let me start by stating that scoring on > > > byte frequency alone will not always predict well how Deflate will work, > > > since Deflate also uses pattern matching as well. > > > Frequency-based block scoring will only ever be a heuristic. There may be > > > scope for improving the scoring, but it would add a lot of complexity, and > > > I think it would be easy to make things worse rather than better. (I > > > found > > > this in my brief experiments with including the frequencies from the > > > previous block.) > > > > implementing the exact scoring by using Deflate itself would allow us to > > understand how much there is between the heuristic and what is achievable. > > If there is a gap that is significant then it may be interresting to > > look at improving the heuristic if the gap is small than the whole > > heuristic improvment path can be discarded. > > The obvious improvment for the heuristic would be to move to higher > > order entropy estimation. ATM IIUC this uses a 0th order. 1st order could > > be tried or any fast compression algorithm could also be used to estimate > > entropy. > > We tried a sliding window order-0 model, which did worse. An order-1 > model might be useful, but would need quite a bit of context for it to > be meaningful > > > It also would be interresting to see how a higher order heuristic or > > some simple fast LZ like compressor would perform with the previous block. > > That is if the issue you saw with the previous block inclusion was because > > of the very simplistic entropy estimation. > > zlib at level 1 maybe? You could also have a multi-pass thing where it > first does a coarse pass then tries to jiggle MVs and greedily accept > any change that results in smaller output. yes, but i would avoid using external code like zlib for this. Its not designed for fast entropy estimation of many blocks. [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Opposition brings concord. Out of discord comes the fairest harmony. -- Heraclitus signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 2/2] avcodec/pnm: Avoid structure pointer dereferences in inner loop in pnm_get()
On Fri, Feb 22, 2019 at 09:10:55AM +0200, Lauri Kasanen wrote: > On Thu, 21 Feb 2019 20:34:29 +0100 > Michael Niedermayer wrote: > > > Improves speed from 5.4 to 4.2 seconds > > Fixes: > > 13149/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PGM_fuzzer-5760833622114304 > > LGTM will apply > > Though, I really would expect the compiler to detect and optimize that. > I wonder if "PNMContext * const sc" would help it any. i doubt that would help. the char * pointer both the one we are reding from and the one we write to could in principle alias anything else (this is allowed in C) So the compiler would have to proof every time it writes to str/s that this cannot alias anything in the structure. And every time it writes to the structure that it cannot alias the bytestream its reading from. Otherwise it cannot optimize the operations out thanks [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Whats the most studid thing your enemy could do ? Blow himself up Whats the most studid thing you could do ? Give up your rights and freedom because your enemy blew himself up. signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 1/2] avcodec/zmbv: obtain frame later
On Fri, Feb 22, 2019 at 01:45:25PM +0100, Tomas Härdin wrote: > tor 2019-02-21 klockan 20:34 +0100 skrev Michael Niedermayer: > > The frame is not needed that early so obtaining it later avoids > > the costly operation in case other checks fail. > > > > Fixes: Timeout (14sec -> 4sec) > > Fixes: > > 13140/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ZMBV_fuzzer-5738330308739072 > > > > Found-by: continuous fuzzing process > > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg > > > Signed-off-by: Michael Niedermayer > > --- > > libavcodec/zmbv.c | 6 +++--- > > 1 file changed, 3 insertions(+), 3 deletions(-) > > Looks OK, passes FATE will apply thanks [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Democracy is the form of government in which you can choose your dictator signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] Reimbursement request for LDP
On Sun, Feb 10, 2019 at 07:53:50PM +0100, Thilo Borgmann wrote: > Hi, > > I'd like to request reimbursement for my travel to the LDP [1]. > > Accomodation was sponsored by the LDP team, so travel costs are rather low: > 21.90€ LGTM thx > > Will send the paperwork to Stefano. > > Thanks, > Thilo > > > [1] https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2018-October/235352.html > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB If you fake or manipulate statistics in a paper in physics you will never get a job again. If you fake or manipulate statistics in a paper in medicin you will get a job for life at the pharma industry. signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] rtpenc_chain: forward strict_std_compliance flags to rtp muxer
fixes: https://trac.ffmpeg.org/ticket/6713 --- libavformat/rtpenc_chain.c | 1 + 1 file changed, 1 insertion(+) diff --git a/libavformat/rtpenc_chain.c b/libavformat/rtpenc_chain.c index d3c1bc96dc..e7a4dffaba 100644 --- a/libavformat/rtpenc_chain.c +++ b/libavformat/rtpenc_chain.c @@ -59,6 +59,7 @@ int ff_rtp_chain_mux_open(AVFormatContext **out, AVFormatContext *s, /* Copy other stream parameters. */ rtpctx->streams[0]->sample_aspect_ratio = st->sample_aspect_ratio; rtpctx->flags |= s->flags & AVFMT_FLAG_BITEXACT; +rtpctx->strict_std_compliance = s->strict_std_compliance; /* Get the payload type from the codec */ if (st->id < RTP_PT_PRIVATE) -- 2.17.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] avformat/matroskadec: Check parents remaining length
+steve who submitted the original patch.
- dale
On Thu, Feb 21, 2019 at 2:30 PM Dale Curtis wrote:
> One of our test clips is behaving differently after this patch:
> https://cs.chromium.org/chromium/src/media/test/data/bear-320x240-live.webm
>
> The printed log message is:
> [matroska,webm @ 0x1516c84f4e00] Invalid length 0xff >
> 0x12f in parent
>
> Looking through the code I'm unsure which of the mixed usage
> "(uint64_t)-1" and "2**56-1" as marker values is correct. Changing the code
> to:
> diff --git a/libavformat/matroskadec.c b/libavformat/matroskadec.c
> index 9b706ab4e0..3015a0b230 100644
> --- a/libavformat/matroskadec.c
> +++ b/libavformat/matroskadec.c
> @@ -1205,7 +1205,7 @@ static int ebml_parse_elem(MatroskaDemuxContext
> *matroska,
> MatroskaLevel *level = >levels[matroska->num_levels
> - 1];
> AVIOContext *pb = matroska->ctx->pb;
> int64_t pos = avio_tell(pb);
> -if (level->length != (uint64_t) -1 &&
> +if (level->length != 0xffULL &&
> (pos + length) > (level->start + level->length)) {
> av_log(matroska->ctx, AV_LOG_ERROR,
> "Invalid length 0x%"PRIx64" > 0x%"PRIx64" in
> parent\n",
>
> Resolves our issues. Should all other length tests be done the same way?
>
> - dale
>
> On Sun, Feb 17, 2019 at 12:45 AM Michael Niedermayer
> wrote:
>
>> On Wed, Feb 13, 2019 at 01:41:31PM +0100, Michael Niedermayer wrote:
>> > Reported-by: Steve Lhomme
>> > This was found through the Hacker One program on VLC but is not a
>> security issue in libavformat
>> > Signed-off-by: Michael Niedermayer
>> > ---
>> > libavformat/matroskadec.c | 21 +
>> > 1 file changed, 21 insertions(+)
>>
>> will apply an equivalent variant from steve
>>
>> [...]
>> --
>> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>>
>> Asymptotically faster algorithms should always be preferred if you have
>> asymptotical amounts of data
>> ___
>> ffmpeg-devel mailing list
>> ffmpeg-devel@ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] ffmpeg_filter: switch sub2video.end_pts initialization value to 0
On Fri, Feb 22, 2019 at 9:56 PM Jan Ekström wrote:
>
> This seems to fix use cases where the sub2video output is the only
> input to a filter chain.
>
> Additionally, this is the value to which the structure's values
> would implicitly get initialized to. The ballooning buffering case
> would not get hit by this as the value of end_pts after
> (re-)initialization would be less than INT64_MAX, thus hitting the
> sub2video AVFrame initialization logic via sub2video_update in
> sub2video_heartbeat.
> ---
> fftools/ffmpeg_filter.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
> index 72838de1e2..536f9c9c46 100644
> --- a/fftools/ffmpeg_filter.c
> +++ b/fftools/ffmpeg_filter.c
> @@ -739,7 +739,7 @@ static int sub2video_prepare(InputStream *ist,
> InputFilter *ifilter)
> if (!ist->sub2video.frame)
> return AVERROR(ENOMEM);
> ist->sub2video.last_pts = INT64_MIN;
> -ist->sub2video.end_pts = INT64_MIN;
> +ist->sub2video.end_pts = 0;
> return 0;
> }
>
> --
> 2.20.1
>
An alternative for this would be to:
1. add a flag to sub2video structure that timestamp initialization is
required, and set it as required during (re-)initializations.
2. pass heartbeat pts to sub2video_update
3. finally, in sub2video_update
} else {
pts = !ist->sub2video.timestamp_initialized ?
heartbeat_pts : ist->sub2video.end_pts;
end_pts = INT64_MAX;
num_rects = 0;
}
and ist->sub2video.timestamp_initialized = 1; at the end.
This way the sub2video logic would start with the first heartbeat's
timestamp (often zero, but not always), as opposed to the initial
value of end_pts.
Jan
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] ffmpeg_filter: switch sub2video.end_pts initialization value to 0
This seems to fix use cases where the sub2video output is the only input to a filter chain. Additionally, this is the value to which the structure's values would implicitly get initialized to. The ballooning buffering case would not get hit by this as the value of end_pts after (re-)initialization would be less than INT64_MAX, thus hitting the sub2video AVFrame initialization logic via sub2video_update in sub2video_heartbeat. --- fftools/ffmpeg_filter.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 72838de1e2..536f9c9c46 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -739,7 +739,7 @@ static int sub2video_prepare(InputStream *ist, InputFilter *ifilter) if (!ist->sub2video.frame) return AVERROR(ENOMEM); ist->sub2video.last_pts = INT64_MIN; -ist->sub2video.end_pts = INT64_MIN; +ist->sub2video.end_pts = 0; return 0; } -- 2.20.1 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] doc: add missing hyphen prefix
On Fri, Feb 22, 2019, at 4:57 AM, Nicolas George wrote: > > That would be an argument for using something way more specific than a > dash. Like what? Do you have an example? > Anyway, that kind of marking would belong in the definition of the macro > @option, not in the body of the documentation. If this is possible it could be an acceptable solution. However, I haven't looked into it, and anticipating submitting even trivial patches to this mailing list usually generates negative motivation, so I don't know if I will ever look into it. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] ffmpeg_filter: initialize sub2video.end_pts together with last_pts
On Fri, Feb 22, 2019 at 3:11 PM Michael Niedermayer
wrote:
>
> On Thu, Feb 21, 2019 at 01:16:00PM +0200, Jan Ekström wrote:
> > This fixes buffering of samples which causes sudden ballooning of
> > memory usage in case of no subtitle samples coming in for a while if
> > the filter chain had been re-initialized.
> >
> > You can also see messages a la:
> > "Error while add the frame to buffer source(Invalid argument)."
> > disappearing after filter chain re-initializations.
> >
> > Passes fate-sub2video.
> >
> > Example (memory usage before patch around 700+ MiB, after around 150MiB) :
> > /usr/bin/time -v ffmpeg -v verbose \
> > -i
> > "https://megumin.fushizen.eu/samples/2019-01-18-audio_reconfig_causes_buffer_growth.ts;
> > \
> > -filter_complex
> > '[0:v:0]yadif=deint=interlaced[yadif_out];[yadif_out][0:s:0]overlay=eof_action=pass:repeatlast=0[overlay_out];[overlay_out]scale=1024:-2[video_out];[0:a:0]aresample=48000:async=1,aformat=channel_layouts=stereo[filtered_audio]'
> > \
> > -map "[video_out]" \
> > -c:v mpeg4 \
> > -b:v 750k \
> > -map "[filtered_audio]" \
> > -c:a aac \
> > -b:a 192k \
> > "test.mp4"
> >
> > Best regards,
> > Jan
>
> > ffmpeg_filter.c |1 +
> > 1 file changed, 1 insertion(+)
> > 5f88558fef759023173b4c8efe157aa30fc9e337
> > 0001-ffmpeg_filter-initialize-sub2video.end_pts-together-.patch
> > From 9c824c36c972aca19f2747437c8edc71b6c0886c Mon Sep 17 00:00:00 2001
> > From: =?UTF-8?q?Jan=20Ekstr=C3=B6m?=
> > Date: Thu, 20 Feb 2019 20:54:11 +0200
> > Subject: [PATCH] ffmpeg_filter: initialize sub2video.end_pts together with
> > last_pts
>
> Breaks: (video stream is empty after this) when a duration is used
> ./ffmpeg -i in.mkv -filter_complex '[0:s:1]scale=800:600' -t 15 -qscale 2
> test.avi
>
> both video and subtitles appear in the first 5 seconds
>
> input is a pgs in mkv file.
> If you cannot reproduce with a random file then say so and ill try to
> turn the file i have into a small testcase i can share
>
> thanks
>
I can replicate this.
Old behavior is kept if end_pts is initialized to its previous
implicit initialization value of zero.
I initialized this originally as INT64_MIN because that way it was
matching, and there were some comments about negative PTS values in
the commit that added the other value's initialization.
In my testing earlier I did find both values working for the use cases
I tested with, so I will send out a patch to switch around to zero
initialization for now, as the main part for not causing possibly
infinite buffering in lavfi is that when sub2video_heartbeat gets
called after a filter chain (re-)initialization sub2video.end_pts is
less than INT64_MAX.
This might be an issue with something expecting timestamps that start
from zero for some sort of "vsync", though, as the heartbeats are
nicely sent out and we are getting actual AVFrames pushed into the
filter chain. The major difference seems to be that after X frames
failed requests start appearing in the filter chain, and the current
sub2video AVFrame gets fed into the filter chain again. Additionally,
the ffmpeg.c vsync logic seems to not duplicate or drop frames when
starting from INT64_MIN.
My debugging av_log code for sub2video can be found at
https://github.com/jeeb/ffmpeg/commit/f1750f489be345f74525f7a2ad89e40c4d7e0493
.
Jan
Excerpt by git diff --no-index --word-diff:
diff --git a/pushing_log_before.log b/pushing_log_int64_min.log
index c163c36c37..c521380b37 100644
--- a/pushing_log_before.log
+++ b/pushing_log_int64_min.log
@@ -1,86 +1,74 @@
[mpegts @ [-0x2fe5b00]-]{+0x263eb00]+} sub2video: using 1920x1080 canvas
sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS -1
(last_pts: -9223372036854775808, end_pts:
[-0).-]{+-9223372036854775808).+}
sub2video: Update for stream 2, (pts: [-0,-]{+-9223372036854775808,+}
end_pts: 9223372036854775807, num_rects: 0)
sub2video: Pushing a 1920x1080 frame to 'graph 0 input from stream
0:2' with PTS [-0-]{+-9223372036854775808+}
sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS
449 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts:
9223372036854775807).
sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS
899 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts:
9223372036854775807).
sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS
1349 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts:
9223372036854775807).
sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS
1799 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts:
9223372036854775807).
sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS
2249 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts:
9223372036854775807).
sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS
2699 (last_pts: [-0,-]{+-9223372036854775808,+} end_pts:
9223372036854775807).
sub2video: Heartbeat from stream 1 (audio, PID: 4352) stream at PTS
3149 (last_pts:
Re: [FFmpeg-devel] [PATCH] doc: add missing hyphen prefix
On Fri, Feb 22, 2019, at 4:21 AM, Nicolas George wrote: > > There are other users than API and command-line. Think of a GUI that > presents all options as editable fields. That seems like an empty argument. Which GUI? Do you have an example? I don't see how some unnamed third-party GUI should have any basis on what we do here. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] doc: add missing hyphen prefix
On Fri, Feb 22, 2019, at 2:29 AM, Nicolas George wrote: > > I do not think this is correct: the dash is not part of the option name, > it is part of the Unix command-line tradition. For consistency, it > should be removed when it is there, possibly replaced by the word > "option" if necessary. I doubt anyone is ever going to do that, I certainly won't, but feel free to supply such a patch. However, I'm having trouble imagining what the result of your suggestion would look like. From this: -version Show version. -formats Show available formats (including devices). -demuxers Show available demuxers. ...to this? option version Show version. option formats Show available formats (including devices). option demuxers Show available demuxers. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 1/2] libavcodec/zmbvenc: block scoring improvements/bug fixes
fre 2019-02-22 klockan 13:09 +0100 skrev Tomas Härdin: > fre 2019-02-22 klockan 00:11 +0100 skrev Michael Niedermayer: > > On Thu, Feb 21, 2019 at 09:26:44PM +, Matthew Fearnley wrote: > > > - To clear up the effects: the change from 'i=1' to 'i=0' was the only > > > change that should make any difference in the output. > > > The rest of the changes were to improve the speed of an inner loop, and to > > > fix two bugs that happily cancelled each other out, but would have broken > > > if the code were changed to emit larger blocks. These should not lead to > > > changes in the blocks chosen or the final compression size. > > > > > > - Regarding the worse compression: let me start by stating that scoring on > > > byte frequency alone will not always predict well how Deflate will work, > > > since Deflate also uses pattern matching as well. > > > Frequency-based block scoring will only ever be a heuristic. There may be > > > scope for improving the scoring, but it would add a lot of complexity, and > > > I think it would be easy to make things worse rather than better. (I > > > found > > > this in my brief experiments with including the frequencies from the > > > previous block.) > > > > implementing the exact scoring by using Deflate itself would allow us to > > understand how much there is between the heuristic and what is achievable. > > If there is a gap that is significant then it may be interresting to > > look at improving the heuristic if the gap is small than the whole > > heuristic improvment path can be discarded. > > The obvious improvment for the heuristic would be to move to higher > > order entropy estimation. ATM IIUC this uses a 0th order. 1st order could > > be tried or any fast compression algorithm could also be used to estimate > > entropy. > > We tried a sliding window order-0 model, which did worse. An order-1 > model might be useful, but would need quite a bit of context for it to > be meaningful I did some looking into this because I was unsure, and it turns out evaluating the entropy of an order-1 model involves computing the eigenvector of the transfer matrix corresponding to the eigenvalue 1, and computing the scalar product between that and the order-0 entropy of each column of the transfer matrix. A pricey operation. With some luck computing the entropy change for a small perturbation (MV change) over the transfer matrix for the entire frame *might* be cheap enough to be useful. Maybe. The resulting matrix difference has a particular structure that might be exploitable.. /Tomas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] http: Do not try to make a new request when seeking past the end of the file
On Wed, Feb 20, 2019 at 9:54 AM Vittorio Giovara
wrote:
> From: Justin Ruggles
>
> This avoids making invalid HTTP Range requests for a byte range past the
> known end of the file during a seek. Those requests generally return a HTTP
> response of 416 Range Not Satisfiable, which causes an error response.
>
> Reference: https://tools.ietf.org/html/rfc7233
>
> Signed-off-by: Vittorio Giovara
> ---
> libavformat/http.c | 7 +++
> 1 file changed, 7 insertions(+)
>
> diff --git a/libavformat/http.c b/libavformat/http.c
> index a0a0636cf2..1e40268599 100644
> --- a/libavformat/http.c
> +++ b/libavformat/http.c
> @@ -1691,6 +1691,13 @@ static int64_t http_seek_internal(URLContext *h,
> int64_t off, int whence, int fo
> if (s->off && h->is_streamed)
> return AVERROR(ENOSYS);
>
> +/* do not try to make a new connection if seeking past the end of the
> file */
> +if (s->end_off || s->filesize != UINT64_MAX) {
> +uint64_t end_pos = s->end_off ? s->end_off : s->filesize;
> +if (s->off >= end_pos)
> +return s->off;
> +}
> +
> /* we save the old context in case the seek fails */
> old_buf_size = s->buf_end - s->buf_ptr;
> memcpy(old_buf, s->buf_ptr, old_buf_size);
> --
> 2.20.1
>
>
If no objections, I'd like to push this to master later today.
--
Vittorio
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] doc: add missing hyphen prefix
On 22.02.2019 14:57, Nicolas George wrote: Gyan (12019-02-22): '-key' will be easier to search for these users as well. It's also a low-cost arrangement. I trust adept API users will quickly suss out that the hyphen represents CLI. GUI users won't be entering the key string, only the values*, and casual CLI users will immediately recognize which entries pertain to options. That would be an argument for using something way more specific than a dash. Anyway, that kind of marking would belong in the definition of the macro @option, not in the body of the documentation. Yeah, that would be helpful. Prior art for adding an invisible prefix to HTML markup for exactly this use-case: https://duktape.org/api.html Regards, Tobias ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] doc: add missing hyphen prefix
Gyan (12019-02-22): > '-key' will be easier to search for these users as well. It's also a > low-cost arrangement. I trust adept API users will quickly suss out that the > hyphen represents CLI. GUI users won't be entering the key string, only the > values*, and casual CLI users will immediately recognize which entries > pertain to options. That would be an argument for using something way more specific than a dash. Anyway, that kind of marking would belong in the definition of the macro @option, not in the body of the documentation. Regards, -- Nicolas George ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] doc: add missing hyphen prefix
On 22-02-2019 06:50 PM, Nicolas George wrote: Tobias Rapp (12019-02-22): In my understanding the main source of documentation for API users are the Doxygen HTML output files while the Texinfo output is for both, command-line and API users -- with a slight bias towards command-line, at least from looking at all the ffmpeg/ffplay examples. There are other users than API and command-line. Think of a GUI that presents all options as editable fields. '-key' will be easier to search for these users as well. It's also a low-cost arrangement. I trust adept API users will quickly suss out that the hyphen represents CLI. GUI users won't be entering the key string, only the values*, and casual CLI users will immediately recognize which entries pertain to options. Gyan *if custom text is allowed, like in myffmpeg, that's in CLI form. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] doc: add missing hyphen prefix
Tobias Rapp (12019-02-22): > In my understanding the main source of documentation for API users are the > Doxygen HTML output files while the Texinfo output is for both, command-line > and API users -- with a slight bias towards command-line, at least from > looking at all the ffmpeg/ffplay examples. There are other users than API and command-line. Think of a GUI that presents all options as editable fields. Regards, -- Nicolas George ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] doc: add missing hyphen prefix
On 22.02.2019 12:43, Hendrik Leppkes wrote: On Fri, Feb 22, 2019 at 12:29 PM Nicolas George wrote: Lou Logan (12019-02-21): For consistency. Fixes #7740. Signed-off-by: Lou Logan I do not think this is correct: the dash is not part of the option name, it is part of the Unix command-line tradition. For consistency, it should be removed when it is there, possibly replaced by the word "option" if necessary. I agree. You don't pass the dash when yo use avoptions through API, for example, so this would only add to the confusion. In my understanding the main source of documentation for API users are the Doxygen HTML output files while the Texinfo output is for both, command-line and API users -- with a slight bias towards command-line, at least from looking at all the ffmpeg/ffplay examples. The current dash prefixes allow me to jump to specific options more easily (Ctrl+F "-b" for example). I agree that this is not a strong argument, but wanted to mention it anyway :-) Regards, Tobias ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] ffmpeg_filter: initialize sub2video.end_pts together with last_pts
On Thu, Feb 21, 2019 at 01:16:00PM +0200, Jan Ekström wrote: > This fixes buffering of samples which causes sudden ballooning of > memory usage in case of no subtitle samples coming in for a while if > the filter chain had been re-initialized. > > You can also see messages a la: > "Error while add the frame to buffer source(Invalid argument)." > disappearing after filter chain re-initializations. > > Passes fate-sub2video. > > Example (memory usage before patch around 700+ MiB, after around 150MiB) : > /usr/bin/time -v ffmpeg -v verbose \ > -i > "https://megumin.fushizen.eu/samples/2019-01-18-audio_reconfig_causes_buffer_growth.ts; > \ > -filter_complex > '[0:v:0]yadif=deint=interlaced[yadif_out];[yadif_out][0:s:0]overlay=eof_action=pass:repeatlast=0[overlay_out];[overlay_out]scale=1024:-2[video_out];[0:a:0]aresample=48000:async=1,aformat=channel_layouts=stereo[filtered_audio]' > \ > -map "[video_out]" \ > -c:v mpeg4 \ > -b:v 750k \ > -map "[filtered_audio]" \ > -c:a aac \ > -b:a 192k \ > "test.mp4" > > Best regards, > Jan > ffmpeg_filter.c |1 + > 1 file changed, 1 insertion(+) > 5f88558fef759023173b4c8efe157aa30fc9e337 > 0001-ffmpeg_filter-initialize-sub2video.end_pts-together-.patch > From 9c824c36c972aca19f2747437c8edc71b6c0886c Mon Sep 17 00:00:00 2001 > From: =?UTF-8?q?Jan=20Ekstr=C3=B6m?= > Date: Thu, 20 Feb 2019 20:54:11 +0200 > Subject: [PATCH] ffmpeg_filter: initialize sub2video.end_pts together with > last_pts Breaks: (video stream is empty after this) when a duration is used ./ffmpeg -i in.mkv -filter_complex '[0:s:1]scale=800:600' -t 15 -qscale 2 test.avi both video and subtitles appear in the first 5 seconds input is a pgs in mkv file. If you cannot reproduce with a random file then say so and ill try to turn the file i have into a small testcase i can share thanks [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB I have never wished to cater to the crowd; for what I know they do not approve, and what they approve I do not know. -- Epicurus signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 1/2] avcodec/zmbv: obtain frame later
tor 2019-02-21 klockan 20:34 +0100 skrev Michael Niedermayer: > The frame is not needed that early so obtaining it later avoids > the costly operation in case other checks fail. > > Fixes: Timeout (14sec -> 4sec) > Fixes: > 13140/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ZMBV_fuzzer-5738330308739072 > > Found-by: continuous fuzzing process > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg > > Signed-off-by: Michael Niedermayer > --- > libavcodec/zmbv.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) Looks OK, passes FATE /Tomas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 1/2] libavcodec/zmbvenc: block scoring improvements/bug fixes
fre 2019-02-22 klockan 00:11 +0100 skrev Michael Niedermayer: > On Thu, Feb 21, 2019 at 09:26:44PM +, Matthew Fearnley wrote: > > - To clear up the effects: the change from 'i=1' to 'i=0' was the only > > change that should make any difference in the output. > > The rest of the changes were to improve the speed of an inner loop, and to > > fix two bugs that happily cancelled each other out, but would have broken > > if the code were changed to emit larger blocks. These should not lead to > > changes in the blocks chosen or the final compression size. > > > > - Regarding the worse compression: let me start by stating that scoring on > > byte frequency alone will not always predict well how Deflate will work, > > since Deflate also uses pattern matching as well. > > Frequency-based block scoring will only ever be a heuristic. There may be > > scope for improving the scoring, but it would add a lot of complexity, and > > I think it would be easy to make things worse rather than better. (I found > > this in my brief experiments with including the frequencies from the > > previous block.) > > implementing the exact scoring by using Deflate itself would allow us to > understand how much there is between the heuristic and what is achievable. > If there is a gap that is significant then it may be interresting to > look at improving the heuristic if the gap is small than the whole > heuristic improvment path can be discarded. > The obvious improvment for the heuristic would be to move to higher > order entropy estimation. ATM IIUC this uses a 0th order. 1st order could > be tried or any fast compression algorithm could also be used to estimate > entropy. We tried a sliding window order-0 model, which did worse. An order-1 model might be useful, but would need quite a bit of context for it to be meaningful > It also would be interresting to see how a higher order heuristic or > some simple fast LZ like compressor would perform with the previous block. > That is if the issue you saw with the previous block inclusion was because > of the very simplistic entropy estimation. zlib at level 1 maybe? You could also have a multi-pass thing where it first does a coarse pass then tries to jiggle MVs and greedily accept any change that results in smaller output. /Tomas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 1/2] libavcodec/zmbvenc: block scoring improvements/bug fixes
tor 2019-02-21 klockan 15:26 +0100 skrev Carl Eugen Hoyos: > > 2019-02-21 11:26 GMT+01:00, Tomas Härdin : > > ons 2019-02-20 klockan 23:33 +0100 skrev Carl Eugen Hoyos: > > > > > > > > 2019-02-10 16:42 GMT+01:00, Tomas Härdin : > > > > lör 2019-02-09 klockan 13:10 + skrev Matthew Fearnley: > > > > > - Improve block choices by counting 0-bytes in the entropy score > > > > > - Make histogram use uint16_t type, to allow byte counts from 16*16 > > > > > (current block size) up to 255*255 (maximum allowed 8bpp block size) > > > > > - Make sure score table is big enough for a full block's worth of > > > > > bytes > > > > > - Calculate *xored without using code in inner loop > > > > > --- > > > > > libavcodec/zmbvenc.c | 22 -- > > > > > 1 file changed, 16 insertions(+), 6 deletions(-) > > > > > > > > Passes FATE, looks good to me > > > > > > I believe you asked on irc about fate tests: > > > Since such a test would depend on the zlib version, you cannot test > > > exact output, you could only test a round-trip (assuming the codec > > > really is lossless). > > > > Right, we'd need to embed a specific version of zlib. But we probably > > don't want to do that, for many reasons. > > > > A dummy DEFLATE implementation that just passes the bytes along could > > be useful. I know there's a mode in the DEFLATE format for blocks that > > fail to compress. That way we could test many encoders that depend on > > zlib, and their output should decode just fine. > > What's wrong with a round-trip test? Nothing I suppose. But it might be nice to catch unintended changes to the encoder's output /Tomas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] doc: add missing hyphen prefix
On Fri, Feb 22, 2019 at 12:29 PM Nicolas George wrote: > > Lou Logan (12019-02-21): > > For consistency. Fixes #7740. > > > > Signed-off-by: Lou Logan > > I do not think this is correct: the dash is not part of the option name, > it is part of the Unix command-line tradition. For consistency, it > should be removed when it is there, possibly replaced by the word > "option" if necessary. > I agree. You don't pass the dash when yo use avoptions through API, for example, so this would only add to the confusion. - Hendrik ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] doc: add missing hyphen prefix
Lou Logan (12019-02-21): > For consistency. Fixes #7740. > > Signed-off-by: Lou Logan I do not think this is correct: the dash is not part of the option name, it is part of the Unix command-line tradition. For consistency, it should be removed when it is there, possibly replaced by the word "option" if necessary. Regards, -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] doc: add missing hyphen prefix
On 22-02-2019 06:12 AM, Lou Logan wrote: For consistency. Fixes #7740. Signed-off-by: Lou Logan --- Does this need to be checked for errant added hypens, like before entries in a value table? If not, LGTM. Gyan ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH] avformat/mov: fix hang while seek on a kind of fragmented mp4.
I have updated a new patch that has passed valgrind as follows: *$ ./configure --samples=fate-suite/ --disable-doc --fatal-warnings --valgrind=VALGRIND* *$ make -j4 && make fate-mov* But I never encountered a ‘stream 0, timescale not set’ log in here. Please let me know if you have any questions. Sorry for the previous trouble. Thanks and Regards, Charles Liu Michael Niedermayer 于2019年2月22日周五 上午9:40写道： > On Wed, Feb 20, 2019 at 11:54:56PM +0800, Charles Liu wrote: > > 1. organize fragmented information according to the tracks. > > 2. do NOT skip the last boxes of fragmented info. > > > > ticket #7572 > > > > Signed-off-by: Charles Liu > > --- > > libavformat/isom.h | 10 +- > > libavformat/mov.c | 375 ++--- > > 2 files changed, 185 insertions(+), 200 deletions(-) > > this causes OOM and crashes > > [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3796f00] stream 0, timescale not set > [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3796f00] stream 1, timescale not set > [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3796f00] error reading header > *** Error in `./ffmpeg': double free or corruption (fasttop): > 0x0379b100 *** > Aborted (core dumped) > > > [...] > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > No snowflake in an avalanche ever feels responsible. -- Voltaire > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
[FFmpeg-devel] [PATCH] avformat/mov: fix hang while seek on a kind of fragmented mp4.
1. organize fragmented information according to the tracks.
2. do NOT skip the last boxes of fragmented info.
ticket #7572
Signed-off-by: Charles Liu
---
libavformat/isom.h | 10 +-
libavformat/mov.c | 374 +
2 files changed, 181 insertions(+), 203 deletions(-)
diff --git a/libavformat/isom.h b/libavformat/isom.h
index 69452cae8e..eea8fa4e8f 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -125,7 +125,7 @@ typedef struct MOVEncryptionIndex {
} MOVEncryptionIndex;
typedef struct MOVFragmentStreamInfo {
-int id;
+unsigned stsd_id;
int64_t sidx_pts;
int64_t first_tfra_pts;
int64_t tfdt_dts;
@@ -136,14 +136,13 @@ typedef struct MOVFragmentStreamInfo {
typedef struct MOVFragmentIndexItem {
int64_t moof_offset;
int headers_read;
-int current;
-int nb_stream_info;
-MOVFragmentStreamInfo * stream_info;
+MOVFragmentStreamInfo stream_info;
} MOVFragmentIndexItem;
typedef struct MOVFragmentIndex {
int allocated_size;
int complete;
+int id; // track id
int current;
int nb_items;
MOVFragmentIndexItem * item;
@@ -274,7 +273,8 @@ typedef struct MOVContext {
int moov_retry;
int use_mfra_for;
int has_looked_for_mfra;
-MOVFragmentIndex frag_index;
+unsigned nb_frag_indices;
+MOVFragmentIndex *frag_indices;
int atom_depth;
unsigned int aax_mode; ///< 'aax' file has been detected
uint8_t file_key[20];
diff --git a/libavformat/mov.c b/libavformat/mov.c
index bbd588c705..a16788fd21 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -1153,59 +1153,29 @@ static int mov_read_moov(MOVContext *c, AVIOContext
*pb, MOVAtom atom)
return 0; /* now go for mdat */
}
-static MOVFragmentStreamInfo * get_frag_stream_info(
-MOVFragmentIndex *frag_index,
-int index,
-int id)
+static MOVFragmentIndex *mov_find_frag_index(MOVFragmentIndex *frag_indices,
int nb_frag_indices, int track_id)
{
-int i;
-MOVFragmentIndexItem * item;
+unsigned i;
+MOVFragmentIndex *frag_index = NULL;
-if (index < 0 || index >= frag_index->nb_items)
-return NULL;
-item = _index->item[index];
-for (i = 0; i < item->nb_stream_info; i++)
-if (item->stream_info[i].id == id)
-return >stream_info[i];
+for (i = 0; i < nb_frag_indices; i++)
+if (frag_indices[i].id == track_id)
+frag_index = _indices[i];
-// This shouldn't happen
-return NULL;
+return frag_index;
}
-static void set_frag_stream(MOVFragmentIndex *frag_index, int id)
+static MOVFragmentStreamInfo *get_current_frag_stream_info(MOVContext *c, int
id)
{
-int i;
-MOVFragmentIndexItem * item;
-
-if (frag_index->current < 0 ||
-frag_index->current >= frag_index->nb_items)
-return;
-
-item = _index->item[frag_index->current];
-for (i = 0; i < item->nb_stream_info; i++)
-if (item->stream_info[i].id == id) {
-item->current = i;
-return;
-}
+MOVFragmentIndex *frag_index = NULL;
-// id not found. This shouldn't happen.
-item->current = -1;
-}
-
-static MOVFragmentStreamInfo * get_current_frag_stream_info(
-MOVFragmentIndex *frag_index)
-{
-MOVFragmentIndexItem *item;
-if (frag_index->current < 0 ||
+frag_index = mov_find_frag_index(c->frag_indices, c->nb_frag_indices, id);
+if (!frag_index ||
+frag_index->current < 0 ||
frag_index->current >= frag_index->nb_items)
return NULL;
-item = _index->item[frag_index->current];
-if (item->current >= 0 && item->current < item->nb_stream_info)
-return >stream_info[item->current];
-
-// This shouldn't happen
-return NULL;
+return _index->item[frag_index->current].stream_info;
}
static int search_frag_moof_offset(MOVFragmentIndex *frag_index, int64_t
offset)
@@ -1232,9 +1202,9 @@ static int search_frag_moof_offset(MOVFragmentIndex
*frag_index, int64_t offset)
return b;
}
-static int64_t get_stream_info_time(MOVFragmentStreamInfo * frag_stream_info)
-{
-av_assert0(frag_stream_info);
+static int64_t get_frag_time(MOVFragmentIndex *frag_index,
+ int index, int track_id) {
+MOVFragmentStreamInfo *frag_stream_info =
_index->item[index].stream_info;
if (frag_stream_info->sidx_pts != AV_NOPTS_VALUE)
return frag_stream_info->sidx_pts;
if (frag_stream_info->first_tfra_pts != AV_NOPTS_VALUE)
@@ -1242,31 +1212,9 @@ static int64_t
get_stream_info_time(MOVFragmentStreamInfo * frag_stream_info)
return frag_stream_info->tfdt_dts;
}
-static int64_t get_frag_time(MOVFragmentIndex *frag_index,
- int index, int track_id)
-{
-MOVFragmentStreamInfo * frag_stream_info;
-int64_t timestamp;
-int i;
-
-if (track_id >= 0) {
-frag_stream_info = get_frag_stream_info(frag_index, index, track_id);
-
