#2675: postproc crashes with -fstack-protector-all
------------------------------------+------------------------------------
Reporter: cehoyos | Owner: michael
Type: defect | Status: new
Priority: normal | Component: postproc
Version: git-master | Resolution:
Keywords: crash | Blocked By:
Blocking: | Reproduced by developer: 0
Analyzed by developer: 0 |
------------------------------------+------------------------------------
Comment (by cehoyos):
The reason is apparently that gcc miscompiles dering_SSE2() and
dering_MMX2() in libpostproc/postproc_template.c:
{{{
1182 "movq %%mm6, %%mm0 \n\t" // max
1183 "psubb %%mm7, %%mm6 \n\t" // max - min
1184 "push %4 \n\t"
1184 "movd %%mm6, %k4 \n\t"
1185 "cmpb "MANGLE(deringThreshold)", %b4 \n\t"
1186 "pop %4 \n\t"
...
: : "r" (src), "r" ((x86_reg)stride), "m" (c->pQPb),
"m"(c->pQPb2), "q"(tmp)
: "%"REG_a, "%"REG_d
}}}
{{{
0x0000000000bd40eb <dering_SSE2+187>: movq %mm6,%mm0
0x0000000000bd40ee <dering_SSE2+190>: psubb %mm7,%mm6
0x0000000000bd40f1 <dering_SSE2+193>: push %rsp
0x0000000000bd40f2 <dering_SSE2+194>: movd %mm6,%esp
0x0000000000bd40f5 <dering_SSE2+197>: cmp 0xe3c6c0,%spl
=> 0x0000000000bd40fd <dering_SSE2+205>: pop %rsp
}}}
--
Ticket URL: <https://ffmpeg.org/trac/ffmpeg/ticket/2675#comment:1>
FFmpeg <http://ffmpeg.org>
FFmpeg issue tracker
_______________________________________________
FFmpeg-trac mailing list
FFmpeg-trac@avcodec.org
http://avcodec.org/mailman/listinfo/ffmpeg-trac
