Bug#870676: ffmpeg requires NEON on armhf, which is not part of the ARMv7 ABI
Hi, On 09/08/17 04:59, Edmund Grimley Evans wrote: > James: >> I think the runtime is working, but this is the testcase from the SIGBUS >> bug which you can use: >> >> ffmpeg -f lavfi -i testsrc=s=32x32:d=0.1 -strict -2 -c:v libx264rgb -f avi >> libx264rgb.avi -y -hide_banner -nostdin >> ffmpeg -strict -2 -i libx264rgb.avi -t 1 -c:v rawvideo -c:a pcm_s32le -f nut >> /dev/null -y -hide_banner -nostdin >> >> Since the SIGBUS bug occurs in NEON code, if the runtime detection is >> working, this will _only_ fail on machines with NEON and work on >> non-NEON machines. > > This bug was closed while I was sleeping, but I will mention anyway > that with 7:3.3.3-1 those commands give a SIGBUS, as expected, but > only if /proc is mounted. > > Presumably with the fixed version, 7:3.3.3-2, performance will be > measurably worse on a system with NEON when /proc is not mounted. > > Upstream should probably switch to using getauxval(). Do you want to > suggest it to them? I've filed it upstream here: https://trac.ffmpeg.org/ticket/6578 James signature.asc Description: OpenPGP digital signature ___ pkg-multimedia-maintainers mailing list pkg-multimedia-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-multimedia-maintainers
Bug#870676: ffmpeg requires NEON on armhf, which is not part of the ARMv7 ABI
James: > I think the runtime is working, but this is the testcase from the SIGBUS > bug which you can use: > > ffmpeg -f lavfi -i testsrc=s=32x32:d=0.1 -strict -2 -c:v libx264rgb -f avi > libx264rgb.avi -y -hide_banner -nostdin > ffmpeg -strict -2 -i libx264rgb.avi -t 1 -c:v rawvideo -c:a pcm_s32le -f nut > /dev/null -y -hide_banner -nostdin > > Since the SIGBUS bug occurs in NEON code, if the runtime detection is > working, this will _only_ fail on machines with NEON and work on > non-NEON machines. This bug was closed while I was sleeping, but I will mention anyway that with 7:3.3.3-1 those commands give a SIGBUS, as expected, but only if /proc is mounted. Presumably with the fixed version, 7:3.3.3-2, performance will be measurably worse on a system with NEON when /proc is not mounted. Upstream should probably switch to using getauxval(). Do you want to suggest it to them? ___ pkg-multimedia-maintainers mailing list pkg-multimedia-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-multimedia-maintainers
Bug#870676: ffmpeg requires NEON on armhf, which is not part of the ARMv7 ABI
Hi, On 06/08/17 04:01, Edmund Grimley Evans wrote: > As far as I know, the best way to implement run-time detection of > hardware capabilities is with getauxval(AT_HWCAP) and > getauxval(AT_HWCAP2). > > There is some kind of NEON detection in ffmeg. See, for example: > > https://sources.debian.net/src/ffmpeg/7:3.3.3-1/libavutil/arm/cpu.c/ > > That code appears to use /proc rather than getauxval. Is there a good > reason for that? I guess you would have to ask upstream to get an answer to this question. > In case someone reading this has access to hardware (or a simulator) > without NEON but is not familiar with ffmpeg, what is a quick and easy > way of checking whether ffmpeg is working? I think the runtime is working, but this is the testcase from the SIGBUS bug which you can use: ffmpeg -f lavfi -i testsrc=s=32x32:d=0.1 -strict -2 -c:v libx264rgb -f avi libx264rgb.avi -y -hide_banner -nostdin ffmpeg -strict -2 -i libx264rgb.avi -t 1 -c:v rawvideo -c:a pcm_s32le -f nut /dev/null -y -hide_banner -nostdin Since the SIGBUS bug occurs in NEON code, if the runtime detection is working, this will _only_ fail on machines with NEON and work on non-NEON machines. Thanks, James signature.asc Description: OpenPGP digital signature ___ pkg-multimedia-maintainers mailing list pkg-multimedia-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-multimedia-maintainers
Bug#870676: ffmpeg requires NEON on armhf, which is not part of the ARMv7 ABI
As far as I know, the best way to implement run-time detection of hardware capabilities is with getauxval(AT_HWCAP) and getauxval(AT_HWCAP2). There is some kind of NEON detection in ffmeg. See, for example: https://sources.debian.net/src/ffmpeg/7:3.3.3-1/libavutil/arm/cpu.c/ That code appears to use /proc rather than getauxval. Is there a good reason for that? In case someone reading this has access to hardware (or a simulator) without NEON but is not familiar with ffmpeg, what is a quick and easy way of checking whether ffmpeg is working? There are other libraries in Debian that do run-time detection of NEON: fftw3 is one, I think. ___ pkg-multimedia-maintainers mailing list pkg-multimedia-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-multimedia-maintainers
Bug#870676: ffmpeg requires NEON on armhf, which is not part of the ARMv7 ABI
Control: forcemerge 870622 -1 Hi James, On Sat, Aug 05, 2017 at 02:40:12PM -0400, James Cowgill wrote: > > The latest release of ffmpeg enables NEON support by default when building > > on armhf; however, NEON support is not a standard part of the ARMv7 ABI, and > > Debian supports running armhf on chips that do not implement NEON. > > Using NEON based on runtime detection of support for it is fine, but the > > existing ffmpeg implementation doesn't appear to do this, instead using NEON > > based on build-time configuration with no fallback. > Are you sure this is true? I tried running the failing test on abel.d.o > (which AFAIK does not have NEON) and harris (which does). The test only > caused ffmpeg to crash on harris, which seems to suggest that the > runtime NEON detection is working properly. Nope, not sure at all, I only know what I saw from code inspection where I failed to find where the code falls back on non-NEON systems. I see you're right that abel does not have NEON (it's not exposed in /proc/cpuinfo feature flags for the CPU), so if ffmpeg runs without crashing there, I guess that's pretty clear evidence that there is a runtime fallback that I overlooked. > These are the commands to reproduce the autopkgtest fail if you want to > try it: > ffmpeg -f lavfi -i testsrc=s=32x32:d=0.1 -strict -2 -c:v libx264rgb -f avi > libx264rgb.avi -y -hide_banner -nostdin > ffmpeg -strict -2 -i libx264rgb.avi -t 1 -c:v rawvideo -c:a pcm_s32le -f nut > /dev/null -y -hide_banner -nostdin > > This issue was noticed in Ubuntu only because the autopkgtests for ffmpeg > > and x264 triggered an unaligned access in the NEON code, which is *also* not > > a portable assumption on armhf; however, if the NEON code had not had any > > unaligned access, the fact that NEON was used would have gone unnoticed on > > Ubuntu infrastructure. > > > > http://autopkgtest.ubuntu.com/packages/f/ffmpeg/artful/armhf > > http://autopkgtest.ubuntu.com/packages/x/x264/artful/armhf > > > > (And if upstream does fix their code to support runtime detection of NEON > > support, then there will be a different bug for us to worry about fixing!) > > This is #870622 BTW. If possible, I would much rather fix these bugs > without having to disable all the NEON optimizations. So, marking this bug as a duplicate of the real bug (the unaligned trap problem due to binutils). Thanks, -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature ___ pkg-multimedia-maintainers mailing list pkg-multimedia-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-multimedia-maintainers
Processed (with 1 error): Re: Bug#870676: ffmpeg requires NEON on armhf, which is not part of the ARMv7 ABI
Processing control commands: > forcemerge 870622 -1 Bug #870622 [src:ffmpeg] ffmpeg: autopkgtest SIGBUS on armhf with binutils 2.29 Unable to merge bugs because: package of #870676 is 'ffmpeg' not 'src:ffmpeg' Failed to forcibly merge 870622: Did not alter merged bugs. -- 870622: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=870622 870676: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=870676 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems ___ pkg-multimedia-maintainers mailing list pkg-multimedia-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-multimedia-maintainers
Bug#870676: ffmpeg requires NEON on armhf, which is not part of the ARMv7 ABI
Hi Steve, On 03/08/17 21:03, Steve Langasek wrote: > Package: ffmpeg > Version: 7:3.3.3-1 > Severity: important > Tags: patch > User: ubuntu-de...@lists.ubuntu.com > Usertags: origin-ubuntu artful ubuntu-patch autopkgtest > > Dear maintainers, > > The latest release of ffmpeg enables NEON support by default when building > on armhf; however, NEON support is not a standard part of the ARMv7 ABI, and > Debian supports running armhf on chips that do not implement NEON. > > Using NEON based on runtime detection of support for it is fine, but the > existing ffmpeg implementation doesn't appear to do this, instead using NEON > based on build-time configuration with no fallback. Are you sure this is true? I tried running the failing test on abel.d.o (which AFAIK does not have NEON) and harris (which does). The test only caused ffmpeg to crash on harris, which seems to suggest that the runtime NEON detection is working properly. These are the commands to reproduce the autopkgtest fail if you want to try it: ffmpeg -f lavfi -i testsrc=s=32x32:d=0.1 -strict -2 -c:v libx264rgb -f avi libx264rgb.avi -y -hide_banner -nostdin ffmpeg -strict -2 -i libx264rgb.avi -t 1 -c:v rawvideo -c:a pcm_s32le -f nut /dev/null -y -hide_banner -nostdin > This issue was noticed in Ubuntu only because the autopkgtests for ffmpeg > and x264 triggered an unaligned access in the NEON code, which is *also* not > a portable assumption on armhf; however, if the NEON code had not had any > unaligned access, the fact that NEON was used would have gone unnoticed on > Ubuntu infrastructure. > > http://autopkgtest.ubuntu.com/packages/f/ffmpeg/artful/armhf > http://autopkgtest.ubuntu.com/packages/x/x264/artful/armhf > > (And if upstream does fix their code to support runtime detection of NEON > support, then there will be a different bug for us to worry about fixing!) This is #870622 BTW. If possible, I would much rather fix these bugs without having to disable all the NEON optimizations. Thanks, James signature.asc Description: OpenPGP digital signature ___ pkg-multimedia-maintainers mailing list pkg-multimedia-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-multimedia-maintainers
Bug#870676: ffmpeg requires NEON on armhf, which is not part of the ARMv7 ABI
Package: ffmpeg Version: 7:3.3.3-1 Severity: important Tags: patch User: ubuntu-de...@lists.ubuntu.com Usertags: origin-ubuntu artful ubuntu-patch autopkgtest Dear maintainers, The latest release of ffmpeg enables NEON support by default when building on armhf; however, NEON support is not a standard part of the ARMv7 ABI, and Debian supports running armhf on chips that do not implement NEON. Using NEON based on runtime detection of support for it is fine, but the existing ffmpeg implementation doesn't appear to do this, instead using NEON based on build-time configuration with no fallback. This issue was noticed in Ubuntu only because the autopkgtests for ffmpeg and x264 triggered an unaligned access in the NEON code, which is *also* not a portable assumption on armhf; however, if the NEON code had not had any unaligned access, the fact that NEON was used would have gone unnoticed on Ubuntu infrastructure. http://autopkgtest.ubuntu.com/packages/f/ffmpeg/artful/armhf http://autopkgtest.ubuntu.com/packages/x/x264/artful/armhf (And if upstream does fix their code to support runtime detection of NEON support, then there will be a different bug for us to worry about fixing!) Cheers, -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org diff -Nru ffmpeg-3.3.3/debian/rules ffmpeg-3.3.3/debian/rules --- ffmpeg-3.3.3/debian/rules 2017-08-01 07:33:41.0 -0700 +++ ffmpeg-3.3.3/debian/rules 2017-08-03 17:43:44.0 -0700 @@ -127,6 +127,14 @@ --enable-libiec61883 endif +# upstream does not implement runtime detection of NEON support at runtime; +# therefore NEON must be disabled at build time. (This also works around +# the fact that the NEON implementation does unaligned access, which is not +# a portable assumption for armhf.) +ifeq ($(DEB_HOST_ARCH),armhf) + CONFIG += --disable-neon +endif + # Some build-dependencies are not installable on some architectures. ifeq (,$(filter $(DEB_HOST_ARCH),powerpcspe)) CONFIG_extra += --enable-netcdf ___ pkg-multimedia-maintainers mailing list pkg-multimedia-maintainers@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-multimedia-maintainers
