Re: [Rpm-maint] Multiple ABI architectures (parallel installation, detection, handling)
On 11/03/2011 09:34 PM, Jon Masters wrote: Panu Matilainenpmati...@laiskiainen.org wrote: On 10/21/2011 10:20 PM, Dennis Gilmore wrote: long term maybe we can do something better. right now im looking for a way to enable yum to detect the arches correctly without it resorting to things like running readelf to determine the abi of the running system. since its noarch we cant just patch things in conditionally. BTW I was under the impression the ABI information for hard vs softfp simply DOES NOT EXIST in the ELF headers (or anywhere else for that matter). If http://wiki.debian.org/ArmHardFloatPort/VfpComparison#ld.so_hwcaps is wrong (or I'm misreading it) and you know of a way to detect it afterall, do let me know. - Panu - ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint It is in an optional section we do carry - see readelf -A. At an ARM conference so will discuss with Steve McIntyre of ARM (who is driving this ABI on their end). Okay, in that case there's hope. Looking at what I hope to be a suitable sample specimen: Attribute Section: aeabi File Attributes Tag_CPU_name: 7-A Tag_CPU_arch: v7 Tag_CPU_arch_profile: Application Tag_ARM_ISA_use: Yes Tag_THUMB_ISA_use: Thumb-2 Tag_FP_arch: VFPv3-D16 Tag_ABI_PCS_wchar_t: 4 Tag_ABI_FP_denormal: Needed Tag_ABI_FP_exceptions: Needed Tag_ABI_FP_number_model: IEEE 754 Tag_ABI_align_needed: 8-byte Tag_ABI_align_preserved: 8-byte, except leaf SP Tag_ABI_enum_size: int Tag_ABI_HardFP_use: SP and DP Tag_ABI_VFP_args: VFP registers Tag_DIV_use: Not allowed ...the interesting part would be Tag_ABI_VFP_args, or...? Would that happen to be available at runtime through the auxiliary vector (hwcap or wherever)? At buildtime, one (easy, since this IS recorded in the ELF headers) possibility is making elfdeps generate different looking dependencies when the VFP flag is present, eg something like libfoo.1.so vs libfoo.1.so()(vfp), so packages for the other ABI wont satisfy dependencies of the other one, despite both being technically compatible architecture-wise (on a system capable of using VFP ABI) - Panu - ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] Multiple ABI architectures (parallel installation, detection, handling)
On 10/27/2011 12:00 AM, Dennis Gilmore wrote: [...big snip...] its not sufficient. it evaluates to armv7l on a hardware floating point system. we would need it to be armv7hl or armv7hnl to give yum the info it needs maybe _target_cpu could be used though it would need a dict of compatiable arches [dennis@panda02 ~]$ rpm --showrc |grep arm build arch: armv7hnl compatible build archs: armv7hnl armv7hl noarch install arch : armv7hnl compatible archs : armv7hnl armv7hl noarch optflags : -O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=neon -mthumb gpg --batch --no-verbose --no-armor --passphrase-fd 3 - -14: __isa_name armv7hnl - -14: _archarm - -14: _build_arch arm - -14: _hostarmv7l-unknown-linux-gnueabi - -14: _host_cpuarmv7l - -11: _target armv7hnl-linux - -11= _target_cpu armv7hnl - -14: arm armv3l armv4b armv4l armv4tl armv5tel armv5tejl armv6l armv7l armv7hl armv7nhl - -11: optflags -O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=neon -mthumb [root@trimslice01 ~]# rpm --showrc |grep arm build arch: armv7hl compatible build archs: armv7hl noarch install arch : armv7hl compatible archs : armv7hl noarch optflags : -O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -mthumb gpg --batch --no-verbose --no-armor --passphrase-fd 3 - -14: __isa_name armv7hl - -14: _archarm - -14: _build_arch arm - -14: _hostarmv7l-unknown-linux-gnueabi - -14: _host_cpuarmv7l - -11: _target armv7hl-linux - -11= _target_cpu armv7hl - -14: arm armv3l armv4b armv4l armv4tl armv5tel armv5tejl armv6l armv7l armv7hl armv7nhl - -11: optflags -O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -mthumb the first output is a system that has a neon simd unit and the second is without it. any software that can run on the second can also run on the first. __isa_name really should be the same on both. long term maybe we can do something better. right now im looking for a way to enable yum to detect the arches correctly without it resorting to things like running readelf to determine the abi of the running system. since its noarch we cant just patch things in conditionally. Dennis ok since i have a ton of the oddball hardware here is there someway to get whats setting compatible archs in --showrc? i ask because rpm --showrc |grep armv7 build arch: armv7hnl compatible build archs: armv7hnl armv7hl noarch install arch : armv7hnl compatible archs : armv7hnl armv7hl noarch optflags : -O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=neon -mthumb -14: __isa_namearmv7hnl - -14: _hostarmv7l-unknown-linux-gnueabi - -14: _host_cpuarmv7l - -11: _target armv7hnl-linux - -11= _target_cpu armv7hnl - -14: arm armv3l armv4b armv4l armv4tl armv5tel armv5tejl armv6l armv7l armv7hl armv7nhl -11: optflags -O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=neon -mthumb rpm --showrc |grep sparc build arch: sparc64 compatible build archs: sparc64v sparc64 sparcv9v sparcv9 sparcv8 sparc noarch install arch : sparc64v compatible archs : sparc64v sparc64 sparcv9v sparcv9 sparcv8 sparc noarch -14: _arch sparc - -14: _build_arch sparc - -14: _hostsparc-unknown-linux-gnu - -14: _host_cpusparc - -11: _target sparc64-linux - -11= _target_cpu sparc64 - -11: optflags %{__global_cflags} -m64 -mcpu=ultrasparc - -14: sparcsparc sparcv8 sparcv9 sparcv9v sparc64 sparc64v _host_cpu and _target_cpu on sparc doesnt give me a way to work out that im on a niagara box but in both cases i could work out whats needed by the info thats in compatible archs looking at rpmrc.c RPM_MACHTABLE_INSTARCH is whats used internal. is that available at all? on a niagara box In [2]: rpm.expandMacro('%{_target_cpu}') Out[2]: 'sparc64' on a trimslice running hardfp but without a neon simd In [9]: rpm.expandMacro('%{_target_cpu}') Out[9]: 'armv7hl' Yup, my bad - neither macro ends up containing the install arch in the end. The target gets set for install arch during initialization (to read the correct platform file) but is then reset in the bowels of rpmrc. The arch (compat and other) tables aren't really exported anywhere at all. In python the only thing is rpm.archscore() which returns non-zero values for arch strings that are compatible with the system architecture according to rpm's internal knowledge (configuration detected stuff). But then yum doesn't use even that, it does its own hardware detection by reading /proc/cpuinfo and all (see rpmUtils/arch.py) and uses its own hardcoded compatibility tables. Maybe rpm should export more of selected bits of its internal hw info. But even if it did, yum is unlikely to rely on that info being available anytime soon. So you'll end up (re)writing the ARM detection in yum anyway, might as well just add it there now and move on to other issues -
Re: [Rpm-maint] Multiple ABI architectures (parallel installation, detection, handling)
On 10/21/2011 10:20 PM, Dennis Gilmore wrote: long term maybe we can do something better. right now im looking for a way to enable yum to detect the arches correctly without it resorting to things like running readelf to determine the abi of the running system. since its noarch we cant just patch things in conditionally. BTW I was under the impression the ABI information for hard vs softfp simply DOES NOT EXIST in the ELF headers (or anywhere else for that matter). If http://wiki.debian.org/ArmHardFloatPort/VfpComparison#ld.so_hwcaps is wrong (or I'm misreading it) and you know of a way to detect it afterall, do let me know. - Panu - ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] Multiple ABI architectures (parallel installation, detection, handling)
Panu Matilainen pmati...@laiskiainen.org wrote: On 10/21/2011 10:20 PM, Dennis Gilmore wrote: long term maybe we can do something better. right now im looking for a way to enable yum to detect the arches correctly without it resorting to things like running readelf to determine the abi of the running system. since its noarch we cant just patch things in conditionally. BTW I was under the impression the ABI information for hard vs softfp simply DOES NOT EXIST in the ELF headers (or anywhere else for that matter). If http://wiki.debian.org/ArmHardFloatPort/VfpComparison#ld.so_hwcaps is wrong (or I'm misreading it) and you know of a way to detect it afterall, do let me know. - Panu - ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint It is in an optional section we do carry - see readelf -A. At an ARM conference so will discuss with Steve McIntyre of ARM (who is driving this ABI on their end). -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] Multiple ABI architectures (parallel installation, detection, handling)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 El Thu, 03 Nov 2011 14:26:58 +0200 Panu Matilainen pmati...@laiskiainen.org escribió: On 10/27/2011 12:00 AM, Dennis Gilmore wrote: [...big snip...] its not sufficient. it evaluates to armv7l on a hardware floating point system. we would need it to be armv7hl or armv7hnl to give yum the info it needs maybe _target_cpu could be used though it would need a dict of compatiable arches [dennis@panda02 ~]$ rpm --showrc |grep arm build arch: armv7hnl compatible build archs: armv7hnl armv7hl noarch install arch : armv7hnl compatible archs : armv7hnl armv7hl noarch optflags : -O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=neon -mthumb gpg --batch --no-verbose --no-armor --passphrase-fd 3 - -14: __isa_name armv7hnl - -14: _arch arm - -14: _build_arch arm - -14: _host armv7l-unknown-linux-gnueabi - -14: _host_cpu armv7l - -11: _target armv7hnl-linux - -11= _target_cpu armv7hnl - -14: arm armv3l armv4b armv4l armv4tl armv5tel armv5tejl armv6l armv7l armv7hl armv7nhl - -11: optflags-O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=neon -mthumb [root@trimslice01 ~]# rpm --showrc |grep arm build arch: armv7hl compatible build archs: armv7hl noarch install arch : armv7hl compatible archs : armv7hl noarch optflags : -O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -mthumb gpg --batch --no-verbose --no-armor --passphrase-fd 3 - -14: __isa_name armv7hl - -14: _arch arm - -14: _build_arch arm - -14: _host armv7l-unknown-linux-gnueabi - -14: _host_cpu armv7l - -11: _target armv7hl-linux - -11= _target_cpu armv7hl - -14: arm armv3l armv4b armv4l armv4tl armv5tel armv5tejl armv6l armv7l armv7hl armv7nhl - -11: optflags-O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -mthumb the first output is a system that has a neon simd unit and the second is without it. any software that can run on the second can also run on the first. __isa_name really should be the same on both. long term maybe we can do something better. right now im looking for a way to enable yum to detect the arches correctly without it resorting to things like running readelf to determine the abi of the running system. since its noarch we cant just patch things in conditionally. Dennis ok since i have a ton of the oddball hardware here is there someway to get whats setting compatible archs in --showrc? i ask because rpm --showrc |grep armv7 build arch: armv7hnl compatible build archs: armv7hnl armv7hl noarch install arch : armv7hnl compatible archs : armv7hnl armv7hl noarch optflags : -O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=neon -mthumb -14: __isa_namearmv7hnl - -14: _hostarmv7l-unknown-linux-gnueabi - -14: _host_cpuarmv7l - -11: _target armv7hnl-linux - -11= _target_cpu armv7hnl - -14: arm armv3l armv4b armv4l armv4tl armv5tel armv5tejl armv6l armv7l armv7hl armv7nhl -11: optflags-O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=neon -mthumb rpm --showrc |grep sparc build arch: sparc64 compatible build archs: sparc64v sparc64 sparcv9v sparcv9 sparcv8 sparc noarch install arch : sparc64v compatible archs : sparc64v sparc64 sparcv9v sparcv9 sparcv8 sparc noarch -14: _arch sparc - -14: _build_arch sparc - -14: _hostsparc-unknown-linux-gnu - -14: _host_cpusparc - -11: _target sparc64-linux - -11= _target_cpu sparc64 - -11: optflags %{__global_cflags} -m64 -mcpu=ultrasparc - -14: sparcsparc sparcv8 sparcv9 sparcv9v sparc64 sparc64v _host_cpu and _target_cpu on sparc doesnt give me a way to work out that im on a niagara box but in both cases i could work out whats needed by the info thats in compatible archs looking at rpmrc.c RPM_MACHTABLE_INSTARCH is whats used internal. is that available at all? on a niagara box In [2]: rpm.expandMacro('%{_target_cpu}') Out[2]: 'sparc64' on a trimslice running hardfp but without a neon simd In [9]: rpm.expandMacro('%{_target_cpu}') Out[9]: 'armv7hl' Yup, my bad - neither macro ends up containing the install arch in the end. The target gets set for install arch during initialization (to read the correct platform file) but is then reset in the bowels of rpmrc. The arch (compat and other) tables aren't really exported anywhere at all. In python the only thing is rpm.archscore() which returns non-zero values for arch strings that are compatible with the system architecture according to rpm's internal knowledge (configuration detected stuff). But then yum doesn't use even that, it does its own hardware detection by reading /proc/cpuinfo and all (see rpmUtils/arch.py) and uses its own
Re: [Rpm-maint] Multiple ABI architectures (parallel installation, detection, handling)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 El Fri, 21 Oct 2011 14:20:13 -0500 Dennis Gilmore den...@ausil.us escribió: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, 21 Oct 2011 14:00:56 -0500 Dennis Gilmore den...@ausil.us wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sat, 17 Sep 2011 16:02:52 +0300 Panu Matilainen pmati...@laiskiainen.org wrote: On 09/16/2011 04:58 PM, Dennis Gilmore wrote: On Friday, September 16, 2011 04:46:33 AM Panu Matilainen wrote: Oh I knew this would happen... the send-button is a magic memory-enhancer in disguise. One (of probably many) forgotten bits, this time hopefully with working address for Dennish too: On 09/15/2011 01:55 PM, Jon Masters wrote: All of the existing stuff assumes that if we have vfpv3 on ARMv7 we're going to be running in hardfp. We can keep that notion around in the very short term, but we need a longer term solution. And although we don't necessarily plan actual parallel installs of different ABIs, it is not technically the case that all ARMv7 systems are going to be running the hardfp port. We may e.g. install an ARMv5 port therein. From what I've heard + read about the hardfp ABI, its not actually possible to parallel-install with anything else because even the dynamic linker has no clue whether hardfp or some other ABI was used (see http://wiki.debian.org/ArmHardFloatPort/VfpComparison, cheers to our friends over Debian side again for a nice article :) Which makes this a trulycough uniquecough piece of work. I'd love to know why it was done that way - my layman sense thinks the compiler toolchain should be able to mark the ABI in the elf headers (heck this is what the whole hwcap thing is about). ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint Since the ABI difference is apparently not recorded anywhere, there's little chance of rpm being able to automatically do the right thing really, and I dunno if its worth it trying to jump through a whole lot of hoops for what seems an unachievable task. One simple brute-force solution that should work right now is to just keep the softfp/hardfp architectures incompatible from rpm POV and make the assumption that hardfp ABI will be used if the system is capable of it. And if somebody wants to override this assumption, that's what /etc/rpm/platform is for (allow overriding rpm's hw detection). im not the biggest fan of this, Maybe we can hard code it somehow at compilation time? Care to elaborate why would you want to hardcode it instead of just utilizing an existing method for special-case override, which is what we have here? as well as extend rpm-python to be able to expose it so yum can do the right thing. The arch used by rpm (whether detected or overridden) is available in %{_host_cpu} macro. If something else is needed then I need to know what that would be. its not sufficient. it evaluates to armv7l on a hardware floating point system. we would need it to be armv7hl or armv7hnl to give yum the info it needs maybe _target_cpu could be used though it would need a dict of compatiable arches [dennis@panda02 ~]$ rpm --showrc |grep arm build arch: armv7hnl compatible build archs: armv7hnl armv7hl noarch install arch : armv7hnl compatible archs : armv7hnl armv7hl noarch optflags : -O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=neon -mthumb gpg --batch --no-verbose --no-armor --passphrase-fd 3 - -14: __isa_name armv7hnl - -14: _arch arm - -14: _build_archarm - -14: _host armv7l-unknown-linux-gnueabi - -14: _host_cpu armv7l - -11: _targetarmv7hnl-linux - -11= _target_cpuarmv7hnl - -14: armarmv3l armv4b armv4l armv4tl armv5tel armv5tejl armv6l armv7l armv7hl armv7nhl - -11: optflags -O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=neon -mthumb [root@trimslice01 ~]# rpm --showrc |grep arm build arch: armv7hl compatible build archs: armv7hl noarch install arch : armv7hl compatible archs : armv7hl noarch optflags : -O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -mthumb gpg --batch --no-verbose --no-armor --passphrase-fd 3 - -14: __isa_name armv7hl - -14: _arch arm - -14: _build_archarm - -14: _host armv7l-unknown-linux-gnueabi - -14: _host_cpu armv7l - -11: _targetarmv7hl-linux - -11= _target_cpuarmv7hl - -14: armarmv3l armv4b armv4l armv4tl armv5tel armv5tejl armv6l armv7l armv7hl armv7nhl
Re: [Rpm-maint] Multiple ABI architectures (parallel installation, detection, handling)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sat, 17 Sep 2011 16:02:52 +0300 Panu Matilainen pmati...@laiskiainen.org wrote: On 09/16/2011 04:58 PM, Dennis Gilmore wrote: On Friday, September 16, 2011 04:46:33 AM Panu Matilainen wrote: Oh I knew this would happen... the send-button is a magic memory-enhancer in disguise. One (of probably many) forgotten bits, this time hopefully with working address for Dennish too: On 09/15/2011 01:55 PM, Jon Masters wrote: All of the existing stuff assumes that if we have vfpv3 on ARMv7 we're going to be running in hardfp. We can keep that notion around in the very short term, but we need a longer term solution. And although we don't necessarily plan actual parallel installs of different ABIs, it is not technically the case that all ARMv7 systems are going to be running the hardfp port. We may e.g. install an ARMv5 port therein. From what I've heard + read about the hardfp ABI, its not actually possible to parallel-install with anything else because even the dynamic linker has no clue whether hardfp or some other ABI was used (see http://wiki.debian.org/ArmHardFloatPort/VfpComparison, cheers to our friends over Debian side again for a nice article :) Which makes this a trulycough uniquecough piece of work. I'd love to know why it was done that way - my layman sense thinks the compiler toolchain should be able to mark the ABI in the elf headers (heck this is what the whole hwcap thing is about). ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint Since the ABI difference is apparently not recorded anywhere, there's little chance of rpm being able to automatically do the right thing really, and I dunno if its worth it trying to jump through a whole lot of hoops for what seems an unachievable task. One simple brute-force solution that should work right now is to just keep the softfp/hardfp architectures incompatible from rpm POV and make the assumption that hardfp ABI will be used if the system is capable of it. And if somebody wants to override this assumption, that's what /etc/rpm/platform is for (allow overriding rpm's hw detection). im not the biggest fan of this, Maybe we can hard code it somehow at compilation time? Care to elaborate why would you want to hardcode it instead of just utilizing an existing method for special-case override, which is what we have here? as well as extend rpm-python to be able to expose it so yum can do the right thing. The arch used by rpm (whether detected or overridden) is available in %{_host_cpu} macro. If something else is needed then I need to know what that would be. its not sufficient. it evaluates to armv7l on a hardware floating point system. we would need it to be armv7hl or armv7hnl to give yum the info it needs we have no plans to support multi-arch here, and anaconda (which we dont have yet but is underway) doesnt write out a /etc/rpm/platform file any longer. ive not looked to see if that code is removed from anaconda or just disabled. our plan is if you install a softfloat on the system thats all you can run. and same for hardfloat. while the buildsystem will have v7 builders that are building for v5 and therfore running softfp, most people should be running the hardfp port since its in the order of 300% faster for floating point operations. the only common case of softfp use i see on v7 hardware is those already running it. Anaconda stopped writing /etc/rpm/platform for a few crazy architectures when it no longer needed to do so, no reason why it couldn't do so again. Whether thats the best option or not I dunno. If there's a way (define or such) that can be used to detect which ABI we're compiling for then sure ifdef'ing the entire hardfp detection out is one option. Another option might be just ship with a hardwired /etc/rpm/platform for softfp arch version of rpm. Ill revise my patch to use hwcaps rather than reading /proc/cpuinfo id really like to wrap the code path around something define or something set when we build for armv7hl No need to revise the patch, what you currently have works for now and there's not much of a benefit for doing it just for ARM. If/when the whole detection is changed to use AT_PLATFORM/AT_HWCAP it'll go naturally along with that. FWIW the auxiliary vector has issues of its own: to reliably access it from a library requires reading /proc/self/auxv, and there we have that /proc dependency again. Of course the entire hw-detection and arch-compatibility system is seriously outdated and inadequate for todays needs, back in the nineties systems were a lot less schizophrenic
Re: [Rpm-maint] Multiple ABI architectures (parallel installation, detection, handling)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, 21 Oct 2011 14:00:56 -0500 Dennis Gilmore den...@ausil.us wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sat, 17 Sep 2011 16:02:52 +0300 Panu Matilainen pmati...@laiskiainen.org wrote: On 09/16/2011 04:58 PM, Dennis Gilmore wrote: On Friday, September 16, 2011 04:46:33 AM Panu Matilainen wrote: Oh I knew this would happen... the send-button is a magic memory-enhancer in disguise. One (of probably many) forgotten bits, this time hopefully with working address for Dennish too: On 09/15/2011 01:55 PM, Jon Masters wrote: All of the existing stuff assumes that if we have vfpv3 on ARMv7 we're going to be running in hardfp. We can keep that notion around in the very short term, but we need a longer term solution. And although we don't necessarily plan actual parallel installs of different ABIs, it is not technically the case that all ARMv7 systems are going to be running the hardfp port. We may e.g. install an ARMv5 port therein. From what I've heard + read about the hardfp ABI, its not actually possible to parallel-install with anything else because even the dynamic linker has no clue whether hardfp or some other ABI was used (see http://wiki.debian.org/ArmHardFloatPort/VfpComparison, cheers to our friends over Debian side again for a nice article :) Which makes this a trulycough uniquecough piece of work. I'd love to know why it was done that way - my layman sense thinks the compiler toolchain should be able to mark the ABI in the elf headers (heck this is what the whole hwcap thing is about). ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint Since the ABI difference is apparently not recorded anywhere, there's little chance of rpm being able to automatically do the right thing really, and I dunno if its worth it trying to jump through a whole lot of hoops for what seems an unachievable task. One simple brute-force solution that should work right now is to just keep the softfp/hardfp architectures incompatible from rpm POV and make the assumption that hardfp ABI will be used if the system is capable of it. And if somebody wants to override this assumption, that's what /etc/rpm/platform is for (allow overriding rpm's hw detection). im not the biggest fan of this, Maybe we can hard code it somehow at compilation time? Care to elaborate why would you want to hardcode it instead of just utilizing an existing method for special-case override, which is what we have here? as well as extend rpm-python to be able to expose it so yum can do the right thing. The arch used by rpm (whether detected or overridden) is available in %{_host_cpu} macro. If something else is needed then I need to know what that would be. its not sufficient. it evaluates to armv7l on a hardware floating point system. we would need it to be armv7hl or armv7hnl to give yum the info it needs maybe _target_cpu could be used though it would need a dict of compatiable arches [dennis@panda02 ~]$ rpm --showrc |grep arm build arch: armv7hnl compatible build archs: armv7hnl armv7hl noarch install arch : armv7hnl compatible archs : armv7hnl armv7hl noarch optflags : -O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=neon -mthumb gpg --batch --no-verbose --no-armor --passphrase-fd 3 - -14: __isa_name armv7hnl - -14: _archarm - -14: _build_arch arm - -14: _hostarmv7l-unknown-linux-gnueabi - -14: _host_cpuarmv7l - -11: _target armv7hnl-linux - -11= _target_cpu armv7hnl - -14: arm armv3l armv4b armv4l armv4tl armv5tel armv5tejl armv6l armv7l armv7hl armv7nhl - -11: optflags -O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=neon -mthumb [root@trimslice01 ~]# rpm --showrc |grep arm build arch: armv7hl compatible build archs: armv7hl noarch install arch : armv7hl compatible archs : armv7hl noarch optflags : -O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -mthumb gpg --batch --no-verbose --no-armor --passphrase-fd 3 - -14: __isa_name armv7hl - -14: _archarm - -14: _build_arch arm - -14: _hostarmv7l-unknown-linux-gnueabi - -14: _host_cpuarmv7l - -11: _target armv7hl-linux - -11= _target_cpu armv7hl - -14: arm armv3l armv4b armv4l armv4tl armv5tel armv5tejl armv6l armv7l armv7hl armv7nhl - -11: optflags -O2 -g -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -mthumb the first output is a system that has a neon simd unit and the second is without it. any software that can run on the second can also run on
Re: [Rpm-maint] Multiple ABI architectures (parallel installation, detection, handling)
On 09/16/2011 10:40 PM, Mark Hatle wrote: On 9/16/11 4:02 AM, Panu Matilainen wrote: On 09/15/2011 01:55 PM, Jon Masters wrote: ... What I want out of this discussion is a decision around how multiple ABIs within an architecture will be handled in general. If you're allergic to ARM, consider that in the Intel space there is now IA32, X32, and X64. The former is a 32-bit architecture, and the latter are both ABIs layered upon the 64-bit architecture. ARM and Intel aren't alone in this, there are others out there doing similarly fun things. Rpm does have limited provisions for multilib/multiarch of course (that's how x86_64 etc multilib archs work on Fedora/RHEL), but what's there is not sufficient for generic multiarch/multiabi parallel support. To outline the issues: Actually rpm already supports (and has a for a while) tri-arch/abi setups on MIPS. MIPS64 already has o32, n32 and n64 support. Generally speaking they're installed into /lib, /lib32 and /lib64 respectively. Yes, this is what I was referring to as limited provisions. Note I said for a *generic* multiarch setup, this just doesn't scale. 1) Location of libraries and the like: the libraries for different ABI's need to go to separate paths. Currently the non-debian world knows /lib and /lib64, and while it's of course possible to add /libx32 and whatnot, that gets *extremely* ugly real soon and simply does not scale to things like cross-toolchains. The debian multiarch spec seems rather nice in this regard: simply stuff it all into separate libraries in $(prefix)/lib/$(arch)-$(abi) style distinct paths, without differentiating between native and other archs/abis. As mentioned above, /lib, /lib32 and /lib64 are known. We intend to add /libx32 to our work in the near future. It's already part of the embedded OE-Core work. Nod, adding more /libxyz is probably the path of least immediate pain and works... as long as we're only talking about natively supported ABIs. This is something that can't be decided and solved by rpm alone, the entire distro and the compiler/linker toolchains are involved. Yes, the distro as a whole needs to agree on naming. Not something that a packaging system can do. BTW just to clarify this: whatever happens (or not) in rpm's multiarch/lib support, the current way of doing this has to be supported pretty much forever, and anything incompatible needs to be an opt-in mode that distros can choose if they so wish. 2) Dependencies: there needs to be a way to distinguish between (including but not limited to) libraries with identical soname-version but different ABI. Rpm currently only knows of two kinds of soname dependencies: ones with ()(64bit) postfix and ones without it. This kinda works for what it gets used now, but is full of weird wrinkles like some 64bit architectures not getting the (64bit) marker etc, and obviously does not scale to full-fledged multi/cross-arch dependencies. So the autogenerated soname dependencies would need to encode full arch+abi(+maybesomethingelsetoo) into the dependencies. The problem here is that changing the way dependencies are encoded is a MASSIVE backwards-compatibility breakage, and probably would need both old-style and new-style dependencies to be generated (although some of this might be possible to arrange at runtime) for a transition period. On mips, we have a ()(32bit) as well.. So we end up with (o32) libfoo.so, (n32) libfoo.so()(32bit), (n64) libfoo.so()(64bit)... Again, sure that works now (for a limited number of native ABIs) - it's the duct-tape path of least immediate pain. For manually added dependencies %{_isa} goes a long way, but probably not sufficient in its current form for a full-scale multiarch, multiabi system. So far this has scaled well in our trials. Soon to be adding in x32 may test this further. BTW n32 is our default ABI for mips64 in our products. Then there's the issue of dependency satisfiability across architectures and abi's. Rpm has some provisions for this (dependency colors), but its currently limited to just differentiate between 32bit vs 64bit (and obviously other) and there's no way to express allowed/prohibited dependency matches across architectures. Actually there are three bits defined.. 32-bit, 64-bit and n32. Ah, yes the mips bit, I'd mostly forgotten about that. Its the same story with that - the color bits work for a limited number of native ABI's, but a bitfield does not scale beyond that. The mips bit should be renamed to something more general though :) If my google search source is right, n32 is has 32bit pointers but 64bit integers, whereas x32 has 32bit pointers and 32bit integers, so while we could probably just hijack the mips bit for x32, technically it should be different (in case some day somebody decides x32 doesn't cut it and adds an n32-like ABI for x86_64 ;) The same coloring system is used to resolve elf-file conflicts on common paths. This rather crazy
Re: [Rpm-maint] Multiple ABI architectures (parallel installation, detection, handling)
Oh I knew this would happen... the send-button is a magic memory-enhancer in disguise. One (of probably many) forgotten bits, this time hopefully with working address for Dennish too: On 09/15/2011 01:55 PM, Jon Masters wrote: All of the existing stuff assumes that if we have vfpv3 on ARMv7 we're going to be running in hardfp. We can keep that notion around in the very short term, but we need a longer term solution. And although we don't necessarily plan actual parallel installs of different ABIs, it is not technically the case that all ARMv7 systems are going to be running the hardfp port. We may e.g. install an ARMv5 port therein. From what I've heard + read about the hardfp ABI, its not actually possible to parallel-install with anything else because even the dynamic linker has no clue whether hardfp or some other ABI was used (see http://wiki.debian.org/ArmHardFloatPort/VfpComparison, cheers to our friends over Debian side again for a nice article :) Which makes this a truly cough unique cough piece of work. I'd love to know why it was done that way - my layman sense thinks the compiler toolchain should be able to mark the ABI in the elf headers (heck this is what the whole hwcap thing is about). Since the ABI difference is apparently not recorded anywhere, there's little chance of rpm being able to automatically do the right thing really, and I dunno if its worth it trying to jump through a whole lot of hoops for what seems an unachievable task. One simple brute-force solution that should work right now is to just keep the softfp/hardfp architectures incompatible from rpm POV and make the assumption that hardfp ABI will be used if the system is capable of it. And if somebody wants to override this assumption, that's what /etc/rpm/platform is for (allow overriding rpm's hw detection). Of course the entire hw-detection and arch-compatibility system is seriously outdated and inadequate for todays needs, back in the nineties systems were a lot less schizophrenic :) I've various vagueish ideas in this area, but the current concept of arch is built so deep into tools around rpm that fundamentally changing it is likely to be a long and hard road. - Panu - ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] Multiple ABI architectures (parallel installation, detection, handling)
On Fri, 2011-09-16 at 12:46 +0300, Panu Matilainen wrote: One simple brute-force solution that should work right now is to just keep the softfp/hardfp architectures incompatible from rpm POV and make the assumption that hardfp ABI will be used if the system is capable of it. And if somebody wants to override this assumption, that's what /etc/rpm/platform is for (allow overriding rpm's hw detection). I'm ok with this at the moment - we aren't actually looking to support multi-arch at this point, just wanting to plan for the future. Jon. ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] Multiple ABI architectures (parallel installation, detection, handling)
On Fri, 2011-09-16 at 12:02 +0300, Panu Matilainen wrote: On 09/15/2011 01:55 PM, Jon Masters wrote: 1) Location of libraries and the like: the libraries for different ABI's need to go to separate paths. Currently the non-debian world knows /lib and /lib64, and while it's of course possible to add /libx32 and whatnot, that gets *extremely* ugly real soon and simply does not scale to things like cross-toolchains. The debian multiarch spec seems rather nice in this regard: simply stuff it all into separate libraries in $(prefix)/lib/$(arch)-$(abi) style distinct paths, without differentiating between native and other archs/abis. Yeh, if everything used it it'd be a bit nicer, but good luck getting Fedora/RHEL to sign off on moving everything to the Debian model :). For manually added dependencies %{_isa} goes a long way, but probably not sufficient in its current form for a full-scale multiarch, multiabi system. I'd assume that's a simple fix though, no? At least compared with all the other stuff that needs to be done :). 3) Package naming resolution: when in multilib mode (in rpm jargon, colored transaction), packages with identical name but different architecture can co-exist relatively peacefully. However the use of arch alone is limiting from multiabi perspective, it'd require every single different arch/abi combination to have arch of their own. Other possibility is differentiating in the package name, but its klunky too. I didn't (yet) see what debian is planning wrt this. Why would you want i686 and x32 to have arch as i686 ... that just seems insane. Also the embed the arch in the package name thing seems like a pretty big hack for x86_64 adding some 32bit packages (rpms need to be built twice, pkg. management needs to know about the special naming rules etc.) ... for i686 vs. x32 it seems much worse (AIUI, you'll have pretty much 100% x32 packages and no multilib). Thanks for your time. I'm attaching Dennis' original patch. Just so you can see what it does, not so that you can directly consider that as the solution that will be necessarily eventually in upstream if you can figure out a preferred means to generically handle multiABI. FWIW, I'm going to have a stab at converting the existing arch-detection (on linux) to use the auxiliary vector AT_* data. If that goes in, doing the same for ARM should be a no-brainer (and well, can be of course be done regardless of what other archs do) If we are going to change a bunch of the arch handling for this, it might be worth trying to merge the rpm/yum needs ... so we can just call into rpm for what we need, instead of duplicating 75% of the problem. ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] Multiple ABI architectures (parallel installation, detection, handling)
Resending i posted to list from address not subscibed On Friday, September 16, 2011 04:46:33 AM Panu Matilainen wrote: Oh I knew this would happen... the send-button is a magic memory-enhancer in disguise. One (of probably many) forgotten bits, this time hopefully with working address for Dennish too: On 09/15/2011 01:55 PM, Jon Masters wrote: All of the existing stuff assumes that if we have vfpv3 on ARMv7 we're going to be running in hardfp. We can keep that notion around in the very short term, but we need a longer term solution. And although we don't necessarily plan actual parallel installs of different ABIs, it is not technically the case that all ARMv7 systems are going to be running the hardfp port. We may e.g. install an ARMv5 port therein. From what I've heard + read about the hardfp ABI, its not actually possible to parallel-install with anything else because even the dynamic linker has no clue whether hardfp or some other ABI was used (see http://wiki.debian.org/ArmHardFloatPort/VfpComparison, cheers to our friends over Debian side again for a nice article :) Which makes this a truly cough unique cough piece of work. I'd love to know why it was done that way - my layman sense thinks the compiler toolchain should be able to mark the ABI in the elf headers (heck this is what the whole hwcap thing is about). Since the ABI difference is apparently not recorded anywhere, there's little chance of rpm being able to automatically do the right thing really, and I dunno if its worth it trying to jump through a whole lot of hoops for what seems an unachievable task. One simple brute-force solution that should work right now is to just keep the softfp/hardfp architectures incompatible from rpm POV and make the assumption that hardfp ABI will be used if the system is capable of it. And if somebody wants to override this assumption, that's what /etc/rpm/platform is for (allow overriding rpm's hw detection). im not the biggest fan of this, Maybe we can hard code it somehow at compilation time? as well as extend rpm-python to be able to expose it so yum can do the right thing. we have no plans to support multi-arch here, and anaconda (which we dont have yet but is underway) doesnt write out a /etc/rpm/platform file any longer. ive not looked to see if that code is removed from anaconda or just disabled. our plan is if you install a softfloat on the system thats all you can run. and same for hardfloat. while the buildsystem will have v7 builders that are building for v5 and therfore running softfp, most people should be running the hardfp port since its in the order of 300% faster for floating point operations. the only common case of softfp use i see on v7 hardware is those already running it. Ill revise my patch to use hwcaps rather than reading /proc/cpuinfo id really like to wrap the code path around something define or something set when we build for armv7hl Of course the entire hw-detection and arch-compatibility system is seriously outdated and inadequate for todays needs, back in the nineties systems were a lot less schizophrenic :) I've various vagueish ideas in this area, but the current concept of arch is built so deep into tools around rpm that fundamentally changing it is likely to be a long and hard road. rather than having yum do its own arch detection it should be able to grab it from rpm that way both tools use the same code path and will always give the same result. We also have an issue where we are setting the isa to the arch-32 we really should use arm-32 and armhfp-32 or armhf-32 that way the sub arches can be used correctly. some packages especially in the softfloat port could do with having additional arches built. pixman for example could do with armv6l and armv7l builds on softfp. yum has its basearch variable that it uses which currently is arm for all arm arches but should be armhfp for the new abi, I do think that the best way to deal with the new abi is to treat it as a new seperate arch. not sure if thats true of the new x86 abi. Dennis signature.asc Description: This is a digitally signed message part. ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] Multiple ABI architectures (parallel installation, detection, handling)
On 9/16/11 4:02 AM, Panu Matilainen wrote: On 09/15/2011 01:55 PM, Jon Masters wrote: ... What I want out of this discussion is a decision around how multiple ABIs within an architecture will be handled in general. If you're allergic to ARM, consider that in the Intel space there is now IA32, X32, and X64. The former is a 32-bit architecture, and the latter are both ABIs layered upon the 64-bit architecture. ARM and Intel aren't alone in this, there are others out there doing similarly fun things. Rpm does have limited provisions for multilib/multiarch of course (that's how x86_64 etc multilib archs work on Fedora/RHEL), but what's there is not sufficient for generic multiarch/multiabi parallel support. To outline the issues: Actually rpm already supports (and has a for a while) tri-arch/abi setups on MIPS. MIPS64 already has o32, n32 and n64 support. Generally speaking they're installed into /lib, /lib32 and /lib64 respectively. 1) Location of libraries and the like: the libraries for different ABI's need to go to separate paths. Currently the non-debian world knows /lib and /lib64, and while it's of course possible to add /libx32 and whatnot, that gets *extremely* ugly real soon and simply does not scale to things like cross-toolchains. The debian multiarch spec seems rather nice in this regard: simply stuff it all into separate libraries in $(prefix)/lib/$(arch)-$(abi) style distinct paths, without differentiating between native and other archs/abis. As mentioned above, /lib, /lib32 and /lib64 are known. We intend to add /libx32 to our work in the near future. It's already part of the embedded OE-Core work. This is something that can't be decided and solved by rpm alone, the entire distro and the compiler/linker toolchains are involved. Yes, the distro as a whole needs to agree on naming. Not something that a packaging system can do. 2) Dependencies: there needs to be a way to distinguish between (including but not limited to) libraries with identical soname-version but different ABI. Rpm currently only knows of two kinds of soname dependencies: ones with ()(64bit) postfix and ones without it. This kinda works for what it gets used now, but is full of weird wrinkles like some 64bit architectures not getting the (64bit) marker etc, and obviously does not scale to full-fledged multi/cross-arch dependencies. So the autogenerated soname dependencies would need to encode full arch+abi(+maybesomethingelsetoo) into the dependencies. The problem here is that changing the way dependencies are encoded is a MASSIVE backwards-compatibility breakage, and probably would need both old-style and new-style dependencies to be generated (although some of this might be possible to arrange at runtime) for a transition period. On mips, we have a ()(32bit) as well.. So we end up with (o32) libfoo.so, (n32) libfoo.so()(32bit), (n64) libfoo.so()(64bit)... For manually added dependencies %{_isa} goes a long way, but probably not sufficient in its current form for a full-scale multiarch, multiabi system. So far this has scaled well in our trials. Soon to be adding in x32 may test this further. BTW n32 is our default ABI for mips64 in our products. Then there's the issue of dependency satisfiability across architectures and abi's. Rpm has some provisions for this (dependency colors), but its currently limited to just differentiate between 32bit vs 64bit (and obviously other) and there's no way to express allowed/prohibited dependency matches across architectures. Actually there are three bits defined.. 32-bit, 64-bit and n32. The same coloring system is used to resolve elf-file conflicts on common paths. This rather crazy system can be entirely avoided by better/more fine-grained packaging: always place libraries and the like into separate (sub)packages to eliminate conflict potential. Based on the debian notes, dpkg apparently has issues with identical files shared across multiple packages, but this is not an issue for rpm: it's always been possible for multiple packages to share files, as long as the files are identical. The conflict resolution works well, but it's much better to break the problem up so you don't need to use it much. That is what we've tried to do for the most part. 3) Package naming resolution: when in multilib mode (in rpm jargon, colored transaction), packages with identical name but different architecture can co-exist relatively peacefully. However the use of arch alone is limiting from multiabi perspective, it'd require every single different arch/abi combination to have arch of their own. Other possibility is differentiating in the package name, but its klunky too. I didn't (yet) see what debian is planning wrt this. I didn't think that was true. I thought that the coloring system would allow you to use the same arch. Maybe this isn't true in the rpm.org world, but it is in
[Rpm-maint] Multiple ABI architectures (parallel installation, detection, handling)
Folks, I've taken the liberty of inviting myself to the party :) I'm also on IRC on Freenode, and have been following RPM development for some time in a lurking capacity. A number of you know me from the office, but greetings to everyone else! We're currently engaged in bootstrapping support for hardfp[0] in the Fedora Linux distribution (specifically, in release 15 thereof) for use by systems featuring an ARM version 7 A (Application) processor or later ARM. This is a new ABI as far as Linux is concerned, and it is parallel installable with the older EABI or (loosely) softfp that has been supported up until this point (for example in our ARM version 5 port, as released in Fedora 13). Currently, we have a hacked up RPM binary that introduces e.g.: * armv7hl * armv7nhl (MeeGo disagrees on this and calls it hnl btw) * armv7thl [1] Dennis has some code (he can forward) that is carried in F15 currently, and which Panu has previously taken a look at. It uses the typical parsing of /proc as an approach, though we agree that the correct way to do that would be through AT_HWCAPS, as in here: https://wiki.linaro.org/Resources/HowTo/DeterminingCPUFeatures (last raised on rpm-maint@ in 2008 as it turns out :) ) All of the existing stuff assumes that if we have vfpv3 on ARMv7 we're going to be running in hardfp. We can keep that notion around in the very short term, but we need a longer term solution. And although we don't necessarily plan actual parallel installs of different ABIs, it is not technically the case that all ARMv7 systems are going to be running the hardfp port. We may e.g. install an ARMv5 port therein. Add to this the fact that we now have X32 (a new x86 32-bit ABI) in the Intel world, and it's clear that RPM needs a generic way to handle multiple possible ABIs within the same architecture. I personally favor the multiarch approach being taken by Debian, but I don't really want to espouse one particular approach here. What I do want to do is to kick off a discussion thread in which you RPM folks can decide how you want to handle multiple ABIs in both the ARM and non-ARM worlds that myself, Dennis, and others occupy. I'm sure Dennis, Panu, and others can point folks at existing patches. What I want out of this discussion is a decision around how multiple ABIs within an architecture will be handled in general. If you're allergic to ARM, consider that in the Intel space there is now IA32, X32, and X64. The former is a 32-bit architecture, and the latter are both ABIs layered upon the 64-bit architecture. ARM and Intel aren't alone in this, there are others out there doing similarly fun things. Thanks for your time. I'm attaching Dennis' original patch. Just so you can see what it does, not so that you can directly consider that as the solution that will be necessarily eventually in upstream if you can figure out a preferred means to generically handle multiABI. Thanks for your collective time, Jon. P.S. Officially I took the day off today so I might not respond if you ping me on IRC. But email should be working :) [0] Begin TMI. hardfp is really a newer ABI extension to the existing ARM ABI. The existing ABI is sometimes referred to as EABI, but that term is no longer officially used by ARM. When we (the Linux community - we've even agreed between different distributions!) refer to hardfp, we mean the presence of a version 3 VFP unit, and conformance to section 6 of the ARM Architecture Procedure Calling Standard (AAPCS), in which an extension to the ABI is detailed that involves the lower 16 registers of a -d16 or -d32 (but -d16 is all we rely upon) VFPv3. Anyway. It's an ABI that is actually standardized :) [1] The t stands for Thumb2, a compressed in size instruction set that is also supported by ARMv7 Application processors. Using Thumb can buy some increases in performance, but it is a different ISA, and some of the packages involved do not yet build cleanly. Thumb(2) uses a concept known as interworking, in which the PLT (Procedure Linkage Table) is used to trampoline into Thumb2 code at all call points. When we do that, we are always in ARM mode. Consequently, we arbitrarily may switch over to Thumb2 if we care in the future, replacing one library at a time without breaking any existing stuff. Not an ABI issue and not really directly relevant to this thread, but some are confuzzled by the mention of Thumb, and hence this clarification. diff -uNr rpm-4.9.0-orig//installplatform rpm-4.9.0/installplatform --- rpm-4.9.0-orig//installplatform 2010-12-03 06:11:57.0 -0600 +++ rpm-4.9.0/installplatform 2011-08-05 12:25:13.0 -0500 @@ -19,7 +19,7 @@ case $arch in i[3456]86|pentium[34]|athlon|geode) SUBSTS='s_i386_i386_ s_i386_i486_ s_i386_i586_ s_i386_i686_ s_i386_pentium3_ s_i386_pentium4_ s_i386_athlon_ s_i386_geode_' ;; alpha*) SUBSTS='s_alpha_alpha_ s_alpha_alphaev5_ s_alpha_alphaev56_ s_alpha_alphapca56_ s_alpha_alphaev6_ s_alpha_alphaev67_' ;; - arm*)