Hi Tony, Thanks for the extremely helpful response.
I presume this means that it is too much to hope for a way to compile multiple architectures into a single sys.so binary? That's OK. I can probably figure out a way to swap in the right one in a login script. Thanks again! Jim On Sunday, July 20, 2014 1:32:00 AM UTC-4, Tony Kelman wrote: > > Hi Jim. OPENBLAS_CORETYPE doesn't have to be set, but it can be to > optionally override the automatic processor family detection in OpenBLAS. > This can be a useful data point for when we trigger bugs or other strange > behavior in OpenBLAS. You can see the list of possible options here > https://github.com/xianyi/OpenBLAS/blob/e6668dd83b653a2cf61b2993b49b9023b1e52029/driver/others/dynamic.c#L296-L318 > > I think Piledriver and Bulldozer use AVX but not AVX2, so you'll probably > want a tweaked version of what I added in > https://github.com/JuliaLang/julia/pull/7476 but using the NO_AVX2 flag > rather than NO_AVX. The former was only recently added in OpenBLAS. > > The "Target architecture mismatch" error is actually not to do with > OpenBLAS at all, rather that has to do with pre-compilation of the base > Julia standard library. The default code generation through LLVM is native > to a specific machine, so the precompiled sys.so is not portable. You can > delete that file but then your Julia executable will take about 10 times > longer to start, or you can build Julia setting JULIA_CPU_TARGET to some > minimum setting. I'm not sure what settings are valid there, probably at > least "core2" but maybe others as well. There are some comments about this > here > https://github.com/JuliaLang/julia/blob/e76eced81953976f4457384ae5dabfd75de27029/Make.inc#L353-L360 > > but it is generally a bit underdocumented how to properly build Julia > across heterogenous processors. > > > > On Saturday, July 19, 2014 10:15:13 PM UTC-7, James Delaney wrote: >> >> Hi Tony, >> >> So are you saying that OPENBLAS_CORETYPE is a shell environment variable >> that has to be set by the user? >> >> I have been trying to figure out how to compile on my HPC. This cluster >> is heterogeneous: some Intel, but also Opterons of at least Piledriver, >> Bulldozer, and Barcelona types. >> >> We're running RHEL 6.5, so we've got that binutils issue. >> >> After a bit of searching around, >> https://github.com/xianyi/OpenBLAS/issues/401 >> >> I see that since it is only vpermpd that throws an error, it is actually >> the flag NO_AVX2=1 that should be set in the build of OpenBlas. So perhaps >> the patch should be changed accordingly? >> https://github.com/JuliaLang/julia/issues/7482 >> >> Anyhow setting that flag "worked" for me so that I didn't have to set >> DYNAMIC_ARCH=0. >> >> But even so, my julia executable fails to run on different architectures. >> I've been able to build it separately for Barcelona, Bulldozer, and >> Piledriver, each running on that specific hardware, but not on any other. >> The error I get from julia is: >> >> "Target architecture mismatch. Please delete or regenerate >> sys.{so,dll,dylib}." >> >> So in order for a single binary with DYNAMIC_ARCH=1, to work on these >> other architectures, do I have to write a shell script that sets this >> OPENBLAS_CORETYPE >> environment variable? >> >> If so, what are the possible settings? >> >> Thanks, >> >> Jim >> >> On Saturday, July 19, 2014 9:42:54 PM UTC-4, Tony Kelman wrote: >>> >>> As of the latest version of OpenBLAS, it now reports which CPU family >>> it's using the optimized kernels for in its info string. If you build for a >>> particular fixed family it appears in upper case, if it's selected >>> dynamically then it shows with just the first letter capitalized. If your >>> OpenBLAS was built with DYNAMIC_ARCH enabled, then you can actually >>> dynamically choose a different family by setting the environment variable >>> OPENBLAS_CORETYPE. >>> >>> >>> On Saturday, July 19, 2014 7:51:59 AM UTC-7, Elliot Saba wrote: >>>> >>>> You can get some limited information by running versioninfo(true) at >>>> the julia prompt. There are certain things that are recorded, and one of >>>> them is the commandline that is passed to OpenBLAS when building Julia. I >>>> believe that is what you are referring to when talking about Sandy Bridge >>>> and Nehalem. By default, OpenBLAS will target the highest CPU capability >>>> that your CPU advertises it is capable of; so if your CPU is capable of >>>> Sandy Bridge instructions, it will target Sandy Bridge and not Nehalem. >>>> You can force it to target a certain architecture however, and that >>>> forcing would get recorded. >>>> >>>> >>>> On Sat, Jul 19, 2014 at 10:45 AM, Tomas Lycken <[email protected]> >>>> wrote: >>>> >>>>> I decided to rebuild my Julia installation, and in the process try to >>>>> optimize the build to my machine. As a first attempt, I just added a file >>>>> Make.user to the Julia directory, with the single line "MARCH=native" in >>>>> it. No other changes to the current master. >>>>> >>>>> Is there any way to inspect what options were actually used when >>>>> building, other than digging around in the build scripts and try to >>>>> deduce >>>>> it myself? For example, I have a SandyBridge CPU, but if I understand the >>>>> docs correctly the default arch is Nehalem. Is there a (simple) way to >>>>> check that this was taken into account in my build? >>>>> >>>>> // Tomas >>>>> >>>> >>>>
