You're welcome. Compiling multiple architectures into the same sys.so is a very interesting question - that's not possible right now, not easily, as far as I'm aware. But it would be interesting to take a page out of what OpenBLAS does and set up something similar, both for the pre-compiled portions of Julia's standard library and as part of the forthcoming static compilation features that will likely see some major development after 0.3 is released.
If you're feeling adventurous and want to roll your own substitute right now, you might be able to do some clever things with the -J startup option. I'd have to look into the code to figure out whether that applies just to the sys.ji slow version of the system image, or also the faster but architecture-dependent sys.so. On Saturday, July 19, 2014 10:49:14 PM UTC-7, James Delaney wrote: > > Hi Tony, > > Thanks for the extremely helpful response. > > I presume this means that it is too much to hope for a way to compile > multiple architectures into a single sys.so binary? > > That's OK. I can probably figure out a way to swap in the right one in a > login script. > > Thanks again! > > Jim > > > On Sunday, July 20, 2014 1:32:00 AM UTC-4, Tony Kelman wrote: >> >> Hi Jim. OPENBLAS_CORETYPE doesn't have to be set, but it can be to >> optionally override the automatic processor family detection in OpenBLAS. >> This can be a useful data point for when we trigger bugs or other strange >> behavior in OpenBLAS. You can see the list of possible options here >> https://github.com/xianyi/OpenBLAS/blob/e6668dd83b653a2cf61b2993b49b9023b1e52029/driver/others/dynamic.c#L296-L318 >> >> I think Piledriver and Bulldozer use AVX but not AVX2, so you'll probably >> want a tweaked version of what I added in >> https://github.com/JuliaLang/julia/pull/7476 but using the NO_AVX2 flag >> rather than NO_AVX. The former was only recently added in OpenBLAS. >> >> The "Target architecture mismatch" error is actually not to do with >> OpenBLAS at all, rather that has to do with pre-compilation of the base >> Julia standard library. The default code generation through LLVM is native >> to a specific machine, so the precompiled sys.so is not portable. You can >> delete that file but then your Julia executable will take about 10 times >> longer to start, or you can build Julia setting JULIA_CPU_TARGET to some >> minimum setting. I'm not sure what settings are valid there, probably at >> least "core2" but maybe others as well. There are some comments about this >> here >> https://github.com/JuliaLang/julia/blob/e76eced81953976f4457384ae5dabfd75de27029/Make.inc#L353-L360 >> >> but it is generally a bit underdocumented how to properly build Julia >> across heterogenous processors. >> >> >> >> On Saturday, July 19, 2014 10:15:13 PM UTC-7, James Delaney wrote: >>> >>> Hi Tony, >>> >>> So are you saying that OPENBLAS_CORETYPE is a shell environment variable >>> that has to be set by the user? >>> >>> I have been trying to figure out how to compile on my HPC. This cluster >>> is heterogeneous: some Intel, but also Opterons of at least Piledriver, >>> Bulldozer, and Barcelona types. >>> >>> We're running RHEL 6.5, so we've got that binutils issue. >>> >>> After a bit of searching around, >>> https://github.com/xianyi/OpenBLAS/issues/401 >>> >>> I see that since it is only vpermpd that throws an error, it is actually >>> the flag NO_AVX2=1 that should be set in the build of OpenBlas. So perhaps >>> the patch should be changed accordingly? >>> https://github.com/JuliaLang/julia/issues/7482 >>> >>> Anyhow setting that flag "worked" for me so that I didn't have to set >>> DYNAMIC_ARCH=0. >>> >>> But even so, my julia executable fails to run on different >>> architectures. I've been able to build it separately for Barcelona, >>> Bulldozer, and Piledriver, each running on that specific hardware, but not >>> on any other. The error I get from julia is: >>> >>> "Target architecture mismatch. Please delete or regenerate >>> sys.{so,dll,dylib}." >>> >>> So in order for a single binary with DYNAMIC_ARCH=1, to work on these >>> other architectures, do I have to write a shell script that sets this >>> OPENBLAS_CORETYPE >>> environment variable? >>> >>> If so, what are the possible settings? >>> >>> Thanks, >>> >>> Jim >>> >>> On Saturday, July 19, 2014 9:42:54 PM UTC-4, Tony Kelman wrote: >>>> >>>> As of the latest version of OpenBLAS, it now reports which CPU family >>>> it's using the optimized kernels for in its info string. If you build for >>>> a >>>> particular fixed family it appears in upper case, if it's selected >>>> dynamically then it shows with just the first letter capitalized. If your >>>> OpenBLAS was built with DYNAMIC_ARCH enabled, then you can actually >>>> dynamically choose a different family by setting the environment variable >>>> OPENBLAS_CORETYPE. >>>> >>>> >>>> On Saturday, July 19, 2014 7:51:59 AM UTC-7, Elliot Saba wrote: >>>>> >>>>> You can get some limited information by running versioninfo(true) at >>>>> the julia prompt. There are certain things that are recorded, and one of >>>>> them is the commandline that is passed to OpenBLAS when building Julia. >>>>> I >>>>> believe that is what you are referring to when talking about Sandy Bridge >>>>> and Nehalem. By default, OpenBLAS will target the highest CPU capability >>>>> that your CPU advertises it is capable of; so if your CPU is capable of >>>>> Sandy Bridge instructions, it will target Sandy Bridge and not Nehalem. >>>>> You can force it to target a certain architecture however, and that >>>>> forcing would get recorded. >>>>> >>>>> >>>>> On Sat, Jul 19, 2014 at 10:45 AM, Tomas Lycken <[email protected]> >>>>> wrote: >>>>> >>>>>> I decided to rebuild my Julia installation, and in the process try to >>>>>> optimize the build to my machine. As a first attempt, I just added a >>>>>> file >>>>>> Make.user to the Julia directory, with the single line "MARCH=native" in >>>>>> it. No other changes to the current master. >>>>>> >>>>>> Is there any way to inspect what options were actually used when >>>>>> building, other than digging around in the build scripts and try to >>>>>> deduce >>>>>> it myself? For example, I have a SandyBridge CPU, but if I understand >>>>>> the >>>>>> docs correctly the default arch is Nehalem. Is there a (simple) way to >>>>>> check that this was taken into account in my build? >>>>>> >>>>>> // Tomas >>>>>> >>>>> >>>>>
