Re: [julia-users] Re: Failure of installing Julia 0.4 (latest master) on Mac OS X 10.10
one option would be to split deps/Makefile into N different deps/ABC.make files (one for each dependency), so that each makefile could have a dependency on itself otherwise, authors just need to be careful to introduce a makefile dependency on the new build artifact whenever they add a patch file, or a different output, etc. On Thu, Oct 30, 2014 at 1:46 AM, Tony Kelman t...@kelman.net wrote: Anyone have a dependency graph in mind? I think a dozen or so of us have the dependency graph more or less memorized. Since I'm pseudo-patching arpack via command-line -D flags, a stale copy of it seems to reuse cached autotools-generated copies of the makefiles. I think what we would effectively need to make rebuilds happen reliably would be some way of having any git pull that modifies the relevant flags or section of deps/Makefile also touch the corresponding configure scripts to make them rerun. And any time we add or modify a patch to a dependency, the only reliable ways I've found of forcing them to apply are either manually applying them, or cleaning out the existing sources which requires make -C deps distclean-xyz (and hence a re-downloading of the tarball and complete rebuild of that dependency). I'm sort of wishing we could do out-of-tree builds for some of this, to better separate sources from build objects. But some of the deps like openblas don't really support out-of-tree builds at all. Grand plan would be rewriting the thousands of lines of Makefile logic into CMake, which might handle all of this better with its caching mechanism. That might be a terrible idea and not work at all though, someone probably just has to try it and see whether it ends up better or worse. Cross-compilation (which only a few of us do very often) with CMake was... difficult... initially, but I eventually changed my mind on it and came to the conclusion that it just takes a really verbose invocation (or a toolchain file where you write down the long set of flags you need) to get working.
[julia-users] Re: Failure of installing Julia 0.4 (latest master) on Mac OS X 10.10
Also having problems after a fetch... lots of link errors followed by _ztrsen_, referenced from _zneupd_ in zneupd.o _zunm2r_, referenced from: _zneupd_ in zneupd.o ld: symbol(s) not found for architecture x86_64 collect2: error: ld returned 1 exit status make[4]: *** [libarpack.la] Error 1 make[3]: *** [all-recursive] Error 1 make[2]: *** [arpack-ng-3.1.5/.libs/libarpack.dylib] Error 2 make[1]: *** [julia-release] Error 2 make: *** [release] Error 2 was fixed by rm -rf libarpack-ng and SuiteSparce and then make previous to that I had to remove and rebuild llvm I think what happens is that sometimes if you let deps grow too far out of date, they don't work with each other any more (makes sense) and the build process get wedged, it can't even figure out what to do. but who wants to rebuild deps entirely every time? no one. Seems like you need to have a way to indicate inter-dep dependency, so that change to 1 particular dep will cascade rebuild to all the other dependencies necessary but no more. it may be in this case that if any one of arpack-ng SuiteSparce objconv change major or minor numbers, they all have to be rebuilt. Anyone have a dependency graph in mind?
[julia-users] Re: Failure of installing Julia 0.4 (latest master) on Mac OS X 10.10
Part of that might be my fault, we added the objconv utility recently as a build-time dependency to handle renaming of symbols in OpenBLAS on Mac, to avoid LP64-vs-ILP64 ABI incompatibilities. Can you tell me if you have a deps/objconv folder, and if so what's in it? Does clang++ -o objconv -O2 *.cpp in that folder work? On Tuesday, October 28, 2014 3:53:57 AM UTC-7, Dahua Lin wrote: I just make a fresh clone of Julia 0.4, and when I did make, I get the following error: Note: The following floating-point exceptions are signalling: IEEE_DENORMAL make[4]: /Users/dhlin/julia-0.4/deps/objconv/objconv: No such file or directory make[4]: *** [../libopenblasp-r0.2.12.a.renamed] Error 1 make[3]: *** [shared] Error 2 *** Clean the OpenBLAS build with 'make -C deps clean-openblas'. Rebuild with 'make OPENBLAS_USE_THREAD=0 if OpenBLAS had trouble linking libpthread.so, and with 'make OPENBLAS_TARGET_ARCH=NEHALEM' if there were errors building SandyBridge support. Both these options can also be used simultaneously. *** make[2]: *** [openblas-v0.2.12/libopenblas.dylib] Error 1 make[1]: *** [julia-release] Error 2 make: *** [release] Error 2 Not sure why it raises IEEE_DENORMAL, or OpenBLAS just doesn't work in Yosemite? Dahua
[julia-users] Re: Failure of installing Julia 0.4 (latest master) on Mac OS X 10.10
deps/objconv is not there. On Tuesday, October 28, 2014 7:14:11 PM UTC+8, Tony Kelman wrote: Part of that might be my fault, we added the objconv utility recently as a build-time dependency to handle renaming of symbols in OpenBLAS on Mac, to avoid LP64-vs-ILP64 ABI incompatibilities. Can you tell me if you have a deps/objconv folder, and if so what's in it? Does clang++ -o objconv -O2 *.cpp in that folder work? On Tuesday, October 28, 2014 3:53:57 AM UTC-7, Dahua Lin wrote: I just make a fresh clone of Julia 0.4, and when I did make, I get the following error: Note: The following floating-point exceptions are signalling: IEEE_DENORMAL make[4]: /Users/dhlin/julia-0.4/deps/objconv/objconv: No such file or directory make[4]: *** [../libopenblasp-r0.2.12.a.renamed] Error 1 make[3]: *** [shared] Error 2 *** Clean the OpenBLAS build with 'make -C deps clean-openblas'. Rebuild with 'make OPENBLAS_USE_THREAD=0 if OpenBLAS had trouble linking libpthread.so, and with 'make OPENBLAS_TARGET_ARCH=NEHALEM' if there were errors building SandyBridge support. Both these options can also be used simultaneously. *** make[2]: *** [openblas-v0.2.12/libopenblas.dylib] Error 1 make[1]: *** [julia-release] Error 2 make: *** [release] Error 2 Not sure why it raises IEEE_DENORMAL, or OpenBLAS just doesn't work in Yosemite? Dahua
Re: [julia-users] Re: Failure of installing Julia 0.4 (latest master) on Mac OS X 10.10
Damn, I seem to have gotten the dependencies wrong here https://github.com/JuliaLang/Julia/pull/8734/files#diff-3ba529ae517f6b57803af0502f52a40bR840 then? Steven Johnson said he was able to get things to work, I thought. Did it look like all of openblas was rebuilding from scratch? You may have to do `make -C deps install-objconv` manually, but this really shouldn’t be necessary for everyone to do manually if I did things right. For everyone else who builds Julia from source regularly, if you want to rename your openblas symbols, follow the steps here https://github.com/JuliaLang/julia/pull/8734#issuecomment-59978800 - this assumes you’ve built openblas recently enough that you have an existing copy of v0.2.12. Check the results with `Base.blas_vendor()`, it should say :openblas64 if your library has renamed symbols. I also hope completely fresh from-scratch builds on mac should work properly, or if you don’t want to rename your openblas symbols yet (particularly if you plan on bisecting back and forth through the git history) it should leave your existing built openblas library alone. Let’s see if we can figure out what’s going wrong in Dahua’s case. Sorry for the trouble. From: Dahua Lin Sent: Tuesday, October 28, 2014 4:58 AM To: julia-users@googlegroups.com Subject: [julia-users] Re: Failure of installing Julia 0.4 (latest master) on Mac OS X 10.10 deps/objconv is not there. On Tuesday, October 28, 2014 7:14:11 PM UTC+8, Tony Kelman wrote: Part of that might be my fault, we added the objconv utility recently as a build-time dependency to handle renaming of symbols in OpenBLAS on Mac, to avoid LP64-vs-ILP64 ABI incompatibilities. Can you tell me if you have a deps/objconv folder, and if so what's in it? Does clang++ -o objconv -O2 *.cpp in that folder work? On Tuesday, October 28, 2014 3:53:57 AM UTC-7, Dahua Lin wrote: I just make a fresh clone of Julia 0.4, and when I did make, I get the following error: Note: The following floating-point exceptions are signalling: IEEE_DENORMAL make[4]: /Users/dhlin/julia-0.4/deps/objconv/objconv: No such file or directory make[4]: *** [../libopenblasp-r0.2.12.a.renamed] Error 1 make[3]: *** [shared] Error 2 *** Clean the OpenBLAS build with 'make -C deps clean-openblas'. Rebuild with 'make OPENBLAS_USE_THREAD=0 if OpenBLAS had trouble linking libpthread.so, and with 'make OPENBLAS_TARGET_ARCH=NEHALEM' if there were errors building SandyBridge support. Both these options can also be used simultaneously. *** make[2]: *** [openblas-v0.2.12/libopenblas.dylib] Error 1 make[1]: *** [julia-release] Error 2 make: *** [release] Error 2 Not sure why it raises IEEE_DENORMAL, or OpenBLAS just doesn't work in Yosemite? Dahua
Re: [julia-users] Re: Failure of installing Julia 0.4 (latest master) on Mac OS X 10.10
This is a completely fresh clone (directly from Github) -- so there is nothing there left from some previous version. I open an issue: https://github.com/JuliaLang/julia/issues/8842 Dahua On Tuesday, October 28, 2014 8:13:21 PM UTC+8, Tony Kelman wrote: Damn, I seem to have gotten the dependencies wrong here https://github.com/JuliaLang/Julia/pull/8734/files#diff-3ba529ae517f6b57803af0502f52a40bR840 then? Steven Johnson said he was able to get things to work, I thought. Did it look like all of openblas was rebuilding from scratch? You may have to do `make -C deps install-objconv` manually, but this really shouldn’t be necessary for everyone to do manually if I did things right. For everyone else who builds Julia from source regularly, if you want to rename your openblas symbols, follow the steps here https://github.com/JuliaLang/julia/pull/8734#issuecomment-59978800 - this assumes you’ve built openblas recently enough that you have an existing copy of v0.2.12. Check the results with `Base.blas_vendor()`, it should say :openblas64 if your library has renamed symbols. I also hope completely fresh from-scratch builds on mac should work properly, or if you don’t want to rename your openblas symbols yet (particularly if you plan on bisecting back and forth through the git history) it should leave your existing built openblas library alone. Let’s see if we can figure out what’s going wrong in Dahua’s case. Sorry for the trouble. *From:* Dahua Lin javascript: *Sent:* Tuesday, October 28, 2014 4:58 AM *To:* julia...@googlegroups.com javascript: *Subject:* [julia-users] Re: Failure of installing Julia 0.4 (latest master) on Mac OS X 10.10 deps/objconv is not there. On Tuesday, October 28, 2014 7:14:11 PM UTC+8, Tony Kelman wrote: Part of that might be my fault, we added the objconv utility recently as a build-time dependency to handle renaming of symbols in OpenBLAS on Mac, to avoid LP64-vs-ILP64 ABI incompatibilities. Can you tell me if you have a deps/objconv folder, and if so what's in it? Does clang++ -o objconv -O2 *.cpp in that folder work? On Tuesday, October 28, 2014 3:53:57 AM UTC-7, Dahua Lin wrote: I just make a fresh clone of Julia 0.4, and when I did make, I get the following error: Note: The following floating-point exceptions are signalling: IEEE_DENORMAL make[4]: /Users/dhlin/julia-0.4/deps/objconv/objconv: No such file or directory make[4]: *** [../libopenblasp-r0.2.12.a.renamed] Error 1 make[3]: *** [shared] Error 2 *** Clean the OpenBLAS build with 'make -C deps clean-openblas'. Rebuild with 'make OPENBLAS_USE_THREAD=0 if OpenBLAS had trouble linking libpthread.so, and with 'make OPENBLAS_TARGET_ARCH=NEHALEM' if there were errors building SandyBridge support. Both these options can also be used simultaneously. *** make[2]: *** [openblas-v0.2.12/libopenblas.dylib] Error 1 make[1]: *** [julia-release] Error 2 make: *** [release] Error 2 Not sure why it raises IEEE_DENORMAL, or OpenBLAS just doesn't work in Yosemite? Dahua
Re: [julia-users] Re: Failure of installing Julia 0.4 (latest master) on Mac OS X 10.10
It builds successfully after I did make -C deps install-objconv. However, I think the make procedure should automatically do that ... On Wednesday, October 29, 2014 8:32:32 AM UTC+8, Dahua Lin wrote: This is a completely fresh clone (directly from Github) -- so there is nothing there left from some previous version. I open an issue: https://github.com/JuliaLang/julia/issues/8842 Dahua On Tuesday, October 28, 2014 8:13:21 PM UTC+8, Tony Kelman wrote: Damn, I seem to have gotten the dependencies wrong here https://github.com/JuliaLang/Julia/pull/8734/files#diff-3ba529ae517f6b57803af0502f52a40bR840 then? Steven Johnson said he was able to get things to work, I thought. Did it look like all of openblas was rebuilding from scratch? You may have to do `make -C deps install-objconv` manually, but this really shouldn’t be necessary for everyone to do manually if I did things right. For everyone else who builds Julia from source regularly, if you want to rename your openblas symbols, follow the steps here https://github.com/JuliaLang/julia/pull/8734#issuecomment-59978800 - this assumes you’ve built openblas recently enough that you have an existing copy of v0.2.12. Check the results with `Base.blas_vendor()`, it should say :openblas64 if your library has renamed symbols. I also hope completely fresh from-scratch builds on mac should work properly, or if you don’t want to rename your openblas symbols yet (particularly if you plan on bisecting back and forth through the git history) it should leave your existing built openblas library alone. Let’s see if we can figure out what’s going wrong in Dahua’s case. Sorry for the trouble. *From:* Dahua Lin *Sent:* Tuesday, October 28, 2014 4:58 AM *To:* julia...@googlegroups.com *Subject:* [julia-users] Re: Failure of installing Julia 0.4 (latest master) on Mac OS X 10.10 deps/objconv is not there. On Tuesday, October 28, 2014 7:14:11 PM UTC+8, Tony Kelman wrote: Part of that might be my fault, we added the objconv utility recently as a build-time dependency to handle renaming of symbols in OpenBLAS on Mac, to avoid LP64-vs-ILP64 ABI incompatibilities. Can you tell me if you have a deps/objconv folder, and if so what's in it? Does clang++ -o objconv -O2 *.cpp in that folder work? On Tuesday, October 28, 2014 3:53:57 AM UTC-7, Dahua Lin wrote: I just make a fresh clone of Julia 0.4, and when I did make, I get the following error: Note: The following floating-point exceptions are signalling: IEEE_DENORMAL make[4]: /Users/dhlin/julia-0.4/deps/objconv/objconv: No such file or directory make[4]: *** [../libopenblasp-r0.2.12.a.renamed] Error 1 make[3]: *** [shared] Error 2 *** Clean the OpenBLAS build with 'make -C deps clean-openblas'. Rebuild with 'make OPENBLAS_USE_THREAD=0 if OpenBLAS had trouble linking libpthread.so, and with 'make OPENBLAS_TARGET_ARCH=NEHALEM' if there were errors building SandyBridge support. Both these options can also be used simultaneously. *** make[2]: *** [openblas-v0.2.12/libopenblas.dylib] Error 1 make[1]: *** [julia-release] Error 2 make: *** [release] Error 2 Not sure why it raises IEEE_DENORMAL, or OpenBLAS just doesn't work in Yosemite? Dahua
