Ivar described the situation to a T. We have pretty good coverage of problems everywhere except Windows thanks to Travis. Occasionally things pop up that are FreeBSD specific, but that's not as common or hard to fix. I've been working sporadically on https://github.com/JuliaLang/julia/pull/6028 since the start of March. I opened that PR just 2 weeks after my first exposure to Julia. Unfortunately there was only a couple of days' time where I could get Julia to build and run all the tests within 30 minutes on AppVeyor. Both Julia and the AppVeyor VM's have gotten slower since then.
Currently downloading and running the binary installer, then running the tests can pass on 32-bit Windows in just under 30 minutes on an AppVeyor VM, but 64 bit has a current failure (backtrace test) and would time out if it didn't. We could split the tests up in a matrix, but you need to build the binaries somewhat more regularly than the current "nightlies" for it to be useful for CI purposes. Doing "make dist" in AppVeyor can't finish within 30 minutes either, even using binary dependencies for everything. I was playing with a different CI service called Wercker that doesn't run on pull requests and doesn't have build matrix functionality, but crucially caches dependencies and allows you to pre-build custom build boxes (so you can for example upgrade the VM to Ubuntu 14.04, install a bunch of customized packages without needing to wait for apt-get on every build, etc). With all the dependencies cached, it is actually possible to cross-compile both 32 bit and 64 bit Julia in about 20-25 minutes. I had a temporary location I was able to upload some test binaries to, but would need something with larger capacity (probably configured to throw away all but the most recent few builds?) to set up this kind of thing permanently. We could figure out some kind of webhook or something to send a notification from wherever the cross-compile build happens and uploads a binary to trigger an AppVeyor build to install and run the tests. I was a little hesitant to give AWS my credit card info (poor grad student here), otherwise I would set up a trial on S3. There's another one of these CI services called Shippable that I also looked at, it's Docker-based and can run on pull requests (I think) and do build matrices, but they don't have the same build environment customization features as Wercker does (yet). If they eventually let you provide your own custom Docker image that would be perfect for cross-compiling binaries, just have to find someplace to upload them to. If someone has the resources and abilities to set up a "machine farm" I'm more than willing to share my "build Julia for Windows as quickly as possible" scripts. The hard part is the farm, since we aren't Mozilla. -Tony On Monday, September 1, 2014 10:40:13 AM UTC-7, Ivar Nesje wrote: > > Travis also runs on Pull Requests, so we get a warning as long as the > tests covers the issue on Linux or OSX. It would be really great to test on > windows and different os versions as well, but afaik Travis does not > support more than we do now. There has been some suggestions about using > AppVeyor for windows testing, but there is some issues about their VMs > being too slow to complete within their 30 minute time limit. > > I don't understand how "really simple" can describe a "machine farm", but > everybody would be really happy if we could get more extensive build > statuses on pull requests. Unfortuenatly it requires a nonzero amount of > maintenance, and someone will actually have to do that work and set it up. > > Regards > Ivar > > kl. 16:59:45 UTC+2 mandag 1. september 2014 skrev Dan Luu følgende: >> >> Something that might help prevent issues like this in the future is >> using something like bors (http://graydon.livejournal.com/186550.html >> <http://www.google.com/url?q=http%3A%2F%2Fgraydon.livejournal.com%2F186550.html&sa=D&sntz=1&usg=AFQjCNEyFlNExquxny_xXxdnk5FjS-i3KQ>) >> >> >> instead of Travis for this kind of thing, since Travis only notifies >> people after the failure. Rust uses this, and I like it a lot. IIRC, >> it's prevented me from breaking a test on some obscure platform I >> don't own at least once. >> >> If you don't want to read all of the text in the link, the idea is >> really simple: when someone creates a pull request, tests get run on >> some machine farm. Instead of having maintainers merge pull requests, >> they approve them. If a PR is approved and tests pass, the PR will get >> merged (with some logic to make sure nothing can fail due to a race >> condition). >> >> On Mon, Sep 1, 2014 at 3:36 AM, Kevin Squire <[email protected]> >> wrote: >> > See https://github.com/JuliaLang/julia/issues/8200. >> > >> > >> > On Sun, Aug 31, 2014 at 7:45 PM, Dan Luu <[email protected]> wrote: >> >> >> >> I'm also having problems, and I wonder if I've run into the same >> issue. >> >> >> >> When I updated Julia today on my Mac (10.9.2), I got the following >> error: >> >> >> >> /bin/sh: line 1: 23089 Segmentation fault: 11 >> >> /Users/danluu/dev/julia/usr/bin/julia --build >> >> /Users/danluu/dev/julia/usr/lib/julia/sys >> >> -J/Users/danluu/dev/julia/usr/lib/julia/$([ -e >> >> /Users/danluu/dev/julia/usr/lib/julia/sys.ji ] && echo sys.ji || echo >> >> sys0.ji) -f sysimg.jl >> >> * This error is usually fixed by running 'make clean'. If the error >> >> persists, try 'make cleanall'. * >> >> make[1]: * [/Users/danluu/dev/julia/usr/lib/julia/sys.o] Error 1 >> >> make: * [release] Error 2 >> >> >> >> I've tried doing make cleanall, and even wiping out my repository and >> >> re-cloning in case it's a problem with deps, and I still get the same >> >> error. >> >> >> >> On Linux (64-bit, 3.2.0-65-generic), the build doesn't error out, but >> >> Julia segfaults on startup. The gdb backtrace for that is: >> >> Program received signal SIGSEGV, Segmentation fault. >> >> 0x00007ffff6e2328c in jl_deserialize_gv (v=0x7bb138, s=0x7fffffffdcc0) >> >> at dump.c:145 >> >> 145 *sysimg_gvars[gvname_index] = v; >> >> (gdb) bt >> >> #0 0x00007ffff6e2328c in jl_deserialize_gv (v=0x7bb138, >> >> s=0x7fffffffdcc0) at dump.c:145 >> >> #1 jl_deserialize_value_internal (s=0x7fffffffdcc0) at dump.c:854 >> >> #2 0x00007ffff6e233e5 in jl_deserialize_value (s=0x7fffffffdcc0) at >> >> dump.c:950 >> >> #3 jl_deserialize_value_internal (s=0x7fffffffdcc0) at dump.c:937 >> >> #4 0x00007ffff6e2350d in jl_deserialize_value (s=0x7fffffffdcc0) at >> >> dump.c:950 >> >> #5 jl_deserialize_datatype (pos=403560, s=0x7fffffffdcc0) at >> dump.c:646 >> >> #6 jl_deserialize_value_internal (s=0x7fffffffdcc0) at dump.c:886 >> >> #7 0x00007ffff6e22818 in jl_deserialize_value (s=0x7fffffffdcc0) at >> >> dump.c:950 >> >> #8 jl_deserialize_value_internal (s=0x7fffffffdcc0) at dump.c:715 >> >> ... >> >> #134 jl_deserialize_value_internal (s=0x7fffffffdcc0) at dump.c:715 >> >> #135 0x00007ffff6e233e5 in jl_deserialize_value (s=0x7fffffffdcc0) at >> >> dump.c:950 >> >> #136 jl_deserialize_value_internal (s=0x7fffffffdcc0) at dump.c:937 >> >> #137 0x00007ffff6e233e5 in jl_deserialize_value (s=0x7fffffffdcc0) at >> >> dump.c:950 >> >> #138 jl_deserialize_value_internal (s=0x7fffffffdcc0) at dump.c:937 >> >> #139 0x00007ffff6e23881 in jl_deserialize_value (s=0x7fffffffdcc0) at >> >> dump.c:950 >> >> #140 jl_restore_system_image (fname=<optimized out>) at dump.c:1060 >> >> #141 0x00007ffff6e1f33b in julia_init ( >> >> imageFile=0x608e60 >> >> "/home/dluu/dev/julia/usr/bin/../lib/julia/sys.ji") at init.c:826 >> >> #142 0x000000000040140a in main (argc=0, argv=0x7fffffffe1c0) at >> >> repl.c:378 >> >> >> >> >> >> >> >> On Sun, Aug 31, 2014 at 8:39 AM, Andrea Vigliotti >> >> <[email protected]> wrote: >> >> > Hi all! >> >> > >> >> > >> >> > I am having problems in updating Julia from the git. As usual, every >> >> > three >> >> > four days I download the last updates from the git and >> >> > compile them, this is what I do (I'm running ubuntu with KDE, and >> Julia >> >> > is >> >> > v0.4) from the source directory I typed >> >> > >> >> > git pull && make >> >> > >> >> > >> >> > >> >> > then I got this >> >> > >> >> > ... >> >> > ... >> >> > ... >> >> > iterator.jl >> >> > inference.jl >> >> > ERROR: >> >> > >> >> > >> LoadError("/usr/local/julia/v0.4/base/sysimg.jl",65,LoadError("inference.jl",134,UndefVarError(:sizeof))) >> >> >> >> > in include at ./boot.jl:245 (repeats 2 times) >> >> > in include_from_node1 at loading.jl:128 >> >> > in process_options at ./client.jl:285 >> >> > in _start at ./client.jl:354 >> >> > in _start_3B_13569 at /usr/local/julia/v0.4/usr/lib/julia/sys.so >> >> > >> >> > *** This error is usually fixed by running 'make clean'. If the >> error >> >> > persists, try 'make cleanall'. *** >> >> > make[1]: *** [/usr/local/julia/v0.4/usr/lib/julia/sys.o] Error 1 >> >> > make: *** [release] Error 2 >> >> > >> >> > >> >> > after doing the make cleanall, tried make again and got >> >> > >> >> > ... >> >> > ... >> >> > ... >> >> > /usr/bin/install -c -m 644 '_U_dyn_register.man' >> >> > '/usr/local/julia/v0.4/usr/share/man/man3/_U_dyn_register.3' >> >> > /usr/bin/install -c -m 644 '_U_dyn_cancel.man' >> >> > '/usr/local/julia/v0.4/usr/share/man/man3/_U_dyn_cancel.3' >> >> > /usr/bin/install -c -m 644 include/libunwind-dynamic.h >> >> > include/libunwind-ptrace.h include/libunwind-coredump.h >> >> > include/libunwind-x86_64.h include/libunwind.h include/unwind.h >> >> > '/usr/local/julia/v0.4/usr/include' >> >> > /usr/bin/install -c -m 644 include/libunwind-common.h >> >> > '/usr/local/julia/v0.4/usr/include' >> >> > Makefile:141: /Makefile.rules: No such file or directory >> >> > make[3]: *** No rule to make target `/Makefile.rules'. Stop. >> >> > make[2]: *** [/usr/local/julia/v0.4/usr/lib/libLLVMJIT.a] Error 2 >> >> > make[1]: *** [julia-release] Error 2 >> >> > make: *** [release] Error 2 >> >> > >> >> > >> >> > now I'm stucked, I looked on the internet but could find reason or >> >> > solutions >> >> > for this, it happened before and I had to remove everything and >> download >> >> > and >> >> > make everything from scratch, >> >> > >> >> > does anybody know how it is possible to fix this without completely >> >> > reinstalling Julia?? >> >> > >> >> > thanks! >> >> > >> >> > andrea >> > >> > >> >
