Hi all, First of all: welcome to the wonderful world of EasyBuild Raik! And thanks for not giving up on it instantly after something went wrong...
Sometimes this problem occurs not because of a parallel build (like Fotis mentioned), but because you cancelled the 'eb' run because you realized you wanted to change something (e.g. enable debug logging, or whatever). If you cancel eb while it's downloading stuff, you'll corrupt the file that's being downloaded at that point. In that case: simply remove that file, start EB again. Stijn: it's not always possible to make EB clean up 'partial downloads', e.g. when it's cancelled or when the connection goes down (well, maybe then it is possible, depends). We can resort to MD5 sums though, but that's a huge effort, and I'm pretty sure there be dragons there too (e.g. upstream sources that get changed, like rgputools, WPS, ...). regards, Kenneth On 04 Dec 2013, at 10:05, Fotis Georgatos wrote: > > Hi Raik, > > On Dec 4, 2013, at 9:42 AM, <[email protected]> > <[email protected]> wrote: >> parse_cmd_output): cmd "tar xjf /home/eb/.local/easybuild/sources/g/GCC/ >> gmp-5.0.4.tar.bz2" exited with exitcode 2 and output: >> bzip2: (stdin) is not a bzip2 file. >> tar: Child returned status 2 >> tar: Error is not recoverable: exiting now >> --- >> >> Somebody an idea? > > I often get these kind of errors as well, it universally reduces to the > following situation: > * the cause is my bad, because if running multiple concurrent builds (via > xargs or parallel) > * I manually try to bunzip/untar, to verify that the sourceball is indeed > corrupted > * generally it is, so I simply remove it, and run eb exactly once, to > re-download it > * eb should go through the extract step unobstructed, that's your green light > * by this moment you can now spawn 10+ builds over the same source w/out > issues > > The good news are, that as you progress in the effort over multiple weeks, > and build your "source" dir, it becomes less and less probable to hit this > issue. > > Potential solutions for the community: > * (short-term) add md5/sha hashes, to warn the user gracefully about this > issue > * (long-term) use some scheduling of short that prevents parallel downloads > (ie. either make each write step atomic or, simply re-schedule builds) > > I am totally interested to heat if anybody has to propose a way > to avoid certain parallel builds, because it also solves some other issues > (some packages eg. NCBI vs BLAST do not like to be build alongside each other) > > cheers, > Fotis > > -- > echo "sysadmin know better bash than english" | sed s/min/mins/ \ > | sed 's/better bash/bash better/' # Yelling in a CERN forum > > > >
