The layout of #6 has served me well for a year now. It allows me to keep several such workspaces around to work with different Impala versions, I just have to be control carefully which workspace the minicluster runs from -- but that's not an issue here.
On Thu, Aug 31, 2017 at 2:44 AM, Philip Zeyliger <[email protected]> wrote: > On Wed, Aug 30, 2017 at 4:45 PM, Jim Apple <[email protected]> wrote: > > > The workspace cleanup isn't working - see the last bit of any recent > > ub1604 job: https://jenkins.impala.io/view/Utility/job/ubuntu-16.04- > > from-scratch/206/console > > > > 03:56:40.920 [WS-CLEANUP] Deleting project workspace...Cannot delete > > workspace :remote file operation failed: /home/ubuntu at > > hudson.remoting.Channel@4384d5b9:ubuntu-16.04 (i-032d527b9c801df4c): > > java.io.IOException: Unable to delete '/home/ubuntu'. Tried 3 times > > (of a maximum of 3) waiting 0.1 sec between attempts. > > 03:56:48.161 ERROR: Step ‘Delete workspace when build is done’ failed: > > Cannot delete workspace: remote file operation failed: /home/ubuntu at > > hudson.remoting.Channel@4384d5b9:ubuntu-16.04 (i-032d527b9c801df4c): > > java.io.IOException: Unable to delete '/home/ubuntu'. Tried 3 times > > (of a maximum of 3) waiting 0.1 sec between attempts. > > > > The workspace is $HOME, so you can't just delete it without being root. > > > > This could be changed to > > > > 1. A post-build script to "rm -rf ~/*". This doesn't reset everything, > > though - the job makes changes to other parts of the filesystem. > > > > 2. A post-build script to "sudo shutdown -h now" to make sure ec2 > > instances are not re-used. I'm not sure how Jenkins would feel about > > this. :-) > > > > 3. A post-build script to move $HOME to some archived location on the > > disk, to preserve debuggability. > > > > 4. A bash trap in the script to do one of the above. > > > > 5. Run the whole thing in a docker in the build machine, then delete > > the container when the script is done. Or don't, if there's enough > > disk space to not worry about that. > > > > 6. Do all of the work in a workspace inside $HOME. This would require > > some changes to bootstrap_development.sh. > > > > #5 is the most hermetic, I'd guess. > > > > I like #6 in the short term: I don't think anything is too bound to $HOME > except a few "~/" in that script which are easily approached. If you do #5, > you have to teach Jenkins about how to save the build output logs from > inside of Docker, which is, I think, more work. > > -- Philip > > > > > > On Thu, Aug 24, 2017 at 8:29 AM, Michael Brown <[email protected]> > wrote: > > > Looks like someone has done this. > > > > > > On Wed, Aug 23, 2017 at 8:16 PM, Alexander Behm < > [email protected]> > > > wrote: > > > > > >> Yes, let's please add the post-build action for sanity and consistency > > with > > >> our other jobs. > > >> > > >> On Wed, Aug 23, 2017 at 7:42 PM, Tim Armstrong < > [email protected] > > > > > >> wrote: > > >> > > >> > Maybe the workspace just got left in a weird state - I think in most > > >> cases > > >> > "git init" followed by checking out a branch and doing a clean would > > >> work. > > >> > > > >> > Should we add the delete workspace post-build action? > > >> > > > >> > On Wed, Aug 23, 2017 at 5:32 PM, Michael Brown <[email protected]> > > >> wrote: > > >> > > > >> > > Not a known issue. I noticed ubuntu-16.04-from-scratch is not set > to > > >> > clean > > >> > > up its workspace, and its config has not been touched since Aug > 11. > > It > > >> > > seems strange we only saw this now.... > > >> > > > > >> > > On Wed, Aug 23, 2017 at 5:25 PM, Tim Armstrong < > > >> [email protected]> > > >> > > wrote: > > >> > > > > >> > > > Is this a known problem? My job failed because the Impala repo > > >> already > > >> > > > existed on the machine: > > >> > > > > > >> > > > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/164/ > > >> > > > > > >> > > > *23:00:24* + /usr/bin/git init /home/ubuntu/Impala*23:00:24* > > >> > > > Reinitialized existing Git repository in > /home/ubuntu/Impala/.git/ > > >> > > > .... > > >> > > > *23:02:18* + for ITER in '$(seq 1 10)'*23:02:18* + echo > 'ATTEMPT: > > >> > > > 1'*23:02:18* ATTEMPT: 1*23:02:18* + /usr/bin/git checkout > > >> > > > FETCH_HEAD*23:02:18* + cat > > >> > > > /home/ubuntu/Impala/tmp.3tYBn0GUga*23:02:18* 23:02:18.712300 > > >> git.c:344 > > >> > > > trace: built-in: git 'checkout' > > 'FETCH_HEAD'*23:02:18* > > >> > > > error: The following untracked working tree files would be > > >> overwritten > > >> > > > by checkout:*23:02:18* .clang-format*23:02:18* > > >> > > > .clang-tidy*23:02:18* > > >> > > > .gitignore*23:02:18* CMakeLists.txt*23:02:18* > > >> > > > DISCLAIMER*23:02:18* > > >> > > > EXPORT_CONTROL.md*23:02:18* LICENSE.txt*23:02:18* > > >> > > > LOGS.md*23:02:18* > > >> > > > NOTICE.txt*23:02:18* README.md*23:02:18* > > >> > > > be/.gitignore*23:02:18* > > >> > > > be/.impala.doxy*23:02:18* > > be/CMakeLists.txt*23:02:18* > > >> > > > be/src/benchmarks/CMakeLists.txt*23:02:18* > > >> > > > be/src/benchmarks/atod-benchmark.cc*23:02:18* > > >> > > > be/src/benchmarks/atof-benchmark.cc*23:02:18* > > >> > > > be/src/benchmarks/atoi-benchmark.cc*23:02:18* > > >> > > > be/src/benchmarks/bit-packing-benchmark.cc*23:02:18* > > >> > > > be/src/benchmarks/bitmap-benchmark.cc > > >> > > > ... > > >> > > > > > >> > > > > >> > > > >> > > >
