Good news: The server is back up. The installation and docker cache regeneration should succeed now.
-Marco On Fri, Jan 12, 2018 at 1:02 AM, Marco de Abreu < [email protected]> wrote: > While it's a quite harsh solution to delete the entire workspace, I think > that it's a good way. Git checkout takes between 2 and 10 seconds, so I > don't think we need to optimize in that regard. > > git clean is our 'soft' approach to clean up. Deleting the workspace is > the 'hard' approach, so this shouldn't be an issue. > > But there is one catch: Windows builds are not containerized and while we > delete the workspace, there could still be a lot of files which are not > being tracked. In future I'd like to have at least a file-system-layer in > between our tests and the host, but we will have to analyze if something > like this exists. At the moment, we even got tests writing to system32. > > -Marco > > On Fri, Jan 12, 2018 at 12:44 AM, Chris Olivier <[email protected]> > wrote: > >> Ok, but still on that note. I remember before that when some problems were >> being fixed in CI (before your time), they switched to deleting the entire >> source directory, ".git" subdirectory and all. At the time, the CI was in >> such an chaotic state that I didn't make an issue of it, but now that it >> has stabilized (for the most part, today's incident notwithstanding), I >> think that we may want to revisit it if it is still doing that. you >> could, >> for example, just delete everything except the .git directory and then do >> a >> 'git reset --hard' to get back a baseline before having to re-download >> everything every tim e(also should speed up the builds). >> >> Note that 'git clean' was not working as it doesn't delete 'unknown' >> directories, which was the problem. >> >> WDYT? >> >> On Thu, Jan 11, 2018 at 3:26 PM, Marco de Abreu < >> [email protected]> wrote: >> >> > This happens because we just merged the clang compilation >> > https://github.com/apache/incubator-mxnet/commit/ >> > 2b73aac527a3439ec0dc9b1e76c6df09ea347eb1. >> > This means that clang has to get installed on all slaves and after some >> > time, the docker images will be cached. The problem right now is that >> their >> > apt-server is unavailable, means the initial installation to create the >> > docker cache doesn't succeed. In future, this will be cached. >> > >> > -Marco >> > >> > On Thu, Jan 11, 2018 at 11:45 PM, Chris Olivier <[email protected]> >> > wrote: >> > >> > > do we download all submodules from scratch every build? if we do >> then >> > we >> > > should probably find a way not to suggest just doing git reset or >> > something >> > > like that >> > > >> > > >> > > >> > > On Thu, Jan 11, 2018 at 1:47 PM Marco de Abreu < >> > > [email protected]> >> > > wrote: >> > > >> > > > Hello, >> > > > >> > > > we're currently experiencing a CI outage caused by >> http://apt.llvm.org >> > > not >> > > > being reachable. >> > > > >> > > > Best regards, >> > > > Marco >> > > > >> > > >> > >> > >
