Seems right to me, but I will have to investigate. I noted it down. -Marco
Am 12.01.2018 1:21 nachm. schrieb "Pedro Larroy" < [email protected]>: > I think Chris is right, git clean with the right options plus proper > initialization of the submodules should not make any difference versus > deleting the entire workspace. Right? > > On Fri, Jan 12, 2018 at 8:56 AM, kellen sunderland > <[email protected]> wrote: > > Doing a few searches I see that llvm.org <http://apt.llvm.org> doesn't > > appear to be stable enough for CI. I'm going to write something to > > hopefully make it a little more stable today, while still allowing those > at > > home to have easily reproducible build steps through docker. What I'd > > propose is we cache the 15 or so deb packages that get installed with > clang > > in s3 in the CI env. For home users who can't reach the cached s3 bucket > > we fall back to apt.llvm.org installation. Sound like a reasonable plan > > Marco? > > > > On Fri, Jan 12, 2018 at 8:21 AM, Marco de Abreu < > > [email protected]> wrote: > > > >> Aah I understand, you're right, we should revisit our decisions. I'll > put > >> it into the backlog so I don't forget it. > >> > >> -Marco > >> > >> Am 12.01.2018 2:48 vorm. schrieb "Chris Olivier" <[email protected] > >: > >> > >> Yeah, I'm just saying the whole delete was done as a drastic measure at > the > >> time. It may not be necessary do re-pull everything. Instead of deleting > >> everything, you could delete everything *except* the .git dir. and then > >> checkout the commit you want and it'll regenerate the sources from the > .git > >> database. > >> > >> This, of course, assuming the .git database is never wrong... If > something > >> goes wrong, you can nuke the whole dir. > >> > >> > >> On Thu, Jan 11, 2018 at 5:42 PM, Marco de Abreu < > >> [email protected]> wrote: > >> > >> > Exactly > >> > > >> > -Marco > >> > > >> > On Fri, Jan 12, 2018 at 2:40 AM, Chris Olivier <[email protected] > > > >> > wrote: > >> > > >> > > Actrually, this is the commit related to it. > >> > > https://github.com/cjolivier01/mxnet/commit/ > >> > 573a010879583885a0193e30dc0b8c > >> > > 848d80869b > >> > > > >> > > Before, the workspace directory wasn't being deleted. Now it is, > >> > correct? > >> > > Everything under the top directory, right? > >> > > > >> > > So a git clone re-pulls everything? > >> > > > >> > > On Thu, Jan 11, 2018 at 4:51 PM, Marco de Abreu < > >> > > [email protected]> wrote: > >> > > > >> > > > deleteDir() deletes the content of the current workspace > >> > > > > >> > > > Okay, I haven't seen any errors related to lua-package not being > >> > deleted. > >> > > > Do you have a CI-link by any chance? > >> > > > > >> > > > -Marco > >> > > > > >> > > > On Fri, Jan 12, 2018 at 1:49 AM, Chris Olivier < > >> [email protected]> > >> > > > wrote: > >> > > > > >> > > > > what is deleteDir() call doing in Jenkinsfile? > >> > > > > Yes, I mentioned the case where it wasn't getting cleaned. > >> > > > > > >> > > > > On Thu, Jan 11, 2018 at 4:41 PM, Marco de Abreu < > >> > > > > [email protected]> wrote: > >> > > > > > >> > > > > > During git_init: First we're just using git clean, if checkout > >> > fails, > >> > > > > we're > >> > > > > > deleting the entire workspace and retrying. > >> > > > > > > >> > > > > > During build: First we're using regular make. If build fails, > >> we're > >> > > > using > >> > > > > > make clean before executing make again. > >> > > > > > > >> > > > > > During test: No cleanup happening in case of failure. > >> > > > > > > >> > > > > > So far, I haven't noticed any files not being deleted in the > >> > > workspace. > >> > > > > Do > >> > > > > > you know an example? > >> > > > > > > >> > > > > > -Marco > >> > > > > > > >> > > > > > On Fri, Jan 12, 2018 at 1:34 AM, Chris Olivier < > >> > > [email protected]> > >> > > > > > wrote: > >> > > > > > > >> > > > > > > What approach is used now? I see in Jenkinsfile() that > >> > deleteDir() > >> > > > is > >> > > > > > > called at the top of init_git() and init_git_win(). That > >> > dele5tes > >> > > > the > >> > > > > > > whole directory, correct? > >> > > > > > > > >> > > > > > > Before there were problems with 'git clean -d -f' *not* > >> deleting > >> > > some > >> > > > > > > directories which were tracked on one branch and not on > >> another, > >> > > > which > >> > > > > I > >> > > > > > > believe is why deletDir() was put there. The directory I > recall > >> > was > >> > > > > > > something like lua-package or something that was in > someone's > >> > > private > >> > > > > > repo > >> > > > > > > or something like that... > >> > > > > > > > >> > > > > > > On Thu, Jan 11, 2018 at 4:02 PM, Marco de Abreu < > >> > > > > > > [email protected]> wrote: > >> > > > > > > > >> > > > > > > > While it's a quite harsh solution to delete the entire > >> > > workspace, I > >> > > > > > think > >> > > > > > > > that it's a good way. Git checkout takes between 2 and 10 > >> > > seconds, > >> > > > > so I > >> > > > > > > > don't think we need to optimize in that regard. > >> > > > > > > > > >> > > > > > > > git clean is our 'soft' approach to clean up. Deleting the > >> > > > workspace > >> > > > > is > >> > > > > > > the > >> > > > > > > > 'hard' approach, so this shouldn't be an issue. > >> > > > > > > > > >> > > > > > > > But there is one catch: Windows builds are not > containerized > >> > and > >> > > > > while > >> > > > > > we > >> > > > > > > > delete the workspace, there could still be a lot of files > >> which > >> > > are > >> > > > > not > >> > > > > > > > being tracked. In future I'd like to have at least a > >> > > > > file-system-layer > >> > > > > > in > >> > > > > > > > between our tests and the host, but we will have to > analyze > >> if > >> > > > > > something > >> > > > > > > > like this exists. At the moment, we even got tests > writing to > >> > > > > system32. > >> > > > > > > > > >> > > > > > > > -Marco > >> > > > > > > > > >> > > > > > > > On Fri, Jan 12, 2018 at 12:44 AM, Chris Olivier < > >> > > > > [email protected] > >> > > > > > > > >> > > > > > > > wrote: > >> > > > > > > > > >> > > > > > > > > Ok, but still on that note. I remember before that when > >> some > >> > > > > problems > >> > > > > > > > were > >> > > > > > > > > being fixed in CI (before your time), they switched to > >> > deleting > >> > > > the > >> > > > > > > > entire > >> > > > > > > > > source directory, ".git" subdirectory and all. At the > >> time, > >> > > the > >> > > > CI > >> > > > > > was > >> > > > > > > > in > >> > > > > > > > > such an chaotic state that I didn't make an issue of it, > >> but > >> > > now > >> > > > > that > >> > > > > > > it > >> > > > > > > > > has stabilized (for the most part, today's incident > >> > > > > > notwithstanding), I > >> > > > > > > > > think that we may want to revisit it if it is still > doing > >> > that. > >> > > > > you > >> > > > > > > > could, > >> > > > > > > > > for example, just delete everything except the .git > >> directory > >> > > and > >> > > > > > then > >> > > > > > > > do a > >> > > > > > > > > 'git reset --hard' to get back a baseline before having > to > >> > > > > > re-download > >> > > > > > > > > everything every tim e(also should speed up the builds). > >> > > > > > > > > > >> > > > > > > > > Note that 'git clean' was not working as it doesn't > delete > >> > > > > 'unknown' > >> > > > > > > > > directories, which was the problem. > >> > > > > > > > > > >> > > > > > > > > WDYT? > >> > > > > > > > > > >> > > > > > > > > On Thu, Jan 11, 2018 at 3:26 PM, Marco de Abreu < > >> > > > > > > > > [email protected]> wrote: > >> > > > > > > > > > >> > > > > > > > > > This happens because we just merged the clang > compilation > >> > > > > > > > > > https://github.com/apache/incubator-mxnet/commit/ > >> > > > > > > > > > 2b73aac527a3439ec0dc9b1e76c6df09ea347eb1. > >> > > > > > > > > > This means that clang has to get installed on all > slaves > >> > and > >> > > > > after > >> > > > > > > some > >> > > > > > > > > > time, the docker images will be cached. The problem > right > >> > now > >> > > > is > >> > > > > > that > >> > > > > > > > > their > >> > > > > > > > > > apt-server is unavailable, means the initial > installation > >> > to > >> > > > > create > >> > > > > > > the > >> > > > > > > > > > docker cache doesn't succeed. In future, this will be > >> > cached. > >> > > > > > > > > > > >> > > > > > > > > > -Marco > >> > > > > > > > > > > >> > > > > > > > > > On Thu, Jan 11, 2018 at 11:45 PM, Chris Olivier < > >> > > > > > > [email protected] > >> > > > > > > > > > >> > > > > > > > > > wrote: > >> > > > > > > > > > > >> > > > > > > > > > > do we download all submodules from scratch every > >> build? > >> > > if > >> > > > we > >> > > > > > do > >> > > > > > > > then > >> > > > > > > > > > we > >> > > > > > > > > > > should probably find a way not to suggest just doing > >> git > >> > > > reset > >> > > > > or > >> > > > > > > > > > something > >> > > > > > > > > > > like that > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > On Thu, Jan 11, 2018 at 1:47 PM Marco de Abreu < > >> > > > > > > > > > > [email protected]> > >> > > > > > > > > > > wrote: > >> > > > > > > > > > > > >> > > > > > > > > > > > Hello, > >> > > > > > > > > > > > > >> > > > > > > > > > > > we're currently experiencing a CI outage caused by > >> > > > > > > > > http://apt.llvm.org > >> > > > > > > > > > > not > >> > > > > > > > > > > > being reachable. > >> > > > > > > > > > > > > >> > > > > > > > > > > > Best regards, > >> > > > > > > > > > > > Marco > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> >
