The disk issue is fixed now, but one day later Gitlab is broken again. This time I'm getting random 502s.
I tired to check logs, I don't know what the problem is but I saw one error line which may be relevant Aug 11 12:21:09 gitlab.haskell.org unicorn[3605]: /nix/store/h6ppx34ccb3binw7awbphaicv5q938z5-ruby2.5.5-prometheus-client-mmap-0.9.8/lib/ruby/gems/2.5.0/gems/prometheus-client-mmap-0.9.8/lib/prometheus/client/mmaped_dict.rb:42: [BUG] Bus Error at 0x00007fe3cd6f6000 Ömer Ömer Sinan Ağacan <omeraga...@gmail.com>, 10 Ağu 2019 Cmt, 11:48 tarihinde şunu yazdı: > > Hi, > > Just yesterday Gitlab was giving 500 because the disk was full. Ben deleted > some > files, but in less than 24h it's full again. This started happening > regularly, I > wonder if we could do something about this. > > The reason this time seems to be that Gitlab started generating 22G-large > backups daily since the 7th. I'm not sure how important those backups are so > I'm > not deleting them. > > There's also a large docker-registry directory (101G). > > I think it might be good to set up some kind of downtime monitoring or maybe > something on the Gitlab server to send an email when the disk is nearly full. > It > could send an email to people who has access to the server. > > It'd also be good to come up with an action plan when this happens. I have > access to the server, but I have no idea which files are important. > Documenting > Gitlab setup (and the server details) in more details might be helpful. > > Does anyone have any other ideas to keep the server running? > > Ömer _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs