Re: Gitlab's disk full again
The disk issue is fixed now, but one day later Gitlab is broken again. This time I'm getting random 502s. I tired to check logs, I don't know what the problem is but I saw one error line which may be relevant Aug 11 12:21:09 gitlab.haskell.org unicorn[3605]: /nix/store/h6ppx34ccb3binw7awbphaicv5q938z5-ruby2.5.5-prometheus-client-mmap-0.9.8/lib/ruby/gems/2.5.0/gems/prometheus-client-mmap-0.9.8/lib/prometheus/client/mmaped_dict.rb:42: [BUG] Bus Error at 0x7fe3cd6f6000 Ömer Ömer Sinan Ağacan , 10 Ağu 2019 Cmt, 11:48 tarihinde şunu yazdı: > > Hi, > > Just yesterday Gitlab was giving 500 because the disk was full. Ben deleted > some > files, but in less than 24h it's full again. This started happening > regularly, I > wonder if we could do something about this. > > The reason this time seems to be that Gitlab started generating 22G-large > backups daily since the 7th. I'm not sure how important those backups are so > I'm > not deleting them. > > There's also a large docker-registry directory (101G). > > I think it might be good to set up some kind of downtime monitoring or maybe > something on the Gitlab server to send an email when the disk is nearly full. > It > could send an email to people who has access to the server. > > It'd also be good to come up with an action plan when this happens. I have > access to the server, but I have no idea which files are important. > Documenting > Gitlab setup (and the server details) in more details might be helpful. > > Does anyone have any other ideas to keep the server running? > > Ömer ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: PseudoOps in primops.txt.pp
This turned out to be rather lengthy and ambivalent, but my current TLDR;
of this is that GHC.Magic Ids could all be PseudoOps, because we don't use
their definitions anyway.
---
Regarding 2., the answer has been right before my eyes in the form of Note
[ghcPrimIds (aka pseudoops)] and Note [magicIds]. The most important
difference I guess is that we can give meaningful, yet forgetful
definitions for functions in GHC.Magic, whereas we can't for proper
pseudoops.
Note [ghcPrimIds (aka pseudoops)] also answers 3.: IIUC if PseudoOps aren't
free abstractions already (proxy#), we try to inline them immediately in
Core. For example `noinline`, which is never inlined, could never be a
PseudoOp. As a side note: `noinline` seems to lack proper handling in the
demand analyser. For example, `noinline id x` should be detected as strict
in `x` by unleashing `id`s strictness signature. Not sure if we currently
do that.
The "What are PseudoOps" part of 1. is thus mostly resolved: PseudoOps are
functions with special semantics that can be lowered or erased in Core or
STG, so we will never have to think about generating code for them. We
still need to treat them specially, because we have no way to encode their
semantics at the source level. Examples (these are all current PseudoOps):
- `seq` works on functions, but Haskell's `case` doesn't
- `proxy#` is a symbolic inhabitant of `Proxy#`, which will be erased in
code generation. I guess with -XUnliftedNewtypes we can finally define
`Proxy#` in source Haskell as `newtype Proxy# a = Proxy# (# #)`
- `unsafeCoerce#` can only be erased when going to STG, where we don't
type check as part of linting.
- `coerce` gets translated to casts as part of desugaring.
- `nullAddr#` get inlined immediately to corresponding literal in Core.
This is so that source syntax doesn't have to introduce a new literal.
Similarly, the definitions of GHC.Magic all seem to vanish after CorePrep.
In fact, I begin to think that GHC.Magic is just a subset of PseudoOps that
have semantics expressible in source Haskell (thus have a meaningful
definition). Which somewhat contradicts my observation above that
`noinline` couldn't be a PseudoOp: Clearly it could, because it is lowered
to id by the time we go to STG. This lowering (even in higher-order
situations, which is why we actually don't need the definition) seems to be
the whole point about having the compiler be aware of these special
identifiers.
So, for a concrete question: What are the reasons that we don't make i.e.
`lazy` a PseudoOp?
Am So., 11. Aug. 2019 um 12:42 Uhr schrieb Sebastian Graf <
sgraf1...@gmail.com>:
> Hey fellow devs,
>
> While implementing new PseudoOps, a couple of questions popped up:
>
>1. What are PseudoOps? When do we want to declare one? There doesn't
>seem to be any documentation around them. I only figured out that I
>probably want a PseudoOp by comparing to PrimOps I thought would be lowered
>at a similar stage (i.e. somewhere in Core or STG).
>2. Why aren't GHC.Magic.{lazy,noinline,oneShot} PseudoOps?
>3. Since we have to set all the IdInfo for `seq` and `noinline`
>manually, why is this incomplete? I.e., I'd expect a useful strictness
>signature and arity for both of these.
>
> Thanks!
> Sebastian
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
PseudoOps in primops.txt.pp
Hey fellow devs,
While implementing new PseudoOps, a couple of questions popped up:
1. What are PseudoOps? When do we want to declare one? There doesn't
seem to be any documentation around them. I only figured out that I
probably want a PseudoOp by comparing to PrimOps I thought would be lowered
at a similar stage (i.e. somewhere in Core or STG).
2. Why aren't GHC.Magic.{lazy,noinline,oneShot} PseudoOps?
3. Since we have to set all the IdInfo for `seq` and `noinline`
manually, why is this incomplete? I.e., I'd expect a useful strictness
signature and arity for both of these.
Thanks!
Sebastian
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Gitlab's disk full again
Hi Artem, I would say it's usual operations practice to keep N>1 backups of a system as assurance against corrupted backups. But maybe they could be stored on another server/service? Other suggestions: Gitlab stores both artifacts and caches for the CI pipelines. By default, archives are stored on the same machine as the GitLab service, creating the risk of resource contention. But there is an option to store them in an object storage service e.g. S3. The same goes for caches, but I think they are stored on the CI runner machine by default (is it separate from the GitLab machine?). Plus, caches are shared across many jobs while artifacts are unique to a job, so there are many less caches than artifacts. Still, it might be valuable to audit the use of both artifacts and caches. On Sun, 11 Aug 2019, 1.23 Artem Pelenitsyn, wrote: > Hello, > > Is there a reason to keep more than one backup of GitLab ever? > > -- > Best, Artem > > On Sat, Aug 10, 2019, 4:49 AM Ömer Sinan Ağacan > wrote: > >> Hi, >> >> Just yesterday Gitlab was giving 500 because the disk was full. Ben >> deleted some >> files, but in less than 24h it's full again. This started happening >> regularly, I >> wonder if we could do something about this. >> >> The reason this time seems to be that Gitlab started generating 22G-large >> backups daily since the 7th. I'm not sure how important those backups are >> so I'm >> not deleting them. >> >> There's also a large docker-registry directory (101G). >> >> I think it might be good to set up some kind of downtime monitoring or >> maybe >> something on the Gitlab server to send an email when the disk is nearly >> full. It >> could send an email to people who has access to the server. >> >> It'd also be good to come up with an action plan when this happens. I have >> access to the server, but I have no idea which files are important. >> Documenting >> Gitlab setup (and the server details) in more details might be helpful. >> >> Does anyone have any other ideas to keep the server running? >> >> Ömer >> ___ >> ghc-devs mailing list >> ghc-devs@haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
