Re: [Savannah-hackers-public] Web CVS server unavailable

[Ludovic Courtès]

Bob Proulx  skribis:

> Ludovic Courtès wrote:
>> Bob Proulx skribis:
>> >   -r--r--r--+ 1 civodul guix 1243780406 Apr 19 11:07 index.html,v
>> >
>> > Oh my goodness.  That is a very large rcs file!
>> 
>> We have a solution to this problem (we no longer need the 7 MiB
>> index.html \o/),
>
> Yes.  Yay!  :-)
>
>> but now I can neither update nor remove index.html: CVS
>> fails out with “cannot allocate memory” on the server side.
>
> Right.  Same problem.  It a very large rcs file.
>
>> Could you nuke it somehow?  (History doesn’t matter.)
>
> It's gone.  (I moved it out of the way.)

Great, thanks!

So what we have now is a low-tech solution that has the advantage of not
requiring users to run JavaScript code and not requiring special
server-side support:

  https://www.gnu.org/software/guix/packages/

>> Also, it might be related, but updates to one the Guix web site CSS
>> files fail to lead to an update on the web site.  That is:
>
> I will lazily say that I believe that to be the same problem but this
> time on the www.gnu.org web site side of things where I can't look at
> it.
>
>> This works fine, but several hours later:
>> 
>>   wget -q -O - http://www.gnu.org/software/guix/static/base/css/packages.css 
>> |head
>> 
>> ... is still showing me the old version of the file.
>> 
>> Ideas?
>
> That large index.html,v file is gone now.  I triggered an update.  Did
> it update?

It did!

We’re all set now.

Thank you, Bob!

Ludo’.

Re: [Savannah-hackers-public] Web CVS server unavailable

[Bob Proulx]

Ludovic Courtès wrote:
> Bob Proulx skribis:
> >   -r--r--r--+ 1 civodul guix 1243780406 Apr 19 11:07 index.html,v
> >
> > Oh my goodness.  That is a very large rcs file!
> 
> We have a solution to this problem (we no longer need the 7 MiB
> index.html \o/),

Yes.  Yay!  :-)

> but now I can neither update nor remove index.html: CVS
> fails out with “cannot allocate memory” on the server side.

Right.  Same problem.  It a very large rcs file.

> Could you nuke it somehow?  (History doesn’t matter.)

It's gone.  (I moved it out of the way.)

> Also, it might be related, but updates to one the Guix web site CSS
> files fail to lead to an update on the web site.  That is:

I will lazily say that I believe that to be the same problem but this
time on the www.gnu.org web site side of things where I can't look at
it.

> This works fine, but several hours later:
> 
>   wget -q -O - http://www.gnu.org/software/guix/static/base/css/packages.css 
> |head
> 
> ... is still showing me the old version of the file.
> 
> Ideas?

That large index.html,v file is gone now.  I triggered an update.  Did
it update?  I forgot to look before I triggered it and I don't know
what to look for afterward to tell if it worked or not.

Please check it again now.  If it was the large ,v file then the
problem should be resolved now.  If it isn't then it is something else
and will need Ruben to look behind the scenes.

Bob

Re: [Savannah-hackers-public] Web CVS server unavailable

[Ludovic Courtès]

Hello!

Bob Proulx  skribis:

> Ludovic Courtès wrote:
>> This particular file takes 7 MiB and has a large history (it is updated
>> daily), which perhaps makes things worse.
>
> Oh!  I apologize that I didn't fully investigate this with the
> previous exchange.  Because vcs often gets overloaded I assumed that
> was the problem again.  Because when I looked it was all very happy at
> that time.  That was bad of me.  Sorry.
>
> But reading this I now realize that it is a somewhat different
> problem.
>
>   vcs:/srv/cvs/web/guix/guix# du -sh
>   1.9G.
>
> Wow!  1.9G of web page!  That is a LOT.  Most of it in one file.
>
>   vcs:/srv/cvs/web/guix/guix/packages# ll
>   total 1267580
>   -r--r--r--+ 1 civodul guix 1243780406 Apr 19 11:07 index.html,v
>   -r--r--r--+ 1 civodul guix   54200311 Apr 14 01:29 issues.html,v
>
> Oh my goodness.  That is a very large rcs file!

We have a solution to this problem (we no longer need the 7 MiB
index.html \o/), but now I can neither update nor remove index.html: CVS
fails out with “cannot allocate memory” on the server side.

Could you nuke it somehow?  (History doesn’t matter.)

Also, it might be related, but updates to one the Guix web site CSS
files fail to lead to an update on the web site.  That is:

--8<---cut here---start->8---
$ cvs ci -m Update. static/base/css
cvs commit: Examining static/base/css
/web/guix/guix/static/base/css/packages.css,v  <--  static/base/css/packages.css
new revision: 1.5; previous revision: 1.4
Triggering webpages update...
--8<---cut here---end--->8---

This works fine, but several hours later:

  wget -q -O - http://www.gnu.org/software/guix/static/base/css/packages.css 
|head

… is still showing me the old version of the file.

Ideas?

Thanks,
Ludo’.

Re: [Savannah-hackers-public] Web CVS server unavailable

[Bob Proulx]

Ludovic Courtès wrote:
> In the meantime, I’ve changed the code that generates HTML to insert
> newlines; revision 1.294 of this file that I just committed is more
> RCS-friendly.
> 
> Feel free to remove history for that file if it’s possible!

Thanks!  I am planning on doing something about this as soon as I get
some free moments.  But if we ever converge about making this a
dynamic page elsewhere then this file can just be dropped without
doing anything.

> >> What about providing rsync/ssh access like for nongnu.org?  After all,
> >> HTML pages that get displayed are often not source nowadays, so people
> >> could use a separate repo for the source and simply rsync the HTML.
> >
> > Like for nongnu.org?  I have only been actively working on the system
> > for five years or so now.  I haven't had time to learn *everything*
> > that is happening everywhere upon it.  :-)  To what access for
> > nongnu.org are you referring to?
> 
> My mistake: nongnu.org releases are uploaded over scp, but web pages are
> uploaded over CVS as for gnu.org:
> 
>   https://savannah.gnu.org/maintenance/HomepageUpload/
>   https://savannah.gnu.org/maintenance/DownloadArea/

Right.

>>Bob Proulx wrote:
>>> What is the input source feeding into that script?

You didn't answer my question about what inputs go into your script.
If it is something within my control I might be able to help set up a
dynamic page for you.  But if I don't know then I can't even think
about it.

Bob

Re: [Savannah-hackers-public] Web CVS server unavailable

[Ludovic Courtès]

Bob Proulx  skribis:

> Ludovic Courtès wrote:
>> Ouch, I didn’t know CVS was this bad.  Sorry that it causes problems!
>
> Just trying to make some of the invisible workings visible. :-)
>
>> I can change the program that generates it to include newlines, but we’d
>> still have to remove history.  Would it help?
>
> Yes.  Is the history for this file important?

Not at all.

> But wait!  Because I suggest we convert that page to a dynamically
> created and cached page instead.  What do you think?

Sure, I’ll email the sysadmins as suggested and see what they think.

In the meantime, I’ve changed the code that generates HTML to insert
newlines; revision 1.294 of this file that I just committed is more
RCS-friendly.

Feel free to remove history for that file if it’s possible!

>> What about providing rsync/ssh access like for nongnu.org?  After all,
>> HTML pages that get displayed are often not source nowadays, so people
>> could use a separate repo for the source and simply rsync the HTML.
>
> Like for nongnu.org?  I have only been actively working on the system
> for five years or so now.  I haven't had time to learn *everything*
> that is happening everywhere upon it.  :-)  To what access for
> nongnu.org are you referring to?

My mistake: nongnu.org releases are uploaded over scp, but web pages are
uploaded over CVS as for gnu.org:

  https://savannah.gnu.org/maintenance/HomepageUpload/
  https://savannah.gnu.org/maintenance/DownloadArea/

Thank you,
Ludo’.

Re: [Savannah-hackers-public] Web CVS server unavailable

[Ludovic Courtès]

John Sullivan  skribis:

> l...@gnu.org (Ludovic Courtès) writes:
>
>> I emailed this list because… the FSF sysadmin (Ineiev) told me they
>> cannot do anything about my CVS issue (RT ticket # 1102191).
>>
>
> Ineiev is a GNU webmaster and translation coordinator, not an FSF
> sysadmin. To contact our sysadmins, it's sysad...@gnu.org.
> webmast...@gnu.org goes primarily to the gnu.org volunteer webmaster
> team.

Oops, my bad, sorry for the confusion.

I’ll email them right away.

Thank you,
Ludo’.

Re: [Savannah-hackers-public] Web CVS server unavailable

[John Sullivan]

l...@gnu.org (Ludovic Courtès) writes:

> I emailed this list because… the FSF sysadmin (Ineiev) told me they
> cannot do anything about my CVS issue (RT ticket # 1102191).
>

Ineiev is a GNU webmaster and translation coordinator, not an FSF
sysadmin. To contact our sysadmins, it's sysad...@gnu.org.
webmast...@gnu.org goes primarily to the gnu.org volunteer webmaster
team.

-john

-- 
John Sullivan | Executive Director, Free Software Foundation
GPG Key: 61A0963B | http://status.fsf.org/johns | http://fsf.org/blogs/RSS

Do you use free software? Donate to join the FSF and support freedom at
.

Re: [Savannah-hackers-public] Web CVS server unavailable

[Ludovic Courtès]

Karl Berry  skribis:

> In addition to what Bob said ...
>
> What about providing rsync/ssh access like for nongnu.org?  
>
> For the umpteenth time: www.gnu.org (and ftp.gnu.org and alpha.gnu.org
> and ...) is controlled by FSF sysadmin.  Not us Savannah volunteers.  We
> have no more access to it than you do.  There are plenty of possible
> solutions, but we cannot implement them.  You want changes, talk to
> them.

I emailed this list because… the FSF sysadmin (Ineiev) told me they
cannot do anything about my CVS issue (RT ticket #1102191).

> It’s clear that CVS is a hindrance for such purposes so if there’s
> another possibility,
>
> Ludo, why must we go through this yet again?  Surely you know perfectly
> well by now that there is no other possibility at present.  And if any
> changes happen, they surely will not happen quickly.  So fix your tool
> -- just running the output through fmt comes to mind, HTML being what it
> is.  Or something like that.

Sure, I’m fine with such a solution, as I wrote earlier.

> Or: stop hosting your web pages through savannah -- having your own site
> at guix.gnu.org comes to mind.  Talk to the FSF about it.  There is
> nothing we can do to set that up.  Good luck.

So you’re asking GNU packages to host their web pages elsewhere.

I think this is detrimental to GNU.

Ludo’.

Re: [Savannah-hackers-public] Web CVS server unavailable

[Alfred M. Szmidt]

Maybe incorperate it with the Free Software Directory somehow, there
could be an icon showing that a package is available in Guix; and
which version. 

Re: [Savannah-hackers-public] Web CVS server unavailable

[Assaf Gordon]

Hello,

On 04/21/2016 02:35 PM, Bob Proulx wrote:

It is a very *heavy* file.  It took a significant amount of wait time
on my browser for it to load.

[...]

This makes me wonder how useful it is in the present state.  Are
people really needing the entire state of the entire system all in one
page?


A suggestion:

If this a file that is commonly updated (perhaps 
automatically/programmatically),
such as a list of available packages, perhaps consider putting it on the 
downloads server ?

Files can be uploaded with rsync/scp,
there's no versioning, and files will also be mirrored (usually within 24 
hours).

Example (a dummy file I just created):
   http://download-mirror.savannah.gnu.org/releases/guix/
   http://download-mirror.savannah.gnu.org/releases/guix/packages.html

   $ rsync rsync://dl.sv.gnu.org/releases/guix/
   drwxrwsr-x  4,096 2016/04/21 14:57:21 .
   -rw-rw-r-- 57 2016/04/21 14:55:01 packages.html

Access permissions to this directory are the same as commit-access to the guix 
repository, e.g.

   rsync -avhP LOCALFILE civo...@dl.sv.gnu.org:/srv/download/guix/


NOTE:
When visiting http://download.savannah.gnu.org/releases/guix/
you'll be redirected to one of the mirrors - which might not yet have the most 
up-to-date file.
to skip the redirection (not recommended), use  
http://download-mirror.savannah.gnu.org/releases/guix/ .


regards,
 - assaf

Re: [Savannah-hackers-public] Web CVS server unavailable

[Bob Proulx]

Alfred M. Szmidt wrote:
> Not to mention that it isn't a problem with CVS; git would have
> similar issues down the line; and the file as such is heavy for the
> network pipe on both ends -- it is a 7MiB download.

It is a very *heavy* file.  It took a significant amount of wait time
on my browser for it to load.  I paged down as it was loading.  And
paged down.  And paged down.  It took a while.

  time wget http://www.gnu.org/software/guix/packages/
...
index.html [   <=>]  6.77M  51.4KB/s in 3m 55s 
2016-04-21 12:32:19 (29.6 KB/s) - ‘index.html’ saved [7098898]
real3m55.844s
user0m0.092s
sys 0m0.216s

This makes me wonder how useful it is in the present state.  Are
people really needing the entire state of the entire system all in one
page?  Probably not.  Seems like the kind of thing people would only
be interested in a much smaller set of information.  Such as only
packages they own.  Or only packages of a certain name.  It definitely
needs something different than it is now.  Just gzip'ing the file
would be a huge improvement.  www.gnu.org doesn't offer transport
compression.

  gzip -v index.html
index.html:  90.1% -- replaced with index.html.gz

Bob

Re: [Savannah-hackers-public] Web CVS server unavailable

[Alfred M. Szmidt]

   It’s clear that CVS is a hindrance for such purposes so if there’s
   another possibility,

   Ludo, why must we go through this yet again?  Surely you know perfectly
   well by now that there is no other possibility at present.  

Not to mention that it isn't a problem with CVS; git would have
similar issues down the line; and the file as such is heavy for the
network pipe on both ends -- it is a 7MiB download.

Re: [Savannah-hackers-public] Web CVS server unavailable

[Karl Berry]

In addition to what Bob said ...

What about providing rsync/ssh access like for nongnu.org?  

For the umpteenth time: www.gnu.org (and ftp.gnu.org and alpha.gnu.org
and ...) is controlled by FSF sysadmin.  Not us Savannah volunteers.  We
have no more access to it than you do.  There are plenty of possible
solutions, but we cannot implement them.  You want changes, talk to
them.

Itœôòùs clear that CVS is a hindrance for such purposes so if thereœôòùs
another possibility,

Ludo, why must we go through this yet again?  Surely you know perfectly
well by now that there is no other possibility at present.  And if any
changes happen, they surely will not happen quickly.  So fix your tool
-- just running the output through fmt comes to mind, HTML being what it
is.  Or something like that.

Or: stop hosting your web pages through savannah -- having your own site
at guix.gnu.org comes to mind.  Talk to the FSF about it.  There is
nothing we can do to set that up.  Good luck.

Karl

Re: [Savannah-hackers-public] Web CVS server unavailable

[Bob Proulx]

Ludovic Courtès wrote:
> Ouch, I didn’t know CVS was this bad.  Sorry that it causes problems!

Just trying to make some of the invisible workings visible. :-)

> I can change the program that generates it to include newlines, but we’d
> still have to remove history.  Would it help?

Yes.  Is the history for this file important?  If it is then we can
certainly preserve it.  But I think for this file we could choose to
trim it down.  It looks like we could start again and it would be okay
for this file.

But wait!  Because I suggest we convert that page to a dynamically
created and cached page instead.  What do you think?

> What about providing rsync/ssh access like for nongnu.org?  After all,
> HTML pages that get displayed are often not source nowadays, so people
> could use a separate repo for the source and simply rsync the HTML.

Like for nongnu.org?  I have only been actively working on the system
for five years or so now.  I haven't had time to learn *everything*
that is happening everywhere upon it.  :-)  To what access for
nongnu.org are you referring to?

> > How is the data on that page generated?
> 
> Using
> .

What is the input source feeding into that script?

> Can www.gnu.org host such pages at all?

Only if an FSF admin sets it up.  We have no access there.  However I
doubt that is the best place to host this script.  It depends upon the
source of the data feeding into it.  Is that on download.sv.gnu.org?

> It’s clear that CVS is a hindrance for such purposes so if there’s
> another possibility, I’m all for it!  :-)

I am going to presume (likely wrongly, but just for the purpose of
furthering this discussion) that the input source is on download.  In
which case that would be the best place for that script to generate
the information page.  We can set that up there.  For example the
mirror processing and status pages are hosted there.

Bob

Re: [Savannah-hackers-public] Web CVS server unavailable

[Ludovic Courtès]

Bob Proulx  skribis:

> There are 293 versions checked in.  Which means that pretty much every
> version is a complete copy of the file since line based diffs are
> saved and there is only one line.  That isn't going to be
> sustainable.  The actual root error is this one.

Ouch, I didn’t know CVS was this bad.  Sorry that it causes problems!

I can change the program that generates it to include newlines, but we’d
still have to remove history.  Would it help?

> But I wanted to say that Assaf Gordon has spent a significant amount
> of time trying to improve the web page system on the savannah.gnu.org
> side of things.  Primarily to enable use of git for web page updates.

What about providing rsync/ssh access like for nongnu.org?  After all,
HTML pages that get displayed are often not source nowadays, so people
could use a separate repo for the source and simply rsync the HTML.

> But there are limits to what we can do on our (savannah) side of
> things.  Presently his efforts are waiting for help on the fsf.org
> side of things.  The improvements are stuck until more resources are
> available to help there.

:-(

> My feedback is that something different needs to be done for this
> index.html page.  I don't want to suggest something without thinking
> about it.  I might suggest something worse!  But the first thought I
> had was that this needs some kind of dedicated status page.  This
> shouldn't really be checked into cvs.  This looks to be some type of
> real time dynamically created status page that shows the information
> you want.  I would put this on a different location entirely.  Perhaps
> a dedicated page?  That seems like the way it should go.  If you want
> to have it displayed right where it is displayed then an  can
> be used to place it there.
>
> How is the data on that page generated?

Using
.

Can www.gnu.org host such pages at all?  It’s clear that CVS is a
hindrance for such purposes so if there’s another possibility, I’m all
for it!  :-)

Thank you,
Ludo’.

Re: [Savannah-hackers-public] Web CVS server unavailable

[Bob Proulx]

> There are 293 versions checked in.  Which means that pretty much every
> version is a complete copy of the file since line based diffs are
> saved and there is only one line.  That isn't going to be
> sustainable.  The actual root error is this one.
> 
>   vcs:/tmp/junk# cvs -d /srv/cvs/web/guix co guix/packages
>   cvs checkout: Updating guix/packages
>   cvs [checkout aborted]: Could not map memory to RCS archive
>   /srv/cvs/web/guix/guix/packages/index.html,v: Cannot allocate memory
> 
> vcs has 6 gig of virtual memory available.  This must be exceeding a
> different limit.  But this is why the web page isn't updating.  It is
> hard stuck and isn't going to make any more progress until something
> changes.

I noticed that one of the swap partitions was offline.  Apparently
since the move to the new storage array a couple of weeks ago.  Fixed
that and added it back online bringing vcs from 6 gig to 12 gig of
virtual memory available.

Apparently cvs needs more than 6 gig of space to work on that file!
Because 6 wasn't enough but 12 now is enough.  I could check out that
file.  I will trigger the web page to try an update.  I expect it will
update now.

Bob

Re: [Savannah-hackers-public] Web CVS server unavailable

[Bob Proulx]

Ludovic Courtès wrote:
> This particular file takes 7 MiB and has a large history (it is updated
> daily), which perhaps makes things worse.

Oh!  I apologize that I didn't fully investigate this with the
previous exchange.  Because vcs often gets overloaded I assumed that
was the problem again.  Because when I looked it was all very happy at
that time.  That was bad of me.  Sorry.

But reading this I now realize that it is a somewhat different
problem.

  vcs:/srv/cvs/web/guix/guix# du -sh
  1.9G.

Wow!  1.9G of web page!  That is a LOT.  Most of it in one file.

  vcs:/srv/cvs/web/guix/guix/packages# ll
  total 1267580
  -r--r--r--+ 1 civodul guix 1243780406 Apr 19 11:07 index.html,v
  -r--r--r--+ 1 civodul guix   54200311 Apr 14 01:29 issues.html,v

Oh my goodness.  That is a very large rcs file!

Looking in the file I see that the main body of the web page is all on
*one* line.  There are 29 lines in total.  Wow.  If it were newline'd
normally that would probably have made a huge reduction in the diff
sizes.

  wget -O-.index.html https://www.gnu.org/software/guix/packages/ | wc 
index.html
 29  427729 7002030

There are 293 versions checked in.  Which means that pretty much every
version is a complete copy of the file since line based diffs are
saved and there is only one line.  That isn't going to be
sustainable.  The actual root error is this one.

  vcs:/tmp/junk# cvs -d /srv/cvs/web/guix co guix/packages
  cvs checkout: Updating guix/packages
  cvs [checkout aborted]: Could not map memory to RCS archive
  /srv/cvs/web/guix/guix/packages/index.html,v: Cannot allocate memory

vcs has 6 gig of virtual memory available.  This must be exceeding a
different limit.  But this is why the web page isn't updating.  It is
hard stuck and isn't going to make any more progress until something
changes.

> Also, I’m under the impression that web page updates are queued.  For
> instance the timestamp at the bottom of the page above says “April 14”,
> whereas the commit from now succeeded (and so it should read “April
> 19”.)

Normally they are updated reasonably quickly.  But the above problem
has this stuck and is why it isn't updating at the moment.  It can't
be checked out.

Normally there is a convoluted Rube Goldberg machine that hits a URL
on the FSF machine which touches a file which then runs a script and
sees the flag files telling it to do a cvs update right now on the
page.  That usually flows through in the amount of time it takes the
machine to operate and check out the file.  (This was broken about a
month ago and fixed and therefore I believe it is operating now.)  So
normally there is no queue as such but not quite immediately.  There
are propagation delays in the mirroring.

> Regardless, if GNU packages and other projects hosted on Savannah can no
> longer update their web pages, then it’s the beginning of the end.  We
> should make sure the FSF shares this conclusion and that the issue will
> be addressed in a timely fashion, IMO.

Obviously my first feedback was wrong.  Sorry.  And I know that led to
the above which is cascade failure on the top of my misdirection.  So
I will ignore that.

But I wanted to say that Assaf Gordon has spent a significant amount
of time trying to improve the web page system on the savannah.gnu.org
side of things.  Primarily to enable use of git for web page updates.
But there are limits to what we can do on our (savannah) side of
things.  Presently his efforts are waiting for help on the fsf.org
side of things.  The improvements are stuck until more resources are
available to help there.

> Thanks for your feedback!

My feedback is that something different needs to be done for this
index.html page.  I don't want to suggest something without thinking
about it.  I might suggest something worse!  But the first thought I
had was that this needs some kind of dedicated status page.  This
shouldn't really be checked into cvs.  This looks to be some type of
real time dynamically created status page that shows the information
you want.  I would put this on a different location entirely.  Perhaps
a dedicated page?  That seems like the way it should go.  If you want
to have it displayed right where it is displayed then an  can
be used to place it there.

How is the data on that page generated?

Bob

Re: [Savannah-hackers-public] Web CVS server unavailable

[Ludovic Courtès]

Bob Proulx  skribis:

> Ludovic Courtès wrote:
>> It seems the www CVS server is currently in a bad shape:
>> 
>> --8<---cut here---start->8---
>> $ cvs up index.html
>> cvs [update aborted]: Could not map memory to RCS archive 
>> /web/guix/guix/packages/index.html,v: Cannot allocate memory
>> --8<---cut here---end--->8---
>
> This is a long running resource problem with vcs.  The outside world
> can sometimes overwhelm the machine with too many things happening.
> The FSF has informed us that new hardware is very soon to be
> available.  But as of yet we don't have it yet.  And frankly there is
> no guarentee that the new hardware will be sufficient yet.  I worry it
> won't be enough.

This is worrisome, indeed.

This particular file takes 7 MiB and has a large history (it is updated
daily), which perhaps makes things worse.  I managed to commit an update
just now, maybe a sign that the machine is in a better state?

  https://www.gnu.org/software/guix/packages/

Also, I’m under the impression that web page updates are queued.  For
instance the timestamp at the bottom of the page above says “April 14”,
whereas the commit from now succeeded (and so it should read “April
19”.)

Regardless, if GNU packages and other projects hosted on Savannah can no
longer update their web pages, then it’s the beginning of the end.  We
should make sure the FSF shares this conclusion and that the issue will
be addressed in a timely fashion, IMO.

Thanks for your feedback!

Ludo’.

Re: [Savannah-hackers-public] Web CVS server unavailable

[Bob Proulx]

Hello Ludovic,

Ludovic Courtès wrote:
> It seems the www CVS server is currently in a bad shape:
> 
> --8<---cut here---start->8---
> $ cvs up index.html
> cvs [update aborted]: Could not map memory to RCS archive 
> /web/guix/guix/packages/index.html,v: Cannot allocate memory
> --8<---cut here---end--->8---

This is a long running resource problem with vcs.  The outside world
can sometimes overwhelm the machine with too many things happening.
The FSF has informed us that new hardware is very soon to be
available.  But as of yet we don't have it yet.  And frankly there is
no guarentee that the new hardware will be sufficient yet.  I worry it
won't be enough.

Looking at the system right now (Mon, 18 Apr 2016 20:20:54 -0600)
things look fine.  But I don't doubt that it had failed for you
because there are frequent droppouts.  And unfortunately when it drops
out the Munin data isn't able to record either.  

> Ideas?

My only advice is patience.  Please feel free to post to us here and
let us know when it isn't working.  Including a timestamp so that we
know when to look in logs.  But at this point all we can really do is
wait for the load to subside and try again.

Bob