Re: [Savannah-hackers-public] [gnu.org #912039] Migrate gnu.org/nongnu.org from CVS

[Sylvain Beucler via RT]

Hi,

On Mon, May 05, 2014 at 05:22:34PM -0400, Karl Berry via RT wrote:
> Second of all, I would be vehemently opposed to anything that 
> changes all web repos over to git, *forcing* people to use git.
> An option to use git would be one thing.  Forcing it, quite another.
> I doubt Sylvain had a forced switch in mind.

The idea appeals to me by its simplicity, but I expect this will cause
enough people as grumpy as you to track me down and roast me over a
celebration bonfire ;)

More seriously it should ease the migration *not* to change all
projects at once.

I also heard good feedback from github's static pages system, so I'll
have a look at it too.

> Third of all, I cannot envision any changes along these lines happening
> any time soon, since it's already been umpteen years since the original
> wishlist item, and the sysadmins and savannah volunteers have no more
> time than they ever did.  Hacking an update script is only a small part
> of the job.

I was about to say such a demotivator would turn into a
self-fulfilling prophecy, when I realized you gave everybody to
motivation to prove you wrong ;)

Cheers! 
Sylvain

Re: [Savannah-hackers-public] [gnu.org #912039] Migrate gnu.org/nongnu.org from CVS

[beuc]

Hi,

On Mon, May 05, 2014 at 05:22:34PM -0400, Karl Berry via RT wrote:
> Second of all, I would be vehemently opposed to anything that 
> changes all web repos over to git, *forcing* people to use git.
> An option to use git would be one thing.  Forcing it, quite another.
> I doubt Sylvain had a forced switch in mind.

The idea appeals to me by its simplicity, but I expect this will cause
enough people as grumpy as you to track me down and roast me over a
celebration bonfire ;)

More seriously it should ease the migration *not* to change all
projects at once.

I also heard good feedback from github's static pages system, so I'll
have a look at it too.

> Third of all, I cannot envision any changes along these lines happening
> any time soon, since it's already been umpteen years since the original
> wishlist item, and the sysadmins and savannah volunteers have no more
> time than they ever did.  Hacking an update script is only a small part
> of the job.

I was about to say such a demotivator would turn into a
self-fulfilling prophecy, when I realized you gave everybody to
motivation to prove you wrong ;)

Cheers! 
Sylvain

Re: [Savannah-hackers-public] [gnu.org #912039] Migrate gnu.org/nongnu.org from CVS

[Lisa Maginnis via RT]

Per your request on IRC, please see attached for a tarball of the requested 
files.

/home/wwwcvs/update-symlinks.sh and /home/wwwcvs/update-gnu-rewritemaps

Thanks :)

-- 
~Lisa Marie Maginnis
Senior System Administrator
Free Software Foundation
http://fsf.org http://gnu.org
GPG Key: 61EEC710




wwwcvs.tar.gz
Description: Binary data


signature.asc
Description: PGP signature

Re: [Savannah-hackers-public] [gnu.org #912039] Migrate gnu.org/nongnu.org from CVS

[Lisa Maginnis via RT]

Please see attached for update-cvs.sh (per your request over IRC).

Thanks :)

-- 
~Lisa Marie Maginnis
Senior System Administrator
Free Software Foundation
http://fsf.org http://gnu.org
GPG Key: 61EEC710




update-cvs.sh
Description: Bourne shell script


signature.asc
Description: PGP signature

Re: [Savannah-hackers-public] [gnu.org #912039] Migrate gnu.org/nongnu.org from CVS

[Karl Berry via RT]

John,

I believe the CVS checkout-by-cron system has been identified as a
major source of load on Savannah in the past. 

I am skeptical.  I have seen no evidence of such.  Or of anything except
overloaded hardware in general.  (It seems impossible to make progress
on this.  The problem continues unabated.  Very frustrating.)

With git's hook system (and better efficiency), this could help
address that current problem -- changes would be pushed when they
are made rather than doing polling.

Gee.  Must you introduce another recipe for inaction :)?

First of all, I see no fundamental reason why git would be more
efficient than cvs in this context.  (As Sylvain just replied.)

Second of all, I would be vehemently opposed to anything that 
changes all web repos over to git, *forcing* people to use git.
An option to use git would be one thing.  Forcing it, quite another.
I doubt Sylvain had a forced switch in mind.

Third of all, I cannot envision any changes along these lines happening
any time soon, since it's already been umpteen years since the original
wishlist item, and the sysadmins and savannah volunteers have no more
time than they ever did.  Hacking an update script is only a small part
of the job.

Please, don't wait for this to magically solve the load problem.

k

Re: [Savannah-hackers-public] [gnu.org #912039] Migrate gnu.org/nongnu.org from CVS

[Karl Berry]

John,

I believe the CVS checkout-by-cron system has been identified as a
major source of load on Savannah in the past. 

I am skeptical.  I have seen no evidence of such.  Or of anything except
overloaded hardware in general.  (It seems impossible to make progress
on this.  The problem continues unabated.  Very frustrating.)

With git's hook system (and better efficiency), this could help
address that current problem -- changes would be pushed when they
are made rather than doing polling.

Gee.  Must you introduce another recipe for inaction :)?

First of all, I see no fundamental reason why git would be more
efficient than cvs in this context.  (As Sylvain just replied.)

Second of all, I would be vehemently opposed to anything that 
changes all web repos over to git, *forcing* people to use git.
An option to use git would be one thing.  Forcing it, quite another.
I doubt Sylvain had a forced switch in mind.

Third of all, I cannot envision any changes along these lines happening
any time soon, since it's already been umpteen years since the original
wishlist item, and the sysadmins and savannah volunteers have no more
time than they ever did.  Hacking an update script is only a small part
of the job.

Please, don't wait for this to magically solve the load problem.

k

Re: [Savannah-hackers-public] [gnu.org #912039] Migrate gnu.org/nongnu.org from CVS

[Sylvain Beucler via RT]

On Mon, May 05, 2014 at 04:57:22PM -0400, John Sullivan via RT wrote:
> > [karl - Thu May 01 17:51:59 2014]:
> 
> > 3. regardless of all that, it would be a new and significant source of
> > complication on the savannah side. we already cannot come close to
> > keeping up with support requests.
> 
> I believe the CVS checkout-by-cron system has been identified as a major
> source of load on Savannah in the past. With git's hook system (and
> better efficiency), this could help address that current problem --
> changes would be pushed when they are made rather than doing polling.

This was fixed a few years ago, all updates are currently done
following a notification from a CVS hook (hence the "triggering
webpages update" output on commit).

-- 
Sylvain

Re: [Savannah-hackers-public] [gnu.org #912039] Migrate gnu.org/nongnu.org from CVS

[beuc]

On Mon, May 05, 2014 at 04:57:22PM -0400, John Sullivan via RT wrote:
> > [karl - Thu May 01 17:51:59 2014]:
> 
> > 3. regardless of all that, it would be a new and significant source of
> > complication on the savannah side. we already cannot come close to
> > keeping up with support requests.
> 
> I believe the CVS checkout-by-cron system has been identified as a major
> source of load on Savannah in the past. With git's hook system (and
> better efficiency), this could help address that current problem --
> changes would be pushed when they are made rather than doing polling.

This was fixed a few years ago, all updates are currently done
following a notification from a CVS hook (hence the "triggering
webpages update" output on commit).

-- 
Sylvain

[Savannah-hackers-public] [gnu.org #912039] Migrate gnu.org/nongnu.org from CVS

[John Sullivan via RT]

> [karl - Thu May 01 17:51:59 2014]:

> 3. regardless of all that, it would be a new and significant source of
> complication on the savannah side. we already cannot come close to
> keeping up with support requests.
> 

I believe the CVS checkout-by-cron system has been identified as a major
source of load on Savannah in the past. With git's hook system (and
better efficiency), this could help address that current problem --
changes would be pushed when they are made rather than doing polling.

-john

-- 
John Sullivan | Executive Director, Free Software Foundation
GPG Key: 61A0963B | http://status.fsf.org/johns | http://fsf.org/blogs/RSS

Do you use free software? Donate to join the FSF and support freedom at

[Savannah-hackers-public] [gnu.org #912039] Migrate gnu.org/nongnu.org from CVS

[Lisa Maginnis via RT]

> Staying on (obsolete) (unmaintained) CVS is also a bad idea :)
> I don't have read access so fencepost:/srv/data/www-config, can you
> check?

I've dumped a tarball of the Apache configs in your home directory:
/home/b/beuc/gnu.www-config.tar.gz
 
> @sysadmin: still willing to see the latest Apache configuration and as
> well as update scripts including http://www.gnu.org/new-savannah-
> project/new.py

Please see attached for a copy of new.py

Let me know if you need anything else!

Thank you for contributing time to look at this :)

Happy Hacking,

-- 
~Lisa Marie Maginnis
Senior System Administrator
Free Software Foundation
http://fsf.org http://gnu.org
GPG Key: 61EEC710


import os,sys
import string
from mod_python import apache, util
from syslog import *
import stat

# baughj, 2007.07.06 - add logging, usage of subprocess
# baughj, 2007.07.16 - make sure to chgrp/chmod checked out directory so that
# it can be updated by wwwcvs user later - why doesn't everything (cron job,
# etc) run as one user?
# add new 'translation' type for translation team webspace, see RT #348523.
# ward, 2008-08-18

# Stripped out actual cvs update, this script now just requests an update.
# See /usr/local/bin/update-cvs.sh for the actual cvs update code (and that 
# script is run from cron)
# ward, 2012-09-26

CHECKOUT = "/var/www/savannah-checkouts/"
LOGFILE = "/var/log/wwwcvs/new-savannah-project.log"

def touch(fname, times=None):
with file(fname, 'a'):
os.utime(fname, times)
os.chmod(fname, stat.S_IRUSR | stat.S_IWUSR | stat.S_IRGRP | stat.S_IWGRP | stat.S_IROTH | stat.S_IWOTH)

def request_update (type, project=''):
if type == 'www':
touch('/usr/local/to-update/www')
syslog(LOG_INFO, "Update requested for type: %s" % type)
return
touch('/usr/local/to-update/%s___%s' % (type, project))
syslog(LOG_INFO, "Update requested for type: %s (project %s)" % (type,project))
return

def handler (req):
openlog('savannah-update', 0, LOG_LOCAL6)
form = util.FieldStorage (req)

req.content_type = 'text/html'
req.send_http_header ()

type = None
project = None
if form.has_key('type'):
type = form['type']
if form.has_key('project'):
project = form['project']

if (type == 'www'):
request_update('www')
req.content_type = 'text/html'
req.send_http_header ()
return apache.OK 

if not (type and project):
syslog(LOG_ERR, 'Error: missing type or project. Aborting.')
return apache.HTTP_NOT_FOUND

if (type != 'gnu') and (type != 'non-gnu') and (type != 'translations'):
syslog(LOG_ERR, "Error: Type unknown, was %s. Aborting." % type)
return apache.HTTP_NOT_FOUND

if string.find (project, "/") != -1:
syslog(LOG_ERR, "Error: / found in project name %s" % project)
return apache.HTTP_NOT_FOUND

request_update(type,project)
req.content_type = 'text/html'
req.send_http_header ()

return apache.OK

[Savannah-hackers-public] [gnu.org #912039] Migrate gnu.org/nongnu.org from CVS

[Lisa Maginnis via RT]

Acknowledged.

Looking at this ticket.

Thanks,

-- 
~Lisa Marie Maginnis
Senior System Administrator
Free Software Foundation
http://fsf.org http://gnu.org
GPG Key: 61EEC710