Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > - Subversion over SSH, using SSH key pairs. This would require > to give committers accounts on the machine, which currently is > ruled out by the administration policy of svn.python.org. Would it work/how much risk would it be to create accounts with shell /bin/false? Cheers, mwh (still faintly bothered by ~/.subversion/auth/svn.simple/*...) -- If trees could scream, would we be so cavalier about cutting them down? We might, if they screamed all the time, for no good reason. -- Jack Handey ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: > I'd like to see the Python source be stored in Subversion instead > of CVS, +1 > and on python.org instead of sf.net. To facilitate discussion, > I have drafted a PEP describing the rationale for doing so, and > the technical procedure to be performed. Not sure about the move to svn.python.org. This would bind additional developer resources for doing administration work. If SF is such a show-stopper, would it be reasonable to look for managed alternatives, such as eg. CollabNet (who funded Subversion development) ? Perhaps we could get a good deal from them given that the PSF is a non-profit organization. Greg Stein could probably provide contacts to the right people to talk to. http://www.collab.net/products/team_edition/index.html Just an idea, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 29 2005) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Tim Peters wrote: ... > I'm sending this to Jim Fulton because he did the conversion of Zope > Corp's code base to SVN. Unfortunately, Jim will soon be out of touch > for several weeks. Jim, do you have time to summarize the high bits > of the problems you hit? IIRC, you didn't find any conversion script > at the time capable of doing the whole job as-is. If that's wrong, it > would be good to know that too. I had two problems, one of which was zope specific: 1. We were making extensive use of symbolic links to share packages among various Zope projects. This requires special care and was the main reason I wrote my own scrips. I don't expect that this would be an issue for Python. 2. I initially tried to conver our entire repository, including all branches. This would have taken days. I finally decided to just convert our trunk, which took several hours. The main time sink was in the load step of the conversion process. I suspect that much of the time hit was due to the Berkely DB back end. I think I've heard that the file-system-based back end is much faster in general. We're using the BDB back end because that's all that was available at the time we converted. Every few weeks, the database gets wedged and I have to do a recovery operation. Every few months, the machine gets wedged and required a reboot. I'm pretty sure the later is due to locking issues. I definately recommend the file-system back-end, even though I haven't tried it myself. I haven't had the time to do the conversion myself. :( I'll also say that, on balance, I'm *very* *very* happy with subversion. I recommend it highly. > Other than that, I'd just like to see an explicit mention in the PEP > of a plan to preserve the pre-conversion CVS tarball forever. +1 Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Tim Peters wrote: > [Jeff Rush] > >>The conversion script isn't perfect and does fail on complex CVS >>arrangements or where there is extensive history to migate. However it >>appears above that Martin has already tried the script out, with success. > > > I'd still like to hear from Jim, as I don't believe all serious > problems were identified by eyeball at once. I'm not aware of any errors in the conversion. ... > Ah, before I forget, "single repository" has worked very well for Zope Yup. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: > Tim Peters wrote: > >>Ah, before I forget, "single repository" has worked very well for Zope >>(which includes top-level Zope2, Zope3, ZODB, ZConfig, zdaemon, ... >>projects): >> >>http://svn.zope.org/ >> >>Long URLs don't really get in the way in practice (rarely a need to >>type one after initial checkout; even "svn switch" is usually just a >>tail-end cmdline edit starting from a copy+paste of "svn info" >>output). > > > That would indeed give conversion problems: Nah > cvs2svn automatically > generates tags/trunk/branches in the root, with the original CVS > modules below; there is a single space for tags and branches > (as is in the original CVS repository). > > I would see two possible conversion procedures: > 1. convert the modules one-by-one, adding to the same repository. >I.e. python would get all the small revision numbers, then >peps, then sandbox, and so on. Cross-module tags would not >be supported (but likely don't exist in the first place), >and the revision number would not increase in historical order. > 2. convert the entire CVS, then rearrange things through >svn move. This would be tedious to do (atleast for tags/branches), >and it would cause all files to be renamed right from the >scratch (some svn clients fail to display history past moves). You reminded me of another reason why I used a custom conversion script. :) I did convert projects individually. I told cvs2svn to just create dump files. I then used svnload to load the dump files myself so that I could make each project a top-level directory with it's own trunk, branches and tags. I'd be happy to share my scrips, although there's nothing all that complicated about them. > So for all who would prefer to see a single repository, could > somebody please > 1. state how precisely the repository should be organized >(with exact URLs on svn.python.org - eg. which of the >non-dist directories becomes toplevel, which ones >get their own tags/branches/trunk, etc). I'm not close enough to the Python repo to offer much of an opinion other than: - IMO, a single repository is good - The repository should be orgabized by "projects", where separate projects reflect more or less independent development efforts with their own teams and schedules. (Maybe Python doesn't have many of these.) > 2. state how the conversion should be performed Once you decide what the projects are, I suggest converting each project separately. I'd be happy to share my scrips and experenice, although, as Tim noted I'll be off-line for two or three weeks starting in the next few days. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Friday 29 July 2005 06:34, M.-A. Lemburg wrote: > If SF is such a show-stopper, would it be reasonable to > look for managed alternatives, such as eg. CollabNet (who > funded Subversion development) ? docutils has been using berlios.de for Subversion; that might be a reasonable option. I'm not sure if there are any political issues about that, but I think everyone actively developing on that project has been happy with the move. (Only the berlios.de Subversion is being used; everything else remains at SourceForge IIRC.) -Fred -- Fred L. Drake, Jr. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: > So do you use project/trunk or trunk/project? If the former, I would > need to get instructions on how to do the conversion from CVS. project/trunk/ On Friday 29 July 2005 02:12, Fernando Perez wrote: > For ipython, which recently went through cvs2svn, I found that moving over > to a project/trunk structure was a few minutes worth of work. Since svn > has moving commands, it was just a matter of making the extra project/ > directory and moving things one level down the hierarchy. Even if cvs2svn > doesn't quite create things the way you want them in the long run, svn is > flexible enough that a few manual tweaks should be quite easy to perform. Yes, this will work. I vaguely recall that Jim converted the zope.org repository one project at a time, so that made it easier to keep them separate. We didn't decommission the CVS entirely, though; Zope 2.7 is still maintained in CVS since we didn't want to risk worrying about the branch structure too much; cvs2svn still had a few issues with the complex branch structure combined with the use of symlinks within the repository (one of the reasons we were so keen to move to Subversion). I'm sure Jim can provide more details of what he had to do; I was only slightly involved in the discussions. -Fred -- Fred L. Drake, Jr. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] math.fabs redundant?
Why does math have an fabs function? Both it and the abs builtin function wind up calling fabs() for floats. abs() is faster to boot. Skip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Friday 29 July 2005 09:17, Jim Fulton wrote: > 1. We were making extensive use of symbolic links to share packages > among various Zope projects. This requires special care and > was the main reason I wrote my own scrips. > > I don't expect that this would be an issue for Python. I know of three symlinks in the Python repository, all from the distutils project. There's one for the package and one for each of the documents. > 2. I initially tried to conver our entire repository, including all > branches. This would have taken days. I finally decided to just > convert our trunk, which took several hours. The main time > sink was in the load step of the conversion process. This might be a possibility for Python as well, though we have a much less complex branching structure, so the conversion may not be so difficult. -Fred -- Fred L. Drake, Jr. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] math.fabs redundant?
[Skip] > Why does math have an fabs function? Both it and the abs builtin function > wind up calling fabs() for floats. abs() is faster to boot. Nothing deep -- the math module supplies everything in C89's standard libm (+ a few extensions), fabs() is a std C89 libm function. There isn't a clear (to me) reason why one would be faster than the other; sounds accidental; math.fabs() could certainly be made faster (as currently implemented (via math_1), it endures a pile of general-purpose "try to guess whether libm should have set errno" boilerplate that's wasted (there are no domain or range errors possible for fabs())). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
M.-A. Lemburg wrote: >>and on python.org instead of sf.net. To facilitate discussion, >>I have drafted a PEP describing the rationale for doing so, and >>the technical procedure to be performed. > > > Not sure about the move to svn.python.org. This would > bind additional developer resources for doing administration > work. True. OTOH, there already is a subversion on svn.python.org, and administrative work is likely only about creating new accounts. I guess the current SF project admins would help out if help is needed. > If SF is such a show-stopper It is at the moment, atleast: there currently is not svn support on sf.net. > would it be reasonable to > look for managed alternatives, such as eg. CollabNet (who > funded Subversion development) ? Perhaps. Somebody would need to research the precise migration procedure. I once tried to move the Python CVS to Sunsite (or its successors), and gave up after half a year of getting nowhere; I'm personally not keen on repeating such an experience. However, if somebody stepped forward to do the actual work (perhaps based on the draft PEP), I would happily hand this project over (it would be good if that volunteer would set a deadline for negotiations with the new host). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Jim Fulton wrote: >I don't expect that this would be an issue for Python. Right. > 2. I initially tried to conver our entire repository, including all >branches. This would have taken days. I finally decided to just >convert our trunk, which took several hours. The main time >sink was in the load step of the conversion process. > >I suspect that much of the time hit was due to the Berkely DB >back end. I think I've heard that the file-system-based back end >is much faster in general. Dunno. For the Python CVS (which translates into some 40,000 revisions), on the machine which I was originally using (1GHz Pentium), conversion took 7h. On my current workstation, it takes a little over an hour. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Jim Fulton wrote: > I did convert projects individually. I told cvs2svn to just create dump > files. I then used svnload to load the dump files myself so that > I could make each project a top-level directory with it's own > trunk, branches and tags. > > I'd be happy to share my scrips, although there's nothing all that > complicated about them. If that's how it worked, I'm sure I can cook my own. Just for confirmation: the svn revision numbers don't increase chronologically across 'modules', right: i.e. the first revision of the module that was converted second has a higher revision than the last revision of the first module, even though historically, the order should have been reverse. Not that this worries me much, but I'd like to confirm there is no other way. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: > Jim Fulton wrote: > >> I don't expect that this would be an issue for Python. > > > Right. > > >>2. I initially tried to conver our entire repository, including all >> branches. This would have taken days. I finally decided to just >> convert our trunk, which took several hours. The main time >> sink was in the load step of the conversion process. >> >> I suspect that much of the time hit was due to the Berkely DB >> back end. I think I've heard that the file-system-based back end >> is much faster in general. > > > Dunno. For the Python CVS (which translates into some 40,000 revisions), > on the machine which I was originally using (1GHz Pentium), conversion > took 7h. On my current workstation, it takes a little over an hour. Was this with the file-system back end? What is your current system? Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: > Jim Fulton wrote: > >>I did convert projects individually. I told cvs2svn to just create dump >>files. I then used svnload to load the dump files myself so that >>I could make each project a top-level directory with it's own >>trunk, branches and tags. >> >>I'd be happy to share my scrips, although there's nothing all that >>complicated about them. > > > If that's how it worked, I'm sure I can cook my own. Just for > confirmation: the svn revision numbers don't increase > chronologically across 'modules', right: i.e. the first > revision of the module that was converted second has a higher > revision than the last revision of the first module, even > though historically, the order should have been reverse. > > Not that this worries me much, but I'd like to confirm there > is no other way. Right. The revision numbers reflect the order in which they are added to the svn repo. I'm pretty sure the revision meta data, in particular the date, was retained. This is an advantage advantage of using the low-level dump/load mechanism. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Fernando Perez wrote: > For ipython, which recently went through cvs2svn, I found that moving over to > a > project/trunk structure was a few minutes worth of work. Since svn has moving > commands, it was just a matter of making the extra project/ directory and > moving things one level down the hierarchy. Even if cvs2svn doesn't quite > create things the way you want them in the long run, svn is flexible enough > that a few manual tweaks should be quite easy to perform. Doesn't this give issues as *every* file the starts out renamed? e.g. what if you want "revision 100 of project/trunk/foo", but, at revision 100, it really was trunk/project/foo? Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Michael Hudson wrote: > Would it work/how much risk would it be to create accounts with shell > /bin/false? It seems that the pydotorg admins are worried about such a prospect. I believe this alone either won't work or won't be good enough (not sure which one): If you have /bin/false as login shell, and still manage to invoke /usr/bin/svnserve remotely, you can likely also invoke /usr/bin/cat /etc/passwd remotely (or download and build the root exploit via ssh). So you would have restrict the set of valid programs to *only* svnserve. This is possible, but difficult to manage (AFAIK). > (still faintly bothered by ~/.subversion/auth/svn.simple/*...) Indeed. I personally would prefer SSL client certificates. This is still tricky (where do you get the passphrase for the private key from (*)), but slightly better. Regards, Martin (*) In case you wonder, I'm personally using the following techniques here: - on windows, remove the passphrase from the private key/.p12 file, and encrypt the file through NTFS encryption. Then you don't need to enter a passphrase, but still nobody can steal the private key from your disk (as long as you are not logged in; the same holds for ssh-agent) - on Linux, my issue is that .subversion is on NFS, so any root user in our net can connect to the file. Therefore, I copy the .p12 file to /tmp/private_dir, and remove the passphrase there. No other machine can read the file (as /tmp is not exported), and the file goes away after machine shutdown latest (as tmp is cleaned on reboot). - again on windows, I'm working on teaching subversion the Microsoft certificate store. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Jim Fulton wrote: >> Dunno. For the Python CVS (which translates into some 40,000 revisions), >> on the machine which I was originally using (1GHz Pentium), conversion >> took 7h. On my current workstation, it takes a little over an hour. > > > Was this with the file-system back end? What is your current system? Yes, and it is a 3 GHz dual processor (although I don't think it uses two processors) with 1GB RAM (I believe RAM size matters a lot for this process; the older machine has 512MB). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Fri, 2005-07-29 at 01:00, "Martin v. Löwis" wrote: > Barry Warsaw wrote: > > We won't use plain text, but we may (or, we currently do) use basic auth > > over ssl. The security then is in the passwords, so we have to make > > sure they're generated securely. > > That (sort of) *is* plain text passwords. Somebody who took over > svn.python.org can get the password. In public-key or digest > authentication, this won't be possible. Actually, the passwords are still hashed in the file, so they wouldn't be able to extract the plain text password. They definitely are vulnerable to brute force attack, though probably not to a dictionary attack. In practice I've been using a password generated based on os.urandom() -- we generate the password and get it to the Subversion user via a "secure route" . I'd be happy to share my password generation script with anybody who wants to audit it. Public/private keys would be better, and if anybody knows how to set up a Subversion server to use these without having to create accounts for everyone, I think we (the pythong.org admins) would love your help. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Fri, 2005-07-29 at 00:44, "Martin v. Löwis" wrote: > - assignment of passwords. This I don't like about the current > pydotorg setup - there should be a way to chose your own password; > perhaps without involving an administrator. > I could imagine a web form for password change, and administrator > interaction in case of a lost password. I disagree. By reserving password generation to the pydotorg admins, we can better insure the passwords are more robust against dictionary attacks. See my previous message. I actually /don't/ want individuals to be able to set their own passwords. In practice, you only have to know your password once, because svn caches the authentication (yes, that opens up opportunities for compromise, but that's how svn works). > - compromised passwords. The only tricky question then is: was the > repository altered? Fortunately, for Subversion, there should be > an easy way to tell: in fsfs, files never change (only new files > are added). So we could generate md5sums of all files in the > repository, and download these to an offsite place. If the md5sum > of an immutable file changes, we were compromised (there are, > of course, a few files that do change regularly). > Of course, we also need regular backups of the entire data > so we can restore them if they got compromised. +1 to all that. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
"Martin v. Löwis" wrote: > Fernando Perez wrote: >> For ipython, which recently went through cvs2svn, I found that moving over >> to a >> project/trunk structure was a few minutes worth of work. Since svn has >> moving commands, it was just a matter of making the extra project/ directory >> and >> moving things one level down the hierarchy. Even if cvs2svn doesn't quite >> create things the way you want them in the long run, svn is flexible enough >> that a few manual tweaks should be quite easy to perform. > > Doesn't this give issues as *every* file the starts out renamed? e.g. > what if you want "revision 100 of project/trunk/foo", but, at revision > 100, it really was trunk/project/foo? To be honest, I don't really know the details, but it seems to work fine. A quick look at ipython: planck[IPython]> svn update At revision 661. planck[IPython]> svn diff -r 10 genutils.py | tail - -Deprecated: this function has been superceded by timing() which has better -fucntionality.""" - -rng = range(reps) -start = time.clock() -for dummy in rng: func(*args,**kw) -return time.clock()-start - -#*** end of file ** Revision 10 was most certainly back in the early CVS days, and the wholesale renaming happened when I started using svn, which was around revision 600 or so. There may be other subtleties I'm missing, but so far I haven't experienced any problems. Cheers, f ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Barry Warsaw wrote: > >Public/private keys would be better, and if anybody knows how to set up >a Subversion server to use these without having to create accounts for >everyone, I think we (the pythong.org admins) would love your help. > > I'll play around with it some this weekend, see what's possible, and was isn't. I'm thinking we ought to be able to use SSH's configuration to only allow one specific command, e.g. something like this in the authorized_keys: no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,command="/usr/bin/svnserve" -- Leif ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Fri, 2005-07-29 at 17:19, "Martin v. Löwis" wrote: > I believe this alone either won't work or won't be good enough (not > sure which one): If you have /bin/false as login shell, and still > manage to invoke /usr/bin/svnserve remotely, you can likely also > invoke /usr/bin/cat /etc/passwd remotely (or download and build > the root exploit via ssh). > > So you would have restrict the set of valid programs to *only* > svnserve. This is possible, but difficult to manage (AFAIK). I think that's basically right. > - on Linux, my issue is that .subversion is on NFS, so any root > user in our net can connect to the file. Therefore, I copy > the .p12 file to /tmp/private_dir, and remove the passphrase > there. No other machine can read the file (as /tmp is not > exported), and the file goes away after machine shutdown > latest (as tmp is cleaned on reboot). I don't think that's true on all Linuxes though (or even all *nixes). -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Fri, 2005-07-29 at 00:35, "Martin v. Löwis" wrote: > It's also a convenience issue, and has social aspects. For example, > firstname.lastname does not work quite well for me, either > (martin.v.loewis or martin.von.loewis would work better; likewise > guido.van.rossum?), other users prefer not to use their "true" > real name (e.g. tim_one vs. tim.peters). Yep, we would use guido.van.rossum. I think it would be fine for you to use martin.v.loewis or martin.von.loewis, just as MAL could use m.a.lemburg or marc.andre.lemberg. Our general concern was people hiding behind obscure nicknames like 'pumpichank' and no one really knowing who's behind that . > Another issue is password assignment - currently, users cannot choose > their passwords on svn.python.org, right? Correct, which I think is a feature. :) > Then you could give different access to peps and to sandbox. > Perhaps there isn't even a need for tags/branches on PEPs? Probably so. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Barry Warsaw wrote: > I disagree. By reserving password generation to the pydotorg admins, we > can better insure the passwords are more robust against dictionary > attacks. See my previous message. I actually /don't/ want individuals > to be able to set their own passwords. In practice, you only have to > know your password once, because svn caches the authentication (yes, > that opens up opportunities for compromise, but that's how svn works). See Michael's (I think) message: that is a much greater risk than the one of a brute-force attack. In our environment, a determined student could easily read out my home directory, and get at my pydotorg password (if I would allow svn to cache it). They would have to break all kinds of rules in doing so; yet, it would be technically possible - so I just can't turn on this svn setting, and have to type the password every time. This is surely inconvenient, as I cannot even remember the password. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Fri, 2005-07-29 at 17:32, "Martin v. Löwis" wrote: > > Was this with the file-system back end? What is your current system? > > Yes, and it is a 3 GHz dual processor (although I don't think it uses > two processors) with 1GB RAM (I believe RAM size matters a lot for > this process; the older machine has 512MB). That matches my experience w/ the fsfs backend. If we do the conversion on one of the new python.org boxes, I expect it to go pretty quickly. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Barry Warsaw wrote: >>That (sort of) *is* plain text passwords. Somebody who took over >>svn.python.org can get the password. In public-key or digest >>authentication, this won't be possible. > > > Actually, the passwords are still hashed in the file, so they wouldn't > be able to extract the plain text password. Nah. Somebody who takes over svn.python.org can replace Apache, and that will receive plain text passwords over the wire (in case you wonder: modules/aaa/mod_auth.c:authenticate_real_user - you can even write an Apache module that gets hold of the sent password). An intruder would have to wait some time before the password come in, instead of being able to read them all from a file at once - that's true. > Public/private keys would be better, and if anybody knows how to set up > a Subversion server to use these without having to create accounts for > everyone, I think we (the pythong.org admins) would love your help. Ok. Since this falls in my research interest, I definitely want to give it a try. I think I would set up PyCA to let users generate their private keys in the browser. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Fri, 2005-07-29 at 00:50, Christopher Petrilli wrote: > Another thing to look at would be Trac, which we are in the process of > moving to from the horrendous nightmare of Bugzilla. It's integration > with SVN as well as Wiki is quite amazing. Now's the time I pipe in to remind everyone that Atlassian has offered free (as in beer) versions of Jira and Confluence for the Python project (actually any open source project). If you haven't used these, they're definitely worth a look. Jira is the issue tracker, Confluence the wiki. They're extremely professional, usable, well-integrated, and you can integrate them with Subversion. We've used them at work for maybe a year now and I've been very happy with them. Jira is definitely one of the better issue trackers, free or not free, that I've used. www.atlassian.com They're not Python and they're not open source, so perhaps it's legitimate to dismiss them because of that. But they are also definitely cool. At the Atlassian folks are very cool too, and fans of FOSS. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
At 05:54 PM 7/29/2005 -0400, Barry Warsaw wrote: >Public/private keys would be better, and if anybody knows how to set up >a Subversion server to use these without having to create accounts for >everyone, I think we (the pythong.org admins) would love your help. From the svnserve man page: -t, --tunnel Causes svnserve to run in tunnel mode, which is just like the inetd mode of operation (serve one connection over stdin/stdout) except that the connection is considered to be pre-authenticated with the username of the current uid. This flag is selected by the client when running over a tunnel agent. --tunnel-user=username When combined with --tunnel, overrides the pre-authenticated username with the supplied username. This is useful in combina- tion with the ssh authorized_key file's "command" directive to allow a single system account to be used by multiple committers, each having a distinct ssh identity. So, it looks like you'd just need to set up public keys for each user, and list them in authorized_keys. Presumably doing something like this: command="/usr/bin/svnserve --root=/svnroot -t --tunnel-user=username",no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa [key info here] would therefore do the trick. I've used a similar arrangement for my own CVS repository, but haven't tried it for SVN yet. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Fri, 2005-07-29 at 17:21, "Martin v. Löwis" wrote: > Doesn't this give issues as *every* file the starts out renamed? e.g. > what if you want "revision 100 of project/trunk/foo", but, at revision > 100, it really was trunk/project/foo? I think it only matters if you use urls. I working directories, I think everything gets tracked correctly. I could be wrong about that though. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Fri, 2005-07-29 at 16:59, "Martin v. Löwis" wrote: > Perhaps. Somebody would need to research the precise migration > procedure. I once tried to move the Python CVS to Sunsite > (or its successors), and gave up after half a year of getting > nowhere; I'm personally not keen on repeating such an experience. I'm with you. SF is a devil we know. If we don't stick with them (and it looks like that's not an option if we want to switch to svn soon), then I say we move it to svn.python.org, where we already have a server set up and running. If we need more volunteers, well the community has always stepped up before and I'm sure it will when we come asking again. Actually, it's a good idea to /always/ be soliciting for help. People get burned out and busy and I'd love to have a bullpen of volunteers that gets constantly refreshed. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Fri, 2005-07-29 at 18:12, Leif Hedstrom wrote: > Barry Warsaw wrote: > > >Public/private keys would be better, and if anybody knows how to set up > >a Subversion server to use these without having to create accounts for > >everyone, I think we (the pythong.org admins) would love your help. > I'll play around with it some this weekend, see what's possible, and was > isn't. I'm thinking we ought to be able to use SSH's configuration to > only allow one specific command, e.g. something like this in the > authorized_keys: > no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,command="/usr/bin/svnserve" But that would still require us to create accounts for every committer, right? -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
At 06:39 PM 7/29/2005 -0400, Barry Warsaw wrote: >But that would still require us to create accounts for every committer, >right? No. Just one account. You can have more than one key listed in authorized_keys, using svnserve --tunnel-user and sshd will select the right command based on the public key the client authenticates with. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Barry Warsaw wrote: > Now's the time I pipe in to remind everyone that Atlassian has offered > free (as in beer) versions of Jira and Confluence for the Python project > (actually any open source project). If you haven't used these, they're > definitely worth a look. Jira is the issue tracker, Confluence the > wiki. They're extremely professional, usable, well-integrated, and you > can integrate them with Subversion. We've used them at work for maybe a > year now and I've been very happy with them. Jira is definitely one of > the better issue trackers, free or not free, that I've used. > > www.atlassian.com I've also used Confluence at work, and been very impressed. A very nice feature which I haven't seen anywhere else is that the Wiki pages allow comments as well as direct editing - it allows discussion *about* the page to occur in a normal answer/response fashion, possibly leading to eventual update of the page text itself. > They're not Python and they're not open source, so perhaps it's > legitimate to dismiss them because of that. But they are also > definitely cool. At the Atlassian folks are very cool too, and fans of > FOSS. I'd hope we wouldn't dismiss them out of hand - one of the things that appeals to me about Python is the philosophy that open-source and protected-source groups can both benefit from working together. Regards, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Pre-PEP: Exception Reorganization for Python 3.0
Well, it has been discussed at multiple times in the past and I have promised to write this PEP several times, so I finally found enough time to write a PEP on reorganizing exceptions for Python 3.0 . Key points in this PEP is the reworking the hierarchy, requiring anything raised to inherit from a certain superclass, and to change bare 'except' clauses to catch a specific superclass. The first and last points I expect some contention, but the middle point I expect people are okay with (Guido liked the idea when Paul Prescod brought it up and the only person who didn't like it, Holger, ended up being okay with it when the superclass had a reasonable name). One thing people might not notice is that I have some minor ideas listed in the tree in parentheses. If people have an opinion on the ideas please speak up. Otherwise the other major points of contention are covered in the Open Issues section or will be brought up in the usual trashing of PEPs that cover contraversial changes. And please realize this is for Python 3.0! None of this is being proposed for any version before then (they could be, but that is another PEP entirely). Hopefully this PEP along with Ping's PEP 344 will cover the major ideas for exceptions for Python 3.0 . -Brett -- PEP: XXX Title: Exception Reorganization for Python 3.0 Version: $Revision: 1.5 $ Last-Modified: $Date: 2005/06/07 13:17:37 $ Author: Brett Cannon <[EMAIL PROTECTED]> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Jul-2005 Post-History: XX-XXX-XXX Abstract Python, as of version 2.4, has 38 exceptions (including warnings) in the built-in namespace in a rather shallow hierarchy. This list of classes has grown over the years without a chance to learn from mistakes and cleaning up the hierarchy. This PEP proposes doing a reorganization for Python 3000 when backwards-compatibility is not an issue. It also proposes changing bare ``except`` clauses to catch only exceptions inheriting from a specific superclass. Lastly, this PEP proposes, in Python 3000, that all objects to be passed to a ``raise`` statement must inherit from a specific superclass. Rationale = Exceptions are a critical part of Python. While exceptions are traditionally used to signal errors in a program, they have also grown to be used for flow control for things such as iterators. There importance is great. But the organization of the exception hierarchy has not been maintained. Mostly for backwards-compatibility reasons, the hierarchy has stayed very flat and old exceptions who usefulness have not been proven have been left in. Making exceptions more hierarchical would help facilitate exception handling by making catching exceptions much more logical with use of superclasses. This should also help lead to less errors from being too broad in what exceptions are caught in an ``except`` clause. A required superclass for all exceptions is also being proposed [Summary2004-08-01]_. By requiring any object that is used in a ``raise`` statement to inherit from a specific superclass, certain attributes (such as those laid out in PEP 344 [PEP344]_) can be guaranteed to exist. This also will lead to the planned removal of string exceptions. Lastly, bare ``except`` clauses can be made less error-prone [Summary2004-09-01]_. Often people use a bare ``except`` when what they really wanted were non-critical exceptions to be caught while more system-specific ones, such as MemoryError, to pass through and to halt the interpreter. With a hierarchy reorganization, bare ``except`` clauses can be changed to only catch exceptions that subclass a non-critical exception superclass, allowing more critical exceptions to propagate to the top of the execution stack as was most likely intended. Philosophy of Reorganization There are several goals in this reorganization that defined the philosophy used to guide the work. One goal was to prune out unneeded exceptions. Extraneous exceptions should not be left in since it just serves to clutter the built-in namespace. Unneeded exceptions also dilute the importance of other exceptions by splitting uses between several exceptions when all uses should have been under a single exception. Another goal was to introduce any exceptions that were deemed needed to fill any holes in the hierarchy. Most new exceptions were done to flesh out the inheritance hierarchy to make it easier to catch a category of exceptions with a simpler ``except`` clause. Changing inheritance to make it more reasonable was a goal. As stated above, having proper inheritance allows for more accurate ``except`` statements when catching exceptions based on the inheritance tree. Lastly, any renaming to make an exception's use more obvious from its name was done. Having to look up what an exception is meant to be used for because the name does not proper reflect its usage is a
Re: [Python-Dev] Pre-PEP: Exception Reorganization for Python 3.0
On 7/29/05, Brett Cannon <[EMAIL PROTECTED]> wrote: > Well, it has been discussed at multiple times in the past and I have > promised to write this PEP several times, so I finally found enough > time to write a PEP on reorganizing exceptions for Python 3.0 . Thanks for getting this ball rolling! (I wonder what happened to Ping's PEP 344 -- he just dropped out, it seems.) Below is some feedback. > Often people use a bare ``except`` when what they really wanted were > non-critical exceptions to be caught while more system-specific ones, > such as MemoryError, to pass through and to halt the interpreter. > With a hierarchy reorganization, bare ``except`` clauses can be > changed to only catch exceptions that subclass a non-critical > exception superclass, allowing more critical exceptions to propagate > to the top of the execution stack as was most likely intended. I appreciate the attempt to make bare except: less dangerous by not catching certain exceptions, but I worry that these changes might actually make bare except: *more* likely to be used, which is contrary to the intention. Maybe we should just get rid of it, and encourage people to write except Exception: or except Raisable: depending on what they want. > MacError > UnixError Do we really need these? Let's not add things that won't be used. > NamespaceException > > To provide a common superclass for exceptions dealing with namespace > issues, this exception is introduced. > Both UnboundLocalError and UnboundGlobalError (the new name for > NameError) inherit from this class. OTOH there's something to say to unify NameError and AttributeError, isn't there? > EnvironmentError > > Originally meant as an exception for when an event outside of the > interpreter needed to raise an exception, its use has been deemed > unneeded. > While both IOError and OSError inherited this class, the distinction > between OS and I/O is significant enough to not warrant having a > common subclass that one might base an ``except`` clause on. -1000. Depending on whether you use open() or os.open() you'll get one or the other for pretty much the same thing. Also, I think that socket.error and select.error should inherit from EnvironmentError (or maybe even from OSError). > RuntimeError > Meant to be used when an existing exception does not fit, its > usefulness is consider meager in Python 2.4 [exceptionsmodule]_. > Also, with the CriticalException/Exception dichotomy, Exception or > CriticalException can be raised directly for the same use. -0.5. I use it all the time as the "application" exception in apps (scripts) that are too small to define their own exception hierarchy. > StopIteration and GeneratorExit > > Inherit from ControlFlowException. > > Collecting all control flow-related exceptions under a common > superclass continues with the theme of maintaining a hierarchy. Yes, but how useful it this? I don't expect to ever come across a situation where I'd want to catch both, so this is more for organization of the docs than for anything else. IMO a good principle for determining the ideal exception hierarchy is whether there's a use case for catching the common base class. > IOError > > No longer subclasses EnvironmentError. -1000, see above. > Change the Name of Raisable? > > It has been argued that Raisable is a bad name to use since it is not > an actual word [python-dev1]_. Then we'll make it a word. Is Java's equivalent, Throwable, any more a word? In this case I like the parallel with Java. > Should Bare ``except`` Clauses be Removed? Yes, see above. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pre-PEP: Exception Reorganization for Python 3.0
Brett Cannon wrote: > New Hierarchy > = > > Raisable (new; rename BaseException?) > +-- CriticalException (new) > +-- KeyboardInterrupt > +-- MemoryError > +-- SystemExit > +-- SystemError (subclass SystemExit?) I'd recommend not subclassing SystemExit--there are too many programs out there which expect the argument (e.g. sys.exit(3)) to mean something specific, but that expectation doesn't apply at all to SystemError. > +-- Exception (replaces StandardError) >... > +-- ControlFlowException (new) I'd definitely appreciate ControlFlowException--there are a number of exceptions in CherryPy which should subclass from that. Um, CherryPy 3.0, that is. ;) > +-- LookupError (better name?) SubscriptError, perhaps? But LookupError could become the "I didn't find obj X in the container you specified" superclass, in which case LookupError is perfect. > +-- TypeError > +-- AttributeError (subclassing new) "Since most attribute access errors can be attributed to an object not being the type that one expects, it makes sense for AttributeError to be considered a type error." Very hmmm. I would have thought it would be a LookupError instead, only because I favor the duck-typing parts of Python. Making AttributeError subclass from TypeError leans toward stronger typing models a bit too much, IMO. > +-- WeakReferenceError (rename for ReferenceError) This also has a LookupError feel to it. > It has been argued that Raisable is a bad name to use since it is not > an actual word [python-dev1]_. Perhaps, but compare http://www.onelook.com/?w=raisable to http://www.onelook.com/?w=raiseable. The only sources "raiseable" has going for it are Dictionary.com (which aggregates lots of questionable sources), Rhymezone, and LookWAYup. I think "raisable" is the clear winner. Robert Brewer System Architect Amor Ministries [EMAIL PROTECTED] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pre-PEP: Exception Reorganization for Python 3.0
On 7/29/05, Robert Brewer <[EMAIL PROTECTED]> wrote: > > +-- SystemExit > > +-- SystemError (subclass SystemExit?) > > I'd recommend not subclassing SystemExit--there are too many programs > out there which expect the argument (e.g. sys.exit(3)) to mean something > specific, but that expectation doesn't apply at all to SystemError. Agreed. SystemExit is used by sys.exit(); SystemError is something completely different, used by the interpreter when it finds an internal invariant is broken. It is one step short of a fatal error -- the latter is used when we have evidence of random memory scribbling, the former when the interpreter is still intact. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pre-PEP: Exception Reorganization for Python 3.0
On Friday 29 July 2005 23:07, Robert Brewer wrote: > > +-- WeakReferenceError (rename for ReferenceError) > > This also has a LookupError feel to it. I disagree. LookupError is used when looking for an object within some containing object according to some sort of key (dict key, index, etc.). It's usually a reasonable expectation that there might not be an object associated with that key. You also know that a different object may be returned at different times; you care about the association of the object with the key. For weak references, you're not using a key in a container, you're resolving a specific reference that you know exists; what you don't know is whether the object still exists. There's no indirection through a key as there is for LookupError. I'm +0 on the WeakReferenceError name, but -1 on making it a subclass of LookupError. -Fred -- Fred L. Drake, Jr. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
