Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On 8/8/05, Martin v. Löwis [EMAIL PROTECTED] wrote: Nicholas Bastin wrote: It's a mature product. I would hope that that would count for something. Sure. But so is subversion. I will then assume that you and I have different ideas of what 'mature' means. So I should then remove your offer to host a perforce installation, as you never made such an offer, right? Correct. . Yes. That's what this PEP is for. So I guess you are -1 on the PEP. Not completely. More like -0 at the moment. We need a better system, but I think we shouldn't just pick a system because it's the one the PEP writer preferred - there should be some sort of effort to test a few systems (including bug trackers). I know this is work, but this isn't just something we can change easily again later. -- Nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Mon, 2005-08-15 at 12:27 -0400, Nicholas Bastin wrote: On 8/8/05, Martin v. Löwis [EMAIL PROTECTED] wrote: Nicholas Bastin wrote: It's a mature product. I would hope that that would count for something. Sure. But so is subversion. I will then assume that you and I have different ideas of what 'mature' means. Bigger projects than Python use it and consider it mature for real use (All the Apache projects, all of KDE, GNOME is planning on switching soon, etc). I've never seen a corrupted FSFS repo, only corrupted BDB repos, and I will happily grant that using BDB ended up being a big mistake for Subversion. Not one that could have easily been foreseen at the time, but such is life. But this is why FSFS is the default for 1.2+ I've never seen you post about a corrupted repository to svn-users or svn-dev or file a bug, so i can't say why you see corrupted repositories if they are FSFS ones. --Dan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Nicholas Bastin wrote: Not completely. More like -0 at the moment. We need a better system, but I think we shouldn't just pick a system because it's the one the PEP writer preferred - there should be some sort of effort to test a few systems (including bug trackers). But that's how the PEP process works: the PEP author is supposed to collect feedback from the community in a fair way, but he is not required to implement every suggestion that the community makes. People who strongly disagree that the entire approach should be taken should write an alternative (counter) PEP, proposing their strategy. In the end, the BDFL will pronounce which approach (if any) should be implemented. In the specific case, I'm personally not willing to discuss every SCM system out there. If somebody manages to make me curious (as Guido did with the bazaar posts), I will try it out, if I can find an easy way to do so. Your comments about (what was the name again) did not make me curious. As for bug trackers: this PEP is specifically *not* about bug trackers at all. If you think the SourceForge bugtracker should be replaced with something else, write a PEP. I really don't see a reasonable alternative to the SF bugtracker. I know this is work, but this isn't just something we can change easily again later. I don't bother asking who we is, here: apparently not you. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Sunday 07 August 2005 15:33, Martin v. Löwis wrote: Ah, ok. That's true. It doesn't mean you can't do proper merging with subversion - it only means that it is harder, as you need to figure out the revision range that you want to merge. If this is too painful, you can probably use subversion to store the relevant information. For example, you could define a custom property on the directory, last_merge_from_trunk, which you would always update after you have done a merge operation. Then you don't have to look through history to find out when you last merged. This is what I do with shtoom - I have properties branchURI and branchRev on the root of the branch. I can then use these when landing the branch. It seems to work well enough for me. Anthony -- Anthony Baxter [EMAIL PROTECTED] It's never too late to have a happy childhood. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Nicholas Bastin wrote: It's a mature product. I would hope that that would count for something. I've had enough corrupted subversion repositories that I'm not crazy about the thought of using it in a production system. I know I'm not the only person with this experience. compared to Perforce, SVN is extremely fragile. I've used both for years, and I've never had Perforce repository break down on me. our SVN repositories are relatively stable these days, but the clients are still buggy as hell (mostly along the I don't feel like doing this today, despite the fact that it worked yesterday, and I don't feel like telling you what's wrong either lines. having to nuke workspaces from time to time gets boring, quickly.) in contrast, Perforce just runs and runs and runs. the clients always do what you tell them. and server maintenance is trivial; just make sure that the server starts when the host computer boots, and if you have enough disk, just leave it running. if you're tight on disk space, trim away some log files now and then. that's it. but despite this, if all you need is a better CVS, I'd say SVN is good enough for today's python-dev. I'd still think that a more distributed, mail-driven system (built on top of Mercurial, Bazaar-NG, or some such (*)) would speed up both development and patch processing, and also make it a lot easier for casual contributors and drive-by developers to help develop Python, but that's another story. /F *) being able to ship a fully working Python-powered SCM with the Python source code would be an extra coolness bonus, of course. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On 8/10/05, Fredrik Lundh [EMAIL PROTECTED] wrote: in contrast, Perforce just runs and runs and runs. the clients always do what you tell them. and server maintenance is trivial; just make sure that the server starts when the host computer boots, and if you have enough disk, just leave it running. if you're tight on disk space, trim away some log files now and then. that's it. We've used P4 at Elemental for two years now; I mostly agree with this assessment, although occasionally the server becomes unbearably slow and a sysadmin does some painful magic to rescue it. Maybe that's just because the box is underpowered. More troublesome is that I've seen a few client repositories getting out of sync; one developer spent a lot of time tracking down mysterious compilation errors that went away after forced resync'ing. We never figured out the cause, but (since he swears he didn't touch the affected files) most likely hitting ^C during a previous sync could've broken some things. Another problem with P4 is that local operation is lousy -- if you can't reach the server, you can't do *anything* -- while svn always lets you edit and diff. Also, P4 has *no* command to tell you which files you've created without adding them to the repository yet -- so the most frequent build breakage is caused by missing new files. Finally, while I hear that P4's branching support is superior over SVN's, I find it has a steep learning curve -- almost every developer needs some serious hand-holding before they understand P4 branches correctly. I'm intrigued by Linus Torvald's preference for extremely distributed source control, but I have no experience and it seems a bit, um, experimental. Someone should contact Steve Alexander, who I believe is excited about Bazaar-NG. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Guido van Rossum [EMAIL PROTECTED] wrote: I'm intrigued by Linus Torvald's preference for extremely distributed source control, but I have no experience and it seems a bit, um, experimental. git, which is Linus' home-grown replacement for BitKeeper, quickly attracted a development community and has grown into a reasonably full-featured distributed RCS. It is apparently already stable enough for serious use. If I was trying to pick an RCS for a large, distributed project, I would at least investigate it as a possibility. Charles -- --- Charles Cazabon [EMAIL PROTECTED] GPL'ed software available at: http://pyropus.ca/software/ --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
[Guido van Rossum wrote] Also, P4 has *no* command to tell you which files you've created without adding them to the repository yet -- so the most frequent build breakage is caused by missing new files. This one is a frequent complaint from CVS-heads here at ActiveState. I have a p4 wrapper called px that extends some p4 commands (and adds a couple). One of the commands that it extends is diff to add a -sn (new) option similar to the -se (edit), -sd (delete). $ px help diff ...the usual 'p4 help diff'... new px options: [-sn -c changelist#] Px adds another -sflag option: -sn Local files not in the p4 client. Px also adds the --skip option (which only makes sense together with -sn) to specify that regularly skipped file (CVS control files, *~) should be skipped. The '-c' option can be used to limit diff'ing to files in the given changelist. '-c' cannot be used with any of the '-s' options. 'px' should grow a px status a la svn|cvs status to give a quick summary of local differences. Other additions: $ px help px 'px' entensions to 'p4': px --help Add px-specific help output to the usual 'p4 -h' and 'p4 -?'. See 'px help usage'. px -V, --version Print px-specific version information in addition to the usage 'p4 -V' output. See 'px help usage'. px -g ... Format input/output as *un*marshalled Python objects. Compare to the usual 'p4 -G ...'. See 'px help usage'. px annotate ... Identify last change to each line in given file, like 'cvs annotate' or 'p4pr.pl'. See 'px help annotate'. px backout ... Provide all the general steps for rolling back a perforce change as described in Perforce technote 14. See 'px help backout'. px changes -d ... Print the full 'p4 describe -du' output for each listed change. See 'px help changes'. px diff -sn --skip ... List local files not in the p4 depot. Useful for importing new files into a depot via 'px diff -sn --skip ./... | px -x - add'. See 'px help diff'. px diff -c change ... Limit diffing to files opened in the given pending change. See 'px help diff'. px genpatch [change] Generate a patch (usable by the GNU 'patch' program) from a pending or submitted chagelist. See 'px help genpatch'. Available here: http://starship.python.net/~tmick/#px Pure python. Works on Python =2.2. Windows, Linux, Mac OS X, Unix. Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Trent Mick wrote: One feature I like in Perforce (which Subversion doesn't have) is the ability to have pending changesets. That sounds useful. Currently with svn you have to manually specify those 9 to be sure to not include the remaining one. With p4 you just say to check-in the whole tree and then remove that one from the list give you in your editor with entering the check-in message. Not that big of a deal. Depends on the client also. With Tortoise SVN, you do get a checkbox list where you can exclude files from the checkin. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Mon, 2005-08-08 at 19:29, Tim Peters wrote: Currently with svn you have to manually specify those 9 to be sure to not include the remaining one. With p4 you just say to check-in the whole tree and then remove that one from the list give you in your editor with entering the check-in message. Not that big of a deal. As a purely theoretical exercise wink, the last time I faced that under SVN, I opened the single file I didn't want to check-in in my editor, did svn revert on it from the cmdline, checked in the whole tree, and then hit the editor's save button. This doesn't scale well to skipping 25 of 50, but it's effective enough for 1 or 2. Or one could use a decent client, like say psvn under XEmacs wink which presents you a list of all modified files and lets you select which ones you want to commit. The one thing I dislike about svn (in my day-to-day use of it) is that it can take a VERY long time to do updates at the roots of very large trees. I once tried to check out the root of our dev tree, which contains all branches and tags. Of course the initial checkout took forever. But an update at the root made this approach unusable. svn would sit there, seemingly idle for 30-45 minutes and then take another 30-45 minutes updating the changes, which typically consisted of maybe 50 files out of thousands. And this on a gig LAN with fast h/w all around (and for Tim's sake I won't even complain about how some operating systems appear to perform much worse than others :). The smaller you can keep your working copies, the better. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Nicholas Bastin wrote: It's a mature product. I would hope that that would count for something. Sure. But so is subversion. I've had enough corrupted subversion repositories that I'm not crazy about the thought of using it in a production system. I had the last corrupted repository with subversion 0.23. It has matured since then. Even then, invoking db_recover would restore the operation, without losing data (i.e. I did not need to go to backup). Interesting offer. I'll add this to the PEP - who is we in this context? Uh, the Python community. Which is currently hosting a subversion repository, so it doesn't seem like a stretch to imagine that p4.python.org could exist just as easily. Ah. But these people have no expertise with Perforce, and there is no Debian Perforce package, so it *is* a stretch assuming that they could also host a perforce directory. So I should then remove your offer to host a perforce installation, as you never made such an offer, right? Pardon me if I don't feel that I'd like to see a system in production for a few weeks before we declare victory. The problems with this kind of conversion can be very subtle, and very painful. I'm not saying we shouldn't do this, I'm just saying that we should take an appropriate measure of how much greener the grass really is on the other side, and how much work we're willing to put in to make it that way. Yes. That's what this PEP is for. So I guess you are -1 on the PEP. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Barry Warsaw [EMAIL PROTECTED] writes: Unfortunately, I don't think we (meaning specifically the collective python.org admins) have much if any operational experience with Perforce. Also (from someone who is on the fringes of the pydotorg admin set): I don't know that much about subversion administration. But, if it proves necessary, as it's an open source project and all, I'm willing to put some time into learning about it. I'm *much* less likely to do this for a closed source package unless someone is paying me to do it. Maybe I'm the only person who thinks this way, but if not, it's something to think about. Cheers, mwh -- 42. You can measure a programmer's perspective by noting his attitude on the continuing vitality of FORTRAN. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Sun, Aug 07, 2005, Barry Warsaw wrote: We'd also have to teach the current crop of developers how to use the client tools effectively. I think it's fairly simple to teach a CVS user how to use Subversion, but have no idea if translating CVS experience to Perforce is as straightforward. The impression I got from Alex Martelli is that it's not particularly straightforward. (Google apparently uses Perforce.) -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Mon, Aug 08, 2005, Trent Mick wrote: Martin: No. The PEP is only about Subversion. Why should we be looking at Per Force? Only because Python is Open Source? Perforce offers free licensing to open source projects. So did BitKeeper. Linux got bitten by that. We'd need a strong incentive to consider Perforce over Subversion just because of that issue. -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Trent Mick wrote: No. The PEP is only about Subversion. Why should we be looking at Per Force? Only because Python is Open Source? Perforce offers free licensing to open source projects. Ok, so I now got it's mature, and it would be without charges. Given that it is now running against Subversion, I would be still interested in advantages that it offers compared to svn. The biggest disadvantage, to me, is that few people know how to use it (myself included). I don't trust tools I've never used. So for me, as the author of this PEP, usage of the revsion control software is non-negotiable (selection of hoster, to a limited degree, is). If you want to see Perforce used for the Python development, you should write a counter-PEP, so we could let the BDFL decide. [This is a theoretical you here, since you then explain that you would still prefer to use subversion] Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
One feature I like in Perforce (which Subversion doesn't have) is the ability to have pending changesets. A changeset is, as with subversion, something you check-in atomically. Pending changesets in Perforce allow you to (1) group related files in a source tree where you might be working on multiple things at once to ensure and (2) to build a change description as you go. In a large source tree this can be useful for separating chunks of work. There are other little things, like not being able to trim the check-in filelist when editing the check-in message. For example, say you have 10 files checked out scattered around the Python source tree and you want to check 9 of those in. Currently with svn you have to manually specify those 9 to be sure to not include the remaining one. With p4 you just say to check-in the whole tree and then remove that one from the list give you in your editor with entering the check-in message. Not that big of a deal. [Martin v. L?wis on Perforce] The biggest disadvantage, to me, is that few people know how to use it (myself included). Granted. For that reason and for a couple of others I mentioned (SVN will probably work better for offline and distributed developers) I think Subversion wins over Perforce. That is presuming, of course, that we find Subversion to be acceptibly stable/robust/manageble. Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
[Trent Mick] ... There are other little things, like not being able to trim the check-in filelist when editing the check-in message. For example, say you have 10 files checked out scattered around the Python source tree and you want to check 9 of those in. This seems dubious, since you're not checking in the state you actually have locally, and you were careful to run the full Python test suite with your full local state ;-) Currently with svn you have to manually specify those 9 to be sure to not include the remaining one. With p4 you just say to check-in the whole tree and then remove that one from the list give you in your editor with entering the check-in message. Not that big of a deal. As a purely theoretical exercise wink, the last time I faced that under SVN, I opened the single file I didn't want to check-in in my editor, did svn revert on it from the cmdline, checked in the whole tree, and then hit the editor's save button. This doesn't scale well to skipping 25 of 50, but it's effective enough for 1 or 2. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Mon, 2005-08-08 at 15:49, Trent Mick wrote: One feature I like in Perforce (which Subversion doesn't have) is the ability to have pending changesets. A changeset is, as with subversion, something you check-in atomically. Pending changesets in Perforce allow you to (1) group related files in a source tree where you might be working on multiple things at once to ensure and (2) to build a change description as you go. In a large source tree this can be useful for separating chunks of work. This seems like a poor workaround for crappy branch/merge support. I'm new to perforce, but the pending changesets seem dodgey to me... you are accumulating changes gradually without recording any history during the process... ie, no checkins until the end. Even worse, perforce seems to treat clients like unversioned branches, allowing you to review and test pending changesets in other clients. This supposedly allows people to review/test each others changes before they are committed. The problem is, since these changes are not committed, there is no firm history of what what was reviewed/tested vs what gets committed... ie they could be different. Having multiple different pending changesets in one large source tree also feels like a workaround for high client overheads. Trying to develop and test a mixture of different changes in one source tree is asking for trouble... they can interact. Maybe I just haven't grokked perforce yet... which might be considered a black mark against it's learning curve :-) For me, the logical way to group a collection of changes is in a branch. This allows you to commit and track history of the collection of changes. You check out each branch into different directories and develop/test them independantly. The branch can then be reviewed and merged when it is complete. -- Donovan Baarda [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
[Tim Peters wrote] [Trent Mick] ... There are other little things, like not being able to trim the check-in filelist when editing the check-in message. For example, say you have 10 files checked out scattered around the Python source tree and you want to check 9 of those in. This seems dubious, since you're not checking in the state you actually have locally, Say that 10th file is a documentation fix for a module unrelated to the other 9 files. and you were careful to run the full Python test suite with your full local state ;-) Absolutely. Er. Always. :) Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Who made me the Perforce-bitch? Here I am screaming Subversion! Subversion! and y'all think I just using that as cover for a p4 lover affair. :) [Donovan Baarda wrote] On Mon, 2005-08-08 at 15:49, Trent Mick wrote: One feature I like in Perforce (which Subversion doesn't have) is the ability to have pending changesets. A changeset is, as with subversion, something you check-in atomically. Pending changesets in Perforce allow you to (1) group related files in a source tree where you might be working on multiple things at once to ensure and (2) to build a change description as you go. In a large source tree this can be useful for separating chunks of work. This seems like a poor workaround for crappy branch/merge support. More like a pretty nice independent self-organizing feature that was necessitated as a workaround for a crappy solution (clientspecs) for huge data trees. I'm new to perforce, but the pending changesets seem dodgey to me... you are accumulating changes gradually without recording any history during the process... ie, no checkins until the end. You want to do checkins of code in a consisten state. Some large changes take a couple of days to write. During which one may have to do a couple minor things in unrelated sections of a project. Having some mechanism to capture some thoughts and be able to say these are the relevant source files for this work is handy. Creating a branch for something that takes a couple of days is overkill. Perforce branching is pretty good in my experience. For very long projects one can easily create a branch. Even worse, perforce seems to treat clients like unversioned branches, allowing you to review and test pending changesets in other clients. I'm not sure what you are talking about here. Yes, client information is stored on the server, but the *changes* (i.e. the diffs) on the client aren't so you must be talking about some other tool. Actually, if there *were* such a feature that would be quite handy. I'd love to be able to easily transfer my diffs developed on my Windows box to my Linux or Mac OS X box to quickly test changes there before checking in. This supposedly allows people to review/test each others changes before they are committed. The problem is, since these changes are not committed, there is no firm history of what what was reviewed/tested vs what gets committed... ie they could be different. The alternative being either that you have separate branches for everything (can be a pain) or just check-in for review (possibly breaking the build or functionality for other developers until the review is done). Actually the Perl guys working on PureMessage downstairs have two branches going in Perforce: one for checking into right away and then a cleaner tree to which only reviewed check-ins from the first are integrated. I'm not saying I am awash in pending changelists here. Nor that they should be used for what is better handled with branching. It is a tool (and a minor one). Trying to develop and test a mixture of different changes in one source tree is asking for trouble... they can interact. ...within reason. Trent -- Trent Mick [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Mon, 2005-08-08 at 17:51, Trent Mick wrote: [...] [Donovan Baarda wrote] On Mon, 2005-08-08 at 15:49, Trent Mick wrote: [...] You want to do checkins of code in a consisten state. Some large changes take a couple of days to write. During which one may have to do a couple minor things in unrelated sections of a project. Having some mechanism to capture some thoughts and be able to say these are the relevant I don't need to checkin in a consitent state if I'm working on a seperate branch. I can checkin any time I want to record a development checkpoint... I can capture the thoughts in the version history of the branch. source files for this work is handy. Creating a branch for something that takes a couple of days is overkill. [...] The alternative being either that you have separate branches for everything (can be a pain) or just check-in for review (possibly It all comes down to how painless branch/merge is. Many esoteric features of version control systems feel like they are there to workaround the absence of proper branch/merge histories. Note: SVN doesn't have branch/merge histories either. -- Donovan Baarda [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Donovan == Donovan Baarda [EMAIL PROTECTED] writes: Donovan It all comes down to how painless branch/merge is. Many Donovan esoteric features of version control systems feel like Donovan they are there to workaround the absence of proper Donovan branch/merge histories. It's not that simple. I've followed both the Arch and the darcs lists---they handle a lot more branch/merge scenarios than Subversion does, but you still can't get away with zero discipline. On the other hand, for the purpose of the main repository for a well-factored project with disciplined workflow like Python, it's not obvious to me that the middle-complexity scenarios are that important. Furthermore, taking good advantage of the more complex branch/merge scenarios will require a change to Python workflow (for example, push- to-tracker will no longer be a convenient way to submit patches for most developers); that will be costly. More important, since none of the core Python people have spoken up strongly in favor of an advanced system, I would guess there's little experience to support planning a new workflow. (Cf. the Linux case, where Linus opted to roll his own.) I know many people in the Emacs communities who are successfully using CVS for the main repositories and various advanced systems (prcs, darcs, arch at least) for local branching and small group project communication. It seems fairly easy to automate that (much easier than extracting changeset information from CVS!) I think that as developers find they have need for such capabilities, the practice will grow in Python too, and then there may be a case to be built for moving the main repository to such a system. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can do free software business; ask what your business can do for free software. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
M.-A. Lemburg wrote: BTW, in one of your replies I read that you had a problem with how cvs2svn handles trunk, branches and tags. In reality, this is no problem at all, since Subversion is very good at handling moves within the repository: you can easily change the repository layout after the import to whatevery layout you see fit - without losing any of the version history. Yes, however, I recall that some clients have problems with displaying history across renames (in particular, I believe viewcvs has this problem); also, it becomes difficult to refer to an old version by path name, since the old versions had all different path names. Jim Fulton has suggested a different approach: cvs2svn can create a dump file, and svnadmin load accepts a parent directory. Then, no renames are necessary. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Phillip J. Eby wrote: Yeah, in my use of SVN I find that this is more theoretical than actual for certain use cases. You can see the history of a file including the history of any file it was copied from. However, if you want to try to look at the whole layout, you can't easily get to the old locations. This can be a royal pain, whereas at least in CVS you can use viewcvs to show you the attic. Subversion doesn't have an attic, which makes looking at structural history very difficult. I guess this is a client issue also; in websvn, you can browse an older revision to see what the structure looked at that point. If you made tags, you can also browse the tags through the standard HTTP interface. I don't know a client, off-hand, which would answer the question which files have been moved since tag xyz?. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Jeff Rush wrote: BTW, re SSH access on python.org, using Apache's SSL support re https would provide as good of security without the risk of giving out shell accounts. SSL would encrypt the link and require a password or permit cert auth instead, same as SSH. Cert admin needn't be hard if only a single server cert is used, with client passwords, instead of client certs. That is the currently-proposed setup. However, with the current subversion clients, you will have to save your password to disk, or type it in every time. This is the real security disk: if somebody attacks the client machine, they get access to the python source repository. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Fernando Perez wrote: I know Joe was in contact with the SVN devs to work on this, so perhaps he's using a patched version of cvs2svn, I simply don't know. But I mention it in case it proves useful to the python.org conversion. Thanks for the pointer. It turns out that I could resolve all my conversion problems myself (following Jim Fulton's suggestion of creating dump files). I found that somebody created a patch to support different structures in cvs2svn directly, but these patches have not been integrated yet. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Donovan Baarda wrote: Yeah. IMHO the sadest thing about SVN is it doesn't do branch/merge properly. All the other cool stuff like renames etc is kinda undone by that. For a definition of properly, see; http://prcs.sourceforge.net/merge.html Can you please elaborate? I read the page, and it seems to me that subversion's merge command works exactly the way described on the page. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: M.-A. Lemburg wrote: BTW, in one of your replies I read that you had a problem with how cvs2svn handles trunk, branches and tags. In reality, this is no problem at all, since Subversion is very good at handling moves within the repository: you can easily change the repository layout after the import to whatevery layout you see fit - without losing any of the version history. Yes, however, I recall that some clients have problems with displaying history across renames (in particular, I believe viewcvs has this problem); also, it becomes difficult to refer to an old version by path name, since the old versions had all different path names. Since I only use trac to view the source code (which doesn't have this problem), I can't comment on this. Jim Fulton has suggested a different approach: cvs2svn can create a dump file, and svnadmin load accepts a parent directory. Then, no renames are necessary. Good idea. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 07 2005) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Donovan Baarda wrote: What this means is SVN has no way of automatically identifying the common version. Ah, ok. That's true. It doesn't mean you can't do proper merging with subversion - it only means that it is harder, as you need to figure out the revision range that you want to merge. If this is too painful, you can probably use subversion to store the relevant information. For example, you could define a custom property on the directory, last_merge_from_trunk, which you would always update after you have done a merge operation. Then you don't have to look through history to find out when you last merged. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: Donovan Baarda wrote: What this means is SVN has no way of automatically identifying the common version. If this is too painful, you can probably use subversion to store the relevant information. For example, you could define a custom property on the directory A script named svnmerge that does just that is included in the contrib directory of the Subversion tar. We (ZC) have just started using it to track two-way merge operations, but I don't have much experience with it personally yet. -- Benji York ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On 8/4/05, Martin v. Löwis [EMAIL PROTECTED] wrote: Nicholas Bastin wrote: Perforce is a commercial product, but it can be had for free for verified Open Source projects, which Python shouldn't have any problem with. There are other problems, like you have to renew the agreement every year, but it might be worth considering, given the fact that it's an excellent system. So we should consider it because it is an excellent system... I don't know what that means, in precise, day-to-day usage terms (i.e. what precisely would it do for us that, say, Subversion can't do). It's a mature product. I would hope that that would count for something. I've had enough corrupted subversion repositories that I'm not crazy about the thought of using it in a production system. I know I'm not the only person with this experience. Sure, you can keep backups, and not really lose any work, but we're moving over because we have uptime and availability problems, so lets not just create them again. I think anything but Subversion is ruled out because: - there is no offer to host that anywhere (for subversion, there is already svn.python.org) We could host a Perforce repository just as easily, I would think. Interesting offer. I'll add this to the PEP - who is we in this context? Uh, the Python community. Which is currently hosting a subversion repository, so it doesn't seem like a stretch to imagine that p4.python.org could exist just as easily. - there is no support for converting a CVS repository (for subversion, there is cvs2svn) I'd put $20 on the fact that cvs2svn will *not* work out of the box for converting the python repository. Just call it a hunch. You could have read the PEP before losing that money :-) It did work out of the box. Pardon me if I don't feel that I'd like to see a system in production for a few weeks before we declare victory. The problems with this kind of conversion can be very subtle, and very painful. I'm not saying we shouldn't do this, I'm just saying that we should take an appropriate measure of how much greener the grass really is on the other side, and how much work we're willing to put in to make it that way. -- Nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Sun, 2005-08-07 at 21:52, Nicholas Bastin wrote: I've had enough corrupted subversion repositories that I'm not crazy about the thought of using it in a production system. I know I'm not the only person with this experience. Sure, you can keep backups, and not really lose any work, but we're moving over because we have uptime and availability problems, so lets not just create them again. Has anyone experienced svn corruptions with the fsfs backend? I haven't, across quite a few repositories. Uh, the Python community. Which is currently hosting a subversion repository, so it doesn't seem like a stretch to imagine that p4.python.org could exist just as easily. Unfortunately, I don't think we (meaning specifically the collective python.org admins) have much if any operational experience with Perforce. We do with Subversion though and that's a big plus. If we were to host a Perforce repository, we'd need significant commitments from several somebodies to get things set up, keep it running, and help socialize long-term institutional knowledge amongst the other admins. We'd also have to teach the current crop of developers how to use the client tools effectively. I think it's fairly simple to teach a CVS user how to use Subversion, but have no idea if translating CVS experience to Perforce is as straightforward. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: M.-A. Lemburg wrote: I haven't received any offers to make a qualified statement. I only know that I would oppose an approach to ask somebody but our volunteers to do it for free, and I also know that I don't want to spend my time researching commercial alternatives (although I wouldn't mind if you spent your time). I don't quite understand what you meant here: are you opposing spending PSF money on a hosting company if and only if volunteers who take on the job don't get paid ? No. I'm opposed to approaching somebody to do it for free, except the somebody are the pydotorg volunteers (IOW, I won't take gifts from anybody else in this matter). Ok. I've done a bit of research on the subject and so far only found CollabNet and VA offering commercial services in this area. VA hosts SourceForge so that's a non-option, I guess :-) It's not that I dislike VA - I personally think they are doing a great job with SourceForge, and I like SourceForge a lot. There are just some issues with it (like that they offer no Subversion). The question would be: what precisely is the commercial offering from VA: does it provide subversion? how is the user management done? etc. I guess this was a misunderstanding on my part: VA doesn't offer their commercial solution in an ASP-like way. Their product, called SourceForge Enterprise, is a J2EE application which we'd have to install and run. They do mention Subversion as being supported by the Enterprise edition. I know that Greg Stein worked for CollabNet, so thought it might be a good idea to ask him about the idea to move things to CollabNet. Of course, before taking this route, I wanted to get a feeling for the general attitude towards a commercial approach, which is why I tossed in the idea. Ok - I expect that the project might be *done* before we even have a single commercial offer, with a precise service description, and a precise price tag. That makes commercial offers so difficult: that it is so time expensive to use them, that you might spend less time doing it yourself. For (more or less) simple things like setting up SVN, I'd agree, but for hosting a complete development system, I have my doubts - things start to get rather complicated and integration of various different tools tends to be very time consuming. Sysadmin tasks like doing backups, emergency recovery, etc. also get more complicated once you have to deal with many different ways of data storage deployed by such tools, e.g. many of them require use of special tools to do hot backups. Other non-commercial alternatives are Berlios and Savannah, but I'm not sure whether they'd offer Subversion support. For me, they fall into the I won't take gifts category. Ok, I'll drop the idea. BTW, have you considered using Trac as issue tracker on svn.python.org ? You mean, me personally? I quite like the Subversion tracker, and don't want to trade it for anything else. I know Guido wants to use Roundup (which is also written in Python), and obviously so does Richard Jones. The main questions are the same as with this PEP: how to do the migration from SF (without losing data), and how to do the ongoing maintenance. It's just that finding answers to these questions is so much harder, therefore, this PEP is *only* about CVS. Ok. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 04 2005) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: M.-A. Lemburg wrote: True, but if we never ask, we'll never know :-) My question was: Would asking a professional hosting company be a reasonable approach ? It would be an option, yes, of course. It's not an approach that *I* would be willing to implement, though. Fair enough. From the answers, I take it that there's not much trust in these offers, so I guess there's not much desire to PSF money into this. I haven't received any offers to make a qualified statement. I only know that I would oppose an approach to ask somebody but our volunteers to do it for free, and I also know that I don't want to spend my time researching commercial alternatives (although I wouldn't mind if you spent your time). I don't quite understand what you meant here: are you opposing spending PSF money on a hosting company if and only if volunteers who take on the job don't get paid ? I've done a bit of research on the subject and so far only found CollabNet and VA offering commercial services in this area. VA hosts SourceForge so that's a non-option, I guess :-) I know that Greg Stein worked for CollabNet, so thought it might be a good idea to ask him about the idea to move things to CollabNet. Of course, before taking this route, I wanted to get a feeling for the general attitude towards a commercial approach, which is why I tossed in the idea. Other non-commercial alternatives are Berlios and Savannah, but I'm not sure whether they'd offer Subversion support. BTW, have you considered using Trac as issue tracker on svn.python.org ? They have a very good subversion integration, it's easy to use, comes with a wiki and looks great. Oh, and it's written in Python :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 03 2005) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
aahz == aahz [EMAIL PROTECTED] writes: aahz I'd rather not rely on licensing of a closed-source system; aahz one of the points made during the talk was that the Linux aahz project had to scramble when they lost their Bitkeeper aahz license Python is unlikely to throw away its license in the same way, I should think. For additional security, you could try to negotiate a perpetual license on a particular version, or a license that required substantial notice (say, six months) for termination. I would imagine you could get them; the only reason for the vendor not to give them would be spite. The problem with both of those options is the one that Martin already pointed out: negotiation takes effort. There are several good open source alternatives, one of which (svn) is well-established and gets excellent reviews for those goals it sets itself, which happen to be solving the problems (as opposed to missing features) of CVS. Why spend effort on negotiating licenses and preparing for potential vendor relationship problems, unless there's acknowledged need for features svn doesn't provide? -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can do free software business; ask what your business can do for free software. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
M == M.-A. Lemburg [EMAIL PROTECTED] writes: M Other non-commercial alternatives are Berlios and Savannah, but M I'm not sure whether they'd offer Subversion support. Savannah doesn't offer great reliability or support, at least to judge by the frequency with which the GNU Emacs and GNU Arch projects have been unable to access various services on Savannah, including mailing lists and CVS. I also wonder if Savannah poses security risks. They've been successfully cracked (ISTR more than once) in the last couple of years, and took 6-10 weeks to get back to normal. This makes them reluctant to make minor variations in their established procedures for the convenience of projects. For example, it took a couple of months for GNU Arch to arrange sftp access so that they could host the Arch project in an Arch repository (Arch can use sftp but not plain ssh as a transport). SunSITE.dk does provide reliable service and timely support. XEmacs has been very happy with it. But Martin v. Loewis apparently hasn't had the same good experience with negotiating with them, and at least some negotiation and relationship maintenance is necessary---it's a closer, more personal relationship than with SF or Savannah. In particular for Subversion support (I was told they allow it on a case by case basis, and once success is demonstrated they plan to offer it in general). As I say, we've been happy with SunSITE, but the amount of effort is basically the same as if we ran our own repository, just directed more toward vendor relations and away from sys admin (which suits us). FWIW, XEmacs has moved or reorganized CVS repositories five times since 1999. Although it's not all in the PEP, if you add the discussion on this list Martin has covered the important issues we encountered or worried about. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can do free software business; ask what your business can do for free software. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Nicholas Bastin wrote: No. The PEP is only about Subversion. Why should we be looking at Per Force? Only because Python is Open Source? Perforce is a commercial product, but it can be had for free for verified Open Source projects, which Python shouldn't have any problem with. There are other problems, like you have to renew the agreement every year, but it might be worth considering, given the fact that it's an excellent system. So we should consider it because it is an excellent system... I don't know what that means, in precise, day-to-day usage terms (i.e. what precisely would it do for us that, say, Subversion can't do). I think anything but Subversion is ruled out because: - there is no offer to host that anywhere (for subversion, there is already svn.python.org) We could host a Perforce repository just as easily, I would think. Interesting offer. I'll add this to the PEP - who is we in this context? - there is no support for converting a CVS repository (for subversion, there is cvs2svn) I'd put $20 on the fact that cvs2svn will *not* work out of the box for converting the python repository. Just call it a hunch. You could have read the PEP before losing that money :-) It did work out of the box. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
George V. Neville-Neil wrote: Since Python is Open Source are you looking at Per Force which you can use for free and seems to be a happy medium between something like CVS and something horrific like Clear Case? No. The PEP is only about Subversion. Why should we be looking at Per Force? Only because Python is Open Source? I think anything but Subversion is ruled out because: - there is no offer to host that anywhere (for subversion, there is already svn.python.org) - there is no support for converting a CVS repository (for subversion, there is cvs2svn) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: M.-A. Lemburg wrote: The PSF does have a reasonable budget, so why not use it to maintain the infrastructure needed for Python development and let a company do the administration of the needed servers and the importing of the CSV and tracker items into their systems ? In principle, this might be a good idea. In practice, it falls short of details: which company, what precisely are their procedures, etc. It's not always the case that giving money to somebody really gives you back the value you expect. True, but if we never ask, we'll never know :-) My question was: Would asking a professional hosting company be a reasonable approach ? From the answers, I take it that there's not much trust in these offers, so I guess there's not much desire to PSF money into this. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 02 2005) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Donovan Baarda [EMAIL PROTECTED] writes: This is why I don't bother migrating any existing CVS projects to SVN; the benefits don't yet outweigh the pain of migrating. I think they do. I was on dialup for a while, and would have _loved_ Python to be using SVN then -- and given how long diffs can take even over my broadband connection... Cheers, mwh PS: Wot, noone's suggested git yet? :) -- C++ is a siren song. It *looks* like a HLL in which you ought to be able to write an application, but it really isn't. -- Alain Picard, comp.lang.lisp ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
[Martin von Löwis] The PEP is only about Subversion. I think anything but Subversion is ruled out because: - there is no offer to host that anywhere (for subversion, there is already svn.python.org) - there is no support for converting a CVS repository (for subversion, there is cvs2svn) I quickly discussed Subversion with a few friends. While some say Subversion is the most reasonable avenue nowadays, others them told me they found something more appealing than Subversion: http://www.venge.net/monotone/ The hosting paradigm is fairly different, and for a few weeks now, they have a CVS repository converter. In my very naive eyes, the centralised aspects of Python development are be better represented with Subversion. It is notable also that Subversion if more Python-friendly than Monotone, with its Lua-based scripting. I did not deepen why, but at first glance, Monotone does not seduce me. On the other hand, the two guys saying good about Monotone are well informed (and also well known), so I would not dismiss their opinion so lightly. So, it might be worth at least a quick look? :-) -- François Pinard http://pinard.progiciels-bpi.ca ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
[François Pinard] While some say Subversion is the most reasonable avenue nowadays, others them told me they found something more appealing than Subversion: http://www.venge.net/monotone/ The current release is 0.21 which suggests that it is not ready for primetime. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
[Raymond Hettinger] http://www.venge.net/monotone/ The current release is 0.21 which suggests that it is not ready for primetime. It suggests it, yes, and to me as well. On the other hand, there is a common prejudice that something requires many releases, or frequent releases, to be qualified as good. While it might be true on average, this is not necessarily true: some packages need not so many steps for becoming very usable, mature or stable. (Note that I'm not asserting anything about Monotone, here.) We should merely keep an open mind. -- François Pinard http://pinard.progiciels-bpi.ca ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
M.-A. Lemburg wrote: True, but if we never ask, we'll never know :-) My question was: Would asking a professional hosting company be a reasonable approach ? It would be an option, yes, of course. It's not an approach that *I* would be willing to implement, though. From the answers, I take it that there's not much trust in these offers, so I guess there's not much desire to PSF money into this. I haven't received any offers to make a qualified statement. I only know that I would oppose an approach to ask somebody but our volunteers to do it for free, and I also know that I don't want to spend my time researching commercial alternatives (although I wouldn't mind if you spent your time). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
François Pinard wrote: So, it might be worth at least a quick look? :-) Certainly not my look - although I'm willing to integrate anything that people contribute into the PEP. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
[Donovan Baarda] It is true that some well designed/developed software becomes reliable very quicky. However, it still takes heavy use over time to prove that. There is wisdom in your say! :-) -- François Pinard http://pinard.progiciels-bpi.ca ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
BAW == Barry Warsaw [EMAIL PROTECTED] writes: BAW So are you saying that moving to svn will let us do more long BAW lived branches? Yay! Yes, but you still have to be disciplined about it. svn is not much better than cvs about detecting and ignoring spurious conflicts due to code that gets merged from branch A to branch B, then back to branch A. Unrestricted cherry-picking is still out. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can do free software business; ask what your business can do for free software. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Sun, 2005-07-31 at 23:54, Stephen J. Turnbull wrote: BAW == Barry Warsaw [EMAIL PROTECTED] writes: BAW So are you saying that moving to svn will let us do more long BAW lived branches? Yay! Yes, but you still have to be disciplined about it. svn is not much better than cvs about detecting and ignoring spurious conflicts due to code that gets merged from branch A to branch B, then back to branch A. Unrestricted cherry-picking is still out. Yeah. IMHO the sadest thing about SVN is it doesn't do branch/merge properly. All the other cool stuff like renames etc is kinda undone by that. For a definition of properly, see; http://prcs.sourceforge.net/merge.html This is why I don't bother migrating any existing CVS projects to SVN; the benefits don't yet outweigh the pain of migrating. For new projects sure, SVN is a better choice than CVS. -- Donovan Baarda [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Donovan == Donovan Baarda [EMAIL PROTECTED] writes: Donovan Yeah. IMHO the sadest thing about SVN is it doesn't do Donovan branch/merge properly. All the other cool stuff like Donovan renames etc is kinda undone by that. [...] This is why Donovan I don't bother migrating any existing CVS projects to Donovan SVN; the benefits don't yet outweigh the pain of Donovan migrating. FWIW, XEmacs just had this discussion, and we basically came to the conclusion that for a multi-developer project it's _definitely_ worth the effort if it can be done by cvs2svn (which for us it probably can't, due to some black magic we did on the CVS repository a few years ago :-( ). For the record, I was opposed for exactly the reason you give, but changed my mind. The point is that with several developers there's almost surely someone enthusiastic enough about svn to bear the burden of fooling with the script for a couple of hours to see if it works, a fascist policy about migrating account names makes that almost trivial, and after that it's all gravy: the administration does not look any worse, the security issues are similar, and the change is likely to incite only a few people to press for account name changes after the move. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can do free software business; ask what your business can do for free software. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
M.-A. Lemburg wrote: The PSF does have a reasonable budget, so why not use it to maintain the infrastructure needed for Python development and let a company do the administration of the needed servers and the importing of the CSV and tracker items into their systems ? In principle, this might be a good idea. In practice, it falls short of details: which company, what precisely are their procedures, etc. It's not always the case that giving money to somebody really gives you back the value you expect. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: Fred L. Drake, Jr. wrote: More interestingly, keeping it in a single repository makes it easier to merge projects, or parts of projects, together, without losing the history. This would be useful when developing packages that may be considered for the standard library, but which also need to continue separate releases to support older versions of Python. We've found it very handy to keep multiple projects in a single repository for zope.org. So do you use project/trunk or trunk/project? If the former, I would need to get instructions on how to do the conversion from CVS. For ipython, which recently went through cvs2svn, I found that moving over to a project/trunk structure was a few minutes worth of work. Since svn has moving commands, it was just a matter of making the extra project/ directory and moving things one level down the hierarchy. Even if cvs2svn doesn't quite create things the way you want them in the long run, svn is flexible enough that a few manual tweaks should be quite easy to perform. Cheers, f ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis [EMAIL PROTECTED] writes: - Subversion over SSH, using SSH key pairs. This would require to give committers accounts on the machine, which currently is ruled out by the administration policy of svn.python.org. Would it work/how much risk would it be to create accounts with shell /bin/false? Cheers, mwh (still faintly bothered by ~/.subversion/auth/svn.simple/*...) -- If trees could scream, would we be so cavalier about cutting them down? We might, if they screamed all the time, for no good reason. -- Jack Handey ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: I'd like to see the Python source be stored in Subversion instead of CVS, +1 and on python.org instead of sf.net. To facilitate discussion, I have drafted a PEP describing the rationale for doing so, and the technical procedure to be performed. Not sure about the move to svn.python.org. This would bind additional developer resources for doing administration work. If SF is such a show-stopper, would it be reasonable to look for managed alternatives, such as eg. CollabNet (who funded Subversion development) ? Perhaps we could get a good deal from them given that the PSF is a non-profit organization. Greg Stein could probably provide contacts to the right people to talk to. http://www.collab.net/products/team_edition/index.html Just an idea, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 29 2005) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Tim Peters wrote: ... I'm sending this to Jim Fulton because he did the conversion of Zope Corp's code base to SVN. Unfortunately, Jim will soon be out of touch for several weeks. Jim, do you have time to summarize the high bits of the problems you hit? IIRC, you didn't find any conversion script at the time capable of doing the whole job as-is. If that's wrong, it would be good to know that too. I had two problems, one of which was zope specific: 1. We were making extensive use of symbolic links to share packages among various Zope projects. This requires special care and was the main reason I wrote my own scrips. I don't expect that this would be an issue for Python. 2. I initially tried to conver our entire repository, including all branches. This would have taken days. I finally decided to just convert our trunk, which took several hours. The main time sink was in the load step of the conversion process. I suspect that much of the time hit was due to the Berkely DB back end. I think I've heard that the file-system-based back end is much faster in general. We're using the BDB back end because that's all that was available at the time we converted. Every few weeks, the database gets wedged and I have to do a recovery operation. Every few months, the machine gets wedged and required a reboot. I'm pretty sure the later is due to locking issues. I definately recommend the file-system back-end, even though I haven't tried it myself. I haven't had the time to do the conversion myself. :( I'll also say that, on balance, I'm *very* *very* happy with subversion. I recommend it highly. Other than that, I'd just like to see an explicit mention in the PEP of a plan to preserve the pre-conversion CVS tarball forever. +1 Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Tim Peters wrote: [Jeff Rush] The conversion script isn't perfect and does fail on complex CVS arrangements or where there is extensive history to migate. However it appears above that Martin has already tried the script out, with success. I'd still like to hear from Jim, as I don't believe all serious problems were identified by eyeball at once. I'm not aware of any errors in the conversion. ... Ah, before I forget, single repository has worked very well for Zope Yup. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: Tim Peters wrote: Ah, before I forget, single repository has worked very well for Zope (which includes top-level Zope2, Zope3, ZODB, ZConfig, zdaemon, ... projects): http://svn.zope.org/ Long URLs don't really get in the way in practice (rarely a need to type one after initial checkout; even svn switch is usually just a tail-end cmdline edit starting from a copy+paste of svn info output). That would indeed give conversion problems: Nah cvs2svn automatically generates tags/trunk/branches in the root, with the original CVS modules below; there is a single space for tags and branches (as is in the original CVS repository). I would see two possible conversion procedures: 1. convert the modules one-by-one, adding to the same repository. I.e. python would get all the small revision numbers, then peps, then sandbox, and so on. Cross-module tags would not be supported (but likely don't exist in the first place), and the revision number would not increase in historical order. 2. convert the entire CVS, then rearrange things through svn move. This would be tedious to do (atleast for tags/branches), and it would cause all files to be renamed right from the scratch (some svn clients fail to display history past moves). You reminded me of another reason why I used a custom conversion script. :) I did convert projects individually. I told cvs2svn to just create dump files. I then used svnload to load the dump files myself so that I could make each project a top-level directory with it's own trunk, branches and tags. I'd be happy to share my scrips, although there's nothing all that complicated about them. So for all who would prefer to see a single repository, could somebody please 1. state how precisely the repository should be organized (with exact URLs on svn.python.org - eg. which of the non-dist directories becomes toplevel, which ones get their own tags/branches/trunk, etc). I'm not close enough to the Python repo to offer much of an opinion other than: - IMO, a single repository is good - The repository should be orgabized by projects, where separate projects reflect more or less independent development efforts with their own teams and schedules. (Maybe Python doesn't have many of these.) 2. state how the conversion should be performed Once you decide what the projects are, I suggest converting each project separately. I'd be happy to share my scrips and experenice, although, as Tim noted I'll be off-line for two or three weeks starting in the next few days. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Friday 29 July 2005 06:34, M.-A. Lemburg wrote: If SF is such a show-stopper, would it be reasonable to look for managed alternatives, such as eg. CollabNet (who funded Subversion development) ? docutils has been using berlios.de for Subversion; that might be a reasonable option. I'm not sure if there are any political issues about that, but I think everyone actively developing on that project has been happy with the move. (Only the berlios.de Subversion is being used; everything else remains at SourceForge IIRC.) -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: So do you use project/trunk or trunk/project? If the former, I would need to get instructions on how to do the conversion from CVS. project/trunk/ On Friday 29 July 2005 02:12, Fernando Perez wrote: For ipython, which recently went through cvs2svn, I found that moving over to a project/trunk structure was a few minutes worth of work. Since svn has moving commands, it was just a matter of making the extra project/ directory and moving things one level down the hierarchy. Even if cvs2svn doesn't quite create things the way you want them in the long run, svn is flexible enough that a few manual tweaks should be quite easy to perform. Yes, this will work. I vaguely recall that Jim converted the zope.org repository one project at a time, so that made it easier to keep them separate. We didn't decommission the CVS entirely, though; Zope 2.7 is still maintained in CVS since we didn't want to risk worrying about the branch structure too much; cvs2svn still had a few issues with the complex branch structure combined with the use of symlinks within the repository (one of the reasons we were so keen to move to Subversion). I'm sure Jim can provide more details of what he had to do; I was only slightly involved in the discussions. -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Friday 29 July 2005 09:17, Jim Fulton wrote: 1. We were making extensive use of symbolic links to share packages among various Zope projects. This requires special care and was the main reason I wrote my own scrips. I don't expect that this would be an issue for Python. I know of three symlinks in the Python repository, all from the distutils project. There's one for the package and one for each of the documents. 2. I initially tried to conver our entire repository, including all branches. This would have taken days. I finally decided to just convert our trunk, which took several hours. The main time sink was in the load step of the conversion process. This might be a possibility for Python as well, though we have a much less complex branching structure, so the conversion may not be so difficult. -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
M.-A. Lemburg wrote: and on python.org instead of sf.net. To facilitate discussion, I have drafted a PEP describing the rationale for doing so, and the technical procedure to be performed. Not sure about the move to svn.python.org. This would bind additional developer resources for doing administration work. True. OTOH, there already is a subversion on svn.python.org, and administrative work is likely only about creating new accounts. I guess the current SF project admins would help out if help is needed. If SF is such a show-stopper It is at the moment, atleast: there currently is not svn support on sf.net. would it be reasonable to look for managed alternatives, such as eg. CollabNet (who funded Subversion development) ? Perhaps. Somebody would need to research the precise migration procedure. I once tried to move the Python CVS to Sunsite (or its successors), and gave up after half a year of getting nowhere; I'm personally not keen on repeating such an experience. However, if somebody stepped forward to do the actual work (perhaps based on the draft PEP), I would happily hand this project over (it would be good if that volunteer would set a deadline for negotiations with the new host). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Jim Fulton wrote: I don't expect that this would be an issue for Python. Right. 2. I initially tried to conver our entire repository, including all branches. This would have taken days. I finally decided to just convert our trunk, which took several hours. The main time sink was in the load step of the conversion process. I suspect that much of the time hit was due to the Berkely DB back end. I think I've heard that the file-system-based back end is much faster in general. Dunno. For the Python CVS (which translates into some 40,000 revisions), on the machine which I was originally using (1GHz Pentium), conversion took 7h. On my current workstation, it takes a little over an hour. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Jim Fulton wrote: I did convert projects individually. I told cvs2svn to just create dump files. I then used svnload to load the dump files myself so that I could make each project a top-level directory with it's own trunk, branches and tags. I'd be happy to share my scrips, although there's nothing all that complicated about them. If that's how it worked, I'm sure I can cook my own. Just for confirmation: the svn revision numbers don't increase chronologically across 'modules', right: i.e. the first revision of the module that was converted second has a higher revision than the last revision of the first module, even though historically, the order should have been reverse. Not that this worries me much, but I'd like to confirm there is no other way. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: Jim Fulton wrote: I don't expect that this would be an issue for Python. Right. 2. I initially tried to conver our entire repository, including all branches. This would have taken days. I finally decided to just convert our trunk, which took several hours. The main time sink was in the load step of the conversion process. I suspect that much of the time hit was due to the Berkely DB back end. I think I've heard that the file-system-based back end is much faster in general. Dunno. For the Python CVS (which translates into some 40,000 revisions), on the machine which I was originally using (1GHz Pentium), conversion took 7h. On my current workstation, it takes a little over an hour. Was this with the file-system back end? What is your current system? Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Fernando Perez wrote: For ipython, which recently went through cvs2svn, I found that moving over to a project/trunk structure was a few minutes worth of work. Since svn has moving commands, it was just a matter of making the extra project/ directory and moving things one level down the hierarchy. Even if cvs2svn doesn't quite create things the way you want them in the long run, svn is flexible enough that a few manual tweaks should be quite easy to perform. Doesn't this give issues as *every* file the starts out renamed? e.g. what if you want revision 100 of project/trunk/foo, but, at revision 100, it really was trunk/project/foo? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Michael Hudson wrote: Would it work/how much risk would it be to create accounts with shell /bin/false? It seems that the pydotorg admins are worried about such a prospect. I believe this alone either won't work or won't be good enough (not sure which one): If you have /bin/false as login shell, and still manage to invoke /usr/bin/svnserve remotely, you can likely also invoke /usr/bin/cat /etc/passwd remotely (or download and build the root exploit via ssh). So you would have restrict the set of valid programs to *only* svnserve. This is possible, but difficult to manage (AFAIK). (still faintly bothered by ~/.subversion/auth/svn.simple/*...) Indeed. I personally would prefer SSL client certificates. This is still tricky (where do you get the passphrase for the private key from (*)), but slightly better. Regards, Martin (*) In case you wonder, I'm personally using the following techniques here: - on windows, remove the passphrase from the private key/.p12 file, and encrypt the file through NTFS encryption. Then you don't need to enter a passphrase, but still nobody can steal the private key from your disk (as long as you are not logged in; the same holds for ssh-agent) - on Linux, my issue is that .subversion is on NFS, so any root user in our net can connect to the file. Therefore, I copy the .p12 file to /tmp/private_dir, and remove the passphrase there. No other machine can read the file (as /tmp is not exported), and the file goes away after machine shutdown latest (as tmp is cleaned on reboot). - again on windows, I'm working on teaching subversion the Microsoft certificate store. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Jim Fulton wrote: Dunno. For the Python CVS (which translates into some 40,000 revisions), on the machine which I was originally using (1GHz Pentium), conversion took 7h. On my current workstation, it takes a little over an hour. Was this with the file-system back end? What is your current system? Yes, and it is a 3 GHz dual processor (although I don't think it uses two processors) with 1GB RAM (I believe RAM size matters a lot for this process; the older machine has 512MB). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Fri, 2005-07-29 at 01:00, Martin v. Löwis wrote: Barry Warsaw wrote: We won't use plain text, but we may (or, we currently do) use basic auth over ssl. The security then is in the passwords, so we have to make sure they're generated securely. That (sort of) *is* plain text passwords. Somebody who took over svn.python.org can get the password. In public-key or digest authentication, this won't be possible. Actually, the passwords are still hashed in the file, so they wouldn't be able to extract the plain text password. They definitely are vulnerable to brute force attack, though probably not to a dictionary attack. In practice I've been using a password generated based on os.urandom() -- we generate the password and get it to the Subversion user via a secure route heh. I'd be happy to share my password generation script with anybody who wants to audit it. Public/private keys would be better, and if anybody knows how to set up a Subversion server to use these without having to create accounts for everyone, I think we (the pythong.org admins) would love your help. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Fri, 2005-07-29 at 00:44, Martin v. Löwis wrote: - assignment of passwords. This I don't like about the current pydotorg setup - there should be a way to chose your own password; perhaps without involving an administrator. I could imagine a web form for password change, and administrator interaction in case of a lost password. I disagree. By reserving password generation to the pydotorg admins, we can better insure the passwords are more robust against dictionary attacks. See my previous message. I actually /don't/ want individuals to be able to set their own passwords. In practice, you only have to know your password once, because svn caches the authentication (yes, that opens up opportunities for compromise, but that's how svn works). - compromised passwords. The only tricky question then is: was the repository altered? Fortunately, for Subversion, there should be an easy way to tell: in fsfs, files never change (only new files are added). So we could generate md5sums of all files in the repository, and download these to an offsite place. If the md5sum of an immutable file changes, we were compromised (there are, of course, a few files that do change regularly). Of course, we also need regular backups of the entire data so we can restore them if they got compromised. +1 to all that. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: Fernando Perez wrote: For ipython, which recently went through cvs2svn, I found that moving over to a project/trunk structure was a few minutes worth of work. Since svn has moving commands, it was just a matter of making the extra project/ directory and moving things one level down the hierarchy. Even if cvs2svn doesn't quite create things the way you want them in the long run, svn is flexible enough that a few manual tweaks should be quite easy to perform. Doesn't this give issues as *every* file the starts out renamed? e.g. what if you want revision 100 of project/trunk/foo, but, at revision 100, it really was trunk/project/foo? To be honest, I don't really know the details, but it seems to work fine. A quick look at ipython: planck[IPython] svn update At revision 661. planck[IPython] svn diff -r 10 genutils.py | tail - -Deprecated: this function has been superceded by timing() which has better -fucntionality. - -rng = range(reps) -start = time.clock() -for dummy in rng: func(*args,**kw) -return time.clock()-start - -#*** end of file genutils.py ** Revision 10 was most certainly back in the early CVS days, and the wholesale renaming happened when I started using svn, which was around revision 600 or so. There may be other subtleties I'm missing, but so far I haven't experienced any problems. Cheers, f ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Barry Warsaw wrote: Public/private keys would be better, and if anybody knows how to set up a Subversion server to use these without having to create accounts for everyone, I think we (the pythong.org admins) would love your help. I'll play around with it some this weekend, see what's possible, and was isn't. I'm thinking we ought to be able to use SSH's configuration to only allow one specific command, e.g. something like this in the authorized_keys: no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,command=/usr/bin/svnserve -- Leif ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Fri, 2005-07-29 at 17:19, Martin v. Löwis wrote: I believe this alone either won't work or won't be good enough (not sure which one): If you have /bin/false as login shell, and still manage to invoke /usr/bin/svnserve remotely, you can likely also invoke /usr/bin/cat /etc/passwd remotely (or download and build the root exploit via ssh). So you would have restrict the set of valid programs to *only* svnserve. This is possible, but difficult to manage (AFAIK). I think that's basically right. - on Linux, my issue is that .subversion is on NFS, so any root user in our net can connect to the file. Therefore, I copy the .p12 file to /tmp/private_dir, and remove the passphrase there. No other machine can read the file (as /tmp is not exported), and the file goes away after machine shutdown latest (as tmp is cleaned on reboot). I don't think that's true on all Linuxes though (or even all *nixes). -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Fri, 2005-07-29 at 17:32, Martin v. Löwis wrote: Was this with the file-system back end? What is your current system? Yes, and it is a 3 GHz dual processor (although I don't think it uses two processors) with 1GB RAM (I believe RAM size matters a lot for this process; the older machine has 512MB). That matches my experience w/ the fsfs backend. If we do the conversion on one of the new python.org boxes, I expect it to go pretty quickly. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Barry Warsaw wrote: That (sort of) *is* plain text passwords. Somebody who took over svn.python.org can get the password. In public-key or digest authentication, this won't be possible. Actually, the passwords are still hashed in the file, so they wouldn't be able to extract the plain text password. Nah. Somebody who takes over svn.python.org can replace Apache, and that will receive plain text passwords over the wire (in case you wonder: modules/aaa/mod_auth.c:authenticate_real_user - you can even write an Apache module that gets hold of the sent password). An intruder would have to wait some time before the password come in, instead of being able to read them all from a file at once - that's true. Public/private keys would be better, and if anybody knows how to set up a Subversion server to use these without having to create accounts for everyone, I think we (the pythong.org admins) would love your help. Ok. Since this falls in my research interest, I definitely want to give it a try. I think I would set up PyCA to let users generate their private keys in the browser. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Fri, 2005-07-29 at 00:50, Christopher Petrilli wrote: Another thing to look at would be Trac, which we are in the process of moving to from the horrendous nightmare of Bugzilla. It's integration with SVN as well as Wiki is quite amazing. Now's the time I pipe in to remind everyone that Atlassian has offered free (as in beer) versions of Jira and Confluence for the Python project (actually any open source project). If you haven't used these, they're definitely worth a look. Jira is the issue tracker, Confluence the wiki. They're extremely professional, usable, well-integrated, and you can integrate them with Subversion. We've used them at work for maybe a year now and I've been very happy with them. Jira is definitely one of the better issue trackers, free or not free, that I've used. www.atlassian.com They're not Python and they're not open source, so perhaps it's legitimate to dismiss them because of that. But they are also definitely cool. At the Atlassian folks are very cool too, and fans of FOSS. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Fri, 2005-07-29 at 17:21, Martin v. Löwis wrote: Doesn't this give issues as *every* file the starts out renamed? e.g. what if you want revision 100 of project/trunk/foo, but, at revision 100, it really was trunk/project/foo? I think it only matters if you use urls. I working directories, I think everything gets tracked correctly. I could be wrong about that though. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Fri, 2005-07-29 at 18:12, Leif Hedstrom wrote: Barry Warsaw wrote: Public/private keys would be better, and if anybody knows how to set up a Subversion server to use these without having to create accounts for everyone, I think we (the pythong.org admins) would love your help. I'll play around with it some this weekend, see what's possible, and was isn't. I'm thinking we ought to be able to use SSH's configuration to only allow one specific command, e.g. something like this in the authorized_keys: no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,command=/usr/bin/svnserve But that would still require us to create accounts for every committer, right? -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
At 06:39 PM 7/29/2005 -0400, Barry Warsaw wrote: But that would still require us to create accounts for every committer, right? No. Just one account. You can have more than one key listed in authorized_keys, using svnserve --tunnel-user and sshd will select the right command based on the public key the client authenticates with. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Barry Warsaw wrote: Now's the time I pipe in to remind everyone that Atlassian has offered free (as in beer) versions of Jira and Confluence for the Python project (actually any open source project). If you haven't used these, they're definitely worth a look. Jira is the issue tracker, Confluence the wiki. They're extremely professional, usable, well-integrated, and you can integrate them with Subversion. We've used them at work for maybe a year now and I've been very happy with them. Jira is definitely one of the better issue trackers, free or not free, that I've used. www.atlassian.com I've also used Confluence at work, and been very impressed. A very nice feature which I haven't seen anywhere else is that the Wiki pages allow comments as well as direct editing - it allows discussion *about* the page to occur in a normal answer/response fashion, possibly leading to eventual update of the page text itself. They're not Python and they're not open source, so perhaps it's legitimate to dismiss them because of that. But they are also definitely cool. At the Atlassian folks are very cool too, and fans of FOSS. I'd hope we wouldn't dismiss them out of hand - one of the things that appeals to me about Python is the philosophy that open-source and protected-source groups can both benefit from working together. Regards, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On 7/28/05, Martin v. Löwis [EMAIL PROTECTED] wrote: I'd like to see the Python source be stored in Subversion instead of CVS, and on python.org instead of sf.net. To facilitate discussion, I have drafted a PEP describing the rationale for doing so, and the technical procedure to be performed. In principle I'm +1 on this (both aspects). I don't know enough to understand all the subtleties. I hope we're correctly estimating the effort required to manage the server and the users. Managing users is especially important -- if a user is compromised (as has happened in the past for python.org users) the whole repository is compromised. Now this could happen to SF users too, but I'm not sure that we know all the tricks in the book to prevent attacks; SF has been doing this for years and that's an aspect of SF that I trust (I think I've heard that they have even modified their SSH server to be stricter). -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Thu, Jul 28, 2005 at 10:00:00PM +0200, Martin v. L?wis wrote: I'd like to see the Python source be stored in Subversion instead of CVS, and on python.org instead of sf.net. +1, +1. CVS has a number of limitations that have been elimintation by Subversion. For the development of Python, the most notable improvements are: - ability to rename files and directories, and to remove directories, while keeping the history of these files. - support for change sets (sets of correlated changes to multiple files) through global revision numbers. - support for offline diffs, which is useful when creating patches. -- transactional operation - a changeset is either committed or rolled back at once; -- very effective (both in terms of speed and memory) tagging and branching; tags and branches are very easy to understand and use in SVN. Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[EMAIL PROTECTED] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Jul 28, 2005, at 4:20 PM, Guido van Rossum wrote: Managing users is especially important -- if a user is compromised (as has happened in the past for python.org users) the whole repository is compromised. Now this could happen to SF users too, but I'm not sure that we know all the tricks in the book to prevent attacks; SF has been doing this for years and that's an aspect of SF that I trust (I think I've heard that they have even modified their SSH server to be stricter). If you use the fsfs storage mechanism for subversion, it is somewhat simpler to verify that the repository is not compromised. Each commit is represented as a separate file, and thus old commits are never modified. Only new files are appended to the directory. If you have a filesystem that allows append-only permissions on a directory, you can enforce this directly. Additionally, it is possible in your backup script to verify that only new files were added and nothing else changed. Then at least you know how much you need to examine instead of having to treat the entire repository as possibly contaminated. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Thu, 2005-07-28 at 16:00, Martin v. Löwis wrote: I'd like to see the Python source be stored in Subversion instead of CVS +1 , and on python.org instead of sf.net. +0 I know that SF has promised svn access to projects for a long time now, but I haven't heard anything from them in a long time. It's listed under their Strategic Projects but the last update to that news item was back in April. Question: do we wait for SF to make the service available (after getting more up-to-date status and a realistic timetable), or do we go straight to svn.python.org? This is for discussion on python-dev and eventual BDFL pronouncement; if you see a reason not to make such a change, or if you would prefer a different procedure, please speak up. Encouragement and support is welcome, as well :-) Thanks for writing this PEP Martin! 1. Assign passwords for all current committers for use on svn.python.org. User names on SF and svn.python.org should be identical, unless some committer requests a different user name. We've been going with firstname.lastname (with some exceptions -- hi Aahz! :) for the current svn access. Is it better to stay with that convention or to maintain consistency with SF's CVS committer names? Maybe the latter for revision history consistency. 2. At the beginning of the migration, announce that the repository on SourceForge closed. We can probably play games with the various CVS hooks to disable all checkins during this time. We might also want to disable the u/i access to CVS at the same time. 4. Convert the CVS repository into two subversion repositories, one for distutils and one for Python. Do we also want to split off nondist and encodings? IWBNI the Python source code proper weren't buried too deep in the directory structure. Note that we might want to provide different access permission to different parts of the repository (but I think we can do that even if we don't split those off into separate repos). As the repository format, fsfs should be used (requires Subversion 1.1). fsfs has the advantage of being more backup-friendly, as it allows to backup a repository incrementally, without requiring to run any dump commands. Definitely +1 on fsfs. Again, thanks Martin! -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Thu, 2005-07-28 at 16:20, Guido van Rossum wrote: I hope we're correctly estimating the effort required to manage the server and the users. Yah, me too! ;) We are building some experience with this though, having moved many of the system files, and all of the web pages, to svn. So far, the management overhead has been almost nil (um, None :). We'll have a bit of ongoing work to add users, but the infrastructure team is also building up some community knowledge about how to do that. Managing users is especially important -- if a user is compromised (as has happened in the past for python.org users) the whole repository is compromised. Now this could happen to SF users too, but I'm not sure that we know all the tricks in the book to prevent attacks; SF has been doing this for years and that's an aspect of SF that I trust (I think I've heard that they have even modified their SSH server to be stricter). James has a very interesting idea for mitigating this. Presumably heh, we'll have backups of everything. I'll feel better when we have coverage from maybe 6 admins spanning as many timezones as possible. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Thu, 2005-07-28 at 17:58, James Y Knight wrote: If you use the fsfs storage mechanism for subversion, it is somewhat simpler to verify that the repository is not compromised. Each commit is represented as a separate file, and thus old commits are never modified. Only new files are appended to the directory. If you have a filesystem that allows append-only permissions on a directory, you can enforce this directly. Additionally, it is possible in your backup script to verify that only new files were added and nothing else changed. Then at least you know how much you need to examine instead of having to treat the entire repository as possibly contaminated. Would it buy us any additional piece of mind to checksum the transaction files as they're committed and store those checksums outside the repository? -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
[...] Publish the Repositories [...] As an option, websvn (available e.g. from the Debian websvn package) could be provided. Is there any reason that this should be an option, and not just done? For occasional source (particularly C source) lookups, I've found webcvs really useful (especially when on a machine without cvs or ssh). I presume that I'm not alone here. If there are issues with it (stability, security, whatever), then I could understand making it optional, but otherwise I think it would be great if the PEP just included it. =Tony.Meyer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Do we also want to split off nondist and encodings? IWBNI the Python source code proper weren't buried too deep in the directory structure. +1 =Tony.Meyer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On 7/28/05, Barry Warsaw [EMAIL PROTECTED] wrote: On Thu, 2005-07-28 at 16:00, Martin v. Löwis wrote: I'd like to see the Python source be stored in Subversion instead of CVS +1 +1 from me as well; single commit numbers for commits across multiple files will be wonderful. , and on python.org instead of sf.net. +0 I know that SF has promised svn access to projects for a long time now, but I haven't heard anything from them in a long time. It's listed under their Strategic Projects but the last update to that news item was back in April. Question: do we wait for SF to make the service available (after getting more up-to-date status and a realistic timetable), or do we go straight to svn.python.org? I say forget SF and we move it. Of course I won't be involved with the migration so me saying this doesn't mean too much. =) 1. Assign passwords for all current committers for use on svn.python.org. User names on SF and svn.python.org should be identical, unless some committer requests a different user name. We've been going with firstname.lastname (with some exceptions -- hi Aahz! :) for the current svn access. Is it better to stay with that convention or to maintain consistency with SF's CVS committer names? Maybe the latter for revision history consistency. I say go with the first.last naming. While this might put committer names out of sync, we could keep a mapping of SF names to the new names in developers.txt for easy referencing. But it would be handy to have actual name references since I know I don't always remember who is whom on SF since some people go with nicks that are not based on their name at all. [SNIP] 4. Convert the CVS repository into two subversion repositories, one for distutils and one for Python. Do we also want to split off nondist and encodings? IWBNI the Python source code proper weren't buried too deep in the directory structure. Note that we might want to provide different access permission to different parts of the repository (but I think we can do that even if we don't split those off into separate repos). Seems like a reasonable thing. Would make it easier for occasional committers as well as people who check out the code just for generating a patch. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: Converting the CVS Repository = The Python CVS repository contains two modules: distutils and python. Keeping them together will produce quite long repository URLs, so it is more convenient if the Python CVS and the distutils CVS are converted into two separate repositories. If I understand things correctly, one project/one repo creates a 'hard' barrier for moving code across projects (while retaining history, so done via an svn command). Is the 'long url' really the only argument for this, and is it significant enough? Instead of: https://svn.python.org/python https://svn.python.org/distutils you could have https://svn.python.org/main/python https://svn.python.org/main/distutils or something similar. It's an extra few chars, and it would give a convenient way to branch off pieces of the main code into their own subprojects in the future if needed. For more experimental things, you can always have other repos: https://svn.python.org/someotherrepo/... But maybe the issue of moving code isn't too important, I'm certainly no expert on svn. Cheers, f ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
I theory Subversion should allow you to be more secure. CVS has a very limited concept of security and for the most part you need to rely on SSH. Subversion makes use of Apache as one of its server options. Any authentication method you can use in Apache 2 you can use for Subversion. Once Apache does the authentication, Subversion allows fairly fine grained access control. SSL can be used for encrypting communication. Note that there are sites like Sourceforge that do provide Subversion hosting to open source projects. I don't know enough about them to be able to recommend any. -Chris On Thu, Jul 28, 2005 at 01:20:14PM -0700, Guido van Rossum wrote: On 7/28/05, Martin v. L?wis [EMAIL PROTECTED] wrote: I'd like to see the Python source be stored in Subversion instead of CVS, and on python.org instead of sf.net. To facilitate discussion, I have drafted a PEP describing the rationale for doing so, and the technical procedure to be performed. In principle I'm +1 on this (both aspects). I don't know enough to understand all the subtleties. I hope we're correctly estimating the effort required to manage the server and the users. Managing users is especially important -- if a user is compromised (as has happened in the past for python.org users) the whole repository is compromised. Now this could happen to SF users too, but I'm not sure that we know all the tricks in the book to prevent attacks; SF has been doing this for years and that's an aspect of SF that I trust (I think I've heard that they have even modified their SSH server to be stricter). -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/lambacck%40computer.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Martin v. Löwis wrote: Currently, access to Subversion on svn.python.org uses WebDAV over https, using basic authentication. For this to work, authorized users need to provide a password. This mechanism should be used, atleast initially, for the Python CVS as well, since various committers also have a username/password pair for the www SVN repository already. Alternatives to password-based access include: - Subversion over SSH, using SSH key pairs. This would require to give committers accounts on the machine, which currently is ruled out by the administration policy of svn.python.org. - Subversion over WebDAV, using SSL client certificates. This would work, but would require to administrate a certificate authority. I'm definitely positive to a migration to Subversion, but I'd be really concerned about using plain text authentication mechanisms. SSH obviously is much prefered, and clearly there are ways to limit the accounts on the svn.python.org, many other projects does this already for cvs. E.g thor (17:11) 350/0 $ ssh -l 'username' cvs.mozilla.org To use anonymous CVS install the latest version of CVS on your local machine. Then set your CVSROOT environment variable to the following value: [EMAIL PROTECTED]:/cvsroot Connection to cvs.mozilla.org closed. We should have enough man powers to come up with some secure solution here :). -- Leif ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
[Martin v. Löwis] I'd like to see the Python source be stored in Subversion instead of CVS, and on python.org instead of sf.net. To facilitate discussion, I have drafted a PEP describing the rationale for doing so, and the technical procedure to be performed. This is for discussion on python-dev and eventual BDFL pronouncement; if you see a reason not to make such a change, or if you would prefer a different procedure, please speak up. Encouragement and support is welcome, as well :-) Encouragement and support on SVN, undecided on moving to python.org (don't know when SF intends to support SVN, don't have a feel for the state of-- or propsects for ongoing --python.org volunteerism). ... The conversion should be done using cvs2svn utility, available e.g. in the cvs2svn Debian package. The command for converting the Python repository is cvs2svn -q --encoding=latin1 --force-branch=cnri-16-start --force-branch=descr-branch --force-branch=release152p1-patches --force-tag=r16b1 --fs-type=fsfs -s py.svn.new python/python The command to convert the distutils repository is cvs2svn -q --encoding=latin1 --fs-type=fsfs -s dist.svn.new python/distutils Sample results of this conversion are available at http://www.dcl.hpi.uni-potsdam.de/python/ http://www.dcl.hpi.uni-potsdam.de/distutils/ I'm sending this to Jim Fulton because he did the conversion of Zope Corp's code base to SVN. Unfortunately, Jim will soon be out of touch for several weeks. Jim, do you have time to summarize the high bits of the problems you hit? IIRC, you didn't find any conversion script at the time capable of doing the whole job as-is. If that's wrong, it would be good to know that too. Other than that, I'd just like to see an explicit mention in the PEP of a plan to preserve the pre-conversion CVS tarball forever. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Thursday 28 July 2005 07:21 pm, Tim Peters wrote: [Martin v. Löwis] The conversion should be done using cvs2svn utility, available e.g. in the cvs2svn Debian package. The command for converting the Python repository is snip Sample results of this conversion are available at http://www.dcl.hpi.uni-potsdam.de/python/ http://www.dcl.hpi.uni-potsdam.de/distutils/ I'm sending this to Jim Fulton because he did the conversion of Zope Corp's code base to SVN. Unfortunately, Jim will soon be out of touch for several weeks. Jim, do you have time to summarize the high bits of the problems you hit? IIRC, you didn't find any conversion script at the time capable of doing the whole job as-is. If that's wrong, it would be good to know that too. The conversion script isn't perfect and does fail on complex CVS arrangements or where there is extensive history to migate. However it appears above that Martin has already tried the script out, with success. BTW, re SSH access on python.org, using Apache's SSL support re https would provide as good of security without the risk of giving out shell accounts. SSL would encrypt the link and require a password or permit cert auth instead, same as SSH. Cert admin needn't be hard if only a single server cert is used, with client passwords, instead of client certs. -Jeff ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Thu, 2005-07-28 at 20:15, Leif Hedstrom wrote: I'm definitely positive to a migration to Subversion, but I'd be really concerned about using plain text authentication mechanisms. We won't use plain text, but we may (or, we currently do) use basic auth over ssl. The security then is in the passwords, so we have to make sure they're generated securely. We should have enough man powers to come up with some secure solution here :). Volunteers (i.e. those with actual free time to give on implementing any solution) are definitely welcome. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Jul 28, 2005, at 3:19 PM, Barry Warsaw wrote: On Thu, 2005-07-28 at 20:15, Leif Hedstrom wrote: I'm definitely positive to a migration to Subversion, but I'd be really concerned about using plain text authentication mechanisms. We won't use plain text, but we may (or, we currently do) use basic auth over ssl. The security then is in the passwords, so we have to make sure they're generated securely. Users can generate their hashed password with the htpasswd command locally, and send those to the server. That's what most subversion hosts seem to prefer. -bob ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
Tim Peters wrote: [Martin v. Löwis] The conversion should be done using cvs2svn utility, available e.g. in the cvs2svn Debian package. The command for converting the Python repository is [...] I'm sending this to Jim Fulton because he did the conversion of Zope Corp's code base to SVN. Unfortunately, Jim will soon be out of touch for several weeks. Jim, do you have time to summarize the high bits of the problems you hit? IIRC, you didn't find any conversion script at the time capable of doing the whole job as-is. If that's wrong, it would be good to know that too. If you hit any snags, you may be interested in contacting the admin for scipy.org. The scipy CVS repo choked cvs2svn pretty badly a while ago, but recent efforts eventually prevailed. This afternoon an email arrived from him: Original Message Subject: [SciPy-dev] SciPy CVS to Subversion migration Date: Thu, 28 Jul 2005 20:02:59 -0500 From: Joe Cooper [EMAIL PROTECTED] Reply-To: SciPy Developers List [EMAIL PROTECTED] To: SciPy Dev List [EMAIL PROTECTED] Hi all, The issues with converting our CVS repository to Subversion have been resolved, and so I'd like to make the switch tomorrow (Friday) afternoon. [...] I know Joe was in contact with the SVN devs to work on this, so perhaps he's using a patched version of cvs2svn, I simply don't know. But I mention it in case it proves useful to the python.org conversion. cheers, f ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
[Jeff Rush] The conversion script isn't perfect and does fail on complex CVS arrangements or where there is extensive history to migate. However it appears above that Martin has already tried the script out, with success. I'd still like to hear from Jim, as I don't believe all serious problems were identified by eyeball at once. That said, the Python project has made relatively very little use of complex (for CVS) concepts like branching, but in Zopeland it wasn't unusual, over long stretches, for someone to make a new branch every day. Ah, before I forget, single repository has worked very well for Zope (which includes top-level Zope2, Zope3, ZODB, ZConfig, zdaemon, ... projects): http://svn.zope.org/ Long URLs don't really get in the way in practice (rarely a need to type one after initial checkout; even svn switch is usually just a tail-end cmdline edit starting from a copy+paste of svn info output). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Thu, 2005-07-28 at 22:14, Tim Peters wrote: Ah, before I forget, single repository has worked very well for Zope (which includes top-level Zope2, Zope3, ZODB, ZConfig, zdaemon, ... projects): http://svn.zope.org/ Long URLs don't really get in the way in practice (rarely a need to type one after initial checkout; even svn switch is usually just a tail-end cmdline edit starting from a copy+paste of svn info output). It depends. In my use of svn, I do a lot of cross-branch merging and repo-side tagging. Those are done with urls and in those cases, long urls can suck. But we may not do a ton of that with the Python project, and besides it might not be important enough to split the directories. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
[Tim] Ah, before I forget, single repository has worked very well for Zope (which includes top-level Zope2, Zope3, ZODB, ZConfig, zdaemon, ... projects): http://svn.zope.org/ Long URLs don't really get in the way in practice (rarely a need to type one after initial checkout; even svn switch is usually just a tail-end cmdline edit starting from a copy+paste of svn info output). [Barry] It depends. In my use of svn, I do a lot of cross-branch merging and repo-side tagging. Yup, me too -- between the two of us, we don't have enough fingers to count how many trunks, branches, and tags of ZODB and Zope I have to fiddle with. Those are done with urls and in those cases, long urls can suck. They're all still copy, paste, tail-edit for me, and-- indeed! --having them all in the same repository is what makes just-the-tail editing possible. Merges I do from the cmdline, but repo-side tagging I do with the TortoiseSVN GUI, and the latter gives easy-to-copy/paste/edit URL fields. So switch to Windows for that part ;-) But we may not do a ton of that with the Python project, and besides it might not be important enough to split the directories. Ya, in Python we make a branch about once per release, + once per 5 years when Jeremy underestimates how long it will take to finish a crusade wink. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Thu, Jul 28, 2005, Barry Warsaw wrote: On Thu, 2005-07-28 at 22:14, Tim Peters wrote: Ah, before I forget, single repository has worked very well for Zope (which includes top-level Zope2, Zope3, ZODB, ZConfig, zdaemon, ... projects): http://svn.zope.org/ Long URLs don't really get in the way in practice (rarely a need to type one after initial checkout; even svn switch is usually just a tail-end cmdline edit starting from a copy+paste of svn info output). It depends. In my use of svn, I do a lot of cross-branch merging and repo-side tagging. Those are done with urls and in those cases, long urls can suck. But we may not do a ton of that with the Python project, and besides it might not be important enough to split the directories. Why can't you write a Python script to generate the URLs? 0.3 wink -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Migrating the Python CVS to Subversion
On Thu, 2005-07-28 at 22:59, Tim Peters wrote: Yup, me too -- between the two of us, we don't have enough fingers to count how many trunks, branches, and tags of ZODB and Zope I have to fiddle with. I have a small inkling of your pain. They're all still copy, paste, tail-edit for me, and-- indeed! --having them all in the same repository is what makes just-the-tail editing possible. Good point! Ya, in Python we make a branch about once per release, + once per 5 years when Jeremy underestimates how long it will take to finish a crusade wink. So are you saying that moving to svn will let us do more long lived branches? Yay! -B signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
