Re: [fossil-users] Outside contribution feature
On Thu, 05 Jun 2014 03:24:10 +0200, Warren Young war...@etr-usa.com wrote: As far as I can tell, Fossil offers two ways to allow outsiders to submit changes to an open source software project: 1. Give them Develop privs 2. Ask them to fossil diff and mail you a patch file In my open source project, I only give repo checkin privileges to contributors who have submitted a few quality patches first, at least. Most contributors submit only a patch or two and then go away, never to be heard from again, their itch scratched. The thing is, a unified diff cannot express all the information Fossil knows: file moves/renames/deletions, commit comments, the difference between a changeset and a monolithic patch, the point in the revision history the diff was made against... Git solves this problem with the pull request. Since it is one Git repository talking to another, no information is lost. I agree that Git is far more complex than necessary for most projects, but to the extent that Fossil tries to be Github-in-a-box, it would be nice to have something more analogous to a pull request than fossil diff. just my 2c (caveat: I've used (unwillingly but unavoidably) git/github in a single instance recently -- which overall confirmed all of my preconceptions about git and then some -- I _don't_ know git(hub) well): the pull request AFAIK is _not_ a git capability but resides in the layer put on top of it by the github people. essentially what github does is to provide facilities to manage communication between 3 repositories: the master repo of the project (commit access only for devs), the users's server-side clone (on the github site) and the local clone where the work is done. after initial setup there is sort of a circular traffic: pull recent changes (if any) from master to local repo, start editing and ci locally, push to user's server-side repo. that's it for the git capabilities. only at that point the pull-request is initiated on the github site and some developer can decide whether or not to pull the receptive change(sets) from the user's server-side repo. and -- as I learned on github -- it is expected to put all such potential contributions on separate branches which are usually closed afterwards. so if I am not missing anything the exact same thing could be done with any dvcs (and it is e.g. perfectly possible to use hg with the hggit extension locally which maps git branches to hg bookmarks and handles the communication with github just fine). one only needs the infrastructure to host all the server-side repos for all contributors. basically, the only thing that is missing (but not within fossil...) is a way to notify the devs (pull request) that something has been contributed. or what am I missing? so I don' think there is any need to modify fossil as it stands. what _is_ missing (presuming that there would be a real demand, which I doubt...) is the GitHub layer (web interface, scripting and what else managing the pull requests and communication between devs and contributors). another observation: the recent quite heavy traffic on the mailing list discussing at length (mis?)features of git that are allegedly missing in fossil is slightly over the top in my view (simply because git is there and could be used if someone so desires and it would not make sense to transform fossil to a git look-alike even if git were unquestionably good in every aspect which it is not). and I am fully on the side of drh in this discussion: fossil should keep clear of the rather obvious flaws (for most users and use cases) the git community sells as features (for all users and use cases) which indeed quickly lead to a situation where working with (or rather fighting against) the dvcs consumes more time than working at the project proper. too long. sorry... I think you could do it with a variant of the Fossil repo file format. It would be the same as the current format, with only two minor differences. First, it would be a subset of the contributor's repository, not a full clone, containing just the range of changes the contributor made. Second, it should have an attribute that tells where this repo fragment started from, relative to the central repo. Because it is still the Fossil DB format, it would record all the changes the outsider made, with no information loss. The workflow from the contributor's side would be: $ fossil clone http://blabla ; fossil open, etc ...hack, hack, hack... $ fossil submit -f contrib.fossil ...send contrib.fossil to the project's mailing list; OR: $ fossil submit ...local Fossil attempts to create a new ticket on central repo I am assuming that fossil submit wouldn't need to be told the starting point of the submission since the outsider's first ci will create a fork when autosync fails. True? If Fossil can't reliably know where to start, it could
Re: [fossil-users] Outside contribution feature
On 6/5/2014 01:51, j. van den hoff wrote: what _is_ missing is the GitHub layer I wish to do without that layer. I like Fossil the way it is: simple to use, yet powerful. I like being able to host my Fossil repo on my own server. Perhaps you're getting hung up on the fact that I described it as similar in spirit to a Git pull request. What it *really* is is a request for an uber-patch mechanism. Another way to think of it is an external form of fossil stash, with the ability to reintegrate the private fork with the main repo when the outside contributor is finished with their little side project. The only reason we need it is that we don't want to give out Develop privs to everyone who wants to contribute a patch or two, and the alternative (fossil diff) is lossy. fossil should keep clear of the rather obvious flaws the git community sells as features Agreed. That's why my proposal doesn't look anything like a Github pull request. It doesn't require that you Fork Me On FossilHub, it doesn't require a formal connection between two repositories, it doesn't require the contributor to make their repo public, etc. I don't see that I've added anything complex to the Fossil file format. It adds: 1. A starting point attribute: This Fossil repo's initial commit has hash b49b9b65cb3e3dd3d0970949b5f72e95e7478d84, so if you apply me to another repo that contains a version with that ID, it forks off from that version. This shouldn't be more complex than a new single-row table. 2. The ability to intelligently extract a subset of a repo, which I expect amounts to a CREATE DATABASE call to SQLite followed by a single clever SELECT statement to copy the changeset into the new DB. 3. The ability to apply that subset to another Fossil repo, using the starting point version ID as the patch base. I'm thinking of ATTACH DATABASE, followed by some data copying from one DB to another, followed by a merge attempt. 4. A few commands to manage all this: fossil submit, fossil apply, plus extensions to fossil commit and fossil revert to understand temporary forks. I realize implementing all this will take a fair bit of work. When I describe it as simple, I mean that I don't see that it changes a lot about how Fossil works internally. I also don't expect this to make the user's life any more complicated. In the end, it should be scarcely more difficult to use than fossil diff + patch, but will give a lot of new power. The temporary fork concept isn't really new. We already have forks, and we already have the stash. Once you commit a temporary fork, it will be an indelible part of the repo, just as if it had been committed and autosync'd. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Outside contribution feature
On 5 Jun 2014, at 02:24, Warren Young war...@etr-usa.com wrote: The thing is, a unified diff cannot express all the information Fossil knows: file moves/renames/deletions, commit comments, the difference between a changeset and a monolithic patch, the point in the revision history the diff was made against... I can see a use for this, but I think it can be achieved with relatively little development 1) Get the outsider to do an anonymous clone of the repository with the feature branch in it 2) Get them to email you the clone 3) Starting with the latest checkin in the feature branch (on the clone), use fossil info to find the parent 4) Add the latest hash to your list and Repeat step 3 until you have a commit hash that the primary repository has 5) Replay through the list of hashes you've built up using fossil commit to add them with the appropriate message. Steps 3 to 5 can be written fairly trivially I think, as a script. My initial worry would be how to deal with commit messages that reference tickets, apart from that I think it should be fine. Thanks, Kevin ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Outside contribution feature
On 6/5/2014 04:35, Kevin Martin wrote: 2) Get them to email you the clone My Fossil DB file is currently only 1.4M, but that's only because I didn't bother importing the previous 10 years of history into it. If I had chosen to do that, I expect the file would be bigger than the 10-20 MB message size limit of many email systems. Many mailing lists are even more restrictive about the size of attachments. Fossil itself currently costs 54 MB per clone. A large patch might be a few 10's of kB. 3 orders of magnitude difference per changeset. Steps 3 to 5 can be written fairly trivially I think, as a script. I gave a GUI method in my first message, but that only because most things in Fossil seem to have a GUI option. For myself, I only care about the command line workflow. The only GUI bit I care about is seeing the bulge in the timeline view. My initial worry would be how to deal with commit messages that reference tickets, apart from that I think it should be fine. Why would that be a problem? The outside contributor is starting from a clone of the main repo, so shouldn't they get a copy of the tickets and thus their IDs? If they mark a ticket closed in their copy, I don't see why that wouldn't replay into the main repo during fossil apply. It does complicate the idea of temporary branches, of course, since pruning the branch would have to undo the closing of the ticket. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Outside contribution feature
Suppose you had the ability to create a sub-repository - a kind of clone of a full fossil repo but that only contains a small subset of the check-ins (and/or wiki and tickets, etc.) A sub-repository would not even be self-complete: It would only contain artifacts for the file that changed in the check-ins that it contains and would have to refer back to the original repository for the unchanged files. With the ability to create a sub-repository, a non-credentialed contributor could clone the original repo, make his suggested changes in a branch, then create a sub-repository of just that branch. Since the sub-repository would be relatively small (hopefully!), it could be sent in via email. Or maybe even attached to a ticket! When the project leader receives the sub-repository, he does: fossil ui subrepo.fossil --master-repo original-project.fossil Fossil works on subrepo.fossil just like it would with a regular repository, except that if it is unable to find an artifact that it needs, it spills over to original-project.fossil. original-project.fossil is treated as read-only for that particular fossil instance, so no information from subrepo.fossil can leak into original-project.fossil. Perhaps you should persistently attach the subrepo to the original project like this: fossil set master-repository ~/myrepos/original-project.fossil -R subrepo.fossil Thereafter, subrepo.fossil works normally, except that it looks to original-project.fossil for any artifacts that it is missing. So you can then check-out the subproject in a separate directory: fossil open ~/tmp/subrepo.fossil Compile it. Diff it. Etc. And if you are satisfied that you want it in the core, you can incorporate the changes into the original project using something like this: fossil push file:/home/me/myrepos/original-project.fossil The key point of this design is that it requires very few modifications to the current Fossil implementation. All we have to do is add the master-repository property that identifies a read-only fossil file that is used to look up missing artifacts. Oh, and also add the command to create a subrepository. -- D. Richard Hipp d...@sqlite.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Outside contribution feature
On 6/5/2014 07:14, Richard Hipp wrote: Suppose you had the ability to create a sub-repository That was the idea, yes. A sub-repository would not even be self-complete: It would only contain artifacts for the file that changed in the check-ins that it contains I expected that limitation as well. The subrepo is an uber-patch. It only needs to make sense while grafted to the master repo. Since the sub-repository would be relatively small (hopefully!), it could be sent in via email. Or maybe even attached to a ticket! I proposed both ideas in my original post with the fossil submit syntax. You can give -f to create a file, but without arguments it would attempt to create a ticket on the master repo, since it knows where it was cloned from. The user would need to be logged into an account with ticket creation privileges, of course, but this is truly the Github-in-a-box experience, Fossil style. When the project leader receives the sub-repository, he does: fossil ui subrepo.fossil --master-repo original-project.fossil The link between the two repos is not a natural attribute of the UI. How about you push it a level lower, so that the same mechanism works for command line users as well: fossil open ../subrepo.fossil --master-repo ../master.fossil Then you can fossil ui if you want, or just start playing with the contributor's new view of the tree right there at the command line. Perhaps you should persistently attach the subrepo to the original project like this: fossil set master-repository ~/myrepos/original-project.fossil -R subrepo.fossil That looks like plumbing to me. fossil open --master-repo is the porcelain, and it can set this property implicitly. Compile it. Diff it. Etc. And if you are satisfied that you want it in the core, you can incorporate the changes into the original project using something like this: fossil push file:/home/me/myrepos/original-project.fossil original-project.fossil is already linked to subrepo.fossil in the scheme above. Could fossil push not simply use that as the default target? Perhaps there are advanced uses where you might want to push to a *third* repo, but I'm not looking for anything that complicated. If there is some conflict with the current behavior of fossil push without args, how about fossil push --master or fossil push --graft? The key point of this design is that it requires very few modifications to the current Fossil implementation. I'm all for that. All we have to do is add the master-repository property that identifies a read-only fossil file that is used to look up missing artifacts. Kind of like a union mount. Oh, and also add the command to create a subrepository. If you don't like fossil submit, how about fossil slice or fossil contribute? ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Outside contribution feature
Thus said Warren Young on Thu, 05 Jun 2014 04:06:09 -0600: I realize implementing all this will take a fair bit of work. When I describe it as simple, I mean that I don't see that it changes a lot about how Fossil works internally. I wonder if it could start out as an external tool which records just the ordered changes in a simple SQLite database that can be emailed to a developer with a simple script that will extract the content and use appropriate fossil commands to import with appropriate --date-override and --user-override options? Andy -- TAI64 timestamp: 400053908516 ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Outside contribution feature
Thus said Richard Hipp on Thu, 05 Jun 2014 09:14:17 -0400: Suppose you had the ability to create a sub-repository - a kind of clone of a full fossil repo but that only contains a small subset of the check-ins (and/or wiki and tickets, etc.) A sub-repository would not even be self-complete: It would only contain artifacts for the file that changed in the check-ins that it contains and would have to refer back to the original repository for the unchanged files. This really sounds like it can almost be achieved with the normal Fossil workflow with extremely minor changes to Fossil. Clone the project. Make changes. Clone the clone with a filter that filters out all artifacts but the ones desired. Email the clone and then the developer can do: fossil pull subrepo.fossil -R project.fossil Would this work? Seems almost too simple, but perhaps all that is missing is a way to clone a repository with just a subset of artifacts. Andy -- TAI64 timestamp: 400053908a5b ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Outside contribution feature
On 6/5/2014 09:18, Andy Bradford wrote: fossil pull subrepo.fossil -R project.fossil A contribution from an untrusted outsider needs to be checked carefully before it is committed to the master repo. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Outside contribution feature
Thus said Warren Young on Thu, 05 Jun 2014 09:25:39 -0600: A contribution from an untrusted outsider needs to be checked carefully before it is committed to the master repo. Certainly. fossil open subrepo.fossil # inspect fossil ui subrepo.fossil # inspect I'm not sure how many artifacts would actually be required to make this work, but it does seem possib le. Maybe it's too simplistic an approach and it would actually require attaching another DB as has been suggested. Andy -- TAI64 timestamp: 40005390ab0f ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Outside contribution feature
On Thu, Jun 5, 2014 at 1:37 PM, Andy Bradford amb-fos...@bradfords.org wrote: Thus said Warren Young on Thu, 05 Jun 2014 09:25:39 -0600: A contribution from an untrusted outsider needs to be checked carefully before it is committed to the master repo. Certainly. fossil open subrepo.fossil # inspect fossil ui subrepo.fossil # inspect I'm not sure how many artifacts would actually be required to make this work, but it does seem possib le. Maybe it's too simplistic an approach and it would actually require attaching another DB as has been suggested. If you included all the artifacts that you need, that would immediately make the subrepo larger than you might expect. Certainly the subrepo would be smaller if you only included artifacts that actually changed, and even then only included deltas and not the complete text. But maybe there should be an open to the fossil submit or fossil subrepo command (whatever it ends up being called) so that you can specify either a dependent or an independent subrepository. Dependent would be the default. But you could request an independent subrepo if you wanted to and space wasn't an issue. -- D. Richard Hipp d...@sqlite.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Outside contribution feature
On 6/5/2014 11:54, Richard Hipp wrote: But maybe there should be an open to the fossil submit or fossil subrepo command (whatever it ends up being called) so that you can specify either a dependent or an independent subrepository. Doesn't the outside developer already have a self-contained repo before calling fossil submit? It's the clone the outside developer is working from. If they want to send that copy off for review, fine, I guess. :) ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Outside contribution feature
Couldn’t this also be used to solve another problem, that of keeping separate repos for various projects that depend on (sometimes massive) common library files? Currently to deal with that I have to put all projects under the same repo, which is not ideal. The alternative, keeping multiple copies of shared code is also problematic. But this proposed change sounds like a possible solution for the above problem also, no? The main repo would hold the shared code, and each sub-repo the individual projects that depend on the shared code. Regards, From: Richard Hipp Sent: Thursday, June 05, 2014 4:14 PM To: Fossil SCM user's discussion Subject: Re: [fossil-users] Outside contribution feature Suppose you had the ability to create a sub-repository - a kind of clone of a full fossil repo but that only contains a small subset of the check-ins (and/or wiki and tickets, etc.) A sub-repository would not even be self-complete: It would only contain artifacts for the file that changed in the check-ins that it contains and would have to refer back to the original repository for the unchanged files. With the ability to create a sub-repository, a non-credentialed contributor could clone the original repo, make his suggested changes in a branch, then create a sub-repository of just that branch. Since the sub-repository would be relatively small (hopefully!), it could be sent in via email. Or maybe even attached to a ticket! When the project leader receives the sub-repository, he does: fossil ui subrepo.fossil --master-repo original-project.fossil Fossil works on subrepo.fossil just like it would with a regular repository, except that if it is unable to find an artifact that it needs, it spills over to original-project.fossil. original-project.fossil is treated as read-only for that particular fossil instance, so no information from subrepo.fossil can leak into original-project.fossil. Perhaps you should persistently attach the subrepo to the original project like this: fossil set master-repository ~/myrepos/original-project.fossil -R subrepo.fossil Thereafter, subrepo.fossil works normally, except that it looks to original-project.fossil for any artifacts that it is missing. So you can then check-out the subproject in a separate directory: fossil open ~/tmp/subrepo.fossil Compile it. Diff it. Etc. And if you are satisfied that you want it in the core, you can incorporate the changes into the original project using something like this: fossil push file:/home/me/myrepos/original-project.fossil The key point of this design is that it requires very few modifications to the current Fossil implementation. All we have to do is add the master-repository property that identifies a read-only fossil file that is used to look up missing artifacts. Oh, and also add the command to create a subrepository. -- D. Richard Hipp d...@sqlite.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Outside contribution feature
On Jun 5, 2014 10:55 AM, Richard Hipp d...@sqlite.org wrote: On Thu, Jun 5, 2014 at 1:37 PM, Andy Bradford amb-fos...@bradfords.org wrote: Thus said Warren Young on Thu, 05 Jun 2014 09:25:39 -0600: A contribution from an untrusted outsider needs to be checked carefully before it is committed to the master repo. Certainly. fossil open subrepo.fossil # inspect fossil ui subrepo.fossil # inspect I'm not sure how many artifacts would actually be required to make this work, but it does seem possib le. Maybe it's too simplistic an approach and it would actually require attaching another DB as has been suggested. If you included all the artifacts that you need, that would immediately make the subrepo larger than you might expect. Certainly the subrepo would be smaller if you only included artifacts that actually changed, and even then only included deltas and not the complete text. But maybe there should be an open to the fossil submit or fossil subrepo command (whatever it ends up being called) A fossil partial? so that you can specify either a dependent or an independent subrepository. Dependent would be the default. But you could request an independent subrepo if you wanted to and space wasn't an issue. -- D. Richard Hipp d...@sqlite.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Outside contribution feature
On Thu, Jun 5, 2014 at 6:38 PM, B Harder brad.har...@gmail.com wrote: On Jun 5, 2014 10:55 AM, Richard Hipp d...@sqlite.org wrote: If you included all the artifacts that you need, that would immediately make the subrepo larger than you might expect. Certainly the subrepo would be smaller if you only included artifacts that actually changed, and even then only included deltas and not the complete text. But maybe there should be an open to the fossil submit or fossil subrepo command (whatever it ends up being called) A fossil partial? Thinking about this, I am wondering what would happen if: 1. clone the main repo 2. Mark the root of the clone private, making the whole clone private 3. Start a branch for the local changes and mark that branch not private 4. Commit changes to the new branch 5. Clone the clone Would the second clone be just the new branch? ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
