Re: [HACKERS] Last gasp
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 If the feature set is desirable, though, I wonder if Postgres is big/high profile enough for them to figure out some sort of better arrangement. They *love* it when big open-source projects use GitHub as their public repo - they'll email and blog announcements about it - and if there's interest I'd be happy to open a conversation with them. No need to wonder, we've been in contact with them before and they are very pro Postgres. I've looked at it in conjunction with Jenkins CI; it looked nice but was way too heavy-weight for a four-person startup (what's code review?). It's probably much more suitable for this sized project. Gerrit's a full-featured code review app with a tolerable UI; MediaWiki is just starting to jump into git/Gerrit and there are definitely a lot of rough edges in that workflow still. ... Someone mentioned upthread that github spam was a problem. I'm not sure I see the issue here - wouldn't mail from them still go through our lists and out current anti-spam measures anyway? - -- Greg Sabino Mullane g...@turnstep.com End Point Corporation http://www.endpoint.com/ PGP Key: 0x14964AC8 201204170623 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAk+NRO4ACgkQvJuQZxSWSshPpACg9+ZB6NzCsvnkQwuoD/BzIHgL yMkAn3zwksbKxaSDt3k/YzKY7UVLmUZb =igZu -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Jay Levitt jay.lev...@gmail.com writes: No meaningful search, eh? Works for me. Redmine searches return partial-word matches, and there's no way to disable that. Searching for test finds latest. To me, that's broken. Well, I believe one can plug in a different search engine, like lucene or xapian. However it doesn't look like some one already did (for ticket/wiki history, but there's xapian search plugin[1] to index attachments.) Also, the UI is very 5 years ago; e.g., compare revisions uses the same columns-of-radio-buttons approach as MediaWiki. If the goal is a tool to reduce friction and increase involvement, you want a smoother UX. Nothing that could not be tweaked with a plugin or core code modification here either. Not sure about the magnitude of the effort required, though. -- Alex [1] https://github.com/xelkano/redmine_xapian -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Sun, Apr 15, 2012 at 8:23 AM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, Apr 7, 2012 at 9:51 PM, Robert Haas robertmh...@gmail.com wrote: I think this basically just boils down to too many patches and not enough people. I was interested in Command Triggers from the beginning of this CommitFest, and I would have liked to pick it up sooner, but there were a LOT of patches to work on for this CommitFest. The first three CommitFests of this cycle each had between 52 and 60 patches, while this one had 106 which included several very complex and invasive patches, command triggers among them. So there was just a lot more to do, and a number of the people who submitted all of those patches didn't do a whole lot to help review them, sometimes because they were still furiously rewriting their submissions. It's not surprising that more patches + fewer reviewers = each patch getting less attention, or getting it later. This is a good point. The current process lacks inherent scalability. I would really like us to enforce a policy of 1 patch = 1 review. That way we automatically have enough review time, no matter how many patches we get. If we don't enforce that, then patch sponsors are more likely to take the attitude that review isn't something they need to pay for, just the dev work. I would be generally in favor of that policy, but I would relax it for people who have only ever submitted a handful of patches, so as to continue encouraging them to become involved in the community. It's also worth noting that not all reviews are created equal. It takes a lot more time to review command triggers than it does to review pg_archivecleanup extension-skipping. It's not important that the review effort is *exactly* proportional to the size of what has been submitted, but it *is* important that when we're having a CommitFest, people are spending the majority of their time on reviewing, rather than continuing to spend it developing patches that aren't done yet. It's perfectly right to say that people should be given a chance to finish up patches that are almost there, but our definition of almost there has a tendency to expand to the point where it's laughable when the last CommitFest of the cycle rolls along. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Sun, Apr 15, 2012 at 11:31 PM, Greg Smith g...@2ndquadrant.com wrote: On 04/15/2012 05:46 AM, Simon Riggs wrote: Our problem is not lack of resource, it is ineffective delegation. As Hannu points out, he didn't know the patch would be rejected, so he didn't know help was needed to save something useful. I considered that the job of the CF manager, but perhaps it is was not. Note that one of the influences on the death march here was lack of a fully devoted CF manager for the full duration. I did some rabble rousing to get things started in the usual way, but my time for this only budgeted for six weeks in that role. And that was quite optimistic for this one. Trying to quantify how much time investment the CF manager role really involves is one of my important projects to chew on. Whoever ends up doing that should at least have an idea what scale of problem they're getting into. Again, if the CF manager role is too big, then we delegate. No reason why we can't have a CF team, with different people responsible for different areas. The key thing is working out how to mobilise the resources we have to best effect. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Alvaro Herrera alvhe...@commandprompt.com writes: I've used Redmine a lot, as you know, and I only keep using it because it's a requirement at work. It is certainly not close to usable for general pgsql stuff. (Trac, which we used to use prior to Redmine, was certainly much worse, though). Same story here, still using redmine a lot, all with custom reports etc. I can't say that it's all that slow, or that there's a problem with the code, or that the search doesn't work right (and I've never had a wiki edit disappear, either, and I've used that a lot). It's just the wrong tool altogether. It's indeed slow here, and I agree that's not the problem. Not the tool we need, +1. Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 04/15/2012 12:01 PM, Tom Lane wrote: Where I think we have been fooling ourselves is in failing to tell the difference between a patch that is committable in the current fest, versus one that is still WIP and is going to need more development time. I wonder if this bit of state might be worth extending the UI to include. Just a little toggle box with the options WIP and Commit Submission there. [I am unattached to those particular terms] I think everyone is clear that Command Triggers is an example that reflects a more general problem seen many times before; I'll continue using it as a fresh example here without meaning to pick on Dimitri in particular. If Dimitri had submitted that in January while ticking Commit Submission, it might have sparked a talk about the difference in expectations earlier. If we use Robert as the bad guy watching the CF progress (again, just as an example, not trying to paint him with that title), I think it would have been easier for him to write an e-mail like this: It's March now, and this feature has been under heavy review and development for 6 weeks. This looks more like a WIP feature to me, not one that arrived as a Commit Submission. Is there any useful subset to consider instead? Compared to the current way such things happen, that's a more factual style of message without as much emotion or judging, and one that can be raised much earlier in the CF cycle. I think it will be easier for people to write those, compared with having to be the person saying this isn't ready to commit only at the end. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Alex wrote: Jay Levittjay.lev...@gmail.com writes: Alex wrote: I didn't follow this whole thread, but have we considered Redmine[1]? As the resident Ruby is shiny, let's do everything in Rails on my MacBook guy, I'd like to make a statement against interest: I've tried Redmine a few times and it's been painful. Much of the codebase is deprecated, it's slow, it has no meaningful search (in 2012?!), I've seen wiki edits disappear, and at the moment pulling up its own FAQ page at redmine.org times out. Yay, that's totally FUD to me. You're right, it was. My bad. Someday I will find the balance between precision and concision. Could you please elaborate a bit on your points? Deprecated codebase? Let me guess... It runs on an outdated version of Rails (2.3) but only because Rails is changing so rapidly, I believe. There is work in progress[1] to move to the supported branch Rails-3.x. I wasn't even thinking of that; I know many production systems still run on Rails 2.3, and in fact it probably even performs better for some workloads. 3.x is a mixed bag. I don't hold that against Redmine. But it's still FUD, because I can't remember where I saw this information. So: withdrawn. Slow? Do you have any data to back this point up? No measurable data; just a sigh of relief when switching from Redmine to Github - and GitHub ain't a speed demon. In general, I've seen multi-second page load times on crazy-simple things like wiki edits; this was on a hosted provider (sourcerepo.com), but they also hosted our git repo and we had no speed problems there. No meaningful search, eh? Works for me. Redmine searches return partial-word matches, and there's no way to disable that. Searching for test finds latest. To me, that's broken. Also, the UI is very 5 years ago; e.g., compare revisions uses the same columns-of-radio-buttons approach as MediaWiki. If the goal is a tool to reduce friction and increase involvement, you want a smoother UX. Jay -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Sat, Apr 14, 2012 at 2:28 PM, Jay Levitt jay.lev...@gmail.com wrote: Christopher Browne wrote: On Thu, Apr 12, 2012 at 6:11 PM, Jay Levittjay.lev...@gmail.com wrote: Rather than extend the CF app into a trivial-patch workflow app, it might be worth looking at integrating it with github. There's a reluctance to require a proprietary component that could disappear on us without notice. Excellent point. I was thinking that GitHub's API would allow archival exporting to counter that, along the lines of let's take advantage of it for the next five years until it goes south, and THEN we could write our own. But I can see how that might not be the best choice for a project that expects to preserve history for a few decades. GitHub does offer an enterprise version that you can self-host, but it seems to be priced per-user and intended for solely intranet use. If the feature set is desirable, though, I wonder if Postgres is big/high profile enough for them to figure out some sort of better arrangement. They *love* it when big open-source projects use GitHub as their public repo - they'll email and blog announcements about it - and if there's interest I'd be happy to open a conversation with them. The existence of git itself is a result of *exactly* that circumstance, as Linux kernel developers had gotten dependent on BitKeeper, whereupon the owner decided to take his toys home, at which point they were left bereft of their SCM tool. http://kerneltrap.org/node/4966 Good history lesson there, with a great outcome. I expect that it would be more worthwhile to look into enhancements to git workflow such ashttp://code.google.com/p/gerrit/ Gerrit. I don't know that Gerrit is THE answer, but there are certainly projects that have found it of value, and it doesn't have the oops, it's proprietary problem. I've looked at it in conjunction with Jenkins CI; it looked nice but was way too heavy-weight for a four-person startup (what's code review?). It's probably much more suitable for this sized project. Gerrit's a full-featured code review app with a tolerable UI; I was thinking of GitHub more as a great lightweight UI for doc patches and other trivial patches where you might have lots of casual review and comments but no need for, say, recording regression tests against each patch version. e.g.: https://github.com/rails/rails/pull/5730 Also, for doc patches, GitHub has the great advantage of in-place editing right from the web UI. I don't know if GitHub's pull request or Gerrit is a good tool (I doubt, actually), but I've been thinking how we could improve our review process in terms of both of human process perspective and tool process. As we have our simple CF app (while there are a bunch of tools like JIRA or something), I'd think we could have our own review UI connected to the rest of our toolset including CF app. I know we want the mail archive history of the whole discussion, but still giving feedback to the submitter via email is hard-work and the successors cannot read it entirely. From a human- rather than technology-oriented perspective: I was shocked to find that you folks *WANT* reviews from non-contributors. It was my assumption as a newcomer that if I don't feel well-versed enough to submit patches yet, the last thing you'd want me to do was to look over someone else's patch and say Yeah, that looks good, any more than I care if my mom thinks my latest web app is very nice. I see now that the Reviewing a Patch wiki page explains this, but maybe this info should be pushed higher into the docs and web site; a How can I contribute page, open calls for reviewers on the non-hackers mailing lists, things like that. Or maybe just make the wiki page bright red and blink a lot. I found myself enjoying reviewing other patches where I don't have strong knowledge. I strongly believe we should encourage more and more people who haven't worked particular patches in that area, to review patches. The more eyeballs there are, the more quality we get. Thanks, -- Hitoshi Harada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Fri, Apr 6, 2012 at 8:19 AM, Tom Lane t...@sss.pgh.pa.us wrote: Greg Smith g...@2ndquadrant.com writes: On 04/05/2012 04:27 PM, Simon Riggs wrote: It's shocking since after months of work and an especially extended edition CF, we expect people to deliver something, not just shunt the whole thing off as rejected with 1 days's notice to alter that outcome. I don't think this is being fair to Robert. If we're going to ship a release at all, somebody's got to be willing to say no. Personally, been there, done that, got the t-shirt [1]. Robert's just pointing out what has to be pointed out. Just returned from a week away, so I'm chipping in on key points only. I accept the command trigger patch is gone now and I would add comments only about our processes. The problem remains that we have wasted many months of development and slipped a release on what appears to be an important, universally popular feature that had bucket loads of early planning. We shouldn't hide from recognising that as an issue. I completely agree that somebody has to be willing to say No, since we all agree that the default for any patch is non-acceptance. My first observation is that if No is received early enough for something to be done, then the outcome could be different. It was not clear that this important patch was going to be totally refused and many people have expressed their surprise about that. Noah signalled to everybody that the FK locks patch was likely to be rejected and a number of us have tried hard to save that, unluckily as it turns out. So an early No helped people allocate their time on what they considered to be important. In contrast the Command Triggers and FKs for arrays patches received a No so late that nothing could be done. So I fully agree that people should say No, but the key point is *when* they say it. For the future, I think we should have a triage week as the first week in each CF; lets shake out the No comments early - in some cases nothing can be done and we can refocus attention onto important topics. Second, my point was that No should not be applied in black/white form if at all possible. If some aspect of a patch is unworkable, it may be possible to provide some of the functionality, rather than simply nothing at all. That wasn't possible with FK locks, and maybe it was possible with command triggers. I do consider it the responsibility of a reviewer to salvage as much as is easily and reasonably possible from each contribution. In most cases that requires a few words from the reviewer, since many patch contributors listen carefully to what is said and try hard to make changes that work. Frequently I see reviewers simply making authors dance around with various additions and tweaks; all very well if the patch is acceptable, but its just a waste of time if there is no route to acceptance laid out clearly by the reviewer. That's something I've mentioned before: the reviewer *must* supply a list of things which, if solved, would allow the patch to be accepted. Clearly there will always be last minute issues that cause rejection. If we can get our information transfer a little slicker we'd be able to find more review time. Instead of spending months on their own dead patches, people would be available to assist on those with a chance to be saved. Our problem is not lack of resource, it is ineffective delegation. As Hannu points out, he didn't know the patch would be rejected, so he didn't know help was needed to save something useful. I considered that the job of the CF manager, but perhaps it is was not. In any case, it would seem best for the future if the CF manager was not also a committer, since those people are clearly too busy to do both roles as well as the project needs them to be. Just as our processes evolved into the creation of CFs and a CF manager, we must evolve again towards someone/team whose main task is ensuring that the delegation problem is solved. That won't work by bossing people around, it has to work by informing and encouraging people to contribute in the ways that they wish. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 6:59 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Apr 11, 2012 at 1:39 PM, Joshua Berkus j...@agliodbs.com wrote: Ultimately, we're herding cats here. I don't think you're going to get the community to suddenly be willing to march in lockstep instead. If you, Peter, Simon, Robert, Heikki, Magnus, Peter G., Greg, Bruce and Andrew agreed on a calendar-driven, mostly unambiguous process and adhered to that process, then the one or two people who didn't follow along wouldn't matter. Everyone else would follow you. The reason things are chaotic now is that our lead committers do not have consensus and are even inconsistent from CF to CF individually. In other words: the problem is only unsolvable because *you* think it's unsolvable. If you decide the problem is solvable, you already have the means to solve it. That's a somewhat bizarre list of people. It both includes people who haven't expressed many concerns about our process one way or the other and excludes some who have. At any rate, clearly the problem is exactly that there isn't consensus on this. I would generally say that Tom, Greg Smith, and I are pretty close together on this issue, and Peter G., Simon, and Dimitri are pretty close together on this issue, but with a big gap in between those two groups. I am less clear on how everyone else feels, but I think that saying that all we need is to get consensus among those people is to define the problem, not the solution. Having just come back from a week away, I'm not really clear what you mean by this issue. It worries me greatly to see my name in a divisive list like that. There are no teams here; each issue needs to be judged on its own, independently of each other and without regard to the person speaking. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Sat, Apr 7, 2012 at 10:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: [ among other good points ] ... On a related note, letting CommitFests go on for three months because there's insufficient reviewer activity to get them done in one or two is, in my opinion, not much of a solution. If there's even less reviewer activity next time, are we going to let it go on for four months? Six months? Twelve months? At some point, it boils down to we're just going to stop accepting patches for an indefinite period of time. Yuck. Yeah, this is something I was thinking about yesterday. In the first couple of release cycles with the CommitFest process, we were willing to let the last fest of a release cycle go on for as long as it takes, or at least that was what I felt the policy to be. This time we eventually gave up and declared closure, but in hindsight we should likely have done that a month earlier. The fact of the matter is that quite a few of the patches we were dealing with were *not* ready to commit, or even close to that, at the start of the fest. If it weren't the last fest they would have gotten marked Returned With Feedback a lot sooner. I wonder whether we ought to have a preset schedule for last fests just like the others. I'd be willing to let them run, say, 2 months instead of 1, but no deadline at all risks turning the whole affair into a death march, which is no fun for anybody and threatens the quality of the end result too. There's too much temptation to commit patches that are not really ready, just to get them out of the way. In short, the idea of strongly calendar-driven releases looks more and more attractive to me the more times we go through this process. If your patch isn't ready on date X, then it's not getting into this release; but there'll be another bus coming along before long. Stretching out release cycles to get in those last few neat features just increases the pressure for more of the same, because people don't know how long it will be to the next release. A 2 month hard deadline seems enough for me. I spoke in favour of reasonableness previously, but that didn't mean no dates at all. If we can do Triage Week at the beginning, that will keep out the ones that aren't ready and allow us to focus our attention on the ones we really care about. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Sat, Apr 7, 2012 at 9:51 PM, Robert Haas robertmh...@gmail.com wrote: I think this basically just boils down to too many patches and not enough people. I was interested in Command Triggers from the beginning of this CommitFest, and I would have liked to pick it up sooner, but there were a LOT of patches to work on for this CommitFest. The first three CommitFests of this cycle each had between 52 and 60 patches, while this one had 106 which included several very complex and invasive patches, command triggers among them. So there was just a lot more to do, and a number of the people who submitted all of those patches didn't do a whole lot to help review them, sometimes because they were still furiously rewriting their submissions. It's not surprising that more patches + fewer reviewers = each patch getting less attention, or getting it later. This is a good point. The current process lacks inherent scalability. I would really like us to enforce a policy of 1 patch = 1 review. That way we automatically have enough review time, no matter how many patches we get. If we don't enforce that, then patch sponsors are more likely to take the attitude that review isn't something they need to pay for, just the dev work. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Tue, Apr 10, 2012 at 3:26 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Christopher Browne cbbro...@gmail.com wrote: Robert Haas robertmh...@gmail.com wrote: CommitFests are a time for patches that are done or very nearly done to get committed, and a time for other patches to get reviewed if they haven't been already. If we make it clear that the purpose of the CommitFest is to assess whether the patch is committable, rather than to provide an open-ended window for it to become committable, we might do better. Yeah, I think there's pretty good room for a +1 on that. Yeah, +1 for sure. The top comment that has been +1'd presents just 2 states; 1 is deliberately phrased to be ridiculous, so of course everybody will vote for the other one. What is missing there is all of the other possible states between those two extremes. Let me re-phrase that: I think open ended CFs aren't much use. Hard edges are needed. But having said that, I can't think of a major feature that didn't have some tweaking after commit, and after end of CF. One other sort of mechanical test which I think can and should be applied to patches submitted to the last CF is that if *at the start of the CF* the patch doesn't apply, compile, pass regression tests, and demonstrably provide the functionality claimed for the patch, it should not be a candidate for inclusion in the release. A patch on which the author is continuing to work even in the absence of review should be considered a WIP want feedback submission; it should not be allowed to constitute a placeholder for inclusion in the release. It's one thing if review turns up corner case bugs missed by the author; it's quite another if there is a month or two of solid development left to be done. The CF period is not the time for now I'll get serious about wrapping this up. Agreed. But again, mistakes do happen, so reasonableness is required. CommitFests should be a finalisation period where submissions get tweaked to fix problems/bugs and allow them to be committed by the end of the CF. Again, in some cases that might be on the last day of the CF (else its not the last day...). In the past, patches could starve on the queue for very long periods, sometimes years. Having a too-harsh process makes it then easy to go back to the old way of quickly bouncing things that lack popularity from committers. Anyway, this discussion is just the annual make things better discussion. Our process was good to start with and has get better each release for years and years now, so objectively we are doing quite well. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Simon Riggs si...@2ndquadrant.com writes: I completely agree that somebody has to be willing to say No, since we all agree that the default for any patch is non-acceptance. My first observation is that if No is received early enough for something to be done, then the outcome could be different. It was not clear that this important patch was going to be totally refused and many people have expressed their surprise about that. Noah signalled to everybody that the FK locks patch was likely to be rejected and a number of us have tried hard to save that, unluckily as it turns out. So an early No helped people allocate their time on what they considered to be important. In contrast the Command Triggers and FKs for arrays patches received a No so late that nothing could be done. I think this is a rather unfair summary of the history. It was clear very early in the CF that people thought Command Triggers had major design problems, and Dimitri was doing significant rewrites to try to fix that. Anyone who did not think that patch was at serious risk of not being committed simply wasn't paying attention. Given the range of different design options that were considered, I think it's just as well that the patch has been put off till 9.3: we will probably get a better feature than if it had made it this time, and we will certainly be taking less schedule risk. I will agree that the array-FKs patch got the short end of the stick: had we been willing to let the CF go on for another month, it would have gotten looked at more carefully, and quite possibly committed. But once we made the decision to cut off the CF, there was not time to look at it closely enough. Again, to my mind this was mostly a risk minimization decision: when you make up a fundamental feature out of whole cloth, there's substantial risk that you didn't get the design right. At this point we don't have enough time for the feature to settle in and get used before 9.2 will ship and it'll be too late to correct any design problems. The more general point here is that the last fest of a release cycle is the worst possible time to be landing big, destabilizing patches. I think we ought to be conservative at this stage of the cycle, in hopes of keeping beta phase short and predictable. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Simon Riggs si...@2ndquadrant.com writes: If we can do Triage Week at the beginning, that will keep out the ones that aren't ready and allow us to focus our attention on the ones we really care about. I think there's some merit in this idea, but there needs to be time allocated to examine all the large patches before we make any hard go/no-go decisions. Maybe we could make such choices about two weeks in, rather than at the very start? Another thought is that triage is probably not the right image to have here. Patches that are obviously going to be rejected altogether are not that common, and they don't take up much time when they do show up. Where I think we have been fooling ourselves is in failing to tell the difference between a patch that is committable in the current fest, versus one that is still WIP and is going to need more development time. Now the latter category *is still deserving of review*, just as much as the former. So even if we can correctly determine early on which patches are WIP, it doesn't mean we should bounce them out of the fest. But it would mean they get approached differently by the reviewers. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Sun, Apr 15, 2012 at 4:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: I think this is a rather unfair summary of the history. It was clear very early in the CF that people thought Command Triggers had major design problems, and Dimitri was doing significant rewrites to try to fix that. Anyone who did not think that patch was at serious risk of not being committed simply wasn't paying attention. Fair comment, since I was definitely not paying attention. My I-Want-a-Pony idea is some kind of rating system that allows us all to judge patches in terms of importance/popularity, complexity and maturity. I guess a Balanced Scorecard for the development process. So we can all see whats going on. We already do this when we speak to each other in hushed tones that so-and-so a patch looks unlikely etc.. If we could do that more openly it would help. The more general point here is that the last fest of a release cycle is the worst possible time to be landing big, destabilizing patches. I think we ought to be conservative at this stage of the cycle, in hopes of keeping beta phase short and predictable. There is a definite selection effect that means the bigger the patch the more likely it is to land later in the release cycle, regrettably. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 04/14/2012 06:03 PM, Robert Haas wrote: If someone's work is going to require substantial revision, it is much better and much less work to do that revision before the code goes into our repository (and particularly, before it gets released) rather than after. I would think one of the major factors in deciding who should be able to commit code is whether they'll likely to commit substandard material. Someone who reviews and is seen to mark patches ready for commit, and they're not, should surely not be committing things either. The review process we have now does a pretty good job of identifying which submissions are baked and which aren't. I'd never argue that there should be more people to commit so they can slip in half baked material. Someone doesn't need to know how to bake everything to be useful as a committer though; they just need to know what they can and can't handle. And, on a related note, I am having a hard time imagining that it's a good idea to give very many people commit bits primarily so that they can commit their own work. If someone has committed their own work after that submission went through the full CF and review process, I don't see a lot of harm in them committing the result. I'd certainly never suggest that the reason to have more committers is so that the CF workflow was easier to subvert. Yes, there are problems with having enough reviewers and ushering large patches through the CF process. But it seems to me there are a fair number of submission that start solid, turn excellent through thorough review, and once they do hit ready for committer they could be picked up for commit by more people than the existing very small pool (committers who process other people's submissions regularly). Also, and I'm aware this is a more controversial point, I believe there are some people who would do more review if they could just move toward committing the stuff that looks good without going through quite as much process. At some times, if you realize something is close and just needs a bit more work, the easy path is to just do it yourself and be done. Non committing reviewers can't get that efficiency boost in the cases it's appropriate. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 04/14/2012 05:28 PM, Jay Levitt wrote: I see now that the Reviewing a Patch wiki page explains this, but maybe this info should be pushed higher into the docs and web site; a How can I contribute page, open calls for reviewers on the non-hackers mailing lists, things like that. Or maybe just make the wiki page bright red and blink a lot. It is too bad the page we used to have like this on MySpace isn't around anymore. Right now there's some rumbling at multiple companies around things like how to sponsor new features for PostgreSQL. Tying that into a larger page covering the various ways people can contribute is an interesting idea. I think we've only recently crossed the point where there's enough details on pages like Reviewing a Patch that it's clear to people what they should do on the technical side. That pretty well completely work could use something that's more like marketing now though. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 04/15/2012 05:46 AM, Simon Riggs wrote: Our problem is not lack of resource, it is ineffective delegation. As Hannu points out, he didn't know the patch would be rejected, so he didn't know help was needed to save something useful. I considered that the job of the CF manager, but perhaps it is was not. Note that one of the influences on the death march here was lack of a fully devoted CF manager for the full duration. I did some rabble rousing to get things started in the usual way, but my time for this only budgeted for six weeks in that role. And that was quite optimistic for this one. Trying to quantify how much time investment the CF manager role really involves is one of my important projects to chew on. Whoever ends up doing that should at least have an idea what scale of problem they're getting into. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Excerpts from Alex's message of dom abr 15 01:52:16 -0300 2012: Jay Levitt jay.lev...@gmail.com writes: Alex wrote: I didn't follow this whole thread, but have we considered Redmine[1]? As the resident Ruby is shiny, let's do everything in Rails on my MacBook guy, I'd like to make a statement against interest: I've tried Redmine a few times and it's been painful. Much of the codebase is deprecated, it's slow, it has no meaningful search (in 2012?!), I've seen wiki edits disappear, and at the moment pulling up its own FAQ page at redmine.org times out. Yay, that's totally FUD to me. I've used Redmine a lot, as you know, and I only keep using it because it's a requirement at work. It is certainly not close to usable for general pgsql stuff. (Trac, which we used to use prior to Redmine, was certainly much worse, though). I can't say that it's all that slow, or that there's a problem with the code, or that the search doesn't work right (and I've never had a wiki edit disappear, either, and I've used that a lot). It's just the wrong tool altogether. -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Peter Eisentraut pete...@gmx.net writes: On ons, 2012-04-11 at 23:30 -0400, Robert Haas wrote: Now what would be sort of neat is if we had a way to keep all the versions of patch X plus author and reviewer information, links to reviews and discussion, etc. in some sort of centralized place. Well, a properly linked email thread contains all this. I have seen a couple of anti-patterns evolving, though, including patch authors starting a new email thread for each patch version, and reviewers starting a new email thread for each review. When that happens, the existence of the commitfest app makes things worse, in a way. (Of course, any discussion here about bug trackers emphasizes the need for email to be the primary communications method for this very reason.) I didn't follow this whole thread, but have we considered Redmine[1]? We at CMD rely on it as the primary means of customer communication, via email. New email to a mailing list backed by Redmine triggers new ticket creation and any followups result into an update to the ticket. The ticket history is searchable, browsable (and updateable) on the web. It's GPL-ed, uses Ruby on Rails, and comes with a broad range of plugins[2]. Also it is quite hackable (for a RoR guy, of course.) We could integrate Redmine with the CommitFest or even replace it completely with a custom-written plugin. A command line interface[3] exists, so 'cf close ###' suggested further in this thread should be also possible. -- Regards, Alex [1] http://www.redmine.org/ [2] http://www.redmine.org/plugins [3] https://github.com/diasjorge/redmine-cli -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 04/14/2012 03:02 AM, Alex wrote: I didn't follow this whole thread, but have we considered Redmine[1]? It comes up every couple of years in contexts near this one, such as http://wiki.postgresql.org/wiki/TrackerDiscussion -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Greg Smith g...@2ndquadrant.com writes: On 04/14/2012 03:02 AM, Alex wrote: I didn't follow this whole thread, but have we considered Redmine[1]? It comes up every couple of years in contexts near this one, such as http://wiki.postgresql.org/wiki/TrackerDiscussion Oh, I see. I wonder maybe it is time to actually take some action? Given that the list of disadvantages on that wiki page looks really bad (to me.) -- Alex -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Alex a...@commandprompt.com writes: Greg Smith g...@2ndquadrant.com writes: On 04/14/2012 03:02 AM, Alex wrote: I didn't follow this whole thread, but have we considered Redmine[1]? It comes up every couple of years in contexts near this one, such as http://wiki.postgresql.org/wiki/TrackerDiscussion Oh, I see. I wonder maybe it is time to actually take some action? Given that the list of disadvantages on that wiki page looks really bad (to me.) Err, the list of Disadvantages of Current Situation I mean. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Christopher Browne wrote: On Thu, Apr 12, 2012 at 6:11 PM, Jay Levittjay.lev...@gmail.com wrote: Rather than extend the CF app into a trivial-patch workflow app, it might be worth looking at integrating it with github. There's a reluctance to require a proprietary component that could disappear on us without notice. Excellent point. I was thinking that GitHub's API would allow archival exporting to counter that, along the lines of let's take advantage of it for the next five years until it goes south, and THEN we could write our own. But I can see how that might not be the best choice for a project that expects to preserve history for a few decades. GitHub does offer an enterprise version that you can self-host, but it seems to be priced per-user and intended for solely intranet use. If the feature set is desirable, though, I wonder if Postgres is big/high profile enough for them to figure out some sort of better arrangement. They *love* it when big open-source projects use GitHub as their public repo - they'll email and blog announcements about it - and if there's interest I'd be happy to open a conversation with them. The existence of git itself is a result of *exactly* that circumstance, as Linux kernel developers had gotten dependent on BitKeeper, whereupon the owner decided to take his toys home, at which point they were left bereft of their SCM tool. http://kerneltrap.org/node/4966 Good history lesson there, with a great outcome. I expect that it would be more worthwhile to look into enhancements to git workflow such ashttp://code.google.com/p/gerrit/ Gerrit. I don't know that Gerrit is THE answer, but there are certainly projects that have found it of value, and it doesn't have the oops, it's proprietary problem. I've looked at it in conjunction with Jenkins CI; it looked nice but was way too heavy-weight for a four-person startup (what's code review?). It's probably much more suitable for this sized project. Gerrit's a full-featured code review app with a tolerable UI; I was thinking of GitHub more as a great lightweight UI for doc patches and other trivial patches where you might have lots of casual review and comments but no need for, say, recording regression tests against each patch version. e.g.: https://github.com/rails/rails/pull/5730 Also, for doc patches, GitHub has the great advantage of in-place editing right from the web UI. Peter mentioned the desire to bring more eyes and hands onto these type of patches - I think the phrase was enthusiast power users who wouldn't really consider themselves hackers. The advantage of GitHub here would be its (current) widespread adoption; the perceived barrier to entry is far lower and the workflow is far more obvious than a mailing list, formatting patches by email, not quite knowing what the process is. I mention all this not to try to push GitHub specifically, but to say these are the types of features that modern, open-source collaborative systems offer, and could improve community involvement. From a human- rather than technology-oriented perspective: I was shocked to find that you folks *WANT* reviews from non-contributors. It was my assumption as a newcomer that if I don't feel well-versed enough to submit patches yet, the last thing you'd want me to do was to look over someone else's patch and say Yeah, that looks good, any more than I care if my mom thinks my latest web app is very nice. I see now that the Reviewing a Patch wiki page explains this, but maybe this info should be pushed higher into the docs and web site; a How can I contribute page, open calls for reviewers on the non-hackers mailing lists, things like that. Or maybe just make the wiki page bright red and blink a lot. Jay -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Alex wrote: I didn't follow this whole thread, but have we considered Redmine[1]? As the resident Ruby is shiny, let's do everything in Rails on my MacBook guy, I'd like to make a statement against interest: I've tried Redmine a few times and it's been painful. Much of the codebase is deprecated, it's slow, it has no meaningful search (in 2012?!), I've seen wiki edits disappear, and at the moment pulling up its own FAQ page at redmine.org times out. Maybe you've had better luck with it, but whenever I've Googled for Redmine questions, the collective Internet has sighed and said Yeah, it was a really good idea, though. Jay Levitt -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Tue, Apr 10, 2012 at 8:43 PM, Greg Smith g...@2ndquadrant.com wrote: The main reason I worry about this is because of a very real chicken/egg problem here that I keep banging into. Since the commit standards for so many other open-source projects are low, there are a non trivial number of business people who assume !committer == ![trusted|competent]. That makes having such a limited number of people who can commit both a PR issue (this project must not be very important if there are only 19 committers) and one limiting sponsorship (I'm not going to pay someone to work on this feature who's been working on it for years but isn't even a committer). I'm not sure what the best way to address this problem is, but I don't think it's to make more people committers if they aren't truly qualified. If someone's work is going to require substantial revision, it is much better and much less work to do that revision before the code goes into our repository (and particularly, before it gets released) rather than after. And, on a related note, I am having a hard time imagining that it's a good idea to give very many people commit bits primarily so that they can commit their own work. We could create quite a few such committers and not really solve whatever bottleneck exists here. What we really need are committers who are willing and able to spend the time to do high-quality reviews of other people's work. I find this whole line of thinking particularly troubling in view of Peter's comments that contributors face a prisoner's dilemma situation and will inevitably push their own patches forward at the expense of other people's patches. That behavior is only a mild nuisance when people without a commit bit do it, but it's potentially a lot more disruptive if someone who has a commit bit does it, because the default for whether the patch gets committed flips from no to yes. I think our committers need to be people we can trust enough NOT to engage in such behavior. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Jay Levitt jay.lev...@gmail.com writes: Alex wrote: I didn't follow this whole thread, but have we considered Redmine[1]? As the resident Ruby is shiny, let's do everything in Rails on my MacBook guy, I'd like to make a statement against interest: I've tried Redmine a few times and it's been painful. Much of the codebase is deprecated, it's slow, it has no meaningful search (in 2012?!), I've seen wiki edits disappear, and at the moment pulling up its own FAQ page at redmine.org times out. Yay, that's totally FUD to me. Could you please elaborate a bit on your points? Deprecated codebase? Let me guess... It runs on an outdated version of Rails (2.3) but only because Rails is changing so rapidly, I believe. There is work in progress[1] to move to the supported branch Rails-3.x. Slow? Do you have any data to back this point up? No meaningful search, eh? Works for me. Disappearing wiki edits? Never seen that, but you can always file a bug. Maybe you've had better luck with it, but whenever I've Googled for Redmine questions, the collective Internet has sighed and said Yeah, it was a really good idea, though. Certainly *you've* had some terrible luck with it. ;-) -- Regards, Alex [1] http://www.redmine.org/issues/4796 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Robert Haas robertmh...@gmail.com writes: Even before this CommitFest, it's felt to me like this hasn't been a great cycle for reviewing. I think we have generally had fewer people doing reviews than we did during the 9.0 and 9.1 cycles. I think we had a lot of momentum with the CommitFest process when it was new, but three years on I think there's been some ebbing of the relative enthusiastic volunteerism that got off the ground. I don't have a I have another analysis here. Tom once said that commiters won't grow on tree, and that is true for submitters too, obviously. I think the CF process has been good into growing more submitters and growing existing ones abilities too. So you don't have that fewer reviewers available, it may be that they're just heavily involved into being submitters too. Tom Lane t...@sss.pgh.pa.us writes: Yeah, this is something I was thinking about yesterday. In the first couple of release cycles with the CommitFest process, we were willing to let the last fest of a release cycle go on for as long as it takes, or at least that was what I felt the policy to be. This time we eventually gave up and declared closure, but in hindsight we should likely have done that a month earlier. The fact of the matter is that quite a few of the patches we were dealing with were *not* ready to commit, or even close to that, at the start of the fest. If it weren't the last fest they would have gotten marked Returned With Feedback a lot sooner. This and other posts in this threads are all hinting the same thing to me: the last commit fest is *not* about feedback at all. If you still need reviewers rather than commiters, you're out of luck for this release, see you next time. Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Tom Lane t...@sss.pgh.pa.us writes: Andres Freund and...@anarazel.de writes: They might have been half-baked. But several of those didn't get design-level review for several weeks which makes it rather hard to fully bake them in time... But if they didn't already have design-level review, that means they were not seen in any previous CF, which means they were not following the expectation that nontrivial patches should be submitted earlier than the last CF. That's not true, unfortunately. I had several rounds of design review in the previous CF, then some more in the last CF, then another one that basically meant baking another patch: same feature but spelled out differently (renaming catalog, files and APIs is no fun in a big patch) to be able to have a new way to organize the code and docs. It's for the best, and it's better to do that before commit, no question. I appreciate having had that review, really. The problem with the command trigger has been another one. First, if we want the last CF to be only about commits and not about feedback, we should simply bite the bullet and spell it out loud. Second, we should have been talking about the hard decision rather than pretending we could still do something in this release. That's mainly on me and I know it, but not being a commiter I have no other power than influence, and I've been optimistic (and playing by the rules). I think the key point here is that people have to expect that it's going to take more than one round of review to land most nontrivial patches. And we have to discourage the expectation that that can happen within the last CF of a release cycle. If anything, the last CF has to be tighter not looser than others on what we will accept, because there is no time to recover if something proves wrong with a patch after a month or three. The more I think about that, the more I think we should declare the last CF to be about preparing a release, not providing another round of feedback. That could well be all we need here. I keep coming back to the thought that we need more and shorter CFs, and/or ReviewFests that are meant to help push WIP patches further down the track. We need to make it easier to get those early reviews done while there's still development time left. Separating away commit fest made for commits and those meant for feedback is a good idea too. I'm not sure how much benefit we would get there for non-last CFs, though. Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Robert Haas robertmh...@gmail.com writes: The real problem with the command triggers patch is that we got a blizzard of code. It's unrealistic to expect anyone to devote serious review time to a patch that's under constant development. It also strikes me that a tremendous amount of pain could have been avoided by posting a clear and detailed design sketch for that patch before beginning to code. Dimitri contended that without code, no one will read design sketches, but that doesn't for the most part jive with my experience, and I think that the strategy he actually chose backfired, because it was clear that any review would be hitting a moving target. In my mind at least it's been more subtle. I had an agreed-on design months before I started to code anything, at the Cluster Hackers Meeting in Ottawa, where several commiters and long term contributors have been participating in the discussion. The big mistake seems to be starting to code with that rather than spending another couple of months (or easily way more than that) of rehashing it on-list. About the design sketches, IME, what you can agree on on-list is a very high level view about how to implement a feature, anything detailed enough to have practical impact on the code you're writing happens while reviewing code. Again, as I though the high level view was ok as of the Cluster Hackers Meeting, I skipped that part and went directly to the code level review. Greg Smith will certainly stamp that email with a big red “Lesson Learned” stamp here: nothing happened if you don't have a link to a pgsql-hackers thread on the archives. Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On tor, 2012-04-12 at 10:12 -0500, Joshua Berkus wrote: Well actually, the other advantage of using branches is that it would encourage committers to bounce a patch back to the submitter for modification *instead of* doing it themselves. This would both have the advantage of saving time for the committer, and doing a better job of teaching submitters how to craft patches which don't need to be modified. I don't see how using branches changes that. You could (or should) be doing that already anyway. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On ons, 2012-04-11 at 23:30 -0400, Robert Haas wrote: Now what would be sort of neat is if we had a way to keep all the versions of patch X plus author and reviewer information, links to reviews and discussion, etc. in some sort of centralized place. Well, a properly linked email thread contains all this. I have seen a couple of anti-patterns evolving, though, including patch authors starting a new email thread for each patch version, and reviewers starting a new email thread for each review. When that happens, the existence of the commitfest app makes things worse, in a way. (Of course, any discussion here about bug trackers emphasizes the need for email to be the primary communications method for this very reason.) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On tor, 2012-04-12 at 07:59 +0200, Magnus Hagander wrote: It might be helpful (if the CF app had a trivial API) with a small tool that could run from a git hook (or manual script or alias) that would prompt for which cf entry, if any, did this commit close? An API for the CF app would actually be nice in general, because then I could write an small command-line tool and just type 'cf close 123' or something instead of having to click around a bunch of times. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On tor, 2012-04-12 at 18:19 -0400, Christopher Browne wrote: On Thu, Apr 12, 2012 at 6:11 PM, Jay Levitt jay.lev...@gmail.com wrote: Rather than extend the CF app into a trivial-patch workflow app, it might be worth looking at integrating it with github. There's a reluctance to require a proprietary component that could disappear on us without notice. Yeah, I think initially this was more of a matter-of-principle decision, but seeing the current spam problem on github.com, I'm glad we didn't do it. For me it has disqualified itself as a platform for intense collaborative development. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Thu, Apr 12, 2012 at 05:49, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Apr 11, 2012 at 5:36 PM, Peter Eisentraut pete...@gmx.net wrote: I'd still review it, but I'd be able to spend say 3 minutes on review and 30 seconds on committing it, versus 3 minutes on review, 3 minutes on research, and 8 minutes on bookkeeping. Well, I am not averse to figuring out a better workflow, or some better tools. In practice, I think it's going to be hard to reduce the time to review a trivial patch much below 5-10 minutes, which is what it takes me now, because you've got to read the email, download the patch, check that it doesn't break the build, review, commit, and push, and I can't really see any of those steps going away. But that doesn't mean we shouldn't make the attempt, because I've got to admit that the current workflow seems a little cumbersome to me, too. I'm not sure I have a better idea, though. git remotes seem useful for collaborating on topic branches, but I don't think they can really be expected to save much of anything during the final commit process - which is basically all the process there is, when the patch is trivial. Now what would be sort of neat is if we had a way to keep all the versions of patch X plus author and reviewer information, links to reviews and discussion, etc. in some sort of centralized place. The CommitFest app was actually designed to track a lot of this information, but it's obviously not completely succeeding in tracking everything that people care about - it only contains links to patches and not patches themselves; it doesn't have any place to store proposed commit messages; etc. There might be room for improvement there, although getting consensus on what improvement looks like may not be totally straightforward, since I think Tom's ideal process for submitting a patch starts with attaching a file to an email and many other people I think would like to see it start with a pull request. This is not entirely a tools issue, of course, but it's in there somewhere. It strikes me that there are two different scenarios being discussed here, and we'd better be sure we keep them straight: small-to-trivial patches, and complex patches. I think that for the trivial case, what we need is less tooling not more. Entering a patch in the CF app, updating and closing it will add a not-small percentage to the total effort required to deal with a small patch (as Peter already noted, and he wasn't even counting the time to put the patch into CF initially). The only reason to even consider doing that is to make sure the patch doesn't get forgotten. Perhaps we could have some lighter-weight method of tracking such things? If we were actually using git branches for it, the CF app could automatically close entries when they were committed. But that requires them to be committed *unmodified*, and I'm not sure that's reasonable. I also think requiring a git branch for the *simple* changes is adding more tooling and not less, and thus fails on that suggestion. It might be helpful (if the CF app had a trivial API) with a small tool that could run from a git hook (or manual script or alias) that would prompt for which cf entry, if any, did this commit close? At the other end of the scale, I think it's true that the CF app could be more helpful than it is for tracking the state of complex patches. I don't really have any concrete suggestions, other than that I've seen far too many cases where the latest version of a patch was not linked into the CF entry. Somehow we've got to make that more robust. Maybe the answer is to tie things more directly into git workflows, though I'm not sure about details. I am concerned about losing traceability of submissions if all that ever shows up in the list archives is a URL. I've suggested before that it would be a good idea to be able to register a git repo + branch name in the commitfest app, and be able to track that. If it was smart enough to figure out that for something like github or bitbucket it could also add a web link (but keep the git link for whoever wants to pull it remotely) with the full differences to master, that would make *some* of those issues go away. (Certainly not all, it's not a magic solution, but I believe it would be a tool that could help). I've pretty much given up on that happening though... -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Tue, Apr 10, 2012 at 08:43:12PM -0400, Greg Smith wrote: The main reason I worry about this is because of a very real chicken/egg problem here that I keep banging into. Since the commit standards for so many other open-source projects are low, there are a non trivial number of business people who assume !committer == ![trusted|competent]. That makes having such a limited number of people who can commit both a PR issue (this project must not be very important if there are only 19 committers) and one limiting sponsorship (I'm not going to pay someone to work on this feature who's been working on it for years but isn't even a committer). There are a significant number of companies who are willing to sponsor committers to open-source projects; there are almost none who will sponsor reviewers or contributors of any stature unless they're already deep into the PostgreSQL community. That's one of the many reasons it's easier for a committer to attract funding for core PostgreSQL work, be it in the form of a full-time job or project-oriented funding. The corresponding flip side to that is that the small number of committers is limiting the scope of funding the project can accumulate. I want to caution against adjusting things to improve funding possibilities. There is nothing wrong with increasing funding possibilities, per say, but such changes often distort behavior in unforeseen ways that adversely affect our community process. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Excerpts from Tom Lane's message of jue abr 12 00:49:38 -0300 2012: At the other end of the scale, I think it's true that the CF app could be more helpful than it is for tracking the state of complex patches. I don't really have any concrete suggestions, other than that I've seen far too many cases where the latest version of a patch was not linked into the CF entry. Somehow we've got to make that more robust. Maybe the answer is to tie things more directly into git workflows, though I'm not sure about details. I am concerned about losing traceability of submissions if all that ever shows up in the list archives is a URL. Two suggestions: 1. it might be convenient to have the patch author attach a suggested commit message to the patch entry in the commifest site. Would save some jiffies for the trivial patch case, I hope. 2. instead of just sending a URL to the list, maybe it'd be better if the patch is uploaded to the CF site, and the CF site sends it to pgsql-hackers for archival and reference, with appropriate In-Reply-To headers so that it is appropriately linked to the thread. But since the patch has been registered into the CF, the site can additionally present a link to download the patch directly instead of sending you to the archives. So redundant storage, for convenience. (Alternatively, the CF app could reach into archives to grab the patch file. With some appropriate ajaxy stuff this shouldn't be particularly hard.) -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 12 April 2012 13:45, Bruce Momjian br...@momjian.us wrote: I want to caution against adjusting things to improve funding possibilities. There is nothing wrong with increasing funding possibilities, per say, but such changes often distort behavior in unforeseen ways that adversely affect our community process. Funding is a necessary component of what we do. So, for example, while I'm very glad that EnterpriseDB afford various people the opportunity to work on community stuff for a significant proportion of their time - I do, after all, indirectly benefit from it - it is rather obviously the case that the particular things that those people work on is influenced to some degree by management. That is an assessment that isn't based on any particular observation about the things that EDB people work on. It's just common sense. This generally isn't a bad thing, since I think that the goals of the Postgres companies are broadly aligned with those of the community. When you get right down to it though, as Tom said, we are a herd of cats, and it isn't particularly obvious that we've zeroed in on some specific vision that we all agree on that must be pursued without diversion. Given the extensibility of Postgres, it isn't usually necessary for anyone to pursue development of a feature that is clearly of niche interest, that we don't really want to have to support. I cannot think of any example of a proposed patch that mostly just scratched some particular organisation's itch. No one is able to hoodwink the community like that. People have always wanted to get their patches accepted, and we've always had high standards. The fact that there might be an additional financial incentive to do so doesn't seem to fundamentally alter that dynamic. It is not a coincidence that I did not send any code to -hackers prior to joining 2ndQuadrant. I certainly had the enthusiasm for it, but I could not afford to dedicate sufficient time. With the kind of dedication required to make a noticeable contribution, this is hardly surprising. There are some good counter-examples of this of course - one in particular that comes to mind is Greg Smith's work on the background writer that made it into 8.3 . However, the general trend is that somebody has to pay for this work for it to be maintainable over months and years, even with the level of dedication that we all have. Something that I would suggest is that those that are receiving funding be transparent about it. It isn't essential of course, but to do any less might lead to the perception of there being a conflict of interests in some people's minds, which is best avoided. I am conscious of the fact that I've expressed lots of opinions on this thread on our processes and so on, some of which, if followed through on, would be quite large departures. I hope that they were received as modest suggestions. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 04/11/2012 10:24 AM, Tom Lane wrote: Greg Smithg...@2ndquadrant.com writes: I'd like to dump around 50 pages of new material into the docs as a start, but I don't want to take so much time away from the code oriented committers to chew on that much. Well, with all due respect, that does not sound like a change that doesn't need review. I wasn't trying to suggest large changes should be made without review. I'd just like some new paths for work to progress without one of the more coding oriented committers being compelled to join and keep up with everything. The quality level I aimed for in my book wouldn't have been possible without Kevin Grittner, Scott Marlowe, and Jim Mlodgenski as reviewers; it didn't require anyone with commit bits though. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
If we were actually using git branches for it, the CF app could automatically close entries when they were committed. But that requires them to be committed *unmodified*, and I'm not sure that's reasonable. I also think requiring a git branch for the *simple* changes is adding more tooling and not less, and thus fails on that suggestion. Well actually, the other advantage of using branches is that it would encourage committers to bounce a patch back to the submitter for modification *instead of* doing it themselves. This would both have the advantage of saving time for the committer, and doing a better job of teaching submitters how to craft patches which don't need to be modified. Ultimately, we need to train new major contributors in order to get past the current bottleneck. Of course, this doesn't work as well for contributors who *can't* improve their patches, such as folks who have a language barrier with the comments. But it's something to think about. --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 I want to caution against adjusting things to improve funding possibilities. There is nothing wrong with increasing funding possibilities, per say, but such changes often distort behavior in unforeseen ways that adversely affect our community process. I don't see this as much of a problem. If somewhat arbitrary labels and powers allow the project to succeed, we should think long and hard before rejecting the idea. It's not like we are going to make anyone who asks a committer, like MediaWiki does. Indeed, we have been super cautious about handing out both commit bits, and labels (e.g. Major Developer). One wrinkle is the subsystems: there are some people who only work on certain parts, yet have a commit bit (with the understanding that they won't start editing core or other parts). From an outside perspective however, a Postgres committer [of certain subsystems] is a Postgres committer. One thing I think would help potential and current developers, and act as a further code review and safety valve, is to have a mailing list that actually shows the committed diffs. Links to a webpage showing the diff is just not the same. pgsql-commit-di...@postgresql.org, anyone? - -- Greg Sabino Mullane g...@turnstep.com End Point Corporation http://www.endpoint.com/ PGP Key: 0x14964AC8 201204121121 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAk+G8/sACgkQvJuQZxSWSsh7HACgn7Wf/AQyUJwtvxgjYSHSIHkJ hq4AnjMgPlDakupg4mo204+N1p4C0mMZ =z+cR -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Thu, Apr 12, 2012 at 03:34:31PM +0100, Peter Geoghegan wrote: Something that I would suggest is that those that are receiving funding be transparent about it. It isn't essential of course, but to do any less might lead to the perception of there being a conflict of interests in some people's minds, which is best avoided. I am conscious of the fact that I've expressed lots of opinions on this thread on our processes and so on, some of which, if followed through on, would be quite large departures. I hope that they were received as modest suggestions. I appreciate everything everyone said in this thread, and I can't think of an example off the top of my head where vendors adversely affected our process. I think the _big_ reason for that is that our community members have always acted with a community first attitude that has insulated us from many of the pressures vendors can place on the development process. I am sure that protection will continue --- I just wanted to point out that it is a necessary protection so we can all be proud of our released code and feature set, and continue working as a well-coordinated team. The specific suggestion that vendors are not taking contributors seriously unless they have commit-bits is perhaps something that requires education of vendors, or perhaps my blogging about this will help. Greg Smith's analysis really hit home with me: a non trivial number of business people who assume !committer == ![trusted|competent]. That makes having such a limited number of people who can commit both a PR issue (this project must not be very important if there are only 19 committers) and one limiting sponsorship (I'm not going to pay someone to work on this feature who's been working on it for years but isn't even a committer). I think the big take-away, education-wise, is that for our project, committer == grunt work. Remember, I used to be the big committer of non-committer patches --- need I say more. ;-) LOL -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
I think the big take-away, education-wise, is that for our project, committer == grunt work. Remember, I used to be the big committer of non-committer patches --- need I say more. ;-) LOL Well, promoting several people to committer specifically and publically because of their review work would send that message a lot more strongly than your blog would. It would also provide an incentive for a few of our major contributors to do more review work, if it got them to committer. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Thu, Apr 12, 2012 at 11:34:48AM -0400, Bruce Momjian wrote: On Thu, Apr 12, 2012 at 03:34:31PM +0100, Peter Geoghegan wrote: Something that I would suggest is that those that are receiving funding be transparent about it. It isn't essential of course, but to do any less might lead to the perception of there being a conflict of interests in some people's minds, which is best avoided. I am conscious of the fact that I've expressed lots of opinions on this thread on our processes and so on, some of which, if followed through on, would be quite large departures. I hope that they were received as modest suggestions. I appreciate everything everyone said in this thread, and I can't think of an example off the top of my head where vendors adversely affected our process. I think the _big_ reason for that is that our community members have always acted with a community first attitude that has insulated us from many of the pressures vendors can place on the development process. I am sure that protection will continue --- I just wanted to point out that it is a necessary protection so we can all be proud of our released code and feature set, and continue working as a well-coordinated team. Let me add one more thing. As someone who has been funded for Postgres work since 2000, I am certainly pro-funding! Since our community members have a community first attitude, it is the community's responsibility to help them get funding. We have thrown around a few ideas in this thread, but perhaps someone should start a new email thread with first-hand suggestions of how we can help people get funding. I am certainly ready to help however I can. My reason for replying to this thread was to highlight our valuable community first attitude. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 04/12/2012 11:34 AM, Bruce Momjian wrote: The specific suggestion that vendors are not taking contributors seriously unless they have commit-bits is perhaps something that requires education of vendors, or perhaps my blogging about this will help. I'm glad I managed to vent my frustration in this area in a way that was helpful. Just recognize that any experienced person at pitching software solutions will tell you to never wander down this path at all. If you have to tell someone a story and make them admit they're wrong about something as an early step toward adoption, you've just dumped a home-made FUD bomb on them. It's not a high percentage path toward credibility. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 12:00:39PM -0300, Alvaro Herrera wrote: remote in their main PG tree, and so changesets could be pulled into the same clone and cherry-picked into the master branch. If you're talking about a way of using git to support reviewing, the Gerrit tool has an interesting workflow. Essentially anything you want reviewed you push to a fake tag refs/for/master which always creates a new branch. As such you have a repository which contains every patch ever submitted, but it simultaneously tracks the parents so you know which version of the tree a patch was against. In the case of Postgres each entry in the CF app would have its own tag (say refs/cf/234) which would create a new patch for that entry. In the end accepted patches are cherry-picked onto the real tree. But because all patches are now in the same place you can build tooling around it easier, like testing: does this patch cherry-pick cleanly or is there a conflict. No merge commits, just using git purely as patch storage. (Note to make this work it has a git server emulation which may or may not be easy to do, but it's just a thought about workflow.) Have a nice day, -- Martijn van Oosterhout klep...@svana.org http://svana.org/kleptog/ He who writes carelessly confesses thereby at the very outset that he does not attach much importance to his own thoughts. -- Arthur Schopenhauer signature.asc Description: Digital signature
Re: [HACKERS] Last gasp
Alvaro Herrera wrote: Now what would be sort of neat is if we had a way to keep all the versions of patch X plus author and reviewer information, links to reviews and discussion, etc. in some sort of centralized place. FWIW: y'all might have discussed to death during the git migration, so *please* do not let me derail you if so... github does a great job of exactly this. You open an issue, you reference it from commits, all the related commits are listed in (and browseable from) the issue, you can comment on specific lines of the commit, it integrates w/email, it has an API to write tools (both workflow and archival) against, etc. Rather than extend the CF app into a trivial-patch workflow app, it might be worth looking at integrating it with github. Jay Levitt -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Thu, Apr 12, 2012 at 6:11 PM, Jay Levitt jay.lev...@gmail.com wrote: Rather than extend the CF app into a trivial-patch workflow app, it might be worth looking at integrating it with github. There's a reluctance to require a proprietary component that could disappear on us without notice. The existence of git itself is a result of *exactly* that circumstance, as Linux kernel developers had gotten dependent on BitKeeper, whereupon the owner decided to take his toys home, at which point they were left bereft of their SCM tool. http://kerneltrap.org/node/4966 I expect that it would be more worthwhile to look into enhancements to git workflow such as http://code.google.com/p/gerrit/ Gerrit. I don't know that Gerrit is THE answer, but there are certainly projects that have found it of value, and it doesn't have the oops, it's proprietary problem. -- When confronted by a difficult problem, solve it by reducing it to the question, How would the Lone Ranger handle this? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wednesday, April 11, 2012, Robert Haas wrote: On Tue, Apr 10, 2012 at 10:05 PM, Peter Geoghegan pe...@2ndquadrant.comjavascript:; wrote: On 11 April 2012 02:14, Robert Haas robertmh...@gmail.comjavascript:; wrote: My perception of what's going on here is dramatically different from yours. I don't think there was any overflow of submissions for 9.2. That is just not true. See the attached graph (couldn't produce one with better resolution at short notice) - I've just eyeballed the graph, but it looks like an upward trend to me. I don't know what this is a graph of, but if you look at the number of patches in each of the CommitFests for the last couple of releases, you see this: 9.0: 66, 40, 38, 60 9.1: 53, 52, 43, 96 9.2: 60, 52, 53, 104 There might be an upward trend there, but it isn't a very steep one. It also seems pretty clear to me (although you are welcome to disagree) that even if the *number* of patches in 9.2 was higher than 9.1, the average complexity was less, at least for the first three CommitFests. By the way, let's take a look at the major features list for 9.0 and 9.1, as well as who wrote them. From the 9.0 release notes: - Streaming Replication (Fujii Masao - not a committer) - Hot Standby (Simon Riggs - committer - though he wasn't when he wrote the first version of this patch) - GRANT/REVOKE IN SCHEMA (Petr Jelinek - not a committer) - ALTER DEFAULT PRIVILEGES (Petr Jelinek - not a committer) - DO (Peter Jelinek - not a committer) - 64-bit Windows (Tsutomu Yamada, Magnus Hagander - not a committer and a committer, respectively) - Better window functions (Hitoshi Harada - not a committer) - ORDER BY for aggregates (Andrew Gierth - not a committer) - Deferrable unique constraints (Dean Rasheed - not a committer) - Exclusion constraints (Jeff Davis - not a committer) - RADIUS (Magnus Hagander - a committer) - LDAP improvements (Robert Fleming, Magnus Hagander - not a committer and a committer, respectively) - Better LISTEN/NOTIFY (Joachim Wieland - not a committer) - Better VACUUM FULL (Itagaki Takahiro, Tom Lane - both now committers, but Itagaki Takahiro became one only later) - pg_upgrade (Bruce Momjian - a committer) - join removal (Robert Haas - now a committer, but not then) - EXPLAIN enhancements (Robert Haas - now a committer, but not then) - hstore enhancments (Andrew Gierth - now a committer, but not then) For the sake of the archives - Andrew Gierth is not a committer now either. //Magnus -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Re: [HACKERS] Last gasp
On Tue, Apr 10, 2012 at 8:12 PM, Robert Haas robertmh...@gmail.com wrote: However exactly the list turns out, there is no question that non-committers have been quite successful in getting significant feature enhancements committed in each of the last three releases, and I'm pretty confident it goes back a lot further than that. I agree that in practice non-committers do get a lot of great stuff done, but the question is if *more* stuff would get done if things were slightly different. To that end, I'd like to share my own anecdote on why I don't attempt large projects to Postgres at this time: I used to work on a proprietary postgres offspring and ship quite a bit of internals code. A couple of people I worked with are frequenters of this list, even. I spent nearly two years doing it, full time, without having to (or being able to) go through a full-blown community process from design to ship: I got a lot of nuts-and-bolts practice, and I really enjoyed it. Yet I don't take on large projects in the project now, and I'm even employed in a place where I could start doing that on-the-job. Why don't I? One reason I don't do that is because there is an unavoidable information asymmetry problem between myself and the committers. When I think of a committer and what makes me different than them, this is what I come up with: * More experience and expertise, both in general and with the project * Proven intent to maintain the work submitted by others for a long time. In a word, stewardship or ownership I'm grateful for both, but the latter point is one where some mind-reading is required: what's strategically important enough that's it is important enough to compromise on something? What compromises are acceptable? That is tantamount to guessing what compromises is a committer willing to maintain? And that can be a pretty personal thing and is hard to guess, and I don't think that's solvable as long as there seems to be this handoff from the contributor to the project. It's hard to feel a sense of ownership -- and thus commitment -- if one cannot simply change one's own code, especially for trivia or churning around a future intent or purpose. If there is a bottleneck with the development process that is having a chilling effect on my ability to justify larger projects, it is this. I don't know what the most apparent way to optimize that bottleneck is, but there's my thoughts. I think part of the answer is more hooks, even if they come with reduced contracts in terms of maintenance (here this release, gone the next release), and less required justification to commit those; consider https://github.com/dimitri/pgextwlist, which relies on such a hook and is the only thing that makes 9.1's extension support viable for Heroku, yet is cohesive feeling with the rest of the system to the point that it pretty much goes unnoticed. That's a great property I wish I could see more of. Also, I am not able to spend as much time on large Postgres-projects because some of the tooling outside the database is still the weakest link in my chain for now, so the good news is that I've shifted my attention to projects that are very much related but not part postgres-proper. The point of having more hooks is to open up more opportunities for such tools without making the result feel incohesive and terrible for reasons not intrinsic to the idea it is trying to add (consider: using the statement hook in pgextwlist rather than preloaded security-definer UDFs that would run CREATE EXTENSION for you. Yuck.) -- fdr -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 04/10/2012 09:14 PM, Robert Haas wrote: I wouldn't object to creating some doc-only committers. OTOH, I would object to anyone making non-trivial documentation enhancements without posting their patches first and having a second person look it over, so how much difference is there, really? This workflow is the easy one: -Committer suggests doc change -No one has an opinion against it strong enough to comment, or minor review comments are made -Commit change with feedback when received That's a predictable, short process unless the change is controversial, in which case good feedback normally comes out of that discussion. And if I feel review is needed but don't catch any volunteers, there is a much broader pool of people who can be hired to help with review work, relative to the size of the committer one. Compare with: -Submitter suggests doc change -No one has a strong opinion on it, may not be picked up at all -Submitter adds to the next CF -Wait for review -[Possible repost update with reviewer changes] -Ready for committer -Committer takes time away from code review to look at it -Possibly another feedback/review resubmission -Commit final versions It's usually not this bad, but in every case it's pulling resources off of more valuable jobs. I'd like to dump around 50 pages of new material into the docs as a start, but I don't want to take so much time away from the code oriented committers to chew on that much. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Greg Smith g...@2ndquadrant.com writes: On 04/10/2012 09:14 PM, Robert Haas wrote: I wouldn't object to creating some doc-only committers. OTOH, I would object to anyone making non-trivial documentation enhancements without posting their patches first and having a second person look it over, so how much difference is there, really? ... I'd like to dump around 50 pages of new material into the docs as a start, but I don't want to take so much time away from the code oriented committers to chew on that much. Well, with all due respect, that does not sound like a change that doesn't need review. I have not noticed people adding docs-only changes to the CFs; usually it's more like post a patch, somebody looks it over and commits it. I agree that this is still too much overhead for simple fixes, such as the editorial glitches that Thom Brown is so good at finding (and I'm about ready to vote to give him a commit bit for that work). But a fifty-page chunk is not that, indeed it sounds like it would have enough technical content that it might actually merit a full-scale review. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 11 April 2012 03:26, Tom Lane t...@sss.pgh.pa.us wrote: [ scratches head... ] That's supposed to be total lines of code in our source tree? What's the big drop in late 2009, then? I had wondered about that myself - all I can tell you is that I used the tool as advertised, without any adornments. This particular tool is a bit misleading, because it counts lines of code as lines of checked-in text, which can include things that are very verbose without necessarily being what we really think of as code - it doesn't make any effort to discriminate against non-code, nor does it expose the option of doing so. I chose it at short notice simply because it produces graphs. I do intend to take a look at this problem in more detail, and get better statistics on changes to our codebase - it's a tricky proposition, though. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 16:24, Tom Lane t...@sss.pgh.pa.us wrote: Greg Smith g...@2ndquadrant.com writes: On 04/10/2012 09:14 PM, Robert Haas wrote: I wouldn't object to creating some doc-only committers. OTOH, I would object to anyone making non-trivial documentation enhancements without posting their patches first and having a second person look it over, so how much difference is there, really? ... I'd like to dump around 50 pages of new material into the docs as a start, but I don't want to take so much time away from the code oriented committers to chew on that much. Well, with all due respect, that does not sound like a change that doesn't need review. I have not noticed people adding docs-only changes to the CFs; usually it's more like post a patch, somebody looks it over and commits it. I agree that this is still too much overhead for simple fixes, such as the editorial glitches that Thom Brown is so good at finding (and I'm about ready to vote to give him a commit bit for that work). But a fifty-page chunk is not that, indeed it sounds like it would have enough technical content that it might actually merit a full-scale review. Since the topic is somewhat drifting here anyway.. :-) Might it be worthwhile to allow some sort of staging repository and actually start using the git stuff a bit more around this? E.g. we could have a docs repo somewhere where more people have commit bits, and then they are just regularly merged back into the main tree? Sort of the way different people can own different subsystems in other projects, but someone does the trusted merge? For example, Thom (and others) could collect a number of typo fixes in their own repo and then just ask for a merge.The advantage over just staging multiple commits and then submitting a patch would be that multiple people could work on it... -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 04/10/2012 08:43 PM, Greg Smith wrote: On 04/10/2012 01:28 PM, Robert Haas wrote: The fact is that we have no shortage of committers - there are 19 people who have access to push code into our master git repository. A handful of those people have basically completely left the project and their commit rights should probably be revoked on that basis; most of them are still involved in one way or another but just not able to devote a lot of time to reviewing other people's code. Let's use realistic numbers here: I count 7 people who regularly review and commit other people's code in a variety of areas. There are also several subject matter experts who commit things in a relatively narrow area. But most bigger patches are going to hit a bottleneck whose capacity could be measured in 3 bits. Robert's point is fairly important. A few years ago one of the Postgres companies hired me for a few weeks to push along some of the stuff that was holding up a release at a critical time. (This was in the pre CF days.) Several important features made it into that release that almost certainly would not have otherwise. But generally that hasn't happened, and the time I've devoted has been my own, and thus subject to competing demands of business and the mythical life AFK. Sometimes that makes it quite hard to arrange time to spend on reviewing and committing large patches. There are a significant number of companies who are willing to sponsor committers to open-source projects; there are almost none who will sponsor reviewers or contributors of any stature unless they're already deep into the PostgreSQL community. That's one of the many reasons it's easier for a committer to attract funding for core PostgreSQL work, be it in the form of a full-time job or project-oriented funding. The corresponding flip side to that is that the small number of committers is limiting the scope of funding the project can accumulate. Yep. The plus side is that Postgres business seems to be growing quite rapidly (if our experience at PostgreSQL Experts Inc is anything to go by), so maybe there will be more sources of funding available. More work needs to be done to generate funds to support getting features reviewed and committed. Patches seems to be getting ever larger and more complex, and that makes reviewing them harder and much more time consuming. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Excerpts from Magnus Hagander's message of mié abr 11 11:35:10 -0300 2012: For example, Thom (and others) could collect a number of typo fixes in their own repo and then just ask for a merge.The advantage over just staging multiple commits and then submitting a patch would be that multiple people could work on it... The other advantage is that committers could have that tree as another remote in their main PG tree, and so changesets could be pulled into the same clone and cherry-picked into the master branch. (I'd also set up a clone for Alex Hunsaker to commit to pl/perl). -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 10:35 AM, Magnus Hagander mag...@hagander.net wrote: Since the topic is somewhat drifting here anyway.. :-) Might it be worthwhile to allow some sort of staging repository and actually start using the git stuff a bit more around this? E.g. we could have a docs repo somewhere where more people have commit bits, and then they are just regularly merged back into the main tree? Sort of the way different people can own different subsystems in other projects, but someone does the trusted merge? For example, Thom (and others) could collect a number of typo fixes in their own repo and then just ask for a merge.The advantage over just staging multiple commits and then submitting a patch would be that multiple people could work on it... If our goal is to give people more or less unfettered access to certain areas of the tree, but not the whole thing, we should perhaps consider just doing that directly. There's no particular reason why Thom Brown can't be made a committer just for docs, or why Alex Hunsaker can't be made a committer just for PL/perl (and presumably docs, since he'd need to update the docs if he updates the code), if that's actually what we want to do. Now, the advantage of a staging tree is that it gives the people who have commit rights to the main repository the ability to decline to merge. The question is - what happens then, especially given that we have so many committers already? In Linux-land, it becomes the subsystem maintainer's responsibility to put the tree into a state where Linus will again become willing to merge it, or he can fire the subsystem maintainer and pick a new one that'll do what he wants. But we don't work that way. Instead, the committers as a group have the responsibility for not breaking stuff. So who would decide whether to do the merge, and who would be responsible for fixing things if the merge were refused? As far as I can see, this basically amounts to bundling lots of unrelated changes into one big pile and then asking to have them all committed at once instead of one at a time, which sounds like more work not less, unless we're just going to blindly merge without reviewing, in which case we may as well just let people commit to the main repository themselves. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Robert Haas robertmh...@gmail.com writes: On Wed, Apr 11, 2012 at 10:35 AM, Magnus Hagander mag...@hagander.net wrote: Might it be worthwhile to allow some sort of staging repository and actually start using the git stuff a bit more around this? ... As far as I can see, this basically amounts to bundling lots of unrelated changes into one big pile and then asking to have them all committed at once instead of one at a time, which sounds like more work not less, unless we're just going to blindly merge without reviewing, in which case we may as well just let people commit to the main repository themselves. Yeah. I'm also worried that we would either lose linear history, or (if we squash the merge commits) lose change log history for the individual fixes. Neither of those sounds terribly attractive. We've frequently had, and still have today, committers who are understood to have limited areas of expertise and are given commit bits on the honor system to not break what they don't know well. I don't have any problem with continuing in that line. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 11:38 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Apr 11, 2012 at 10:35 AM, Magnus Hagander mag...@hagander.net wrote: Might it be worthwhile to allow some sort of staging repository and actually start using the git stuff a bit more around this? ... As far as I can see, this basically amounts to bundling lots of unrelated changes into one big pile and then asking to have them all committed at once instead of one at a time, which sounds like more work not less, unless we're just going to blindly merge without reviewing, in which case we may as well just let people commit to the main repository themselves. Yeah. I'm also worried that we would either lose linear history, or (if we squash the merge commits) lose change log history for the individual fixes. Neither of those sounds terribly attractive. We've frequently had, and still have today, committers who are understood to have limited areas of expertise and are given commit bits on the honor system to not break what they don't know well. I don't have any problem with continuing in that line. Me neither, but I don't know how far it scales. Having certain people who are defined as, say, doc-only committers will not only make it clear to those people what they're expected to commit, but also clear to everyone else who the people are who might commit any given patch they might write. If we just end up with 50 committers and you have to follow pgsql-hackers to understand who knows what and which people are even still around, it's not going to make anything easier for anyone. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Excerpts from Robert Haas's message of mié abr 11 12:44:02 -0300 2012: Me neither, but I don't know how far it scales. Having certain people who are defined as, say, doc-only committers will not only make it clear to those people what they're expected to commit, but also clear to everyone else who the people are who might commit any given patch they might write. If we just end up with 50 committers and you have to follow pgsql-hackers to understand who knows what and which people are even still around, it's not going to make anything easier for anyone. Since we're so keen on copying what Linux does, we could just have a MAINTAINERS file. -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
All, From my observation, the CF process ... in fact, all development processes we've had in Postgres ... have suffered from only one problem: lack of consensus on how the process should work. For example, we've *never* had consensus around the criteria for kicking a patch out of a commitfest. This lack of consensus has resulted in disorganization, ennui towards the process, deadline overruns, and a lot of general unhappiness. People have stopped believing in the CF system because we've stopped running it. I'm encouraged at this point that we've seen where this lack of consensus can lead us, maybe at this point we're willing to set aside individual differences of opinion on what the criteria should be (especially when it comes to the patches we each individually care about) in service of a smoother-running process. Some suggestions: - for the first 2 weeks of each CF, there should be a *total* moritorium on discussing any features not in the current CF on -hackers. - the CF manager should have unquestioned authority to kick patches. As in, no arguing. - we should have simple rules for the CF manager for kicking patches, as in: * no response from author in 5 days * judged as needing substantial work by reviewer * feature needs spec discussion However, the real criteria don't matter as much as coming up with a set of criteria we're all willing to obey, whatever they are. We also need better tools for the CF, but frankly better tools is a minor issue and easily solved if we have a consensus which people are willing to obey. For that matter, if we have a smooth and impartial process, we can do other things, including: training new reviewers, promoting new committers, changing the length of the CF cycle, or changing the PostgreSQL release cycle (yes, really). While our review and commit process is completely subjective and inconsistent, though, we can't do any of these things. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 11 April 2012 15:35, Magnus Hagander mag...@hagander.net wrote: Might it be worthwhile to allow some sort of staging repository and actually start using the git stuff a bit more around this? E.g. we could have a docs repo somewhere where more people have commit bits, and then they are just regularly merged back into the main tree? Sort of the way different people can own different subsystems in other projects, but someone does the trusted merge? For example, Thom (and others) could collect a number of typo fixes in their own repo and then just ask for a merge.The advantage over just staging multiple commits and then submitting a patch would be that multiple people could work on it... This is hardly a radical idea at all - it's basically how git was really intended to be used at scale. Of course, unless some committer is going to make it their responsibility to merge those commits say every 3 months, there's no point in bothering. This could consolidate the number of typo commits to boot, as they could be rebased. TBH, I find it slightly embarrassing to have to ask a committer to fix a minor typo, and it's hardly reasonable to expect me to save my typos up. Big +1 from me. It might be the case that over time, we become comfortable with this approach and upgrade the tree to a linux-next style tree (much like the -mm tree was repurposed into the progenitor of linux-next), with a lesser (though still substantial) standard for committers to meet. There could be an understanding that by committing to the tree, the developer makes a representation that they are confident that the feature is ready for prime-time, in just the same way that a commit currently represents - don't underestimate the power of that ceremony. Less senior contributors could have their work scrutinised by a wider body of people that haven't necessarily taken enough of an interest in the contributor's work to want to follow them on github or whatever - enthusiast power users who wouldn't really consider themselves hackers. This partially solves the you want us to fund feature development but you're not even a committer? problem that Greg referred to. It's also how big projects scale - technically, there are relatively few committers to the linux-stable tree. This approach formalises Tom's view that I think the key point here is that people have to expect that it's going to take more than one round of review to land most nontrivial patches. On 11 April 2012 16:27, Robert Haas robertmh...@gmail.com wrote: Now, the advantage of a staging tree is that it gives the people who have commit rights to the main repository the ability to decline to merge. The question is - what happens then, especially given that we have so many committers already? In Linux-land, it becomes the subsystem maintainer's responsibility to put the tree into a state where Linus will again become willing to merge it, or he can fire the subsystem maintainer and pick a new one that'll do what he wants. But we don't work that way. Instead, the committers as a group have the responsibility for not breaking stuff. So who would decide whether to do the merge, and who would be responsible for fixing things if the merge were refused? This seems like a non-issue to me. We just try and match the Linux model. In practice I'd imagine that the roles would not really be perfectly delineated like that, and I find it really doubtful that they are in Linux land either. There was a time when some people were somewhat skeptical of the git migration (at least privately), and that was, as far as I can tell, a roaring success. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Tom Lane t...@sss.pgh.pa.us wrote: What I'd be interested to see is number of lines changed per unit time; that would be a much better measure of patch rate IMHO. Based on `git diff --shortstat` between tags, for the whole tree, this is what shows up: files git tag changed insertions deletions === === == = PG95-1_01 Release_1_0_2133 10373 1046 REL2_0 1079 65115 41987 REL6_1 770 70656 42995 REL6_2 919 195720 141153 REL6_3 1140 104845 34326 REL6_4_21583 171053 69832 REL6_5 1366 136665 85013 REL7_0 1991 272086 142201 REL7_1 2164 441090 212481 REL7_2 1826 314583 139238 REL7_3 2258 631434 245295 REL7_4 2018 436323 283074 REL8_0_02398 446392 258825 REL8_1_01796 235680 181193 REL8_2_02343 288467 168438 REL8_3_02355 405974 174798 REL8_4_02464 339085 462065 REL9_0_02270 292187 191852 REL9_1_03225 443037 186042 HEAD2014 189648 94 -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Robert Haas robertmh...@gmail.com writes: On Wed, Apr 11, 2012 at 11:38 AM, Tom Lane t...@sss.pgh.pa.us wrote: We've frequently had, and still have today, committers who are understood to have limited areas of expertise and are given commit bits on the honor system to not break what they don't know well. I don't have any problem with continuing in that line. Me neither, but I don't know how far it scales. Having certain people who are defined as, say, doc-only committers will not only make it clear to those people what they're expected to commit, but also clear to everyone else who the people are who might commit any given patch they might write. If we just end up with 50 committers and you have to follow pgsql-hackers to understand who knows what and which people are even still around, it's not going to make anything easier for anyone. When and if we have 50 committers, we can worry about that problem ;-). But in practice people have different areas of expertise already. There was some mumbling upthread about trying to identify early who would be the responsible committer for any given CF submission --- doesn't seem to me like that is much different. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 8:59 AM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 11 April 2012 15:35, Magnus Hagander mag...@hagander.net wrote: For example, Thom (and others) could collect a number of typo fixes in their own repo and then just ask for a merge.The advantage over just staging multiple commits and then submitting a patch would be that multiple people could work on it... This is hardly a radical idea at all - it's basically how git was really intended to be used at scale. Of course, unless some committer is going to make it their responsibility to merge those commits say every 3 months, there's no point in bothering. This could consolidate the number of typo commits to boot, as they could be rebased. TBH, I find it slightly embarrassing to have to ask a committer to fix a minor typo, and it's hardly reasonable to expect me to save my typos up. Big +1 from me. Particularly for the docs, it'd be nice to have more committer bandwidth available, if there's a reasonable way to do so without causing needless merge work for existing committers. Like Peter, I hate to bother busy committers with trivial typofixes, and sometimes I just don't bother sending such changes in, and they get lost :-( Maybe keeping doc/ as a 'git submodule' could work? Or, as Tom suggests, adding a global committer who could focus on docs changes would effectively solve the problem as well. Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 18:29, Josh Kupershmidt schmi...@gmail.com wrote: On Wed, Apr 11, 2012 at 8:59 AM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 11 April 2012 15:35, Magnus Hagander mag...@hagander.net wrote: For example, Thom (and others) could collect a number of typo fixes in their own repo and then just ask for a merge.The advantage over just staging multiple commits and then submitting a patch would be that multiple people could work on it... This is hardly a radical idea at all - it's basically how git was really intended to be used at scale. Of course, unless some committer is going to make it their responsibility to merge those commits say every 3 months, there's no point in bothering. This could consolidate the number of typo commits to boot, as they could be rebased. TBH, I find it slightly embarrassing to have to ask a committer to fix a minor typo, and it's hardly reasonable to expect me to save my typos up. Big +1 from me. Particularly for the docs, it'd be nice to have more committer bandwidth available, if there's a reasonable way to do so without causing needless merge work for existing committers. Like Peter, I hate to bother busy committers with trivial typofixes, and sometimes I just don't bother sending such changes in, and they get lost :-( Maybe keeping doc/ as a 'git submodule' could work? Or, as Tom suggests, adding a global committer who could focus on docs changes would effectively solve the problem as well. git submodule would be a really bad idea imho. Then you couldn't make a single commit that deals with both code and docs. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Joshua Berkus j...@agliodbs.com writes: From my observation, the CF process ... in fact, all development processes we've had in Postgres ... have suffered from only one problem: lack of consensus on how the process should work. For example, we've *never* had consensus around the criteria for kicking a patch out of a commitfest. True, but put that decision entirely in the hands of the CF manager doesn't seem to me to be a workable solution. Half the time we don't even have a CF manager, AFAICT. Now admittedly the opportunity to wield absolute power might attract more interest in the position ;-) but I don't think we want people who are attracted by that. Some suggestions: - for the first 2 weeks of each CF, there should be a *total* moritorium on discussing any features not in the current CF on -hackers. We've tried that in the past, and it's never been adhered to very well, and I think it's folly to assume that we'll get much better at it. The nature of a mailing list is that there's a lot of noise. Even if 95% of the membership knows about and agrees with the restriction, the other 5% will still post about non-CF stuff. - we should have simple rules for the CF manager for kicking patches, as in: * no response from author in 5 days * judged as needing substantial work by reviewer * feature needs spec discussion These rules still seem to me to require a lot of judgment, hence opportunity for argument. What's substantial work? How big a quibble about the spec is big enough to get a patch booted? However, the real criteria don't matter as much as coming up with a set of criteria we're all willing to obey, whatever they are. Ultimately, we're herding cats here. I don't think you're going to get the community to suddenly be willing to march in lockstep instead. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 11:49 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of mié abr 11 12:44:02 -0300 2012: Me neither, but I don't know how far it scales. Having certain people who are defined as, say, doc-only committers will not only make it clear to those people what they're expected to commit, but also clear to everyone else who the people are who might commit any given patch they might write. If we just end up with 50 committers and you have to follow pgsql-hackers to understand who knows what and which people are even still around, it's not going to make anything easier for anyone. Since we're so keen on copying what Linux does, we could just have a MAINTAINERS file. I'm actually not particularly keen on copying what Linux does. It seems that they have a lot of the same problems that we have, only worse. Unless you can get the proper person to take notice of your patch and merge it into his tree, from where it will get merged into somebody else's tree, from where it will eventually get merged into Linus's tree (maybe with one more tree in the path just for good measure), your patch is just going to fall on the floor and die. The lseek contention stuff in Linux 3.2 was submitted in substantially identical form years ago and didn't get merged - mostly, AFAICT, just because nobody cared about it enough. Our process is not perfect, but in recent years we have at least done a fairly good job preventing things from dying of apathy, even if not everybody agrees on which things ultimately should or should not have gotten committed. Small doc changes might be an exception. I used to apply those regularly, but I've had to fall back to making occasional sweeps of my pgsql-docs email box and looking for unresolved issues. Having a few more committers, and specifically people focused on documentation, would, I think, be a a step forward. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On ons, 2012-04-11 at 06:04 -0400, Greg Smith wrote: Compare with: -Submitter suggests doc change -No one has a strong opinion on it, may not be picked up at all -Submitter adds to the next CF -Wait for review -[Possible repost update with reviewer changes] -Ready for committer -Committer takes time away from code review to look at it -Possibly another feedback/review resubmission -Commit final versions I totally get that. Just as a personal view, if people were to send me doc or trivial patches in git-am format, with proper commit message, and Acked or Signed-off etc. lines from recognized contributors, and proper References: mail header linked to the discussion or suggestion message, I could probably commit 20 of those in an hour. Instead, I have to review the entire email thread for discussion, any possible reviews or test runs, extract the patch from the email, mangle it into proper form, apply it, think of a commit message, make sure I register all the right people in the message, re-review the commit, push, reply to email, optionally, log into commit fest, find the patch, click a bunch of times, close it, done -- I think. That takes 15 minutes per patch, and after two patches like that I'm tired. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Ultimately, we're herding cats here. I don't think you're going to get the community to suddenly be willing to march in lockstep instead. If you, Peter, Simon, Robert, Heikki, Magnus, Peter G., Greg, Bruce and Andrew agreed on a calendar-driven, mostly unambiguous process and adhered to that process, then the one or two people who didn't follow along wouldn't matter. Everyone else would follow you. The reason things are chaotic now is that our lead committers do not have consensus and are even inconsistent from CF to CF individually. In other words: the problem is only unsolvable because *you* think it's unsolvable. If you decide the problem is solvable, you already have the means to solve it. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On ons, 2012-04-11 at 12:48 -0400, Tom Lane wrote: However, the real criteria don't matter as much as coming up with a set of criteria we're all willing to obey, whatever they are. Ultimately, we're herding cats here. I don't think you're going to get the community to suddenly be willing to march in lockstep instead. Yeah, I think what's forgotten in this discussion is that we're all volunteers. (Even those who have jobs related to this are still volunteers with respect to the project.) So people will do whatever they want. If you don't let them do what they want, they will do it elsewhere. Now it's fine to encourage a little bit of team spirit and community values here and there, but you can't build your process on making people do what they don't want to do. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 1:39 PM, Joshua Berkus j...@agliodbs.com wrote: Ultimately, we're herding cats here. I don't think you're going to get the community to suddenly be willing to march in lockstep instead. If you, Peter, Simon, Robert, Heikki, Magnus, Peter G., Greg, Bruce and Andrew agreed on a calendar-driven, mostly unambiguous process and adhered to that process, then the one or two people who didn't follow along wouldn't matter. Everyone else would follow you. The reason things are chaotic now is that our lead committers do not have consensus and are even inconsistent from CF to CF individually. In other words: the problem is only unsolvable because *you* think it's unsolvable. If you decide the problem is solvable, you already have the means to solve it. That's a somewhat bizarre list of people. It both includes people who haven't expressed many concerns about our process one way or the other and excludes some who have. At any rate, clearly the problem is exactly that there isn't consensus on this. I would generally say that Tom, Greg Smith, and I are pretty close together on this issue, and Peter G., Simon, and Dimitri are pretty close together on this issue, but with a big gap in between those two groups. I am less clear on how everyone else feels, but I think that saying that all we need is to get consensus among those people is to define the problem, not the solution. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Peter Eisentraut pete...@gmx.net writes: Just as a personal view, if people were to send me doc or trivial patches in git-am format, with proper commit message, and Acked or Signed-off etc. lines from recognized contributors, and proper References: mail header linked to the discussion or suggestion message, I could probably commit 20 of those in an hour. Instead, I have to review the entire email thread for discussion, any possible reviews or test runs, extract the patch from the email, mangle it into proper form, apply it, think of a commit message, make sure I register all the right people in the message, re-review the commit, push, reply to email, optionally, log into commit fest, find the patch, click a bunch of times, close it, done -- I think. That takes 15 minutes per patch, and after two patches like that I'm tired. I hear you ... but, given that the source material is a mailing-list thread, *somebody* has to do all that work to produce an acceptable commit. And if you're just going to commit what that somebody sends you without further review, then you might as well give that person a commit bit, because you're trusting them to get all this right. So I'm not sure how this moves us forward, other than to the obvious observation that it'd be great if we had more qualified committers. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On ons, 2012-04-11 at 14:29 -0400, Tom Lane wrote: I hear you ... but, given that the source material is a mailing-list thread, *somebody* has to do all that work to produce an acceptable commit. And if you're just going to commit what that somebody sends you without further review, then you might as well give that person a commit bit, because you're trusting them to get all this right. I'd still review it, but I'd be able to spend say 3 minutes on review and 30 seconds on committing it, versus 3 minutes on review, 3 minutes on research, and 8 minutes on bookkeeping. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 5:36 PM, Peter Eisentraut pete...@gmx.net wrote: On ons, 2012-04-11 at 14:29 -0400, Tom Lane wrote: I hear you ... but, given that the source material is a mailing-list thread, *somebody* has to do all that work to produce an acceptable commit. And if you're just going to commit what that somebody sends you without further review, then you might as well give that person a commit bit, because you're trusting them to get all this right. I'd still review it, but I'd be able to spend say 3 minutes on review and 30 seconds on committing it, versus 3 minutes on review, 3 minutes on research, and 8 minutes on bookkeeping. Well, I am not averse to figuring out a better workflow, or some better tools. In practice, I think it's going to be hard to reduce the time to review a trivial patch much below 5-10 minutes, which is what it takes me now, because you've got to read the email, download the patch, check that it doesn't break the build, review, commit, and push, and I can't really see any of those steps going away. But that doesn't mean we shouldn't make the attempt, because I've got to admit that the current workflow seems a little cumbersome to me, too. I'm not sure I have a better idea, though. git remotes seem useful for collaborating on topic branches, but I don't think they can really be expected to save much of anything during the final commit process - which is basically all the process there is, when the patch is trivial. Now what would be sort of neat is if we had a way to keep all the versions of patch X plus author and reviewer information, links to reviews and discussion, etc. in some sort of centralized place. The CommitFest app was actually designed to track a lot of this information, but it's obviously not completely succeeding in tracking everything that people care about - it only contains links to patches and not patches themselves; it doesn't have any place to store proposed commit messages; etc. There might be room for improvement there, although getting consensus on what improvement looks like may not be totally straightforward, since I think Tom's ideal process for submitting a patch starts with attaching a file to an email and many other people I think would like to see it start with a pull request. This is not entirely a tools issue, of course, but it's in there somewhere. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Robert Haas robertmh...@gmail.com writes: On Wed, Apr 11, 2012 at 5:36 PM, Peter Eisentraut pete...@gmx.net wrote: I'd still review it, but I'd be able to spend say 3 minutes on review and 30 seconds on committing it, versus 3 minutes on review, 3 minutes on research, and 8 minutes on bookkeeping. Well, I am not averse to figuring out a better workflow, or some better tools. In practice, I think it's going to be hard to reduce the time to review a trivial patch much below 5-10 minutes, which is what it takes me now, because you've got to read the email, download the patch, check that it doesn't break the build, review, commit, and push, and I can't really see any of those steps going away. But that doesn't mean we shouldn't make the attempt, because I've got to admit that the current workflow seems a little cumbersome to me, too. I'm not sure I have a better idea, though. git remotes seem useful for collaborating on topic branches, but I don't think they can really be expected to save much of anything during the final commit process - which is basically all the process there is, when the patch is trivial. Now what would be sort of neat is if we had a way to keep all the versions of patch X plus author and reviewer information, links to reviews and discussion, etc. in some sort of centralized place. The CommitFest app was actually designed to track a lot of this information, but it's obviously not completely succeeding in tracking everything that people care about - it only contains links to patches and not patches themselves; it doesn't have any place to store proposed commit messages; etc. There might be room for improvement there, although getting consensus on what improvement looks like may not be totally straightforward, since I think Tom's ideal process for submitting a patch starts with attaching a file to an email and many other people I think would like to see it start with a pull request. This is not entirely a tools issue, of course, but it's in there somewhere. It strikes me that there are two different scenarios being discussed here, and we'd better be sure we keep them straight: small-to-trivial patches, and complex patches. I think that for the trivial case, what we need is less tooling not more. Entering a patch in the CF app, updating and closing it will add a not-small percentage to the total effort required to deal with a small patch (as Peter already noted, and he wasn't even counting the time to put the patch into CF initially). The only reason to even consider doing that is to make sure the patch doesn't get forgotten. Perhaps we could have some lighter-weight method of tracking such things? At the other end of the scale, I think it's true that the CF app could be more helpful than it is for tracking the state of complex patches. I don't really have any concrete suggestions, other than that I've seen far too many cases where the latest version of a patch was not linked into the CF entry. Somehow we've got to make that more robust. Maybe the answer is to tie things more directly into git workflows, though I'm not sure about details. I am concerned about losing traceability of submissions if all that ever shows up in the list archives is a URL. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Christopher Browne cbbro...@gmail.com wrote: Robert Haas robertmh...@gmail.com wrote: CommitFests are a time for patches that are done or very nearly done to get committed, and a time for other patches to get reviewed if they haven't been already. If we make it clear that the purpose of the CommitFest is to assess whether the patch is committable, rather than to provide an open-ended window for it to become committable, we might do better. Yeah, I think there's pretty good room for a +1 on that. Yeah, +1 for sure. One other sort of mechanical test which I think can and should be applied to patches submitted to the last CF is that if *at the start of the CF* the patch doesn't apply, compile, pass regression tests, and demonstrably provide the functionality claimed for the patch, it should not be a candidate for inclusion in the release. A patch on which the author is continuing to work even in the absence of review should be considered a WIP want feedback submission; it should not be allowed to constitute a placeholder for inclusion in the release. It's one thing if review turns up corner case bugs missed by the author; it's quite another if there is a month or two of solid development left to be done. The CF period is not the time for now I'll get serious about wrapping this up. onlyhalfkiddingPerhaps we should have a concept of feature months -- so that when we look at holding up a release with 20 features for two months so that one more feature can make it in, the cost side of the equation is 40 feature-months, and the benefit is 10 feature-months. (Remember, you can't count the added feature as though it's there for a year before the next release if it holds the release up.)/onlyhalfkidding -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 6 April 2012 01:19, Noah Misch n...@leadboat.com wrote: On Thu, Apr 05, 2012 at 02:34:30PM -0400, Robert Haas wrote: On Thu, Apr 5, 2012 at 2:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: The FK arrays one I'm kind of queasy about. ?It's a cool-sounding idea but I'm not convinced that all the corner-case details have been adequately thought through, and I'm scared of being unable to fix any such bugs in later versions because of backwards compatibility worries. It'd be a lot better to be pushing this in at the start of a devel cycle than the end. I've been feeling that that patch has been suffering from a lack of reviewer attention, which is a real shame, because I think the functionality is indeed really cool. But I haven't looked at it enough to know what kind of shape it's in. As the reviewer, I'm not aware of any unexplored corner cases or problems that ought to preclude commit. That said, it is a large patch; I doubt anyone could pick it up from scratch and commit it with less than a solid day's effort, and 2-3 days might be more likely. In retrospect, I should have suggested splitting the new ON DELETE/ON UPDATE actions into their own patch. That would have nicely slimmed the base patch and also isolated it from the ON DELETE EACH CASCADE judgement call. As a likely user of this feature (not sure if this needs a disclaimer, but my employer offered a small bounty towards the development), I'd only need ON DELETE RESTRICT behaviour, currently, and wouldn't ever need ON UPDATE ... as the referent column would always be a SERIAL. In the meantime, I'm pretty sure the restriction could be handled by a hand-rolled trigger on insert and delete, but the delete one would be a lot slower without some kind of indexing. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 10 April 2012 15:26, Kevin Grittner kevin.gritt...@wicourts.gov wrote: A patch on which the author is continuing to work even in the absence of review should be considered a WIP want feedback submission; it should not be allowed to constitute a placeholder for inclusion in the release. To be fair, I doubt that anyone actually believes that. If they did, they wouldn't have to pay attention very long to receive a rude awakening. onlyhalfkiddingPerhaps we should have a concept of feature months -- so that when we look at holding up a release with 20 features for two months so that one more feature can make it in, the cost side of the equation is 40 feature-months, and the benefit is 10 feature-months. (Remember, you can't count the added feature as though it's there for a year before the next release if it holds the release up.)/onlyhalfkidding I am broadly in favour of assessing the value of features in the same way that a commercial organisation might - the more adoption a feature spurs, the more valuable it is, and the more hesitant we should be to bump it (though other factors are also very important). I take this idea seriously, or at the very least share the mentality of the idea - I'm just not sure that we can formalise it, or that we should. I also think that we should try and reward good will. I think that we generally do so, but an automatic cut-off date seems contrary to that. The law of unintended consequences might see us lower our standards to commit something to meet the deadline, that would otherwise not be immediately committed. We're only human, and it would be foolish to assume that committers don't feel that kind of pressure. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Tue, Apr 10, 2012 at 11:24 AM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 10 April 2012 15:26, Kevin Grittner kevin.gritt...@wicourts.gov wrote: A patch on which the author is continuing to work even in the absence of review should be considered a WIP want feedback submission; it should not be allowed to constitute a placeholder for inclusion in the release. To be fair, I doubt that anyone actually believes that. If they did, they wouldn't have to pay attention very long to receive a rude awakening. Uhm, this has been done, repeatedly. It is not an imaginary problem. I've observed people doing the following anti-social things: 1. Adding patches to the CommitFest after the deadline, sometimes weeks after. The system doesn't prohibit this because there can be legitimate reasons for doing it, such as when a patch submitted on time gets a minor piece of it split out into a separate entry. 2. Adding fake CommitFest entries that point to a non-existing email, and then editing them later to point to the real patch. 3. Posting a patch in time for the CommitFest deadline that is not even code-complete and then continuing to hack on it vigorously throughout the CommitFest. Or, a variant: it's code completed, but not debugged. Both command triggers and foreign key locks fell into this category, AFAICT. When these things are pointed out to the people who are doing them, the response is often either (a) this feature is so important we're all going to die if it's not in the release how can you even think about bouncing it or (b) I'm not really still hacking on it these are all just minor changes. It's surprisingly easy to hoodwink even experienced contributors into thinking that your patch is really, really almost done, honest, it just needs a couple more tweaks when in fact it's nowhere close. I try not to attribute to bad faith what can be explained by incurable optimism, so maybe we just have a lot of incurable optimism. But it's doing nobody any good. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 10 April 2012 16:51, Robert Haas robertmh...@gmail.com wrote: When these things are pointed out to the people who are doing them, the response is often either (a) this feature is so important we're all going to die if it's not in the release how can you even think about bouncing it or (b) I'm not really still hacking on it these are all just minor changes. It's surprisingly easy to hoodwink even experienced contributors into thinking that your patch is really, really almost done, honest, it just needs a couple more tweaks when in fact it's nowhere close. I try not to attribute to bad faith what can be explained by incurable optimism, so maybe we just have a lot of incurable optimism. But it's doing nobody any good. I think that you may be missing the greater point here. The people that do this are kind of like the defectors in prisoner's dilemma - at a certain point, some people cannot resist the temptation to push their own patch forward at the expense of others by asserting dubiously that it's ready-for-committer, or maybe they really do incorrectly believe it to be so, or maybe, unlike you, they understand that term to mean I've done as much as I can, as has my reviewer, or whatever. To play devil's advocate, that might be an anti-social act, but at a certain point, who wants to be the last honest sap? Besides, isn't everyone's crime no crime at all? ISTM that this is symptomatic of the wider problem of a dire shortage of committer resources. 100% of my non-doc patches so far have been committed by 3 people. I would really like to see us figure out a way of making more hackers committers, perhaps subject to certain conditions that don't currently exist for committers. You might find that given commit bits, some people will take their responsibilities as a reviewer far more seriously. Maybe you don't think that any of the likely candidates are quite ready for that responsibility, but you must admit that it's a serious problem. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Kevin Grittner kevin.gritt...@wicourts.gov writes: One other sort of mechanical test which I think can and should be applied to patches submitted to the last CF is that if *at the start of the CF* the patch doesn't apply, compile, pass regression tests, and demonstrably provide the functionality claimed for the patch, it should not be a candidate for inclusion in the release. I would not be in favor of inflexible application of such a rule. For instance, if a patch had gotten broken by a conflict with some other patch applied the day before the CF starts, it would be unfair to not give the patch author a reasonable amount of time to rebase. And such conflicts occurring after the CF starts are hardly unusual either. A patch on which the author is continuing to work even in the absence of review should be considered a WIP want feedback submission; it should not be allowed to constitute a placeholder for inclusion in the release. It's one thing if review turns up corner case bugs missed by the author; it's quite another if there is a month or two of solid development left to be done. The CF period is not the time for now I'll get serious about wrapping this up. Agreed here, though. Chris Browne mentioned upthread that we really need a somewhat different process for WIP patches as opposed to those that are thought to be committable or nearly so. I don't know if we should institute his idea of a separate series of HackFest events, but at the very least we should try harder to draw a distinction between WIP and finished patches. They need different sorts of reviewing. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Robert Haas robertmh...@gmail.com writes: ... It's surprisingly easy to hoodwink even experienced contributors into thinking that your patch is really, really almost done, honest, it just needs a couple more tweaks when in fact it's nowhere close. I try not to attribute to bad faith what can be explained by incurable optimism, so maybe we just have a lot of incurable optimism. But it's doing nobody any good. The first, and possibly most significant, observation in Brooks' classic _The Mythical Man-Month_ is: All programmers are optimists. I don't think you're going to get rid of the incurable-optimism problem. Rather, we'd better design around it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: One other sort of mechanical test which I think can and should be applied to patches submitted to the last CF is that if *at the start of the CF* the patch doesn't apply, compile, pass regression tests, and demonstrably provide the functionality claimed for the patch, it should not be a candidate for inclusion in the release. I would not be in favor of inflexible application of such a rule. For instance, if a patch had gotten broken by a conflict with some other patch applied the day before the CF starts, it would be unfair to not give the patch author a reasonable amount of time to rebase. And such conflicts occurring after the CF starts are hardly unusual either. I didn't mean to exclude rebasing because of conflicts with recent commits, so perhaps mechanical was overstating it. But maybe not -- perhaps each patch submission should state which commit it was last confirmed to compile and work with, and if there are problems against HEAD that could be confirmed before asking for the rebase. That wouldn't be too much extra work for the initial reviewer, and it would help establish objective criteria for categorizing whether a patch should be treated as WIP. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 04/09/2012 11:12 PM, Christopher Browne wrote: It seems as though we need to have a bad guy that will say, that sure isn't ready to COMMIT, so we'd better step back from imagining that it ought to be completed as part of this COMMITfest. There's no reward for anyone in the PostgreSQL community to be a bad guy. If you're too aggressive about it, submitters get mad; too loose, and you get both committers and people worried about the release schedule mad. And the community is tight enough that the person you tick off today might be someone you have to work with next week. Having sat in this particular seat several times now, I'd say the role needed here is more mediator than pointy-haired boss. When I write bad news e-mail to submitters, I try to make the tone more about clarifying what was learned and what is needed to improve things for a next round of submissions. It's not easy to adopt a writing tone for that sort of message while not coming off as insulting to someone. Getting a feature punted forward is easier to take if a submitter leaves with a better roadmap and idea what standards they have to meet. On bigger features in particular, that sometimes requires feedback from a committer earlier in the process, even if they haven't reached Ready for Committer via a reviewer yet. My comment upthread about nailing down the committer for big features earlier than smaller ones was along these same lines. I wonder if we're starting to have enough data to establish meaningful statistics on feedback. I had Robert send me a dump of the data that's in the CF app the other day. I'm hoping to do some useful data mining on it before PGCon. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Tue, Apr 10, 2012 at 12:28 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: I think that you may be missing the greater point here. The people that do this are kind of like the defectors in prisoner's dilemma - at a certain point, some people cannot resist the temptation to push their own patch forward at the expense of others ... The question is - what exactly are we supposed to do about that? I think that when prisoner A rats out prisoner B and prisoner B spends 10 years in Leavenworth, prisoner B is unlikely to be very happy with the situation, even though he's surely not confused about *why* prisoner A did it. So here. If we accept your argument that some people simply cannot help themselves, then the only solution is to make it cease to be a prisoner's dilemma, and that can only be done by changing the incentives, which presumably means handing down punishments to people who push their own patches forward at the expense of others. Unless we care to choose a benevolent dictator, I don't see any way to accomplish that. It's feasible to think that we might be able to streamline the process of booting patches that are not close to committable at the start of a CommitFest, and especially at the start of the final CommitFest. For example, limiting patches to a small number of days in the Waiting on Author state would help a great deal. But the more general problem of people arguing that *their* patch is the special one without which the earth will cease to revolve about its axis is more difficult to solve, or that it's ready when it's really not, is more difficult to solve. How would you propose we deal with that problem? ISTM that this is symptomatic of the wider problem of a dire shortage of committer resources. 100% of my non-doc patches so far have been committed by 3 people. I would really like to see us figure out a way of making more hackers committers, perhaps subject to certain conditions that don't currently exist for committers. You might find that given commit bits, some people will take their responsibilities as a reviewer far more seriously. Maybe you don't think that any of the likely candidates are quite ready for that responsibility, but you must admit that it's a serious problem. I don't agree with that. I think that there are a few people who don't now have commit bits who should be given them - in particular, Fujii Masao and Kevin Grittner, both of whom have been doing consistently excellent work for several years. But giving people a commit bit in the hopes that they will do better reviews seems completely backwards to me: we should instead give commit bits to people who have *already* demonstrated that they can be trusted to do good reviews and exercise good judgement, and no one else. The fact is that we have no shortage of committers - there are 19 people who have access to push code into our master git repository. A handful of those people have basically completely left the project and their commit rights should probably be revoked on that basis; most of them are still involved in one way or another but just not able to devote a lot of time to reviewing other people's code. The problem is even more acute for large patches, which only a handful of people are qualified to review, and which also take a lot of wall clock time to review thoroughly. But that's not a problem that's going to go away because we make more committers. Giving more people the ability to commit stuff will neither force them to devote time to it nor make them qualified to do it if they aren't already. Every time someone's favorite patch gets rejected, there is an outcry of - the standards for commit are too high! But this overlooks the fact that there are some people who regularly meet them. A patch from Fujii Masao, Kevin, or Noah is about ten times more likely to be applied without comment than one from the average submitter. That's not because I like them (although I have to admit to liking Kevin quite a lot; I have met Fujii Masao at most briefly and Noah not at all), or because I necessarily care about their patches more than anyone else's; it's because they do really good work. Over time, such people tend to become committers, and then everyone complains that the committers have high standards. Well, yes. They are committers *because* they have high standards, and that is exactly as it should be. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Tue, Apr 10, 2012 at 1:27 PM, Greg Smith g...@2ndquadrant.com wrote: There's no reward for anyone in the PostgreSQL community to be a bad guy. If you're too aggressive about it, submitters get mad; too loose, and you get both committers and people worried about the release schedule mad. And the community is tight enough that the person you tick off today might be someone you have to work with next week. Yep. Having sat in this particular seat several times now, I'd say the role needed here is more mediator than pointy-haired boss. When I write bad news e-mail to submitters, I try to make the tone more about clarifying what was learned and what is needed to improve things for a next round of submissions. It's not easy to adopt a writing tone for that sort of message while not coming off as insulting to someone. Agreed. I used to be better at this, but the increasing volume of patches that get reviewed by no one else has forced me to curtail the amount of time I spend on each one, and that is causing my natural bluntness to come to the fore. Unfortunately. I also think that people were more receptive to my reviews before I got a commit bit. Back then, I was the guy who was telling you what you were going to have to fix so Tom didn't boot your patch. Now, I'm the guy who is threatening to boot your patch if you don't fix what I want fixed. It comes off differently, even if the content is identical. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 10 April 2012 18:28, Robert Haas robertmh...@gmail.com wrote: If we accept your argument that some people simply cannot help themselves, then the only solution is to make it cease to be a prisoner's dilemma, and that can only be done by changing the incentives, which presumably means handing down punishments to people who push their own patches forward at the expense of others. Unless we care to choose a benevolent dictator, I don't see any way to accomplish that. Well, I was really pointing out that people are somewhat forced into a corner by the current state of affairs, because committers are not typically able to look at anything in sufficient detail that isn't ready for committer, particularly as we approach crunch-time - their time is simply too precious. By not marking the patch ready for committer, they are basically asking for their patch to be passed over, and they may be incapable of bridging the chasm between what really is their best effort, and what'd you'd consider to be the ready-for-committer gold standard. Some people cannot exclusively dedicate their time to their patch, or lack sufficient experience to meet that standard. It's feasible to think that we might be able to streamline the process of booting patches that are not close to committable at the start of a CommitFest, and especially at the start of the final CommitFest. For example, limiting patches to a small number of days in the Waiting on Author state would help a great deal. But the more general problem of people arguing that *their* patch is the special one without which the earth will cease to revolve about its axis is more difficult to solve, or that it's ready when it's really not, is more difficult to solve. How would you propose we deal with that problem? As I've already said, I think that needs to be decided impartially, ideally by people who are removed from the engineering process. I don't mean that we'd get a marketing person to make those decisions - far from it. I just mean that some separation of powers can be a good thing in some circumstances. I don't agree with that. I think that there are a few people who don't now have commit bits who should be given them - in particular, Fujii Masao and Kevin Grittner, both of whom have been doing consistently excellent work for several years. I agree with you about both individuals. I hope that this happens sooner rather than later. Giving more people the ability to commit stuff will neither force them to devote time to it nor make them qualified to do it if they aren't already. One major component of being qualified, is, of course, knowing what you don't know, and the risk of being left with egg on your face turns out to be a pretty effective way of preventing new committers from being too eager. Giving more people bits has a cost: in general, I'd expect it to result in a higher bug-to-line ratio when code is committed. However, not doing so has an opportunity cost: less code is committed, which may, on balance, result in an inferior release than what we could have had. Maybe you think that we have the balance perfectly right, and you are of course perfectly entitled to that view, as well as being perfectly entitled to having your opinion more heavily weighed than mine, but I'd like to see a dialogue about it at some point. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Peter Geoghegan pe...@2ndquadrant.com writes: On 10 April 2012 18:28, Robert Haas robertmh...@gmail.com wrote: I don't agree with that. I think that there are a few people who don't now have commit bits who should be given them - in particular, Fujii Masao and Kevin Grittner, both of whom have been doing consistently excellent work for several years. I agree with you about both individuals. I hope that this happens sooner rather than later. FYI, the core committee traditionally has a discussion about whom to appoint as new committers at the end of each release cycle. I'm sure we'll be thinking about these names this time. Giving more people the ability to commit stuff will neither force them to devote time to it nor make them qualified to do it if they aren't already. One major component of being qualified, is, of course, knowing what you don't know, and the risk of being left with egg on your face turns out to be a pretty effective way of preventing new committers from being too eager. Giving more people bits has a cost: in general, I'd expect it to result in a higher bug-to-line ratio when code is committed. However, not doing so has an opportunity cost: less code is committed, which may, on balance, result in an inferior release than what we could have had. Maybe you think that we have the balance perfectly right, and you are of course perfectly entitled to that view, as well as being perfectly entitled to having your opinion more heavily weighed than mine, but I'd like to see a dialogue about it at some point. We've done pretty well over the past fifteen years by being chary in handing out commit bits. I don't particularly want to change that policy. Obviously we do need a steady supply of new blood, since people do leave the project, but relaxing our standards doesn't seem like the way to get it. The impression I have is that we have a pretty good and even increasing supply of new interested people, so letting them acquire experience with the code base and eventually mature into qualified committers doesn't look like a dead-end strategy from here. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Tue, Apr 10, 2012 at 2:49 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: Well, I was really pointing out that people are somewhat forced into a corner by the current state of affairs, because committers are not typically able to look at anything in sufficient detail that isn't ready for committer, particularly as we approach crunch-time - their time is simply too precious. By not marking the patch ready for committer, they are basically asking for their patch to be passed over, and they may be incapable of bridging the chasm between what really is their best effort, and what'd you'd consider to be the ready-for-committer gold standard. Some people cannot exclusively dedicate their time to their patch, or lack sufficient experience to meet that standard. I think that's partly true and partly false. I actually spend a lot of time looking at patches that are not marked Ready for Committer, on the theory that I'd like to move things along that haven't been formally tagged with that designation if they are nevertheless ready to go, whereas Tom I think actively avoids it, on the theory that no one else will volunteer to review if the committers just do everything. These positions are in tension but neither seems to me to be without merit. I do understand that not everyone is going to write code that meets our standards for commit, and I have rewritten my share of patches - sometimes, even quite large patches - to try to bring them up to that level. I had more time to do that last year than I have this year, because this year I've been focused on performance stuff. But, on the flip side, I think we still did a pretty good job handling pretty much everything submitted before November. The stuff that ran into trouble was the stuff that came in at the end, which in many cases was not only late to the table but overly ambitious in its scope. I think the question should be not so much why didn't those big patches get committed? as why does anyone think that they have a right to be upset that they didn't?. They were given, basically, two to three extra months to become committable, and still fell short. And I'm still very willing to devote more time to them to make them committable *even though they are not my projects*, but I am *not* willing to do it while the features I worked hard on to get ready early sit there and don't get released. That might be a reasonable expectation if the original patches were submitted in May and I had blithely ignored them in favor of my own work all year, but that ain't what happened. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Mon, 2012-04-09 at 23:12 -0400, Christopher Browne wrote: But there is also a flip side to that, namely that if we do so, there ought to be some aspect to the process to help guide those items that *aren't* particularly close to being committable. I have benefited immensely from review of my WIP patches, and a lot of the serious review tends to happen during commitfests. This is most important for features with a significant user interface, where it's harder to guess what people will want. My current strategy is to submit WIP-marked patches during a commitfest. I agree that we should continue to have a mechanism to review patches that aren't ready for commit, though I'm fine if we change it. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 04/10/2012 01:33 PM, Robert Haas wrote: I also think that people were more receptive to my reviews before I got a commit bit. That's not true; many people were just as annoyed at you back then. (Robert knows I'm kidding. I hope.) -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Tue, Apr 10, 2012 at 11:53:23AM -0500, Kevin Grittner wrote: Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: One other sort of mechanical test which I think can and should be applied to patches submitted to the last CF is that if *at the start of the CF* the patch doesn't apply, compile, pass regression tests, and demonstrably provide the functionality claimed for the patch, it should not be a candidate for inclusion in the release. I would not be in favor of inflexible application of such a rule. For instance, if a patch had gotten broken by a conflict with some other patch applied the day before the CF starts, it would be unfair to not give the patch author a reasonable amount of time to rebase. And such conflicts occurring after the CF starts are hardly unusual either. I didn't mean to exclude rebasing because of conflicts with recent commits, so perhaps mechanical was overstating it. But maybe not -- perhaps each patch submission should state which commit it was last confirmed to compile and work with, and if there are problems against HEAD that could be confirmed before asking for the rebase. That wouldn't be too much extra work for the initial reviewer, and it would help establish objective criteria for categorizing whether a patch should be treated as WIP. Of the patches I've reviewed that fall into one the problem categories Robert outlined, all applied cleanly and passed regression tests. On the flip side, I have submitted at least two patches that failed regression tests for the reviewer due to isolated, easily-fixed blunders. Consequently, I'm not hopeful about these checks as coarse indicators of patch readiness. I would certainly like an objective test for assigning patches to those categories, but I don't have a better idea for such a test. nm -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 04/10/2012 01:28 PM, Robert Haas wrote: The fact is that we have no shortage of committers - there are 19 people who have access to push code into our master git repository. A handful of those people have basically completely left the project and their commit rights should probably be revoked on that basis; most of them are still involved in one way or another but just not able to devote a lot of time to reviewing other people's code. Let's use realistic numbers here: I count 7 people who regularly review and commit other people's code in a variety of areas. There are also several subject matter experts who commit things in a relatively narrow area. But most bigger patches are going to hit a bottleneck whose capacity could be measured in 3 bits. I would actually be happy to have more of the people whose commits were expected to be in targeted areas. It's not like anyone who commits outside of their scope of expertise is going to survive doing that for long before getting publicly shamed and/or booted. The pressure to not screw up is so high in this project, I suspect concerns over making a mistake is behind some people's reticence to commit other people's work. Committing sketchy code that blows up later is still going to haunt its committer, regardless of the original author. Giving more people the ability to commit stuff will neither force them to devote time to it nor make them qualified to do it if they aren't already. There are a couple of directions from which I don't completely agree with this. To use a personal example I don't think is unique, I would set aside more time to hack on the documentation if I didn't have to bug one of the existing committers each time I wanted to work on something there. It really feels like I'm wasting the time of someone who could be doing more difficult things every time I submit a doc patch. I'd merrily write more of those and consume things like the never ending stream of corrections from Thom Browne if I could turn that into a larger part of my job. I don't do more of that now because it's very unsatisfying work unless you can do the whole thing yourself. Knowing everything is going to pass through another person regardless removes some of the incentive to polish something until it's perfect for submitters. As for broader concerns about whether people will alter their quality of work based on being able to commit, I'd suggest turning a look at yourself. Your quality of work was high before it was a primary job goal, but it's surely gotten better now that it is, right? Seems that way to me at least. But it's really hard to get funding to work full-time on this project unless someone can commit their work. There are plenty of people contributing here who rummage up enough part-time hours to develop the occasional feature, but not quite enough to make things perfect even by their own standards. And not being recognized for your work on the project can be a self-fulfilling prophecy. The main reason I worry about this is because of a very real chicken/egg problem here that I keep banging into. Since the commit standards for so many other open-source projects are low, there are a non trivial number of business people who assume !committer == ![trusted|competent]. That makes having such a limited number of people who can commit both a PR issue (this project must not be very important if there are only 19 committers) and one limiting sponsorship (I'm not going to pay someone to work on this feature who's been working on it for years but isn't even a committer). There are a significant number of companies who are willing to sponsor committers to open-source projects; there are almost none who will sponsor reviewers or contributors of any stature unless they're already deep into the PostgreSQL community. That's one of the many reasons it's easier for a committer to attract funding for core PostgreSQL work, be it in the form of a full-time job or project-oriented funding. The corresponding flip side to that is that the small number of committers is limiting the scope of funding the project can accumulate. I'm somewhat oddly pleased at how the overflow of incoming submissions for 9.2 has raised questions around not having enough active committers. I hope decisions about adding more recognizes that distributing that power really does influence the ability of people to contribute, on average in a positive way. All I see coming for next year is a dramatic increase in this class of problem. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Tue, Apr 10, 2012 at 8:43 PM, Greg Smith g...@2ndquadrant.com wrote: To use a personal example I don't think is unique, I would set aside more time to hack on the documentation if I didn't have to bug one of the existing committers each time I wanted to work on something there. It really feels like I'm wasting the time of someone who could be doing more difficult things every time I submit a doc patch. I'd merrily write more of those and consume things like the never ending stream of corrections from Thom Browne if I could turn that into a larger part of my job. I don't do more of that now because it's very unsatisfying work unless you can do the whole thing yourself. Knowing everything is going to pass through another person regardless removes some of the incentive to polish something until it's perfect for submitters. I wouldn't object to creating some doc-only committers. OTOH, I would object to anyone making non-trivial documentation enhancements without posting their patches first and having a second person look it over, so how much difference is there, really? As for broader concerns about whether people will alter their quality of work based on being able to commit, I'd suggest turning a look at yourself. Your quality of work was high before it was a primary job goal, but it's surely gotten better now that it is, right? Seems that way to me at least. I think my quantity of work has gone up. I'm not sure the quality is much different, although perhaps I've gotten a bit sharper with practice. I'm somewhat oddly pleased at how the overflow of incoming submissions for 9.2 has raised questions around not having enough active committers. I hope decisions about adding more recognizes that distributing that power really does influence the ability of people to contribute, on average in a positive way. All I see coming for next year is a dramatic increase in this class of problem. My perception of what's going on here is dramatically different from yours. I don't think there was any overflow of submissions for 9.2. For the most part I would describe it as a slow and boring release cycle, with the usual spike in half-baked submissions right near the end, except this release they were less baked than usual, which is why most of them didn't go in. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Tue, Apr 10, 2012 at 10:28 AM, Robert Haas robertmh...@gmail.com wrote: It's feasible to think that we might be able to streamline the process of booting patches that are not close to committable at the start of a CommitFest, and especially at the start of the final CommitFest. I'd usually consider booting a patch to be the domain of committers (or the commit fest manager), not non-committing reviewers. Could enough patches get enough attention from committers early enough in the process to make early booting happen? For example, limiting patches to a small number of days in the Waiting on Author state would help a great deal. Could it be something like a limit on waiting on author days, but only days that are after 2 weeks after the submission deadline count (and days later than 3 weeks after the submission deadline count twice). That would encourage people to submit before the deadline, rather than on the deadline. Cheers, Jeff -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wednesday, April 11, 2012 03:14:41 AM Robert Haas wrote: I'm somewhat oddly pleased at how the overflow of incoming submissions for 9.2 has raised questions around not having enough active committers. I hope decisions about adding more recognizes that distributing that power really does influence the ability of people to contribute, on average in a positive way. All I see coming for next year is a dramatic increase in this class of problem. My perception of what's going on here is dramatically different from yours. I don't think there was any overflow of submissions for 9.2. For the most part I would describe it as a slow and boring release cycle, with the usual spike in half-baked submissions right near the end, except this release they were less baked than usual, which is why most of them didn't go in. They might have been half-baked. But several of those didn't get design-level review for several weeks which makes it rather hard to fully bake them in time... Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Andres Freund and...@anarazel.de writes: On Wednesday, April 11, 2012 03:14:41 AM Robert Haas wrote: My perception of what's going on here is dramatically different from yours. I don't think there was any overflow of submissions for 9.2. For the most part I would describe it as a slow and boring release cycle, with the usual spike in half-baked submissions right near the end, except this release they were less baked than usual, which is why most of them didn't go in. They might have been half-baked. But several of those didn't get design-level review for several weeks which makes it rather hard to fully bake them in time... But if they didn't already have design-level review, that means they were not seen in any previous CF, which means they were not following the expectation that nontrivial patches should be submitted earlier than the last CF. I think the key point here is that people have to expect that it's going to take more than one round of review to land most nontrivial patches. And we have to discourage the expectation that that can happen within the last CF of a release cycle. If anything, the last CF has to be tighter not looser than others on what we will accept, because there is no time to recover if something proves wrong with a patch after a month or three. I keep coming back to the thought that we need more and shorter CFs, and/or ReviewFests that are meant to help push WIP patches further down the track. We need to make it easier to get those early reviews done while there's still development time left. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 11 April 2012 02:14, Robert Haas robertmh...@gmail.com wrote: My perception of what's going on here is dramatically different from yours. I don't think there was any overflow of submissions for 9.2. That is just not true. See the attached graph (couldn't produce one with better resolution at short notice) - I've just eyeballed the graph, but it looks like an upward trend to me. I think I submitted my first patch in late April 2011. Now that Andres has joined 2ndQuadrant, it seems likely that his output will increase by a large amount too. I think that Noah began contributing less than 2 years ago. It certainly seems to me like we're getting more submissions that are more complex...this image was generated by the tool gitstat, and as such does not include figures for the patches that we were unable to commit, though there also appears to be an upward trend there. For the most part I would describe it as a slow and boring release cycle, with the usual spike in half-baked submissions right near the end, except this release they were less baked than usual, which is why most of them didn't go in. That is uncalled for. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services attachment: lines_of_code.png -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
