Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2015-06-16 Thread Noah Misch
On Thu, Dec 11, 2014 at 10:24:20AM -0800, Jeff Janes wrote:
 On Thu, Dec 11, 2014 at 8:03 AM, Tom Lane t...@sss.pgh.pa.us wrote:
  2. The amount of pre-release testing we get from people outside the
  hard-core development crowd seems to be continuing to decrease.
  We were fortunate that somebody found the JSONB issue before it was
  too late to do anything about it.

 We are not particularly inviting of feedback for whatever testing has been
 done.
 
 The definitive guide seems to be
 https://wiki.postgresql.org/wiki/HowToBetaTest, and says:
 
 You can report tests by email. You can subscribe to any PostgreSQL mailing
 list from the subscription form http://www.postgresql.org/community/lists/
 .
 
- pgsql-bugs: this is the preferred mailing list if you think you have
found a bug in the beta. You can also use the Bug Reporting Form
http://www.postgresql.org/support/submitbug/.
- pgsql-hackers: bugs, questions, and successful test reports are
welcome here if you are already subscribed to pgsql-hackers. Note that
pgsql-hackers is a high-traffic mailing list with a lot of development
discussion.
 
 
 =
 
 So if you find a bug, you can report it on the bug reporting form (which
 doesn't have a drop-down entry for 9.4RC1).

Let's get 9.5 alpha/beta/rc releases into that drop-down as we release them.

 If you have positive results rather than negative ones (or even complaints
 that are not actually bugs), you can subscribe to a mailing list which
 generates a lot of traffic which is probably over your head and not
 interesting to you.

Feel welcome to revise that part.  Don't messages from non-subscribed people
make it to the list after manual moderation?  Testers might want to create a
no-delivery subscription to avoid moderation delay, but the decision to
receive all -hackers mail is separate.

 Does the core team keep a mental list of items they want to see tested by
 the public, and they will spend their own time testing those things
 themselves if they don't hear back on some positive tests for them?

Not sure about the core team.  I myself would test essentially the same things
during beta regardless of what end users report having tested, because end
users will pick different test scenarios for the same features.

 If we find reports of public testing that yields good results (or at least
 no bugs) to be useful, we should be more clear on how to go about doing
 it.  But are positive reports useful?  If I report a bug, I can write down
 the steps to reproduce it, and then follow my own instructions to make sure
 it does actually reproduce it.  If I find no bugs, it is just I did a
 bunch of random stuff and nothing bad happened, that I noticed.

Positive reports have potential to be useful.  In particular, mention the new
features you took action to try.  Areas like BRIN, pg_rewind, foreign tables,
event triggers, CREATE POLICY, INSERT ... ON CONFLICT, and GROUPING SETS are
either completely new or have new sub-features.  If nothing else, we can CC
reporters when considering changes to features they reported upon.  Other
analysis would become attractive given a larger corpus of positive reports.

Thanks,
nm


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-16 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote:
 2. It's not clear that we're going to have a particularly-impressive
 list of major features for 9.5.  So far we've got RLS and BRIN. I
 expect that GROUPING SETS is far enough along that it should be
 possible to get it in before development ends, and there are a few
 performance patches pending (Andres's lwlock scalability patches,
 Rahila's work on compressing full-page writes) that I think will
 probably make the grade.  But after that it seems to me that it gets
 pretty thin on the ground.  

When it comes to a list of major features for 9.5, it'd be pretty
terrible, imv, if we manage to screw up and not get UPSERT taken care
of (again..).  BDR will be great to have too, but we lose out far more
often for lack of what those outside the community perceive as a simple
and obvious feature that nearly every other system they deal with has.

 Now, against all that, if we don't get back on our usual release
 schedule then (a) it will look like we're losing momentum, which I'm
 actually afraid may be true rather than merely a perception, and (b)
 people whose stuff did get in will have to wait longer to see it
 released.  So, I'm not sure waiting is any better.  But there are
 certainly some things not to like about where we are.

I agree with both of these.  It doesn't help that we have non-committers
working on major patches for years and aren't able to see the fruits of
their labors.  I'm as much at fault for that happening at times as
anyone and I don't have any silver bullets but I certainly don't like
it.

Thanks,

Stephen


signature.asc
Description: Digital signature


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Peter Geoghegan
On Wed, Dec 10, 2014 at 10:18 PM, Josh Berkus j...@agliodbs.com wrote:

 So far, I haven't seen any features for 9.5 which would delay a more
 timely release the way we did for 9.4.  Anybody know of a bombshell
 someone's going to drop on us for CF5?


I had wondered about that myself. What about jsquery? Is that something
that is planned for submission some time in the current cycle?

FWIW, I strongly doubt that I'll find time to work on anything like that
for 9.5.

-- 
Peter Geoghegan


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Robert Haas
On Thu, Dec 11, 2014 at 12:35 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 Quite.  So, here's a new thread.

 MHO is that, although 9.4 has slipped more than any of us would like,
 9.5 development launched right on time in August.  So I don't see a
 good reason to postpone 9.5 release just because 9.4 has slipped.
 I think we should stick to the schedule agreed to in Ottawa last spring.

 Comments?

I'm fine with that, but in the spirit of playing the devil's advocate:

1. At the development meeting, Simon argued for the 5CF schedule for
this release, with CF5 not starting until February, as a way of making
sure that there was time after the release of 9.4 to get feedback from
actual users in time to do something about it for 9.5.  If anything,
we're going to end up being significantly worse off in that regard
than we would have been, because we're releasing in late December
instead of early September; an extra month of time to get patches in
does not make up for a release that was delayed nearly three months.

2. It's not clear that we're going to have a particularly-impressive
list of major features for 9.5.  So far we've got RLS and BRIN. I
expect that GROUPING SETS is far enough along that it should be
possible to get it in before development ends, and there are a few
performance patches pending (Andres's lwlock scalability patches,
Rahila's work on compressing full-page writes) that I think will
probably make the grade.  But after that it seems to me that it gets
pretty thin on the ground.  Are we going to bill commit timestamp
tracking - with replication node ID tracking as the real goal, despite
the name - as a major feature, or DDL deparsing if that goes in, as
major features?  As useful as they may be for BDR, they don't strike
me as things we can publicize as major features independent of BDR.
And it's getting awfully late for any other major work that people are
thinking of to start showing up.

Now, against all that, if we don't get back on our usual release
schedule then (a) it will look like we're losing momentum, which I'm
actually afraid may be true rather than merely a perception, and (b)
people whose stuff did get in will have to wait longer to see it
released.  So, I'm not sure waiting is any better.  But there are
certainly some things not to like about where we are.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Robert Haas
On Thu, Dec 11, 2014 at 1:18 AM, Josh Berkus j...@agliodbs.com wrote:
 While there were technical
 issues, 9.4 dragged a considerable amount because most people were
 ignoring it in favor of 9.5 development.

I think 9.4 dragged almost entirely because of one issue: the
compressibility of JSONB.  And it became pretty clear early on in that
discussion that it was not going to be resolved until Tom signed off
on some proposed fix.  Which I think points to the issue Bruce
mentioned about how busy many of our key contributors are.  I could
have easily spent four times as much time doing reviewing and
committing over the last few months, but I'm not willing to work an 80
or 100 hour week to make that happen.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Thu, Dec 11, 2014 at 1:18 AM, Josh Berkus j...@agliodbs.com wrote:
 While there were technical
 issues, 9.4 dragged a considerable amount because most people were
 ignoring it in favor of 9.5 development.

 I think 9.4 dragged almost entirely because of one issue: the
 compressibility of JSONB.

Meh.  While we certainly weren't very speedy about resolving that,
I don't think that issue deserves all or even most of the blame.
I agree with Josh: the problem really was that people were not focusing
on getting 9.4 tested and releasable.  One way in which that lack of
focus manifested was not having any urgency about resolving JSONB ...
but it struck me as a systemic problem and not that specific issue's
fault.

I'd finger two underlying issues here:

1. As Bruce points out in a nearby thread, we've been in commitfest mode
more or less continuously since August.  That inherently sucks away
developer mindshare from testing already-committed stuff.

2. The amount of pre-release testing we get from people outside the
hard-core development crowd seems to be continuing to decrease.
We were fortunate that somebody found the JSONB issue before it was
too late to do anything about it.  Personally, I'm very worried that
there are other such bugs in 9.4.  But I've given up hoping that any
more testing will happen until we put out something that calls itself
9.4.0, which is why I voted to release in the core discussion about it.

I don't know what to do about either of those things, but I predict
future release cycles will be just as bad unless we can fix them.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Bruce Momjian
On Thu, Dec 11, 2014 at 10:37:32AM -0500, Robert Haas wrote:
 2. It's not clear that we're going to have a particularly-impressive
 list of major features for 9.5.  So far we've got RLS and BRIN. I
 expect that GROUPING SETS is far enough along that it should be
 possible to get it in before development ends, and there are a few
 performance patches pending (Andres's lwlock scalability patches,
 Rahila's work on compressing full-page writes) that I think will
 probably make the grade.  But after that it seems to me that it gets
 pretty thin on the ground.  Are we going to bill commit timestamp
 tracking - with replication node ID tracking as the real goal, despite
 the name - as a major feature, or DDL deparsing if that goes in, as
 major features?  As useful as they may be for BDR, they don't strike
 me as things we can publicize as major features independent of BDR.
 And it's getting awfully late for any other major work that people are
 thinking of to start showing up.

How bad is the 9.5 feature list going to be compared to the 9.4 one that
had JSONB, but also a lot of infrastructure additions.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + Everyone has their own god. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Tom Lane
Bruce Momjian br...@momjian.us writes:
 On Thu, Dec 11, 2014 at 10:37:32AM -0500, Robert Haas wrote:
 2. It's not clear that we're going to have a particularly-impressive
 list of major features for 9.5.

 How bad is the 9.5 feature list going to be compared to the 9.4 one that
 had JSONB, but also a lot of infrastructure additions.

Well, whatever the list ends up being, let's wait until we have some
more features isn't a tenable scheduling policy.  We learned years ago
the folly of delaying a release until not-quite-ready feature X was ready.
Are we going to delay 9.5 until not-even-proposed-yet features are ready?

More abstractly, there's a lot of value in having a predictable release
schedule.  That's going to mean that some release cycles are thin on
user-visible features, even if just as much work went into them.  It's
the nature of the game.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Robert Haas
On Thu, Dec 11, 2014 at 11:03 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 I think 9.4 dragged almost entirely because of one issue: the
 compressibility of JSONB.

 Meh.  While we certainly weren't very speedy about resolving that,
 I don't think that issue deserves all or even most of the blame.
 I agree with Josh: the problem really was that people were not focusing
 on getting 9.4 tested and releasable.  One way in which that lack of
 focus manifested was not having any urgency about resolving JSONB ...
 but it struck me as a systemic problem and not that specific issue's
 fault.

 I'd finger two underlying issues here:

 1. As Bruce points out in a nearby thread, we've been in commitfest mode
 more or less continuously since August.  That inherently sucks away
 developer mindshare from testing already-committed stuff.

The problem is that, on the one hand, we have a number of serious
problems with things that got committed and turned out to have
problems - the multixact stuff, and JSONB, in particular - and on the
other hand, we are lacking in adequate committer bandwidth to properly
handle all of the new patches that come in.  We can fix the first
problem by tightening up on the requirements for committing things,
but that exacerbates the second problem.  Or we can fix the second
problem by loosening up on the requirements for commit, but that
exacerbates the first problem.  Promoting more or fewer committers is
really the same trade-off: if you're very careful about who you
promote, you'll get better people but not as many of them, so less
will get done but with fewer mistakes; if you're more generous in
handing out commit bits, you reduce the bottleneck to stuff getting
done but, inevitably, you'll be trusting people in whom you have at
least slightly less confidence.  There's an inherent tension between
quality and rate of progress that we can't get rid of, and the fact
that some of our best people are busier than ever with things other
than PostgreSQL hacking is not helping - not only because less actual
review/commit happens, but because newcomers to the community don't
have as much contact with the more senior people who could help mentor
them if they only had the time.

 2. The amount of pre-release testing we get from people outside the
 hard-core development crowd seems to be continuing to decrease.
 We were fortunate that somebody found the JSONB issue before it was
 too late to do anything about it.  Personally, I'm very worried that
 there are other such bugs in 9.4.  But I've given up hoping that any
 more testing will happen until we put out something that calls itself
 9.4.0, which is why I voted to release in the core discussion about it.

 I don't know what to do about either of those things, but I predict
 future release cycles will be just as bad unless we can fix them.

I agree.  We have had a few people, Jeff Janes perhaps foremost among
them, who have done a lot of really useful testing, but overall it
does feel pretty thin.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread David G Johnston
Tom Lane-2 wrote
 Robert Haas lt;

 robertmhaas@

 gt; writes:
 On Thu, Dec 11, 2014 at 1:18 AM, Josh Berkus lt;

 josh@

 gt; wrote:
 While there were technical
 issues, 9.4 dragged a considerable amount because most people were
 ignoring it in favor of 9.5 development.
 
 I think 9.4 dragged almost entirely because of one issue: the
 compressibility of JSONB.
 
 2. The amount of pre-release testing we get from people outside the
 hard-core development crowd seems to be continuing to decrease.
 We were fortunate that somebody found the JSONB issue before it was
 too late to do anything about it.  Personally, I'm very worried that
 there are other such bugs in 9.4.  But I've given up hoping that any
 more testing will happen until we put out something that calls itself
 9.4.0, which is why I voted to release in the core discussion about it.

The compressibility properties of a new type seem like something that should
be mandated before it is committed - it shouldn't require good fortune that



--
View this message in context: 
http://postgresql.nabble.com/logical-column-ordering-tp5829775p5830115.html
Sent from the PostgreSQL - hackers mailing list archive at Nabble.com.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread David G Johnston
David G Johnston wrote
 
 Tom Lane-2 wrote
 Robert Haas lt;

 robertmhaas@

 gt; writes:
 On Thu, Dec 11, 2014 at 1:18 AM, Josh Berkus lt;

 josh@

 gt; wrote:
 While there were technical
 issues, 9.4 dragged a considerable amount because most people were
 ignoring it in favor of 9.5 development.
 
 I think 9.4 dragged almost entirely because of one issue: the
 compressibility of JSONB.
 
 2. The amount of pre-release testing we get from people outside the
 hard-core development crowd seems to be continuing to decrease.
 We were fortunate that somebody found the JSONB issue before it was
 too late to do anything about it.  Personally, I'm very worried that
 there are other such bugs in 9.4.  But I've given up hoping that any
 more testing will happen until we put out something that calls itself
 9.4.0, which is why I voted to release in the core discussion about it.
 The compressibility properties of a new type seem like something that
 should be mandated before it is committed - it shouldn't require good
 fortune that

... someone thought to test it out.  Is there anywhere to effectively add
that to maximize the chance someone remembers for next time?

3. Effort and brain power was also diverted to fixing 9.3 this time around.

I don't have any answers but if a release is thin on features but thick on
review and testing I wouldn't complain.  That testing, especially applied to
existing releases, would have considerable benefit to those still using
older supported releases instead of only benefitting those who can upgrade
to the latest and greatest.  An existing user on an older release is just,
if not more, important than the potential user who is looking for new
features before deciding to migrate or begin using PostgreSQL.

David J.




--
View this message in context: 
http://postgresql.nabble.com/logical-column-ordering-tp5829775p5830116.html
Sent from the PostgreSQL - hackers mailing list archive at Nabble.com.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Heikki Linnakangas

On 12/11/2014 06:59 PM, Robert Haas wrote:

On Thu, Dec 11, 2014 at 11:03 AM, Tom Lane t...@sss.pgh.pa.us wrote:

I think 9.4 dragged almost entirely because of one issue: the
compressibility of JSONB.


Meh.  While we certainly weren't very speedy about resolving that,
I don't think that issue deserves all or even most of the blame.
I agree with Josh: the problem really was that people were not focusing
on getting 9.4 tested and releasable.  One way in which that lack of
focus manifested was not having any urgency about resolving JSONB ...
but it struck me as a systemic problem and not that specific issue's
fault.

I'd finger two underlying issues here:

1. As Bruce points out in a nearby thread, we've been in commitfest mode
more or less continuously since August.  That inherently sucks away
developer mindshare from testing already-committed stuff.


The problem is that, on the one hand, we have a number of serious
problems with things that got committed and turned out to have
problems - the multixact stuff, and JSONB, in particular - and on the
other hand, we are lacking in adequate committer bandwidth to properly
handle all of the new patches that come in.  We can fix the first
problem by tightening up on the requirements for committing things,
but that exacerbates the second problem.  Or we can fix the second
problem by loosening up on the requirements for commit, but that
exacerbates the first problem.


There is a third option: reject more patches, more quickly, with less 
discussion. IOW, handle new patches less properly.


The commitfest is good at making sure that no patch completely falls off 
the radar. That's also a problem. Before the commitfest process, many 
patches were not actively rejected, but they just fell to the side if no 
committer was interested enough in it to pick it up, review, and commit.


There are a lot of patches in the October commitfest that I personally 
don't care about, and if I was a dictator I would just drop as not 
worth the trouble to review. Often a patch just doesn't feel quite 
right, or would require some work to clean up, but you can't immediately 
point to anything outright wrong with it. It takes some effort to review 
such a patch enough to give feedback on it, if you want more meaningful 
feedback than This smells bad.


I imagine that it's the same for everyone else. Many of the patches that 
sit in the commitfest for weeks are patches that no-one really cares 
much about. I'm not sure what to do about that. It would be harsh to 
reject a patch just because no-one's gotten around to reviewing it, and 
if we start doing that, it's not clear what the point of a commitfest is 
any more.


Perhaps we should change the process so that it is the patch author's 
responsibility to find a reviewer, and a committer, for the patch. If 
you can't cajole anyone to review your patch, it's a sign that no-one 
cares about it, and the patch is rejected by default. Or put a quick 
+1/-1 voting option to each patch in the commitfest, to get a quick 
gauge of how the committers feel about it.


- Heikki



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Peter Geoghegan
On Thu, Dec 11, 2014 at 7:37 AM, Robert Haas robertmh...@gmail.com wrote:
 2. It's not clear that we're going to have a particularly-impressive
 list of major features for 9.5.  So far we've got RLS and BRIN. I
 expect that GROUPING SETS is far enough along that it should be
 possible to get it in before development ends, and there are a few
 performance patches pending (Andres's lwlock scalability patches,
 Rahila's work on compressing full-page writes) that I think will
 probably make the grade.  But after that it seems to me that it gets
 pretty thin on the ground.

I'm slightly surprised that you didn't mention abbreviated keys in
that list of performance features. You're reviewing that patch; how do
you feel about it now?

 Are we going to bill commit timestamp
 tracking - with replication node ID tracking as the real goal, despite
 the name - as a major feature, or DDL deparsing if that goes in, as
 major features?  As useful as they may be for BDR, they don't strike
 me as things we can publicize as major features independent of BDR.
 And it's getting awfully late for any other major work that people are
 thinking of to start showing up.

Version 1.0 of INSERT ... ON CONFLICT UPDATE was posted in August -
when development launched. It still doesn't have a reviewer, and it
isn't actually in evidence that someone else has so much as downloaded
and applied the patch (I'm sure someone has done that much, but the
fact is that all the feedback that I've received this year concerns
the semantics/syntax, which you can form an opinion on by just looking
at the extensive documentation and other supplementary material I've
written). It's consistently one of the most requested features, and
yet major aspects of the design, that permeate through every major
subsystem go unremarked on for months now. This feature is
*definitely* major feature list material, since people have been
loudly requesting it for over a decade, and yet no one mentions it in
this thread (only Bruce mentioned it in the other thread about the
effectiveness of the CF process). It's definitely in the top 2 or 3
most requested features, alongside much harder problems like parallel
query and comprehensive partitioning support.

If there is a lesson here for people that are working on major
features, or me personally, I don't know what it is -- if anyone else
knows, please tell me. I've bent over backwards to make the patch as
accessible as possible, and as easy to review as possible. I also
think its potential to destabilize the system (as major features go)
is only about average. What am I doing wrong here?

There is an enormous amount of supplementary documentation associated
with the patch:

https://wiki.postgresql.org/wiki/UPSERT
https://wiki.postgresql.org/wiki/Value_locking

Both of these pages are far larger than the Wiki page for RLS, for
example. The UPSERT wiki page is kept right up to date.

-- 
Peter Geoghegan


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Jeff Janes
On Thu, Dec 11, 2014 at 8:03 AM, Tom Lane t...@sss.pgh.pa.us wrote:


 2. The amount of pre-release testing we get from people outside the
 hard-core development crowd seems to be continuing to decrease.
 We were fortunate that somebody found the JSONB issue before it was
 too late to do anything about it.  Personally, I'm very worried that
 there are other such bugs in 9.4.  But I've given up hoping that any
 more testing will happen until we put out something that calls itself
 9.4.0, which is why I voted to release in the core discussion about it.



We are not particularly inviting of feedback for whatever testing has been
done.

The definitive guide seems to be
https://wiki.postgresql.org/wiki/HowToBetaTest, and says:

You can report tests by email. You can subscribe to any PostgreSQL mailing
list from the subscription form http://www.postgresql.org/community/lists/
.

   - pgsql-bugs: this is the preferred mailing list if you think you have
   found a bug in the beta. You can also use the Bug Reporting Form
   http://www.postgresql.org/support/submitbug/.
   - pgsql-hackers: bugs, questions, and successful test reports are
   welcome here if you are already subscribed to pgsql-hackers. Note that
   pgsql-hackers is a high-traffic mailing list with a lot of development
   discussion.


=

So if you find a bug, you can report it on the bug reporting form (which
doesn't have a drop-down entry for 9.4RC1).

If you have positive results rather than negative ones (or even complaints
that are not actually bugs), you can subscribe to a mailing list which
generates a lot of traffic which is probably over your head and not
interesting to you.

Does the core team keep a mental list of items they want to see tested by
the public, and they will spend their own time testing those things
themselves if they don't hear back on some positive tests for them?

If we find reports of public testing that yields good results (or at least
no bugs) to be useful, we should be more clear on how to go about doing
it.  But are positive reports useful?  If I report a bug, I can write down
the steps to reproduce it, and then follow my own instructions to make sure
it does actually reproduce it.  If I find no bugs, it is just I did a
bunch of random stuff and nothing bad happened, that I noticed.

Chees,

Jeff


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Robert Haas
On Thu, Dec 11, 2014 at 1:06 PM, Peter Geoghegan p...@heroku.com wrote:
 On Thu, Dec 11, 2014 at 7:37 AM, Robert Haas robertmh...@gmail.com wrote:
 2. It's not clear that we're going to have a particularly-impressive
 list of major features for 9.5.  So far we've got RLS and BRIN. I
 expect that GROUPING SETS is far enough along that it should be
 possible to get it in before development ends, and there are a few
 performance patches pending (Andres's lwlock scalability patches,
 Rahila's work on compressing full-page writes) that I think will
 probably make the grade.  But after that it seems to me that it gets
 pretty thin on the ground.

 I'm slightly surprised that you didn't mention abbreviated keys in
 that list of performance features. You're reviewing that patch; how do
 you feel about it now?

I'm not sure it's where I think it needs to be yet, but yeah, I think
that will get in.  I thought of it after I hit send.

 Are we going to bill commit timestamp
 tracking - with replication node ID tracking as the real goal, despite
 the name - as a major feature, or DDL deparsing if that goes in, as
 major features?  As useful as they may be for BDR, they don't strike
 me as things we can publicize as major features independent of BDR.
 And it's getting awfully late for any other major work that people are
 thinking of to start showing up.

 Version 1.0 of INSERT ... ON CONFLICT UPDATE was posted in August -
 when development launched. It still doesn't have a reviewer, and it
 isn't actually in evidence that someone else has so much as downloaded
 and applied the patch (I'm sure someone has done that much, but the
 fact is that all the feedback that I've received this year concerns
 the semantics/syntax, which you can form an opinion on by just looking
 at the extensive documentation and other supplementary material I've
 written). It's consistently one of the most requested features, and
 yet major aspects of the design, that permeate through every major
 subsystem go unremarked on for months now. This feature is
 *definitely* major feature list material, since people have been
 loudly requesting it for over a decade, and yet no one mentions it in
 this thread (only Bruce mentioned it in the other thread about the
 effectiveness of the CF process). It's definitely in the top 2 or 3
 most requested features, alongside much harder problems like parallel
 query and comprehensive partitioning support.

I'm not sure whether that patch is going to go anywhere.  There has
been so much arguing about different aspects of the design that felt
rather unproductive that I think most of the people who would be
qualified to commit it have grown a bit gun-shy.  That would be a good
problem to get fixed, but I don't have a proposal.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Josh Berkus
On 12/11/2014 09:22 AM, Heikki Linnakangas wrote:

 I imagine that it's the same for everyone else. Many of the patches that
 sit in the commitfest for weeks are patches that no-one really cares
 much about. I'm not sure what to do about that. It would be harsh to
 reject a patch just because no-one's gotten around to reviewing it, and
 if we start doing that, it's not clear what the point of a commitfest is
 any more.

So the nobody cares argument is manifestly based on false assumptions.
 Are you contending that nobody cares about UPSERT or GROUPING SETS?  In
my experience, the patches that sit for weeks on the CF fall into 4 groups:

1. Large complicated patches that only a handful of committers are even
capable of reviewing.

2. Obscure patches which require specialized knowledge our outside input
to review.

3. Inconsequential patches where the submitter doesn't work the social
process.

4. Patches with contentious spec or syntax.

5. Patches which everyone wants but have persistent hard-to-resolve issues.

Of these only (3) would fit into nobody cares, and that's a pretty
small group.

There's also a chicken-and-egg problem there.  Say that we started not
reviewing DW features because nobody cares.  Then the people who
contributed those features don't go on to become major contributors,
which means they won't review further DW patches.  Which means that
we've just closed off an entire use-case for PostgreSQL.  I'd think that
PostGIS would have taught us the value of the nobody cares fallacy.

Also, if we go back on the promise that every patch gets a review,
then we're definitely headed towards no more new contributors.  As Noah
said at one developer meeting (to paraphrase), one of the few things
which keeps contributors persisting through our baroque,
poorly-documented, excessively political contribution process is the
promise that they'll get a fair shake for their invested time.  If we
drop that promise, we'll solve our workflow problem by cutting off the
flow of new patches entirely.

 Perhaps we should change the process so that it is the patch author's
 responsibility to find a reviewer, and a committer, for the patch. If
 you can't cajole anyone to review your patch, it's a sign that no-one
 cares about it, and the patch is rejected by default. Or put a quick
 +1/-1 voting option to each patch in the commitfest, to get a quick
 gauge of how the committers feel about it.

Again, that process would favor existing contributors and other folks
who know how to work the Postgres community.  It would be effectively
the same as hanging up a sign which says no new contributors wanted.
It would also be dramatically increasing the amount of politics around
submitted patches, which take up far more time than the technical work.

Overall, we're experiencing this issue because of a few predictable reasons:

1. a gradual increase in the number of submitted patches, especially
large patches

2. Tom Lane cutting back on the number of other people's patches he
reviews and revises.

3. Other major committers getting busier with their day jobs.

4. Failure to recruit, mentor and promote new committers at a rate
proportional to the number of new contributors or the size of our community.

5. Failure to adopt or develop automated systems to remove some of the
grunt work from patch submission and review.

6. Failure to adhere to any uniform standards of patch handing for the
Commitfests.

(2) out of these is actually the biggest thing we're seeing right now, I
think.   Tom was historically a one-man-patch-fixing machine, at one
time responsible for 70% of the patches committed to PostgreSQL.  This
was never a good thing for the community, even if it was a good thing
for the code base, becuase it discouraged others from stepping into a
senior committer role.  Now Tom has cut back, and others have to take up
the slack.  In the long run this will be good for our development
community; in the short run it's kind of painful.

I will also point out that some of the existing senior committers, who
are the heaviest burdened under the existing system, have also been the
most resistant to any change in the system.  You (Heikki) yourself
expressed a strong opposition to any attempt to recruit more reviewers.
So, given that you are inside the heart of the problem, do you have a
solution other than your proposal above that we simply stop accepting
new contributors?  Or is that your solution?  It would work, for some
definition of work.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Josh Berkus
On 12/11/2014 08:59 AM, Tom Lane wrote:
 More abstractly, there's a lot of value in having a predictable release
 schedule.  That's going to mean that some release cycles are thin on
 user-visible features, even if just as much work went into them.  It's
 the nature of the game.

+ 1,000,000 from me.  ;-)

Frankly, BRIN, UPSERT and a couple other things are plenty for a
Postgres release.  Other SQL databases would be thrilled to have that
much ... can you name 3 major advances in the last MySQL release?

And given that I've seen nothing about jquery/VODKA since pgCon, I'm
expecting them for 9.6/whatever, not 9.5.  There's a whole longish
syntax discussion we haven't even started yet, let alone actual
technical review.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Heikki Linnakangas

On 12/11/2014 08:51 PM, Josh Berkus wrote:

On 12/11/2014 09:22 AM, Heikki Linnakangas wrote:


I imagine that it's the same for everyone else. Many of the patches that
sit in the commitfest for weeks are patches that no-one really cares
much about. I'm not sure what to do about that. It would be harsh to
reject a patch just because no-one's gotten around to reviewing it, and
if we start doing that, it's not clear what the point of a commitfest is
any more.


So the nobody cares argument is manifestly based on false assumptions.
  Are you contending that nobody cares about UPSERT or GROUPING SETS?


No. I was thinking of patches like Add IF NOT EXISTS to CREATE TABLE AS 
and CREATE MATERIALIZED VIEW, event triggers: more DROP info, Point 
to polygon distance operator and pgcrypto: support PGP signatures. 
And nobody was an exaggeration; clearly *someone* cares about those 
things, or they wouldn't have written a patch in the first place. But 
none of the committers care enough to pick them up. Even in the case 
that someone reviews them, it's often not because the reviewer is 
interested in the patch, it's just to help out with the process.


(Apologies to the authors of those patches; they were just the first few 
that caught my eye)



There's also a chicken-and-egg problem there.  Say that we started not
reviewing DW features because nobody cares.  Then the people who
contributed those features don't go on to become major contributors,
which means they won't review further DW patches.  Which means that
we've just closed off an entire use-case for PostgreSQL.  I'd think that
PostGIS would have taught us the value of the nobody cares fallacy.

Also, if we go back on the promise that every patch gets a review,
then we're definitely headed towards no more new contributors.  As Noah
said at one developer meeting (to paraphrase), one of the few things
which keeps contributors persisting through our baroque,
poorly-documented, excessively political contribution process is the
promise that they'll get a fair shake for their invested time.  If we
drop that promise, we'll solve our workflow problem by cutting off the
flow of new patches entirely.


Yeah, there is that.


Perhaps we should change the process so that it is the patch author's
responsibility to find a reviewer, and a committer, for the patch. If
you can't cajole anyone to review your patch, it's a sign that no-one
cares about it, and the patch is rejected by default. Or put a quick
+1/-1 voting option to each patch in the commitfest, to get a quick
gauge of how the committers feel about it.


Again, that process would favor existing contributors and other folks
who know how to work the Postgres community.  It would be effectively
the same as hanging up a sign which says no new contributors wanted.
It would also be dramatically increasing the amount of politics around
submitted patches, which take up far more time than the technical work.


I was thinking that by getting candid feedback that none of the existing 
contributors are interested to look at your patch, the author could 
revise the patch to garner more interest, or perhaps promise to review 
someone else's patch in return. Right now, the patches just linger, and 
the author doesn't know why, what's going to happen or what to do about it.



I will also point out that some of the existing senior committers, who
are the heaviest burdened under the existing system, have also been the
most resistant to any change in the system.  You (Heikki) yourself
expressed a strong opposition to any attempt to recruit more reviewers.


Huh, I did? Can you elaborate?


So, given that you are inside the heart of the problem, do you have a
solution other than your proposal above that we simply stop accepting
new contributors?  Or is that your solution?  It would work, for some
definition of work.


I don't have any good solutions, I'm afraid. It might help to ping the 
reviewers who have signed up more often, to make the reviews to happen 
more quickly. I did some nudge people in the August commitfest, but I 
felt that it didn't actually achieve much. The most efficient way to 
move the commitfest forward was to just review more patches myself.


That's one thought. Robert said the same thing about when he was the 
commitfest manager; he just reviewed most the patches himself in the 
end. And you mentioned that Tom used to review 70% of all incoming 
patches. How about we make that official? It's the commitfest manager's 
duty to review all the patches. He can recruit others if he can, but at 
the end of the day he's expected to take a look at every patch, and 
commit or return with some feedback.


The problem with that is that we'll have a hard time to find volunteers 
for that. But we only need to find one sucker for each commitfest. I can 
volunteer to do that once a year; if the other active committers do the 
same, we're covered.


- Heikki



--
Sent via pgsql-hackers mailing list 

Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Alvaro Herrera
Heikki Linnakangas wrote:

 That's one thought. Robert said the same thing about when he was the
 commitfest manager; he just reviewed most the patches himself in the end.
 And you mentioned that Tom used to review 70% of all incoming patches. How
 about we make that official? It's the commitfest manager's duty to review
 all the patches. He can recruit others if he can, but at the end of the day
 he's expected to take a look at every patch, and commit or return with some
 feedback.
 
 The problem with that is that we'll have a hard time to find volunteers for
 that. But we only need to find one sucker for each commitfest. I can
 volunteer to do that once a year; if the other active committers do the
 same, we're covered.

Hm, I was commitfest manager once, too, and this exact same thing
happened.  Maybe we need to make that official, and give the sucker some
perk.

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Peter Geoghegan
On Thu, Dec 11, 2014 at 11:52 AM, Alvaro Herrera
alvhe...@2ndquadrant.com wrote:
 The problem with that is that we'll have a hard time to find volunteers for
 that. But we only need to find one sucker for each commitfest. I can
 volunteer to do that once a year; if the other active committers do the
 same, we're covered.

 Hm, I was commitfest manager once, too, and this exact same thing
 happened.  Maybe we need to make that official, and give the sucker some
 perk.

I should do it at least once, if only because I haven't experienced it.
-- 
Peter Geoghegan


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Peter Geoghegan
On Thu, Dec 11, 2014 at 10:43 AM, Robert Haas robertmh...@gmail.com wrote:
 Version 1.0 of INSERT ... ON CONFLICT UPDATE was posted in August -
 when development launched. It still doesn't have a reviewer, and it
 isn't actually in evidence that someone else has so much as downloaded
 and applied the patch

 I'm not sure whether that patch is going to go anywhere.  There has
 been so much arguing about different aspects of the design that felt
 rather unproductive that I think most of the people who would be
 qualified to commit it have grown a bit gun-shy.  That would be a good
 problem to get fixed, but I don't have a proposal.

Really?

I have acted comprehensively on 100% of feedback to date on the
semantics/syntax of the ON CONFLICT UPDATE patch. The record reflects
that. I don't believe that the problem is that people are gun shy. We
haven't even had a real disagreement since last year, and last year's
discussion of value locking was genuinely very useful (I hate to say
it, but the foreign key locking patch might have considered the
possibility of unprincipled deadlocks more closely, as we saw
recently).

Lots of other systems have a comparable feature. Most recently, VoltDB
added a custom UPSERT, even though they don't have half the SQL
features we do. It's a complicated feature, but it's not a
particularly complicated feature as big, impactful features go (like
LATERAL, or logical decoding, or the foreign key locking stuff). It's
entirely possible to get the feature in in the next few months, if
someone will work with me on it.

Even Heikki, who worked on this with me far more than everyone else,
found the value locking page a useful summary. He didn't even know
that there was a third design advocated by Simon and Andres until he
saw it! The discussion was difficult, but had a useful outcome,
because accounting for unprincipled deadlocks is a major source of
complexity. I have seen a lot of benefit from comprehensively
documenting stuff in one place (the wiki pages), but that has tapered
off.
-- 
Peter Geoghegan


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Alvaro Herrera
Peter Geoghegan wrote:
 On Thu, Dec 11, 2014 at 11:52 AM, Alvaro Herrera
 alvhe...@2ndquadrant.com wrote:
  The problem with that is that we'll have a hard time to find volunteers for
  that. But we only need to find one sucker for each commitfest. I can
  volunteer to do that once a year; if the other active committers do the
  same, we're covered.
 
  Hm, I was commitfest manager once, too, and this exact same thing
  happened.  Maybe we need to make that official, and give the sucker some
  perk.
 
 I should do it at least once, if only because I haven't experienced it.

We could make an slogan out of it: you've never experienced the
PostgreSQL development process if you haven't been a CFM at least once
in your life.

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Bruce Momjian
On Thu, Dec 11, 2014 at 11:59:58AM -0500, Robert Haas wrote:
 The problem is that, on the one hand, we have a number of serious
 problems with things that got committed and turned out to have
 problems - the multixact stuff, and JSONB, in particular - and on the
 other hand, we are lacking in adequate committer bandwidth to properly
 handle all of the new patches that come in.  We can fix the first
 problem by tightening up on the requirements for committing things,
 but that exacerbates the second problem.  Or we can fix the second
 problem by loosening up on the requirements for commit, but that
 exacerbates the first problem.  Promoting more or fewer committers is
 really the same trade-off: if you're very careful about who you
 promote, you'll get better people but not as many of them, so less
 will get done but with fewer mistakes; if you're more generous in
 handing out commit bits, you reduce the bottleneck to stuff getting
 done but, inevitably, you'll be trusting people in whom you have at
 least slightly less confidence.  There's an inherent tension between
 quality and rate of progress that we can't get rid of, and the fact
 that some of our best people are busier than ever with things other
 than PostgreSQL hacking is not helping - not only because less actual
 review/commit happens, but because newcomers to the community don't
 have as much contact with the more senior people who could help mentor
 them if they only had the time.

Great outline of the tradeoffs involved in  being more aggressive about
committing. I do think the multixact and JSONB problems have spooked
some of us to be slower/more careful about committing.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + Everyone has their own god. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Jeff Janes
On Thu, Dec 11, 2014 at 11:40 AM, Heikki Linnakangas 
hlinnakan...@vmware.com wrote:

 On 12/11/2014 08:51 PM, Josh Berkus wrote:

 On 12/11/2014 09:22 AM, Heikki Linnakangas wrote:


  Perhaps we should change the process so that it is the patch author's
 responsibility to find a reviewer, and a committer, for the patch. If
 you can't cajole anyone to review your patch, it's a sign that no-one
 cares about it, and the patch is rejected by default. Or put a quick
 +1/-1 voting option to each patch in the commitfest, to get a quick
 gauge of how the committers feel about it.


 Again, that process would favor existing contributors and other folks
 who know how to work the Postgres community.  It would be effectively
 the same as hanging up a sign which says no new contributors wanted.
 It would also be dramatically increasing the amount of politics around
 submitted patches, which take up far more time than the technical work.


 I was thinking that by getting candid feedback that none of the existing
 contributors are interested to look at your patch, the author could revise
 the patch to garner more interest, or perhaps promise to review someone
 else's patch in return. Right now, the patches just linger, and the author
 doesn't know why, what's going to happen or what to do about it.


I agree.  Having your patch disappear into the void is not friendly at
all.  But I don't think a commentless -1 is the answer, either.  That
might one of the few things worse than silence.  Even if the comment is
just This seems awfully complicated for a minimally useful feature or
This will probably have unintended side effects in the executor that I
don't have time to trace down, can someone make an argument for
correctness, or even this format is unreadable, I won't review this until
it is fixed then at least the author (and perhaps more important, junior
contributors who are not the author) will know either what argument to make
to lobby for it, what avenue to take when reviewing it, or whether to just
forget it and work on something more important.


Cheers,

Jeff


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Robert Haas
On Thu, Dec 11, 2014 at 3:47 PM, Jeff Janes jeff.ja...@gmail.com wrote:
 I agree.  Having your patch disappear into the void is not friendly at all.
 But I don't think a commentless -1 is the answer, either.  That might one
 of the few things worse than silence.  Even if the comment is just This
 seems awfully complicated for a minimally useful feature or This will
 probably have unintended side effects in the executor that I don't have time
 to trace down, can someone make an argument for correctness, or even this
 format is unreadable, I won't review this until it is fixed then at least
 the author (and perhaps more important, junior contributors who are not the
 author) will know either what argument to make to lobby for it, what avenue
 to take when reviewing it, or whether to just forget it and work on
 something more important.

Agreed!

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Bruce Momjian
On Thu, Dec 11, 2014 at 10:04:43AM -0700, David G Johnston wrote:
 Tom Lane-2 wrote
  Robert Haas lt;
 
  robertmhaas@
 
  gt; writes:
  On Thu, Dec 11, 2014 at 1:18 AM, Josh Berkus lt;
 
  josh@
 
  gt; wrote:
  While there were technical
  issues, 9.4 dragged a considerable amount because most people were
  ignoring it in favor of 9.5 development.
  
  I think 9.4 dragged almost entirely because of one issue: the
  compressibility of JSONB.
  
  2. The amount of pre-release testing we get from people outside the
  hard-core development crowd seems to be continuing to decrease.
  We were fortunate that somebody found the JSONB issue before it was
  too late to do anything about it.  Personally, I'm very worried that
  there are other such bugs in 9.4.  But I've given up hoping that any
  more testing will happen until we put out something that calls itself
  9.4.0, which is why I voted to release in the core discussion about it.
 
 The compressibility properties of a new type seem like something that should
 be mandated before it is committed - it shouldn't require good fortune that

Odd are the next problem will have nothing to do with compressibility 
--- we can't assume old failure will repeat themselves.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + Everyone has their own god. +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread David Johnston
On Thu, Dec 11, 2014 at 2:05 PM, Bruce Momjian br...@momjian.us wrote:

 On Thu, Dec 11, 2014 at 10:04:43AM -0700, David G Johnston wrote:
  Tom Lane-2 wrote
   Robert Haas lt;
 
   robertmhaas@
 
   gt; writes:
   On Thu, Dec 11, 2014 at 1:18 AM, Josh Berkus lt;
 
   josh@
 
   gt; wrote:
   While there were technical
   issues, 9.4 dragged a considerable amount because most people were
   ignoring it in favor of 9.5 development.
  
   I think 9.4 dragged almost entirely because of one issue: the
   compressibility of JSONB.
  
   2. The amount of pre-release testing we get from people outside the
   hard-core development crowd seems to be continuing to decrease.
   We were fortunate that somebody found the JSONB issue before it was
   too late to do anything about it.  Personally, I'm very worried that
   there are other such bugs in 9.4.  But I've given up hoping that any
   more testing will happen until we put out something that calls itself
   9.4.0, which is why I voted to release in the core discussion about it.
 
  The compressibility properties of a new type seem like something that
 should
  be mandated before it is committed - it shouldn't require good fortune
 that

 Odd are the next problem will have nothing to do with compressibility
 --- we can't assume old failure will repeat themselves.




​tl;dr: assign two people, a manager/curator and a lead reviewer​.  Give
the curator better tools and the responsibility to engage the community.
If the primary reviewer cannot review a patch in the current commitfest it
can be marked awaiting reviewer and moved to the next CF for evaluation
by the next pair of individuals.  At minimum, though, if it does get moved
the manager AND reviewer need to comment on why it was not handled during
their CF.  Also, formalize documentation targeting developers and reviewers
just like the documentation for users has been formalized and committed to
the repository.  That knowledge and history is probably more important that
source code commit log and definitely should be more accessible to
newcomers.


​While true a checklist of things to look for and evaluate when adding a
new type to the system still has value.  How new types interact, if at all,
with TOAST seems like something that warrants explicit attention before
commit, and there are probably others (e.g., OIDs, input/output function
volatility and configuration, etc...)​.  Maybe this exists somewhere but it
you are considering improvements to the commitfest application having an
top-of-page  always-visible checklist that can bootstrapped based upon the
patch/feature type and modified for specific nuances for the item in
question seems like it would be valuable.

If this was in place before 9.3 then the whatever category the multi-xact
patches fall into would want to have their template modified to incorporate
the various research and testing - along with links to the discussions -
the resulted from the various bug reports that were submitted.  It could
even be structured to both be an interactive checklist as well as acting as
curated documentation for developers and reviewers.  The wiki has some of
this but if the goal is to encourage people to learn how to contribute to
PostgreSQL it should receive a similar level of attention and quality
control that our documentation for people wanting to learn how to use
PostgreSQL receive.  But that is above and beyond the simple goal of having
meaningful checklists attached to each of the major commit-fest items whose
TODO items can be commented upon and serve as a reference for how close to
commit a feature may be.  Comments can be as simple as a place for people
to upload a psql script and say this is what I did and everything seemed
to work/fail in the way I expected - on this platform.

Curation is hard though so I get why easier - just provide links to the
mailing list mainly - actions are what is currently being used.  Instead of
the CF manager being a reviewer (which is a valid approach) having them be
a curator and providing better tools geared toward that role (both to do
the work and to share the results) seem like a better ideal.  Having that
role a CF reviewer should maybe also be assigned.  The two people -
reviewer and manager - would then be responsible for ensuring that reviews
are happening and that the communication to and recruitment from the
community is being handled - respectively.

Just some off-the-cuff thoughts...

David J.


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Alvaro Herrera
FWIW I don't think any amount of process would have gotten multixact to
not have the copious bugs it had.  It was just too complex a patch,
doing ugly things to parts too deeply linked to the inner guts of the
server.  We might have spared a few with some extra testing (such as the
idiotic wraparound bugs, and perhaps the pg_upgrade problems), but most
of the rest of it would have taken a real production load to notice --
which is exactly what happened.

(Of course, review from Tom might have unearthed many of these bugs
before they hit final release, but we're saying we don't want to be
dependent on Tom's review for every patch so that doesn't count.)

Review is good, but (as history shows) some bugs can slip through even
extensive review such as the one multixacts got from Noah and Andres.
Had anyone put some real stress on the beta, we could have noticed some
of these bugs much earlier.  One of the problems we have is that our
serious users don't seem to take the beta period seriously.  All our
users are too busy for that, it seems, and they expect that somebody
else will test the point-zero release, and that by the time it hits
point-one or point-two it will be bug-free, which is quite clearly not
the case.

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training  Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-11 Thread Peter Geoghegan
On Thu, Dec 11, 2014 at 1:49 PM, Alvaro Herrera
alvhe...@2ndquadrant.com wrote:
 Review is good, but (as history shows) some bugs can slip through even
 extensive review such as the one multixacts got from Noah and Andres.
 Had anyone put some real stress on the beta, we could have noticed some
 of these bugs much earlier.  One of the problems we have is that our
 serious users don't seem to take the beta period seriously.  All our
 users are too busy for that, it seems, and they expect that somebody
 else will test the point-zero release, and that by the time it hits
 point-one or point-two it will be bug-free, which is quite clearly not
 the case.

Good, strategic stress testing has a big role to play here IMV. I have
had some difficulty generating interest in that, but I think it's
essential.

-- 
Peter Geoghegan


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] 9.5 release scheduling (was Re: logical column ordering)

2014-12-10 Thread Josh Berkus
On 12/10/2014 09:35 PM, Tom Lane wrote:
 Josh Berkus j...@agliodbs.com writes:
 On 12/10/2014 05:14 PM, Stephen Frost wrote:
 * Andres Freund (and...@2ndquadrant.com) wrote:
 But the scheduling of commits with regard to the 9.5 schedule actually
 opens a relevant question: When are we planning to release 9.5? Because
 If we try ~ one year from now it's a whole different ballgame than if we
 try to go back to september. And I think there's pretty good arguments
 for both.
 
 This should really be on its own thread for discussion...  I'm leaning,
 at the moment at least, towards the September release schedule.  I agree
 that having a later release would allow us to get more into it, but
 there's a lot to be said for the consistency we've kept up over the past
 few years with a September (our last non-September release was 8.4).
 
 Can we please NOT discuss this in the thread about someone's patch?  Thanks.
 
 Quite.  So, here's a new thread.
 
 MHO is that, although 9.4 has slipped more than any of us would like,
 9.5 development launched right on time in August.  So I don't see a
 good reason to postpone 9.5 release just because 9.4 has slipped.
 I think we should stick to the schedule agreed to in Ottawa last spring.
 
 Comments?

If anything, it seems like a great reason to try to get 9.5 out BEFORE
we open the tree for 9.6/10.0/Cloud++.  While there were technical
issues, 9.4 dragged a considerable amount because most people were
ignoring it in favor of 9.5 development.

So far, I haven't seen any features for 9.5 which would delay a more
timely release the way we did for 9.4.  Anybody know of a bombshell
someone's going to drop on us for CF5?


-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers