Re: [HACKERS] open items for 9.4
On 09/30/2014 09:10 PM, Gregory Smith wrote: On 9/29/14, 2:30 PM, Andres Freund wrote: Can we explain those reasons in the form of documentation? Yes. Try and benchmark it. It'll be hardware and workload dependant. I missed this whole thing, and eventually I have to circle back to it. I could do it this week. Ah, that would be great! Could you (or someone else familiar with the useful benchmarks) give me more detail on how to replicate one case where things should improve? I can do that, and I have a lab full of hardware if it's easier to spot on one type of server. That exercise will probably lead to a useful opinion on the feature in its final form, any associated GUC, tunables, and necessary level of associated documentation in a day or two. To see the most improvement from the patch, try a workload that's otherwise bottlenecked on XLogInsert. For example, with pgbench: psql postgres -c create table foo (id int4) pgbench postgres -n -f fooinsert.sql -c 4 -T10 and in the test script: insert into foo select g from generate_series(1, 1) g; - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] open items for 9.4
On 09/30/2014 04:56 AM, Heikki Linnakangas wrote: There seems to be no decisive consensus here. I'm going to put my foot on the ground and go remove it, as I'm leaning towards that option, and we need to get the release out. But if someone objects loudly enough to actually write the documentation and commit it that way, I'm happy with that too. I'm also in favor of removing it. We really don't need more GUCs which nobody has any clear idea how to change or why. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] open items for 9.4
On 9/29/14, 2:30 PM, Andres Freund wrote: Can we explain those reasons in the form of documentation? Yes. Try and benchmark it. It'll be hardware and workload dependant. I missed this whole thing, and eventually I have to circle back to it. I could do it this week. Could you (or someone else familiar with the useful benchmarks) give me more detail on how to replicate one case where things should improve? I can do that, and I have a lab full of hardware if it's easier to spot on one type of server. That exercise will probably lead to a useful opinion on the feature in its final form, any associated GUC, tunables, and necessary level of associated documentation in a day or two. This is a pretty low bar, right? Examples and documentation sufficient for just me to catch up is not asking for very much. If it isn't possible to do that in a very short period of time, it would not bode well for broader consumption. -- Greg Smith greg.sm...@crunchydatasolutions.com Chief PostgreSQL Evangelist - http://crunchydatasolutions.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] open items for 9.4
On Sun, Sep 28, 2014 at 11:36 PM, Tom Lane t...@sss.pgh.pa.us wrote: Josh Berkus j...@agliodbs.com writes: So, can we get Beta3 out now? If nobody else steps up and says they want to do some performance testing, I'll push the latest lengths+offsets patch tomorrow. Are any of the other open items listed at https://wiki.postgresql.org/wiki/PostgreSQL_9.4_Open_Items things that we must-fix-before-beta3? The items I see are: - Remove xloginsert_slots/xloginsert_locks GUC - Not yet!! The text seems to indicate that there's some disagreement on this point. I don't have a strong opinion on whether or not to keep the GUC, but if we're going to remove it it should probably happen before beta3. It's going to be impossible to remove once we've released with it, I suspect. - TAP tests still have some pretty severe problems Some of these issues have been fixed; but maybe not all. - psql output change in 9.4 I'm still happy with what's committed, but if somebody else wants to tweak it, it should get done soon. - autovacuum scheduling starvation and frenzy This doesn't seem to be a new problem, so I don't think we should consider it a stop-ship issue. - jsonb data format doesn't work well with toast compression Sounds like we are near to resolving this. - pg_dump fails with --if-exists and blobs This looks like a 9.4 regression. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] open items for 9.4
On Mon, Sep 29, 2014 at 5:53 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-29 11:50:19 -0400, Robert Haas wrote: The items I see are: - Remove xloginsert_slots/xloginsert_locks GUC - Not yet!! The text seems to indicate that there's some disagreement on this point. I don't have a strong opinion on whether or not to keep the GUC, but if we're going to remove it it should probably happen before beta3. It's going to be impossible to remove once we've released with it, I suspect. I vote for keeping it. No real preference. But ISTM that if we're uncertain, it's probably a good idea to keep them. - TAP tests still have some pretty severe problems Some of these issues have been fixed; but maybe not all. If there are, they don't seem blockers to me personally. +1. Good to have, but not release blockers. - autovacuum scheduling starvation and frenzy This doesn't seem to be a new problem, so I don't think we should consider it a stop-ship issue. +1. +1. - pg_dump fails with --if-exists and blobs This looks like a 9.4 regression. Alvaro, IIRC you were looking at this one? This does seem to be the major release stopper, other than the json stuff. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] open items for 9.4
On 2014-09-29 11:50:19 -0400, Robert Haas wrote: The items I see are: - Remove xloginsert_slots/xloginsert_locks GUC - Not yet!! The text seems to indicate that there's some disagreement on this point. I don't have a strong opinion on whether or not to keep the GUC, but if we're going to remove it it should probably happen before beta3. It's going to be impossible to remove once we've released with it, I suspect. I vote for keeping it. - TAP tests still have some pretty severe problems Some of these issues have been fixed; but maybe not all. If there are, they don't seem blockers to me personally. - autovacuum scheduling starvation and frenzy This doesn't seem to be a new problem, so I don't think we should consider it a stop-ship issue. +1. - pg_dump fails with --if-exists and blobs This looks like a 9.4 regression. Alvaro, IIRC you were looking at this one? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] open items for 9.4
Andres Freund wrote: On 2014-09-29 11:50:19 -0400, Robert Haas wrote: - pg_dump fails with --if-exists and blobs This looks like a 9.4 regression. Alvaro, IIRC you were looking at this one? I am. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] open items for 9.4
Dne 29.9.2014 18:00 Magnus Hagander mag...@hagander.net napsal(a): On Mon, Sep 29, 2014 at 5:53 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-29 11:50:19 -0400, Robert Haas wrote: The items I see are: - Remove xloginsert_slots/xloginsert_locks GUC - Not yet!! The text seems to indicate that there's some disagreement on this point. I don't have a strong opinion on whether or not to keep the GUC, but if we're going to remove it it should probably happen before beta3. It's going to be impossible to remove once we've released with it, I suspect. I vote for keeping it. No real preference. But ISTM that if we're uncertain, it's probably a good idea to keep them. - TAP tests still have some pretty severe problems Some of these issues have been fixed; but maybe not all. If there are, they don't seem blockers to me personally. +1. Good to have, but not release blockers. - autovacuum scheduling starvation and frenzy This doesn't seem to be a new problem, so I don't think we should consider it a stop-ship issue. +1. +1. - pg_dump fails with --if-exists and blobs This looks like a 9.4 regression. Alvaro, IIRC you were looking at this one? This does seem to be the major release stopper, other than the json stuff. I sent two variants of fix month ago. It should be somewhere in mailing list archive -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] open items for 9.4
On 2014-09-29 11:28:07 -0700, Josh Berkus wrote: On 09/29/2014 08:53 AM, Andres Freund wrote: - Remove xloginsert_slots/xloginsert_locks GUC - Not yet!! The text seems to indicate that there's some disagreement on this point. I don't have a strong opinion on whether or not to keep the GUC, but if we're going to remove it it should probably happen before beta3. It's going to be impossible to remove once we've released with it, I suspect. I vote for keeping it. Time for me to put on my we have too many darned GUCs curmudgeon hat. 1. What does it do? Change scalability in write heavy loads. 2. Are there reasons why users would want to change it from the default? Yes, to improve scalability. 3. Can we explain those reasons in the form of documentation? Yes. Try and benchmark it. It'll be hardware and workload dependant. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] open items for 9.4
Robert Haas robertmh...@gmail.com writes: The items I see are: - Remove xloginsert_slots/xloginsert_locks GUC - Not yet!! The text seems to indicate that there's some disagreement on this point. I don't have a strong opinion on whether or not to keep the GUC, but if we're going to remove it it should probably happen before beta3. It's going to be impossible to remove once we've released with it, I suspect. The lack of any documentation for the GUC (neither in config.sgml or postgresql.conf.sample) suggests very very strongly that it was not meant to be shipped. If we don't remove it I will certainly insist that it be documented adequately. Personally I think a hardwired #define should be plenty. What's the argument that users will need to tune this at runtime? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] open items for 9.4
Andres Freund and...@2ndquadrant.com writes: On 2014-09-29 14:44:42 -0400, Tom Lane wrote: Personally I think a hardwired #define should be plenty. What's the argument that users will need to tune this at runtime? That right now it can make quite noticeable differences in scalability. And we have not much data to justify 8 as the default. And that's surely not going to change if we require recompiles. I wonder why it's a fixed constant at all, and not something like wal_buffers / 8. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] open items for 9.4
On 2014-09-29 16:35:12 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-09-29 16:16:38 -0400, Tom Lane wrote: I wonder why it's a fixed constant at all, and not something like wal_buffers / 8. Because that'd be horrible performancewise on a system with many wal_buffers. There's several operations where all locks are checked in sequence (to see whether there's any stragglers that need to finish inserting) and even some where they're acquired concurrently (e.g. for xlog switch, checkpoint and such). Hm. Well, if there are countervailing considerations as to how large is a good value, that makes it even less likely that it's sensible to expose it as a user tunable. Aren't there such considerations for most of the performance critical gucs? A relevant analogy is that we don't expose a way to adjust the number of lock table partitions at runtime. Which has worked out badly for e.g. the number of buffer partitions... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] open items for 9.4
On Tue, Sep 30, 2014 at 3:44 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: The items I see are: - Remove xloginsert_slots/xloginsert_locks GUC - Not yet!! The text seems to indicate that there's some disagreement on this point. I don't have a strong opinion on whether or not to keep the GUC, but if we're going to remove it it should probably happen before beta3. It's going to be impossible to remove once we've released with it, I suspect. The lack of any documentation for the GUC (neither in config.sgml or postgresql.conf.sample) suggests very very strongly that it was not meant to be shipped. If we don't remove it I will certainly insist that it be documented adequately. Personally I think a hardwired #define should be plenty. What's the argument that users will need to tune this at runtime? I tend to go in this direction too. It is unclear how it is really able to improve scalability, or at least some documentation should be here to help users to set it. An additional thought as well: set it with a configure switch at compilation instead of a GUC. -- Michael
