Re: [HACKERS] Getting to 9.3 beta
On 29 March 2013 15:05, Tom Lane t...@sss.pgh.pa.us wrote: Bruce Momjian br...@momjian.us writes: Our final 9.3 commit-fest has has exceeded the two-month mark, so it is time to start targeting a date to close it and get to 9.3 beta. I see 25 items will needing attention before we can close it: https://commitfest.postgresql.org/action/commitfest_view?id=17 What is a reasonable timeframe to target for completion of these items? TBH, once Andrew commits the JSON patch, I wouldn't have a problem with moving all the rest to Returned With Feedback or the next CF. None of the others seem to me to be close-to-committable (with the possible exception of the Max LSN patch, which I've not looked at), and April is not the time to be doing development. Agreed I completely agree with the need to end this CF here/now/soon. As someone who has normally tried to stretch the limit on what is possible, I do recognise the need for this to come to an end. (Certainly my own available time is running out.) Thanks very much to everybody that tried to get so much into the release. Next week is going to be tied up with the back-branch releases, but maybe we could target beta for the week after? The main gating factor at this point really would be how quickly we could write some draft release notes, so people know what to test. I think we probably need a little more time for open items discussion and lists (not more feature commits, just docs and I-said-I'd-change-thats). In the past that has taken a couple of weeks to work through, so I'd guess there's a few things to sort out there. So I suggest about 23 April for Beta1. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting to 9.3 beta
On Fri, Mar 29, 2013 at 11:22 AM, Andres Freund and...@2ndquadrant.com wrote: - sepgsql: db_schema:search permission and sepgsql: db_procedure:execute permission: Not much review afaics. = looks unfair, but unless some comitter (robert?) takes it on I don't see much alternative to booting it to the next fest? I'm going to try to pick these up early next week. I suspect they will be straightforward. I'm sorry I've been buried in other things recently. I'm attempting to get unburied. So far it's only sort-of working, but I keep hoping... ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting to 9.3 beta
On Fri, Mar 29, 2013 at 7:22 PM, Andres Freund and...@2ndquadrant.comwrote: - Index based regexp search for pg_trgm: Seems like the patch is undergoing restructuring of the regex access API = move to next fest Last proposal of regex API by Tom seems simple enough in implementation for me. I'm going to post reworked version today or tomorrow. I hope we can do one more review cycle in this CF. -- With best regards, Alexander Korotkov.
[HACKERS] Getting to 9.3 beta
Our final 9.3 commit-fest has has exceeded the two-month mark, so it is time to start targeting a date to close it and get to 9.3 beta. I see 25 items will needing attention before we can close it: https://commitfest.postgresql.org/action/commitfest_view?id=17 What is a reasonable timeframe to target for completion of these items? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting to 9.3 beta
Bruce Momjian br...@momjian.us writes: Our final 9.3 commit-fest has has exceeded the two-month mark, so it is time to start targeting a date to close it and get to 9.3 beta. I see 25 items will needing attention before we can close it: https://commitfest.postgresql.org/action/commitfest_view?id=17 What is a reasonable timeframe to target for completion of these items? TBH, once Andrew commits the JSON patch, I wouldn't have a problem with moving all the rest to Returned With Feedback or the next CF. None of the others seem to me to be close-to-committable (with the possible exception of the Max LSN patch, which I've not looked at), and April is not the time to be doing development. Next week is going to be tied up with the back-branch releases, but maybe we could target beta for the week after? The main gating factor at this point really would be how quickly we could write some draft release notes, so people know what to test. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting to 9.3 beta
On Fri, Mar 29, 2013 at 4:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: Bruce Momjian br...@momjian.us writes: Our final 9.3 commit-fest has has exceeded the two-month mark, so it is time to start targeting a date to close it and get to 9.3 beta. I see 25 items will needing attention before we can close it: https://commitfest.postgresql.org/action/commitfest_view?id=17 What is a reasonable timeframe to target for completion of these items? TBH, once Andrew commits the JSON patch, I wouldn't have a problem with moving all the rest to Returned With Feedback or the next CF. None of the others seem to me to be close-to-committable (with the possible exception of the Max LSN patch, which I've not looked at), and April is not the time to be doing development. I haven't looked through too many of them. as for the one I have on there myself, the pg_retainxlog one, i think the consensus was to turn it into a documentation patch - and we can keep adding doc patches into beta. But, to the point. It might be a decent starting point to at least move all patches that are waiting on author and doesn't get almost daily status updates (or has a very clear statement from the author on when an update can be expected). That'll cut the list down quite a bit I think, and should make it easier to push harder on the others perhaps. Next week is going to be tied up with the back-branch releases, but maybe we could target beta for the week after? The main gating factor at this point really would be how quickly we could write some draft release notes, so people know what to test. I think we need to give it at least one more week before we can target a beta. Packagers will if anything have *more* work to deal with these backbranch releases than usual, and asking them to push another one just one (or even two) weeks after that would be rushing it unnecessarily... Of course, that doesn't prevent from starting work on the release notes meanwhile :) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting to 9.3 beta
On 2013-03-29 10:15:42 -0400, Bruce Momjian wrote: Our final 9.3 commit-fest has has exceeded the two-month mark, so it is time to start targeting a date to close it and get to 9.3 beta. I see 25 items will needing attention before we can close it: Very much agreed! https://commitfest.postgresql.org/action/commitfest_view?id=17 What is a reasonable timeframe to target for completion of these items? Here's my take on it: - Extension templates: Not ready yet, probably takes a bit more work till committable, last version got in rather late. = move to next fest - Performance Improvement by reducing WAL for Update Operation: Not ready yet, some redesigns are recently made, benefits aren't all that great = move to next fest - Index based regexp search for pg_trgm: Seems like the patch is undergoing restructuring of the regex access API = move to next fest - Truncate trailing nulls from heap rows to reduce the size of the null bitmap: Not sure, benefits don't seem to be all that big for realistic usecases? = move or reject? - allowing privileges on untrusted languages: Direction forward seems unclear, discussion required = move - replace plugins directory with GUC Peter has agreed to boot it to the next fest afaics (515357f4.6000...@gmx.net) = move (done) - sepgsql: name qualified default security label has been committed by robert, updated CF - sepgsql: db_schema:search permission and sepgsql: db_procedure:execute permission: Not much review afaics. = looks unfair, but unless some comitter (robert?) takes it on I don't see much alternative to booting it to the next fest? - Patch to compute Max LSN of Data Pages: I *personally* don't really see a point in including it in postgres, there doesn't really seem to be any real demand met by the tool = reject? - Make recovery.conf parameters into GUCs: Nice to see progress, but this seems to be too late for 9.3. = move to -next - pg_retainxlog for contrib: 5102f7c9.6050...@gmx.net sums up my opinion of the utility, just doesn't seem to be reliable enough to be distributed with postgres. = reject/redesign? - REINDEX CONCURRENTLY: Imo pretty close to being comittable and pretty useful, but it got redesigned pretty late and it mostly had review from me and fujii and it could use a bit more input = unclear - built-in/SQL Command to edit postgresql.conf (SET PERSISTENT patch): Different patches afloat, none is really ready. I personally think Zoltan's patch is more realistic, but ... = move - pg_ctl idempotent option: looks ready to me = commit - New statistics for WAL buffer dirty writes: waiting on other for months = returned with feedback - pg_stat_statements: query, session, and eviction identification: Seems to need at least docs = wait for author, seems to be easy enough? - Json API and extraction functions: Seems to be committable = commit - psql watch Waiting on author since 4 days = wait or boot? - User control over psql's error stream Unclear, there doesn't seem to be much progress and review = return with feedback - plpgsql_check_function: Tom says (27661.1364267...@sss.pgh.pa.us) that even if the approach can be aggreed uppon it needs quite a bit more work = move - transforms: I am not sure what solution is proposed for the dependency issue between shared objects. There also hasn't been code level review for a while... = unclear - Check file parameters to psql before prompt for password: No answer to Stephen's review in december = return with feedback - pkg-config for libpq and ecpg: Issues seem to be fixable easy enough = wait Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting to 9.3 beta
On 03/29/2013 11:05 AM, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Our final 9.3 commit-fest has has exceeded the two-month mark, so it is time to start targeting a date to close it and get to 9.3 beta. I see 25 items will needing attention before we can close it: https://commitfest.postgresql.org/action/commitfest_view?id=17 What is a reasonable timeframe to target for completion of these items? TBH, once Andrew commits the JSON patch, I wouldn't have a problem with moving all the rest to Returned With Feedback or the next CF. I can do that pretty much right away. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting to 9.3 beta
Magnus Hagander mag...@hagander.net writes: On Fri, Mar 29, 2013 at 4:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: Next week is going to be tied up with the back-branch releases, but maybe we could target beta for the week after? The main gating factor at this point really would be how quickly we could write some draft release notes, so people know what to test. I think we need to give it at least one more week before we can target a beta. Sure, I was just suggesting a best-case scenario. Probably a more realistic thing would be to target beta wrap for April 15, which would leave an extra week or so for documentation polishing and alpha-level testing. Of course, that doesn't prevent from starting work on the release notes meanwhile :) Right ;-) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting to 9.3 beta
Andres Freund and...@2ndquadrant.com writes: On 2013-03-29 10:15:42 -0400, Bruce Momjian wrote: What is a reasonable timeframe to target for completion of these items? Here's my take on it: Thanks for annotating these! I've commented on the ones where I come to a different conclusion: - replace plugins directory with GUC Peter has agreed to boot it to the next fest afaics (515357f4.6000...@gmx.net) = move (done) It looks to me more like a RWF situation, since Peter has evidently lost interest in committing this feature at all. So I marked it that way. - Patch to compute Max LSN of Data Pages: I *personally* don't really see a point in including it in postgres, there doesn't really seem to be any real demand met by the tool = reject? Not sure. It certainly hasn't drawn that much enthusiasm on the list, but it strikes me as the sort of thing that when you need it you really need it. It would be interesting to hear opinions about it from people who deal with data recovery situations on a regular basis. - REINDEX CONCURRENTLY: Imo pretty close to being comittable and pretty useful, but it got redesigned pretty late and it mostly had review from me and fujii and it could use a bit more input = unclear I think this really has no hope of being bulletproof until we have MVCC-based catalog scans. The work up to now has been good exploratory effort, but it's not going to be committable without that infrastructure, IMHO anyway. - psql watch Waiting on author since 4 days = wait or boot? If there is agreement on the basic design of the feature, ie \watch [n] as a repeating version of \g, it seems to me that this could be fixed and committed in a day or less. I'd be willing to look at it sometime next week. - transforms: I am not sure what solution is proposed for the dependency issue between shared objects. There also hasn't been code level review for a while... = unclear Needs to be moved; no way this is ready in the next week or two. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting to 9.3 beta
On Fri, Mar 29, 2013 at 11:05:46AM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Our final 9.3 commit-fest has has exceeded the two-month mark, so it is time to start targeting a date to close it and get to 9.3 beta. I see 25 items will needing attention before we can close it: https://commitfest.postgresql.org/action/commitfest_view?id=17 What is a reasonable timeframe to target for completion of these items? TBH, once Andrew commits the JSON patch, I wouldn't have a problem with moving all the rest to Returned With Feedback or the next CF. None of the others seem to me to be close-to-committable (with the possible exception of the Max LSN patch, which I've not looked at), and April is not the time to be doing development. Next week is going to be tied up with the back-branch releases, but maybe we could target beta for the week after? The main gating factor at this point really would be how quickly we could write some draft release notes, so people know what to test. I have been clearing my calendar with plans to do the 9.3 release notes, so I am ready to start. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting to 9.3 beta
On 2013-03-29 12:28:59 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-03-29 10:15:42 -0400, Bruce Momjian wrote: What is a reasonable timeframe to target for completion of these items? Here's my take on it: Thanks for annotating these! I've commented on the ones where I come to a different conclusion: - replace plugins directory with GUC Peter has agreed to boot it to the next fest afaics (515357f4.6000...@gmx.net) = move (done) It looks to me more like a RWF situation, since Peter has evidently lost interest in committing this feature at all. So I marked it that way. Yea, after a second reading of Peter's email I agree. - Patch to compute Max LSN of Data Pages: I *personally* don't really see a point in including it in postgres, there doesn't really seem to be any real demand met by the tool = reject? Not sure. It certainly hasn't drawn that much enthusiasm on the list, but it strikes me as the sort of thing that when you need it you really need it. It would be interesting to hear opinions about it from people who deal with data recovery situations on a regular basis. I only have done so infrequently, but I guess my real problem is that I can't see it being all that helpful in such a situation. If you've lost pg_xlog - otherwise there doesn't seem to be much use in this - you can just use pg_resetxlog to set an insanely high lsn and then dump restore the database. The only reason to use something more complex than this is if you want to continue using the cluster which seems like a really bad idea. Am I missing a usecase here? I guess it can be useful as a template utility to start hacking on for more complex scenarios, but I don't think that warrants /contrib. - REINDEX CONCURRENTLY: Imo pretty close to being comittable and pretty useful, but it got redesigned pretty late and it mostly had review from me and fujii and it could use a bit more input = unclear I think this really has no hope of being bulletproof until we have MVCC-based catalog scans. The work up to now has been good exploratory effort, but it's not going to be committable without that infrastructure, IMHO anyway. I think its fair not to commit it in 9.3, the patch came late and has undergone significant issues since then. And there are some things in it I am not sure about yet. But I am not sure why its only acceptable with MVCC scans? Sure, it takes an exclusive lock for a very short time when switching the relations, but thats still a huge step up over the status quo: * toast indexes currently cannot be created/dropped at all. So you have to reindex the table which will need to be locked over the whole time. * you currently cannot replace indexes that are referenced by foreign keys without doing manual catalog hackery which is definitely unsafe * you cannot create constraints that use an index concurrently. Parts of that is emulatable by creating the constraint with an existing index, but that doesn't help e.g. for exclusion constraints. * there's no convenient way to issue CREATE INDEX CONCURRENLTY new_index; DROP INDEX CONCURRENTLY old_index; RENAME new_index old_index; for all indexes. The last alone imnsho is already a good enough usecase for the patch. Also, the part that deals with the (locked) switch of the relfilenodes is a really small part of the patch, improving that later is not going to throw away much. Imo having this would have made e.g. beb850e1d873f8920a78b9b9ee27e9f87c95592f way much less painful. We (as in 2ndq) had to write scripts for customers to deal with this in a remotely sane manner. And those were *ugly*. - psql watch Waiting on author since 4 days = wait or boot? If there is agreement on the basic design of the feature, ie \watch [n] as a repeating version of \g, it seems to me that this could be fixed and committed in a day or less. I'd be willing to look at it sometime next week. Cool. - transforms: I am not sure what solution is proposed for the dependency issue between shared objects. There also hasn't been code level review for a while... = unclear Needs to be moved; no way this is ready in the next week or two. Good. I haven't looked at it in a long time and nobody else did, so I just wasn't sure. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting to 9.3 beta
Hello - plpgsql_check_function: Tom says (27661.1364267...@sss.pgh.pa.us) that even if the approach can be aggreed uppon it needs quite a bit more work = move Can we talk about this patch little bit more before moving to next commitfest? I would to have some plan to next commitfest. We have to remove some wanted functionality - possibility to identify more issues in one run, remove warnings, ... or we have significantly refactor plpgsql parser (two stages) Next possibility - introduce some new API and move plpgsql_check_function to external module, although I am thinking so this is important and very missing functionality (and should be in core) still. There is no reply, comments to my last update - where I rewrote output, what was mayor Tom's objection (well specified). Regards Pavel
Re: [HACKERS] Getting to 9.3 beta
On 2013/03/30, at 2:33, Andres Freund and...@2ndquadrant.com wrote: On 2013-03-29 12:28:59 -0400, Tom Lane - REINDEX CONCURRENTLY: Imo pretty close to being comittable and pretty useful, but it got redesigned pretty late and it mostly had review from me and fujii and it could use a bit more input = unclear I think this really has no hope of being bulletproof until we have MVCC-based catalog scans. The work up to now has been good exploratory effort, but it's not going to be committable without that infrastructure, IMHO anyway. I think its fair not to commit it in 9.3, the patch came late and has undergone significant issues since then. And there are some things in it I am not sure about yet. But I am not sure why its only acceptable with MVCC scans? Sure, it takes an exclusive lock for a very short time when switching the relations, but thats still a huge step up over the status quo: * toast indexes currently cannot be created/dropped at all. So you have to reindex the table which will need to be locked over the whole time. * you currently cannot replace indexes that are referenced by foreign keys without doing manual catalog hackery which is definitely unsafe * you cannot create constraints that use an index concurrently. Parts of that is emulatable by creating the constraint with an existing index, but that doesn't help e.g. for exclusion constraints. * there's no convenient way to issue CREATE INDEX CONCURRENLTY new_index; DROP INDEX CONCURRENTLY old_index; RENAME new_index old_index; for all indexes. Thanks for the detailed explanation! Just adding that this patch needs more review and attention especially for the toast part where multiple indexes are handled. As it looks that it is too late for 9.3, well let's move it to the next commit fest and work on that in the next release cycle. Thanks, -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting to 9.3 beta
On Fri, Mar 29, 2013 at 8:22 AM, Andres Freund and...@2ndquadrant.com wrote: - pg_stat_statements: query, session, and eviction identification: Seems to need at least docs = wait for author, seems to be easy enough? I would have responded by now, but recent events have unfortunately made me put a lot of things that take longer than fifteen minutes on hold. Still, I am watching. Like you say, it's not terribly invasive. The under-estimation error propagation is perhaps a bit too weird to justify (even though it's a simple implementation). -- fdr -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Getting to 9.3 beta
On Friday, March 29, 2013 11:04 PM Andres Freund wrote: On 2013-03-29 12:28:59 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-03-29 10:15:42 -0400, Bruce Momjian wrote: What is a reasonable timeframe to target for completion of these items? Here's my take on it: Thanks for annotating these! I've commented on the ones where I come to a different conclusion: - replace plugins directory with GUC Peter has agreed to boot it to the next fest afaics (515357f4.6000...@gmx.net) = move (done) It looks to me more like a RWF situation, since Peter has evidently lost interest in committing this feature at all. So I marked it that way. Yea, after a second reading of Peter's email I agree. - Patch to compute Max LSN of Data Pages: I *personally* don't really see a point in including it in postgres, there doesn't really seem to be any real demand met by the tool = reject? Not sure. It certainly hasn't drawn that much enthusiasm on the list, but it strikes me as the sort of thing that when you need it you really need it. It would be interesting to hear opinions about it from people who deal with data recovery situations on a regular basis. I only have done so infrequently, but I guess my real problem is that I can't see it being all that helpful in such a situation. If you've lost pg_xlog - otherwise there doesn't seem to be much use in this - you can just use pg_resetxlog to set an insanely high lsn and then dump restore the database. This utility can give user value that is okay to proceed. The only reason to use something more complex than this is if you want to continue using the cluster which seems like a really bad idea. I agree that mostly user doesn't need to use cluster further, but incase if he is aware that only some part of the database (particular tablespace) has problem and he can take dump of that part of database and redo some of lost operations on other tablespaces. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
