Re: [HACKERS] Last gasp
On Thu, Apr 12, 2012 at 05:49, Tom Lane wrote: > Robert Haas writes: >> On Wed, Apr 11, 2012 at 5:36 PM, Peter Eisentraut wrote: >>> I'd still review it, but I'd be able to spend say 3 minutes on review >>> and 30 seconds on committing it, versus 3 minutes on review, 3 minutes >>> on research, and 8 minutes on bookkeeping. > >> Well, I am not averse to figuring out a better workflow, or some >> better tools. In practice, I think it's going to be hard to reduce >> the time to review a trivial patch much below 5-10 minutes, which is >> what it takes me now, because you've got to read the email, download >> the patch, check that it doesn't break the build, review, commit, and >> push, and I can't really see any of those steps going away. But that >> doesn't mean we shouldn't make the attempt, because I've got to admit >> that the current workflow seems a little cumbersome to me, too. I'm >> not sure I have a better idea, though. git remotes seem useful for >> collaborating on topic branches, but I don't think they can really be >> expected to save much of anything during the final commit process - >> which is basically all the process there is, when the patch is >> trivial. > >> Now what would be sort of neat is if we had a way to keep all the >> versions of patch X plus author and reviewer information, links to >> reviews and discussion, etc. in some sort of centralized place. The >> CommitFest app was actually designed to track a lot of this >> information, but it's obviously not completely succeeding in tracking >> everything that people care about - it only contains links to patches >> and not patches themselves; it doesn't have any place to store >> proposed commit messages; etc. There might be room for improvement >> there, although getting consensus on what improvement looks like may >> not be totally straightforward, since I think Tom's ideal process for >> submitting a patch starts with attaching a file to an email and many >> other people I think would like to see it start with a pull request. >> This is not entirely a tools issue, of course, but it's in there >> somewhere. > > It strikes me that there are two different scenarios being discussed > here, and we'd better be sure we keep them straight: small-to-trivial > patches, and complex patches. > > I think that for the trivial case, what we need is less tooling not more. > Entering a patch in the CF app, updating and closing it will add a > not-small percentage to the total effort required to deal with a small > patch (as Peter already noted, and he wasn't even counting the time to > put the patch into CF initially). The only reason to even consider > doing that is to make sure the patch doesn't get forgotten. Perhaps > we could have some lighter-weight method of tracking such things? If we were actually using git branches for it, the CF app could automatically close entries when they were committed. But that requires them to be committed *unmodified*, and I'm not sure that's reasonable. I also think requiring a git branch for the *simple* changes is adding more tooling and not less, and thus fails on that suggestion. It might be helpful (if the CF app had a trivial API) with a small tool that could run from a git hook (or manual script or alias) that would prompt for "which cf entry, if any, did this commit close"? > At the other end of the scale, I think it's true that the CF app could > be more helpful than it is for tracking the state of complex patches. > I don't really have any concrete suggestions, other than that I've > seen far too many cases where the latest version of a patch was not > linked into the CF entry. Somehow we've got to make that more robust. > Maybe the answer is to tie things more directly into git workflows, > though I'm not sure about details. I am concerned about losing > traceability of submissions if all that ever shows up in the list > archives is a URL. I've suggested before that it would be a good idea to be able to register a git repo + branch name in the commitfest app, and be able to track that. If it was smart enough to figure out that for something like github or bitbucket it could also add a web link (but keep the git link for whoever wants to pull it remotely) with the full differences to master, that would make *some* of those issues go away. (Certainly not all, it's not a magic solution, but I believe it would be a tool that could help). I've pretty much given up on that happening though... -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] how to create a non-inherited CHECK constraint in CREATE TABLE
Hi, Cumulative reaction to all the responses first: Whoa! :) I was under the impression that a majority of us felt that the current mechanism was inadequate. Also if you go through the nabble thread, the fact that CREATE TABLE did not support such constraints was considered to be an annoyance. And I was enquired if/when I can provide this functionality. Apologies though with the timing. > > +1 for fixing up the syntax before 9.2 goes out the door. I think the > > original syntax was misguided to begin with. > > Well, it was fine in isolation, but once you consider how to make CREATE > TABLE do this too, it's hard to avoid the conclusion that you need to > attach the modifier to the CHECK constraint not the ALTER TABLE command. > > Yeah, exactly. > > CHECK NO INHERIT sounds fine to me; will that display ALTER TABLE ONLY > > x as the one true way of doing this? > > s/display/displace/, I think you meant? Yeah, that's what I understand > the proposal to be. > > Displace yes. It would error out if someone says ALTER TABLE ONLY... CHECK (); suggesting to use the ONLY with the CHECK. This patch does this and also makes both CREATE TABLE and ALTER TABLE use it in a uniform manner. Regarding "NO INHERIT" versus "ONLY", we again have had discussions on the longish original thread quite a while back: http://postgresql.1045698.n5.nabble.com/Check-constraints-on-partition-parents-only-tt464.html But now if we prefer NO INHERIT, I can live with that. Regards, Nikhils
Re: [HACKERS] Last gasp
Robert Haas writes: > On Wed, Apr 11, 2012 at 5:36 PM, Peter Eisentraut wrote: >> I'd still review it, but I'd be able to spend say 3 minutes on review >> and 30 seconds on committing it, versus 3 minutes on review, 3 minutes >> on research, and 8 minutes on bookkeeping. > Well, I am not averse to figuring out a better workflow, or some > better tools. In practice, I think it's going to be hard to reduce > the time to review a trivial patch much below 5-10 minutes, which is > what it takes me now, because you've got to read the email, download > the patch, check that it doesn't break the build, review, commit, and > push, and I can't really see any of those steps going away. But that > doesn't mean we shouldn't make the attempt, because I've got to admit > that the current workflow seems a little cumbersome to me, too. I'm > not sure I have a better idea, though. git remotes seem useful for > collaborating on topic branches, but I don't think they can really be > expected to save much of anything during the final commit process - > which is basically all the process there is, when the patch is > trivial. > Now what would be sort of neat is if we had a way to keep all the > versions of patch X plus author and reviewer information, links to > reviews and discussion, etc. in some sort of centralized place. The > CommitFest app was actually designed to track a lot of this > information, but it's obviously not completely succeeding in tracking > everything that people care about - it only contains links to patches > and not patches themselves; it doesn't have any place to store > proposed commit messages; etc. There might be room for improvement > there, although getting consensus on what improvement looks like may > not be totally straightforward, since I think Tom's ideal process for > submitting a patch starts with attaching a file to an email and many > other people I think would like to see it start with a pull request. > This is not entirely a tools issue, of course, but it's in there > somewhere. It strikes me that there are two different scenarios being discussed here, and we'd better be sure we keep them straight: small-to-trivial patches, and complex patches. I think that for the trivial case, what we need is less tooling not more. Entering a patch in the CF app, updating and closing it will add a not-small percentage to the total effort required to deal with a small patch (as Peter already noted, and he wasn't even counting the time to put the patch into CF initially). The only reason to even consider doing that is to make sure the patch doesn't get forgotten. Perhaps we could have some lighter-weight method of tracking such things? At the other end of the scale, I think it's true that the CF app could be more helpful than it is for tracking the state of complex patches. I don't really have any concrete suggestions, other than that I've seen far too many cases where the latest version of a patch was not linked into the CF entry. Somehow we've got to make that more robust. Maybe the answer is to tie things more directly into git workflows, though I'm not sure about details. I am concerned about losing traceability of submissions if all that ever shows up in the list archives is a URL. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 5:36 PM, Peter Eisentraut wrote: > On ons, 2012-04-11 at 14:29 -0400, Tom Lane wrote: >> I hear you ... but, given that the source material is a mailing-list >> thread, *somebody* has to do all that work to produce an acceptable >> commit. And if you're just going to commit what that somebody sends >> you without further review, then you might as well give that person a >> commit bit, because you're trusting them to get all this right. > > I'd still review it, but I'd be able to spend say 3 minutes on review > and 30 seconds on committing it, versus 3 minutes on review, 3 minutes > on research, and 8 minutes on bookkeeping. Well, I am not averse to figuring out a better workflow, or some better tools. In practice, I think it's going to be hard to reduce the time to review a trivial patch much below 5-10 minutes, which is what it takes me now, because you've got to read the email, download the patch, check that it doesn't break the build, review, commit, and push, and I can't really see any of those steps going away. But that doesn't mean we shouldn't make the attempt, because I've got to admit that the current workflow seems a little cumbersome to me, too. I'm not sure I have a better idea, though. git remotes seem useful for collaborating on topic branches, but I don't think they can really be expected to save much of anything during the final commit process - which is basically all the process there is, when the patch is trivial. Now what would be sort of neat is if we had a way to keep all the versions of patch X plus author and reviewer information, links to reviews and discussion, etc. in some sort of centralized place. The CommitFest app was actually designed to track a lot of this information, but it's obviously not completely succeeding in tracking everything that people care about - it only contains links to patches and not patches themselves; it doesn't have any place to store proposed commit messages; etc. There might be room for improvement there, although getting consensus on what improvement looks like may not be totally straightforward, since I think Tom's ideal process for submitting a patch starts with attaching a file to an email and many other people I think would like to see it start with a pull request. This is not entirely a tools issue, of course, but it's in there somewhere. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_upgrade improvements
On Wed, Apr 11, 2012 at 5:40 PM, Bruce Momjian wrote: > On Wed, Apr 04, 2012 at 07:26:58PM -0700, Harold Giménez wrote: > > There could be incoming connections for a number of > > reasons: either the user or the user's applications are reestablishing > > connections, or something like collectd on the localhost attempts to > > connect during that small window. > > Well, we did address this in PG 9.2 by having pg_upgrade use a > non-default port number when starting servers, 50432. You can do that > too in PG 9.1 by just specifying non-default port numbers when you run > pg_upgrade. We do start the server with a special --binary-upgrade > mode, and we could do all sorts of connection limits in that mode, but > having the port number be different seemed the clearest solution. > The non-default port number is a good solution, better than modifying pg_hba. Thanks for pointing that out! -Harold
Re: [HACKERS] pg_upgrade improvements
On Wed, Apr 04, 2012 at 07:26:58PM -0700, Harold Giménez wrote: > Hi all, > > I've written a pg_upgrade wrapper for upgrading our users (heroku) > to postgres 9.1. In the process I encountered a specific issue that > could easily be improved. We've had this process work consistently > for many users both internal and external, with the exception of just > a few for whom the process fails and required manual handholding. > > Before it performs the upgrade, the pg_upgrade program starts the > old cluster, does various checks, and then attempts to stop it. On > occasion stopping the cluster fails - I've posted command output > on a gist [1]. Manually running the pg_upgrade shortly afterwards > succeeds. We believe stopping the cluster times out because there > are other connections to the cluster that are established in that > small window. There could be incoming connections for a number of > reasons: either the user or the user's applications are reestablishing > connections, or something like collectd on the localhost attempts to > connect during that small window. Well, we did address this in PG 9.2 by having pg_upgrade use a non-default port number when starting servers, 50432. You can do that too in PG 9.1 by just specifying non-default port numbers when you run pg_upgrade. We do start the server with a special --binary-upgrade mode, and we could do all sorts of connection limits in that mode, but having the port number be different seemed the clearest solution. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_upgrade improvements
On Sat, Apr 07, 2012 at 01:13:23PM +0300, Peter Eisentraut wrote: > On ons, 2012-04-04 at 19:26 -0700, Harold Giménez wrote: > > It would also be nice if the invocation of pg_ctl didn't pipe its > > output to /dev/null. I'm sure it would contain information that would > > directly point at the root cause and could've saved some debugging and > > hand waving time. > > This aspect has been reworked in 9.2devel. Check it out to see if it > works better for you. Yes, we now spit out 5 log files if pg_upgrade fails, and tell you which one to look at. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] About bug #6579
I wrote:
> I've looked into this:
> http://archives.postgresql.org/pgsql-bugs/2012-04/msg00058.php
> and concluded that it's not very practical to fix it properly
> right now. A real fix will involve rearranging things so that
> construction of the filter-condition list happens at Path creation
> time, not createplan time, and that's a rather invasive change.
> So I want to put it off until 9.3.
... and on still further review, I've concluded that this isn't that
expensive to fix locally after all, at least in HEAD; and we get the
further benefit of saner costing of join cases. (As per attached.
Basically it'll cost us one list_difference_ptr operation per IndexPath,
on what will typically be pretty short lists. The cost_qual_eval
operation should be negligible either way, because it will be hitting
RestrictInfo nodes with already-cached costs.)
I'm still inclined to put the quick Min() hack into older branches,
though. While this larger fix could possibly be back-patched, it might
change cost estimates by enough to destabilize plan choices. Given
the small number of complaints about the issue to date, it doesn't seem
worth taking any risk for in released branches.
regards, tom lane
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 9cae27b99eb253362fddf6ec00e6f736a5243c52..0330f01531e5bc1ae2a98c807b60fa212b50ed17 100644
*** a/src/backend/optimizer/path/costsize.c
--- b/src/backend/optimizer/path/costsize.c
*** cost_index(IndexPath *path, PlannerInfo
*** 228,233
--- 228,234
IndexOptInfo *index = path->indexinfo;
RelOptInfo *baserel = index->rel;
bool indexonly = (path->path.pathtype == T_IndexOnlyScan);
+ List *allclauses;
Cost startup_cost = 0;
Cost run_cost = 0;
Cost indexStartupCost;
*** cost_index(IndexPath *path, PlannerInfo
*** 239,244
--- 240,246
spc_random_page_cost;
Cost min_IO_cost,
max_IO_cost;
+ QualCost qpqual_cost;
Cost cpu_per_tuple;
double tuples_fetched;
double pages_fetched;
*** cost_index(IndexPath *path, PlannerInfo
*** 267,274
* Note that we force the clauses to be treated as non-join clauses
* during selectivity estimation.
*/
- List *allclauses;
-
allclauses = list_union_ptr(baserel->baserestrictinfo,
path->indexclauses);
path->path.rows = baserel->tuples *
--- 269,274
*** cost_index(IndexPath *path, PlannerInfo
*** 283,288
--- 283,291
}
else
{
+ /* allclauses should just be the rel's restriction clauses */
+ allclauses = baserel->baserestrictinfo;
+
/*
* The number of rows is the same as the parent rel's estimate, since
* this isn't a parameterized path.
*** cost_index(IndexPath *path, PlannerInfo
*** 442,465
/*
* Estimate CPU costs per tuple.
*
! * Normally the indexquals will be removed from the list of restriction
! * clauses that we have to evaluate as qpquals, so we should subtract
! * their costs from baserestrictcost. But if we are doing a join then
! * some of the indexquals are join clauses and shouldn't be subtracted.
! * Rather than work out exactly how much to subtract, we don't subtract
! * anything.
*/
! startup_cost += baserel->baserestrictcost.startup;
! cpu_per_tuple = cpu_tuple_cost + baserel->baserestrictcost.per_tuple;
!
! if (path->path.required_outer == NULL)
! {
! QualCost index_qual_cost;
! cost_qual_eval(&index_qual_cost, path->indexquals, root);
! /* any startup cost still has to be paid ... */
! cpu_per_tuple -= index_qual_cost.per_tuple;
! }
run_cost += cpu_per_tuple * tuples_fetched;
--- 445,467
/*
* Estimate CPU costs per tuple.
*
! * What we want here is cpu_tuple_cost plus the evaluation costs of any
! * qual clauses that we have to evaluate as qpquals. We approximate that
! * list as allclauses minus any clauses appearing in indexquals (as
! * before, assuming that pointer equality is enough to recognize duplicate
! * RestrictInfos). This method neglects some considerations such as
! * clauses that needn't be checked because they are implied by a partial
! * index's predicate. It does not seem worth the cycles to try to factor
! * those things in at this stage, even though createplan.c will take pains
! * to remove such unnecessary clauses from the qpquals list if this path
! * is selected for use.
*/
! cost_qual_eval(&qpqual_cost,
! list_difference_ptr(allclauses, path->indexquals),
! root);
! startup_cost += qpqual_cost.startup;
! cpu_per_tuple = cpu_tuple_cost + qpqual_cost.per_tuple;
run_cost += cpu_per_tuple * tuples_fetched;
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] About bug #6579
I've looked into this: http://archives.postgresql.org/pgsql-bugs/2012-04/msg00058.php and concluded that it's not very practical to fix it properly right now. A real fix will involve rearranging things so that construction of the filter-condition list happens at Path creation time, not createplan time, and that's a rather invasive change. So I want to put it off until 9.3. However, I did think of a simple one-line hack we could apply to mask the worst effects of the bogus estimate, which is just to clamp the correction factor from the indexquals to be not more than the original cost estimate for the baserestrict quals, at line 461 in HEAD's costsize.c: - cpu_per_tuple -= index_qual_cost.per_tuple; + cpu_per_tuple -= Min(index_qual_cost.per_tuple, +baserel->baserestrictcost.per_tuple); This seems safe and back-patchable. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On ons, 2012-04-11 at 14:29 -0400, Tom Lane wrote: > I hear you ... but, given that the source material is a mailing-list > thread, *somebody* has to do all that work to produce an acceptable > commit. And if you're just going to commit what that somebody sends > you without further review, then you might as well give that person a > commit bit, because you're trusting them to get all this right. I'd still review it, but I'd be able to spend say 3 minutes on review and 30 seconds on committing it, versus 3 minutes on review, 3 minutes on research, and 8 minutes on bookkeeping. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Columns of pg_stat_activity
On Wed, Apr 11, 2012 at 05:14:51PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Wed, Apr 11, 2012 at 09:50:43PM +0100, Thom Brown wrote: > >> On 11 April 2012 21:46, Bruce Momjian wrote: > >>> Arguably: > >>>backend_start -> session_start > >>>query_start -> statment_start > > >> Sounds like a lot of potential breakage to solve something I don't > >> think is a problem. Besides, isn't the door for 9.2 changes now > >> closed and bolted? > > We do still have open issues that include such proposed changes, > so I'd say that "too late" isn't a good argument. However ... > > > Well, we renamed procpid -> pid and I noticed these others. Not sure if > > it is a win or not, but just asking. > > We were talking about renaming columns if we changed their semantics. > I don't think renaming for the sake of a slightly cleaner name will > win us any friends. The "procpid" change was for accuracy, I guess. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Columns of pg_stat_activity
On Wed, Apr 11, 2012 at 11:11:18PM +0200, Magnus Hagander wrote: > >> > Should we make any of these changes? > >> > >> Sounds like a lot of potential breakage to solve something I don't > >> think is a problem. Besides, isn't the door for 9.2 changes now > >> closed and bolted? > > > > Well, we renamed procpid -> pid and I noticed these others. Not sure if > > it is a win or not, but just asking. > > We also renamed current_query -> query, but that was mainly because it > actually changed meaning. > > But. Since we already whacked around procpid->pid, yes, if we're ever > going to change those, now is the time, really. > > I think at least backend_start -> session_start would make sense. > > Not sure about the other one - what's wrong with query_start? We consistently use "statement" for commands, not "queries", because some feel query means SELECT. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Columns of pg_stat_activity
Bruce Momjian writes: > On Wed, Apr 11, 2012 at 09:50:43PM +0100, Thom Brown wrote: >> On 11 April 2012 21:46, Bruce Momjian wrote: >>> Arguably: >>>backend_start -> session_start >>>query_start -> statment_start >> Sounds like a lot of potential breakage to solve something I don't >> think is a problem. Besides, isn't the door for 9.2 changes now >> closed and bolted? We do still have open issues that include such proposed changes, so I'd say that "too late" isn't a good argument. However ... > Well, we renamed procpid -> pid and I noticed these others. Not sure if > it is a win or not, but just asking. We were talking about renaming columns if we changed their semantics. I don't think renaming for the sake of a slightly cleaner name will win us any friends. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Columns of pg_stat_activity
On Wed, Apr 11, 2012 at 23:04, Bruce Momjian wrote: > On Wed, Apr 11, 2012 at 09:50:43PM +0100, Thom Brown wrote: >> On 11 April 2012 21:46, Bruce Momjian wrote: >> > Since we are wacking around pg_stat_activity for 9.2, what do people >> > think about these column names? >> > >> > backend_start | timestamp with time zone | >> > xact_start | timestamp with time zone | >> > query_start | timestamp with time zone | >> > >> > Arguably: >> > >> > backend_start -> session_start >> > query_start -> statment_start >> > >> > Should we make any of these changes? >> >> Sounds like a lot of potential breakage to solve something I don't >> think is a problem. Besides, isn't the door for 9.2 changes now >> closed and bolted? > > Well, we renamed procpid -> pid and I noticed these others. Not sure if > it is a win or not, but just asking. We also renamed current_query -> query, but that was mainly because it actually changed meaning. But. Since we already whacked around procpid->pid, yes, if we're ever going to change those, now is the time, really. I think at least backend_start -> session_start would make sense. Not sure about the other one - what's wrong with query_start? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] man pages for contrib programs
On 11 April 2012 21:58, Peter Eisentraut wrote: > On ons, 2012-04-11 at 21:42 +0100, Thom Brown wrote: >> Could you clarify what you're defining to be a client application and >> a server application? This could be confusing as we already have >> sections under Reference called "PostgreSQL Client Applications" and >> "PostgreSQL Server Applications", visible in the root table of >> contents. >> > By the same criteria as the main reference: client applications can run > anywhere and connect to a server, server applications run on the same > host as the database server. Fair enough. So will you be classifying things like auto_explain and auth_delay as "extensions"? (i.e. things which aren't installed via CREATE EXTENSION) -- Thom -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Columns of pg_stat_activity
On Wed, Apr 11, 2012 at 09:50:43PM +0100, Thom Brown wrote: > On 11 April 2012 21:46, Bruce Momjian wrote: > > Since we are wacking around pg_stat_activity for 9.2, what do people > > think about these column names? > > > > backend_start | timestamp with time zone | > > xact_start | timestamp with time zone | > > query_start | timestamp with time zone | > > > > Arguably: > > > > backend_start -> session_start > > query_start -> statment_start > > > > Should we make any of these changes? > > Sounds like a lot of potential breakage to solve something I don't > think is a problem. Besides, isn't the door for 9.2 changes now > closed and bolted? Well, we renamed procpid -> pid and I noticed these others. Not sure if it is a win or not, but just asking. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] man pages for contrib programs
On ons, 2012-04-11 at 21:42 +0100, Thom Brown wrote: > > G. Additional Supplied Applications > > > > with two subsections Client and Server Applications, and one refentry > > per application. That would end up looking much like the SPI chapter. > > Could you clarify what you're defining to be a client application and > a server application? This could be confusing as we already have > sections under Reference called "PostgreSQL Client Applications" and > "PostgreSQL Server Applications", visible in the root table of > contents. > By the same criteria as the main reference: client applications can run anywhere and connect to a server, server applications run on the same host as the database server. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Columns of pg_stat_activity
On 11 April 2012 21:46, Bruce Momjian wrote: > Since we are wacking around pg_stat_activity for 9.2, what do people > think about these column names? > > backend_start | timestamp with time zone | > xact_start | timestamp with time zone | > query_start | timestamp with time zone | > > Arguably: > > backend_start -> session_start > query_start -> statment_start > > Should we make any of these changes? Sounds like a lot of potential breakage to solve something I don't think is a problem. Besides, isn't the door for 9.2 changes now closed and bolted? -- Thom -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Columns of pg_stat_activity
Since we are wacking around pg_stat_activity for 9.2, what do people think about these column names? backend_start| timestamp with time zone | xact_start | timestamp with time zone | query_start | timestamp with time zone | Arguably: backend_start -> session_start query_start -> statment_start Should we make any of these changes? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] man pages for contrib programs
On 11 April 2012 21:29, Peter Eisentraut wrote: > On ons, 2012-04-04 at 21:53 +0300, Peter Eisentraut wrote: >> I think it would be useful to split this up into three sections: > >> F.1. Extensions >> F.2. Client Applications >> F.3. Server Applications > >> where the first looks like now and the other two contain the refentry >> pages. > >> We could also consider making two separate appendixes. Maybe that >> would result in a better table of contents. > > I've played around with this a little bit to see how the tables of > contents etc. turn out. I think the best approach is to have two > appendixes > > F. Additional Supplied Extensions > > with one sect1 per extension, like now, and > > G. Additional Supplied Applications > > with two subsections Client and Server Applications, and one refentry > per application. That would end up looking much like the SPI chapter. Could you clarify what you're defining to be a client application and a server application? This could be confusing as we already have sections under Reference called "PostgreSQL Client Applications" and "PostgreSQL Server Applications", visible in the root table of contents. -- Thom -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] man pages for contrib programs
On ons, 2012-04-04 at 21:53 +0300, Peter Eisentraut wrote: > I think it would be useful to split this up into three sections: > F.1. Extensions > F.2. Client Applications > F.3. Server Applications > where the first looks like now and the other two contain the refentry > pages. > We could also consider making two separate appendixes. Maybe that > would result in a better table of contents. I've played around with this a little bit to see how the tables of contents etc. turn out. I think the best approach is to have two appendixes F. Additional Supplied Extensions with one sect1 per extension, like now, and G. Additional Supplied Applications with two subsections Client and Server Applications, and one refentry per application. That would end up looking much like the SPI chapter. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] how to create a non-inherited CHECK constraint in CREATE TABLE
Robert Haas writes: > +1 for fixing up the syntax before 9.2 goes out the door. I think the > original syntax was misguided to begin with. Well, it was fine in isolation, but once you consider how to make CREATE TABLE do this too, it's hard to avoid the conclusion that you need to attach the modifier to the CHECK constraint not the ALTER TABLE command. > CHECK NO INHERIT sounds fine to me; will that display ALTER TABLE ONLY > x as the one true way of doing this? s/display/displace/, I think you meant? Yeah, that's what I understand the proposal to be. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] how to create a non-inherited CHECK constraint in CREATE TABLE
On Wed, Apr 11, 2012 at 2:45 PM, Tom Lane wrote: > Alvaro Herrera writes: >> Excerpts from Nikhil Sontakke's message of mié abr 11 15:07:45 -0300 2012: >>> This patch removes the support for : >>> >>> ALTER TABLE ONLY constraint_rename_test ADD CONSTRAINT con2 CHECK (b > 0); >>> >>> and uses >>> >>> ALTER TABLE constraint_rename_test ADD CONSTRAINT con2 CHECK ONLY (b > 0); >>> >>> I know it's a bit late in the commitfest, but if this patch makes this >>> feature more "complete", maybe we should consider... > >> Personally I don't think we should consider this for 9.2. > > Well, if we're going to regret having offered the other syntax, now > would be the time to figure that out, before we ship it not after. > I would go so far as to say that if we don't accept this for 9.2 > we probably shouldn't accept it at all, because two different ways > to spell the same thing isn't nice. +1 for fixing up the syntax before 9.2 goes out the door. I think the original syntax was misguided to begin with. CHECK NO INHERIT sounds fine to me; will that display ALTER TABLE ONLY x as the one true way of doing this? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] how to create a non-inherited CHECK constraint in CREATE TABLE
On 04/11/2012 03:06 PM, Tom Lane wrote: I'd propose "CHECK NO INHERIT", though, as (a) it seems better English and (b) it avoids creating any new keyword. I could live with that too. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] how to create a non-inherited CHECK constraint in CREATE TABLE
Excerpts from Andrew Dunstan's message of mié abr 11 15:51:51 -0300 2012: > > On 04/11/2012 02:45 PM, Tom Lane wrote: > > Alvaro Herrera writes: > >> Excerpts from Nikhil Sontakke's message of mié abr 11 15:07:45 -0300 2012: > >>> This patch removes the support for : > >>> > >>> ALTER TABLE ONLY constraint_rename_test ADD CONSTRAINT con2 CHECK (b> 0); > >>> > >>> and uses > >>> > >>> ALTER TABLE constraint_rename_test ADD CONSTRAINT con2 CHECK ONLY (b> 0); > >>> > >>> I know it's a bit late in the commitfest, but if this patch makes this > >>> feature more "complete", maybe we should consider... > >> Personally I don't think we should consider this for 9.2. > > Well, if we're going to regret having offered the other syntax, now > > would be the time to figure that out, before we ship it not after. > > I would go so far as to say that if we don't accept this for 9.2 > > we probably shouldn't accept it at all, because two different ways > > to spell the same thing isn't nice. > > > > I don't really care for the idea that the ONLY goes in a different place > > for this operation than for every other kind of ALTER TABLE, but it does > > make sense if you subscribe to the quoted theory that ONLY is a property > > of the constraint and not the ALTER command as such. > > I think I rather dislike it. ONLY should be followed by the name of the > parent table whose children it causes us to exclude, IMNSHO. Moving it > elsewhere doesn't seem to me to be a blow for clarity at all. If that's the only objection, maybe we could use a different keyword then, perhaps NOINHERIT: ALTER TABLE constraint_rename_test ADD CONSTRAINT con2 CHECK NOINHERIT (b> 0); -- Álvaro Herrera The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] how to create a non-inherited CHECK constraint in CREATE TABLE
On 04/11/2012 02:45 PM, Tom Lane wrote: Alvaro Herrera writes: Excerpts from Nikhil Sontakke's message of mié abr 11 15:07:45 -0300 2012: This patch removes the support for : ALTER TABLE ONLY constraint_rename_test ADD CONSTRAINT con2 CHECK (b> 0); and uses ALTER TABLE constraint_rename_test ADD CONSTRAINT con2 CHECK ONLY (b> 0); I know it's a bit late in the commitfest, but if this patch makes this feature more "complete", maybe we should consider... Personally I don't think we should consider this for 9.2. Well, if we're going to regret having offered the other syntax, now would be the time to figure that out, before we ship it not after. I would go so far as to say that if we don't accept this for 9.2 we probably shouldn't accept it at all, because two different ways to spell the same thing isn't nice. I don't really care for the idea that the ONLY goes in a different place for this operation than for every other kind of ALTER TABLE, but it does make sense if you subscribe to the quoted theory that ONLY is a property of the constraint and not the ALTER command as such. I think I rather dislike it. ONLY should be followed by the name of the parent table whose children it causes us to exclude, IMNSHO. Moving it elsewhere doesn't seem to me to be a blow for clarity at all. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] how to create a non-inherited CHECK constraint in CREATE TABLE
Alvaro Herrera writes: > Excerpts from Nikhil Sontakke's message of mié abr 11 15:07:45 -0300 2012: >> This patch removes the support for : >> >> ALTER TABLE ONLY constraint_rename_test ADD CONSTRAINT con2 CHECK (b > 0); >> >> and uses >> >> ALTER TABLE constraint_rename_test ADD CONSTRAINT con2 CHECK ONLY (b > 0); >> >> I know it's a bit late in the commitfest, but if this patch makes this >> feature more "complete", maybe we should consider... > Personally I don't think we should consider this for 9.2. Well, if we're going to regret having offered the other syntax, now would be the time to figure that out, before we ship it not after. I would go so far as to say that if we don't accept this for 9.2 we probably shouldn't accept it at all, because two different ways to spell the same thing isn't nice. I don't really care for the idea that the ONLY goes in a different place for this operation than for every other kind of ALTER TABLE, but it does make sense if you subscribe to the quoted theory that ONLY is a property of the constraint and not the ALTER command as such. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] how to create a non-inherited CHECK constraint in CREATE TABLE
Excerpts from Nikhil Sontakke's message of mié abr 11 15:07:45 -0300 2012: > This patch removes the support for : > > ALTER TABLE ONLY constraint_rename_test ADD CONSTRAINT con2 CHECK (b > 0); > > and uses > > ALTER TABLE constraint_rename_test ADD CONSTRAINT con2 CHECK ONLY (b > 0); > > Is this what we want? Or we would want the earlier support in place for > backward compatibility as well? We are actually introducing this in 9.2 so > I guess we can remove this. I'm not quite following that logic. I don't think support for the previous syntax should be removed -- does it cause some serious problem? > This is a much cleaner implementation and we might not even need the > changes in pg_dump now because the pg_get_constraintdef can provide the > info about the ONLY part too. So some cleanup can be done if needed. > > I know it's a bit late in the commitfest, but if this patch makes this > feature more "complete", maybe we should consider... > > Thoughts? Personally I don't think we should consider this for 9.2. -- Álvaro Herrera The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Peter Eisentraut writes: > Just as a personal view, if people were to send me doc or "trivial" > patches in git-am format, with proper commit message, and Acked or > Signed-off etc. lines from recognized contributors, and proper > References: mail header linked to the discussion or "suggestion" > message, I could probably commit 20 of those in an hour. > Instead, I have to review the entire email thread for discussion, any > possible reviews or test runs, extract the patch from the email, mangle > it into proper form, apply it, think of a commit message, make sure I > register all the right people in the message, re-review the commit, > push, reply to email, optionally, log into commit fest, find the patch, > click a bunch of times, close it, done -- I think. That takes 15 > minutes per patch, and after two patches like that I'm tired. I hear you ... but, given that the source material is a mailing-list thread, *somebody* has to do all that work to produce an acceptable commit. And if you're just going to commit what that somebody sends you without further review, then you might as well give that person a commit bit, because you're trusting them to get all this right. So I'm not sure how this moves us forward, other than to the obvious observation that it'd be great if we had more qualified committers. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] [streaming replication] 9.1.3 streaming replication bug ?
On Wed, Apr 11, 2012 at 12:35 PM, Fujii Masao wrote: > On Thu, Apr 12, 2012 at 12:56 AM, Fujii Masao wrote: >> On Wed, Apr 11, 2012 at 3:31 PM, 乔志强 wrote: >>> So in sync streaming replication, if master delete WAL before sent to the >>> only standby, all transaction will fail forever, >>> "the master tries to avoid a PANIC error rather than termination of >>> replication." but in sync replication, termination of replication is THE >>> bigger PANIC error. >> >> I see your point. When there are backends waiting for replication, the WAL >> files >> which the standby might not have received yet must not be removed. If they >> are >> removed, replication keeps failing forever because required WAL files don't >> exist in the master, and then waiting backends will never be released unless >> replication mode is changed to async. This should be avoided. > > On second thought, we can avoid the issue by just increasing > wal_keep_segments enough. Even if the issue happens and some backends > get stuck to wait for replication, we can release them by taking fresh backup > and restarting the standby from that backup. This is the basic procedure to > restart replication after replication is terminated because required WAL files > are removed from the master. So this issue might not be worth implementing > the patch for now (though I'm not against improving things in the future), but > it seems just a tuning-problem of wal_keep_segments. We've talked about teaching the master to keep track of how far back all of its known standbys are, and retaining WAL back to that specific point, rather than the shotgun approach that is wal_keep_segments. It's not exactly clear what the interface to that should look like, though. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] how to create a non-inherited CHECK constraint in CREATE TABLE
Hi, So, I have a patch for this. This patch introduces support for CHECK ONLY syntax while doing a CREATE TABLE as well as during the usual ALTER TABLE command. Example: create table atacc7 (test int, test2 int CHECK ONLY (test>0), CHECK (test2>10)); create table atacc8 () inherits (atacc7); postgres=# \d+ atacc7 Table "public.atacc7" Column | Type | Modifiers | Storage | Description +-+---+-+- test | integer | | plain | test2 | integer | | plain | Check constraints: "atacc7_test2_check" CHECK (test2 > 10) "atacc7_test_check" CHECK ONLY (test > 0) Child tables: atacc8 Has OIDs: no postgres=# \d+ atacc8 Table "public.atacc8" Column | Type | Modifiers | Storage | Description +-+---+-+- test | integer | | plain | test2 | integer | | plain | Check constraints: "atacc7_test2_check" CHECK (test2 > 10) Inherits: atacc7 Has OIDs: no This patch removes the support for : ALTER TABLE ONLY constraint_rename_test ADD CONSTRAINT con2 CHECK (b > 0); and uses ALTER TABLE constraint_rename_test ADD CONSTRAINT con2 CHECK ONLY (b > 0); Is this what we want? Or we would want the earlier support in place for backward compatibility as well? We are actually introducing this in 9.2 so I guess we can remove this. This is a much cleaner implementation and we might not even need the changes in pg_dump now because the pg_get_constraintdef can provide the info about the ONLY part too. So some cleanup can be done if needed. I know it's a bit late in the commitfest, but if this patch makes this feature more "complete", maybe we should consider... Thoughts? P.S Here's the discussion thread in its entirety for reference: http://postgresql.1045698.n5.nabble.com/how-to-create-a-non-inherited-CHECK-constraint-in-CREATE-TABLE-td5152184.html Regards, Nikhils On Thu, Feb 2, 2012 at 1:32 AM, Peter Eisentraut wrote: > On ons, 2012-01-18 at 18:17 -0500, Robert Haas wrote: > > I agree with Peter that we should have we should have CHECK ONLY. > > ONLY is really a property of the constraint, not the ALTER TABLE > > command -- if it were otherwise, we wouldn't need to store it the > > system catalogs, but of course we do. The fact that it's not a > > standard property isn't a reason not to have proper syntax for it. > > Clearly, we will eventually want to support inherited and non-inherited > constraints of all types. Currently, each type of constraint has an > implicit default regarding this property: > > check - inherited > not null - inherited > foreign key - not inherited > primary key - not inherited > unique - not inherited > exclusion - not inherited > > As discussed above, we need to have a syntax that is attached to the > constraint, not the table operation that creates the constraint, so that > we can also create these in CREATE TABLE. > > How should we resolve these different defaults? > > Also, in ALTER TABLE, if you want to add either an inherited or not > inherited constraint to a parent table, you should really say ALTER > TABLE ONLY in either case. Because it's conceivably valid that ALTER > TABLE foo ADD CHECK () NOINHERIT would add an independent, not inherited > check constraint to each child table. > > So, there are all kinds of inconsistencies and backward compatibility > problems lurking here. We might need either a grand transition plan or > document the heck out of these inconsistencies. > > > check_constraint_create_table_support.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 1:39 PM, Joshua Berkus wrote: >> Ultimately, we're herding cats here. I don't think you're going to >> get >> the community to suddenly be willing to march in lockstep instead. > > If you, Peter, Simon, Robert, Heikki, Magnus, Peter G., Greg, Bruce and > Andrew agreed on a calendar-driven, mostly unambiguous process and adhered to > that process, then the one or two people who didn't follow along wouldn't > matter. Everyone else would follow you. The reason things are chaotic now > is that our lead committers do not have consensus and are even inconsistent > from CF to CF individually. > > In other words: the problem is only unsolvable because *you* think it's > unsolvable. If you decide the problem is solvable, you already have the > means to solve it. That's a somewhat bizarre list of people. It both includes people who haven't expressed many concerns about our process one way or the other and excludes some who have. At any rate, clearly the problem is exactly that there isn't consensus on this. I would generally say that Tom, Greg Smith, and I are pretty close together on this issue, and Peter G., Simon, and Dimitri are pretty close together on this issue, but with a big gap in between those two groups. I am less clear on how everyone else feels, but I think that saying that "all we need" is to get consensus among those people is to define the problem, not the solution. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch: bytea_agg
Peter Eisentraut writes: > On lör, 2012-04-07 at 10:38 -0400, Tom Lane wrote: >> Hm. So are you now suggesting we should get rid of one-argument >> bytea_agg and replace it with two-argument string_agg(bytea,bytea)? >> I could support that, since we've not released bytea_agg yet. > Yes, that looks like the best solution. Here is a patch for that. Looks sane in a quick once-over, except for the documentation entry. I'm not really thrilled with "text, text or bytea, bytea" because it seems easy to misparse. Moreover, as written the entry claims that the return type is text either way, which is wrong. You could fix the latter by writing "same as argument data type", but I wonder whether it'd be better to make separate table entries for the two forms. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] patch: bytea_agg
On lör, 2012-04-07 at 10:38 -0400, Tom Lane wrote:
> > Nevertheless, the problem would now be that adding string_agg(bytea)
> > would effectively forbid adding string_agg(bytea, delim) in the
> future.
> > So making a two-argument string_agg(bytea, bytea) now seems like the
> > best solution anyway. (This applies independently of the function
> > renaming, actually.)
>
> Hm. So are you now suggesting we should get rid of one-argument
> bytea_agg and replace it with two-argument string_agg(bytea,bytea)?
> I could support that, since we've not released bytea_agg yet.
Yes, that looks like the best solution. Here is a patch for that.
diff --git i/doc/src/sgml/func.sgml w/doc/src/sgml/func.sgml
index ae22ee5..131384c 100644
--- i/doc/src/sgml/func.sgml
+++ w/doc/src/sgml/func.sgml
@@ -10965,24 +10965,6 @@ SELECT NULLIF(value, '(none)') ...
-bytea_agg
-
-
- bytea_agg(expression)
-
-
-
- bytea
-
-
- bytea
-
- input values concatenated into a bytea
-
-
-
-
-
count
count(*)
@@ -11061,7 +11043,7 @@ SELECT NULLIF(value, '(none)') ...
- text, text
+ text, text or bytea, bytea
text
diff --git i/src/backend/utils/adt/varlena.c w/src/backend/utils/adt/varlena.c
index 65e9af8..a5592d5 100644
--- i/src/backend/utils/adt/varlena.c
+++ w/src/backend/utils/adt/varlena.c
@@ -397,7 +397,7 @@ byteasend(PG_FUNCTION_ARGS)
}
Datum
-bytea_agg_transfn(PG_FUNCTION_ARGS)
+bytea_string_agg_transfn(PG_FUNCTION_ARGS)
{
StringInfo state;
@@ -408,21 +408,28 @@ bytea_agg_transfn(PG_FUNCTION_ARGS)
{
bytea *value = PG_GETARG_BYTEA_PP(1);
+ /* On the first time through, we ignore the delimiter. */
if (state == NULL)
state = makeStringAggState(fcinfo);
+ else if (!PG_ARGISNULL(2))
+ {
+ bytea *delim = PG_GETARG_BYTEA_PP(2);
+
+ appendBinaryStringInfo(state, VARDATA_ANY(delim), VARSIZE_ANY_EXHDR(delim));
+ }
appendBinaryStringInfo(state, VARDATA_ANY(value), VARSIZE_ANY_EXHDR(value));
}
/*
- * The transition type for bytea_agg() is declared to be "internal",
+ * The transition type for string_agg() is declared to be "internal",
* which is a pass-by-value type the same size as a pointer.
*/
PG_RETURN_POINTER(state);
}
Datum
-bytea_agg_finalfn(PG_FUNCTION_ARGS)
+bytea_string_agg_finalfn(PG_FUNCTION_ARGS)
{
StringInfo state;
diff --git i/src/include/catalog/pg_aggregate.h w/src/include/catalog/pg_aggregate.h
index adda07c..461772c 100644
--- i/src/include/catalog/pg_aggregate.h
+++ w/src/include/catalog/pg_aggregate.h
@@ -229,7 +229,7 @@ DATA(insert ( 2335 array_agg_transfn array_agg_finalfn 0 2281 _null_ ));
DATA(insert ( 3538 string_agg_transfn string_agg_finalfn 0 2281 _null_ ));
/* bytea */
-DATA(insert ( 3545 bytea_agg_transfn bytea_agg_finalfn 0 2281 _null_ ));
+DATA(insert ( 3545 bytea_string_agg_transfn bytea_string_agg_finalfn 0 2281 _null_ ));
/*
* prototypes for functions in pg_aggregate.c
diff --git i/src/include/catalog/pg_proc.h w/src/include/catalog/pg_proc.h
index 6414b33..aa4d350 100644
--- i/src/include/catalog/pg_proc.h
+++ w/src/include/catalog/pg_proc.h
@@ -2433,11 +2433,11 @@ DATA(insert OID = 3536 ( string_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f
DESCR("aggregate final function");
DATA(insert OID = 3538 ( string_aggPGNSP PGUID 12 1 0 0 0 t f f f f f i 2 0 25 "25 25" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into a string");
-DATA(insert OID = 3543 ( bytea_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 2 0 2281 "2281 17" _null_ _null_ _null_ _null_ bytea_agg_transfn _null_ _null_ _null_ ));
+DATA(insert OID = 3543 ( bytea_string_agg_transfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 3 0 2281 "2281 17 17" _null_ _null_ _null_ _null_ bytea_string_agg_transfn _null_ _null_ _null_ ));
DESCR("aggregate transition function");
-DATA(insert OID = 3544 ( bytea_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 1 0 17 "2281" _null_ _null_ _null_ _null_ bytea_agg_finalfn _null_ _null_ _null_ ));
+DATA(insert OID = 3544 ( bytea_string_agg_finalfn PGNSP PGUID 12 1 0 0 0 f f f f f f i 1 0 17 "2281" _null_ _null_ _null_ _null_ bytea_string_agg_finalfn _null_ _null_ _null_ ));
DESCR("aggregate final function");
-DATA(insert OID = 3545 ( bytea_aggPGNSP PGUID 12 1 0 0 0 t f f f f f i 1 0 17 "17" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
+DATA(insert OID = 3545 ( string_aggPGNSP PGUID 12 1 0 0 0 t f f f f f i 2 0 17 "17 17" _null_ _null_ _null_ _null_ aggregate_dummy _null_ _null_ _null_ ));
DESCR("concatenate aggregate input into a bytea");
/* To ASCII conversion */
diff --git i/src/include/utils/builtins.h w/src/include/utils/builtins.h
index 9fda7ad..201b23e 100644
--- i/src/include/utils/builtins.h
+++ w/src/include/utils/built
Re: [HACKERS] Last gasp
On ons, 2012-04-11 at 12:48 -0400, Tom Lane wrote: > > However, the real criteria don't matter as much as coming up with a > set of criteria we're all willing to obey, whatever they are. > > Ultimately, we're herding cats here. I don't think you're going to > get the community to suddenly be willing to march in lockstep > instead. Yeah, I think what's forgotten in this discussion is that we're all volunteers. (Even those who have jobs related to this are still volunteers with respect to the project.) So people will do whatever they want. If you don't let them do what they want, they will do it elsewhere. Now it's fine to encourage a little bit of team spirit and community values here and there, but you can't build your process on making people do what they don't want to do. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
> Ultimately, we're herding cats here. I don't think you're going to > get > the community to suddenly be willing to march in lockstep instead. If you, Peter, Simon, Robert, Heikki, Magnus, Peter G., Greg, Bruce and Andrew agreed on a calendar-driven, mostly unambiguous process and adhered to that process, then the one or two people who didn't follow along wouldn't matter. Everyone else would follow you. The reason things are chaotic now is that our lead committers do not have consensus and are even inconsistent from CF to CF individually. In other words: the problem is only unsolvable because *you* think it's unsolvable. If you decide the problem is solvable, you already have the means to solve it. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On ons, 2012-04-11 at 06:04 -0400, Greg Smith wrote: > Compare with: > > -Submitter suggests doc change > -No one has a strong opinion on it, may not be picked up at all > -Submitter adds to the next CF > -Wait for review > -[Possible repost update with reviewer changes] > -Ready for committer > -Committer takes time away from code review to look at it > -Possibly another feedback/review resubmission > -Commit final versions I totally get that. Just as a personal view, if people were to send me doc or "trivial" patches in git-am format, with proper commit message, and Acked or Signed-off etc. lines from recognized contributors, and proper References: mail header linked to the discussion or "suggestion" message, I could probably commit 20 of those in an hour. Instead, I have to review the entire email thread for discussion, any possible reviews or test runs, extract the patch from the email, mangle it into proper form, apply it, think of a commit message, make sure I register all the right people in the message, re-review the commit, push, reply to email, optionally, log into commit fest, find the patch, click a bunch of times, close it, done -- I think. That takes 15 minutes per patch, and after two patches like that I'm tired. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 11:49 AM, Alvaro Herrera wrote: > Excerpts from Robert Haas's message of mié abr 11 12:44:02 -0300 2012: >> Me neither, but I don't know how far it scales. Having certain people >> who are defined as, say, doc-only committers will not only make it >> clear to those people what they're expected to commit, but also clear >> to everyone else who the people are who might commit any given patch >> they might write. If we just end up with 50 committers and you have >> to follow pgsql-hackers to understand who knows what and which people >> are even still around, it's not going to make anything easier for >> anyone. > > Since we're so keen on copying what Linux does, we could just have a > MAINTAINERS file. I'm actually not particularly keen on copying what Linux does. It seems that they have a lot of the same problems that we have, only worse. Unless you can get the proper person to take notice of your patch and merge it into his tree, from where it will get merged into somebody else's tree, from where it will eventually get merged into Linus's tree (maybe with one more tree in the path just for good measure), your patch is just going to fall on the floor and die. The lseek contention stuff in Linux 3.2 was submitted in substantially identical form years ago and didn't get merged - mostly, AFAICT, just because nobody cared about it enough. Our process is not perfect, but in recent years we have at least done a fairly good job preventing things from dying of apathy, even if not everybody agrees on which things ultimately should or should not have gotten committed. Small doc changes might be an exception. I used to apply those regularly, but I've had to fall back to making occasional sweeps of my pgsql-docs email box and looking for unresolved issues. Having a few more committers, and specifically people focused on documentation, would, I think, be a a step forward. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Joshua Berkus writes: > From my observation, the CF process ... in fact, all development > processes we've had in Postgres ... have suffered from only one > problem: lack of consensus on how the process should work. For > example, we've *never* had consensus around the criteria for kicking > a patch out of a commitfest. True, but "put that decision entirely in the hands of the CF manager" doesn't seem to me to be a workable solution. Half the time we don't even have a CF manager, AFAICT. Now admittedly the opportunity to wield absolute power might attract more interest in the position ;-) but I don't think we want people who are attracted by that. > Some suggestions: > - for the first 2 weeks of each CF, there should be a *total* moritorium on > discussing any features not in the current CF on -hackers. We've tried that in the past, and it's never been adhered to very well, and I think it's folly to assume that we'll get much better at it. The nature of a mailing list is that there's a lot of noise. Even if 95% of the membership knows about and agrees with the restriction, the other 5% will still post about non-CF stuff. > - we should have simple rules for the CF manager for kicking patches, as in: >* no response from author in 5 days >* judged as needing substantial work by reviewer >* feature needs spec discussion These rules still seem to me to require a lot of judgment, hence opportunity for argument. What's "substantial work"? How big a quibble about the spec is big enough to get a patch booted? > However, the real criteria don't matter as much as coming up with a set of > criteria we're all willing to obey, whatever they are. Ultimately, we're herding cats here. I don't think you're going to get the community to suddenly be willing to march in lockstep instead. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 18:29, Josh Kupershmidt wrote: > On Wed, Apr 11, 2012 at 8:59 AM, Peter Geoghegan > wrote: >> On 11 April 2012 15:35, Magnus Hagander wrote: > >>> For example, Thom (and others) could collect a number of typo fixes in >>> their own repo and then just ask for a merge.The advantage over just >>> staging multiple commits and then submitting a patch would be that >>> multiple people could work on it... >> >> This is hardly a radical idea at all - it's basically how git was >> really intended to be used at scale. Of course, unless some committer >> is going to make it their responsibility to merge those commits say >> every 3 months, there's no point in bothering. This could consolidate >> the number of typo commits to boot, as they could be rebased. TBH, I >> find it slightly embarrassing to have to ask a committer to fix a >> minor typo, and it's hardly reasonable to expect me to save my typos >> up. >> >> Big +1 from me. > > Particularly for the docs, it'd be nice to have more committer > bandwidth available, if there's a reasonable way to do so without > causing needless merge work for existing committers. Like Peter, I > hate to bother busy committers with trivial typofixes, and sometimes I > just don't bother sending such changes in, and they get lost :-( > > Maybe keeping doc/ as a 'git submodule' could work? Or, as Tom > suggests, adding a global committer who could focus on docs changes > would effectively solve the problem as well. git submodule would be a really bad idea imho. Then you couldn't make a single commit that deals with both code and docs. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] [streaming replication] 9.1.3 streaming replication bug ?
On Thu, Apr 12, 2012 at 12:56 AM, Fujii Masao wrote: > On Wed, Apr 11, 2012 at 3:31 PM, 乔志强 wrote: >> So in sync streaming replication, if master delete WAL before sent to the >> only standby, all transaction will fail forever, >> "the master tries to avoid a PANIC error rather than termination of >> replication." but in sync replication, termination of replication is THE >> bigger PANIC error. > > I see your point. When there are backends waiting for replication, the WAL > files > which the standby might not have received yet must not be removed. If they are > removed, replication keeps failing forever because required WAL files don't > exist in the master, and then waiting backends will never be released unless > replication mode is changed to async. This should be avoided. On second thought, we can avoid the issue by just increasing wal_keep_segments enough. Even if the issue happens and some backends get stuck to wait for replication, we can release them by taking fresh backup and restarting the standby from that backup. This is the basic procedure to restart replication after replication is terminated because required WAL files are removed from the master. So this issue might not be worth implementing the patch for now (though I'm not against improving things in the future), but it seems just a tuning-problem of wal_keep_segments. Regards, -- Fujii Masao -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [Patch] Fix little typo in a comment
"Etsuro Fujita" writes: > This is a little patch to fix a typo in contrib/file_fdw. I think that comment is fine as-is. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 8:59 AM, Peter Geoghegan wrote: > On 11 April 2012 15:35, Magnus Hagander wrote: >> For example, Thom (and others) could collect a number of typo fixes in >> their own repo and then just ask for a merge.The advantage over just >> staging multiple commits and then submitting a patch would be that >> multiple people could work on it... > > This is hardly a radical idea at all - it's basically how git was > really intended to be used at scale. Of course, unless some committer > is going to make it their responsibility to merge those commits say > every 3 months, there's no point in bothering. This could consolidate > the number of typo commits to boot, as they could be rebased. TBH, I > find it slightly embarrassing to have to ask a committer to fix a > minor typo, and it's hardly reasonable to expect me to save my typos > up. > > Big +1 from me. Particularly for the docs, it'd be nice to have more committer bandwidth available, if there's a reasonable way to do so without causing needless merge work for existing committers. Like Peter, I hate to bother busy committers with trivial typofixes, and sometimes I just don't bother sending such changes in, and they get lost :-( Maybe keeping doc/ as a 'git submodule' could work? Or, as Tom suggests, adding a global committer who could focus on docs changes would effectively solve the problem as well. Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Robert Haas writes: > On Wed, Apr 11, 2012 at 11:38 AM, Tom Lane wrote: >> We've frequently had, and still have today, committers who are >> understood to have limited areas of expertise and are given commit >> bits on the honor system to not break what they don't know well. >> I don't have any problem with continuing in that line. > Me neither, but I don't know how far it scales. Having certain people > who are defined as, say, doc-only committers will not only make it > clear to those people what they're expected to commit, but also clear > to everyone else who the people are who might commit any given patch > they might write. If we just end up with 50 committers and you have > to follow pgsql-hackers to understand who knows what and which people > are even still around, it's not going to make anything easier for > anyone. When and if we have 50 committers, we can worry about that problem ;-). But in practice people have different areas of expertise already. There was some mumbling upthread about trying to identify early who would be the responsible committer for any given CF submission --- doesn't seem to me like that is much different. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] [streaming replication] 9.1.3 streaming replication bug ?
On 4/11/12, Fujii Masao wrote: > On Wed, Apr 11, 2012 at 3:31 PM, 乔志强 wrote: >> So in sync streaming replication, if master delete WAL before sent to the >> only standby, all transaction will fail forever, >> "the master tries to avoid a PANIC error rather than termination of >> replication." but in sync replication, termination of replication is THE >> bigger PANIC error. > > I see your point. When there are backends waiting for replication, the WAL > files > which the standby might not have received yet must not be removed. If they > are > removed, replication keeps failing forever because required WAL files don't > exist in the master, and then waiting backends will never be released unless > replication mode is changed to async. This should be avoided. > > To fix this issue, we should prevent the master from deleting the WAL files > including the minimum waiting LSN or bigger ones. I'll think more and > implement > the patch. With asynchonous replication, does the master even know if a slave fails because of a WAL problem? And does/should it care? Isn't there a separate issue with synchronous replication? If it fails, what's the appropriate action to take on the master? PANICing it seems to be a bad idea, but having transactions never complete because they never hear back from the synchronous slave (for whatever reason) seems bad too. -- Mike Nolan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Tom Lane wrote: > What I'd be interested to see is number of lines changed per unit > time; that would be a much better measure of patch rate IMHO. Based on `git diff --shortstat` between tags, for the whole tree, this is what shows up: files git tag changed insertions deletions === === == = PG95-1_01 Release_1_0_2133 10373 1046 REL2_0 1079 65115 41987 REL6_1 770 70656 42995 REL6_2 919 195720 141153 REL6_3 1140 104845 34326 REL6_4_21583 171053 69832 REL6_5 1366 136665 85013 REL7_0 1991 272086 142201 REL7_1 2164 441090 212481 REL7_2 1826 314583 139238 REL7_3 2258 631434 245295 REL7_4 2018 436323 283074 REL8_0_02398 446392 258825 REL8_1_01796 235680 181193 REL8_2_02343 288467 168438 REL8_3_02355 405974 174798 REL8_4_02464 339085 462065 REL9_0_02270 292187 191852 REL9_1_03225 443037 186042 HEAD2014 189648 94 -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 11 April 2012 15:35, Magnus Hagander wrote: > Might it be worthwhile to allow some sort of "staging repository" and > actually start using the git stuff a bit more around this? E.g. we > could have a "docs repo" somewhere where more people have commit bits, > and then they are just regularly merged back into the main tree? Sort > of the way different people can own different subsystems in other > projects, but someone does the "trusted merge"? > > For example, Thom (and others) could collect a number of typo fixes in > their own repo and then just ask for a merge.The advantage over just > staging multiple commits and then submitting a patch would be that > multiple people could work on it... This is hardly a radical idea at all - it's basically how git was really intended to be used at scale. Of course, unless some committer is going to make it their responsibility to merge those commits say every 3 months, there's no point in bothering. This could consolidate the number of typo commits to boot, as they could be rebased. TBH, I find it slightly embarrassing to have to ask a committer to fix a minor typo, and it's hardly reasonable to expect me to save my typos up. Big +1 from me. It might be the case that over time, we become comfortable with this approach and upgrade the tree to a "linux-next" style tree (much like the -mm tree was repurposed into the progenitor of linux-next), with a lesser (though still substantial) standard for committers to meet. There could be an understanding that by committing to the tree, the developer makes a representation that they are confident that the feature is ready for prime-time, in just the same way that a commit currently represents - don't underestimate the power of that ceremony. Less senior contributors could have their work scrutinised by a wider body of people that haven't necessarily taken enough of an interest in the contributor's work to want to follow them on github or whatever - enthusiast power users who wouldn't really consider themselves hackers. This partially solves the "you want us to fund feature development but you're not even a committer?" problem that Greg referred to. It's also how big projects scale - technically, there are relatively few committers to the linux-stable tree. This approach formalises Tom's view that "I think the key point here is that people have to expect that it's going to take more than one round of review to land most nontrivial patches". On 11 April 2012 16:27, Robert Haas wrote: > Now, the advantage of a staging tree is that it gives the people who > have commit rights to the main repository the ability to decline to > merge. The question is - what happens then, especially given that we > have so many committers already? In Linux-land, it becomes the > subsystem maintainer's responsibility to put the tree into a state > where Linus will again become willing to merge it, or he can fire the > subsystem maintainer and pick a new one that'll do what he wants. But > we don't work that way. Instead, the committers as a group have the > responsibility for not breaking stuff. So who would decide whether to > do the merge, and who would be responsible for fixing things if the > merge were refused? This seems like a non-issue to me. We just try and match the Linux model. In practice I'd imagine that the roles would not really be perfectly delineated like that, and I find it really doubtful that they are in Linux land either. There was a time when some people were somewhat skeptical of the git migration (at least privately), and that was, as far as I can tell, a roaring success. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] [streaming replication] 9.1.3 streaming replication bug ?
On Wed, Apr 11, 2012 at 3:31 PM, 乔志强 wrote: > So in sync streaming replication, if master delete WAL before sent to the > only standby, all transaction will fail forever, > "the master tries to avoid a PANIC error rather than termination of > replication." but in sync replication, termination of replication is THE > bigger PANIC error. I see your point. When there are backends waiting for replication, the WAL files which the standby might not have received yet must not be removed. If they are removed, replication keeps failing forever because required WAL files don't exist in the master, and then waiting backends will never be released unless replication mode is changed to async. This should be avoided. To fix this issue, we should prevent the master from deleting the WAL files including the minimum waiting LSN or bigger ones. I'll think more and implement the patch. Regards, -- Fujii Masao -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
All, >From my observation, the CF process ... in fact, all development processes >we've had in Postgres ... have suffered from only one problem: lack of >consensus on how the process should work. For example, we've *never* had >consensus around the criteria for kicking a patch out of a commitfest. This >lack of consensus has resulted in disorganization, ennui towards the process, >deadline overruns, and a lot of general unhappiness. People have stopped >believing in the CF system because we've stopped running it. I'm encouraged at this point that we've seen where this lack of consensus can lead us, maybe at this point we're willing to set aside individual differences of opinion on what the criteria should be (especially when it comes to the patches we each individually care about) in service of a smoother-running process. Some suggestions: - for the first 2 weeks of each CF, there should be a *total* moritorium on discussing any features not in the current CF on -hackers. - the CF manager should have unquestioned authority to kick patches. As in, no arguing. - we should have simple rules for the CF manager for kicking patches, as in: * no response from author in 5 days * judged as needing substantial work by reviewer * feature needs spec discussion However, the real criteria don't matter as much as coming up with a set of criteria we're all willing to obey, whatever they are. We also need better tools for the CF, but frankly better tools is a minor issue and easily solved if we have a consensus which people are willing to obey. For that matter, if we have a smooth and impartial process, we can do other things, including: training new reviewers, promoting new committers, changing the length of the CF cycle, or changing the PostgreSQL release cycle (yes, really). While our review and commit process is completely subjective and inconsistent, though, we can't do any of these things. --Josh Berkus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] [streaming replication] 9.1.3 streaming replication bug ?
On 4/11/12, Kevin Grittner wrote: > Michael Nolan wrote: >> On 4/11/12, 乔志强 wrote: > >>> But when a transaction larger than 1GB... >> >> Then you may need WAL space larger than 1GB as well. For >> replication to work, it seems likely that you may need to have >> sufficient WAL space to handle a row, possibly the entire >> transaction.. But since a single statement can update thousands >> or millions of rows, do you always need enough WAL space to hold >> the entire transaction? > > No. > >>> Does master send WAL to standby before the transaction commit ? > > Yes. > >> A related question is what happens if there is a rollback? > > PostgreSQL doesn't use a rollback log; WAL files can be reclaimed as > soon as the work they represent has been persisted to the database > by a CHECKPOINT, even if it is not committed. Because there can be > multiple versions of each row in the base table, each with its own > xmin (telling which transaction committed it) and xmax (telling > which transaction expired it) visibiliity checking can handle the > commits and rollbacks correctly. It also uses a commit log (CLOG), > hint bits, and other structures to help resolve visibility. It is a > complex topic, but it does work. Thanks, Kevin. That does lead to a question about the problem that started this thread, though. How does one determine how big the WAL space needs to be to not cause streaming replication to fail? Or maybe this is a bug after all? -- Mike Nolan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Excerpts from Robert Haas's message of mié abr 11 12:44:02 -0300 2012: > Me neither, but I don't know how far it scales. Having certain people > who are defined as, say, doc-only committers will not only make it > clear to those people what they're expected to commit, but also clear > to everyone else who the people are who might commit any given patch > they might write. If we just end up with 50 committers and you have > to follow pgsql-hackers to understand who knows what and which people > are even still around, it's not going to make anything easier for > anyone. Since we're so keen on copying what Linux does, we could just have a MAINTAINERS file. -- Álvaro Herrera The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 11:38 AM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Apr 11, 2012 at 10:35 AM, Magnus Hagander >> wrote: >>> Might it be worthwhile to allow some sort of "staging repository" and >>> actually start using the git stuff a bit more around this? > >> ... As far as I can see, this basically amounts to >> bundling lots of unrelated changes into one big pile and then asking >> to have them all committed at once instead of one at a time, which >> sounds like more work not less, unless we're just going to blindly >> merge without reviewing, in which case we may as well just let people >> commit to the main repository themselves. > > Yeah. I'm also worried that we would either lose linear history, or > (if we squash the merge commits) lose change log history for the > individual fixes. Neither of those sounds terribly attractive. > > We've frequently had, and still have today, committers who are > understood to have limited areas of expertise and are given commit > bits on the honor system to not break what they don't know well. > I don't have any problem with continuing in that line. Me neither, but I don't know how far it scales. Having certain people who are defined as, say, doc-only committers will not only make it clear to those people what they're expected to commit, but also clear to everyone else who the people are who might commit any given patch they might write. If we just end up with 50 committers and you have to follow pgsql-hackers to understand who knows what and which people are even still around, it's not going to make anything easier for anyone. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Robert Haas writes: > On Wed, Apr 11, 2012 at 10:35 AM, Magnus Hagander wrote: >> Might it be worthwhile to allow some sort of "staging repository" and >> actually start using the git stuff a bit more around this? > ... As far as I can see, this basically amounts to > bundling lots of unrelated changes into one big pile and then asking > to have them all committed at once instead of one at a time, which > sounds like more work not less, unless we're just going to blindly > merge without reviewing, in which case we may as well just let people > commit to the main repository themselves. Yeah. I'm also worried that we would either lose linear history, or (if we squash the merge commits) lose change log history for the individual fixes. Neither of those sounds terribly attractive. We've frequently had, and still have today, committers who are understood to have limited areas of expertise and are given commit bits on the honor system to not break what they don't know well. I don't have any problem with continuing in that line. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 10:35 AM, Magnus Hagander wrote: > Since the topic is somewhat drifting here anyway.. :-) > > Might it be worthwhile to allow some sort of "staging repository" and > actually start using the git stuff a bit more around this? E.g. we > could have a "docs repo" somewhere where more people have commit bits, > and then they are just regularly merged back into the main tree? Sort > of the way different people can own different subsystems in other > projects, but someone does the "trusted merge"? > > For example, Thom (and others) could collect a number of typo fixes in > their own repo and then just ask for a merge.The advantage over just > staging multiple commits and then submitting a patch would be that > multiple people could work on it... If our goal is to give people more or less unfettered access to certain areas of the tree, but not the whole thing, we should perhaps consider just doing that directly. There's no particular reason why Thom Brown can't be made a committer just for docs, or why Alex Hunsaker can't be made a committer just for PL/perl (and presumably docs, since he'd need to update the docs if he updates the code), if that's actually what we want to do. Now, the advantage of a staging tree is that it gives the people who have commit rights to the main repository the ability to decline to merge. The question is - what happens then, especially given that we have so many committers already? In Linux-land, it becomes the subsystem maintainer's responsibility to put the tree into a state where Linus will again become willing to merge it, or he can fire the subsystem maintainer and pick a new one that'll do what he wants. But we don't work that way. Instead, the committers as a group have the responsibility for not breaking stuff. So who would decide whether to do the merge, and who would be responsible for fixing things if the merge were refused? As far as I can see, this basically amounts to bundling lots of unrelated changes into one big pile and then asking to have them all committed at once instead of one at a time, which sounds like more work not less, unless we're just going to blindly merge without reviewing, in which case we may as well just let people commit to the main repository themselves. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] [streaming replication] 9.1.3 streaming replication bug ?
Michael Nolan wrote: > On 4/11/12, 乔志强 wrote: >> But when a transaction larger than 1GB... > > Then you may need WAL space larger than 1GB as well. For > replication to work, it seems likely that you may need to have > sufficient WAL space to handle a row, possibly the entire > transaction.. But since a single statement can update thousands > or millions of rows, do you always need enough WAL space to hold > the entire transaction? No. >> Does master send WAL to standby before the transaction commit ? Yes. > A related question is what happens if there is a rollback? PostgreSQL doesn't use a rollback log; WAL files can be reclaimed as soon as the work they represent has been persisted to the database by a CHECKPOINT, even if it is not committed. Because there can be multiple versions of each row in the base table, each with its own xmin (telling which transaction committed it) and xmax (telling which transaction expired it) visibiliity checking can handle the commits and rollbacks correctly. It also uses a commit log (CLOG), hint bits, and other structures to help resolve visibility. It is a complex topic, but it does work. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Excerpts from Magnus Hagander's message of mié abr 11 11:35:10 -0300 2012: > For example, Thom (and others) could collect a number of typo fixes in > their own repo and then just ask for a merge.The advantage over just > staging multiple commits and then submitting a patch would be that > multiple people could work on it... The other advantage is that committers could have that tree as another remote in their main PG tree, and so changesets could be pulled into the same clone and cherry-picked into the master branch. (I'd also set up a clone for Alex Hunsaker to commit to pl/perl). -- Álvaro Herrera The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 04/10/2012 08:43 PM, Greg Smith wrote: On 04/10/2012 01:28 PM, Robert Haas wrote: The fact is that we have no shortage of committers - there are 19 people who have access to push code into our master git repository. A handful of those people have basically completely left the project and their commit rights should probably be revoked on that basis; most of them are still involved in one way or another but just not able to devote a lot of time to reviewing other people's code. Let's use realistic numbers here: I count 7 people who regularly review and commit other people's code in a variety of areas. There are also several subject matter experts who commit things in a relatively narrow area. But most bigger patches are going to hit a bottleneck whose capacity could be measured in 3 bits. Robert's point is fairly important. A few years ago one of the Postgres companies hired me for a few weeks to push along some of the stuff that was holding up a release at a critical time. (This was in the pre CF days.) Several important features made it into that release that almost certainly would not have otherwise. But generally that hasn't happened, and the time I've devoted has been my own, and thus subject to competing demands of business and the mythical "life AFK". Sometimes that makes it quite hard to arrange time to spend on reviewing and committing large patches. There are a significant number of companies who are willing to sponsor committers to open-source projects; there are almost none who will sponsor reviewers or "contributors" of any stature unless they're already deep into the PostgreSQL community. That's one of the many reasons it's easier for a committer to attract funding for core PostgreSQL work, be it in the form of a full-time job or project-oriented funding. The corresponding flip side to that is that the small number of committers is limiting the scope of funding the project can accumulate. Yep. The plus side is that Postgres business seems to be growing quite rapidly (if our experience at PostgreSQL Experts Inc is anything to go by), so maybe there will be more sources of funding available. More work needs to be done to generate funds to support getting features reviewed and committed. Patches seems to be getting ever larger and more complex, and that makes reviewing them harder and much more time consuming. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [JDBC] Regarding GSoc Application
On Tue, Apr 10, 2012 at 10:41 PM, Atri Sharma wrote: >>>Well. maybe I spoke too soon...JNI is probably the best route. Since >>>SPI is off the table, all we're really pulling in from pl/java is the >>>(non-trivial) proper installation of a jvm into a postgres process. >>>pl/java is essentially a wrapper to JNI that does postgres to jni type >>>conversion and builds a database driven class lookup system that >>>allows building the java environment inside the database. That part >>>is not needed at all; since we're not going to install our routines as >>>pl/java installed they will be installed into a separate and more >>>typical class path from a jvm point of view. There are no exposed >>>routines in pljava.so that allow for easy manual invocation of a java >>>routine -- it's all pretty much hardwired to the language function >>>call handler system. >>> >>>This begs the question about why pl/java should be used at all. Well, >>>as I said the jvm insertion into postgres is non-trivial so that alone >>>is worth something. Maybe though we should just rip out the jvm guts >>>that we really need (mostly some parts of backend.c and some of the >>>type files) and do a completely standalone jni wrapper. pl/java's >>>build system is a mess anyways so we're not losing all that much in >>>trying to go off on our own and do something. OTOH, throwing a build >>>system on top of the growing pile of things to do is turning this into >>>a daunting project vs the (admittedly cobbled together) approach I was >>>hoping to do earlier. Anyways, it's really Atri's call if he's >>>comfortable proceeding. >> >> Hi All, >> >> I think we are back on the initial approach I proposed(hooking directly into >> the JVM and executing Java code that calls JDBC).I think the best way to do >> this is create a JVM that executes the Java code and give the control of the >> JVM to the native API. >> >> I agree,the only need of Pl/Java that is apparent here is the need of the >> Java internals(JDK et al).If we set them up independently,then,we can have >> the FDW wrapping JDBC directly through JNI.JNI would call pure Java >> functions to connect to the JDBC. >> >> I think we can proceed with this.Once we are done with the API calling Java >> functions,I think the rest of the path is easily mapped(writing Java >> functions to connect to JDBC). > >yeah -- it sound plausible. I think the next step is to pull one of >the fdw projects that is working and currently builds with pgxs. Make >sure it is bsd licensed and that you retain the original copyright in >the code (feel free to add your own). The mysql_fdw project for >example is a good candidate. After that we need to strip out all the >mysql specific parts so that we have a basic skeleton of the fdw >wrapper that still builds. From there we will need to build in the >jni calls as well as jvm initialization code we are going to more or >less directly copy from pl/java, but one thing at a time: IMO, >getting the jvm installed and executing a 'hello world' jni is a good >milestone to reach before getting into all the mechanics of the remote >querying. > >The jvm initialization code in pl/java we are going to use is in >src/C/pljava/backend.c. A lot of the initialization work done there >is irrelevant to what we are trying to do but there are some very >important bits like the on_proc_exit handler that does cleanup when >the postgres process exits. Ideally we can drop just the stuff we >need from there and get our project to compile. > I agree,atm I'll work on getting JNI set up and to build a JVM using it and getting the "Hello,World" running in it in PostGreSQl. I'll keep you posted, Atri -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [JDBC] Regarding GSoc Application
Hi all, In continuation to the discussion regarding my JDBC wrapping FDW,I have been talking to members of the community and I have two approaches on which I would request your suggestions and opinions: >I think we are back on the initial approach I proposed(hooking directly into the JVM and executing Java code that calls JDBC). >I think the best way to do this is create a JVM that executes the Java code and give the control of the JVM to the native API. >I agree,the only need of Pl/Java that is apparent here is the need of the Java internals(JDK et al).If we set them up >independently,then,we can have the FDW wrapping JDBC directly through JNI.JNI would call pure Java functions to connect to the >JDBC. In the above context,I think if we directly wrap JNI and call pure Java routines(not using Pl/Java codes directly),we can build something on the lines of Multicorn(a nice suggestion by Hitoshi) http://multicorn.org/ On the other hand,as suggested by John,we can use the Pl/Java routines that already exist to build our FDW's API.This shall be the flow: user application -> SQL -> PLjava FDW functions -> pljava code -> JDBC -> foreign database The user will have to pass security levels and permissions before being able to get to the FDW.This would look to the user very much like dblink looks now, and have similar security issues and mitigations. Please let me know how to proceed further. Atri -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] [streaming replication] 9.1.3 streaming replication bug ?
On 4/11/12, 乔志强 wrote: > >> Yes, increase wal_keep_segments. Even if you set wal_keep_segments to 64, >> the amount of disk space for WAL files is only 1GB, so there is no need to >> worry so much, I think. No? > > But when a transaction larger than 1GB... Then you may need WAL space larger than 1GB as well. For replication to work, it seems likely that you may need to have sufficient WAL space to handle a row, possibly the entire transaction.. But since a single statement can update thousands or millions of rows, do you always need enough WAL space to hold the entire transaction? > So in sync streaming replication, if master delete WAL before sent to the > only standby, all transaction will fail forever, > "the master tries to avoid a PANIC error rather than termination of > replication." but in sync replication, termination of replication is THE > bigger PANIC error. That's somewhat debatable. Would I rather have a master that PANICED or a slave that lost replication? I would choose the latter. A third option, which may not even be feasible, would be to have the master fail the transaction if synchronous replication cannot be achieved, although that might have negative consequences as well. > Another question: > Does master send WAL to standby before the transaction commit ? That's another question for the core team, I suspect. A related question is what happens if there is a rollback? -- Mike Nolan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [JDBC] Regarding GSoc Application
On Tue, Apr 10, 2012 at 10:41 PM, Atri Sharma wrote: >>Well. maybe I spoke too soon...JNI is probably the best route. Since >>SPI is off the table, all we're really pulling in from pl/java is the >>(non-trivial) proper installation of a jvm into a postgres process. >>pl/java is essentially a wrapper to JNI that does postgres to jni type >>conversion and builds a database driven class lookup system that >>allows building the java environment inside the database. That part >>is not needed at all; since we're not going to install our routines as >>pl/java installed they will be installed into a separate and more >>typical class path from a jvm point of view. There are no exposed >>routines in pljava.so that allow for easy manual invocation of a java >>routine -- it's all pretty much hardwired to the language function >>call handler system. >> >>This begs the question about why pl/java should be used at all. Well, >>as I said the jvm insertion into postgres is non-trivial so that alone >>is worth something. Maybe though we should just rip out the jvm guts >>that we really need (mostly some parts of backend.c and some of the >>type files) and do a completely standalone jni wrapper. pl/java's >>build system is a mess anyways so we're not losing all that much in >>trying to go off on our own and do something. OTOH, throwing a build >>system on top of the growing pile of things to do is turning this into >>a daunting project vs the (admittedly cobbled together) approach I was >>hoping to do earlier. Anyways, it's really Atri's call if he's >>comfortable proceeding. > > Hi All, > > I think we are back on the initial approach I proposed(hooking directly into > the JVM and executing Java code that calls JDBC).I think the best way to do > this is create a JVM that executes the Java code and give the control of the > JVM to the native API. > > I agree,the only need of Pl/Java that is apparent here is the need of the > Java internals(JDK et al).If we set them up independently,then,we can have > the FDW wrapping JDBC directly through JNI.JNI would call pure Java > functions to connect to the JDBC. > > I think we can proceed with this.Once we are done with the API calling Java > functions,I think the rest of the path is easily mapped(writing Java > functions to connect to JDBC). yeah -- it sound plausible. I think the next step is to pull one of the fdw projects that is working and currently builds with pgxs. Make sure it is bsd licensed and that you retain the original copyright in the code (feel free to add your own). The mysql_fdw project for example is a good candidate. After that we need to strip out all the mysql specific parts so that we have a basic skeleton of the fdw wrapper that still builds. From there we will need to build in the jni calls as well as jvm initialization code we are going to more or less directly copy from pl/java, but one thing at a time: IMO, getting the jvm installed and executing a 'hello world' jni is a good milestone to reach before getting into all the mechanics of the remote querying. The jvm initialization code in pl/java we are going to use is in src/C/pljava/backend.c. A lot of the initialization work done there is irrelevant to what we are trying to do but there are some very important bits like the on_proc_exit handler that does cleanup when the postgres process exits. Ideally we can drop just the stuff we need from there and get our project to compile. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Wed, Apr 11, 2012 at 16:24, Tom Lane wrote: > Greg Smith writes: >> On 04/10/2012 09:14 PM, Robert Haas wrote: >>> I wouldn't object to creating some doc-only committers. OTOH, I would >>> object to anyone making non-trivial documentation enhancements without >>> posting their patches first and having a second person look it over, >>> so how much difference is there, really? > >> ... >> I'd like to dump around 50 pages of new material into the docs as a >> start, but I don't want to take so much time away from the code oriented >> committers to chew on that much. > > Well, with all due respect, that does not sound like a change that > doesn't need review. > > I have not noticed people adding docs-only changes to the CFs; usually > it's more like "post a patch, somebody looks it over and commits it". > I agree that this is still too much overhead for simple fixes, such > as the editorial glitches that Thom Brown is so good at finding > (and I'm about ready to vote to give him a commit bit for that work). > But a fifty-page chunk is not that, indeed it sounds like it would have > enough technical content that it might actually merit a full-scale > review. Since the topic is somewhat drifting here anyway.. :-) Might it be worthwhile to allow some sort of "staging repository" and actually start using the git stuff a bit more around this? E.g. we could have a "docs repo" somewhere where more people have commit bits, and then they are just regularly merged back into the main tree? Sort of the way different people can own different subsystems in other projects, but someone does the "trusted merge"? For example, Thom (and others) could collect a number of typo fixes in their own repo and then just ask for a merge.The advantage over just staging multiple commits and then submitting a patch would be that multiple people could work on it... -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 11 April 2012 03:26, Tom Lane wrote: > [ scratches head... ] That's supposed to be total lines of code in our > source tree? What's the big drop in late 2009, then? I had wondered about that myself - all I can tell you is that I used the tool as advertised, without any adornments. This particular tool is a bit misleading, because it counts lines of code as lines of checked-in text, which can include things that are very verbose without necessarily being what we really think of as code - it doesn't make any effort to discriminate against non-code, nor does it expose the option of doing so. I chose it at short notice simply because it produces graphs. I do intend to take a look at this problem in more detail, and get better statistics on changes to our codebase - it's a tricky proposition, though. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
Greg Smith writes: > On 04/10/2012 09:14 PM, Robert Haas wrote: >> I wouldn't object to creating some doc-only committers. OTOH, I would >> object to anyone making non-trivial documentation enhancements without >> posting their patches first and having a second person look it over, >> so how much difference is there, really? > ... > I'd like to dump around 50 pages of new material into the docs as a > start, but I don't want to take so much time away from the code oriented > committers to chew on that much. Well, with all due respect, that does not sound like a change that doesn't need review. I have not noticed people adding docs-only changes to the CFs; usually it's more like "post a patch, somebody looks it over and commits it". I agree that this is still too much overhead for simple fixes, such as the editorial glitches that Thom Brown is so good at finding (and I'm about ready to vote to give him a commit bit for that work). But a fifty-page chunk is not that, indeed it sounds like it would have enough technical content that it might actually merit a full-scale review. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch: add timing of buffer I/O requests
On Wed, Apr 11, 2012 at 01:53:06AM +0100, Peter Geoghegan wrote: > On 11 April 2012 01:16, Tom Lane wrote: > > Peter Geoghegan writes: > >> On 11 April 2012 00:35, Robert Haas wrote: > >>> If people need something like that, couldn't they create it by hashing > >>> the normalized query text with an arbitrary algorithm? > > > >> That supposes that the normalised query text is perfectly stable. It > >> may well not be, particularly for things like ad-hoc queries or > >> queries generated by ORMs, across database clusters and over long > >> periods of time - > > > > Indeed, but the hash value isn't stable either given those sorts of > > assumptions, so I'm not convinced that there's any advantage there. > > Isn't it? The hash captures the true meaning of the query, while > having the database server's platform as a usually irrelevant > artefact. Another thing that I forgot to mention is client encoding - > it may well be fairly inconvenient to have to use the same algorithm > to hash the query string across applications. You also have to hash > the query string yourself again and again, which is expensive to do > from Python or something, and is often inconvenient - differences > beyond track_activity_query_size bytes (default:1024) are not > recognised. Using an SQL code beautifier where a single byte varies > now breaks everything, which developers don't expect at all (we've > trained them not to), so in many ways you're back to the same > limitations as classic pg_stat_statements if you attempt to aggregate > queries over time and across machines, which is a very real use case. > > It's probably pretty annoying to have to get your Python app to use > the same hash function as your Java app or whatever I, unless you want > to use something heavyweight like a cryptographic hash function. By > doing it within Postgres, you avoid those headaches. > > I'm not asking you to very loudly proclaim that it should be used like > this - just expose it, accurately document it, and I'm quite confident > that it will be widely used and relied upon by those that are > reasonably well informed, and understand its limitations, which are > really quite straightforward. > > > What I think people would actually like to know, if they're in a > > situation where distinct query texts are getting hashed to the same > > thing, is *which* different texts got hashed to the same thing. > > But there's no good way to expose that given the pg_stat_statements > > infrastructure, and exposing the hash value doesn't help. > > Apart from detecting the case where we get a straightforward > collision, I don't expect that that would be useful. The whole point > is that the user doesn't care about the difference, and I think we've > specified a practical, widely useful standard for when queries should > be considered equivalent. > -- > Peter Geoghegan http://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Training and Services > By using all 64-bits of the hash that we currently calculate, instead of the current use of 32-bits only, the collision probabilities are very low. Regards, Ken -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On 04/10/2012 09:14 PM, Robert Haas wrote: I wouldn't object to creating some doc-only committers. OTOH, I would object to anyone making non-trivial documentation enhancements without posting their patches first and having a second person look it over, so how much difference is there, really? This workflow is the easy one: -Committer suggests doc change -No one has an opinion against it strong enough to comment, or minor review comments are made -Commit change with feedback when received That's a predictable, short process unless the change is controversial, in which case good feedback normally comes out of that discussion. And if I feel review is needed but don't catch any volunteers, there is a much broader pool of people who can be hired to help with review work, relative to the size of the committer one. Compare with: -Submitter suggests doc change -No one has a strong opinion on it, may not be picked up at all -Submitter adds to the next CF -Wait for review -[Possible repost update with reviewer changes] -Ready for committer -Committer takes time away from code review to look at it -Possibly another feedback/review resubmission -Commit final versions It's usually not this bad, but in every case it's pulling resources off of more valuable jobs. I'd like to dump around 50 pages of new material into the docs as a start, but I don't want to take so much time away from the code oriented committers to chew on that much. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server
Hi all, (2012/03/07 3:39), Tom Lane wrote: > A bigger issue with postgresql_fdw_validator is that it supposes that > the core backend is authoritative as to what options libpq supports, > which is bad design on its face. It would be much more sensible for > dblink to be asking libpq what options libpq supports, say via > PQconndefaults(). > > We might find that we have to leave postgresql_fdw_validator as-is > for backwards compatibility reasons (in particular, being able to load > existing FDW definitions) but I think we should migrate away from using > it. In the discussion about pgsql_fdw which was proposed for 9.2, some issues about postgresql_fdw_validator are pointed out. * The name "postgresql_fdw_validator" conflicts with the name of the FDW for PostgreSQL which follows the naming habit of other FDWs. * dblink depends on postgresql_fdw_validator. * postgresql_fdw_validator assumes that libpq supports some particular options. An idea to resolve these is to add dblink's own validator which doesn't assume much about libpq, and obsolete postgresql_fdw_validator. * Add dblink_fdw_validator to contrib/dblink, which is similar to postgresql_fdw_validator but it assumes less about libpq. * Add dblink_fdw as default FDW of dblink, which uses dblink_fdw_validator, and recommend to use it. This would prevent users from using postgresql_fdw_validator with dblink. * Mention that postgresql_fdw_validator might be obsolete in future release in the document of CREATE FOREIGN DATA WRAPPER. To make the behavior of dblink_fdw_validator similar to that of current postgresql_fdw_validator, we need to assume that libpq supports "user" option, and allow it and secret options in only USER MAPPING options, and allow others in only SERVER options (and reject all debug options). IMO this is not unreasonable assumption. Is this proposal reasonable? Any comments and questions are welcome. Regards, -- Shigeru HANADA -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Last gasp
On Tue, Apr 10, 2012 at 8:12 PM, Robert Haas wrote: > However exactly > the list turns out, there is no question that non-committers have been > quite successful in getting significant feature enhancements committed > in each of the last three releases, and I'm pretty confident it goes > back a lot further than that. I agree that in practice non-committers do get a lot of great stuff done, but the question is if *more* stuff would get done if things were slightly different. To that end, I'd like to share my own anecdote on why I don't attempt large projects to Postgres at this time: I used to work on a proprietary postgres offspring and ship quite a bit of internals code. A couple of people I worked with are frequenters of this list, even. I spent nearly two years doing it, full time, without having to (or being able to) go through a full-blown community process from design to ship: I got a lot of nuts-and-bolts practice, and I really enjoyed it. Yet I don't take on large projects in the project now, and I'm even employed in a place where I could start doing that on-the-job. Why don't I? One reason I don't do that is because there is an unavoidable information asymmetry problem between myself and the committers. When I think of a committer and what makes me different than them, this is what I come up with: * More experience and expertise, both in general and with the project * Proven intent to maintain the work submitted by others for a long time. In a word, "stewardship" or "ownership" I'm grateful for both, but the latter point is one where some mind-reading is required: what's strategically important enough that's it is important enough to compromise on something? What compromises are acceptable? That is tantamount to guessing "what compromises is a committer willing to maintain?" And that can be a pretty personal thing and is hard to guess, and I don't think that's solvable as long as there seems to be this handoff from "the contributor" to "the project." It's hard to feel a sense of ownership -- and thus commitment -- if one cannot simply change one's own code, especially for trivia or churning around a future intent or purpose. If there is a bottleneck with the development process that is having a chilling effect on my ability to justify larger projects, it is this. I don't know what the most apparent way to optimize that bottleneck is, but there's my thoughts. I think part of the answer is more hooks, even if they come with reduced contracts in terms of maintenance (here this release, gone the next release), and less required justification to commit those; consider https://github.com/dimitri/pgextwlist, which relies on such a hook and is the only thing that makes 9.1's extension support viable for Heroku, yet is cohesive feeling with the rest of the system to the point that it pretty much goes unnoticed. That's a great property I wish I could see more of. Also, I am not able to spend as much time on large Postgres-projects because some of the tooling outside the database is still the weakest link in my chain for now, so the good news is that I've shifted my attention to projects that are very much related but not part postgres-proper. The point of having more hooks is to open up more opportunities for such tools without making the result feel incohesive and terrible for reasons not intrinsic to the idea it is trying to add (consider: using the statement hook in pgextwlist rather than preloaded security-definer UDFs that would run CREATE EXTENSION for you. Yuck.) -- fdr -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Another review of URI for libpq, v7 submission
Excerpts from Alvaro Herrera's message of lun abr 09 16:41:50 -0300 2012: > There are three minor things that need to be changed for this to be > committable: Committed this patch after some more editorialization; in particular the test was rewritten so that instead of trying to connect, it uses PQconninfoParse to figure out how the URI is parsed, which makes a lot more sense. Also some other changes to the accepted URI, in particular so that username, pwd, and port are possible to be specified when using unix-domain sockets. Now that it is a done deal I'm sure people will start complaining how bad the documentation change was; please keep the flames up. -- Álvaro Herrera The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [Patch] Fix little typo in a comment
This is a little patch to fix a typo in contrib/file_fdw. Best regards, Etsuro Fujita file_fdw_typo_fix.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
