Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Joe Abbate
On 05/30/2011 10:57 AM, Magnus Hagander wrote:
 The case I want to avoid is (a). And if it's possible to make (b) just
 be the -hackers mailinglist and putting a keyword in the right place,

Did you mean the -bugs mailing list?

On the subject of keywords, considering Robert's suggestion to
Associate some kind of status like OPEN, FIXED, NOTABUG,
WONTFIX, etc. with each such bug via web interface and considering
that most people think a mail interface is more desirable, perhaps any
email response on -bugs that takes a definite stance on a bug, i.e.,
other than keeping it OPEN, could add a status keyword at the end of the
subject line?

Joe

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Stefan Kaltenbrunner
On 05/30/2011 04:26 AM, Greg Stark wrote:
 On Sun, May 29, 2011 at 3:36 PM, Joe Abbate j...@freedomcircle.com wrote:
  Anyone interested in the tracker, please visit
 http://wiki.postgresql.org/wiki/TrackerDiscussion and add your
 feedback/input.
 
 I think this illustrates exactly what we *don't* want to happen with a
 bug tracker. We want the discussion to stay *here* not on some other
 medium accessible only through the web and editable only through a web
 interface
 
 Also your summary seems to have missed the point on the has email
 interface requirement. The table of features you listed has just
 Creation of bugs via mail interface as the only feature that is
 accessible from email.
 
 I'm not sure what Robert meant but I suspect he meant what I would
 want which is the ability to add comments, close bugs, set other
 properties, etc. By email. My biggest gripe about bugzilla was that it
 sent you an email with updates to the bug but you couldn't respond to
 that email.

well bugzilla has an inbound email interface as well that can both be
used to creande and to manipulate bugs (as in mails that have the
bug-id in the subject will be added as a comment).
The demo installation did that by simply being subscribed to -bugs.

 
 My ideal bug tracker is the debian one which basically stays out of
 your way and lets you cc any message to a specific bug at
 n...@bugs.debian.org which archives that message in the bug and sends
 it to anyone listening to the bug. And you can have control commands
 to close it or edit it -- basically making all our existing that's
 not a bug bleah bleah messages into close nnn; that's not a bug
 bleah bleah messages.

that is what every emailinterface should be able to provide ;). However
the real issue with say BZ(or most other trackers) in this role is that
in order to attribute a bug report or a comment to the original
author/person you have to trust the From in the email and basically
autocreate an account based on that information for the tracker to work
with.


Stefan

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Small patch for GiST: move childoffnum to child

2011-05-30 Thread Heikki Linnakangas

On 24.05.2011 15:22, Alexander Korotkov wrote:

During preparing patch of my GSoC project I found reasonable to
move childoffnum (GISTInsertStack structure) from parent to child. This
means that now child have childoffnum of parent's link to child. It allows
to maintain entire parts of tree in that GISTInsertStack structures. Also it
simplifies existing code a bit.
Heikki advice me that since this change simplifies existing code it can be
considered as a separate patch.


Looks good to me.

--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Nested CASE-WHEN scoping

2011-05-30 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
 I think we can work around both of those by just saving and restoring 
 the value of each Param that we set while evaluating an expression,

Huh?  That's a waste of time and effort.  Just make sure that each such
spot has its own Param number.  That's why I'm telling you to do it in
the planner, not the parser --- it is easy to assign globally-unique-
across-the-plan numbers at plan time, in fact we do it already.

 For debugging purposes, it seems like a good idea to invent a new kind 
 of Param for these, and keep them separate from PARAM_EXEC params.

I'd vote against that too.  PARAM_EXEC Params already have more than one
purpose.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Unfriendly handling of pg_hba SSL options with SSL off

2011-05-30 Thread Magnus Hagander
On Fri, May 13, 2011 at 00:21, Tom Lane t...@sss.pgh.pa.us wrote:
 Magnus Hagander mag...@hagander.net writes:
 On Tue, May 10, 2011 at 05:39, Tom Lane t...@sss.pgh.pa.us wrote:
 I wouldn't have a problem with making the Windows port throw an error
 for local lines.  We'd have to fix initdb to remove that line from the
 sample file (if it doesn't already), but that's surely not hard.

 It does already (that's what the @remove-line-for-nolocal@ markup in
 the sample file is for).

 So +1 for making it throw an error.

 Although this should be a simple change, I don't want to do it because
 I'm not in a position to test it.  Do you want to take care of it?

Here's a patch that I think does what we want. It works fine on
Windows - I just want to make sure this is what you meant as well?

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/
commit 4c64761b53f1acec54af3c63b436d9f6defb845c
Author: Magnus Hagander mag...@hagander.net
Date:   Mon May 30 20:11:13 2011 +0200

Refuse local lines in pg_hba.conf on platforms that don't support it

This makes the behavior compatible with that of hostssl, which
also throws an error when there is no SSL support included.

diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index c17863f..f3a3b6e 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -824,7 +824,16 @@ parse_hba_line(List *line, int line_num, HbaLine *parsedline)
 	token = lfirst(line_item);
 	if (strcmp(token, local) == 0)
 	{
+#ifdef HAVE_UNIX_SOCKETS
 		parsedline-conntype = ctLocal;
+#else
+		ereport(LOG,
+(errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg(local connections are not supported by this build),
+ errcontext(line %d of configuration file \%s\,
+			line_num, HbaFileName)));
+		return false;
+#endif
 	}
 	else if (strcmp(token, host) == 0
 			 || strcmp(token, hostssl) == 0

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SSI predicate locking on heap -- tuple or row?

2011-05-30 Thread Kevin Grittner
[The first attempt to send this shows as undeliverable to the list. 
Resending for completeness and coherency of archives.  Apologies to
those getting duplicates.]
 
 Heikki Linnakangas wrote:
 On 26.05.2011 06:19, Kevin Grittner wrote:

 /*
 * Check whether the writer has become a pivot with an
 * out-conflict committed transaction, while neither reader
 * nor writer is committed. If the reader is a READ ONLY
 * transaction, there is only a serialization failure if an
 * out-conflict transaction causing the pivot committed
 * before the reader acquired its snapshot. (That is, the
 * reader must not have been concurrent with the out-conflict
 * transaction.)
 */

 The comment is not in sync with the code. The code is not checking
 that neither reader or writer has committed, but something more
 complicated.

True. We changed the code but not the comment. Sorry for missing
that. Basically the quoted clause would be correct by changing it to
neither reader nor writer committed first. (Your rewrite,
discussed below, is better, though.)

 I find it helps tremendously to draw the dangerous structures being
 checked, in addition to just explaining them in text.

Agreed -- I spent many an hour with the dangerous structure diagram
in front of me when thinking through the mechanics of implementation.

 Ascii art is a bit clunky, but I think we have to do it here.

OK.

 I did some of that in the comments, and I think I understand it
 now. See attached patch. Does that look right to you?

! * A serialization failure can only occur if there is a dangerous
! * structure in the dependency graph:
! *
! * Tin -- Tpivot -- Tout
! * rw rw
! *
! * Furthermore, Tout must commit first.

I agree that this is easier to read and understand than just straight
text; but you dropped one other condition, which we use rather
heavily. We should probably add back something like:

* If Tin is declared READ ONLY (or commits without writing), we can
* only have a problem if Tout committed before Tin acquired its
* snapshot.

 * XXX: Where does that last condition come from?

This optimization is an original one, not yet appearing in any
academic papers; Dan and I are both convinced it is safe, and in
off-list correspondence with Michael Cahill after he left Oracle, he
said that he discussed this with Alan Fekete and they both concur
that it is a safe and good optimization.

This optimization is mentioned in the README-SSI file in the
Innovations section. Do you think that file needs to have more on
the topic?

 * XXX: for an anomaly to occur, T2 must've committed before W.
 * Couldn't we easily check that here, or does the fact that W
 * committed already somehow imply it?

The flags and macros should probably be renamed to make it more
obvious that this is covered. The flag which SxactHasConflictOut is
based on is only set when the conflict out is to a transaction which
committed ahead of the flagged transaction. Regarding the other
test -- since we have two transactions (Tin and Tout) which have not
been summarized, and transactions are summarized in order of commit,
SxactHasSummaryConflictOut implies that the transaction with the flag
set has not committed before the transaction to which it has a
rw-conflict out.

The problem is coming up with a more accurate name which isn't really
long. The best I can think of is SxactHasConflictOutToEarlierCommit,
which is a little long. If nobody has a better idea, I guess it's
better to have a long name than a misleading one. Do you think we
need to rename SxactHasSummaryConflictOut or just add a comment on
that use that a summarized transaction will always be committed ahead
of any transactions which are not summarized?

 (note that I swapped the second and third check in the function, I
 think it's more straightforward that way).

It doesn't matter to me either way, so if it seems clearer to you,
that's a win.

 Should we say Cancelled on identification as pivot, during ...,
 or Cancelled on identification as a pivot, during ... ? We seem
 to use both in the error messages.

Consistency is good. I think it sounds better with the indefinite
article in there.

Do you want me to post another patch based on this, or are these
changes from what you posted small enough that you would prefer to
just do it as part of the commit?

Thanks for taking the time to work through this. As always, your
ideas improve things.

-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Unfriendly handling of pg_hba SSL options with SSL off

2011-05-30 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes:
 On Fri, May 13, 2011 at 00:21, Tom Lane t...@sss.pgh.pa.us wrote:
 Magnus Hagander mag...@hagander.net writes:
 On Tue, May 10, 2011 at 05:39, Tom Lane t...@sss.pgh.pa.us wrote:
 I wouldn't have a problem with making the Windows port throw an error
 for local lines. We'd have to fix initdb to remove that line from the
 sample file (if it doesn't already), but that's surely not hard.

 Here's a patch that I think does what we want. It works fine on
 Windows - I just want to make sure this is what you meant as well?

Looks sane to me.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Alvaro Herrera
Excerpts from Greg Stark's message of dom may 29 22:26:21 -0400 2011:

 My ideal bug tracker is the debian one which basically stays out of
 your way and lets you cc any message to a specific bug at
 n...@bugs.debian.org which archives that message in the bug and sends
 it to anyone listening to the bug. And you can have control commands
 to close it or edit it -- basically making all our existing that's
 not a bug bleah bleah messages into close nnn; that's not a bug
 bleah bleah messages.

Yeah.  The other good thing about the Debian thing is that email is
first-class citizen; each bug history is basically an mbox.  All the
other systems I've looked at try to do the silly thing of extracting
the text from the email and inserting into a comment of some sort,
which is ocassionally problematic because of random annoyances in email
messages; and when you want to get down to investigating exactly what
was discussed in the email thread, the interesting bits aren't there.
In the Debian system, you can get the mbox and open it with your
favorite email reading tool instead.

-- 
Álvaro Herrera alvhe...@commandprompt.com
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Greg Smith

On 05/29/2011 05:17 AM, Peter Eisentraut wrote:

Here is a list to choose from:
http://en.wikipedia.org/wiki/Comparison_of_issue_tracking_systems
   


I turned this into a spreadsheet to sort and prune more easily; if 
anyone wants that let me know, it's not terribly useful beyond what I'm 
posting here.  44 total, 16 that are open-source.  I would say that 
having an e-mail interface is the next major cut to make.  While 
distasteful, it's possible for this project to adopt a solution that 
doesn't use PostgreSQL, and one interesting candidate is in that 
category.  It's not feasible to adopt one that doesn't integrate well 
with e-mail though.


List of software without listed e-mail integration:  Fossil, GNATS, 
Liberum Help Desk, MantisBT, org-mode, Flyspray, ikiwiki, Trac.


The 8 F/OSS programs left after that filter are:

OTRS
Debbugs
Request Tracker
Zentrack
LibreSource
Redmine
Roundup
Bugzilla

The next two filters you might apply are:

Support for Git:  Redmine, Bugzilla
PostgreSQL back-end:  OTRS, Request Tracker, LibreSource, Redmine, 
Roundup, Bugzilla


There are a couple of additional nice to have items I saw on the feature 
list, and they all seem to spit out just Redmine  Bugzilla.  Those are 
the two I've ended up using the most on PostgreSQL related projects, 
too, so that isn't a surprise to me.  While I'm not a strong fan of 
Redmine, it has repeatedly been the lesser of the evils available here 
for three different companies I've worked at or dealt with.


Greg Stark is right that Debbugs has a lot of interesting features 
similar to the desired workflow here.  It's not tied to just Debian 
anymore; the GNU project is also using it now.  And the database backend 
isn't that terrible to consider:  it's flat files with a BerkleyDB index 
built on top.  I think if it was perfect except for that, it would still 
be worth considering.  Debbugs is far from a general purpose solution 
though, so any customization to support differences in this project's 
workflow would likely end up being one-off hacks.  The VCS support might 
be a problem, but I've gotten the impression that git is increasingly 
popular for other Debian work.  Since the program is in Perl, I can't 
imagine it's a gigantic task to switch that out, and probably one other 
people would like to see.


--
Greg Smith   2ndQuadrant USg...@2ndquadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support  www.2ndQuadrant.us



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Peter Eisentraut
On sön, 2011-05-29 at 18:36 -0400, Joe Abbate wrote:
 I've summarizes the main points made in the recent discussion and did
 some minor additional research on the lists suggested by Peter and
 Chris Browne.  Anyone interested in the tracker, please visit
 http://wiki.postgresql.org/wiki/TrackerDiscussion and add your
 feedback/input.

Based on that, and past discussions, and things we've tried in the past,
and gut feeling, and so on, it looks like Request Tracker would appear
to be the next best thing to consider trying out.  What do people think
about that?



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Magnus Hagander
On Mon, May 30, 2011 at 04:26, Greg Stark gsst...@mit.edu wrote:
 On Sun, May 29, 2011 at 3:36 PM, Joe Abbate j...@freedomcircle.com wrote:
  Anyone interested in the tracker, please visit
 http://wiki.postgresql.org/wiki/TrackerDiscussion and add your
 feedback/input.

 I think this illustrates exactly what we *don't* want to happen with a
 bug tracker. We want the discussion to stay *here* not on some other
 medium accessible only through the web and editable only through a web
 interface

+as high number as my quota currently goes

It's fine that a bug tracker *tracks* bugs. It should not control
them. That's not how this community currently works, and a lot of
people have said that's how they want it to stay (at least for now).


 Also your summary seems to have missed the point on the has email
 interface requirement. The table of features you listed has just
 Creation of bugs via mail interface as the only feature that is
 accessible from email.

 I'm not sure what Robert meant but I suspect he meant what I would
 want which is the ability to add comments, close bugs, set other
 properties, etc. By email. My biggest gripe about bugzilla was that it
 sent you an email with updates to the bug but you couldn't respond to
 that email.

I agree with these too :-)

It's also missing what I believe is a very important requirement - it
needs to have an extensive, and fully supported, API. So that we can
easily make it work together with our other services.


 My ideal bug tracker is the debian one which basically stays out of
 your way and lets you cc any message to a specific bug at
 n...@bugs.debian.org which archives that message in the bug and sends
 it to anyone listening to the bug. And you can have control commands
 to close it or edit it -- basically making all our existing that's
 not a bug bleah bleah messages into close nnn; that's not a bug
 bleah bleah messages.

No direct experience with the debian tracker, but I agree that being
able to do all those things from mail is very important. If it *also*
provides a way to do this from the web, that's even better.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Peter Eisentraut
On sön, 2011-05-29 at 11:05 -0400, Tom Lane wrote:
 If it has only a partial view of the set of bugs being worked on, it's
 not going to meet the goals that are being claimed for it.
 
 I don't doubt that somebody could run around and link every discussion
 about a bug into the tracker.  I'm just dubious that that actually
 *will* happen with enough reliability to make the tracker more useful
 than a mailing-list search.

At least initially, the bug tracker is for those who want to use it, to
help with their work.  If it eventually becomes the total-awareness
tool, that would be great, but if we make that the main goal, it will
never get started.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Andres Freund
On Monday, May 30, 2011 07:30:37 AM Greg Smith wrote:
 Trac
While I am not a fan of trac there is a plugin for that that works reasonable 
well and isn't that hard to customize if needed:
https://subtrac.sara.nl/oss/email2trac

Andres

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] [PATCH] Bug in XPATH() if expression returns a scalar value

2011-05-30 Thread Florian Pflug
Hi

The in-core XPATH() function currently only handles XPath expressions which
return a node set correctly. For XPath expressions which return boolean,
numeric or string values, XPATH() returns an empty array. (This is the case
for XPath expressions whose top-level syntactic construct isn't a path
expression but rather one of the built-in functions like name() or count()).

The XPath expression 'name(/*)', for example, is supposed to return 'root'
when applied to the XML fragment 'rootnested/nested//root'. Postgres,
however, currently returns an empty array.

A patch which fixes that is attached. It makes XPATH() return a single-element
array containing a textual representation of the string, numeric value or 
boolean
for XPath expressions which don't return a node set. For numeric and boolean
results, the textual representation is obtained by calling float8out() 
respectively
boolout(). These function's results are assumed to always be well-form XML. For
string results, xml_in() is used to verify that the string returned by the XPath
expression is a well-formed XML fragment. This is required because XPATH() could
otherwise be used to insert invalid XML fragments into columns of type XML.

The patch add a few tests of non-nodeset returning XPath expressions to the
XML regression tests.

I'm not 100% clear on whether I should add this to the next commit fest or not -
after all, it's more a bug-fix rather than a new feature. But to prevent this
from getting lost, I'll add it unless someone tells me not to.

best regards,
Florian Pflug


pg_xpath_returnvalue.v1.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] How can I check the treatment of bug fixes?

2011-05-30 Thread Greg Smith
On 05/27/2011 08:36 AM, MauMau wrote:
 I posted a patch for bug #6011 to pgsql-hackers several days ago. How
 can I check the status of bug fixes? I'm worried that the patch might
 be forgotten, because bug #5842 was missed for two months until Bruce
 noticed it.

Discussion here seems to have wandered far away from useful suggestions
for you, let's see if that's possible to return to that. Best way to
confirm when a bug is resolved is to subscribe to the pgsql-committers
mailing list. If a commit for this fix appears, odds are good the
original bug number will be referenced. Even if it isn't, you may
recognize it via its description. Until you see that, the bug is almost
certainly still open.

Bugs that are considered to impact the current version under development
are sometimes listed at http://wiki.postgresql.org/wiki/Open_Items
Adding a bug to there that's not really specific to the new version may
not be considered good form by some. It is the closest thing to an open
bug tracker around though, and adding items to there means they won't be
forgotten about; it's checked regularly by developers considering when
it's a good time to release another alpha or beta.

In my mind, clarifying what circumstances it's appropriate for people to
put a bug onto the Open Items list would be a useful way to spend a
little time. Certainly more productive than trying firing more bullets
at the unkillable zombie that is bug tracking software.

-- 
Greg Smith   2ndQuadrant USg...@2ndquadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support  www.2ndQuadrant.us



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Magnus Hagander
On Mon, May 30, 2011 at 16:52, Joe Abbate j...@freedomcircle.com wrote:
 Hi Magnus,

 On 05/30/2011 08:45 AM, Magnus Hagander wrote:
 It's fine that a bug tracker *tracks* bugs. It should not control
 them. That's not how this community currently works, and a lot of
 people have said that's how they want it to stay (at least for now).

 If I may belabor the point, what do you see as an example of
 controlling the bugs?  To put some context, there could be at least
 three ways a bug could be closed when someone commits a patch that fixes
 (or claims to fix) a bug:

 a. The committer has to use a web interface to indicate the bug is closed
 b. The committer has to send an email to a mail interface
 c. The commit message gets routed to a mail interface that, seeing
 something like bug #1234 in the first line, automatically closes the bug

 Based on the discussion so far, it's obvious that option b is more
 desired than a (where the tracker is, in a sense, controlling *you*),
 but is option c --while presumably more desirable since there's one less
 thing to do or remember-- an instance of control, since the tracker
 takes an automatic action?  Or do you want the tracker *not* to require
 or take any of the actions, i.e., let someone/thing other than the
 committer/commit message worry about tracking the bug's status, leaving
 it up to volunteers, as Tom said?

I believe b is perfectly fine in this, and to me the preferred way. We
always respond to the original message with something like yeah,
patched over here or something like that anyway, so I don't
(personally) see a need for the actual commit message to be able to do
it.

The case I want to avoid is (a). And if it's possible to make (b) just
be the -hackers mailinglist and putting a keyword in the right place,
that minimizes the impact on those who spend a lot of time with it
(far more than me..), which is always good.

I personally don't think it's good to expect external volunteers
(external when compared to committers) to maintain *all* the bug
statuses. What I want/need those to do is to take care of everything
that the system did *not* pick up properly, or any case when the
hacker/committer forgot something, or things like that.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SSI predicate locking on heap -- tuple or row?

2011-05-30 Thread Heikki Linnakangas

On 26.05.2011 06:19, Kevin Grittner wrote:

/*
 * Check whether the writer has become a pivot with an out-conflict
 * committed transaction, while neither reader nor writer is committed. 
If
 * the reader is a READ ONLY transaction, there is only a serialization
 * failure if an out-conflict transaction causing the pivot committed
 * before the reader acquired its snapshot.  (That is, the reader must 
not
 * have been concurrent with the out-conflict transaction.)
 */
if (!failure)
{
if (SxactHasSummaryConflictOut(writer))
{
failure = true;
conflict = NULL;
}
else
conflict = (RWConflict)
SHMQueueNext(writer-outConflicts,
 writer-outConflicts,
 
offsetof(RWConflictData, outLink));
while (conflict)
{
if (SxactIsCommitted(conflict-sxactIn)
 (!SxactIsCommitted(reader)
|| conflict-sxactIn-commitSeqNo = 
reader-commitSeqNo)
 (!SxactIsCommitted(writer)
|| conflict-sxactIn-commitSeqNo = 
writer-commitSeqNo)
 (!SxactIsReadOnly(reader)
|| conflict-sxactIn-commitSeqNo = 
reader-SeqNo.lastCommitBeforeSnapshot))
{
failure = true;
break;
}
conflict = (RWConflict)
SHMQueueNext(writer-outConflicts,
 conflict-outLink,
 
offsetof(RWConflictData, outLink));
}
}


The comment is not in sync with the code. The code is not checking that 
neither reader or writer has committed, but something more complicated.


Looking at OnConflict_CheckForSerializationFailure(), it's really hard 
to see how it works, and why the conditions it checks are sufficient. I 
find it helps tremendously to draw the dangerous structures being 
checked, in addition to just explaining them in text. Ascii art is a bit 
clunky, but I think we have to do it here.


I did some of that in the comments, and I think I understand it now. See 
attached patch. Does that look right to you? (note that I swapped the 
second and third check in the function, I think it's more 
straightforward that way). I also added a couple of questions about the 
conditions, marked with XXX comments. Can you answer those, please?


PS. Should we say Cancelled on identification as pivot, during ..., or 
Cancelled on identification as a pivot, during ... ? We seem to use 
both in the error messages.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com
*** a/src/backend/access/heap/heapam.c
--- b/src/backend/access/heap/heapam.c
***
*** 1529,1535  heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
  	OffsetNumber offnum;
  	bool		at_chain_start;
  	bool		valid;
- 	bool		match_found;
  
  	if (all_dead)
  		*all_dead = true;
--- 1529,1534 
***
*** 1539,1545  heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
  	Assert(ItemPointerGetBlockNumber(tid) == BufferGetBlockNumber(buffer));
  	offnum = ItemPointerGetOffsetNumber(tid);
  	at_chain_start = true;
- 	match_found = false;
  
  	/* Scan through possible multiple members of HOT-chain */
  	for (;;)
--- 1538,1543 
***
*** 1597,1606  heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
  			PredicateLockTuple(relation, heapTuple);
  			if (all_dead)
  *all_dead = false;
! 			if (IsolationIsSerializable())
! match_found = true;
! 			else
! return true;
  		}
  
  		/*
--- 1595,1601 
  			PredicateLockTuple(relation, heapTuple);
  			if (all_dead)
  *all_dead = false;
! 			return true;
  		}
  
  		/*
***
*** 1629,1635  heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
  			break;/* end of chain */
  	}
  
! 	return match_found;
  }
  
  /*
--- 1624,1630 
  			break;/* end of chain */
  	}
  
! 	return false;
  }
  
  /*
***
*** 2855,2866  l2:
  
  	END_CRIT_SECTION();
  
- 	/*
- 	 * Any existing SIREAD locks on the old tuple must be linked to the new
- 	 * tuple for conflict detection purposes.
- 	 */
- 	PredicateLockTupleRowVersionLink(relation, oldtup, heaptup);
- 
  	if (newbuf != buffer)
  		LockBuffer(newbuf, BUFFER_LOCK_UNLOCK);
  	LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
--- 2850,2855 

[HACKERS] Cube Index Size

2011-05-30 Thread Nick Raj
Hi,

Cube code provided by postgres contrib folder. It uses the NDBOX structure.
On creating index, it's size increase at a high rate.

On inserting some tuple and creating indexes its behaviour is shown below.

1. When there is only one tuple
select pg_size_pretty(pg_relation_
size('cubtest'));   //Table size without index
 pg_size_pretty

 8192 bytes
(1 row)

select pg_size_pretty(pg_total_relation_size('cubtest')); //Table size with
index
 pg_size_pretty

 16 kB
(1 row)

i.e. Index size in nearly 8kB

2. When tuples are 20,000

Table Size without index - 1.6 MB
Table Size with index - 11 MB
i.e. Index size is nearly 9.4 MB

3. When tuples are 5 lakh

Table Size without index - 40 MB
Table Size with index - 2117 MB
i.e. Index size is nearly 2077 MB ~ 2GB
It is taking nearly 20-25 min for creating index for 5 lakh tuples.

Can some one tell me why index is becoming so large?
How to compress or reduce its size?

Thanks
Nick


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Christopher Browne
On Mon, May 30, 2011 at 2:26 AM, Greg Stark gsst...@mit.edu wrote:
 On Sun, May 29, 2011 at 3:36 PM, Joe Abbate j...@freedomcircle.com wrote:
  Anyone interested in the tracker, please visit
 http://wiki.postgresql.org/wiki/TrackerDiscussion and add your
 feedback/input.

 I think this illustrates exactly what we *don't* want to happen with a
 bug tracker. We want the discussion to stay *here* not on some other
 medium accessible only through the web and editable only through a web
 interface

That's more or less why I was suggesting SD as a possible model, as a
bug tracker that begins with a command line interface consciously
analogous to version management software.  (See attachment for samples
of the help...)

 Also your summary seems to have missed the point on the has email
 interface requirement. The table of features you listed has just
 Creation of bugs via mail interface as the only feature that is
 accessible from email.

I recall RT (on one of the lists) having a somewhat sophisticated
email-based interface, however, I'm not at all sure that this would be
considered a good thing, as it would be pretty in your face that you
are submitting specially-constructed email messages to control things.

 I'm not sure what Robert meant but I suspect he meant what I would
 want which is the ability to add comments, close bugs, set other
 properties, etc. By email. My biggest gripe about bugzilla was that it
 sent you an email with updates to the bug but you couldn't respond to
 that email.

Having used a number of versions of Bugzilla over the years, I'm
somewhat comfortable with its foibles, but that's not nearly the same
thing as actually liking it.

 My ideal bug tracker is the debian one which basically stays out of
 your way and lets you cc any message to a specific bug at
 n...@bugs.debian.org which archives that message in the bug and sends
 it to anyone listening to the bug. And you can have control commands
 to close it or edit it -- basically making all our existing that's
 not a bug bleah bleah messages into close nnn; that's not a bug
 bleah bleah messages.

http://www.chiark.greenend.org.uk/~ian/debbugs/

I suppose it would be interesting to inject a little more code into
this that would collect other interesting bits of data, such as the
commit hash of a patch that is believed to fix the bug, and version
numbers believed to include fixes for the bug.  Also interesting would
be a reference to commitfest work relating to the bug.

Perhaps it's enough to just send an email to the bug indicating
appropriate URLs, as opposed to requiring any first-class extensions
to support this sort of data.

I think we'd probably want a web interface that can point not merely
to messages, but also to the whole threads of discussion.  That way,
reporting that an email thread relates to bug 72521 requires only that
*ONE* of the messages in the thread includes cc:
72...@bugs.postgresql.org (or similar).
-- 
When confronted by a difficult problem, solve it by reducing it to the
question, How would the Lone Ranger handle this?


sd.help
Description: Binary data


sd.tickets
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Unfriendly handling of pg_hba SSL options with SSL off

2011-05-30 Thread Magnus Hagander
On Mon, May 30, 2011 at 20:39, Tom Lane t...@sss.pgh.pa.us wrote:
 Magnus Hagander mag...@hagander.net writes:
 On Fri, May 13, 2011 at 00:21, Tom Lane t...@sss.pgh.pa.us wrote:
 Magnus Hagander mag...@hagander.net writes:
 On Tue, May 10, 2011 at 05:39, Tom Lane t...@sss.pgh.pa.us wrote:
 I wouldn't have a problem with making the Windows port throw an error
 for local lines. We'd have to fix initdb to remove that line from the
 sample file (if it doesn't already), but that's surely not hard.

 Here's a patch that I think does what we want. It works fine on
 Windows - I just want to make sure this is what you meant as well?

 Looks sane to me.

Ok, thanks. Applied.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] SSI predicate locking on heap -- tuple or row?

2011-05-30 Thread Heikki Linnakangas

On 30.05.2011 17:10, Kevin Grittner wrote:

Heikki Linnakangas  wrote:
On 26.05.2011 06:19, Kevin Grittner wrote:
I did some of that in the comments, and I think I understand it
now. See attached patch. Does that look right to you?


!  * A serialization failure can only occur if there is a dangerous
!  * structure in the dependency graph:
!  *
!  *Tin --  Tpivot --  Tout
!  * rw rw
!  *
!  * Furthermore, Tout must commit first.

I agree that this is easier to read and understand than just straight
text; but you dropped one other condition, which we use rather
heavily.  We should probably add back something like:

* If Tin is declared READ ONLY (or commits without writing), we can
* only have a problem if Tout committed before Tin acquired its
* snapshot.


 * XXX: Where does that last condition come from?


This optimization is an original one, not yet appearing in any
academic papers; Dan and I are both convinced it is safe, and in
off-list correspondence with Michael Cahill after he left Oracle, he
said that he discussed this with Alan Fekete and they both concur
that it is a safe and good optimization.

This optimization is mentioned in the README-SSI file in the
Innovations section.  Do you think that file needs to have more on
the topic?


Oh, I see this:


  o We can avoid ever rolling back a transaction when the
transaction on the conflict *in* side of the pivot is explicitly or
implicitly READ ONLY unless the transaction on the conflict *out*
side of the pivot committed before the READ ONLY transaction acquired
its snapshot. (An implicit READ ONLY transaction is one which
committed without writing, even though it was not explicitly declared
to be READ ONLY.)


Since this isn't coming from the papers, it would be nice to explain why 
that is safe.



  * XXX: for an anomaly to occur, T2 must've committed before W.
  * Couldn't we easily check that here, or does the fact that W
  * committed already somehow imply it?


The flags and macros should probably be renamed to make it more
obvious that this is covered.  The flag which SxactHasConflictOut is
based on is only set when the conflict out is to a transaction which
committed ahead of the flagged transaction.  Regarding the other
test -- since we have two transactions (Tin and Tout) which have not
been summarized, and transactions are summarized in order of commit,
SxactHasSummaryConflictOut implies that the transaction with the flag
set has not committed before the transaction to which it has a
rw-conflict out.


Ah, got it.


The problem is coming up with a more accurate name which isn't really
long.  The best I can think of is SxactHasConflictOutToEarlierCommit,
which is a little long.  If nobody has a better idea, I guess it's
better to have a long name than a misleading one.  Do you think we
need to rename SxactHasSummaryConflictOut or just add a comment on
that use that a summarized transaction will always be committed ahead
of any transactions which are not summarized?


Dunno. I guess a comment would do. Can you write a separate patch for 
that, please?



(note that I swapped the second and third check in the function, I
think it's more straightforward that way).


It doesn't matter to me either way, so if it seems clearer to you,
that's a win.


FWIW, the reason I prefer it that way is that the function is checking 
three scenarios:


R  W  T2, where W committed after T2
R  W  T2, else
T0  R  W

It seems more straightforward to keep both R  W  T2 cases 
together.



Should we say Cancelled on identification as pivot, during ...,
or Cancelled on identification as a pivot, during ... ? We seem
to use both in the error messages.


Consistency is good.  I think it sounds better with the indefinite
article in there.


Ok, will do.


Do you want me to post another patch based on this, or are these
changes from what you posted small enough that you would prefer to
just do it as part of the commit?


I've committed with the discussed changes.

--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] adding applications to the stack builder

2011-05-30 Thread Gaetano Giunta

Hello

I would like to know what is the process to get new applications accepted for 
inclusion in the stack builder (namely the eZ Publish cms).

I would be ready spend some time to package the application according to some specific format, and to host the built package on some dedicated server if there 
is need - but the only thing I've found so far is a project in pgfoundry that looks abandonware (not a lot of activity since 2007...)


thanks
Gaetano


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Greg Stark
On Sun, May 29, 2011 at 10:09 PM, Stefan Kaltenbrunner
ste...@kaltenbrunner.cc wrote:
 well bugzilla has an inbound email interface as well that can both be
 used to creande and to manipulate bugs (as in mails that have the
 bug-id in the subject will be added as a comment).

Ugh, putting it in the subject plays poorly with MUAs like gmail that
don't understand threading and group messages by subject.

-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Cube Index Size

2011-05-30 Thread Nick Raj
Hi,

Cube code provided by postgres contrib folder. It uses the NDBOX structure.
On creating index, it's size increase at a high rate.

On inserting some tuple and creating indexes its behaviour is shown below.

1. When there is only one tuple
select pg_size_pretty(pg_relation_size('cubtest'));   //Table size
without index
 pg_size_pretty

 8192 bytes
(1 row)

select pg_size_pretty(pg_total_relation_size('cubtest')); //Table size with
index
 pg_size_pretty

 16 kB
(1 row)

i.e. Index size in nearly 8kB

2. When tuples are 20,000

Table Size without index - 1.6 MB
Table Size with index - 11 MB
i.e. Index size is nearly 9.4 MB

3. When tuples are 5 lakh

Table Size without index - 40 MB
Table Size with index - 2117 MB
i.e. Index size is nearly 2077 MB ~ 2GB
It is taking nearly 20-25 min for creating index for 5 lakh tuples.

Can some one tell me why index is becoming so large?
How to compress or reduce its size?

Thanks
Nick


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Joe Abbate
Hi Greg,

On 05/29/2011 10:26 PM, Greg Stark wrote:
 On Sun, May 29, 2011 at 3:36 PM, Joe Abbate j...@freedomcircle.com wrote:
  Anyone interested in the tracker, please visit
 http://wiki.postgresql.org/wiki/TrackerDiscussion and add your
 feedback/input.
 
 I think this illustrates exactly what we *don't* want to happen with a
 bug tracker. We want the discussion to stay *here* not on some other
 medium accessible only through the web and editable only through a web
 interface

I have no problem keeping the discussion here, but I thought perhaps not
everyone on -hackers wanted to see the discussion (there was a -tracker
list that became defunct, according to the page--don't know if people
want to resurrect it).

 Also your summary seems to have missed the point on the has email
 interface requirement. The table of features you listed has just
 Creation of bugs via mail interface as the only feature that is
 accessible from email.

My summary is in the section titled Discussion Points (and it was not
meant to be all-inclusive).  The second section, titled Previous
Content was there before and I didn't want to eliminate it entirely.
You're referring to the second section.

 I'm not sure what Robert meant but I suspect he meant what I would
 want which is the ability to add comments, close bugs, set other
 properties, etc. By email. My biggest gripe about bugzilla was that it
 sent you an email with updates to the bug but you couldn't respond to
 that email.
 
 My ideal bug tracker is the debian one which basically stays out of
 your way and lets you cc any message to a specific bug at
 n...@bugs.debian.org which archives that message in the bug and sends
 it to anyone listening to the bug. And you can have control commands
 to close it or edit it -- basically making all our existing that's
 not a bug bleah bleah messages into close nnn; that's not a bug
 bleah bleah messages.

I see that a full interface is very desirable to you (and others).  That
confirms the order of my summary list of requirements (mail interface is
listed before web interface).  I'll admit that I became interest in
assisting with this effort due to the latter rather than the former, but
I don't mind carrying the ball forward, for now.

All the best,

Joe

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Fix for GiST penalty

2011-05-30 Thread Alexander Korotkov
Hi!

During my work on GSoC project, I found bad perfomace of GiST for point
datatype. It appears even on uniform random data.

test=# create table test as (select point(random(), random()) from
generate_series(1, 100));
SELECT 100
test=# create index test_idx on test using gist(point);
CREATE INDEX
test=# explain (analyze, buffers) select * from test where point @
box(point(0.5,0.5), point(0.505,0.505));
   QUERY PLAN


 Bitmap Heap Scan on test  (cost=48.40..2593.73 rows=1000 width=16) (actual
time=97.479..97.551 rows=24 loops=1)
   Recheck Cond: (point @ '(0.505,0.505),(0.5,0.5)'::box)
   Buffers: shared hit=5126
   -  Bitmap Index Scan on test_idx  (cost=0.00..48.15 rows=1000 width=0)
(actual time=97.462..97.462 rows=24 loops=1)
 Index Cond: (point @ '(0.505,0.505),(0.5,0.5)'::box)
 Buffers: shared hit=5102
 Total runtime: 97.644 ms
(7 rows)

Search for the cause takes relatively long time from me, but finally I did.
In gist_box_penalty function floating point error in line
  *result = (float) (size_box(ud) - size_box(origentry-key));
sometimes makes *result a very small negative number.
I beleive that best place to fix it is gistpenalty function. The attached
patch makes this function treating negative number from user's penalty as
zero. I didn't find mention of negative penalty value in documentation. So,
AFAICS such behaviour shouldn't break anything.
After the patch index performance is ok.

test=# explain (analyze, buffers) select * from test where point @
box(point(0.5,0.5), point(0.505,0.505));
  QUERY PLAN

--
 Bitmap Heap Scan on test  (cost=44.35..2589.68 rows=1000 width=16) (actual
time=0.988..1.116 rows=24 loops=1)
   Recheck Cond: (point @ '(0.505,0.505),(0.5,0.5)'::box)
   Buffers: shared hit=44
   -  Bitmap Index Scan on test_idx  (cost=0.00..44.10 rows=1000 width=0)
(actual time=0.966..0.966 rows=24 loops=1)
 Index Cond: (point @ '(0.505,0.505),(0.5,0.5)'::box)
 Buffers: shared hit=20
 Total runtime: 1.313 ms
(7 rows)

--
With best regards,
Alexander Korotkov.
*** a/src/backend/access/gist/gistutil.c
--- b/src/backend/access/gist/gistutil.c
***
*** 526,536  gistpenalty(GISTSTATE *giststate, int attno,
--- 526,540 
  
  	if (giststate-penaltyFn[attno].fn_strict == FALSE ||
  		(isNullOrig == FALSE  isNullAdd == FALSE))
+ 	{
  		FunctionCall3Coll(giststate-penaltyFn[attno],
  		  giststate-supportCollation[attno],
  		  PointerGetDatum(orig),
  		  PointerGetDatum(add),
  		  PointerGetDatum(penalty));
+ 		if (penalty  0.0)
+ 			penalty = 0.0;
+ 	}
  	else if (isNullOrig  isNullAdd)
  		penalty = 0.0;
  	else

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] switch UNLOGGED to LOGGED

2011-05-30 Thread Leonardo Francalanci
 Why is it necessary to replay the operation only on the slave?   Can we
 just use XLOG_HEAP_NEWPAGE?


Uh, I don't know why but I thought I shouldn't log a page on the master,
since all the pages are already there and fsync-ed. But if it makes no harm,
I can easily use   XLOG_HEAP_NEWPAGE (of course, only in the 
 wal_level != minimal case).

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers



Re: [HACKERS] Nested CASE-WHEN scoping

2011-05-30 Thread Heikki Linnakangas

On 26.05.2011 00:31, Tom Lane wrote:

The main annoyance here is that standalone expressions may now need
Param slots to execute, which will complicate places that need to
evaluate them, but there's probably no way around that


Yeah, I can see two complications:

1. Domain constraints

Domain constraint check expressions are fetched and constant-folded 
separately from the enclosing expression, in ExecInitExpr(). In order to 
allocate distinct paramids for any CASE values within the domain check 
expression, we'd need to know how many paramids are in use in the parent 
expression. We could dig into the parent PlanState and its EState to get 
that, but we don't have that for stand-alone expressions.


2. Index expressions

Index expressions are stored in relcache after constant evaluation, so 
any CaseTestExprs will already be replaced with Params. When the recheck 
expression is evaluated, the paramids used in the recheck expression 
will clash with real PARAM_EXEC params used to pass values up and down 
subqueries, as well as any params used for CASEs.


I think we can work around both of those by just saving and restoring 
the value of each Param that we set while evaluating an expression, as 
the values should not be used simultaneously, but it makes me a bit 
uncomfortable. If we ever try to inline those expressions into other 
expressions, we'll be right back to the issue we have with nested CASE 
now. I'm not sure if we might already do that for index recheck 
expressions. Alternatively, we could adjust the paramids when an 
expression is inlined into another, similar to what OffsetVarNodes does 
for Vars.


For debugging purposes, it seems like a good idea to invent a new kind 
of Param for these, and keep them separate from PARAM_EXEC params. The 
scope would be different, PARAM_EXECs are used to pass values between 
planner nodes, while these new kind of Params would be used to pass 
values within a single expression.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Joe Abbate
Hi Magnus,

On 05/30/2011 08:45 AM, Magnus Hagander wrote:
 It's fine that a bug tracker *tracks* bugs. It should not control
 them. That's not how this community currently works, and a lot of
 people have said that's how they want it to stay (at least for now).

If I may belabor the point, what do you see as an example of
controlling the bugs?  To put some context, there could be at least
three ways a bug could be closed when someone commits a patch that fixes
(or claims to fix) a bug:

a. The committer has to use a web interface to indicate the bug is closed
b. The committer has to send an email to a mail interface
c. The commit message gets routed to a mail interface that, seeing
something like bug #1234 in the first line, automatically closes the bug

Based on the discussion so far, it's obvious that option b is more
desired than a (where the tracker is, in a sense, controlling *you*),
but is option c --while presumably more desirable since there's one less
thing to do or remember-- an instance of control, since the tracker
takes an automatic action?  Or do you want the tracker *not* to require
or take any of the actions, i.e., let someone/thing other than the
committer/commit message worry about tracking the bug's status, leaving
it up to volunteers, as Tom said?

Joe

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Please test peer (socket ident) auth on *BSD

2011-05-30 Thread Tom Lane
I've applied patches to fix Martin Pitt's report of peer auth failing on
FreeBSD-amd64 kernels.  I tested it with FreeBSD but do not have the
resources to check every other platform that uses the same code branch
in auth_peer.  The buildfarm will soon tell us if the patches fail to
compile anywhere, but since the buildfarm doesn't test that
authentication path, it's not going to be as obvious whether it works.

So, if you have a BSD-ish machine, please try HEAD and see if peer auth
(or ident auth in older branches) still works for you.  Extra points
if you find out it used to be broken on your machine.  (Hey Stefan, did
you ever try that on spoonbill?)

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Christopher Browne
On 2011-05-30 4:31 PM, Peter Eisentraut pete...@gmx.net wrote:

 On sön, 2011-05-29 at 18:36 -0400, Joe Abbate wrote:
  I've summarizes the main points made in the recent discussion and did
  some minor additional research on the lists suggested by Peter and
  Chris Browne.  Anyone interested in the tracker, please visit
  http://wiki.postgresql.org/wiki/TrackerDiscussion and add your
  feedback/input.

 Based on that, and past discussions, and things we've tried in the past,
 and gut feeling, and so on, it looks like Request Tracker would appear
 to be the next best thing to consider trying out.  What do people think
 about that?

My suspicion is that RT may be rather a lot heavier weight in terms of how
it would have to affect process than people would be happy with.

What has been pretty clearly expressed is that various of the developers
prefer for the mailing lists and archives thereof to be the primary data
source and the venue for bug discussions.

RT, and Bugzilla, and pretty well the bulk of the issue trackers out there
are designed to themselves be the venue for discussions, and that's not
consistent with the preference for email discussions.

There are Debian packages for RT 3.8, and I imagine it may be worth tossing
an instance, but I'd definitely commend trying to minimize the amount of
deployment effort done, as I think there's a fair chance that a number of
devs (I'll pick on Greg Stark :-)) are liable to rebel against it.  It'd be
interesting to see the reactions to the interaction between RT, -hackers,
and -bugs for a bug or three...

I'd be more optimistic that debbugs, or an adaption thereof, might more
nearly fit into the workflow.


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Robert Haas
On Mon, May 30, 2011 at 8:16 PM, Christopher Browne cbbro...@gmail.com wrote:
 On 2011-05-30 4:31 PM, Peter Eisentraut pete...@gmx.net wrote:
 On sön, 2011-05-29 at 18:36 -0400, Joe Abbate wrote:
  I've summarizes the main points made in the recent discussion and did
  some minor additional research on the lists suggested by Peter and
  Chris Browne.  Anyone interested in the tracker, please visit
  http://wiki.postgresql.org/wiki/TrackerDiscussion and add your
  feedback/input.

 Based on that, and past discussions, and things we've tried in the past,
 and gut feeling, and so on, it looks like Request Tracker would appear
 to be the next best thing to consider trying out.  What do people think
 about that?

 My suspicion is that RT may be rather a lot heavier weight in terms of how
 it would have to affect process than people would be happy with.

 What has been pretty clearly expressed is that various of the developers
 prefer for the mailing lists and archives thereof to be the primary data
 source and the venue for bug discussions.

 RT, and Bugzilla, and pretty well the bulk of the issue trackers out there
 are designed to themselves be the venue for discussions, and that's not
 consistent with the preference for email discussions.

 There are Debian packages for RT 3.8, and I imagine it may be worth tossing
 an instance, but I'd definitely commend trying to minimize the amount of
 deployment effort done, as I think there's a fair chance that a number of
 devs (I'll pick on Greg Stark :-)) are liable to rebel against it.  It'd be
 interesting to see the reactions to the interaction between RT, -hackers,
 and -bugs for a bug or three...

 I'd be more optimistic that debbugs, or an adaption thereof, might more
 nearly fit into the workflow.

Yeah, that's my feeling, as well.  I have used RT and I found that the
web interface was both difficult to use and unwieldly for tickets
containing large numbers of messages.  Maybe those those things have
been improved, but frankly if RT or Bugzilla is the best we can come
up with then I'd rather not have a bug tracker at all.  See also:
Linus's opinion on CVS.

I don't personally care if I need to go to a web interface to mark
bugs closed.  Being able to do it via email is possibly useful, but I
don't really care about it personally.  Sounds like we should have it
for the benefit of those who do, but it's not my priority.  What I do
care about is that the tracker doesn't get in the way of *discussion*
of bugs.  IOW, if people just reply-to-all on bug reports as they do
now, and either include some special tag in the subject line or copy
some special address on the CC list, it should all get sucked into the
appropriate bug report.  The number of people reading and replying to
emails on pgsql-bugs is already insufficient, perhaps because of the
(incorrect) perception that Tom does or will fix everything and no one
else needs to care.  So anything that makes it harder for people to
follow along and participate is a non-starter IMV.

Based on the discussion thus far, it sounds like debbugs might be
reasonably close to what we need.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] switch UNLOGGED to LOGGED

2011-05-30 Thread Robert Haas
On Sun, May 29, 2011 at 4:29 AM, Noah Misch n...@leadboat.com wrote:
 I don't think it is *necessary*.  If we're replaying WAL on a master, we'll 
 also
 be resetting unlogged relations after recovery; what we write or do not write 
 to
 them in the mean time has no functional impact.  Seemed like a sensible
 optimization, but maybe it's premature.

Some jiggering may be necessary, because right now we remove the main
forks at the start of recovery and repopulate them at the end.  It's
not immediately obvious to me that that's going to work well with
HEAP_XLOG_NEWPAGE, but then it's not immediately obvious to me that
it's going to work well with a new WAL record type, either.  I think
we need a detailed design document for how this is all going to work.
We need to not only handle the master properly but also handle the
slave properly.  Consider, for example, the case where the slave
begins to replay the transaction, reaches a restartpoint after
replaying some of the new pages, and then crashes.  If the subsequent
restart from the restartpoint blows away the main relation fork, we're
hosed.  I fear we're plunging into implementation details without
having a good overall design in mind first.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Brendan Jurd
On 31 May 2011 11:52, Robert Haas robertmh...@gmail.com wrote:
 I have used RT and I found that the
 web interface was both difficult to use and unwieldly for tickets
 containing large numbers of messages.

A big loud ditto from me on this point.

Cheers,
BJ

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Tom Lane
Christopher Browne cbbro...@gmail.com writes:
 On 2011-05-30 4:31 PM, Peter Eisentraut pete...@gmx.net wrote:
 Based on that, and past discussions, and things we've tried in the past,
 and gut feeling, and so on, it looks like Request Tracker would appear
 to be the next best thing to consider trying out.  What do people think
 about that?

 I'd be more optimistic that debbugs, or an adaption thereof, might more
 nearly fit into the workflow.

Yeah, that's my impression as well.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread k...@rice.edu
On Mon, May 30, 2011 at 09:52:38PM -0400, Robert Haas wrote:
 On Mon, May 30, 2011 at 8:16 PM, Christopher Browne cbbro...@gmail.com 
 wrote:
  On 2011-05-30 4:31 PM, Peter Eisentraut pete...@gmx.net wrote:
  On sön, 2011-05-29 at 18:36 -0400, Joe Abbate wrote:
   I've summarizes the main points made in the recent discussion and did
   some minor additional research on the lists suggested by Peter and
   Chris Browne.  Anyone interested in the tracker, please visit
   http://wiki.postgresql.org/wiki/TrackerDiscussion and add your
   feedback/input.
 
  Based on that, and past discussions, and things we've tried in the past,
  and gut feeling, and so on, it looks like Request Tracker would appear
  to be the next best thing to consider trying out.  What do people think
  about that?
 
  My suspicion is that RT may be rather a lot heavier weight in terms of how
  it would have to affect process than people would be happy with.
 
  What has been pretty clearly expressed is that various of the developers
  prefer for the mailing lists and archives thereof to be the primary data
  source and the venue for bug discussions.
 
  RT, and Bugzilla, and pretty well the bulk of the issue trackers out there
  are designed to themselves be the venue for discussions, and that's not
  consistent with the preference for email discussions.
 
  There are Debian packages for RT 3.8, and I imagine it may be worth tossing
  an instance, but I'd definitely commend trying to minimize the amount of
  deployment effort done, as I think there's a fair chance that a number of
  devs (I'll pick on Greg Stark :-)) are liable to rebel against it.  It'd be
  interesting to see the reactions to the interaction between RT, -hackers,
  and -bugs for a bug or three...
 
  I'd be more optimistic that debbugs, or an adaption thereof, might more
  nearly fit into the workflow.
 
 Yeah, that's my feeling, as well.  I have used RT and I found that the
 web interface was both difficult to use and unwieldly for tickets
 containing large numbers of messages.  Maybe those those things have
 been improved, but frankly if RT or Bugzilla is the best we can come
 up with then I'd rather not have a bug tracker at all.  See also:
 Linus's opinion on CVS.
 
 I don't personally care if I need to go to a web interface to mark
 bugs closed.  Being able to do it via email is possibly useful, but I
 don't really care about it personally.  Sounds like we should have it
 for the benefit of those who do, but it's not my priority.  What I do
 care about is that the tracker doesn't get in the way of *discussion*
 of bugs.  IOW, if people just reply-to-all on bug reports as they do
 now, and either include some special tag in the subject line or copy
 some special address on the CC list, it should all get sucked into the
 appropriate bug report.  The number of people reading and replying to
 emails on pgsql-bugs is already insufficient, perhaps because of the
 (incorrect) perception that Tom does or will fix everything and no one
 else needs to care.  So anything that makes it harder for people to
 follow along and participate is a non-starter IMV.
 
 Based on the discussion thus far, it sounds like debbugs might be
 reasonably close to what we need.
 
 -- 
 Robert Haas
 EnterpriseDB: http://www.enterprisedb.com
 The Enterprise PostgreSQL Company
 

We use RT here and it is very customizable. In particular, it is easy
to have the basic process be completely via Email, if desired. It seems
that the general opinion is to use Email and consolidate the information
in the bug tracking system. RT can definitely step into the background
as needed.

Regards,
Ken

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Please test peer (socket ident) auth on *BSD

2011-05-30 Thread Tom Lane
I wrote:
 I've applied patches to fix Martin Pitt's report of peer auth failing on
 FreeBSD-amd64 kernels.  I tested it with FreeBSD but do not have the
 resources to check every other platform that uses the same code branch
 in auth_peer.  The buildfarm will soon tell us if the patches fail to
 compile anywhere, but since the buildfarm doesn't test that
 authentication path, it's not going to be as obvious whether it works.

 So, if you have a BSD-ish machine, please try HEAD and see if peer auth
 (or ident auth in older branches) still works for you.  Extra points
 if you find out it used to be broken on your machine.  (Hey Stefan, did
 you ever try that on spoonbill?)

BTW, after looking more closely at the buildfarm configure logs, it
appears that both OpenBSD and NetBSD have getpeereid(), which means
that they don't use this code at all.  It is currently looking to me
like the HAVE_STRUCT_FCRED and HAVE_STRUCT_SOCKCRED variants are dead
code.  They've been in there since before we had the getpeereid() code
path, and presumably were needed at one time ... but does anyone know
of a platform where they're still needed?

I'm a bit inclined to rip that code out of HEAD, if we can't point to a
platform where it'd be needed, just to reduce the #ifdef spaghetti.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Andrew Dunstan



On 05/30/2011 09:52 PM, Robert Haas wrote:

I have used RT and I found that the
web interface was both difficult to use and unwieldly for tickets
containing large numbers of messages.  Maybe those those things have
been improved, but frankly if RT or Bugzilla is the best we can come
up with then I'd rather not have a bug tracker at all.  See also:
Linus's opinion on CVS.




This is just the sort of argument that's stopped us in the past. My 
guess that that everybody's favourite tracker is someone else's least 
favourite.


I have a slight preference for Bugzilla for no other reasons than 
familiarity and the fact that I did a good deal of the work that allowed 
it to run on Postgres some years ago. Also, I'd be happier if we could 
leverage the good work that Stefan did a few years ago.


Some years ago I was involved in doing a substantial study of trackers 
and SCMs for a company I was working for. One of the conclusions the 
study group came to was that there should be good integration between 
the tracker system and the SCM. That was in the days before distributed 
SCMs were common, and in a commercial context, so I'm not sure how well 
our reasoning would stand up for the current context, but I see it's 
been mentioned elsewhere and I think it's a significant consideration, 
at least.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Kim Bisgaard

On 2011-05-30 04:26, Greg Stark wrote:

My biggest gripe about bugzilla was that it sent you an email with updates to 
the bug but you couldn't respond to that email.


Just checked bugzilla's list of features and they *now* lists that as supported:


File/Modify Bugs By Email

In addition to the web interface, you can send Bugzilla an email that will create a new bug, or will modify an existing bug. You can also 
very easily attach files to bugs this way.


http://www.bugzilla.org/features/#email-in

Regards,
Kim


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Greg Stark
On Mon, May 30, 2011 at 6:52 PM, Robert Haas robertmh...@gmail.com wrote:
  The number of people reading and replying to
 emails on pgsql-bugs is already insufficient, perhaps because of the
 (incorrect) perception that Tom does or will fix everything and no one
 else needs to care.  So anything that makes it harder for people to
 follow along and participate is a non-starter IMV.

Actually I think most of our bugs don't come in from pgsql-bugs. I
think we want to add other bugs that come up from discussions on
-hackers or -general which for whatever reason don't get immediately
fixed. The important thing about a bug tracker is that it has all the
bugs (at least all the ones you intend to fix) so they don't get
forgotten about. Keeping a single list takes the stress off
individuals trying to remember what needs to get done.

I'm actually not nearly so concerned as other people that it contain
all the detailed discussion of the bug -- we can always search for the
bug# on the list or follow links on the bug tracker.

Fwiw it is pretty nice to be able to include a Closes: #1001 in the
commit and have that close the bug and associate the commit to the
commit as soon as it's pushed. Anything to make keeping things clean
and up to date as simple and low-overhead as possible.

-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [ADMIN] 'SGT DETAIL: Could not open file pg_clog/05DC: No such file or directory' - what to do now?

2011-05-30 Thread Tom Lane
Tomasz Chmielewski man...@wpkg.org writes:
 bookstor=# SELECT 1 FROM core_wot_seq FOR UPDATE;

Um ... why are you doing that on a sequence?

 ERROR: could not access status of transaction 1573786613
 DETAIL: Could not open file pg_clog/05DC: No such file or directory.

This doesn't surprise me too much, because sequences are not expected
to contain any live XIDs, so the XID freezing mechanism ignores them.
So if you did that in the past, this would eventually happen.

I think the most appropriate solution may be to disallow SELECT FOR
UPDATE/SHARE on sequences ... so if you have a good reason why we
shouldn't do so, please explain it.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Tom Lane
Kim Bisgaard kim...@alleroedderne.adsl.dk writes:
 On 2011-05-30 04:26, Greg Stark wrote:
 My biggest gripe about bugzilla was that it sent you an email with updates 
 to the bug but you couldn't respond to that email.

 Just checked bugzilla's list of features and they *now* lists that as 
 supported:

 File/Modify Bugs By Email
 
 In addition to the web interface, you can send Bugzilla an email that will 
 create a new bug, or will modify an existing bug. You can also 
 very easily attach files to bugs this way.

The claim is there all right, but the feature seems spectacularly
undocumented otherwise.  I wanted to see if it worked like debbugs
(ie, you just cc: some mail to the bug tracker), but there's no
information about exactly how to use it.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Online base backup from the hot-standby

2011-05-30 Thread Jun Ishiduka
 I don't much like that approach. The standby would need to be able to
 write the backup history file to the archive at the end of backup, and
 we'd have to reintroduce the code to fetch it from archive and, when
 streaming, from the master. At the moment, the archiver doesn't even run
 in the standby.

Please teach the reason for The standby would need to be able to write 
the backup history file to the archive at the end of backup .
(I'd like to know why to only pg_xlog is wrong .)

Because there is the opinion of Cascade replication , I don't want to 
realize the function with the method which the standby requests to execute 
it on the primary server .

(The opinion of Cascade replication:
http://archives.postgresql.org/pgsql-hackers/2011-05/msg01150.php)


Jun Ishizuka
NTT Software Corporation
TEL:045-317-7018
E-Mail: ishizuka@po.ntts.co.jp




-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Getting a bug tracker for the Postgres project

2011-05-30 Thread Stefan Kaltenbrunner
On 05/31/2011 05:42 AM, Tom Lane wrote:
 Kim Bisgaard kim...@alleroedderne.adsl.dk writes:
 On 2011-05-30 04:26, Greg Stark wrote:
 My biggest gripe about bugzilla was that it sent you an email with updates 
 to the bug but you couldn't respond to that email.
 
 Just checked bugzilla's list of features and they *now* lists that as 
 supported:
 
 File/Modify Bugs By Email

 In addition to the web interface, you can send Bugzilla an email that will 
 create a new bug, or will modify an existing bug. You can also 
 very easily attach files to bugs this way.
 
 The claim is there all right, but the feature seems spectacularly
 undocumented otherwise.  I wanted to see if it worked like debbugs
 (ie, you just cc: some mail to the bug tracker), but there's no
 information about exactly how to use it.

Depends on what exactly you are looking for...

* that feature relies on finding a valid bugid in the subject, if it
finds one it will add the email ass a comment
* if you would prefer something like nn-...@tracker.postgresql.org
for adding to existing bugs, that would be a trivial thing to add as a
feature(have the MTA split the localpart and pass it as a parameter in
the pipe-transport to the email_in.pl script)
* the challenge is more about creating new bugs, because for that you
need a bz account (or maybe a community account in our case) by default.
We could certainly modify the feature so that it will autocreate bz
accounts as soon as we see a new emailaddress sending email in but that
will be fairly hard to control spamwise.



Stefan

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers