Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-04-26 Thread Simon Riggs
On Thu, Apr 26, 2012 at 1:35 AM, Jameison Martin wrote: > Tom, I whipped up some  INSERT/SELECT tests where I selected into a > temporary table as you suggested. The target temporary table and the source > table were in cache and I basically disabled things that would cause noise. > The source tab

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Simon Riggs
On Wed, Apr 25, 2012 at 7:34 PM, Robert Haas wrote: > On Wed, Apr 25, 2012 at 1:31 PM, Simon Riggs wrote: >> However, it is a fallacy that this is a good solution for using temp >> tables on HS. I think the wish to enhance Oracle compatibility is >> making some wishful thinking happen with regard

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Simon Riggs
On Thu, Apr 26, 2012 at 5:57 AM, Robert Haas wrote: > On Wed, Apr 25, 2012 at 10:04 PM, Noah Misch wrote: >> Based on the range of assessments spanning your "almost useless" to Merlin's >> "killer feature", I gather that its utility is exceptionally site-specific. > > Well said, sir. Not well sa

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Kevin Grittner
> Robert Haas wrote: > Noah Misch wrote: >> Based on the range of assessments spanning your "almost useless" >> to Merlin's "killer feature", I gather that its utility is >> exceptionally site-specific. > > Well said, sir. +1 > I find it a rather elegant and well-thought-out kludge. Global

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Robert Haas
On Thu, Apr 26, 2012 at 3:56 AM, Simon Riggs wrote: > Previous discussions had Tom proposing ways of extending catalogs to > allow exactly this. So designs that address that point are already on > record. Link? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL C

[HACKERS] plpython crash (PG 92)

2012-04-26 Thread Asif Naeem
Hi, PFA test case. It used simple select statement to retrieve data via plpython. It crashes latest pg 9.2 with the following stack trace i.e. #0 0x0073021f in pfree () > #1 0x7fa74b632f7a in PLy_result_dealloc () from > /home/masif/work/postgresql/postgresql/inst/lib/plpython2.so >

Re: [HACKERS] plpython crash (PG 92)

2012-04-26 Thread Asif Naeem
FYI, I have observed this crash on Linux64. Thanks. Best Regards, Muhammad Asif Naeem On Thu, Apr 26, 2012 at 5:32 PM, Asif Naeem wrote: > Hi, > > PFA test case. It used simple select statement to retrieve data via > plpython. It crashes latest pg 9.2 with the following stack trace i.e. > > #0

[HACKERS] Future In-Core Replication

2012-04-26 Thread Simon Riggs
I'm beginning to work on advanced additions to in-core replication for PostgreSQL. There are a number of additional features for existing single-master replication still to achieve, but the key topics to be addressed are major leaps forward in functionality. I hope to add useful features in 9.3, t

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Simon Riggs
On Thu, Apr 26, 2012 at 12:49 PM, Kevin Grittner wrote: >> So, I can't help wonder if what we're really missing here is a >> high-performance, log-based logical replication solution with good >> core support. On that, we do agree completely. That is exactly my goal for 9.3. > The GSoC xReader p

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Simon Riggs
On Thu, Apr 26, 2012 at 12:49 PM, Kevin Grittner wrote: >> I find it a rather elegant and well-thought-out kludge. > > Global temporary tables as a feature are far more than a kludge; I > assume you're talking about that feature as a solution for the "no > temporary tables on a hot standby" probl

Re: [HACKERS] 9.3: summary of corruption detection / checksums / CRCs discussion

2012-04-26 Thread Merlin Moncure
On Wed, Apr 25, 2012 at 9:28 AM, Merlin Moncure wrote: >> That patch actually has more than one optimization in it, I think, but >> the basic idea is that if we could figure out a way to set >> HEAP_XMIN_COMMITTED when loading data into a table created or >> truncated within the same transaction,

[HACKERS] Re: patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-04-26 Thread Greg Stark
On Thu, Apr 26, 2012 at 8:27 AM, Simon Riggs wrote: >> The source table had 5 integer columns, and was populated with 10 million >> rows. ... >>   2) target has all nullable columns, only the first column is set: the >> patch was slightly faster ... >> By slightly faster I'm talking on order of 10

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Greg Stark
On Thu, Apr 26, 2012 at 9:18 AM, Simon Riggs wrote: > Implementing a feature that *requires* those things is madness and > obscuring those crucial points is not balanced or fair. I think this whole discussion started the wrong way around. If the goal of implementing GTTs is to solve a need with r

Re: [HACKERS] Future In-Core Replication

2012-04-26 Thread Simon Riggs
On Thu, Apr 26, 2012 at 1:41 PM, Simon Riggs wrote: > I will also be organising a small-medium sized "Future of In-Core > Replication" meeting in Ottawa on Wed 16 May, 6-10pm. Thanks for such rapid response. I've put up a wiki page and will be adding names as they come through http://wiki.postg

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Robert Haas
On Thu, Apr 26, 2012 at 9:57 AM, Greg Stark wrote: > On Thu, Apr 26, 2012 at 9:18 AM, Simon Riggs wrote: >> Implementing a feature that *requires* those things is madness and >> obscuring those crucial points is not balanced or fair. > > I think this whole discussion started the wrong way around.

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Tom Lane
Robert Haas writes: > On Thu, Apr 26, 2012 at 3:56 AM, Simon Riggs wrote: >> Previous discussions had Tom proposing ways of extending catalogs to >> allow exactly this. So designs that address that point are already on >> record. > Link? There was a thread a couple years ago ... http://archives

Re: [HACKERS] Temporary tables under hot standby

2012-04-26 Thread Robert Haas
On Thu, Apr 26, 2012 at 10:12 AM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Apr 26, 2012 at 3:56 AM, Simon Riggs wrote: >>> Previous discussions had Tom proposing ways of extending catalogs to >>> allow exactly this. So designs that address that point are already on >>> record. > >> Link?

Re: [HACKERS] xReader, double-effort (was: Temporary tables under hot standby)

2012-04-26 Thread Simon Riggs
On Thu, Apr 26, 2012 at 3:40 PM, Kevin Grittner wrote: > [resending because of postgresql.org bounces on first try] > > Simon Riggs wrote: >> Kevin Grittner wrote: > >>> The GSoC xReader project is intended to be a major step toward >>> that, by providing a way to translate the WAL stream to a s

Re: [HACKERS] 9.2 release notes, beta time?

2012-04-26 Thread Simon Riggs
On Wed, Apr 25, 2012 at 7:59 PM, Robert Haas wrote: > On Wed, Apr 25, 2012 at 2:48 PM, Tom Lane wrote: >> Bruce Momjian writes: >>> I had hoped to complete the release notes by March 30, but travel and >>> catching up on old stuff has delayed that.  I will start working on them >>> now and have

[HACKERS] Assertion failure in indxpath.c

2012-04-26 Thread Heikki Linnakangas
I'm getting an assertion failure on a fresh git checkout with this query: create table foo (a int4, b text);CREATE TABLE create index i_foo on foo (b, a);CREATE INDEX select * from foo where a = 0 and (b = 'foo' or b = 'bar'); TRAP: FailedAssertion("!(const Node*)(ipath))->type) == T_IndexP

Re: [HACKERS] Request to add options to tools/git_changelog

2012-04-26 Thread Bruce Momjian
On Thu, Apr 26, 2012 at 01:26:16AM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Wed, Apr 25, 2012 at 05:09:04PM -0400, Tom Lane wrote: > >>> --details-after Show branch and author info after the commit description > > >> I don't understand the point of that. > > > The release notes have

Re: [HACKERS] Assertion failure in indxpath.c

2012-04-26 Thread Thom Brown
On 26 April 2012 17:32, Heikki Linnakangas wrote: > I'm getting an assertion failure on a fresh git checkout with this query: > > create table foo (a int4, b text);CREATE TABLE > create index i_foo on foo (b, a);CREATE INDEX > select * from foo where a = 0 and (b = 'foo' or b = 'bar'); > > TRAP: F

Re: [HACKERS] Request to add options to tools/git_changelog

2012-04-26 Thread Robert Haas
On Thu, Apr 26, 2012 at 1:26 AM, Tom Lane wrote: > Bruce Momjian writes: >> On Wed, Apr 25, 2012 at 05:09:04PM -0400, Tom Lane wrote: --details-after Show branch and author info after the commit description > >>> I don't understand the point of that. > >> The release notes have the author at

Re: [HACKERS] xReader, double-effort (was: Temporary tables under hot standby)

2012-04-26 Thread Robert Haas
On Thu, Apr 26, 2012 at 11:02 AM, Simon Riggs wrote: > On Thu, Apr 26, 2012 at 3:40 PM, Kevin Grittner > wrote: >> [resending because of postgresql.org bounces on first try] >> >> Simon Riggs wrote: >>> Kevin Grittner wrote: >> The GSoC xReader project is intended to be a major step toward

Re: [HACKERS] Request to add options to tools/git_changelog

2012-04-26 Thread Magnus Hagander
On Thu, Apr 26, 2012 at 18:56, Robert Haas wrote: > On Thu, Apr 26, 2012 at 1:26 AM, Tom Lane wrote: >> Bruce Momjian writes: >>> On Wed, Apr 25, 2012 at 05:09:04PM -0400, Tom Lane wrote: > --details-after Show branch and author info after the commit description >> I don't understand th

Re: [HACKERS] Request to add options to tools/git_changelog

2012-04-26 Thread Bruce Momjian
On Thu, Apr 26, 2012 at 06:59:18PM +0200, Magnus Hagander wrote: > On Thu, Apr 26, 2012 at 18:56, Robert Haas wrote: > > On Thu, Apr 26, 2012 at 1:26 AM, Tom Lane wrote: > >> Bruce Momjian writes: > >>> On Wed, Apr 25, 2012 at 05:09:04PM -0400, Tom Lane wrote: > > --details-after Show branch

Re: [HACKERS] xReader, double-effort (was: Temporary tables under hot standby)

2012-04-26 Thread Simon Riggs
On Thu, Apr 26, 2012 at 5:57 PM, Robert Haas wrote: > On Thu, Apr 26, 2012 at 11:02 AM, Simon Riggs wrote: >> On Thu, Apr 26, 2012 at 3:40 PM, Kevin Grittner >> wrote: >>> [resending because of postgresql.org bounces on first try] >>> >>> Simon Riggs wrote: Kevin Grittner wrote: >>> >

Re: [HACKERS] Assertion failure in indxpath.c

2012-04-26 Thread Tom Lane
Heikki Linnakangas writes: > I'm getting an assertion failure on a fresh git checkout with this query: > create table foo (a int4, b text);CREATE TABLE > create index i_foo on foo (b, a);CREATE INDEX > select * from foo where a = 0 and (b = 'foo' or b = 'bar'); > TRAP: FailedAssertion("!(cons

Re: [HACKERS] Request to add options to tools/git_changelog

2012-04-26 Thread Tom Lane
Bruce Momjian writes: > I agree adding rarely-used options to a tool doesn't make sense, but the > question is what percentage of the git_changelog userbase am I? 50% I think. The only thing that's really concerning me here is that the reverse-sort option seems likely to be bug-inducing, and I r

Re: [HACKERS] xReader, double-effort

2012-04-26 Thread Josh Berkus
>> The GSoC xReader project is intended to be a major step toward >> that, by providing a way to translate the WAL stream to a series >> of notifications of logical events to clients which register with >> xReader. >>> >>> Yes, it appears very similar. I didn't know about this. T

Re: [HACKERS] plpython triggers are broken for composite-type columns

2012-04-26 Thread Peter Eisentraut
On mån, 2012-04-23 at 02:25 +0200, Jan Urbański wrote: > It turned out not to be as straightforward as I though :( Yeah, been there ... > > The I/O code in PL/Python is a bit of a mess and that's something that > I'd like to address somewhere in the 9.3 development cycle. Right now > making the

Re: [HACKERS] psql omits row count under "\x auto"

2012-04-26 Thread Peter Eisentraut
On mån, 2012-04-23 at 12:30 -0400, Noah Misch wrote: > I've been enjoying "\x auto" in .psqlrc, but I noticed the row count footer > missing when it chooses ordinary output: > Looks like the logic in printQuery() needs further treatment. Hmm, this looks a bit tricky, because at the time we add th

[HACKERS] ExceptionalCondition() return type

2012-04-26 Thread Peter Eisentraut
I came across this comment: /* * ExceptionalCondition - Handles the failure of an Assert() * * Note: this can't actually return, but we declare it as returning int * because the TrapMacro() macro might get wonky otherwise. */ But it seems to me that this can easily be fixed like shown below,

Re: [HACKERS] Future In-Core Replication

2012-04-26 Thread Jim Nasby
On 4/26/12 7:41 AM, Simon Riggs wrote: 5. WRITE-SCALEABLE - the ability to partition data across nodes in a way that allows the solution to improve beyond the write rate of a single node. It would be valuable to look at READ-SCALEABLE as well; specifically a second form of "synchronous" replic

Re: [HACKERS] Request to add options to tools/git_changelog

2012-04-26 Thread Robert Haas
On Thu, Apr 26, 2012 at 2:05 PM, Tom Lane wrote: > Bruce Momjian writes: >> I agree adding rarely-used options to a tool doesn't make sense, but the >> question is what percentage of the git_changelog userbase am I? > > 50% I think.  The only thing that's really concerning me here is that > the r

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-04-26 Thread Jameison Martin
Simon and Greg, The math on space savings is assuming that columns will be used roughly from first to last as declared in the DDL, not a random distribution of column values. This is the case for the particular schema that I'm looking at. I'm not asserting that it is the common case in general,

Re: [HACKERS] ExceptionalCondition() return type

2012-04-26 Thread Tom Lane
Peter Eisentraut writes: > I came across this comment: > /* > * ExceptionalCondition - Handles the failure of an Assert() > * > * Note: this can't actually return, but we declare it as returning int > * because the TrapMacro() macro might get wonky otherwise. > */ > But it seems to me that t

Re: [HACKERS] Request to add options to tools/git_changelog

2012-04-26 Thread Andrew Dunstan
On 04/26/2012 02:54 PM, Robert Haas wrote: On Thu, Apr 26, 2012 at 2:05 PM, Tom Lane wrote: Bruce Momjian writes: I agree adding rarely-used options to a tool doesn't make sense, but the question is what percentage of the git_changelog userbase am I? 50% I think. The only thing that's rea

Re: [HACKERS] Request to add options to tools/git_changelog

2012-04-26 Thread Bruce Momjian
On Thu, Apr 26, 2012 at 02:05:23PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > I agree adding rarely-used options to a tool doesn't make sense, but the > > question is what percentage of the git_changelog userbase am I? > > 50% I think. The only thing that's really concerning me here is t

Re: [HACKERS] Future In-Core Replication

2012-04-26 Thread Josh Berkus
Simon, So the idea is that you'll present briefly your intentions for 9.3 at the developer meeting, and then have this in-depth afterwards? Sounds great. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make c

Re: [HACKERS] Future In-Core Replication

2012-04-26 Thread Robert Haas
On Thu, Apr 26, 2012 at 6:48 PM, Josh Berkus wrote: > So the idea is that you'll present briefly your intentions for 9.3 at > the developer meeting, and then have this in-depth afterwards?  Sounds > great. I really, really do not want the developer meeting to turn into a series of presentations.

Re: [HACKERS] index-only scans vs. Hot Standby, round two

2012-04-26 Thread Robert Haas
On Mon, Apr 16, 2012 at 4:13 PM, Robert Haas wrote: > But fundamentally we all seem to be converging on some variant of the > "soft conflict" idea. So, as a first step, I've committed a patch that just throws a hard conflict. I think we probably want to optimize this further, and I'm going to wo

Re: [HACKERS] Future In-Core Replication

2012-04-26 Thread Josh Berkus
Simon, > I'm beginning to work on advanced additions to in-core replication for > PostgreSQL. ... > Those are the basic requirements that I am trying to address. There > are a great many important details, but the core of this is probably > what I would call "logical replication", that is shipping

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-04-26 Thread Josh Berkus
Tom, > I can't help thinking that (a) this is an incredibly narrow use-case, > and (b) you'd be well advised to rethink your schema design anyway. It's more common than you'd think. Both EAV and Hstore have their own (severe) drawbacks. For example, I'm working on an application which collects

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-26 Thread Kyotaro HORIGUCHI
Hello, At Wed, 25 Apr 2012 02:31:24 +0900, Fujii Masao wrote in > > If we are allowed to be tolerant of the temporary lack of > > coherence in shared memory there, the spinlock could be removed. > > But the possibility to read garbage by using XLogCtl itself to > > access standbyMode does not s