Re: [HACKERS] [PATCH] Remove some duplicate if conditions

2014-01-02 Thread David Rowley
On Fri, Jan 3, 2014 at 6:59 AM, Tom Lane wrote: > David Rowley writes: > > I've attached a simple patch which removes some duplicate if conditions > > that seemed to have found their way into the code. > > > These are per PVS-Studio's warnings. > > -1. If PVS-Studio is complaining about this ty

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-02 Thread David Rowley
On Fri, Jan 3, 2014 at 12:23 PM, Erik Rijkers wrote: > On Fri, January 3, 2014 00:09, Erik Rijkers wrote: > > > > connection to server was lost > > > > So, to repeat, this runs fine on a server compiled for speed. > > > > I forgot to append the log messages: > > 2014-01-03 00:19:17.073 CET 14054

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Amit Kapila
On Fri, Jan 3, 2014 at 12:51 AM, Heikki Linnakangas wrote: > On 01/02/2014 02:24 PM, Rushabh Lathia wrote: >> >> Hi All, >> >> Test case: >> >> drop table if exists t; >> create table t(c text); >> insert into t values ('x'), (repeat(md5('abcdefghijklmnop'), 1)); >> select pg_column_size(c), p

Re: [HACKERS] [bug fix] connection service file doesn't take effect with ECPG apps

2014-01-02 Thread MauMau
From: "Michael Meskes" However, I'd prefer to solve the problem slightly differently by not creating an empty host variable instead of checking for it after the fact. But I take it you don't mind that. Fixed in HEAD and all back branches. Thanks for the report. Thank you for committing the

Re: [HACKERS] [PATCH] Make various variables read-only (const)

2014-01-02 Thread Wim Lewis
By an odd coincidence, I also decided to try to const-ify libpq recently, and noticed this thread as I was cleaning up my patch for submission. For what it's worth, I've attached my patch to this message. It doesn't move as much data into the text segment as Oskari Saarenmaa's patch does, but it is

Re: [HACKERS] truncating pg_multixact/members

2014-01-02 Thread Robert Haas
On Mon, Dec 30, 2013 at 10:59 PM, Alvaro Herrera wrote: > One problem I see is length of time before freezing multis: they live > for far too long, causing the SLRU files to eat way too much disk space. > I ran burnmulti in a loop, creating multis of 3 members each, with a min > freeze age of 50 m

Re: [HACKERS] fix_PGSTAT_NUM_TABENTRIES_macro patch

2014-01-02 Thread Tom Lane
Mark Dilger writes: > The mechanism that occurs to me (and I'm not wedded to > this idea) is: > typedef uint8 T_HOFF_TYPE; > typedef struct xl_heap_header > { >     uint16  t_infomask2; >     uint16  t_infomask; >     T_HOFF_TYPE t_hoff; > } xl_heap_header;

Re: [HACKERS] [PATCH] Store Extension Options

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 4:19 AM, Andres Freund wrote: > On 2013-12-31 13:37:59 +0100, Pavel Stehule wrote: >> > We use the namespace "ext" to the internal code >> > (src/backend/access/common/reloptions.c) skip some validations and store >> > the custom GUC. >> > >> > Do you think we don't need to

Re: [HACKERS] fix_PGSTAT_NUM_TABENTRIES_macro patch

2014-01-02 Thread Mark Dilger
The mechanism that occurs to me (and I'm not wedded to this idea) is: typedef uint8 T_HOFF_TYPE; typedef struct xl_heap_header {     uint16  t_infomask2;     uint16  t_infomask;     T_HOFF_TYPE t_hoff; } xl_heap_header; #define SizeOfHeapHeader    (offs

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-02 Thread Peter Geoghegan
On Thu, Jan 2, 2014 at 11:58 AM, Peter Geoghegan wrote: > My executive summary is that the exclusion patch performs about the > same on lower client counts, presumably due to not having the > additional window of btree lock contention. By 8 clients, the > exclusion patch does noticeably better, bu

Re: [HACKERS] Streaming replication bug in 9.3.2, "WAL contains references to invalid pages"

2014-01-02 Thread MauMau
From: "Christophe Pettus" We've had two clients experience a crash on the secondary of a streaming replication pair, running PostgreSQL 9.3.2. In both cases, the messages were close to this example: 2013-12-30 18:08:00.464 PST,,,23869,,52ab4839.5d3d,16,,2013-12-13 09:47:37 PST,1/0,0,WARNING

Re: [HACKERS] fix_PGSTAT_NUM_TABENTRIES_macro patch

2014-01-02 Thread Mark Dilger
I completely understand the padding issues that you are dealing with.  I was mostly curious why Tom wanted to use asserts to double-check the code in one place, while happily not doing so in what seemed the same kind of situation elsewhere. He has since made the reason for that clear. On Thurs

Re: [HACKERS] fix_PGSTAT_NUM_TABENTRIES_macro patch

2014-01-02 Thread Tom Lane
Mark Dilger writes: > I still don't understand why this case in src/include/pgstat.h > is different from cases elsewhere in the code. The reason why I'm exercised about it is that (a) somebody actually made a mistake of this type, and (b) it wasn't caught by any automated testing. The catalog an

Re: [HACKERS] fix_PGSTAT_NUM_TABENTRIES_macro patch

2014-01-02 Thread Andres Freund
On 2014-01-02 15:15:58 -0800, Mark Dilger wrote: > I still don't understand why this case in src/include/pgstat.h > is different from cases elsewhere in the code.  Taken from > src/include/access/heapam_xlog.h: > > > typedef struct xl_heap_header > { >     uint16  t_infomask2; >     uint16   

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-02 Thread Erik Rijkers
On Fri, January 3, 2014 00:09, Erik Rijkers wrote: > connection to server was lost > > So, to repeat, this runs fine on a server compiled for speed. > I forgot to append the log messages: 2014-01-03 00:19:17.073 CET 14054 LOG: database system is ready to accept connections TRAP: FailedAsserti

Re: [HACKERS] fix_PGSTAT_NUM_TABENTRIES_macro patch

2014-01-02 Thread Mark Dilger
I still don't understand why this case in src/include/pgstat.h is different from cases elsewhere in the code.  Taken from src/include/access/heapam_xlog.h: typedef struct xl_heap_header {     uint16  t_infomask2;     uint16  t_infomask;     uint8   t_hoff; } xl_heap_header; #define S

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-02 Thread Erik Rijkers
On Thu, January 2, 2014 17:33, Erik Rijkers wrote: > On Thu, January 2, 2014 13:36, Erik Rijkers wrote: >> On Thu, January 2, 2014 13:05, David Rowley wrote: >>> here's a slightly updated patch >>> [inverse_transition_functions_v1.8.patch.gz ] >> >> patch applies, and compiles (although with new wa

Re: [HACKERS] Planning time in explain/explain analyze

2014-01-02 Thread Andreas Karlsson
On 01/02/2014 04:08 AM, Robert Haas wrote: I'm wondering whether the time should be stored inside the PlannedStmt node instead of passing it around separately. One possible problem with the way you've done things here is that, in the case of a prepared statement, EXPLAIN ANALYZE will emit the tim

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-02 Thread Peter Geoghegan
On Thu, Jan 2, 2014 at 2:37 AM, Andres Freund wrote: > Locking the definitely visible row only works if there's a row matching > the index's columns. If the values of the new row don't have > corresponding values in all the indexes you have the same old race > conditions again. I still don't get

Re: [HACKERS] fix_PGSTAT_NUM_TABENTRIES_macro patch

2014-01-02 Thread Tom Lane
I wrote: > It occurs to me that, rather than trying to improve the struct definition > methodology, maybe we should just add static asserts to catch any > inconsistency here. It wouldn't be that hard: > #define PGSTAT_MAX_MSG_SIZE 1000 > #define PGSTAT_MSG_PAYLOAD(PGSTAT_MAX_MSG_SIZE - size

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-02 Thread Peter Geoghegan
On Thu, Jan 2, 2014 at 8:08 AM, Heikki Linnakangas wrote: >> Yeah, it seems like PromiseTupleInsertionLockAcquire should be locking >> the tuple, rather than the XID. > > Well, that would be ideal, because we already have tuple locks. It would be > nice to use the same concept for this. It's a bit

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Andres Freund
On 2014-01-02 16:05:09 -0500, Robert Haas wrote: > On Thu, Jan 2, 2014 at 3:19 PM, Andres Freund wrote: > >> I was wondering if we could somehow arrange to not > >> release the subtransaction's AccessShareLock on the table, as long as it > >> was protecting toasted references someplace. > > > > So

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 3:19 PM, Andres Freund wrote: >> I was wondering if we could somehow arrange to not >> release the subtransaction's AccessShareLock on the table, as long as it >> was protecting toasted references someplace. > > Sounds fairly ugly... I think the only principled fixes are to

Re: [HACKERS] CLUSTER FREEZE

2014-01-02 Thread Robert Haas
On Mon, Dec 23, 2013 at 6:53 AM, Andres Freund wrote: > On 2013-12-22 20:45:02 -0500, Robert Haas wrote: >> I suspect we ought to extend this to rewriting variants of ALTER TABLE >> as well, but a little thought is needed there. ATRewriteTables() >> appears to just call heap_insert() for each upd

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Andres Freund
On 2014-01-02 15:00:58 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2014-01-02 21:21:15 +0200, Heikki Linnakangas wrote: > >> I don't see any other realistic way to fix this, however, so maybe we > >> should just bite the bullet and do it anyway. > > > We could remember the subtransactio

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Andres Freund
On 2014-01-02 14:44:34 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2014-01-02 12:46:34 -0500, Tom Lane wrote: > >> For real > >> forensics work, you need to be able to see all tuples, which makes me > >> think that something akin to pgstattuple is the right API; that is "return > >> a se

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Tom Lane
Robert Haas writes: > Well, that's fair enough. I don't mind having two functions. Should > the whole-table function also include invisible tuples? Certainly, that's exactly why I was proposing it. You can do a join if you want to suppress them. regards, tom lane --

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 2:56 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Jan 2, 2014 at 2:44 PM, Tom Lane wrote: >>> As I commented to Robert, the page-at-a-time behavior of pageinspect >>> is not an API detail we'd want to copy for this. I envision something >>> like >>> >>> select hdr

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Tom Lane
Andres Freund writes: > On 2014-01-02 21:21:15 +0200, Heikki Linnakangas wrote: >> I don't see any other realistic way to fix this, however, so maybe we >> should just bite the bullet and do it anyway. > We could remember the subtransaction a variable was created in and error > out if it the crea

[HACKERS] Streaming replication bug in 9.3.2, "WAL contains references to invalid pages"

2014-01-02 Thread Christophe Pettus
Greetings, We've had two clients experience a crash on the secondary of a streaming replication pair, running PostgreSQL 9.3.2. In both cases, the messages were close to this example: 2013-12-30 18:08:00.464 PST,,,23869,,52ab4839.5d3d,16,,2013-12-13 09:47:37 PST,1/0,0,WARNING,01000,"page 4578

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-02 Thread Peter Geoghegan
I decided to make at least a cursory attempt to measure or characterize the performance of each of our approaches to value locking. Being fair here is a non-trivial matter, because of the fact that upserts can behave quite differently based on the need to insert or update, lock contention and so on

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Tom Lane
Robert Haas writes: > On Thu, Jan 2, 2014 at 2:44 PM, Tom Lane wrote: >> As I commented to Robert, the page-at-a-time behavior of pageinspect >> is not an API detail we'd want to copy for this. I envision something >> like >> >> select hdr.*, foo.* >> from tuple_header_details('foo'::regclass)

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-02 Thread David Rowley
On Fri, Jan 3, 2014 at 5:33 AM, Erik Rijkers wrote: > *** /var/data1/pg_stuff/pg_sandbox/pgsql.inverse/src/ > test/regress/expected/window.out 2014-01-02 16:19:48.0 +0100 > --- > /var/data1/pg_stuff/pg_sandbox/pgsql.inverse/src/test/regress/results/window.out >2014-01-02 16:21:43.00

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Tom Lane
Robert Haas writes: > We could certainly add a function that returns SETOF record, taking > e.g. regclass as an argument, but it doesn't seem a stretch to me to > think that you might want to get tuple header information for some but > not all tuples in the relation, and I don't see any real good

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 2:44 PM, Tom Lane wrote: > As I commented to Robert, the page-at-a-time behavior of pageinspect > is not an API detail we'd want to copy for this. I envision something > like > >select hdr.*, foo.* > from tuple_header_details('foo'::regclass) as hdr >

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Tom Lane
Andres Freund writes: > On 2014-01-02 12:46:34 -0500, Tom Lane wrote: >> For real >> forensics work, you need to be able to see all tuples, which makes me >> think that something akin to pgstattuple is the right API; that is "return >> a set of the header info for all tuples on such-and-such pages

Re: [HACKERS] proposal: multiple read-write masters in a cluster with wal-streaming synchronization

2014-01-02 Thread Andres Freund
On 2014-01-02 11:35:57 -0800, Mark Dilger wrote: > BTW, since the space shuttle has already left orbit, as you > metaphorically put it, maybe there should be more > visibility to the wider world about this?  You can go to > postgresql.org and find diddly squat about it.  I grant you > that it is no

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Andres Freund
On 2014-01-02 21:21:15 +0200, Heikki Linnakangas wrote: > I don't see any other realistic way to fix this, however, so maybe we > should just bite the bullet and do it anyway. We could remember the subtransaction a variable was created in and error out if it the creating subtransaction aborted and

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 2:03 PM, Tom Lane wrote: >> I both agree and disagree with this. I think that pgstattuple is >> useful, and I further agree that adding a stat to it about how much of >> the heap is frozen would be worthwhile. However, an aggregate number >> isn't always what you want, and

Re: [HACKERS] proposal: multiple read-write masters in a cluster with wal-streaming synchronization

2014-01-02 Thread Mark Dilger
Thanks to both of you for all the feedback.  Your reasoning about why it is not worth implementing, what the problems with it would be, etc., are helpful. Sorry about using the word multimaster where it might have been better to say sharded. BTW, since the space shuttle has already left orbit, a

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Tom Lane
Heikki Linnakangas writes: > The simplest fix would be to just detoast everything on assignment but > that was rejected on performance grounds in that previous thread. I > don't see any other realistic way to fix this, however, so maybe we > should just bite the bullet and do it anyway. Or jus

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Andres Freund
On 2014-01-02 12:46:34 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2014-01-02 09:40:54 -0500, Tom Lane wrote: > >> Actually, I thought the function approach was a good proposal. You are > >> right that func(tab.*) isn't going to work, because it's going to get a > >> Datum-ified tuple n

Re: [HACKERS] proposal: multiple read-write masters in a cluster with wal-streaming synchronization

2014-01-02 Thread Merlin Moncure
On Tue, Dec 31, 2013 at 3:51 PM, Mark Dilger wrote: > The BDR documentation > http://wiki.postgresql.org/images/7/75/BDR_Presentation_PGCon2012.pdf > says, > > "Physical replication forces us to use just one > node: multi-master required for write scalability" > > "Physical replicatio

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Heikki Linnakangas
On 01/02/2014 02:24 PM, Rushabh Lathia wrote: Hi All, Test case: drop table if exists t; create table t(c text); insert into t values ('x'), (repeat(md5('abcdefghijklmnop'), 1)); select pg_column_size(c), pg_column_size(c || '') FROM t; CREATE OR REPLACE FUNCTION copy_toast_out() RETURNS V

Re: [HACKERS] proposal: multiple read-write masters in a cluster with wal-streaming synchronization

2014-01-02 Thread Andres Freund
On 2014-01-02 10:18:52 -0800, Mark Dilger wrote: > I anticipated that my proposal would require partitioning the catalogs. > For instance, autovacuum could only run on locally owned tables, and > would need to store the analyze stats data in a catalog partition belonging > to the local server, but

Re: [HACKERS] more psprintf() use

2014-01-02 Thread Alvaro Herrera
Peter Eisentraut wrote: > psprintf() in place of hardcoded palloc(N) + sprintf() and the like. > > + values[j++] = psprintf("%d", stat.blkno); > + values[j++] = psprintf("%c", stat.type); > + values[j++] = psprintf("%d", stat.live_items); > + values[j++] = psprintf("%d", stat.dea

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-02 Thread Tom Lane
"Erik Rijkers" writes: > The TRACE_POSTGRESQL_SORT_DONE warnings were not from your patch; sorry about > that. They occur on HEAD too (with a debug > compile). > tuplesort.c:935:44: warning: comparison between pointer and integer [enabled > by default] > TRACE_POSTGRESQL_SORT_DONE(state->tape

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Tom Lane
Robert Haas writes: > On Thu, Jan 2, 2014 at 12:46 PM, Tom Lane wrote: >> In any case, upon further reflection I'm not convinced that doing this >> with a SELECT-based query is the right thing, no matter whether the query >> looks at a function or a system column; because by definition, you'll on

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 12:46 PM, Tom Lane wrote: > Andres Freund writes: >> On 2014-01-02 09:40:54 -0500, Tom Lane wrote: >>> Actually, I thought the function approach was a good proposal. You are >>> right that func(tab.*) isn't going to work, because it's going to get a >>> Datum-ified tuple n

Re: [HACKERS] RFC: Async query processing

2014-01-02 Thread Claudio Freire
On Wed, Dec 18, 2013 at 1:50 PM, Florian Weimer wrote: > On 11/04/2013 02:51 AM, Claudio Freire wrote: >> >> On Sun, Nov 3, 2013 at 3:58 PM, Florian Weimer wrote: >>> >>> I would like to add truly asynchronous query processing to libpq, >>> enabling >>> command pipelining. The idea is to to allo

Re: [HACKERS] proposal: multiple read-write masters in a cluster with wal-streaming synchronization

2014-01-02 Thread Mark Dilger
My original email was mostly a question about whether WAL data could be merged from multiple servers, or whether I was overlooking some unsolvable difficulty.  I'm still mostly curious about that question. I anticipated that my proposal would require partitioning the catalogs. For instance, autov

Re: [HACKERS] fix_PGSTAT_NUM_TABENTRIES_macro patch

2014-01-02 Thread Tom Lane
Robert Haas writes: > On Tue, Dec 31, 2013 at 12:20 PM, Tom Lane wrote: >> I'm inclined to think that we should look for a less breakable way to >> manage the message size limit; if Robert missed this issue in that patch, >> other people are going to miss it in future patches. The existing codin

Re: [HACKERS] fix_PGSTAT_NUM_TABENTRIES_macro patch

2014-01-02 Thread Robert Haas
On Tue, Dec 31, 2013 at 12:20 PM, Tom Lane wrote: > Mark Dilger writes: >> In src/include/pgstat.h, the PGSTAT_NUM_TABENTRIES macro >> attempts to subtract off the size of the PgStat_MsgTabstat >> struct up to the m_entry[] field. This macro was correct up >> until the fields m_block_read_time a

Re: [HACKERS] [PATCH] Remove some duplicate if conditions

2014-01-02 Thread Tom Lane
David Rowley writes: > I've attached a simple patch which removes some duplicate if conditions > that seemed to have found their way into the code. > These are per PVS-Studio's warnings. -1. If PVS-Studio is complaining about this type of coding, to hell with it; it should just optimize away th

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Tom Lane
Andres Freund writes: > On 2014-01-02 09:40:54 -0500, Tom Lane wrote: >> Actually, I thought the function approach was a good proposal. You are >> right that func(tab.*) isn't going to work, because it's going to get a >> Datum-ified tuple not a pointer to raw on-disk storage. But an inspection

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 11:08 AM, Heikki Linnakangas wrote: > On 01/02/2014 02:53 PM, Robert Haas wrote: >> On Tue, Dec 31, 2013 at 4:12 AM, Peter Geoghegan wrote: >>> >>> On Tue, Dec 31, 2013 at 12:52 AM, Heikki Linnakangas >>> wrote: 1. PromiseTupleInsertionLockAcquire() 2. Inser

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-02 Thread Erik Rijkers
On Thu, January 2, 2014 13:36, Erik Rijkers wrote: > On Thu, January 2, 2014 13:05, David Rowley wrote: >> here's a slightly updated patch >> [inverse_transition_functions_v1.8.patch.gz ] > > patch applies, and compiles (although with new warnings). > But make check complains loudly: see attached.

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-02 Thread Heikki Linnakangas
On 01/02/2014 02:53 PM, Robert Haas wrote: On Tue, Dec 31, 2013 at 4:12 AM, Peter Geoghegan wrote: On Tue, Dec 31, 2013 at 12:52 AM, Heikki Linnakangas wrote: 1. PromiseTupleInsertionLockAcquire() 2. Insert heap tuple 3. Insert index tuples 4. Check if conflict happened. Kill the already-inse

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Andres Freund
On 2014-01-02 09:40:54 -0500, Tom Lane wrote: > Robert Haas writes: > > On Thu, Jan 2, 2014 at 12:26 AM, Greg Stark wrote: > >> 1) it's a bit weird to go to this effort to eliminate system columns by > >> using a scheme that depends on having a system column -- ctid > > > At any rate, my goal is

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Tom Lane
Robert Haas writes: > On Thu, Jan 2, 2014 at 12:26 AM, Greg Stark wrote: >> 1) it's a bit weird to go to this effort to eliminate system columns by >> using a scheme that depends on having a system column -- ctid > At any rate, my goal isn't really to get rid of system columns, but to > address

Re: [HACKERS] Fixing pg_basebackup with tablespaces found in $PGDATA

2014-01-02 Thread Bernd Helmle
--On 1. Januar 2014 23:53:46 +0100 Dimitri Fontaine wrote: Hi, As much as I've seen people frown upon $subject, it still happens in the wild, and Magnus seems to agree that the current failure mode of our pg_basebackup tool when confronted to the situation is a bug. So here's a fix, attac

Re: [HACKERS] more psprintf() use

2014-01-02 Thread Tom Lane
Andres Freund writes: > On 2014-01-02 09:49:48 +0200, Heikki Linnakangas wrote: >> Is it legal to return a constant with PG_RETURN_CSTRING? Grepping around, I >> don't see that being done anywhere else, but there are places that do >> PG_RETURN_CSTRING(pstrdup())... > I don't see why it wouldn't

Re: [HACKERS] Fixing pg_basebackup with tablespaces found in $PGDATA

2014-01-02 Thread Dimitri Fontaine
Magnus Hagander writes: > We can't get away with just comparing the relative part of the pathname. > Because it will fail if there is another path with exactly the same length, > containing the tablespace. Actually… yeah. > I think we might want to store a value in the tablespaceinfo struct > in

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 7:40 AM, Andres Freund wrote: > On 2014-01-02 07:35:11 -0500, Robert Haas wrote: >> On Thu, Jan 2, 2014 at 12:26 AM, Greg Stark wrote: >> >> I fail to see why. What is so ugly about this: >> > >> >> select x.* from pgbench_accounts a, pg_tuple_header(a.tableoid, a.ctid) x;

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-02 Thread Nicolas Barbier
2013/12/15 David Rowley : > I've been working on speeding up aggregate functions when used in the > context of a window's with non fixed frame heads. > 1. Fully implement negative transition functions for SUM and AVG. I would like to mention that this functionality is also extremely useful to ha

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-02 Thread Robert Haas
On Tue, Dec 31, 2013 at 4:12 AM, Peter Geoghegan wrote: > On Tue, Dec 31, 2013 at 12:52 AM, Heikki Linnakangas > wrote: >> 1. PromiseTupleInsertionLockAcquire() >> 2. Insert heap tuple >> 3. Insert index tuples >> 4. Check if conflict happened. Kill the already-inserted tuple on conflict. >> 5. P

Re: [HACKERS] Show lossy heap block info in EXPLAIN ANALYZE for bitmap heap scan

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 4:27 AM, Andres Freund wrote: > On 2014-01-01 21:15:46 -0500, Robert Haas wrote: >> [ sensible reasoning ] However, I'm not sure it's really worth it. >> I think what people really care about is knowing whether the bitmap >> lossified or not, and generally how much got lossi

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Andres Freund
On 2014-01-02 07:35:11 -0500, Robert Haas wrote: > On Thu, Jan 2, 2014 at 12:26 AM, Greg Stark wrote: > >> I fail to see why. What is so ugly about this: > > > >> select x.* from pgbench_accounts a, pg_tuple_header(a.tableoid, a.ctid) x; > > > > Two points: > > > > 1) it's a bit weird to go to th

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 12:26 AM, Greg Stark wrote: >> I fail to see why. What is so ugly about this: > >> select x.* from pgbench_accounts a, pg_tuple_header(a.tableoid, a.ctid) x; > > Two points: > > 1) it's a bit weird to go to this effort to eliminate system columns by > using a scheme that de

[HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Rushabh Lathia
Hi All, Test case: drop table if exists t; create table t(c text); insert into t values ('x'), (repeat(md5('abcdefghijklmnop'), 1)); select pg_column_size(c), pg_column_size(c || '') FROM t; CREATE OR REPLACE FUNCTION copy_toast_out() RETURNS VOID AS $$ declare v text; BEGIN

Re: [HACKERS] Fixing pg_basebackup with tablespaces found in $PGDATA

2014-01-02 Thread Magnus Hagander
On Wed, Jan 1, 2014 at 11:53 PM, Dimitri Fontaine wrote: > Hi, > > As much as I've seen people frown upon $subject, it still happens in the > wild, and Magnus seems to agree that the current failure mode of our > pg_basebackup tool when confronted to the situation is a bug. > > So here's a fix, at

[HACKERS] [PATCH] Remove some duplicate if conditions

2014-01-02 Thread David Rowley
I've attached a simple patch which removes some duplicate if conditions that seemed to have found their way into the code. These are per PVS-Studio's warnings. Regards David Rowley duplicate_if_test.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgres

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-01-02 Thread David Rowley
On Fri, Dec 27, 2013 at 1:36 AM, David Rowley wrote: > From what I can tell adding an inverse transition function to support AVG > for numeric does not affect the number of trailing zeros in the results, so > I've attached a patch which now has inverse transition functions to support > numeric ty

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-01-02 Thread Christian Kruse
Hi, On 02/01/14 10:02, Andres Freund wrote: > > Christian's idea of a context line seems plausible to me. I don't > > care for this implementation too much --- a global variable? Ick. > > Yea, the data should be stored in ErrorContextCallback.arg instead. Fixed. I also palloc() the ErrorConte

Re: [HACKERS] [PATCH] Store Extension Options

2014-01-02 Thread Andres Freund
On 2014-01-02 08:26:20 -0200, Fabrízio de Royes Mello wrote: > On Thu, Jan 2, 2014 at 7:19 AM, Andres Freund > wrote: > > > > On 2013-12-31 13:37:59 +0100, Pavel Stehule wrote: > > > > We use the namespace "ext" to the internal code > > > > (src/backend/access/common/reloptions.c) skip some valid

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-02 Thread Andres Freund
On 2014-01-02 02:20:02 -0800, Peter Geoghegan wrote: > On Thu, Jan 2, 2014 at 1:49 AM, Andres Freund wrote: > >> Well, you're not totally on your own for something like that with this > >> feature. You can project the conflicter's tid, and possibly do a more > >> sophisticated recovery, like inspe

Re: [HACKERS] [PATCH] Store Extension Options

2014-01-02 Thread Fabrízio de Royes Mello
On Thu, Jan 2, 2014 at 7:19 AM, Andres Freund wrote: > > On 2013-12-31 13:37:59 +0100, Pavel Stehule wrote: > > > We use the namespace "ext" to the internal code > > > (src/backend/access/common/reloptions.c) skip some validations and store > > > the custom GUC. > > > > > > Do you think we don't

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-02 Thread Peter Geoghegan
On Thu, Jan 2, 2014 at 1:49 AM, Andres Freund wrote: >> Well, you're not totally on your own for something like that with this >> feature. You can project the conflicter's tid, and possibly do a more >> sophisticated recovery, like inspecting the locked row and iterating. > > Yea, but in that case

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-01-02 Thread Andres Freund
On 2014-01-02 01:40:38 -0800, Peter Geoghegan wrote: > On Thu, Jan 2, 2014 at 12:56 AM, Andres Freund wrote: > > I agree that the message needs improvement, but I don't agree that we > > shouldn't lock the tuple's location. If you manually investigate the > > situation that's where you'll find the

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2014-01-02 Thread Andres Freund
On 2013-12-27 14:11:44 -0800, Peter Geoghegan wrote: > On Fri, Dec 27, 2013 at 12:57 AM, Andres Freund > wrote: > > I don't think the current syntax the feature implements can be used as > > the sole argument what the feature should be able to support. > > > > If you think from the angle of a asy

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-01-02 Thread Peter Geoghegan
On Thu, Jan 2, 2014 at 12:56 AM, Andres Freund wrote: > I agree that the message needs improvement, but I don't agree that we > shouldn't lock the tuple's location. If you manually investigate the > situation that's where you'll find the conflicting tuple - I don't see > what we gain by not loggin

Re: [HACKERS] Show lossy heap block info in EXPLAIN ANALYZE for bitmap heap scan

2014-01-02 Thread Andres Freund
On 2014-01-01 21:15:46 -0500, Robert Haas wrote: > [ sensible reasoning ] However, I'm not sure it's really worth it. > I think what people really care about is knowing whether the bitmap > lossified or not, and generally how much got lossified. The counts of > exact and lossy pages are sufficient

Re: [HACKERS] [PATCH] Store Extension Options

2014-01-02 Thread Andres Freund
On 2013-12-31 13:37:59 +0100, Pavel Stehule wrote: > > We use the namespace "ext" to the internal code > > (src/backend/access/common/reloptions.c) skip some validations and store > > the custom GUC. > > > > Do you think we don't need to use the "ext" namespace? > > > > yes - there be same mechan

Re: [HACKERS] proposal: multiple read-write masters in a cluster with wal-streaming synchronization

2014-01-02 Thread Andres Freund
On 2013-12-31 13:51:08 -0800, Mark Dilger wrote: > The BDR documentation > http://wiki.postgresql.org/images/7/75/BDR_Presentation_PGCon2012.pdf > says, > >     "Physical replication forces us to use just one > node: multi-master required for write scalability" > >     "Physical replication

Re: [HACKERS] What exactly is drop-index-concurrently-1.spec trying to test?

2014-01-02 Thread Andres Freund
On 2013-12-31 17:14:11 -0500, Tom Lane wrote: > Peter pointed out in > http://www.postgresql.org/message-id/527c0fe9.7000...@gmx.net > that Kyotaro-san's patch to treat unique indexes as satisfying any sort > condition that they are a prefix of broke the drop-index-concurrently-1 > isolation test.

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-01-02 Thread Andres Freund
On 2013-12-31 11:36:36 -0500, Tom Lane wrote: > Simon Riggs writes: > > On 31 December 2013 09:12, Christian Kruse > > wrote: > >> Output with patch: > >> > >> LOG: process 24774 acquired ShareLock on transaction 696 after 11688.720 > >> ms > >> CONTEXT: relation name: foo (OID 16385) > >> t

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-01-02 Thread Andres Freund
On 2013-12-31 13:56:53 -0800, Peter Geoghegan wrote: > ISTM that you should be printing just the value and the unique index > there, and not any information about the tuple proper. Doing any less > could be highly confusing. I agree that the message needs improvement, but I don't agree that we sho

Re: [HACKERS] more psprintf() use

2014-01-02 Thread Andres Freund
On 2014-01-02 09:49:48 +0200, Heikki Linnakangas wrote: > On 01/02/2014 05:14 AM, Peter Eisentraut wrote: > >diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c > >index 772a5ca..8331a56 100644 > >--- a/contrib/hstore/hstore_io.c > >+++ b/contrib/hstore/hstore_io.c > >@@ -1114,11 +

Re: [HACKERS] preserving forensic information when we freeze

2014-01-02 Thread Andres Freund
Hi, On 2014-01-02 05:26:26 +, Greg Stark wrote: > 2) refetching a row could conceivably end up retrieving different data than > was present when the row was originally read. (In some cases that might > actually be the intended behaviour) That's possible with system columns as well. In the nor