[HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

On Wed, Jun 03, 2015 at 04:53:46PM -0400, Robert Haas wrote: > So here's a patch taking a different approach. In this approach, if > the multixact whose members we want to look up doesn't exist, we don't > use a later one (that might or might not be valid). Instead, we > attempt to cope with the

Re: [HACKERS] Further issues with jsonb semantics, documentation

On Wed, Jun 3, 2015 at 7:02 PM, Peter Geoghegan wrote: > Consider this case: > > postgres=# select '{"c":5, "a":6, "b":7}'::jsonb - 1; > ?column? > -- > {"a": 6, "c": 5} > (1 row) > > Clearly anyone expecting the value "a" to be removed here would be in > for a surprise. More

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

On Thu, Jun 4, 2015 at 1:52 AM, Andrew Dunstan wrote: > > On 06/02/2015 11:55 PM, Amit Kapila wrote: > > On Tue, Jun 2, 2015 at 10:26 PM, Andrew Dunstan > > wrote: >> >> Well, it seems to me the new function is being altogether way too >> trusting about the na

[HACKERS] Further issues with jsonb semantics, documentation

I've noticed some more issues with the jsonb documentation, and the new jsonb stuff generally. I didn't set out to give Andrew feedback on the semantics weeks after feature freeze, but unfortunately this feels like another discussion that we need to have now rather than later. "operator jsonb - in

Re: [HACKERS] [PATCH] Add error handling to byteaout.

On Thu, Jun 4, 2015 at 1:32 AM, Alvaro Herrera wrote: > Andreas Seltenreich wrote: > > Tom Lane writes: > > > > > Andreas Seltenreich writes: > > >> The scary one is due to an integer overflow the attached patch also > > >> fixes. > > > > > > s/int/Size/ doesn't fix anything on 32-bit machines.

[HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

On Mon, Jun 1, 2015 at 4:55 PM, Noah Misch wrote: > While testing this (with inconsistent-multixact-fix-master.patch applied, > FWIW), I noticed a nearby bug with a similar symptom. TruncateMultiXact() > omits the nextMXact==oldestMXact special case found in each other > find_multixact_start() ca

Re: [HACKERS] pg_xlog -> pg_xjournal?

On 6/2/15 4:58 PM, David Steele wrote: On 5/31/15 1:46 PM, Tom Lane wrote: Hm. I think the impact on third-party backup tools would be rather bad, but there's a simple modification of the idea that might fix that: just always create pg_xlog as a symlink to pg_xjournal during initdb. Anybody who

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

Robert Haas wrote: > So here's a patch taking a different approach. I tried to apply this to 9.3 but it's messy because of pgindent. Anyone would have a problem with me backpatching a pgindent run of multixact.c? Also, you have a new function SlruPageExists, but we already have SimpleLruDoesPhy

Re: [HACKERS] xid wrap / optimize frozen tables?

On May 24, 2015 6:42 AM, "Nils Goroll" wrote: > > Hi Jeff and all, > > On 23/05/15 22:13, Jeff Janes wrote: > > Are you sure it is the read IO that causes the problem? > > Yes. Trouble is here that we are talking about a 361 GB table > >List of relations > Sche

Re: [HACKERS] [PATCH] Document that directly callable functions may use fn_extra

On 5/29/15 10:21 AM, Peter Eisentraut wrote: On 5/28/15 10:15 PM, Craig Ringer wrote: I was a puzzled by src/backend/utils/fmgr/README and fmgr.h's descriptions of fcinfo->flinfo->fn_extra (FmgrInfo.fn_extra) as they seem to conflict with actual usage. The docs suggest that fl_extra is for the

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

On Wed, Jun 3, 2015 at 8:24 AM, Robert Haas wrote: > On Tue, Jun 2, 2015 at 5:22 PM, Andres Freund wrote: >>> > Hm. If GetOldestMultiXactOnDisk() gets the starting point by scanning >>> > the disk it'll always get one at a segment boundary, right? I'm not sure >>> > that's actually ok; because th

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

On 06/02/2015 11:55 PM, Amit Kapila wrote: On Tue, Jun 2, 2015 at 10:26 PM, Andrew Dunstan > wrote: On 05/15/2015 02:21 AM, Amit Kapila wrote: Find the patch which gets rid of rmtree usage. I have made it as a separate function because the sam

Re: [HACKERS] [PATCH] Add error handling to byteaout.

On June 3, 2015 9:42:21 PM GMT+02:00, Andreas Seltenreich wrote: >Piotr Stefaniak writes: s/int/Size/ doesn't fix anything on 32-bit machines. >> >> Postgres requires twos-complement representation, so that the >> assumption that signed integer types wrap around on overflow can be >> safely

Re: [HACKERS] [PATCH] Add error handling to byteaout.

Piotr Stefaniak writes: >>> s/int/Size/ doesn't fix anything on 32-bit machines. > > Postgres requires twos-complement representation, so that the > assumption that signed integer types wrap around on overflow can be > safely made. Thanks for the clarification! -- Sent via pgsql-hackers mailing

Re: [HACKERS] [PATCH] Add error handling to byteaout.

Alvaro Herrera writes: > Why not just use an unsigned 64 bit variable? Also, perhaps > palloc_huge() avoids the whole problem in the first place ... Ja, that crossed my mind too, but the current limit is already far beyond anything that is usually configured for per-backend memory use, so I dism

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

Andres Freund wrote: > On 2015-06-03 15:01:46 -0300, Alvaro Herrera wrote: > > One idea I had was: what if the oldestMulti pointed to another multi > > earlier in the same 0046 file, so that it is read-as-zeroes (and the > > file is created), and then a subsequent multixact truncate tries to read

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

On 2015-06-03 15:01:46 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > That's not necessarily the case though, given how the code currently > > works. In a bunch of places the SLRUs are accessed *before* having been > > made consistent by WAL replay. Especially if several checkpoints/vacuum

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

Alvaro Herrera wrote: > Really, the whole question of how this code goes past the open() failure > in SlruPhysicalReadPage baffles me. I don't see any possible way for > the file to be created ... Hmm, the checkpointer can call TruncateMultiXact when in recovery, on restartpoints. I wonder if in

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

Andres Freund wrote: > On 2015-06-03 00:42:55 -0300, Alvaro Herrera wrote: > > Thomas Munro wrote: > > > On Tue, Jun 2, 2015 at 9:30 AM, Alvaro Herrera > > > wrote: > > > > My guess is that the file existed, and perhaps had one or more pages, > > > > but the wanted page doesn't exist, so we tried

Re: [CORE] [HACKERS] postpone next week's release

On 05/31/2015 03:51 AM, David Steele wrote: > On 5/30/15 8:38 PM, Joshua D. Drake wrote: >> >> On 05/30/2015 03:48 PM, David Steele wrote: >>> On 5/30/15 2:10 PM, Robert Haas wrote: What, in this release, could break things badly? RLS? Grouping sets? Heikki's WAL format changes? That la

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

On 2015-06-03 00:42:55 -0300, Alvaro Herrera wrote: > Thomas Munro wrote: > > On Tue, Jun 2, 2015 at 9:30 AM, Alvaro Herrera > > wrote: > > > My guess is that the file existed, and perhaps had one or more pages, > > > but the wanted page doesn't exist, so we tried to read but got 0 bytes > > > ba

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

Thomas Munro wrote: > I have finally reproduced that error! See attached repro shell script. > > The conditions are: > > 1. next multixact == oldest multixact (no active multixacts, pointing > past the end) > 2. next multixact would be the first item on a new page (multixact % 2048 == > 0) >

Re: [HACKERS] [CORE] Restore-reliability mode

On 2015-06-03 10:21:28 -0700, Josh Berkus wrote: > So, historically, this is what the period between feature freeze and > beta1 was for; the "consolidation" phase was supposed to deal with this. > The problem over the last few years, by my observation, has been that > consolidation has been left t

Re: [HACKERS] [CORE] Restore-reliability mode

On 06/03/2015 06:50 AM, Noah Misch wrote: > Subject changed from "Re: [CORE] postpone next week's release". > > On Sat, May 30, 2015 at 10:48:45PM -0400, Bruce Momjian wrote: >> If we have to totally stop feature development until we are all happy >> with the code we have, so be it. If people fee

Re: [HACKERS] anole: assorted stability problems

Tom Lane wrote: > Andres Freund writes: > > Uh. I'm pretty sure there were some back when that patch went in. And > > there definitely used to be a couple earlier. I guess itanium really is > > dying (mixed bad: It's a horrible architecture, but more coverage would > > still be good). > > Since

Re: [HACKERS] [PATCH] Add error handling to byteaout.

Andreas Seltenreich wrote: > Tom Lane writes: > > > Andreas Seltenreich writes: > >> The scary one is due to an integer overflow the attached patch also > >> fixes. > > > > s/int/Size/ doesn't fix anything on 32-bit machines. > > Well, it changes the signedness of the computation on 32-bit, and

Re: [HACKERS] Restore-reliability mode

On 06/03/2015 07:18 AM, Andres Freund wrote: On 2015-06-03 09:50:49 -0400, Noah Misch wrote: Second, I would define the subject matter as "bug fixes, testing and review", not "restructuring, testing and review." Different code structures are clearest to different hackers. Restructuring, on av

Re: [HACKERS] Restore-reliability mode

On 2015-06-03 09:50:49 -0400, Noah Misch wrote: > Second, I would define the subject matter as "bug fixes, testing and > review", not "restructuring, testing and review." Different code > structures are clearest to different hackers. Restructuring, on > average, adds bugs even more quickly than f

Re: [HACKERS] Restore-reliability mode

On 3 June 2015 at 14:50, Noah Misch wrote: > I > ​ ​ > would define the subject matter as "bug fixes, testing and review", not > "restructuring, testing and review." Different code structures are > clearest > to different hackers. Restructuring, on average, adds bugs even more > quickly > than

[HACKERS] Restore-reliability mode

Subject changed from "Re: [CORE] postpone next week's release". On Sat, May 30, 2015 at 10:48:45PM -0400, Bruce Momjian wrote: > Well, I think we stop what we are doing, focus on restructuring, > testing, and reviewing areas that historically have had problems, and > when we are done, we can look

Re: [HACKERS] auto_explain sample rate

On 3 June 2015 at 20:04, Andres Freund wrote: > On 2015-06-03 18:54:24 +0800, Craig Ringer wrote: > > OK, here we go. > > Hm. Wouldn't random sampling be better than what you do? If your queries > have a pattern to them (e.g. you always issue the same 10 queries in > succession), this will possib

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

On Tue, Jun 2, 2015 at 5:22 PM, Andres Freund wrote: >> > Hm. If GetOldestMultiXactOnDisk() gets the starting point by scanning >> > the disk it'll always get one at a segment boundary, right? I'm not sure >> > that's actually ok; because the value at the beginning of the segment >> > can very wel

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

On Wed, Jun 3, 2015 at 4:48 AM, Thomas Munro wrote: > On Wed, Jun 3, 2015 at 3:42 PM, Alvaro Herrera > wrote: >> Thomas Munro wrote: >>> On Tue, Jun 2, 2015 at 9:30 AM, Alvaro Herrera >>> wrote: >>> > My guess is that the file existed, and perhaps had one or more pages, >>> > but the wanted pa

Re: [HACKERS] auto_explain sample rate

On 2015-06-03 18:54:24 +0800, Craig Ringer wrote: > OK, here we go. Hm. Wouldn't random sampling be better than what you do? If your queries have a pattern to them (e.g. you always issue the same 10 queries in succession), this will possibly only show a subset of the queries. I think a formulatio

Re: [HACKERS] why does txid_current() assign new transaction-id?

On Wed, Jun 3, 2015 at 12:13 PM, Fujii Masao wrote: > On Wed, Jun 3, 2015 at 9:04 AM, Naoya Anzai > wrote: > >> > diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml > >> > index 89a609f..6485b42 100644 > >> > --- a/doc/src/sgml/func.sgml > >> > +++ b/doc/src/sgml/func.sgml > >> > @@ -1

Re: [HACKERS] auto_explain sample rate

On 2 June 2015 at 15:07, Craig Ringer wrote: > On 29 May 2015 at 11:35, Tom Lane wrote: > >> Craig Ringer writes: >> > It's sometimes desirable to collect auto_explain data with ANALYZE in >> order >> > to track down hard-to-reproduce issues, but the performance impacts can >> be >> > pretty he

Re: [HACKERS] Re: [GENERAL] 9.4.1 -> 9.4.2 problem: could not access status of transaction 1

On Wed, Jun 3, 2015 at 3:42 PM, Alvaro Herrera wrote: > Thomas Munro wrote: >> On Tue, Jun 2, 2015 at 9:30 AM, Alvaro Herrera >> wrote: >> > My guess is that the file existed, and perhaps had one or more pages, >> > but the wanted page doesn't exist, so we tried to read but got 0 bytes >> > back

Re: [HACKERS] [GENERAL] psql weird behaviour with charset encodings

On Tue, Jun 2, 2015 at 4:19 PM, Michael Paquier wrote: > On Sun, May 24, 2015 at 2:43 AM, Noah Misch wrote: > > It would be good to purge the code of precisions on "s" conversion > specifiers, > > then Assert(!pointflag) in fmtstr() to catch new introductions. I won't > plan > > to do it mysel

Re: [HACKERS] auto_explain sample rate

> > >> lot of variants - I would to see cost and times for EXPLAIN ANALYZE every > time - but the precision of time can be reduced to 1ms. It is question if > we can significantly reduce the cost (or number of calls) of getting system > time. > > Pavel > > > OK, so you're suggesting a sampling-base

Re: [HACKERS] auto_explain sample rate

2015-06-03 9:46 GMT+02:00 Craig Ringer : > > On 3 June 2015 at 15:22, Pavel Stehule wrote: > >> >> >> 2015-06-03 9:17 GMT+02:00 Craig Ringer : >> >>> >>> >>> On 2 June 2015 at 15:11, Pavel Stehule wrote: >>> 2015-06-02 9:07 GMT+02:00 Craig Ringer : > > For the majorit

Re: [HACKERS] auto_explain sample rate

On 3 June 2015 at 15:22, Pavel Stehule wrote: > > > 2015-06-03 9:17 GMT+02:00 Craig Ringer : > >> >> >> On 2 June 2015 at 15:11, Pavel Stehule wrote: >> >>> >>> >>> 2015-06-02 9:07 GMT+02:00 Craig Ringer : >>> For the majority of users I'm sure it's sufficient to just have a sampl

Re: [HACKERS] auto_explain sample rate

2015-06-03 9:17 GMT+02:00 Craig Ringer : > > > On 2 June 2015 at 15:11, Pavel Stehule wrote: > >> >> >> 2015-06-02 9:07 GMT+02:00 Craig Ringer : >> >>> >>> For the majority of users I'm sure it's sufficient to just have a sample >>> rate. >>> >>> Anything that's trying to match individual queries

Re: [HACKERS] auto_explain sample rate

On 2 June 2015 at 15:11, Pavel Stehule wrote: > > > 2015-06-02 9:07 GMT+02:00 Craig Ringer : > >> >> For the majority of users I'm sure it's sufficient to just have a sample >> rate. >> >> Anything that's trying to match individual queries could be interested in >> all sorts of different things.