Re: [HACKERS] WAL consistency check facility

On Sat, Sep 10, 2016 at 8:33 PM, Robert Haas wrote: > On Sat, Sep 10, 2016 at 3:19 AM, Michael Paquier > wrote: >> On Fri, Sep 9, 2016 at 4:01 PM, Kuntal Ghosh >> wrote: > - If WAL consistency check is enabled

Re: [HACKERS] WAL consistency check facility

On Sat, Sep 10, 2016 at 12:49 PM, Michael Paquier wrote: > On Fri, Sep 9, 2016 at 4:01 PM, Kuntal Ghosh > wrote: > - In recovery tests (src/test/recovery/t), I've added wal_consistency parameter in the existing scripts. This

Re: [HACKERS] pg_sequence catalog

On Sun, Sep 11, 2016 at 12:39 AM, Andres Freund wrote: > On 2016-09-10 17:23:21 +0530, Amit Kapila wrote: >> > >> >> I may be missing something here, but why would it contend on a lock, >> as per locking scheme proposed by Alvaro, access to sequence object >> will need a share

Re: [HACKERS] Write Ahead Logging for Hash Indexes

On Sun, Sep 11, 2016 at 4:10 AM, Mark Kirkwood wrote: > > > performed several 10 hour runs on size 100 database using 32 and 64 clients. > For the last run I rebuilt with assertions enabled. No hangs or assertion > failures. > Thanks for verification. Do you think

Re: [HACKERS] sequence data type

On 9/10/16, Jim Nasby wrote: > On 9/3/16 6:01 PM, Gavin Flower wrote: >> I am curious as to the use cases for other possibilities. > > A hex or base64 type might be interesting, should those ever get created... > -- > Jim Nasby, Data Architect, Blue Treble Consulting,

Re: [HACKERS] Override compile time log levels of specific messages/modules

On 9/6/16 5:18 AM, Craig Ringer wrote: I think something automatic that we clearly define as unstable and not to be relied upon would be preferable. Plus we already have much of the infrastructure in elog.c as used by errcontext etc. Actually, I wish this was a straight-up logging level

Re: [HACKERS] sequence data type

On 9/3/16 6:01 PM, Gavin Flower wrote: I am curious as to the use cases for other possibilities. A hex or base64 type might be interesting, should those ever get created... -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL

Re: [HACKERS] [PATCH] Generic type subscription

On 9/9/16 6:29 AM, Dmitry Dolgov wrote: Regarding to the previous conversations [1], [2], here is a patch (with some improvements from Nikita Glukhov) for the generic type subscription. Awesome! Please make sure to add it to the Commit Fest app. -- Jim Nasby, Data Architect, Blue Treble

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

On 8/1/16 11:38 AM, Bruce Momjian wrote: I am hoping for a "novice" mode that issues warnings about possible bugs, e.g. unintentionally-correlated subselect, and this could be part of that. Somewhat related; I've recently been wondering about a mode that disallows Const's in queries coming

Re: [HACKERS] [PATCH] Alter or rename enum value

On 9/8/16 4:55 AM, Emre Hasegeli wrote: The main problem that has been discussed before was the indexes. One way is to tackle with it is to reindex all the tables after the operation. Currently we are doing it when the datatype of indexed columns change. So it should be possible, but very

[HACKERS] Merge Join with an Index Optimization

Hi, As I understand it, a merge join will currently read all tuples from both subqueries (besides early termination). I believe it should be possible to take advantages of the indexes on one or both of the tables being read from to skip a large number of tuples that would currently be read. As an

Re: [HACKERS] feature request: explain "with details" option

On 9/8/16 11:35 PM, Tom Lane wrote: This isn't simple because there are often *lots* of variants. You > don't just want to see the "top 10" candidate plans, because they're > probably a bunch of small variants on the same plan; the ones you'll > be interested in will probably be very different

Re: [HACKERS] Let file_fdw access COPY FROM PROGRAM

V2 of this patch: Changes: * rebased to most recent master * removed non-tap test that assumed the existence of Unix sed program * added non-tap test that assumes the existence of perl * switched from filename/program to filename/is_program to more closely follow patterns in copy.c * slight

Re: [HACKERS] Write Ahead Logging for Hash Indexes

On 09/09/16 14:50, Mark Kirkwood wrote: Yeah, good suggestion about replacing (essentially) all the indexes with hash ones and testing. I did some short runs with this type of schema yesterday (actually to get a feel for if hash performance vs btree was compareable - does seem tp be) - but

Re: [HACKERS] [GENERAL] C++ port of Postgres

Christian Convey writes: >If that's correct, then it sounds like the only way Joy's commit has >a chance of acceptance is if Peter's commit is rejected. >Because Peter's commit might be merged as part of the 2016-09 >commitfest, but Joy's can show up

Re: [HACKERS] [GENERAL] C++ port of Postgres

Hi Heikki, Could I ask you a newbie-reviewer question about something I'm seeing here? https://commitfest.postgresql.org/10/776/ >From some reading I've done (e.g., Stephen Frost's PGCon 2011 slides), I got the impression that a successful patch would always have this sequence of states in

Re: [HACKERS] High-CPU consumption on information_schema (only) query

> > > Without having at least compared EXPLAIN outputs from the two boxes, you > have no business jumping to that conclusion. > > If EXPLAIN does show different plans, my first instinct would be to wonder > whether the pg_stats data is equally up-to-date on both boxes. > >

Re: [HACKERS] [GENERAL] C++ port of Postgres

> Thanks. It sounds like worst-case scenario, I perform an unneeded > review. I'll give it a shot. Hi guys, Apologies for more boring process-related questions, but any pointers would be greatly appreciated... I'm a bit confused about how PG's code-review process is meant to handle this C++

Re: [HACKERS] High-CPU consumption on information_schema (only) query

Robins Tharakan writes: > I completely agree. With 'irrelevant' I was only trying to imply that > irrespective of the complexity of the query, a replicated box was seeing > similar slowness whereas a Restored DB wasn't. It felt that the SQL itself > isn't to blame here...

Re: [HACKERS] High-CPU consumption on information_schema (only) query

On Fri, 9 Sep 2016 at 09:39 Andres Freund wrote: > On 2016-09-07 23:37:31 +, Robins Tharakan wrote: > > If someone asks for I could provide SQL + EXPLAIN, but it feels > irrelevant > > here. > > Why is that? information_schema are normal sql queries, and some of them >

Re: [HACKERS] Re: [COMMITTERS] pgsql: Use LEFT JOINs in some system views in case referenced row doesn

I wrote: > I looked into this a little. There are at least three things we could > do here: > 1. Decide that showing walsenders is a good thing. I'm not really > sure why it isn't -- for example, we could take the trouble to display > the current replication command as the walsender's activity.

Re: [HACKERS] feature request: explain "with details" option

On Thu, Sep 8, 2016 at 10:40 AM, Roger Pack wrote: > My apologies if this was already requested before... > > I think it would be fantastic if postgres had an "explain the explain" > option: > Today's explain tells us what loops and scans were used, and relative > costs,

Re: [HACKERS] Tuplesort merge pre-reading

On Sat, Sep 10, 2016 at 12:04 AM, Heikki Linnakangas wrote: >> Did I misunderstand something? I'm applying the first patch of Peter's >> series (cap number of tapes), then your first one (remove prefetch) >> and second one (use larger read buffers). > > > Yes. I have been testing

Re: [HACKERS] pg_sequence catalog

On 2016-09-10 17:23:21 +0530, Amit Kapila wrote: > On Thu, Sep 1, 2016 at 12:00 AM, Andres Freund wrote: > > On 2016-08-31 14:23:41 -0400, Tom Lane wrote: > >> Andres Freund writes: > >> > On 2016-08-31 13:59:48 -0400, Tom Lane wrote: > >> >> You are

Re: [HACKERS] GiST penalty functions [PoC]

>Personally I wouldn't be very happy about an IEEE754 assumption. Ok, so let's avoid autoconf man. There is no real explanations of the ground for this assumption, just a reference to paper of David Goldberg (and there is no metion about IEEE754 is absoulte everywhere). BTW, may be we can ask

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Fri, Sep 9, 2016 at 5:22 PM, Claudio Freire wrote: > Since it is true that doing so would make it impossible to keep the > asserts about tupindex in tuplesort_heap_root_displace, I guess it > depends on how useful those asserts are (ie: how likely it is that > those

Re: [HACKERS] Re: [COMMITTERS] pgsql: Use LEFT JOINs in some system views in case referenced row doesn

I wrote: > Fujii Masao writes: >> On Sat, Aug 20, 2016 at 6:13 AM, Tom Lane wrote: >>> Use LEFT JOINs in some system views in case referenced row doesn't exist. >> This change causes pg_stat_activity to report the "bogus" information about >>

Re: [HACKERS] Useless dependency assumption libxml2 -> libxslt in MSVC scripts

Michael Paquier writes: > Thanks. I was a bit too lazy to look at the history to get this piece > of history out... And so the code that is presently in the MSVC > scripts should have been updated at the same time as those > compilations have been relaxed, but it got

Re: [HACKERS] Useless dependency assumption libxml2 -> libxslt in MSVC scripts

Michael Paquier writes: > Thanks. I was a bit too lazy to look at the history to get this piece > of history out... And so the code that is presently in the MSVC > scripts should have been updated at the same time as those > compilations have been relaxed, but it got

Re: [HACKERS] COPY command with RLS bug

> Looking for and improving test coverage for RLS is a good suggestion, > but let's not link the fate of the issue reported here with this > requirement. I have spent some time looking at this patch and this > looks in rather good shape to me (you even remembered to use the > prefix regress_* for

Re: [HACKERS] Allow to_date() and to_timestamp() to accept localized month names

Mattia writes: > attached is a patch which adds support to localized month names in > to_date() and to_timestamp() functions. Seems like a fine goal. > I thought about reusing from_char_seq_search() but localized month > names use different capitalization according to the

Re: [HACKERS] WAL consistency check facility

On Sat, Sep 10, 2016 at 3:19 AM, Michael Paquier wrote: > On Fri, Sep 9, 2016 at 4:01 PM, Kuntal Ghosh > wrote: - If WAL consistency check is enabled for a rmgrID, we always include the backup image in the WAL record. >>> >>> What

Re: [HACKERS] An extra error for client disconnection on Windows

Michael Paquier writes: > On Fri, Sep 9, 2016 at 11:39 PM, Tom Lane wrote: >> So this change would deal nicely with the "peer application on the remote >> host is suddenly stopped" case, at the price of being not nice about any >> of the other

Re: [HACKERS] pg_sequence catalog

On Thu, Sep 1, 2016 at 12:00 AM, Andres Freund wrote: > On 2016-08-31 14:23:41 -0400, Tom Lane wrote: >> Andres Freund writes: >> > On 2016-08-31 13:59:48 -0400, Tom Lane wrote: >> >> You are ignoring the performance costs associated with eating 100x more

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

On Tue, Aug 30, 2016 at 8:01 AM, Andres Freund wrote: > On 2016-08-30 07:57:19 +0530, Amit Kapila wrote: >> I will write such a test case either in this week or early next week. > > Great. > Okay, attached patch adds some simple tests for pg_stat_statements. One thing to note

Re: [HACKERS] improved DefElem list processing

On 8/22/16 10:28 AM, Alvaro Herrera wrote: > Peter Eisentraut wrote: > >> I'm not happy that utils/acl.h has prototypes for aclchk.c, because >> acl.h is included all over the place. Perhaps I should make a >> src/include/catalog/aclchk.c to clean that up. > > I've been bothered by that too in

Re: [HACKERS] pg_sequence catalog

On 9/5/16 10:35 PM, Tom Lane wrote: > In this viewpoint, we'd keep the sequence-specific data in a pg_sequence > catalog. pg_sequence rows would be extensions of the associated pg_class > rows in much the same way that pg_index rows extend the pg_class entries > for indexes. We should supply a

Re: [HACKERS] Block level parallel vacuum WIP

On Wed, Aug 24, 2016 at 3:31 AM, Michael Paquier wrote: > On Tue, Aug 23, 2016 at 10:50 PM, Amit Kapila > wrote: > > On Tue, Aug 23, 2016 at 6:11 PM, Michael Paquier > > wrote: > >> On Tue, Aug 23, 2016 at 8:02 PM,

Re: [HACKERS] pg_basebackup, pg_receivexlog and data durability (was: silent data loss with ext4 / all current versions)

On 3 Sep. 2016 9:22 pm, "Michael Paquier" wrote: > > On Sat, Sep 3, 2016 at 12:42 AM, Magnus Hagander wrote: > > On Fri, Sep 2, 2016 at 8:50 AM, Michael Paquier < michael.paqu...@gmail.com> > > wrote: > >> On Fri, Sep 2, 2016 at 2:20 AM, Peter

Re: [HACKERS] Logical Replication WIP

Review of 0004-Make-libpqwalreceiver-reentrant.patch: This looks like a good change. typo: _PG_walreceirver_conn_init For libpqrcv_create_slot(), slotname should be const char *. Similarly, for slotname in libpqrcv_startstreaming*() and conninfo in libpqrcv_connect(). (the latter two

[HACKERS] Allow to_date() and to_timestamp() to accept localized month names

Hi, attached is a patch which adds support to localized month names in to_date() and to_timestamp() functions. The patch is fairly simple but I want to discuss the approach and implementation: Using the TM modifier as in to_char() was already discussed some years ago:

Re: [HACKERS] pg_basebackup, pg_receivexlog and data durability (was: silent data loss with ext4 / all current versions)

On 9/3/16 9:23 AM, Michael Paquier wrote: > On Sat, Sep 3, 2016 at 10:22 PM, Michael Paquier > wrote: >> On Sat, Sep 3, 2016 at 10:21 PM, Michael Paquier >> wrote: >>> Oh, well. I have just implemented it on top of the two other patches >>>

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

On Fri, Sep 9, 2016 at 10:18 PM, David Steele wrote: > On 9/6/16 10:25 PM, Michael Paquier wrote: >> >> >> On Wed, Sep 7, 2016 at 12:16 AM, David Steele wrote: >> >>> Attached is a new patch that adds sgml documentation. I can expand on >>> each >>>

Re: [HACKERS] An extra error for client disconnection on Windows

On Fri, Sep 9, 2016 at 11:39 PM, Tom Lane wrote: > Haribabu Kommi writes: >> On Thu, Jun 2, 2016 at 6:51 PM, Kyotaro HORIGUCHI < >> horiguchi.kyot...@lab.ntt.co.jp> wrote: >>> After a process termination without PQfinish() of a client, >>> server

Re: [HACKERS] WAL consistency check facility

On Fri, Sep 9, 2016 at 4:01 PM, Kuntal Ghosh wrote: >>> - If WAL consistency check is enabled for a rmgrID, we always include >>> the backup image in the WAL record. >> >> What happens if wal_consistency has different settings on a standby >> and its master? If for

Re: [HACKERS] Tuplesort merge pre-reading

On 09/10/2016 04:21 AM, Claudio Freire wrote: On Fri, Sep 9, 2016 at 9:51 PM, Claudio Freire wrote: On Fri, Sep 9, 2016 at 8:13 AM, Heikki Linnakangas wrote: Claudio, if you could also repeat the tests you ran on Peter's patch set on the other

Re: [HACKERS] Useless dependency assumption libxml2 -> libxslt in MSVC scripts

On Thu, Sep 8, 2016 at 10:04 PM, Tom Lane wrote: > The core code has never used xslt at all. Yes. > Some quick digging in the git > history suggests that contrib/xml2 wasn't very clean about this before > 2008: > [...] > Both of those fixes postdate our native-Windows port,

Re: [HACKERS] COPY command with RLS bug

On Sat, Sep 10, 2016 at 3:55 AM, Adam Brightwell wrote: >> Perhaps we should extend rowsecurity test with a more comprehensive >> set of tests rather than just fix the COPY one? > > I think more tests that provide value are always a *good* thing, > however, would