Re: [HACKERS] Pluggable storage

2016-08-17 Thread Alvaro Herrera
Anastasia Lubennikova wrote: > 13.08.2016 02:15, Alvaro Herrera: > >To support this, we introduce StorageTuple and StorageScanDesc. > >StorageTuples represent a physical tuple coming from some storage AM. > >It is necessary to have a pointer to a StorageAmRoutine in order

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-08-17 Thread Alvaro Herrera
Robert Haas wrote: > Eh ... I doubt very much that it's safe to blow away the entire > contents of an SLRU between shutdown and startup, even if the data is > technically transient data that won't be needed again after the system > is reset. Hmm. At least async.c (pg_notify) deletes the whole di

Re: [HACKERS] Assertion failure in REL9_5_STABLE

2016-08-17 Thread Alvaro Herrera
Marko Tiikkaja wrote: > Running one specific test from our application against REL9_5_STABLE > (current as of today) gives me this: > > #2 0x7effb59595be in ExceptionalCondition ( > conditionName=conditionName@entry=0x7effb5b27a88 "!(CritSectionCount > 0 > || TransactionIdIsCurrentTransa

Re: [HACKERS] Intermittent "cache lookup failed for type" buildfarm failures

2016-08-16 Thread Alvaro Herrera
Tom Lane wrote: > Robert Haas writes: > > It would sure be nice if those cache lookup failure messages printed > > the file and line number. I wonder if we could teach psql to always > > treat the VERBOSITY as verbose when the error code is XX000. > > I looked around when I saw the earlier ones

Re: [HACKERS] Assertion failure in REL9_5_STABLE

2016-08-16 Thread Alvaro Herrera
Andres Freund wrote: > Phew. Alvaro, are you tackling this? Sure. Marko Tiikkaja wrote: > There's a rolled back subtransaction that also did some magic on the rows > AFAICT. I can try and spend some time producing a smaller test case over > the weekend. No hurry since this missed the today's

Re: [HACKERS] max_parallel_degree > 0 for 9.6 beta

2016-08-13 Thread Alvaro Herrera
Michael Paquier wrote: > Being cautious pays more in the long term, so seeing the number of > bugs that showed up I'd rather vote for having it disabled by default > in 9.6 stable, and enabled on master to aim at enabling it in 10.0. I too prefer to keep it turned off in 9.6 and consider enabling

[HACKERS] Pluggable storage

2016-08-12 Thread Alvaro Herrera
Many have expressed their interest in this topic, but I haven't seen any design of how it should work. Here's my attempt; I've been playing with this for some time now and I think what I propose here is a good initial plan. This will allow us to write permanent table storage that works differentl

Re: [HACKERS] Small issues in syncrep.c

2016-08-11 Thread Alvaro Herrera
Simon Riggs wrote: > Good catch. > > I've updated Julien's patch to reflect Michael's suggestion. > > Looks good to apply immediately. > > 14e8803f1 was only a partial patch for the syncrep code, so I don't > see any reason to keep the code as it currently is in 9.5/9.6. > > Any objections to

Re: [HACKERS] Assertion failure in REL9_5_STABLE

2016-08-10 Thread Alvaro Herrera
BTW this is not a regression, right? It's been broken all along. Or am I mistaken? -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Assertion failure in REL9_5_STABLE

2016-08-10 Thread Alvaro Herrera
Marko Tiikkaja wrote: > On 2016-08-10 19:28, Alvaro Herrera wrote: > >Umm. AFAICS HeapTupleSatisfiesUpdate() only returns SelfUpdated after > >already calling HeapTupleHeaderGetCmax() (which obviously hasn't caught > >the same assertion). Something is odd there ... >

Re: [HACKERS] regression test for extended query protocol

2016-08-10 Thread Alvaro Herrera
Michael Paquier wrote: > On Fri, Aug 5, 2016 at 12:21 AM, Alvaro Herrera > wrote: > > If somebody had some spare time to devote to this, I would suggest to > > implement something in core that can be used to specify a list of > > commands to run, and a list of files-to-be

Re: [HACKERS] Assertion failure in REL9_5_STABLE

2016-08-10 Thread Alvaro Herrera
Marko Tiikkaja wrote: > Hi, > > Running one specific test from our application against REL9_5_STABLE > (current as of today) gives me this: > > #2 0x7effb59595be in ExceptionalCondition ( > conditionName=conditionName@entry=0x7effb5b27a88 "!(CritSectionCount > 0 > || TransactionIdIsCurre

Re: [HACKERS] Logical Replication WIP

2016-08-09 Thread Alvaro Herrera
Petr Jelinek wrote: > On 09/08/16 10:13, Craig Ringer wrote: > >The only argument I can see for not using bgworkers is for the > >supervisor worker. It's a singleton that launches the per-database > >workers, and arguably is a job that the postmaster could do better. The > >current design there st

Re: [HACKERS] Logical Replication WIP

2016-08-09 Thread Alvaro Herrera
Petr Jelinek wrote: > On 09/08/16 12:16, Craig Ringer wrote: > >Right. I think that's probably the direction we should be going > >eventually. Personally I don't think such a change should block the > >logical replication work from proceeding with bgworkers, though. > > Yeah that's why I added lo

Re: [HACKERS] patch: xmltable - proof concept

2016-08-09 Thread Alvaro Herrera
Pavel Stehule wrote: > postgres=# SELECT xmltable.* > postgres-#FROM (SELECT data FROM xmldata) x, > postgres-# LATERAL xmltable('/ROWS/ROW' > postgres(# PASSING data > postgres(# COLUMNS id int PATH '@id', > postgres(#

Re: [HACKERS] No longer possible to query catalogs for index capabilities?

2016-08-09 Thread Alvaro Herrera
Tom Lane wrote: > Andrew Gierth writes: > > Where'd be a good place to put that function? ruleutils? catalog/index.c ? > > > (ruleutils is way too big already) > > Agreed. catalog/index.c is not a place that implements SQL-visible > functions, so I don't like that either. > > One idea is utils

Re: [HACKERS] Refactoring of heapam code.

2016-08-08 Thread Alvaro Herrera
Anastasia Lubennikova wrote: > By the way, I thought about looking at the mentioned patch and > probably reviewing it, but didn't find any of your patches on the > current commitfest. Could you point out the thread? Sorry, I haven't posted anything yet. > >Agreed. But changing its name while ke

Re: [HACKERS] No longer possible to query catalogs for index capabilities?

2016-08-07 Thread Alvaro Herrera
Tom Lane wrote: > Building on the has-property approach Andrew suggested, I wonder if > we need something like pg_index_column_has_property(indexoid, colno, > propertyname) with properties like "sortable", "desc", "nulls first". This seems simple enough, on the surface. Why not run with this des

Re: [HACKERS] Refactoring of heapam code.

2016-08-05 Thread Alvaro Herrera
Anastasia Lubennikova wrote: > Working on page compression and some other issues related to > access methods, I found out that the code related to heap > looks too complicated. Much more complicated, than it should be. > Since I anyway got into this area, I want to suggest a set of improvements. H

Re: [HACKERS] regression test for extended query protocol

2016-08-04 Thread Alvaro Herrera
Michael Paquier wrote: > On Thu, Aug 4, 2016 at 12:14 AM, Dave Cramer wrote: > > We currently run tests every time a PR is created on github, but I don't > > think there are any animals running the JDBC test suite > > > > We can add tests, what exactly do we want to test. Then setting up an animal

Re: [HACKERS] PostmasterContext survives into parallel workers!?

2016-08-03 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> It looks to me like the reason for it is simply not having bothered to > >> copy the rw->rw_worker data to somewhere that would survive deletion > >> of the PostmasterContext. I w

Re: [HACKERS] Increasing timeout of poll_query_until for TAP tests

2016-08-03 Thread Alvaro Herrera
Michael Paquier wrote: > On Wed, Aug 3, 2016 at 7:21 AM, Alvaro Herrera > wrote: > > Why not capture both items in a single select, such as in the attached > > patch? > > Let me test this > [... A while after ...] > This looks to work properly. 12 runs in a r

Re: [HACKERS] Increasing timeout of poll_query_until for TAP tests

2016-08-02 Thread Alvaro Herrera
Michael Paquier wrote: > Here using pg_xlog_replay_resume() is not the correct solution because > this would cause the node to finish recovery before we want it to, and > so is recovery_target_action = 'promote'. If we look at the test, it > is doing the following when getting the TXID that is use

Re: [HACKERS] PostgreSQL 10 kick-off

2016-08-02 Thread Alvaro Herrera
Peter Eisentraut wrote: > On 8/1/16 1:08 PM, Fabrízio de Royes Mello wrote: > > What knowledge is expected for a CFM? I'm really would like to help and > > also learn more about our development. > > If you search the wiki for "commitfest" you will find several pages of > varying outdatedness that

Re: [HACKERS] Increasing timeout of poll_query_until for TAP tests

2016-08-02 Thread Alvaro Herrera
Michael Paquier wrote: > On Tue, Aug 2, 2016 at 10:28 AM, Michael Paquier > wrote: > > There is still an issue with pg_basebackup when testing stream mode > > and replication slots. I am digging into this one now.. > > After 5 hours running this test in a row and 30 attempts torturing > hamster w

Re: [HACKERS] PostmasterContext survives into parallel workers!?

2016-08-02 Thread Alvaro Herrera
Tom Lane wrote: > Robert Haas writes: > > On Mon, Aug 1, 2016 at 4:18 PM, Tom Lane wrote: > >> Now, I'm undecided whether to flush that context only in parallel workers, > >> or to try to make it go away for all bgworkers of any stripe. The latter > >> seems a little better from a security stand

Re: [HACKERS] TODO item: Implement Boyer-Moore searching in LIKE queries

2016-08-02 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Aug 1, 2016 at 1:19 PM, Karan Sikka wrote: > > Following the patch to implement strpos with Boyer-Moore-Horspool, > > it was proposed we bring BMH to LIKE as well. > > > > Original thread: > > https://www.postgresql.org/message-id/flat/27645.1220635769%40sss.pgh.pa.us#

Re: [HACKERS] New version numbering practices

2016-08-01 Thread Alvaro Herrera
Michael Paquier wrote: > On Tue, Aug 2, 2016 at 5:25 AM, Tom Lane wrote: > > Andrew Dunstan writes: > >> Somewhat related is how we name the git branches. It would help me from > >> a buildfarm POV if we kept lexically them sortable, which could be done > >> at least for the next 90 major release

Re: [HACKERS] PostmasterContext survives into parallel workers!?

2016-08-01 Thread Alvaro Herrera
Tom Lane wrote: > Andres Freund writes: > > On 2016-08-01 18:09:03 -0400, Robert Haas wrote: > >> (Also vaguely on the list of things to clean up: can't we make it so > >> that bgworkers aren't launched from inside a signal handler? Blech.) > > > Isn't pretty much everything on-demand below post

Re: [HACKERS] HandleParallelMessages contains CHECK_FOR_INTERRUPTS?

2016-08-01 Thread Alvaro Herrera
Tom Lane wrote: > $SUBJECT seems like a pretty bad idea, because it implies a recursive > entry to ProcessInterrupts and thence to HandleParallelMessages itself. > By what reasoning is that call necessary where it's placed? I notice you just removed the CHECK_FOR_INTERRUPTS in HandleParallelMessag

Re: [HACKERS] New version numbering practices

2016-08-01 Thread Alvaro Herrera
Tom Lane wrote: > Also, it strikes me that we need a new convention for how we talk about > release branches informally. Up to now, mentioning say "9.5" without > any further qualification in a PG-list message was usually sufficient > to indicate a branch number, but I do not think that will work

Re: [HACKERS] Design for In-Core Logical Replication

2016-07-29 Thread Alvaro Herrera
Robert Haas wrote: > One minor comment is that this document makes extensive use of Terms > With Initial Capitals, which seems stylistically odd, although I'm not > precisely sure what would be better. We use publication on the first use only, which is turned into italics. -- Álvaro Herrera

Re: [HACKERS] BRIN vs. HOT

2016-07-28 Thread Alvaro Herrera
Robert Haas wrote: > If I understand correctly, we currently deem an update to be non-HOT > whenever any indexed column is updated. This is because tuple > versions created by HOT updates can later be removed by HOT pruning, > which means that they must not be referenced by index entries. > Otherw

Re: [HACKERS] Rethinking TupleTableSlot deforming

2016-07-27 Thread Alvaro Herrera
Andres Freund wrote: > Hi, > > I've previously mentioned (e.g. [1]) that tuple deforming is a serious > bottlneck. I've also experimented successfully [2] making > slot_deform_tuple() faster. I'm currently messing with making heapam.c be just one possible implementation of tuple storage, to allow

Re: [HACKERS] Curing plpgsql's memory leaks for statement-lifespan values

2016-07-25 Thread Alvaro Herrera
Alvaro Herrera wrote: > Tom Lane wrote: > > > Although this is in principle a bug fix, it's invasive enough that I'd > > be pretty hesitant to back-patch it, or even to stick it into HEAD > > post-beta. I'm inclined to sign it up for the next commitfest i

Re: [HACKERS] Curing plpgsql's memory leaks for statement-lifespan values

2016-07-25 Thread Alvaro Herrera
Tom Lane wrote: > Although this is in principle a bug fix, it's invasive enough that I'd > be pretty hesitant to back-patch it, or even to stick it into HEAD > post-beta. I'm inclined to sign it up for the next commitfest instead. Do we have a backpatchable fix for the reported problem? If so,

Re: [HACKERS] Increasing timeout of poll_query_until for TAP tests

2016-07-24 Thread Alvaro Herrera
Michael Paquier wrote: > Lately hamster is failing every 4/5 days on the recovery regression > tests in 003 covering the recovery targets, with that: > # Postmaster PID for node "standby_2" is 20510 > # > Timed out while waiting for standby to catch up at > t/003_recovery_targets.pl line 36. > >

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

2016-07-24 Thread Alvaro Herrera
Andrew Borodin wrote: > >If a feature changes the shape of WAL records, XLOG_PAGE_MAGIC is bumped to > >prevent any problems. > I've attached patch with a bump, but, obviously, it'll be irrelevant > after any other commit changing WAL shape. > > Here is a patch with updated GiST README. > I haven

Re: [HACKERS] Bug with plpgsql handling of NULL argument of compound type

2016-07-22 Thread Alvaro Herrera
Tom Lane wrote: > Merlin Moncure writes: > > Not sure we are guided there. Currently we follow the spec > > specifically with the IS NULL operator but not in other cases. > > Yeah. ExecEvalNullTest() has been taught about this, but most other > places that check null-ness just check overall dat

Re: [HACKERS] DROP OWNED BY ... CACADE & "could not open relation with OID" error

2016-07-20 Thread Alvaro Herrera
Satoshi Nagayasu wrote: > Hi, > > I have been trying MADlib [1], a machine-learning library for PostgreSQL, > and when I was tying it on 9.5 and 9.6beta2, I often got following > error on my box. > > > madpack.py : ERROR : SQL command failed: > SQL: DROP OWNED

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2016-07-20 Thread Alvaro Herrera
Michael Paquier wrote: > On Wed, Jul 6, 2016 at 4:18 PM, Michael Paquier > wrote: > > OK, after hacking that for a bit I have finished with option 2 and the > > set of PG-like set of routines, the use of USE_SSL in the file > > containing all the SHA functions of OpenBSD has proved to be really >

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-07-20 Thread Alvaro Herrera
Andres Freund wrote: > On 2016-07-20 11:26:11 -0400, Robert Haas wrote: > > On Wed, Jul 20, 2016 at 3:39 AM, Andres Freund wrote: > > >>I think Snapshot's members whenTaken and lsn are updated/initialized > > >>only in GetSnapshotData(). So if GetSnapshotData() is not used, how > > >>will you exp

Re: [HACKERS] Adjust recovery test file name

2016-07-19 Thread Alvaro Herrera
Masahiko Sawada wrote: > Hi all, > > The file 006_logical_decoding_timelines.pl was removed by the commit c1543a8. > But currently 005_***.pl and 007_***.pl exist on source tree. > Should we change its file number to 006? I don't think we need to bother about this. Whenever somebody submits a ne

Re: [HACKERS] Regression tests vs existing users in an installation

2016-07-15 Thread Alvaro Herrera
Tom Lane wrote: > We've talked before about how the regression tests should be circumspect > about what role names they create/drop, so as to avoid possibly blowing > up an installation's existing users during "make installcheck". In > particular I believe there was consensus that such names shoul

Re: [HACKERS] Version number for pg_control

2016-07-15 Thread Alvaro Herrera
David Steele wrote: > On 7/15/16 5:47 PM, Alvaro Herrera wrote: > > I can't quite make up my mind about it. It seems pointless to change > > it now, but at the same time it seems wrong to let it continue to be > > unchanged from 9.4. > > > > I slightly le

Re: [HACKERS] Version number for pg_control

2016-07-15 Thread Alvaro Herrera
Tom Lane wrote: > While researching a pgsql-general question, I noticed that commit > 73c986adde5d73a5e2555da9b5c8facedb146dcd added several new fields > to pg_control without bothering to touch PG_CONTROL_VERSION. Thus, > PG_CONTROL_VERSION is still "942" even though the file contents > are not a

Re: [HACKERS] heap_xlog_lock forgets to reset HEAP_XMAX_INVALID

2016-07-15 Thread Alvaro Herrera
Andres Freund wrote: > On 2016-07-15 17:23:57 -0400, Alvaro Herrera wrote: > > Maybe we should change fix_infomask_from_infobits() to reset > > HEAP_XMAX_BITS | HEAP_MOVED too (and HEAP_KEYS_UPDATED I suppose), to > > avoid doing it in each callsite. > > Yea, I was th

Re: [HACKERS] heap_xlog_lock forgets to reset HEAP_XMAX_INVALID

2016-07-15 Thread Alvaro Herrera
Andres Freund wrote: > While validating my patch for > http://archives.postgresql.org/message-id/20160714060607.klwgq2qr7egt3zrr%40alap3.anarazel.de > I noticed that the standby's infomask still had HEAP_XMAX_INVALID set > after replaying a XLOG_HEAP_LOCK record. > > That's bad, but not really c

Re: [HACKERS] unexpected psql "feature"

2016-07-13 Thread Alvaro Herrera
David G. Johnston wrote: > On Wed, Jul 13, 2016 at 5:44 PM, Fabien COELHO wrote: > > > Although "\;" behavior is not documented, I would have expected both > >>> results to be shown one after the other, or having a an error, but not a > >>> quiet discard. > >> > >> See the documentation for PQexe

Re: [HACKERS] Showing parallel status in \df+

2016-07-12 Thread Alvaro Herrera
Peter Eisentraut wrote: > I'm quite fond of having the full source code show in \df+ and I'm > against removing it on short notice past beta2, discussed under a > "false" subject heading. How do you use it? -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x

Re: [HACKERS] Showing parallel status in \df+

2016-07-11 Thread Alvaro Herrera
Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > It would certainly be easy enough to do that, as long as you don't mind > > hard-wiring into psql the knowledge that "internal" and "C" are the > > languages to show prosrc for. "Source code" would no longer be a very > > appropria

[HACKERS] remove checkpoint_warning

2016-07-09 Thread Alvaro Herrera
the checkpoint_warning feature was added by commit 2986aa6a668bce3cfb836 in November 2002 when we didn't have any logging of checkpointing at all. I propose to remove it: surely anyone who cares about analyzing checkpointing behavior nowadays is using the log_checkpoint feature instead, which cont

Re: [HACKERS] \timing interval

2016-07-09 Thread Alvaro Herrera
Andrew Gierth wrote: > How about > > Time: 1234567.666 ms (20m 34.6s) > > ? +1 LGTM -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To ma

Re: [HACKERS] MVCC overheads

2016-07-08 Thread Alvaro Herrera
Peter Geoghegan wrote: > Has anyone ever done any kind of write-up of the "TED" design that was > discussed during FOSDEM (I hope I recall the name it was given > correctly)? Apparently that's something that's been discussed a few > times among senior community members, and I think it has promise.

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

2016-07-08 Thread Alvaro Herrera
Kyotaro HORIGUCHI wrote: > At Fri, 10 Jun 2016 17:39:59 +0900, Michael Paquier > wrote in > > > On Thu, Jun 9, 2016 at 9:55 PM, Kyotaro HORIGUCHI > > wrote: > > Indeed, and you could just do the following to reproduce the failure > > with the recovery test suite, so I would suggest adding th

Re: [HACKERS] MVCC overheads

2016-07-08 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Pete Stevenson wrote: > >> Maybe I could figure out the lines of code that add versions into a > >> table and then those that collect old versions (they do get collected, > >> right?). Anyway, thought being I

Re: [HACKERS] MVCC overheads

2016-07-08 Thread Alvaro Herrera
Pete Stevenson wrote: > Maybe I could figure out the lines of code that add versions into a > table and then those that collect old versions (they do get collected, > right?). Anyway, thought being I could profile while running TPC-C or > similar. I was hoping that someone might be able to jump on

Re: [HACKERS] Showing parallel status in \df+

2016-07-08 Thread Alvaro Herrera
Tom Lane wrote: > Magnus Hagander writes: > > On Friday, July 8, 2016, Michael Paquier wrote: > >> Fujii-san has reminded me of the fact that we do not show in \df+ the > >> parallel status of a function. The output of \df+ is already very > >> large, so I guess that any people mentally sane alre

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-07 Thread Alvaro Herrera
Tom Lane wrote: > Simon Riggs writes: > > pg_am has existed for decades without supporting DDL > > That argument has been obsoleted by events ;-) ... and in any case, the > reason we went without CREATE ACCESS METHOD for so long was not that we > encouraged "INSERT INTO pg_am" but that non-core

Re: [HACKERS] Header and comments describing routines in incorrect shape in visibilitymap.c

2016-07-07 Thread Alvaro Herrera
Michael Paquier wrote: > Hi all, > > I just bumped into a couple of things in visibilitymap.c: > - visibilitymap_clear clears all bits of a visibility map, its header > comment mentions the contrary > - The header of visibilitymap.c mentions correctly "a bit" when > referring to setting them, but

Re: [HACKERS] Reviewing freeze map code

2016-07-07 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Jul 6, 2016 at 6:06 PM, Andres Freund wrote: > > Hm. We can't easily do that in the back-patched version; because a > > standby won't know to check for the flag . That's kinda ok, since we > > don't yet need to clear all-visible yet at that point of > > heap_update. Bu

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-07-06 Thread Alvaro Herrera
Michael Paquier wrote: > On Wed, Jul 6, 2016 at 7:34 PM, Fujii Masao wrote: > > I have one question; why do we call the column "conn_info" instead of > > "conninfo" which is basically used in other places? "conninfo" is better to > > me. > > No real reason for one or the other to be honest. If y

Re: [HACKERS] Typo Patch

2016-07-05 Thread Alvaro Herrera
CharSyam wrote: > Yes, Some typos problems, It is hard to figure them out. > But in this patch, guranteed is just misspelling of guaranteed. I agree with Robert: the sentence as a whole makes no sense: * We need to check all possible distances, so reset Lpos * to guarant

Re: [HACKERS] Re: GiST optimizing memmoves in gistplacetopage for fixed-size updates [PoC]

2016-07-04 Thread Alvaro Herrera
Andrew Borodin wrote: > Thank you, Amit. > > Currently, if WAL reconstruct page in another order it won't be a problem. We require that replay of WAL leads to identical bytes than the original. Among other things, this enables use of a tool that verifies that WAL is working correctly simply by c

Re: [HACKERS] Renaming more clearly SHA functions in pgcrypto/

2016-07-04 Thread Alvaro Herrera
Michael Paquier wrote: > Hi all, > (folks interested in that in CC) > > While looking at some facility in pgcrypto, I have noticed the stanza > created by 56f4478 to prevent conflicts with OpenSSL, like that: > +#define SHA256_Init pg_SHA256_Init > +#define SHA256_Update pg_SHA256_Update > > Woul

Re: [HACKERS] OpenSSL 1.1 breaks configure and more

2016-07-01 Thread Alvaro Herrera
Thanks for this effort. > static BIO_METHOD * > my_BIO_s_socket(void) > { > - if (!my_bio_initialized) > + if (!my_bio_methods) > { > - memcpy(&my_bio_methods, BIO_s_socket(), sizeof(BIO_METHOD)); > - my_bio_methods.bread = my_sock_read; > - my_

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-07-01 Thread Alvaro Herrera
Michael Paquier wrote: > On Fri, Jul 1, 2016 at 8:50 AM, Michael Paquier > wrote: > >> Okay, that argument I buy. > >> > >> I suppose this function/view should report no row at all if there is no > >> wal receiver connected, rather than a view with nulls. > > > > The function returns PG_RETURN_NU

Re: [HACKERS] EXISTS clauses not being optimized in the face of 'one time pass' optimizable expressions

2016-07-01 Thread Alvaro Herrera
Merlin Moncure wrote: > It's pretty easy to craft a query where you're on the winning side, > but what's the worst case of doing two pass...is constant folding a > non trivial fraction of planning time? One thing that has been suggested is to re-examine the plan after planning is done, and if exe

Re: [HACKERS] initdb issue on 64-bit Windows - (Was: [pgsql-packagers] PG 9.6beta2 tarballs are ready)

2016-06-30 Thread Alvaro Herrera
Craig Ringer wrote: > On 30 June 2016 at 20:19, Alvaro Herrera wrote: > > > Hmm, so what about a pure 32bit build, if such a thing still exists? If > > so and it causes the same crash, perhaps we should have one member for > > each VS version running on 32bit x86. >

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-06-30 Thread Alvaro Herrera
Michael Paquier wrote: > On Fri, Jul 1, 2016 at 8:35 AM, Alvaro Herrera > wrote: > > Michael Paquier wrote: > >> On Thu, Jun 30, 2016 at 11:24 PM, Alvaro Herrera > >> wrote: > >> > Also, actually, I see no reason for the conninfo to be shown different

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-06-30 Thread Alvaro Herrera
Michael Paquier wrote: > On Thu, Jun 30, 2016 at 11:24 PM, Alvaro Herrera > wrote: > > Also, actually, I see no reason for the conninfo to be shown differently > > regardless of a connection being already established. If we show the > > conninfo that the server is tr

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-06-30 Thread Alvaro Herrera
Fujii Masao wrote: > On Thu, Jun 30, 2016 at 10:12 PM, Michael Paquier > wrote: > > On Thu, Jun 30, 2016 at 9:41 PM, Alvaro Herrera > > wrote: > >> Fujii Masao wrote: > >>> On Thu, Jun 30, 2016 at 9:30 PM, Michael Paquier > >>> wrote: &

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-06-30 Thread Alvaro Herrera
Fujii Masao wrote: > On Thu, Jun 30, 2016 at 9:30 PM, Michael Paquier > wrote: > > On Thu, Jun 30, 2016 at 8:59 PM, Fujii Masao wrote: > >> ISTM that we will never be able to get out of this loop if walreceiver > >> fails to connect to the master (e.g., password is wrong) after we enter > >> thi

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-06-30 Thread Alvaro Herrera
Fujii Masao wrote: > On Thu, Jun 30, 2016 at 6:01 AM, Alvaro Herrera > wrote: > > Alvaro Herrera wrote: > > > >> I propose to push this patch, closing the open item, and you can rework > >> on top -- I suppose you would completely remove the original conninfo

Re: [HACKERS] A couple of cosmetic changes around shared memory code

2016-06-30 Thread Alvaro Herrera
Robert Haas wrote: > It might be better to document this in bgworker.sgml instead. That > already documents some facts about exiting: > > >If bgw_restart_time for a background worker is >configured as BGW_NEVER_RESTART, or if it exits with an exit >code of 0 or is terminated by Te

Re: [HACKERS] initdb issue on 64-bit Windows - (Was: [pgsql-packagers] PG 9.6beta2 tarballs are ready)

2016-06-30 Thread Alvaro Herrera
Craig Ringer wrote: > On 30 June 2016 at 07:21, Tom Lane wrote: > > > Alvaro Herrera writes: > > > Tom Lane wrote: > > >> Thanks for investigating! I'll go commit that change. I wish someone > > >> would put up a buildfarm critter using VS201

Re: [HACKERS] 10.0

2016-06-30 Thread Alvaro Herrera
Gavin Flower wrote: > I hate the rampant inflation associated with numbering schemes like FireFox > - the numbers of no meaning at all, other than something non-trivial has > been changed, with no indication at all about how non-trivial! I thought this horse had already been beaten to death -- ap

Re: [HACKERS] initdb issue on 64-bit Windows - (Was: [pgsql-packagers] PG 9.6beta2 tarballs are ready)

2016-06-29 Thread Alvaro Herrera
Michael Paquier wrote: > On Fri, Jun 24, 2016 at 11:51 AM, Tsunakawa, Takayuki > wrote: > >> From: pgsql-hackers-ow...@postgresql.org > >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Michael Paquier > >> Sent: Friday, June 24, 2016 11:37 AM > >> On Fri, Jun 24, 2016 at 11:33 AM, Craig

Re: [HACKERS] initdb issue on 64-bit Windows - (Was: [pgsql-packagers] PG 9.6beta2 tarballs are ready)

2016-06-29 Thread Alvaro Herrera
Tom Lane wrote: > "Haroon ." writes: > > The problem appears to be related to 'taking the address of a formal > > parameter'. NOT passing the original formal parameter to > > get_foreign_key_join_selectivity fixes it (dodges the problem) on VS2013. > > Thanks for investigating! I'll go commit t

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-06-29 Thread Alvaro Herrera
Alvaro Herrera wrote: > I propose to push this patch, closing the open item, and you can rework > on top -- I suppose you would completely remove the original conninfo > from shared memory and instead only copy the obfuscated version there > (and probably also remove the ready_to_

Re: [HACKERS] WIP: About CMake v2

2016-06-29 Thread Alvaro Herrera
Yury Zhuravlev wrote: > Hello Hackers. > > I decided to talk about the current state of the project: > 1. Merge with 9.6 master. 2. plpython2, plpython3, plperl, pltcl, plsql all > work correctly (all tests pass). > 3. Works done for all contrib modules. 4. You can use gettext, .po->.mo will > hav

Re: [HACKERS] An unkillable connection caused replication delay on my replica

2016-06-29 Thread Alvaro Herrera
Shawn wrote: > Unfortunately...no. I have been trying to repro this scenario. Is there a > specific way to make a Python connection where this is possible? > > My end game, if this is not something that can be fixed on the Postgres > side, is to come up with a way to automatically cause the conn

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-06-29 Thread Alvaro Herrera
Fujii Masao wrote: > On Thu, Jun 30, 2016 at 2:50 AM, Alvaro Herrera > wrote: > > Fujii Masao wrote: > >> On Wed, Jun 29, 2016 at 12:23 PM, Alvaro Herrera > >> wrote: > >> > Michael Paquier wrote: > >> >> On Wed, Jun 29, 2016 at 6:42 A

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-06-29 Thread Alvaro Herrera
Fujii Masao wrote: > On Wed, Jun 29, 2016 at 12:23 PM, Alvaro Herrera > wrote: > > Michael Paquier wrote: > >> On Wed, Jun 29, 2016 at 6:42 AM, Alvaro Herrera > >> wrote: > > > >> > I have already edited the patch following some of th

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-06-28 Thread Alvaro Herrera
Michael Paquier wrote: > On Wed, Jun 29, 2016 at 6:42 AM, Alvaro Herrera > wrote: > > I have already edited the patch following some of these ideas. Will > > post a new version later. > > Cool, thanks. Here it is. I found it was annoying to maintain the function retu

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-06-28 Thread Alvaro Herrera
Michael Paquier wrote: > I have been thinking more about that, and came up with the following > idea... We do not want to link libpq directly to the server, so let's > add a new routine to libpqwalreceiver that builds an obfuscated > connection string and let's have walreceiver.c save it in shared

Re: [HACKERS] An unkillable connection caused replication delay on my replica

2016-06-28 Thread Alvaro Herrera
Shawn wrote: > strace of long-running query pid 6819 loops like this: > > sendto(10, "" NULL, 0) = ? ERESTARTSYS (To be > restarted if SA_RESTART is set) > --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=8719, > si_uid=3001} --- > rt_sigreturn() > > Where pid 6819 is

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-06-28 Thread Alvaro Herrera
Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Michael Paquier writes: > > > On Tue, Jun 21, 2016 at 11:29 AM, Tom Lane wrote: > > >> What I would want to know is whether this specific change is actually a > > >> good idea. In particular, I'm concerned about the possible secur

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-06-28 Thread Alvaro Herrera
Noah Misch wrote: > On Sun, Jun 19, 2016 at 05:56:12PM +0900, Michael Paquier wrote: > > The new pg_stat_wal_receiver does not include primary_conninfo. > > Looking at that now, it looks almost stupid not to include it... > > Adding it now would require a catalog bump, so I am not sure if this > >

Re: [HACKERS] [HITB-Announce] HITB2016AMS Videos & GSEC Singapore Voting

2016-06-28 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Jun 27, 2016 at 5:29 PM, Alvaro Herrera > wrote: > > Robert Haas wrote: > >> On Mon, Jun 20, 2016 at 5:41 PM, Hafez Kamal > >> wrote: > >> > See you in Singapore! > >> > >> This seems totally off-topic. S

Re: [HACKERS] [HITB-Announce] HITB2016AMS Videos & GSEC Singapore Voting

2016-06-27 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Jun 20, 2016 at 5:41 PM, Hafez Kamal wrote: > > See you in Singapore! > > This seems totally off-topic. Shouldn't a post like this result in a ban? It is off-topic. Sorry that it got through. We get dozens of these every week, and the vast majority are rejected;

Re: [HACKERS] MultiXactId error after upgrade to 9.3.4

2016-06-24 Thread Alvaro Herrera
After some further testing, I noticed a case that wasn't handled in heap_update, which I also fixed. I reworded some comments here and there, and pushed to all branches. Further testing and analysis is welcome. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development,

Re: [HACKERS] Odd behavior with domains

2016-06-24 Thread Alvaro Herrera
Joshua D. Drake wrote: > Yes but what makes it weird is this: > > postgres=# create domain text char(3); > CREATE DOMAIN > > -- cool, no problem > > postgres=# create domain text char(2); > ERROR: type "text" already exists > > -- as expected > > postgres=# \dD > List of domains

Re: [HACKERS] Odd behavior with domains

2016-06-23 Thread Alvaro Herrera
Joshua D. Drake wrote: > Hey, > > So this came across my twitter feed: > > https://pbs.twimg.com/media/ClqIJtmXEAA5IGt.png > > I have verified the oddness with a newer version: Well, it's not specifically related to domains -- it's related to the fact that pg_catalog objects mask the domain you

Re: [HACKERS] Reviewing freeze map code

2016-06-23 Thread Alvaro Herrera
Andres Freund wrote: > I'm looking into three approaches right now: > > 3) Use WAL logging for the already_marked = true case. > 3) This approach so far seems the best. It's possible to reuse the > xl_heap_lock record (in an afaics backwards compatible manner), and in > most cases the overhead

Re: [HACKERS] Rethinking representation of partial-aggregate steps

2016-06-23 Thread Alvaro Herrera
Tom Lane wrote: > What I'm imagining is, say, > > #define AGGOP_COMBINESTATES 0x1 > #define AGGOP_SERIALIZESTATES 0x2 > #define AGGOP_DESERIALIZESTATES 0x4 > #define AGGOP_FINALIZEAGGS 0x8 > > typedef enum AggPartialMode > { > AGGPARTIAL_SIMPLE = AGGOP_FINALIZEAGGS, > AGGPARTIAL_PARTIAL

Re: [HACKERS] Rethinking representation of partial-aggregate steps

2016-06-23 Thread Alvaro Herrera
Tom Lane wrote: > Robert Haas writes: > > I do agree, however, that the three Boolean flags don't make the code > > entirely easy to read. What I might suggest is that we replace the > > three Boolean flags with integer flags, something like this: > > Yeah, that's another way we could go. I ha

Re: [HACKERS] Bug in to_timestamp().

2016-06-23 Thread Alvaro Herrera
David G. Johnston wrote: > On Thursday, June 23, 2016, Alex Ignatov wrote: > > > Arguing just like that one can say that we don't even need exception like > > "division by zero". Just use well-formed numbers in denominator... > > Input data sometimes can be generated automagically. Without excep

Re: [HACKERS] MultiXactId error after upgrade to 9.3.4

2016-06-22 Thread Alvaro Herrera
varo Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services commit 7235c0b120208d73d53d3929fe8243d5e487dca8 Author: Alvaro Herrera AuthorDate: Thu Jun 16 23:33:20 2016 -0400 CommitDate: Wed Jun 22 17:17:15 2016 -0400

Re: [HACKERS] MultiXactId error after upgrade to 9.3.4

2016-06-21 Thread Alvaro Herrera
Alvaro Herrera wrote: > Robert Haas wrote: > > On Fri, Jun 17, 2016 at 9:33 AM, Andrew Gierth > > wrote: > > >>>>>> "Robert" == Robert Haas writes: > > > >> Why is the correct rule not "check for and ignore pre-upgrade mxids &

<    5   6   7   8   9   10   11   12   13   14   >