Re: [HACKERS] 9.6 TAP tests and extensions

2016-09-24 Thread Craig Ringer
On 24 Sep. 2016 04:04, "Tom Lane" wrote:. > > > > > It's thus sufficient to apply the patch to install the perl modules to > > 9.4, 9.5 and 9.6. Nothing else is needed. I've attached backports for > > 9.4 and 9.5. > > Pushed with cosmetic adjustments --- Thanks. > Looking back over the thread, I

Re: [HACKERS] Refactor StartupXLOG?

2016-09-24 Thread Heikki Linnakangas
On 09/24/2016 05:01 AM, Thomas Munro wrote: What would the appetite be for that kind of refactoring work, considering the increased burden on committers who have to backpatch bug fixes? Is it a project goal to reduce the size of large complicated functions like StartupXLOG and heap_update? It s

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2016-09-24 Thread Masahiko Sawada
On Wed, Sep 21, 2016 at 11:22 PM, Robert Haas wrote: > On Wed, Sep 21, 2016 at 5:54 AM, Petr Jelinek wrote: Reading again the thread, it seems that my previous post [1] was a bit misunderstood. My position is to not introduce any new behavior changes in 9.6, so we could just make t

Re: [HACKERS] Refactor StartupXLOG?

2016-09-24 Thread Thomas Munro
On Sat, Sep 24, 2016 at 8:24 PM, Heikki Linnakangas wrote: > On 09/24/2016 05:01 AM, Thomas Munro wrote: >> >> What would the appetite be for that kind of refactoring work, >> considering the increased burden on committers who have to backpatch >> bug fixes? Is it a project goal to reduce the siz

Re: [HACKERS] pgbench - minor fix for meta command only scripts

2016-09-24 Thread Fabien COELHO
Hello Heikki, Yeah, it really is quite a mess. I tried to review your patch, and I think it's correct, but I couldn't totally convince myself, because of the existing messiness of the logic. So I bit the bullet and started refactoring. I came up with the attached. It refactors the logic in d

Re: [HACKERS] ICU integration

2016-09-24 Thread Peter Geoghegan
On Fri, Sep 23, 2016 at 7:27 AM, Thomas Munro wrote: > A couple of thoughts about abbreviated keys: > > #ifndef TRUST_STRXFRM > if (!collate_c) > abbreviate = false; > #endif > > I think this macro should affect only strxfrm, and we should trust > ucol_getSortKey or disable it independ

Re: [HACKERS] Refactoring speculative insertion with unique indexes a little

2016-09-24 Thread Peter Geoghegan
On Tue, Sep 20, 2016 at 8:55 AM, Heikki Linnakangas wrote: > Thanks for working on this, and sorry for disappearing and dropping this on > the floor earlier. This doesn't apply anymore, thanks to 9c810a2e. Shouldn't > be hard to fix. Thanks for looking at it again. > I was in support of this ear

Re: [HACKERS] Refactor StartupXLOG?

2016-09-24 Thread Michael Paquier
On Sat, Sep 24, 2016 at 11:01 AM, Thomas Munro wrote: > What would the appetite be for that kind of refactoring work, > considering the increased burden on committers who have to backpatch > bug fixes? Is it a project goal to reduce the size of large > complicated functions like StartupXLOG and he

Re: [HACKERS] WIP: Covering + unique indexes.

2016-09-24 Thread Amit Kapila
On Wed, Sep 21, 2016 at 6:51 PM, Anastasia Lubennikova wrote: > 20.09.2016 08:21, Amit Kapila: > > On Tue, Sep 6, 2016 at 10:18 PM, Anastasia Lubennikova > wrote: > > 28.08.2016 09:13, Amit Kapila: > > > The problem seems really tricky, but the answer is simple. > We store included columns unorde

Re: [HACKERS] raised checkpoint limit & manual checkpoint

2016-09-24 Thread Amit Kapila
On Sat, Sep 24, 2016 at 12:12 PM, Fabien COELHO wrote: > > Hello, > > The checkpoint time limit has just been raised to one day after a discussion > started by Andres Freund: > > https://www.postgresql.org/message-id/20160202001320.GP8743%40awork2.anarazel.de > > I would have gone further up, say

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2016-09-24 Thread Peter Geoghegan
On Wed, Sep 21, 2016 at 5:52 PM, Heikki Linnakangas wrote: > I find this unification business really complicated. I can certainly understand why you would. As I said, it's the most complicated part of the patch, which overall is one of the most ambitious patches I've ever written. > I think it'd

Re: [HACKERS] Possibly too stringent Assert() in b-tree code

2016-09-24 Thread Amit Kapila
On Fri, Sep 23, 2016 at 12:21 AM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Sep 19, 2016 at 7:07 AM, Amit Kapila wrote: >>> I think you have a valid point. It seems we don't need to write WAL >>> for reuse page (aka don't call _bt_log_reuse_page()), if the page is >>> new, as the only pu

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2016-09-24 Thread Peter Geoghegan
On Thu, Sep 22, 2016 at 8:57 PM, Robert Haas wrote: > On Thu, Sep 22, 2016 at 3:51 PM, Heikki Linnakangas wrote: >> It'd be good if you could overlap the final merges in the workers with the >> merge in the leader. ISTM it would be quite straightforward to replace the >> final tape of each worker

Re: [HACKERS] raised checkpoint limit & manual checkpoint

2016-09-24 Thread Fabien COELHO
I would suggest that a good complementary feature would be to allow a manual checkpoint to run over a period of time, say something like: CHECKPOINT OVER '10 hours'; That would target to complete after this period (whether it succeeds or not is another issue) instead of going as fast as poss

Re: [HACKERS] store narrow values in hash indexes?

2016-09-24 Thread Bruce Momjian
On Sat, Sep 24, 2016 at 10:33:01AM +0530, Amit Kapila wrote: > On Sat, Sep 24, 2016 at 1:02 AM, Robert Haas wrote: > > Currently, hash indexes always store the hash code in the index, but > > not the actual Datum. It's recently been noted that this can make a > > hash index smaller than the corre

[HACKERS] Rebranding OS X as macOS

2016-09-24 Thread Tom Lane
Apple has decided to rename Mac OS X to "macOS", and apparently is now retroactively referring to old releases that way too. I propose that we should do likewise, ie run around and clean up our various references to "Mac OS X", "OS X", or "OSX" to uniformly say "macOS". I think it's considerably

Re: [HACKERS] Hash Indexes

2016-09-24 Thread Greg Stark
On Thu, Sep 22, 2016 at 3:23 AM, Tom Lane wrote: > But to kick the hash AM as such to the curb is to say > "sorry, there will never be O(1) index lookups in Postgres". Well there's plenty of halfway solutions for that. We could move hash indexes to contrib or even have them in core as experimenta

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-09-24 Thread Tomas Vondra
On 09/24/2016 06:06 AM, Amit Kapila wrote: On Fri, Sep 23, 2016 at 8:22 PM, Tomas Vondra wrote: ... >> So I'm using 16GB shared buffers (so with scale 300 everything fits into shared buffers), min_wal_size=16GB, max_wal_size=128GB, checkpoint timeout 1h etc. So no, there are no checkpoints du

Re: [HACKERS] Hash Indexes

2016-09-24 Thread Tom Lane
Greg Stark writes: > On Thu, Sep 22, 2016 at 3:23 AM, Tom Lane wrote: >> But to kick the hash AM as such to the curb is to say >> "sorry, there will never be O(1) index lookups in Postgres". > Well there's plenty of halfway solutions for that. We could move hash > indexes to contrib or even have

Re: [HACKERS] Better tracking of free space during SP-GiST index build

2016-09-24 Thread Tomas Vondra
On 09/22/2016 07:37 PM, Tom Lane wrote: Tomas Vondra writes: ... I've tried increasing the cache size to 768 entries, with vast majority of them (~600) allocated to leaf pages. Sadly, this seems to only increase the CREATE INDEX duration a bit, without making the index significantly smaller (s

Re: [HACKERS] Rebranding OS X as macOS

2016-09-24 Thread Michael Paquier
On Sun, Sep 25, 2016 at 1:47 AM, Tom Lane wrote: > Apple has decided to rename Mac OS X to "macOS", and apparently is now > retroactively referring to old releases that way too. I propose that > we should do likewise, ie run around and clean up our various references > to "Mac OS X", "OS X", or "

[HACKERS] Development build with uuid-ossp support - macOS

2016-09-24 Thread Enrique M
I am trying to do a macOS build of postgresql (9.6 stable branch from github) with the uuid-ossp contrib by typing "make world" but it fails due to an openjade error (I did install openjade using homebrew using this setup https://github.com/petere/homebrew-sgml). Is there a way to build postgresql

Re: [HACKERS] ICU integration

2016-09-24 Thread Thomas Munro
On Sat, Sep 24, 2016 at 10:13 PM, Peter Geoghegan wrote: > On Fri, Sep 23, 2016 at 7:27 AM, Thomas Munro > wrote: >> It looks like varstr_abbrev_convert calls strxfrm unconditionally >> (assuming TRUST_STRXFRM is defined). This needs to >> use ucol_getSortKey instead when appropriate. It looks

Re: [HACKERS] Hash Indexes

2016-09-24 Thread Amit Kapila
On Sat, Sep 24, 2016 at 10:49 PM, Greg Stark wrote: > On Thu, Sep 22, 2016 at 3:23 AM, Tom Lane wrote: >> But to kick the hash AM as such to the curb is to say >> "sorry, there will never be O(1) index lookups in Postgres". > > Well there's plenty of halfway solutions for that. We could move hash

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-09-24 Thread Amit Kapila
On Sun, Sep 25, 2016 at 10:30 AM, Amit Kapila wrote: > On Fri, Sep 23, 2016 at 5:34 PM, Amit Kapila wrote: >> >> I think here I am slightly wrong. For the full page writes, it do use >> RBM_ZERO_AND_LOCK mode to read the page and for such mode we are not >> doing page verification check and rath

Re: [HACKERS] Tracking wait event for latches

2016-09-24 Thread Michael Paquier
On Thu, Sep 22, 2016 at 10:49 PM, Robert Haas wrote: > Finally, extensions got their own category in this taxonomy, though I > wonder if it would be better to instead have > Activity/ExtensionActivity, Client/ExtensionClient, > Timeout/ExtensionTimeout, and IPC/ExtensionIPC instead of making it a

Re: [HACKERS] Hash Indexes

2016-09-24 Thread Mark Kirkwood
On 25/09/16 18:18, Amit Kapila wrote: On Sat, Sep 24, 2016 at 10:49 PM, Greg Stark wrote: On Thu, Sep 22, 2016 at 3:23 AM, Tom Lane wrote: But to kick the hash AM as such to the curb is to say "sorry, there will never be O(1) index lookups in Postgres". Well there's plenty of halfway solut