Re: [HACKERS] store narrow values in hash indexes?

On Sat, Sep 24, 2016 at 1:02 AM, Robert Haas wrote: > Currently, hash indexes always store the hash code in the index, but > not the actual Datum. It's recently been noted that this can make a > hash index smaller than the corresponding btree index would be if the > column

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Fri, Sep 23, 2016 at 8:22 PM, Tomas Vondra wrote: > On 09/23/2016 03:07 PM, Amit Kapila wrote: >> >> On Fri, Sep 23, 2016 at 6:16 PM, Tomas Vondra >> wrote: >>> >>> On 09/23/2016 01:44 AM, Tomas Vondra wrote: ...

[HACKERS] Complete LOCK TABLE ... IN ACCESS|ROW|SHARE

Hi After LOCK TABLE ... IN ACCESS|ROW|SHARE we run out of completions. Here's a patch to improve that, for November. -- Thomas Munro http://www.enterprisedb.com complete-lock-table.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared m

On Fri, Sep 23, 2016 at 7:41 PM, Tom Lane wrote: > Amit Kapila writes: >> Now, it is quite possible >> that error code is set to 0 on success in my windows environment >> (Win7) and doesn't work in some other environment. In any case, if we >> want

[HACKERS] Refactor StartupXLOG?

Hi hackers StartupXLOG is 1549 lines of code. Its unwieldy size came up in a discussion in an IRC channel where some PG hackers lurk and I decided to see how it might be chopped up into subroutines with a clear purpose and reasonable size, as an exercise. I came up with the following on a first

Re: [HACKERS] Logical Replication WIP

On 21/09/16 15:04, Peter Eisentraut wrote: > Some partial notes on 0005-Add-logical-replication-workers.patch: > > Document to what extent other relation types are supported (e.g., > materialized views as source, view or foreign table or temp table as > target). Suggest an updatable view as

Re: [HACKERS] asynchronous execution

[ Adjusting subject line to reflect the actual topic of discussion better. ] On Fri, Sep 23, 2016 at 9:29 AM, Robert Haas wrote: > On Fri, Sep 23, 2016 at 8:45 AM, Amit Khandekar > wrote: >> For e.g., in the above plan which you specified, suppose

Re: [HACKERS] tbm_lossify causes "unbalanced" hashtables / bitmaps

On 2016-09-23 17:40:13 -0400, Tom Lane wrote: > My idea of an appropriate fix would be to resume the scan at the same > point where the last scan stopped, and work circularly around the table > when necessary. I've played with that idea, and it does help a great deal. Not that surprisingly, it's

Re: [HACKERS] tbm_lossify causes "unbalanced" hashtables / bitmaps

Andres Freund writes: > Because iteration (both in my implementation and dynahash) is > essentially in bucket order, the front of the hashtable will be mostly > empty, whereas later parts of the hashtable will be full (i.e. a lot of > collisions). The reason for that is that

Re: [HACKERS] tbm_lossify causes "unbalanced" hashtables / bitmaps

On 2016-09-23 13:58:43 -0700, Andres Freund wrote: > static void > tbm_lossify(TIDBitmap *tbm) > { > ... > pagetable_start_iterate_random(tbm->pagetable, ); Uh, obviously that's from one of my attempts to address the problem. Greetings, Andres Freund -- Sent via pgsql-hackers mailing

[HACKERS] tbm_lossify causes "unbalanced" hashtables / bitmaps

Hi, Playing around with my hashtable implementation [1] and using it for bitmapscans/tidbitmap.c I noticed something curious. When using small work_mem (4MB in my case) for large-ish tables (~5GB), the performance tanks. That's primarily caused by some issues in the hashtable code (since fixed),

Re: [HACKERS] PG 9.6.0 release schedule

On 09/23/2016 01:31 PM, Tom Lane wrote: > Apologies for the late notice on this, but the release team has concluded > that 9.6 is about as ready as it's going to get. We are planning to go > ahead with 9.6.0 release next week --- that is, wrap Monday 26th for > public announcement on Thursday

[HACKERS] PG 9.6.0 release schedule

Apologies for the late notice on this, but the release team has concluded that 9.6 is about as ready as it's going to get. We are planning to go ahead with 9.6.0 release next week --- that is, wrap Monday 26th for public announcement on Thursday 29th. Thanks to the Release Management Team

Re: [HACKERS] patch: function xmltable

Hi 2016-09-23 10:07 GMT+02:00 Craig Ringer : > > Did some docs copy-editing and integrated some examples. > > Whoops, forgot to attach. > > Rather than sending a whole new copy of the patch, here's a diff > against your patched tree of my changes so you can see what I've

Re: [HACKERS] Hash Indexes

On 2016-09-23 15:19:14 -0400, Robert Haas wrote: > On Wed, Sep 21, 2016 at 10:33 PM, Andres Freund wrote: > > On 2016-09-21 22:23:27 -0400, Tom Lane wrote: > >> Andres Freund writes: > >> > Sure. But that can be addressed, with a lot less effort than

Re: [HACKERS] store narrow values in hash indexes?

Robert Haas writes: > Another thought is that hash codes are 32 bits, but a Datum is 64 bits > wide on most current platforms. So we're wasting 4 bytes per index > tuple storing nothing. Datum is not a concept that exists on-disk. What's stored is the 32-bit hash value.

Re: [HACKERS] 9.6 TAP tests and extensions

Craig Ringer writes: > On 23 September 2016 at 00:32, Tom Lane wrote: >> Certainly there are restrictions, but I'd imagine that every new release >> will be adding features to the TAP test infrastructure for some time to >> come. I think it's silly to

[HACKERS] store narrow values in hash indexes?

Currently, hash indexes always store the hash code in the index, but not the actual Datum. It's recently been noted that this can make a hash index smaller than the corresponding btree index would be if the column is wide. However, if the index is being built on a fixed-width column with a

Re: [HACKERS] PL/Python adding support for multi-dimensional arrays

On 9/23/16 2:42 AM, Heikki Linnakangas wrote: How do we handle single-dimensional arrays of composite types at the moment? At a quick glance, it seems that the composite types are just treated like strings, when they're in an array. That's probably OK, but it means that there's nothing special

Re: [HACKERS] Hash Indexes

On Wed, Sep 21, 2016 at 10:33 PM, Andres Freund wrote: > On 2016-09-21 22:23:27 -0400, Tom Lane wrote: >> Andres Freund writes: >> > Sure. But that can be addressed, with a lot less effort than fixing and >> > maintaining the hash indexes, by adding the

Re: [HACKERS] Rename max_parallel_degree?

On Fri, Sep 23, 2016 at 8:54 AM, Peter Eisentraut wrote: > On 9/20/16 4:07 PM, Robert Haas wrote: >> No, I'm assuming that the classes would be built-in. A string tag >> seems like over-engineering to me, particularly because the postmaster >> needs to switch on

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

On 9/12/16 2:50 PM, Peter Eisentraut wrote: > The comments on excludeDirContents are somewhat imprecise. For > example, none of those directories are actually removed or recreated > on startup, only the contents are. Fixed. > The comment for pg_replslot is incorrect. I think you can copy >

Re: [HACKERS] pg_basebackup creates a corrupt file for pg_stat_tmp and pg_replslot on a backup location

On 9/23/16 2:12 PM, Peter Eisentraut wrote: > On 4/26/16 5:02 AM, Ashutosh Sharma wrote: >> Knowing that pg_basebackup always creates an empty directory for >> pg_stat_tmp and pg_replslot in backup location, even i think it would be >> better to handle these directories in such a way that

Re: [HACKERS] pg_basebackup creates a corrupt file for pg_stat_tmp and pg_replslot on a backup location

On 4/26/16 5:02 AM, Ashutosh Sharma wrote: > Knowing that pg_basebackup always creates an empty directory for > pg_stat_tmp and pg_replslot in backup location, even i think it would be > better to handle these directories in such a way that pg_basebackup > generates an empty directory for

Re: [HACKERS] sequences and pg_upgrade

Here is an updated patch set. Compared to the initial set, I have changed pg_dump's sorting priorities so that sequence data is always after table data. This would otherwise have introduced a problem because sortDataAndIndexObjectsBySize() only considers consecutive DO_TABLE_DATA entries. Also,

Re: [HACKERS] pg_upgrade vs user created range type extension

On 09/23/2016 11:55 AM, Tom Lane wrote: Andrew Dunstan writes: On 09/22/2016 07:33 PM, Tom Lane wrote: If that diagnosis is correct, we should either change pg_dump to not dump that function, or change CREATE TYPE AS RANGE to not auto-create the constructor functions in

Re: [HACKERS] pg_upgrade vs user created range type extension

Andrew Dunstan writes: > On 09/22/2016 07:33 PM, Tom Lane wrote: >> If that diagnosis is correct, we should either change pg_dump to not >> dump that function, or change CREATE TYPE AS RANGE to not auto-create >> the constructor functions in binary-upgrade mode. The latter

Re: [HACKERS] pg_basebackup, pg_receivexlog and data durability (was: silent data loss with ext4 / all current versions)

On Fri, Sep 23, 2016 at 10:54 AM, Peter Eisentraut wrote: > This is looking pretty good. More comments on this patch set: Thanks for the review. > 0001: > > Keep the file order alphabetical in Mkvcbuild.pm. > > Needs nls.mk updates for file move (in initdb and

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On 09/23/2016 02:59 PM, Pavan Deolasee wrote: On Fri, Sep 23, 2016 at 6:05 PM, Tomas Vondra > wrote: On 09/23/2016 05:10 AM, Amit Kapila wrote: On Fri, Sep 23, 2016 at 5:14 AM, Tomas Vondra

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On 09/23/2016 03:07 PM, Amit Kapila wrote: On Fri, Sep 23, 2016 at 6:16 PM, Tomas Vondra wrote: On 09/23/2016 01:44 AM, Tomas Vondra wrote: ... The 4.5 kernel clearly changed the results significantly: ... (c) Although it's not visible in the results, 4.5.5

Re: [HACKERS] [PATCH] Remove redundant if clause in standbydesc.c

Michael Paquier writes: > On Fri, Sep 23, 2016 at 6:54 PM, Aleksander Alekseev > wrote: >>> Very simple small patch - see attachment. >>> >>> /* not expected, but print something anyway */ >>> else if (msg->id ==

Re: [HACKERS] Phrase search distance syntax

Teodor Sigaev writes: >> Why does the phrase distance operator assume <1> means adjacent words, >> and not <0>. (FYI, <-> is the same as <1>.) > Because > 1 it is a result of subtruction of word's positions > 2 <0> could be used as special case like a word with two

Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segm

Amit Kapila writes: > Attached patch tightens the error handling. However, on debugging, I > found that CreateFileMapping() always set error code to 0 on success. Ah, interesting. > Now, it is quite possible > that error code is set to 0 on success in my windows

Re: [HACKERS] Phrase search distance syntax

On Fri, Sep 23, 2016 at 05:07:26PM +0300, Teodor Sigaev wrote: > >Sorry to be asking another phrase search syntax question, and so close > >to final release, but ... > Really close... > > > >Why does the phrase distance operator assume <1> means adjacent words, > >and not <0>. (FYI, <-> is the

Re: [HACKERS] Phrase search distance syntax

Sorry to be asking another phrase search syntax question, and so close to final release, but ... Really close... Why does the phrase distance operator assume <1> means adjacent words, and not <0>. (FYI, <-> is the same as <1>.) Because 1 it is a result of subtruction of word's positions 2

Re: [HACKERS] [PATCH] Remove redundant if clause in standbydesc.c

On Fri, Sep 23, 2016 at 6:54 PM, Aleksander Alekseev wrote: >> > Very simple small patch - see attachment. >> >> /* not expected, but print something anyway */ >> else if (msg->id == SHAREDINVALRELMAP_ID) >> -appendStringInfoString(buf, "

Re: [HACKERS] Tracking wait event for latches

On Fri, Sep 23, 2016 at 10:32 AM, Robert Haas wrote: > On Thu, Sep 22, 2016 at 7:10 PM, Thomas Munro > wrote: >> I was thinking about suggesting a category "Replication" to cover the >> waits for client IO relating to replication, as opposed

Re: [HACKERS] asynchronous and vectorized execution

On Fri, Sep 23, 2016 at 8:45 AM, Amit Khandekar wrote: > For e.g., in the above plan which you specified, suppose : > 1. Hash Join has called ExecProcNode() for the child foreign scan b, and so > is > waiting in ExecAsyncWaitForNode(foreign_scan_on_b). > 2. The event wait

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Fri, Sep 23, 2016 at 6:50 AM, Robert Haas wrote: > On Thu, Sep 22, 2016 at 7:44 PM, Tomas Vondra > wrote: >> I don't dare to suggest rejecting the patch, but I don't see how we could >> commit any of the patches at this point. So perhaps

[HACKERS] Phrase search distance syntax

Sorry to be asking another phrase search syntax question, and so close to final release, but ... Why does the phrase distance operator assume <1> means adjacent words, and not <0>. (FYI, <-> is the same as <1>.) For example: select to_tsvector('park a a house') @@ to_tsquery('park <3>

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Fri, Sep 23, 2016 at 6:29 PM, Pavan Deolasee wrote: > On Fri, Sep 23, 2016 at 6:05 PM, Tomas Vondra > wrote: >> >> On 09/23/2016 05:10 AM, Amit Kapila wrote: >>> >>> On Fri, Sep 23, 2016 at 5:14 AM, Tomas Vondra >>>

Re: [HACKERS] pg_ctl promote wait

On 9/22/16 11:16 AM, Masahiko Sawada wrote: > Commit e7010ce seems to forget to change help message of pg_ctl. > Attached patch. Fixed, thanks. I also added the -t option and reformatted a bit. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support,

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Fri, Sep 23, 2016 at 6:16 PM, Tomas Vondra wrote: > On 09/23/2016 01:44 AM, Tomas Vondra wrote: >> >> ... >> The 4.5 kernel clearly changed the results significantly: >> > ... >> >> >> (c) Although it's not visible in the results, 4.5.5 almost perfectly >>

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Fri, Sep 23, 2016 at 6:05 PM, Tomas Vondra wrote: > On 09/23/2016 05:10 AM, Amit Kapila wrote: > >> On Fri, Sep 23, 2016 at 5:14 AM, Tomas Vondra >> wrote: >> >>> On 09/21/2016 08:04 AM, Amit Kapila wrote: >>> >>> (c)

Re: [HACKERS] Rename max_parallel_degree?

On 9/20/16 4:07 PM, Robert Haas wrote: > No, I'm assuming that the classes would be built-in. A string tag > seems like over-engineering to me, particularly because the postmaster > needs to switch on the tag, and we need to be very careful about the > degree to which the postmaster trusts the

Re: [HACKERS] pageinspect: Hash index support

On Fri, Sep 23, 2016 at 6:11 PM, Peter Eisentraut wrote: > On 9/23/16 1:56 AM, Amit Kapila wrote: >> which comment are you referring here? hashm_mapp contains block >> numbers of bitmap pages. > > The comment I'm referring to says > > The blknos of these

Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.85140

Amit Kapila writes: > On Fri, Sep 23, 2016 at 1:21 AM, Tom Lane wrote: >> Robert Haas writes: >>> So, we could have dsm_postmaster_startup() seed the random number >>> generator itself, and then let PostmasterRandom() override

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On 09/23/2016 01:44 AM, Tomas Vondra wrote: ... The 4.5 kernel clearly changed the results significantly: ... > (c) Although it's not visible in the results, 4.5.5 almost perfectly eliminated the fluctuations in the results. For example when 3.2.80 produced this results (10 runs with the same

Re: [HACKERS] asynchronous and vectorized execution

On 13 September 2016 at 20:20, Robert Haas wrote: > On Mon, Aug 29, 2016 at 4:08 AM, Kyotaro HORIGUCHI > wrote: > > [ new patches ] > > +/* > + * We assume that few nodes are async-aware and async-unaware > +

Re: [HACKERS] pageinspect: Hash index support

On 9/23/16 1:56 AM, Amit Kapila wrote: > On Fri, Sep 23, 2016 at 9:40 AM, Peter Eisentraut >> - hash_metap result fields spares and mapp should be arrays of integer. >> > > how would you like to see both those arrays in tuple, right now, I > think Jesper's code is showing it as string. I'm not

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On 09/23/2016 05:10 AM, Amit Kapila wrote: On Fri, Sep 23, 2016 at 5:14 AM, Tomas Vondra wrote: On 09/21/2016 08:04 AM, Amit Kapila wrote: (c) Although it's not visible in the results, 4.5.5 almost perfectly eliminated the fluctuations in the results. For

Re: [HACKERS] Write Ahead Logging for Hash Indexes

On Thu, Sep 22, 2016 at 10:16 AM, Amit Kapila wrote: > On Thu, Sep 22, 2016 at 8:51 AM, Jeff Janes wrote: >> >> >> Correct. But any torn page write must be covered by the restoration of a >> full page image during replay, shouldn't it? And that

Re: [HACKERS] MSVC pl-perl error message is not verbose enough

On 08/02/2016 08:18 PM, John Harvey wrote: On Mon, Aug 1, 2016 at 9:39 PM, Michael Paquier wrote: On Tue, Aug 2, 2016 at 2:08 AM, Robert Haas wrote: Did you add this to the next CommitFest? I have added it here:

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment

On Thu, Sep 22, 2016 at 10:40 PM, Tom Lane wrote: > Amit Kapila writes: >> On Tue, Sep 20, 2016 at 10:33 PM, Tom Lane wrote: >>> ISTM both the previous coding and this version can fail for no good >>> reason, that is what if

Re: [HACKERS] [PATCH] Remove redundant if clause in standbydesc.c

> > Very simple small patch - see attachment. > > /* not expected, but print something anyway */ > else if (msg->id == SHAREDINVALRELMAP_ID) > -appendStringInfoString(buf, " relmap"); > -else if (msg->id == SHAREDINVALRELMAP_ID) >

Re: [HACKERS] [PATCH] Remove redundant if clause in standbydesc.c

On Fri, Sep 23, 2016 at 6:08 PM, Aleksander Alekseev wrote: > Very simple small patch - see attachment. /* not expected, but print something anyway */ else if (msg->id == SHAREDINVALRELMAP_ID) -appendStringInfoString(buf, " relmap"); -

[HACKERS] [PATCH] Remove redundant if clause in standbydesc.c

Hello. Very simple small patch - see attachment. -- Best regards, Aleksander Alekseev diff --git a/src/backend/access/rmgrdesc/standbydesc.c b/src/backend/access/rmgrdesc/standbydesc.c index 13797a3..77983e5 100644 --- a/src/backend/access/rmgrdesc/standbydesc.c +++

Re: [HACKERS] patch: function xmltable

2016-09-23 10:05 GMT+02:00 Craig Ringer : > On 22 September 2016 at 02:31, Pavel Stehule > wrote: > > > another small update - fix XMLPath parser - support multibytes characters > > I'm returning for another round of review. > > The code doesn't

Re: [HACKERS] patch: function xmltable

On 22 September 2016 at 02:31, Pavel Stehule wrote: > another small update - fix XMLPath parser - support multibytes characters I'm returning for another round of review. The code doesn't handle the 5 XML built-in entities correctly in text-typed output. It processes

Re: [HACKERS] PL/Python adding support for multi-dimensional arrays

On 09/22/2016 10:28 AM, Pavel Stehule wrote: Now, the tests are enough - so I'll mark this patch as ready for commiters. I had to fix tests - there was lot of white spaces, and the result for python3 was missing Thanks Pavel! This crashes with arrays with non-default lower bounds:

Re: [HACKERS] ICU integration

On Wed, Aug 31, 2016 at 2:46 PM, Peter Eisentraut wrote: > Here is a patch I've been working on to allow the use of ICU for sorting > and other locale things. This is very interesting work, and it's great to see some development in this area. I've been

Re: [HACKERS] Showing parallel status in \df+

2016-09-23 7:22 GMT+02:00 Rushabh Lathia : > > > On Thu, Sep 22, 2016 at 10:04 PM, Tom Lane wrote: > >> Rushabh Lathia writes: >> > I agree with the argument in this thread, having "Source code" as part >> > of \df+ is bit

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618

On Fri, Sep 23, 2016 at 1:21 AM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Sep 20, 2016 at 12:53 PM, Tom Lane wrote: >>> I'd be the first to agree that this point is inadequately documented >>> in the code, but

Re: [HACKERS] File system operations.

On 22 September 2016 at 20:02, Yury Zhuravlev wrote: > On четверг, 15 сентября 2016 г. 19:09:16 MSK, Tom Lane wrote: >> >> Robert Haas writes: >>> >>> On Thu, Sep 15, 2016 at 11:01 AM, Anastasia Lubennikova >>>