Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

Robert Haas writes: > I agree that we should have some way for foreign data wrappers and > custom scans and perhaps also other executor nodes to find out whether > there's a known limit to the number of tuples that they might need to > produce, but I wonder if we should be

Re: [HACKERS] kqueue

On 13 September 2016 at 08:08, Heikki Linnakangas wrote: > So, if I've understood correctly, the purpose of this patch is to improve > performance on a multi-CPU system, which has the kqueue() function. Most > notably, FreeBSD? I'm getting a little fried from "self-documenting"

Re: [HACKERS] cost_sort() may need to be updated

On Tue, Sep 13, 2016 at 5:59 AM, Robert Haas wrote: > I think that the only real way to judge whether cost_sort() is any > good is to see whether it causes the wrong plan to be chosen in some > cases. For example, it could cause merge joins to be picked too > frequently or

Re: [HACKERS] kqueue

Hi, On 2016-09-13 16:08:39 +0300, Heikki Linnakangas wrote: > So, if I've understood correctly, the purpose of this patch is to improve > performance on a multi-CPU system, which has the kqueue() function. Most > notably, FreeBSD? I think it's not necessarily about the current system, but more

Re: [HACKERS] Push down more full joins in postgres_fdw

On Tue, Sep 6, 2016 at 9:07 AM, Ashutosh Bapat wrote: > That's not true with the alias information. As long as we detect which > relations need subqueries, their RTIs are enough to create unique aliases > e.g. if a relation involves RTIs 1, 2, 3 corresponding

Re: [HACKERS] kqueue

Andres Freund writes: > On 2016-09-13 16:08:39 +0300, Heikki Linnakangas wrote: >> So, if I've understood correctly, the purpose of this patch is to improve >> performance on a multi-CPU system, which has the kqueue() function. Most >> notably, FreeBSD? > I think it's not

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

On Tue, Sep 13, 2016 at 3:48 AM, Kouhei Kaigai wrote: > Sorry for my late. > > The attached patch fixed the wording problems on SGML part. I agree that we should have some way for foreign data wrappers and custom scans and perhaps also other executor nodes to find out

Re: [HACKERS] asynchronous and vectorized execution

On Tue, Aug 2, 2016 at 3:41 AM, Kyotaro HORIGUCHI wrote: > Thank you for the comment. > > At Mon, 1 Aug 2016 10:44:56 +0530, Amit Khandekar > wrote in >> On 21 July 2016

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

On September 13, 2016 9:07:35 AM PDT, Tom Lane wrote: >Andres Freund writes: >> Attached is a significantly updated patch series (see the mail one up >> for details about what this is, I don't want to quote it in its >> entirety). > >I've reviewed the

Re: [HACKERS] Hash Indexes

On Wed, Sep 7, 2016 at 9:32 PM, Amit Kapila wrote: > On Wed, Sep 7, 2016 at 11:49 PM, Jeff Janes wrote: > > On Thu, Sep 1, 2016 at 8:55 PM, Amit Kapila > wrote: > >> > >> > >> I have fixed all other issues you have raised.

Re: [HACKERS] kqueue

On Tue, Sep 13, 2016 at 11:36 AM, Simon Riggs wrote: > On 13 September 2016 at 08:08, Heikki Linnakangas wrote: >> So, if I've understood correctly, the purpose of this patch is to improve >> performance on a multi-CPU system, which has the kqueue()

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

Andres Freund writes: > Attached is a significantly updated patch series (see the mail one up > for details about what this is, I don't want to quote it in its > entirety). I've reviewed the portions of 0005 that have to do with making the parser mark queries with

Re: [HACKERS] kqueue

On 09/13/2016 04:33 PM, Tom Lane wrote: Heikki Linnakangas writes: So, if I've understood correctly, the purpose of this patch is to improve performance on a multi-CPU system, which has the kqueue() function. Most notably, FreeBSD? OS X also has this, so it might be worth

Re: [HACKERS] DISABLE_PAGE_SKIPPING option for vacuumdb

On Thu, Sep 8, 2016 at 1:32 PM, Masahiko Sawada wrote: > Attached patch add --disable-page-skipping option to vacuumed command for 9.6. > If by any chance the freeze map causes the serious data corruption bug > then the user will want to run pg_check_visible() and vacuum

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

On 2016-09-13 12:07:35 -0400, Tom Lane wrote: > diff --git a/src/backend/optimizer/plan/analindex e28a8dc..74e4245 100644 > --- a/src/backend/optimizer/plan/analyzejoins.c > +++ b/src/backend/optimizer/plan/analyzejoins.c > @@ -650,6 +650,11 @@ rel_is_distinct_for(PlannerInfo *root, R > bool >

[HACKERS] Inheriting PostgresNode object

Hi hackers! I've encountered need to extend functionality of PostgresNode class from the TAP test framework. What I want can easily be done via perl object inheritation. But documentation in the PostgresNode.pm recommends to use get_new_node function rather than call PostgresNode constructor

Re: [HACKERS] kqueue

Andres Freund writes: > On 2016-09-13 14:47:08 -0400, Tom Lane wrote: >> Also I notice that the WaitEventSet thread started with a simple >> pgbench test, so I don't really buy the claim that that's not a >> way that will reach the problem. > You can reach it, but not when

Re: [HACKERS] kqueue

Andres Freund writes: > On 2016-09-13 12:43:36 -0400, Tom Lane wrote: >> Also, if it's only a win on machines with dozens of CPUs, how many >> people are running *BSD on that kind of iron? I think Linux is by >> far the dominant kernel for such hardware. For sure Apple isn't

Re: [HACKERS] kqueue

On 2016-09-13 14:47:08 -0400, Tom Lane wrote: > Also I notice that the WaitEventSet thread started with a simple > pgbench test, so I don't really buy the claim that that's not a > way that will reach the problem. You can reach it, but not when using 1 core:one pgbench thread:one client

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Tue, Sep 13, 2016 at 3:51 PM, Robert Haas wrote: > On Fri, Sep 9, 2016 at 3:04 AM, Masahiko Sawada wrote: >> Attached PoC patch changes the representation of dead tuple locations >> to the hashmap having tuple bitmap. >> The one hashmap entry

Re: [HACKERS] Hash Indexes

On 09/13/2016 07:26 AM, Amit Kapila wrote: Attached, new version of patch which contains the fix for problem reported on write-ahead-log of hash index thread [1]. I have been testing patch in various scenarios, and it has a positive performance impact in some cases. This is especially seen

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Tue, Sep 13, 2016 at 4:06 PM, Robert Haas wrote: > On Tue, Sep 13, 2016 at 2:59 PM, Claudio Freire > wrote: >> I've finished writing that patch, I'm in the process of testing its CPU >> impact. >> >> First test seemed to hint at a 40% increase

Re: [HACKERS] Cache Hash Index meta page.

On Thu, Sep 8, 2016 at 11:21 PM, Jesper Pedersen wrote: > > > For the archives, this patch conflicts with the WAL patch [1]. > > > [1] https://www.postgresql.org/message-id/CAA4eK1JS%2BSiRSQBzEFp > nsSmxZKingrRH7WNyWULJeEJSj1-%3D0w%40mail.gmail.com > Updated the

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Fri, Sep 9, 2016 at 3:04 AM, Masahiko Sawada wrote: > Attached PoC patch changes the representation of dead tuple locations > to the hashmap having tuple bitmap. > The one hashmap entry consists of the block number and the TID bitmap > of corresponding block, and the

Re: [HACKERS] GiST penalty functions [PoC]

Hi hackers! Here is gist cube penaly patch v5. Changes: 1. union version of pack_float() is used, without warings and with minimum asm commands emitted 2. test for __STDC_IEC_559__ is added, this is a standard way of C99 to determine float compliance with IEEE754. ANSI-only compiler+libs, along

Re: [HACKERS] kqueue

On 2016-09-13 12:43:36 -0400, Tom Lane wrote: > > I think it's not necessarily about the current system, but more about > > future uses of the WaitEventSet stuff. Some of that is going to use a > > lot more sockets. E.g. doing a parallel append over FDWs. (note that I'm talking about network

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

Andres Freund writes: > On September 13, 2016 9:07:35 AM PDT, Tom Lane wrote: >> I'd like to go ahead and push this, since it's a necessary prerequisite >> for either approach we might adopt for the rest of the patch series, >> and the improved error

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

Andres Freund writes: > On 2016-09-13 12:07:35 -0400, Tom Lane wrote: >> /* >> - * Don't pull up a subquery that has any set-returning functions in its >> - * targetlist. Otherwise we might well wind up inserting set-returning >> - * functions into places where

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Tue, Sep 13, 2016 at 11:51 AM, Robert Haas wrote: > I think it's probably wrong to worry that an array-of-arrays is going > to be meaningfully slower than a single array here. It's basically > costing you some small number of additional memory references per > tuple,

Re: [HACKERS] Patch: Implement failover on libpq connect level.

On Fri, Sep 9, 2016 at 4:49 AM, Victor Wagner wrote: > Random permutation is much more computationally complex than random > picking. It really isn't. The pseudocode given at https://en.wikipedia.org/wiki/Fisher%E2%80%93Yates_shuffle is all of 4 lines long, and one of those

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Tue, Sep 13, 2016 at 2:59 PM, Claudio Freire wrote: > I've finished writing that patch, I'm in the process of testing its CPU > impact. > > First test seemed to hint at a 40% increase in CPU usage, which seems > rather steep compared to what I expected, so I'm trying

Re: [HACKERS] kqueue

On 2016-09-13 15:37:22 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2016-09-13 14:47:08 -0400, Tom Lane wrote: > >> Also I notice that the WaitEventSet thread started with a simple > >> pgbench test, so I don't really buy the claim that that's not a > >> way that will

Re: [HACKERS] Write Ahead Logging for Hash Indexes

On 09/13/2016 07:41 AM, Amit Kapila wrote: README: +in_complete split flag. The reader algorithm works correctly, as it will scan What flag ? in-complete-split flag which indicates that split has to be finished for that particular bucket. The value of these flags are

Re: [HACKERS] Logical Replication WIP

Hi, First read through the current version. Hence no real architectural comments. On 2016-09-09 00:59:26 +0200, Petr Jelinek wrote: > diff --git a/src/backend/commands/publicationcmds.c > b/src/backend/commands/publicationcmds.c > new file mode 100644 > index 000..e0c719d > --- /dev/null >

Re: [HACKERS] autonomous transactions

On Tue, Sep 13, 2016 at 5:06 PM, Petr Jelinek wrote: > I mostly agree. I think if this was called something like background > transactions it might be better. It's definitely useful functionality but > the naming is clearly contentious. It won't stop people using it for same

Re: [HACKERS] kqueue

Andres Freund writes: > On 2016-09-13 15:37:22 -0400, Tom Lane wrote: >> (It's a 4-core CPU so I saw little point in pressing harder than >> that.) > I think in reality most busy machines, were performance and scalability > matter, are overcommitted in the number of

Re: [HACKERS] autonomous transactions

On 13/09/16 22:24, Robert Haas wrote: On Wed, Sep 7, 2016 at 9:14 PM, Craig Ringer wrote: That's probably going to be one of the smaller costs. Doing this with bgworkers won't be cheap, but you need to consider the alternative. Factoring out all

Re: [HACKERS] An extra error for client disconnection on Windows

On Tue, Sep 13, 2016 at 10:00 PM, Alvaro Herrera wrote: > Yeah, I looked into this a few days ago and that was my conclusion also: > let's drop this. Okay. Just done so in the CF app. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] multivariate statistics (v19)

Hi, Thanks for looking into this! On 09/12/2016 04:08 PM, Dean Rasheed wrote: On 3 August 2016 at 02:58, Tomas Vondra wrote: Attached is v19 of the "multivariate stats" patch series Hi, I started looking at this - just at a very high level - I've not read

Re: [HACKERS] Inheriting PostgresNode object

Victor Wagner wrote: > Hi hackers! > > I've encountered need to extend functionality of PostgresNode class > from the TAP test framework. What I want can easily be done via perl > object inheritation. > > But documentation in the PostgresNode.pm recommends to use get_new_node > function rather

Re: [HACKERS] Inheriting PostgresNode object

Alvaro Herrera writes: > Yeah, as I recall the only thing the get_new_node thingy does is assign > a nonconflicting port number to each instance, and make sure the > instances are all teared down at END. I don't remember now why didn't > we just do the port check in the

Re: [HACKERS] kqueue

I wrote: > At -j 10 -c 10, all else the same, I get 84928 TPS on HEAD and 90357 > with the patch, so about 6% better. And at -j 1 -c 1, I get 22390 and 24040 TPS, or about 7% better with the patch. So what I am seeing on OS X isn't contention of any sort, but just a straight speedup that's

Re: [HACKERS] Logical Replication WIP

On 13/09/16 02:55, Craig Ringer wrote: On 13 September 2016 at 06:03, Petr Jelinek wrote: Oh sure, I don't see that as big problem, the TupleData already contains type of the data it sends (to distinguish between nulls and text data) so that's mostly about adding some

Re: [HACKERS] pg_basebackup wish list

On 9/12/16 3:31 PM, Tom Lane wrote: > Hm, there was just a kerfuffle about spelling things like this > "--no-clean" etc. I wasn't on board with removing existing spellings, > but surely all newly added switches should use the "no-" spelling? This is supposed to be parallel to initdb's option.

Re: [HACKERS] autonomous transactions

On Wed, Sep 7, 2016 at 9:14 PM, Craig Ringer wrote: > That's probably going to be one of the smaller costs. Doing this with > bgworkers won't be cheap, but you need to consider the alternative. > Factoring out all transaction-specific data currently stored in or >

Re: [HACKERS] [BUGS] BUG #14244: wrong suffix for pg_size_pretty()

On 7/30/16 2:16 PM, Bruce Momjian wrote: > The second patch does what Tom suggests above by outputting only "KB", > and it supports "kB" for backward compatibility. What it doesn't do is > to allow arbitrary case, which I think would be a step backward. The > second patch actually does match the

Re: [HACKERS] inappropriate use of NameGetDatum macro

I wrote: > Mark Dilger writes: >> there are several places in the code where variables defined as >> (char *) or as (const char *) are passed to the NameGetDatum() >> macro. I believe it would be better form to use CStringGetDatum() >> in these locations. I am aware

Re: [HACKERS] gettimeofday is at the end of its usefulness?

On Thu, Aug 25, 2016 at 3:12 PM, Haribabu Kommi wrote: > On Thu, Jun 9, 2016 at 12:56 AM, Tom Lane wrote: > > Thom Brown writes: > >> On 15 May 2014 at 19:56, Bruce Momjian wrote: > >>> On Tue, May 13, 2014 at

Re: [HACKERS] DISABLE_PAGE_SKIPPING option for vacuumdb

On Wed, Sep 14, 2016 at 2:17 AM, Robert Haas wrote: > On Thu, Sep 8, 2016 at 1:32 PM, Masahiko Sawada wrote: >> Attached patch add --disable-page-skipping option to vacuumed command for >> 9.6. >> If by any chance the freeze map causes the serious

Re: [HACKERS] Hash Indexes

On Tue, Sep 13, 2016 at 5:46 PM, Robert Haas wrote: > On Thu, Sep 8, 2016 at 12:32 AM, Amit Kapila wrote: >> Hmm. I think page or block is a concept of database systems and >> buckets is a general concept used in hashing technology. I think the

[HACKERS] Event trigger and CREATE/ALTER ROLE/USER

Simple question: Is there any reason for event trigger to not support CREATE/ALTER ROLE/USER? Best regards, -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] Event trigger and CREATE/ALTER ROLE/USER

On 14 Sep. 2016 9:44 am, "Alvaro Herrera" wrote: > > Tatsuo Ishii wrote: > > Simple question: Is there any reason for event trigger to not support > > CREATE/ALTER ROLE/USER? > > As I understand the issue, the main reason is that event triggers > execute procedures, and

Re: [HACKERS] pg_basebackup wish list

On Wed, Sep 14, 2016 at 6:52 AM, Peter Eisentraut wrote: > On 9/12/16 3:31 PM, Tom Lane wrote: >> Hm, there was just a kerfuffle about spelling things like this >> "--no-clean" etc. I wasn't on board with removing existing spellings, >> but surely all newly

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

> On Tue, Sep 13, 2016 at 3:48 AM, Kouhei Kaigai wrote: > > Sorry for my late. > > > > The attached patch fixed the wording problems on SGML part. > > I agree that we should have some way for foreign data wrappers and > custom scans and perhaps also other executor nodes to

Re: [HACKERS] condition variables

On Thu, Aug 11, 2016 at 2:47 PM, Robert Haas wrote: > Another approach to the problem is to use a latch wait loop. That > almost works. Interrupts can be serviced, and you can recheck shared > memory to see whether the condition for proceeding is satisfied after > each

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Wed, Sep 14, 2016 at 12:21 AM, Robert Haas wrote: > On Fri, Sep 9, 2016 at 3:04 AM, Masahiko Sawada > wrote: > > Attached PoC patch changes the representation of dead tuple locations > > to the hashmap having tuple bitmap. > > The one hashmap

Re: [HACKERS] Event trigger and CREATE/ALTER ROLE/USER

On Wed, Sep 14, 2016 at 9:26 AM, Tatsuo Ishii wrote: > Simple question: Is there any reason for event trigger to not support > CREATE/ALTER ROLE/USER? Because it performs a global administrative task, no? -- Michael -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Event trigger and CREATE/ALTER ROLE/USER

On Wed, Sep 14, 2016 at 9:34 AM, Tatsuo Ishii wrote: >>> Simple question: Is there any reason for event trigger to not support >>> CREATE/ALTER ROLE/USER? >> >> Because it performs a global administrative task, no? > > So is there any technical difficulty? I don't recall the

Re: [HACKERS] Event trigger and CREATE/ALTER ROLE/USER

Tatsuo Ishii wrote: > Simple question: Is there any reason for event trigger to not support > CREATE/ALTER ROLE/USER? As I understand the issue, the main reason is that event triggers execute procedures, and those exist in a single database only. If you were to create an event trigger in

Re: [HACKERS] Event trigger and CREATE/ALTER ROLE/USER

> As I understand the issue, the main reason is that event triggers > execute procedures, and those exist in a single database only. If you > were to create an event trigger in database A, then a user gets created > in database B, your function would not be invoked, which becomes a > problem.

[HACKERS] Comment typo in execIndexing.c

Spotted a typo in the header comment of execIndexing.c. Attached fixes it. s/exclusive constraints/exclusion constraints/g Thanks, Amit diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c index 0e2d834..009c1b7 100644 --- a/src/backend/executor/execIndexing.c

Re: [HACKERS] Push down more full joins in postgres_fdw

On Tue, Sep 13, 2016 at 10:28 PM, Robert Haas wrote: > On Tue, Sep 6, 2016 at 9:07 AM, Ashutosh Bapat > wrote: >> That's not true with the alias information. As long as we detect which >> relations need subqueries, their RTIs are enough to

Re: [HACKERS] Event trigger and CREATE/ALTER ROLE/USER

>> Simple question: Is there any reason for event trigger to not support >> CREATE/ALTER ROLE/USER? > > Because it performs a global administrative task, no? So is there any technical difficulty? Can you please elaborate? Best regards, -- Tatsuo Ishii SRA OSS, Inc. Japan English:

Re: [HACKERS] Event trigger and CREATE/ALTER ROLE/USER

Simple question: Is there any reason for event trigger to not support CREATE/ALTER ROLE/USER? >>> >>> Because it performs a global administrative task, no? >> >> So is there any technical difficulty? > > I don't recall the exact details but... > >> Can you please elaborate? > > Roles

Re: [HACKERS] WAL consistency check facility

On Tue, Sep 13, 2016 at 6:07 PM, Kuntal Ghosh wrote: >> For now, I've kept this as a WARNING message to detect all inconsistencies >> at once. Once, the patch is finalized, I'll modify it as an ERROR message. > > Or say FATAL. This way the server is taken down. What

Re: [HACKERS] kqueue

On Wed, Sep 14, 2016 at 12:06 AM, Tom Lane wrote: > I wrote: >> At -j 10 -c 10, all else the same, I get 84928 TPS on HEAD and 90357 >> with the patch, so about 6% better. > > And at -j 1 -c 1, I get 22390 and 24040 TPS, or about 7% better with > the patch. So what I am

[HACKERS] palloc() too large on pg_buffercache with large shared_buffers

Hello, It looks to me pg_buffercache tries to allocate more than 1GB using palloc(), when shared_buffers is more than 256GB. # show shared_buffers ; shared_buffers 280GB (1 row) # SELECT buffers, d.datname, coalesce(c.relname, '???') FROM (SELECT count(*) buffers,

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Mon, Sep 5, 2016 at 9:33 AM, Amit Kapila wrote: > USE_CONTENT_LOCK on my windows box, you can try by commenting that as > well, if it works for you). So, in short we have to compare three > approaches here. > > 1) Group mode to reduce CLOGControlLock contention > 2)

Re: [HACKERS] WAL consistency check facility

On Wed, Sep 14, 2016 at 6:34 AM, Michael Paquier wrote: >>>2) Regarding page comparison: >>>We could just use memcpy() here. compareImages was useful to get a >>>clear image of what the inconsistencies were, but you don't do that >>>anymore. >> memcmp(), right? > >Yep

Re: [HACKERS] kqueue

On Wed, Sep 14, 2016 at 7:06 AM, Tom Lane wrote: > It would be good for someone else to reproduce my results though. > For one thing, 5%-ish is not that far above the noise level; maybe > what I'm measuring here is just good luck from relocation of critical > loops into more

Re: [HACKERS] _hash_alloc_buckets() safety

On Tue, Sep 13, 2016 at 6:58 PM, Tom Lane wrote: > Amit Kapila writes: >> While working on write-ahead-logging of hash indexes, I noticed that >> this function allocates buckets in batches and the mechanism it uses >> is that it initialize the last

Re: [HACKERS] asynchronous and vectorized execution

On Mon, Aug 29, 2016 at 4:08 AM, Kyotaro HORIGUCHI wrote: > [ new patches ] +/* + * We assume that few nodes are async-aware and async-unaware + * nodes cannot be revserse-dispatched from lower nodes that is + *

Re: [HACKERS] WAL consistency check facility

- If WAL consistency check is enabled for a rmgrID, we always include the backup image in the WAL record. >>> >>>What happens if wal_consistency has different settings on a standby >>>and its master? If for example it is set to 'all' on the standby, and >>>'none' on the master, or

[HACKERS] Nested loop join condition does not get pushed down to foreign scan

I just noticed something surprising: -- create a larger local table CREATE TABLE llarge (id integer NOT NULL, val integer NOT NULL); INSERT INTO llarge SELECT i, i%100 FROM generate_series(1, 1) i; ALTER TABLE llarge ADD PRIMARY KEY (id); -- create a small local table CREATE TABLE small (id

Re: [HACKERS] Nested loop join condition does not get pushed down to foreign scan

On Tue, Sep 13, 2016 at 4:05 PM, Albe Laurenz wrote: > I just noticed something surprising: > > -- create a larger local table > CREATE TABLE llarge (id integer NOT NULL, val integer NOT NULL); > INSERT INTO llarge SELECT i, i%100 FROM generate_series(1, 1) i; > ALTER

Re: [HACKERS] PoC: Partial sort

On Fri, Apr 8, 2016 at 10:09 PM, Peter Geoghegan wrote: > On Wed, Mar 30, 2016 at 8:02 AM, Alexander Korotkov > wrote: > > Hmm... I'm not completely agree with that. In typical usage partial sort > > should definitely use quicksort. However, fallback to

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Amit, [...] There still seems to be a change in behavior of the -r option due to the patch. Consider the following example: select a from a where a = 1 \; select a+1 from a where a = 1; ... - statement latencies in milliseconds: 2.889 select a from a where a = 1 ; vs select

Re: [HACKERS] 9.6 TAP tests and extensions

Craig Ringer wrote: > I suggest that the above patches be applied to 9.6 and v10. Then for > v10 I don't object to patching 9.6 in this way, but kindly do not pollute this thread with future ideas on what to do on pg10, at least until the current release is sorted out. You'll only distract

Re: [HACKERS] Hash Indexes

Attached, new version of patch which contains the fix for problem reported on write-ahead-log of hash index thread [1]. [1] - https://www.postgresql.org/message-id/CAA4eK1JuKt%3D-%3DY0FheiFL-i8Z5_5660%3D3n8JUA8s3zG53t_ArQ%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB:

Re: [HACKERS] Hash Indexes

On 09/12/2016 10:42 PM, Amit Kapila wrote: The following script hangs on idx_val creation - just with v5, WAL patch not applied. Are you sure it is actually hanging? I see 100% cpu for a few minutes but the index eventually completes ok for me (v5 patch applied to today's master). It

Re: [HACKERS] Write Ahead Logging for Hash Indexes

On Mon, Sep 12, 2016 at 11:29 AM, Jeff Janes wrote: > > > My test program (as posted) injects crashes and then checks the > post-crash-recovery system for consistency, so it cannot be run as-is > without the WAL patch. I also ran the test with crashing turned off (just >

Re: [HACKERS] Hash Indexes

On Thu, Sep 8, 2016 at 12:32 AM, Amit Kapila wrote: > Hmm. I think page or block is a concept of database systems and > buckets is a general concept used in hashing technology. I think the > difference is that there are primary buckets and overflow buckets. I > have

[HACKERS] _hash_alloc_buckets() safety

While working on write-ahead-logging of hash indexes, I noticed that this function allocates buckets in batches and the mechanism it uses is that it initialize the last page of batch with zeros and expect that the filesystem will ensure the intervening pages read as zeroes too. I think to make it

Re: [HACKERS] WAL consistency check facility

On Mon, Sep 12, 2016 at 5:06 AM, Kuntal Ghosh wrote: >>+ void(*rm_checkConsistency) (XLogReaderState *record); >>All your _checkConsistency functions share the same pattern, in short >>they all use a for loop for each block, call each time

Re: [HACKERS] WAL consistency check facility

On Sun, Sep 11, 2016 at 12:03 AM, Robert Haas wrote: > It seems entirely unnecessary for the master and the standby to agree > here. I think what we need is two GUCs. One of them, which affects > only the master, controls whether the validation information is > including

Re: [HACKERS] 9.6 TAP tests and extensions

On 13 September 2016 at 13:27, Craig Ringer wrote: > This was wrong for out-of-tree builds, updated. > > Still pending fix for PG_REGRESS path when invoked using > $(prove_check) from PGXS Looking further at this, I think a pgxs-specific patch to add support for prove

Re: [HACKERS] 9.6 TAP tests and extensions

Craig Ringer writes: > While updating an extension for 9.6 I noticed that while the > $(prove_check) definition is exposed for use by PGXS in > Makefile.global, extensions can't actually use the TAP tests because > we don't install the required Perl modules like

Re: [HACKERS] Supporting SJIS as a database encoding

On 09/08/2016 09:35 AM, Kyotaro HORIGUCHI wrote: Returning in UTF-8 bloats the result string by about 1.5 times so it doesn't seem to make sense comparing with it. But it takes real = 47.35s. Nice! I was hoping that this would also make the binaries smaller. A few dozen kB of storage is

Re: [HACKERS] cost_sort() may need to be updated

On Sun, Sep 11, 2016 at 12:13 PM, Peter Geoghegan wrote: > On Sun, Sep 11, 2016 at 9:01 AM, Tom Lane wrote: >> Peter Geoghegan writes: >>> I think that we *can* refine this guess, and should, because random >>> I/O is really quite unlikely

Re: [HACKERS] An extra error for client disconnection on Windows

Michael Paquier wrote: > On Tue, Sep 13, 2016 at 10:42 AM, Kyotaro HORIGUCHI > wrote: > > If we take a policy to try to imitate the behavior of some > > reference platform (specifically Linux) on other platforms, this > > is required disguising. Another potential

Re: [HACKERS] kqueue

So, if I've understood correctly, the purpose of this patch is to improve performance on a multi-CPU system, which has the kqueue() function. Most notably, FreeBSD? I launched a FreeBSD 10.3 instance on Amazon EC2 (ami-e0682b80), on a m4.10xlarge instance. That's a 40 core system, biggest

Re: [HACKERS] _hash_alloc_buckets() safety

Amit Kapila writes: > While working on write-ahead-logging of hash indexes, I noticed that > this function allocates buckets in batches and the mechanism it uses > is that it initialize the last page of batch with zeros and expect > that the filesystem will ensure the

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On Sun, Sep 11, 2016 at 2:05 PM, Peter Geoghegan wrote: > On Sun, Sep 11, 2016 at 6:28 AM, Heikki Linnakangas wrote: >> Pushed this "displace root" patch, with some changes: > > Attached is rebased version of the entire patch series, which should > be applied on

Re: [HACKERS] kqueue

Heikki Linnakangas writes: > So, if I've understood correctly, the purpose of this patch is to > improve performance on a multi-CPU system, which has the kqueue() > function. Most notably, FreeBSD? OS X also has this, so it might be worth trying on a multi-CPU Mac. > If

Re: [HACKERS] An extra error for client disconnection on Windows

On Tue, Sep 13, 2016 at 10:42 AM, Kyotaro HORIGUCHI wrote: > If we take a policy to try to imitate the behavior of some > reference platform (specifically Linux) on other platforms, this > is required disguising. Another potential policy on this problem > is

Re: [HACKERS] 9.6 TAP tests and extensions

On 13 September 2016 at 14:36, Craig Ringer wrote: > > > prove_check: > rm -rf $(CURDIR)/tmp_check/log > cd $(srcdir) && TESTDIR='$(CURDIR)' PATH="$(shell pg_config > --bindir):$$PATH" PGPORT='6$(DEF_PGPORT)' > top_builddir='$(CURDIR)/$(top_builddir)'

Re: [HACKERS] Refactoring of heapam code.

On Mon, Sep 12, 2016 at 7:12 PM, Pavan Deolasee wrote: > I was thinking about locking, bulk reading (like page-mode API) etc. While > you've added an API for vacuuming, we would probably also need an API to > collect dead tuples, pruning etc (not sure if that can be

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

Sorry for my late. The attached patch fixed the wording problems on SGML part. Best regards, -- NEC Business Creation Division / PG-Strom Project KaiGai Kohei > -Original Message- > From: Jeevan Chalke [mailto:jeevan.cha...@enterprisedb.com] > Sent: Tuesday,

Re: [HACKERS] CVE-2016-1238 fix breaks (at least) pg_rewind tests

On Fri, Sep 9, 2016 at 1:25 PM, Tom Lane wrote: > Andres Freund writes: >> On 2016-09-08 20:15:46 -0400, Peter Eisentraut wrote: >>> We don't support build directories with spaces in them, but we support >>> installation directories with spaces in them.