Re: [HACKERS] free space map and visibility map

2017-03-21 Thread Masahiko Sawada
On Mon, Mar 20, 2017 at 11:28 PM, Robert Haas wrote: > On Sat, Mar 18, 2017 at 5:42 PM, Jeff Janes wrote: >> Isn't HEAP2_CLEAN only issued before an intended HOT update? (Which then >> can't leave the block as all visible or all frozen). I think the

Re: [HACKERS] [PATCH] Generic type subscripting

2017-03-21 Thread David Steele
Hi Dmitry, On 3/14/17 7:10 PM, Tom Lane wrote: Dmitry Dolgov <9erthali...@gmail.com> writes: [ generic_type_subscription_v7.patch ] I looked through this a bit. This thread has been idle for over a week. Please respond and/or post a new patch by 2017-03-24 00:00 AoE (UTC-12) or this

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-21 Thread Robert Haas
On Mon, Mar 20, 2017 at 1:38 AM, Craig Ringer wrote: > On 14 March 2017 at 19:57, Robert Haas wrote: >> On Mon, Mar 13, 2017 at 10:22 PM, Craig Ringer wrote: >>> I'll introduce a new LWLock, ClogTruncationLock, which will be

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Petr Jelinek
On 21/03/17 18:14, Robert Haas wrote: > On Tue, Mar 21, 2017 at 1:08 PM, Peter Geoghegan wrote: >> On Tue, Mar 21, 2017 at 10:04 AM, Robert Haas wrote: >>> I think that's a good question. I previously expressed similar >>> concerns. On the one hand, it's

Re: [HACKERS] segfault in hot standby for hash indexes

2017-03-21 Thread Ashutosh Sharma
Hi Jeff, > > I can confirm that that fixes the seg faults for me. Thanks for confirmation. > > Did you mean you couldn't reproduce the problem in the first place, or that > you could reproduce it and now the patch fixes it? If the first of those, I > forget to say you do have to wait for hot

Re: [HACKERS] Freeze on Cygwin w/ concurrency

2017-03-21 Thread Andrew Dunstan
On 03/20/2017 11:47 PM, Noah Misch wrote: > "pgbench -i -s 50; pgbench -S -j2 -c16 -T900 -P5" freezes consistently on > Cygwin 2.2.1 and Cygwin 2.6.0. (I suspect most other versions are affected.) > I've pinged[1] the Cygwin bug thread with some additional detail. If a Cygwin > buildfarm

Re: [HACKERS] PATCH: recursive json_populate_record()

2017-03-21 Thread Andrew Dunstan
On 03/21/2017 01:37 PM, David Steele wrote: > On 3/16/17 11:54 AM, David Steele wrote: >> On 2/1/17 12:53 AM, Michael Paquier wrote: >>> On Thu, Jan 26, 2017 at 6:49 AM, Tom Lane wrote: Nikita Glukhov writes: > On 25.01.2017 23:58, Tom Lane

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Bruce Momjian
On Tue, Mar 21, 2017 at 11:45:09PM +0530, Pavan Deolasee wrote: > Early in the discussion we talked about allowing multiple changes per > WARM chain if they all changed the same index and were in the same > direction so there were no duplicates, but it was complicated.  There > was

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Peter Geoghegan
On Tue, Mar 21, 2017 at 10:04 AM, Robert Haas wrote: > I think that's a good question. I previously expressed similar > concerns. On the one hand, it's hard to ignore the fact that, in the > cases where this wins, it already buys us a lot of performance > improvement. On

Re: [HACKERS] Partitioned tables and relfilenode

2017-03-21 Thread Robert Haas
On Tue, Mar 21, 2017 at 1:21 PM, Simon Riggs wrote: >> The decision not to require the attribute numbers to match doesn't >> necessarily mean we can't get rid of the Append node, though. First >> of all, in a lot of practical cases the attribute numbers will all >> match.

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Pavan Deolasee
On Tue, Mar 21, 2017 at 10:47 PM, Bruce Momjian wrote: > On Tue, Mar 21, 2017 at 01:04:14PM -0400, Robert Haas wrote: > > > I know we have talked about it, but not recently, and if everyone else > > > is fine with it, I am too, but I have to ask these questions. > > > > I think

Re: [HACKERS] Supporting huge pages on Windows

2017-03-21 Thread David Steele
On 3/8/17 8:36 PM, Tsunakawa, Takayuki wrote: From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Ashutosh Sharma To start with, I ran the regression test-suite and didn't find any failures. But, then I am not sure if huge_pages are getting used or

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan [take-2]

2017-03-21 Thread David Steele
Hi, On 3/13/17 3:25 AM, Jeevan Chalke wrote: I have reviewed this patch further and here are my comments: This thread has been idle for over a week. Please respond and/or post a new patch by 2017-03-24 00:00 AoE (UTC-12) or this submission will be marked "Returned with Feedback".

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-21 Thread Masahiko Sawada
On Wed, Mar 22, 2017 at 2:29 AM, David Steele wrote: > Hi, > > On 3/15/17 9:50 AM, Amit Kapila wrote: >> >> >> What about if somebody does manual vacuum and there are no garbage >> tuples to clean, won't in that case also you want to avoid skipping >> the lazy_cleanup_index?

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Petr Jelinek
On 21/03/17 18:19, Bruce Momjian wrote: > On Tue, Mar 21, 2017 at 01:14:00PM -0400, Robert Haas wrote: >> On Tue, Mar 21, 2017 at 1:08 PM, Peter Geoghegan wrote: >>> On Tue, Mar 21, 2017 at 10:04 AM, Robert Haas wrote: I think that's a good question. I

Re: [HACKERS] [WIP] RE: DECLARE STATEMENT setting up a connection in ECPG

2017-03-21 Thread David Steele
Hi Haribabu, On 3/7/17 12:09 AM, Ideriha, Takeshi wrote: I tried applying your patches. But it failed... The error messages are as below. Attached 004_declareStmt_test_v5.patch is a rebased one. The rest of patches are same as older version. Regards, Ideriha, Takeshi You are signed up to

Re: [HACKERS] PATCH: Make pg_stop_backup() archive wait optional

2017-03-21 Thread Fujii Masao
On Tue, Mar 21, 2017 at 11:03 AM, Tsunakawa, Takayuki wrote: > From: David Steele [mailto:da...@pgmasters.net] >> Well, that's embarrassing. When I recreated the function to add defaults >> I messed up the AS clause and did not pay attention to the results of the

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Robert Haas
On Tue, Mar 21, 2017 at 1:08 PM, Peter Geoghegan wrote: > On Tue, Mar 21, 2017 at 10:04 AM, Robert Haas wrote: >> I think that's a good question. I previously expressed similar >> concerns. On the one hand, it's hard to ignore the fact that, in the >> cases

Re: [HACKERS] make async slave to wait for lsn to be replayed

2017-03-21 Thread Robert Haas
On Sun, Mar 12, 2017 at 10:20 PM, Thomas Munro wrote: > Maybe someone can think of a clever way for an extension to insert a > wait for a user-supplied LSN *before* acquiring a snapshot so it can > work for the higher levels, or maybe the hooks should go into core >

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Bruce Momjian
On Tue, Mar 21, 2017 at 01:04:14PM -0400, Robert Haas wrote: > > I know we have talked about it, but not recently, and if everyone else > > is fine with it, I am too, but I have to ask these questions. > > I think that's a good question. I previously expressed similar > concerns. On the one

Re: [HACKERS] Measuring replay lag

2017-03-21 Thread David Steele
Hi Thomas, On 3/15/17 8:38 PM, Simon Riggs wrote: On 16 March 2017 at 08:02, Thomas Munro wrote: I agree that these states exist, but we disagree on what 'lag' really means, or, rather, which of several plausible definitions would be the most useful here. My

Re: [HACKERS] PATCH: recursive json_populate_record()

2017-03-21 Thread David Steele
On 3/16/17 11:54 AM, David Steele wrote: On 2/1/17 12:53 AM, Michael Paquier wrote: On Thu, Jan 26, 2017 at 6:49 AM, Tom Lane wrote: Nikita Glukhov writes: On 25.01.2017 23:58, Tom Lane wrote: I think you need to take a second look at the code

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-03-21 Thread Peter Geoghegan
On Tue, Mar 21, 2017 at 9:10 AM, Robert Haas wrote: > - * This code is moderately slow (~10% slower) compared to the regular > - * btree (insertion) build code on sorted or well-clustered data. On > - * random data, however, the insertion build code is unusable -- the > -

Re: [HACKERS] PATCH: Configurable file mode mask

2017-03-21 Thread David Steele
On 3/18/17 3:57 PM, Robert Haas wrote: On Wed, Mar 15, 2017 at 3:00 AM, Tsunakawa, Takayuki wrote: I'm on David's side, too. I don't postmaster to always scan all files at startup. +1. Even just doing it during crash recovery, it can take a regrettably long

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Robert Haas
On Tue, Mar 21, 2017 at 2:03 PM, Petr Jelinek wrote: > This is why I like the idea of pluggable storage, if we ever get that it > would buy us ability to implement completely different heap format > without breaking pg_upgrade. You probably won't be surprised to

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-03-21 Thread Robert Haas
On Tue, Mar 21, 2017 at 2:03 PM, Peter Geoghegan wrote: > I think that since the comment refers to code from before 1999, it can > go. Any separate patch to remove it would have an entirely negative > linediff. It's a good general principle that a patch should do one thing well and

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-21 Thread Robert Haas
On Tue, Mar 14, 2017 at 6:10 PM, Peter Geoghegan wrote: > On Tue, Mar 14, 2017 at 2:48 PM, Peter Geoghegan wrote: >> I think that that's safe, but it is a little disappointing that it >> does not allow us to skip work in the case that you really had in mind >> when

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-03-21 Thread Andrew Borodin
Hi, Teodor! 2017-03-21 20:32 GMT+05:00 Teodor Sigaev : > I had a look on patch That's great, thanks! > > /* > * All subtree is empty - just return TRUE to indicate that parent > must > * do a cleanup. Unless we are ROOT an there is way to go upper. >

Re: [HACKERS] perlcritic

2017-03-21 Thread David Steele
Hi Daniel, On 3/6/17 12:02 PM, Dagfinn Ilmari Mannsåker wrote: ilm...@ilmari.org (Dagfinn Ilmari Mannsåker) writes: Hi Peter, Peter Eisentraut writes: I posted this about 18 months ago but then ran out of steam. [ ] Here is an updated patch. The testing

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Bruce Momjian
On Tue, Mar 21, 2017 at 07:05:15PM +0100, Petr Jelinek wrote: > >> Well, I don't want to rule it out either, but if we do a release to > >> which you can't pg_upgrade, it's going to be really painful for a lot > >> of users. Many users can't realistically upgrade using pg_dump, ever. > >> So

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Robert Haas
On Tue, Mar 21, 2017 at 12:49 PM, Bruce Momjian wrote: > On Tue, Mar 21, 2017 at 09:25:49AM -0400, Robert Haas wrote: >> On Tue, Mar 21, 2017 at 8:41 AM, Pavan Deolasee >> > TBH I see many artificial scenarios here. It will be very useful if he can >> > rerun the query with some

Re: [HACKERS] Partitioned tables and relfilenode

2017-03-21 Thread Simon Riggs
On 21 March 2017 at 16:33, Robert Haas wrote: > On Tue, Mar 21, 2017 at 12:19 PM, Simon Riggs wrote: >> On 16 March 2017 at 10:03, Amit Langote >> wrote: >>> On 2017/03/15 7:09, Robert Haas wrote: >> I think that

Re: [HACKERS] logical replication apply to run with sync commit off by default

2017-03-21 Thread Robert Haas
On Mon, Mar 20, 2017 at 7:56 PM, Petr Jelinek wrote: > On 18/03/17 13:31, Petr Jelinek wrote: >> On 07/03/17 06:23, Petr Jelinek wrote: >>> there has been discussion at the logical replication initial copy thread >>> [1] about making apply work with sync commit off

Re: [HACKERS] Page Scan Mode in Hash Index

2017-03-21 Thread Jesper Pedersen
Hi, On 02/14/2017 12:27 AM, Ashutosh Sharma wrote: Currently, Hash Index scan works tuple-at-a-time, i.e. for every qualifying tuple in a page, it acquires and releases the lock which eventually increases the lock/unlock traffic. For example, if an index page contains 100 qualified tuples, the

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Pavan Deolasee
On Tue, Mar 21, 2017 at 10:34 PM, Robert Haas wrote: > On Tue, Mar 21, 2017 at 12:49 PM, Bruce Momjian wrote: > > On Tue, Mar 21, 2017 at 09:25:49AM -0400, Robert Haas wrote: > >> On Tue, Mar 21, 2017 at 8:41 AM, Pavan Deolasee > >> > TBH I see many

Re: [HACKERS] increasing the default WAL segment size

2017-03-21 Thread Robert Haas
On Tue, Mar 21, 2017 at 9:04 AM, Stephen Frost wrote: > In short, I'm also concerned about this change to make WAL file names no > longer match up with LSNs and also about the odd stepping that you get > as a result of this change when it comes to WAL file names. OK, that's a

Re: [HACKERS] Create replication slot in pg_basebackup if requested and not yet present

2017-03-21 Thread Robert Haas
On Sun, Mar 19, 2017 at 12:01 PM, Magnus Hagander wrote: > I think maybe we should output a message when the slot is created, at least > in verbose mode, to make sure people realize that happened. Does that seem > reasonable? Slots are great until you leave one lying around

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Bruce Momjian
On Tue, Mar 21, 2017 at 01:14:00PM -0400, Robert Haas wrote: > On Tue, Mar 21, 2017 at 1:08 PM, Peter Geoghegan wrote: > > On Tue, Mar 21, 2017 at 10:04 AM, Robert Haas wrote: > >> I think that's a good question. I previously expressed similar > >> concerns.

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-21 Thread Masahiko Sawada
On Wed, Mar 15, 2017 at 4:50 PM, Amit Kapila wrote: > On Thu, Mar 9, 2017 at 10:21 PM, Masahiko Sawada > wrote: >> On Wed, Mar 8, 2017 at 1:43 AM, Peter Geoghegan wrote: >>> On Sat, Mar 4, 2017 at 1:30 AM, Amit Kapila

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-21 Thread David Steele
Hi, On 3/15/17 9:50 AM, Amit Kapila wrote: What about if somebody does manual vacuum and there are no garbage tuples to clean, won't in that case also you want to avoid skipping the lazy_cleanup_index? Another option could be to skip updating the relfrozenxid if we have skipped the index

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-21 Thread Robert Haas
On Tue, Mar 21, 2017 at 6:36 AM, Dilip Kumar wrote: > On Tue, Mar 21, 2017 at 3:36 PM, Rafia Sabih > wrote: >> On Wed, Mar 15, 2017 at 8:55 PM, Robert Haas wrote: >>> Note this: >>> >>> if (completed ||

Re: [HACKERS] make async slave to wait for lsn to be replayed

2017-03-21 Thread David Steele
Hi Ivan, On 3/12/17 10:20 PM, Thomas Munro wrote: On Fri, Mar 10, 2017 at 1:49 AM, Ivan Kartyshov wrote: Here I attached rebased patch waitlsn_10dev_v3 (core feature) I will leave the choice of implementation (core/contrib) to the discretion of the community. Will

[HACKERS] Re: [PATCH] guc-ify the formerly hard-coded MAX_SEND_SIZE to max_wal_send

2017-03-21 Thread David Steele
On 3/16/17 11:56 AM, David Steele wrote: My recommendation is that we mark this patch "Returned with Feedback" to allow you time to test and refine the patch. You can resubmit once it is ready. This submission has been marked "Returned with Feedback". Please feel free to resubmit to a

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

2017-03-21 Thread David Steele
On 3/7/17 9:42 PM, Pavan Deolasee wrote: Fair point. I'm not going to "persist" with the idea too long. It seemed like a good, low-risk feature to me which can benefit certain use cases quite reasonably. It's not uncommon to create indexes (or reindex existing indexes to remove index bloats) on

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Bruce Momjian
On Tue, Mar 21, 2017 at 11:54:25PM +0530, Pavan Deolasee wrote: > We can also save HEAP_WARM_UPDATED flag since this is required only for > abort-handling case. We can find a way to push that information down to the > old > tuple if UPDATE aborts and we detect the broken chain. Again, not fully

Re: [HACKERS] [COMMITTERS] pgsql: Add missing support for new node fields

2017-03-21 Thread Andres Freund
Hi, On 2017-03-21 07:22:57 +0100, Fabien COELHO wrote: > > Add missing support for new node fields > > > > Commit b6fb534f added two new node fields but neglected to add copy and > > comparison support for them, Mea culpa, should have checked for that. > > I've been annoyed by these stupid

[HACKERS] Do we create a new roadmap page for development?

2017-03-21 Thread Tsunakawa, Takayuki
Hello, I'd like to share our roadmap for PostgreSQL development, as other companies and individuals do in the following page. But this page is for PostgreSQL 10. PostgreSQL10 Roadmap https://wiki.postgresql.org/wiki/PostgreSQL10_Roadmap Should I create a page for PostgreSQL 11 likewise? Or,

[HACKERS] Asymmetry between parent and child wrt "false" quals

2017-03-21 Thread Ashutosh Bapat
When I run a query like below on a child-less table, the plan comes out to be explain verbose SELECT * FROM uprt1_l WHERE a = 1 AND a = 2; QUERY PLAN -- Result (cost=0.00..11.50 rows=1 width=13)

Re: [HACKERS] Adding support for Default partition in partitioning

2017-03-21 Thread Rushabh Lathia
I picked this for review and noticed that patch is not getting cleanly complied on my environment. partition.c: In function ‘RelationBuildPartitionDesc’: partition.c:269:6: error: ISO C90 forbids mixed declarations and code [-Werror=declaration-after-statement] Const*val = lfirst(c);

Re: [HACKERS] [COMMITTERS] pgsql: Add missing support for new node fields

2017-03-21 Thread Fabien COELHO
Add missing support for new node fields Commit b6fb534f added two new node fields but neglected to add copy and comparison support for them, Mea culpa, should have checked for that. I've been annoyed by these stupid functions and forgetting to update them since I run into them while trying

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-21 Thread Andres Freund
On 2017-03-20 16:06:27 -0400, Tom Lane wrote: > ... is there a reason why resultnum for EEOP_ASSIGN_* steps is declared > size_t and not just int? Since it's an array index, and one that > certainly can't be bigger than AttrNumber, that seems rather confusing. Not that I can see, no. I guess I

Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall

2017-03-21 Thread Haribabu Kommi
Because of this refactor handing of database objects between pg_dump and pg_dumpall, the latest pg_dump tap tests are failing in the following scenarios. 1. CREATE DATABASE postgres Before this patch, the pg_dump uses to dump the CREATE DATABASE command of postgres but not by pg_dumpall. During

[HACKERS] Partition-wise aggregation/grouping

2017-03-21 Thread Jeevan Chalke
Hi all, Declarative partitioning is supported in PostgreSQL 10 and work is already in progress to support partition-wise joins. Here is a proposal for partition-wise aggregation/grouping. Our initial performance measurement has shown 7 times performance when partitions are on foreign servers and

Re: [HACKERS] btree_gin and btree_gist for enums

2017-03-21 Thread Anastasia Lubennikova
28.02.2017 00:22, Andrew Dunstan: OK, here's the whole series of patches. Patch 1 adds the CallerFInfoFunctionCall{1,2} functions. Patch 2 adds btree_gist support for their use for non-varlena types Patch 3 does the same for varlena types (Not required for patch 4, but better to be

Re: [HACKERS] Asymmetry between parent and child wrt "false" quals

2017-03-21 Thread Amit Langote
On 2017/03/21 14:59, Ashutosh Bapat wrote: > When I run a query like below on a child-less table, the plan comes out to be > > explain verbose SELECT * FROM uprt1_l WHERE a = 1 AND a = 2; > QUERY PLAN >

Re: [HACKERS] [PATCH]: fix bug in SP-GiST box_ops

2017-03-21 Thread Anastasia Lubennikova
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:not tested As I can see, this bugfix was already discussed and reviewed.

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-03-21 Thread Ideriha, Takeshi
Hi, I like your suggestion and took a look at your patch though I’m not the expert about psql. I like the idea taking advantage of linestyle utilities to implement rst and markdown format efficiently instead of newly developing pset format things. But I'm thinking two comments below needs

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-03-21 Thread Haribabu Kommi
On Tue, Mar 21, 2017 at 3:16 PM, Seki, Eiji wrote: > > > Thank you for you review. > > I reflected your comment and attach the updated patch. Thanks for the updated patch. +/* Use these flags in GetOldestXmin as "flags" */ How about some thing like the following. /*

[HACKERS] segfault in hot standby for hash indexes

2017-03-21 Thread Jeff Janes
Against an unmodified HEAD (17fa3e8), I got a segfault in the hot standby. Using the attached files, I start the test case like this: nice sh do_nocrash_sr.sh >& do_nocrash_sr.err & And start the replica like: rm -r /tmp/data2_replica/ ; psql -p 9876 -c "select

Re: [HACKERS] segfault in hot standby for hash indexes

2017-03-21 Thread Amit Kapila
On Tue, Mar 21, 2017 at 1:28 PM, Jeff Janes wrote: > Against an unmodified HEAD (17fa3e8), I got a segfault in the hot standby. > I think I see the problem in hash_xlog_vacuum_get_latestRemovedXid(). It seems to me that we are using different block_id for registering the

Re: [HACKERS] extended statistics: n-distinct

2017-03-21 Thread Kyotaro HORIGUCHI
Thank you for finishing this. At Mon, 20 Mar 2017 16:02:20 -0300, Alvaro Herrera wrote in <20170320190220.ixlaueanxegqd5gr@alvherre.pgsql> > Here is a closer to final version of the multivariate statistics series, > last posted at >

Re: [HACKERS] Partition-wise aggregation/grouping

2017-03-21 Thread Antonin Houska
Jeevan Chalke wrote: > Declarative partitioning is supported in PostgreSQL 10 and work is already in > progress to support partition-wise joins. Here is a proposal for > partition-wise > aggregation/grouping. Our initial performance measurement has shown 7 times

[HACKERS] Implementing delete in columnar store fdw

2017-03-21 Thread sri harsha
Hello, I want to implement delete functionality for a column store fdw in postgres. It is similar to file_fdw. I want to use the “AddForeignUpdateTargets” function to implement this , but the junk filter shouldn’t be a column present in the table . Is it possible to add a Expr/Var to the

Re: [HACKERS] Partitioned tables and relfilenode

2017-03-21 Thread Amit Langote
On 2017/03/21 1:16, Robert Haas wrote: > On Fri, Mar 17, 2017 at 4:57 AM, Amit Langote > wrote: >>> Yes, but on the flip side, you're having to add code in a lot of >>> places -- I think I counted 7 -- where you turn around and ignore >>> those AppendRelInfos. >> >>

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Robert Haas
On Tue, Mar 21, 2017 at 8:41 AM, Pavan Deolasee wrote: >> Yeah. So what's the deal with this? Is somebody working on figuring >> out a different approach that would reduce this overhead? Are we >> going to defer WARM to v11? Or is the intent to just ignore the 5-10%

Re: [HACKERS] [PATCH]: fix bug in SP-GiST box_ops

2017-03-21 Thread Teodor Sigaev
Thank you, pushed. I just make test table permanent. Anastasia Lubennikova wrote: The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Alvaro Herrera
Amit Kapila wrote: > I think it is because heap_getattr() is not that cheap. We have > noticed the similar problem during development of scan key push down > work [1]. One possibility to reduce the cost of that is to use whole tuple deform instead of repeated individual heap_getattr() calls.

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-21 Thread Kuntal Ghosh
On Tue, Mar 21, 2017 at 10:52 AM, Michael Paquier wrote: Thank you for the review. > Unfortunately this is true only for background workers that connect to > a database. And this would break for bgworkers that do not do that. > The point to fix is here: > + if

Re: [HACKERS] Partitioned tables and relfilenode

2017-03-21 Thread Robert Haas
On Tue, Mar 21, 2017 at 5:05 AM, Amit Langote wrote: > Attached updated patches. Committed 0001 after removing a comma. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Freeze on Cygwin w/ concurrency

2017-03-21 Thread Robert Haas
On Mon, Mar 20, 2017 at 11:47 PM, Noah Misch wrote: > "pgbench -i -s 50; pgbench -S -j2 -c16 -T900 -P5" freezes consistently on > Cygwin 2.2.1 and Cygwin 2.6.0. (I suspect most other versions are affected.) > I've pinged[1] the Cygwin bug thread with some additional detail.

Re: [HACKERS] postgres_fdw bug in 9.6

2017-03-21 Thread Etsuro Fujita
On 2017/03/17 0:37, David Steele wrote: This patch does not apply cleanly at cccbdde: Marked "Waiting on Author". Ok, I'll update the patch. One thing I'd like to revise in addition to that is (1) add to JoinPathExtraData a flag member to indicate whether to give the FDW a chance to

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-03-21 Thread Pavel Stehule
2017-03-21 9:59 GMT+01:00 Ideriha, Takeshi : > Hi, > > I like your suggestion and took a look at your patch though I’m not the > expert about psql. > > > > I like the idea taking advantage of linestyle utilities > to implement rst and markdown format efficiently

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-21 Thread Rafia Sabih
On Wed, Mar 15, 2017 at 8:55 PM, Robert Haas wrote: > Note this: > > if (completed || !fcache->returnsSet) > postquel_end(es); > > When the SQL function doesn't return a set, then we can allow > parallelism even when lazyEval is set, because we'll only

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-21 Thread Dilip Kumar
On Tue, Mar 21, 2017 at 3:36 PM, Rafia Sabih wrote: > On Wed, Mar 15, 2017 at 8:55 PM, Robert Haas wrote: >> Note this: >> >> if (completed || !fcache->returnsSet) >> postquel_end(es); >> >> When the SQL function doesn't

Re: [HACKERS] segfault in hot standby for hash indexes

2017-03-21 Thread Ashutosh Sharma
Hi Jeff, On Tue, Mar 21, 2017 at 1:54 PM, Amit Kapila wrote: > On Tue, Mar 21, 2017 at 1:28 PM, Jeff Janes wrote: >> Against an unmodified HEAD (17fa3e8), I got a segfault in the hot standby. >> > > I think I see the problem in

Re: [HACKERS] WIP: Covering + unique indexes.

2017-03-21 Thread Anastasia Lubennikova
Patch rebased to the current master is in attachments. -- Anastasia Lubennikova Postgres Professional: http://www.postgrespro.com The Russian Postgres Company commit 497d52b713dd8f926b465ddf22f21db7229b12e3 Author: Anastasia Date: Tue Mar 21 12:58:13 2017 +0300

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-03-21 Thread Pavel Stehule
2017-03-21 10:59 GMT+01:00 Jan Michálek : > > > 2017-03-21 9:59 GMT+01:00 Ideriha, Takeshi >: > >> Hi, >> >> I like your suggestion and took a look at your patch though I’m not the >> expert about psql. >> >> >> >> I like the idea taking

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-21 Thread Rajkumar Raghuwanshi
> On Mon, Mar 20, 2017 at 1:19 PM, Ashutosh Bapat > wrote: I have created some test to cover partition wise joins with postgres_fdw, also verified make check. patch attached. Thanks & Regards, Rajkumar Raghuwanshi QMG, EnterpriseDB Corporation diff --git

Re: [HACKERS] Logical Replication and Character encoding

2017-03-21 Thread Kyotaro HORIGUCHI
At Mon, 6 Mar 2017 11:13:48 +0100, Petr Jelinek wrote in > >>> Well the length is necessary to be able to add binary format support in > >>> future so it's definitely not an omission. > >> > >> Right. So it

Re: [HACKERS] Logical Replication and Character encoding

2017-03-21 Thread Kyotaro HORIGUCHI
Mmm. I shot the previous mail halfway. At Mon, 6 Mar 2017 11:13:48 +0100, Petr Jelinek wrote in > > By the way, I noticed that postmaster launches logical > > replication launcher even if wal_level < logical.

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Amit Kapila
On Fri, Mar 10, 2017 at 11:37 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Wed, Mar 8, 2017 at 2:30 PM, Alvaro Herrera >> wrote: >> > Not really -- it's a bit slower actually in a synthetic case measuring >> > exactly the slowed-down

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Amit Kapila
On Thu, Mar 9, 2017 at 8:43 AM, Robert Haas wrote: > On Wed, Mar 8, 2017 at 2:30 PM, Alvaro Herrera > wrote: >> Not really -- it's a bit slower actually in a synthetic case measuring >> exactly the slowed-down case. See >>

Re: [HACKERS] postgres_fdw: support parameterized foreign joins

2017-03-21 Thread Etsuro Fujita
On 2017/03/16 22:23, Arthur Zakirov wrote: 2017-02-27 12:40 GMT+03:00 Etsuro Fujita : I'd like to propose to support parameterized foreign joins. Attached is a patch for that, which has been created on top of [1]. Can you rebase the patch? It is not applied now.

Re: [HACKERS] Push down more full joins in postgres_fdw

2017-03-21 Thread Etsuro Fujita
On 2017/03/17 2:35, Robert Haas wrote: And ... I don't see anything to complain about, so, committed. Thanks for committing, Robert! Thanks for reviewing, Ashutosh and David! Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-03-21 Thread Jan Michálek
2017-03-21 9:59 GMT+01:00 Ideriha, Takeshi : > Hi, > > I like your suggestion and took a look at your patch though I’m not the > expert about psql. > > > > I like the idea taking advantage of linestyle utilities > > to implement rst and markdown format efficiently

Re: [HACKERS] pageinspect and hash indexes

2017-03-21 Thread Amit Kapila
On Tue, Mar 21, 2017 at 10:15 AM, Ashutosh Sharma wrote: >> >> I think it is not just happening for freed overflow but also for newly >> allocated bucket page. It would be good if we could mark freed >> overflow page as UNUSED page rather than just initialising it's header

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-03-21 Thread Jan Michálek
2017-03-21 11:01 GMT+01:00 Pavel Stehule : > > > 2017-03-21 10:59 GMT+01:00 Jan Michálek : > >> >> >> 2017-03-21 9:59 GMT+01:00 Ideriha, Takeshi > m>: >> >>> Hi, >>> >>> I like your suggestion and took a look at

[HACKERS] Problem in Parallel Bitmap Heap Scan?

2017-03-21 Thread Thomas Munro
Hi, I noticed a failure in the inet.sql test while running the regression tests with parallelism cranked up, and can reproduce it interactively as follows. After an spgist index is created and the plan changes to the one shown below, the query returns no rows. regression=# set

Re: [HACKERS] BRIN cost estimate

2017-03-21 Thread Emre Hasegeli
> Not sure what you mean here. I'm not speaking of the brin index am, I > mean the get_index_stats_hook call which you've added. I see. Actually this part was from Alvaro. I haven't noticed the get_index_stats_hook call before, but it is still the same coding as btcostestimate().

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Pavan Deolasee
On Tue, Mar 21, 2017 at 5:34 PM, Robert Haas wrote: > On Tue, Mar 21, 2017 at 6:56 AM, Amit Kapila > wrote: > >> Hmm, that test case isn't all that synthetic. It's just a single > >> column bulk update, which isn't anything all that crazy, and

[HACKERS] Implementing delete in columnar store fdw

2017-03-21 Thread Hari Krishnan
Hello, I want to implement delete functionality for a column store fdw in postgres. It is similar to file_fdw. I want to use the “AddForeignUpdateTargets” function to implement this , but the junk filter shouldn’t be a column present in the table . Is it possible to add a Expr/Var to

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2017-03-21 Thread Amit Kapila
On Mon, Mar 20, 2017 at 8:27 AM, Robert Haas wrote: > On Fri, Mar 17, 2017 at 2:30 AM, Amit Kapila wrote: >>> I was wondering about doing an explicit test: if the XID being >>> committed matches the one in the PGPROC, and nsubxids matches, and the

Re: [HACKERS] increasing the default WAL segment size

2017-03-21 Thread Beena Emerson
PFA an updated patch. This fixes an issue reported by Tushar internally. Since the patch changes the way min and max wal_size is stored internally from segment count to size in kb, it limited the maximum size of min and max_wal_size to 2GB in 32 bit systems. The minimum required segment is 2 and

Re: [HACKERS] increasing the default WAL segment size

2017-03-21 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Mar 20, 2017 at 7:23 PM, David Steele wrote: > > With 16MB WAL segments the filename neatly aligns with the LSN. For > > example: > > > > WAL FILE 0001000100FE = LSN 1/FE00 > > > > This no longer

Re: [HACKERS] brin autosummarization -- autovacuum "work items"

2017-03-21 Thread Alvaro Herrera
Thomas Munro wrote: > Another thought about this design: Why autovacuum? One reason is that autovacuum is already there, so it's convenient to give it the responsibility for this kind of task. Another reason is that autovacuum is already doing this, via vacuum. I don't see the need to have a

Re: [HACKERS] brin autosummarization -- autovacuum "work items"

2017-03-21 Thread Alvaro Herrera
Thomas Munro wrote: > What is your motivation for using DSA? It seems you are creating an > area and then using it to make exactly one allocation of a constant > size known up front to hold your fixed size workitems array. You > don't do any dynamic allocation at runtime, apart from the detail

Re: [HACKERS] [COMMITTERS] pgsql: Add missing support for new node fields

2017-03-21 Thread Fabien COELHO
Hello Andres, It is not done yet, but it looks that it can work in the end with limited effort. Currently it works for copy & equal. It'd have to do out/read as well imo. Sure. This part is WIP, though. Is there some interest to generate the x00kB of sources rather than edit them

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-03-21 Thread Peter Geoghegan
On Tue, Mar 21, 2017 at 12:06 PM, Robert Haas wrote: > From my point of view, the main point is that having two completely > separate mechanisms for managing temporary files that need to be > shared across cooperating workers is not a good decision. That's a > need that's

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-21 Thread Alvaro Herrera
Bruce Momjian wrote: > On Tue, Mar 21, 2017 at 04:43:58PM -0300, Alvaro Herrera wrote: > > Bruce Momjian wrote: > > > > > I don't think it makes sense to try and save bits and add complexity > > > when we have no idea if we will ever use them, > > > > If we find ourselves in dire need of

Re: [HACKERS] Monitoring roles patch

2017-03-21 Thread Denish Patel
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Looks good. -- Sent via pgsql-hackers mailing list

  1   2   3   >