Re: Regarding ambulkdelete, amvacuumcleanup index methods

2018-01-23 Thread Masahiko Sawada
On Wed, Jan 24, 2018 at 1:27 PM, Abinaya k wrote: > Hai all, > We are building In-memory index extension for postgres. We would > capture table inserts, updates, deletes using triggers. During vacuum > operation, postgres would give calls to ambulkdelete,

Help needed in using 'on_dsm_detach' callback

2018-01-23 Thread Gaddam Sai Ram
Hello people, We are trying to build an in-memory index in postgres using dsa. Here is how we implemented dsa part. We have PROC_DSA_AREA global variable(Process specific DSA Pointer) We have a piece of traditional postgres shared memory to store dsa_handle Each process

Re: [HACKERS] Subscription code improvements

2018-01-23 Thread Masahiko Sawada
On Tue, Jan 23, 2018 at 7:58 AM, Stephen Frost wrote: > Greetings, > > * Michael Paquier (michael.paqu...@gmail.com) wrote: >> On Fri, Aug 18, 2017 at 2:09 PM, Masahiko Sawada >> wrote: >> > On Thu, Aug 17, 2017 at 8:17 PM, Masahiko Sawada

JIT compiling with LLVM v9.0

2018-01-23 Thread Andres Freund
Hi, I've spent the last weeks working on my LLVM compilation patchset. In the course of that I *heavily* revised it. While still a good bit away from committable, it's IMO definitely not a prototype anymore. There's too many small changes, so I'm only going to list the major things. A good bit

Re: Doc tweak for huge_pages?

2018-01-23 Thread Justin Pryzby
On Wed, Jan 24, 2018 at 07:46:41AM +0100, Catalin Iacob wrote: > I see Peter assigned himself as committer, some more information below > for him to decide on the strength of the anti THP message. Thanks for digging this up! > And it would be good if somebody could run benchmarks on pre 4.6 and >

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-01-23 Thread amul sul
On Tue, Jan 23, 2018 at 7:01 PM, Amit Kapila wrote: > On Fri, Jan 12, 2018 at 11:43 AM, amul sul wrote: [] > I have asked to change the message "tuple to be updated .." after > heap_lock_tuple call not in nodeModifyTable.c, so please revert the >

Re: Index-only scan returns incorrect results when using a composite GIST index with a gist_trgm_ops column.

2018-01-23 Thread Kyotaro HORIGUCHI
Thank you for looking this. At Wed, 24 Jan 2018 00:13:51 +0300, Sergei Kornilov wrote in <348951516742...@web54j.yandex.ru> > Hello > I tested this patch and think it can be commited to master. Is there a CF > record? I can not find one. Not yet. I'm thinking of creating an

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Thomas Munro
On Wed, Jan 24, 2018 at 6:43 PM, Amit Kapila wrote: > On Wed, Jan 24, 2018 at 10:36 AM, Thomas Munro > wrote: >> On Wed, Jan 24, 2018 at 5:59 PM, Amit Kapila wrote: > I am going to repeat my previous suggest

Re: Doc tweak for huge_pages?

2018-01-23 Thread Catalin Iacob
On Tue, Jan 23, 2018 at 7:13 PM, Catalin Iacob wrote: > By the way, Fedora 27 does disable THP by default, they deviate from > upstream in this regard: > When I have some time I'll try to do some digging into history of the > Fedora kernel package to see if they provide a

Re: [HACKERS] taking stdbool.h into use

2018-01-23 Thread Michael Paquier
On Tue, Jan 23, 2018 at 11:33:56AM -0500, Peter Eisentraut wrote: > Here is a proposed patch set to clean this up. First, add some test > coverage for record_image_cmp. (There wasn't any, only for > record_image_eq as part of MV testing.) Then, remove the GET_ macros > from

Re: Index-only scan returns incorrect results when using a composite GIST index with a gist_trgm_ops column.

2018-01-23 Thread Andrey Borodin
Hi! > 24 янв. 2018 г., в 2:13, Sergei Kornilov написал(а): > > Should we also make backport to older versions? I test on REL_10_STABLE - > patch builds and works ok, but "make check" fails on new testcase with error: >> CREATE INDEX ON t USING gist (a test_inet_ops, a inet_ops);

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Peter Geoghegan
On Tue, Jan 23, 2018 at 9:38 PM, Amit Kapila wrote: > On Wed, Jan 24, 2018 at 10:38 AM, Peter Geoghegan wrote: >> The leader can go ahead and sort before calling something like a new >> WaitForParallelWorkersToAttach() function (or even >>

Re: Incorrect comments in partition.c

2018-01-23 Thread Etsuro Fujita
(2018/01/24 13:06), Amit Langote wrote: > On 2018/01/23 20:43, Etsuro Fujita wrote: >> Here is a comment for get_qual_for_list in partition.c: >> >>* get_qual_for_list >>* >>* Returns an implicit-AND list of expressions to use as a list partition's >> - * constraint, given the

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Amit Kapila
On Wed, Jan 24, 2018 at 10:36 AM, Thomas Munro wrote: > On Wed, Jan 24, 2018 at 5:59 PM, Amit Kapila wrote: I am going to repeat my previous suggest that we use a Barrier here. Given the discussion subsequent to my original

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Amit Kapila
On Wed, Jan 24, 2018 at 10:38 AM, Peter Geoghegan wrote: > On Tue, Jan 23, 2018 at 9:02 PM, Thomas Munro > wrote: >>> Yes, this is what I am trying to explain on parallel create index >>> thread. I think there we need to either use >>>

Re: Possible performance regression in version 10.1 with pgbench read-write tests.

2018-01-23 Thread Mithun Cy
On Wed, Jan 24, 2018 at 7:36 AM, Amit Kapila wrote: > Both the cases look identical, but from the document attached, it > seems the case-1 is for scale factor 300. Oops sorry it was a typo. CASE 1 is scale factor 300 which will fit in shared buffer =8GB. -- Thanks

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Peter Geoghegan
On Tue, Jan 23, 2018 at 9:02 PM, Thomas Munro wrote: >> Yes, this is what I am trying to explain on parallel create index >> thread. I think there we need to either use >> WaitForParallelWorkersToFinish or WaitForParallelWorkersToAttach (a >> new API as proposed in

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Thomas Munro
On Wed, Jan 24, 2018 at 5:59 PM, Amit Kapila wrote: >>> I am going to repeat my previous suggest that we use a Barrier here. >>> Given the discussion subsequent to my original proposal, this can be a >>> lot simpler than what I suggested originally. Each worker does >>>

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Thomas Munro
On Wed, Jan 24, 2018 at 5:43 PM, Amit Kapila wrote: >> Hmm. Yeah. I can't seem to reach a stuck case and was probably just >> confused and managed to confuse Robert too. If you make >> fork_process() fail randomly (see attached), I see that there are a >> couple of

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Amit Kapila
On Wed, Jan 24, 2018 at 12:20 AM, Peter Geoghegan wrote: > On Tue, Jan 23, 2018 at 10:36 AM, Robert Haas wrote: >> As Amit says, what remains is the case where fork() fails or the >> worker dies before it reaches the line in ParallelWorkerMain that >> reads

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Amit Kapila
On Wed, Jan 24, 2018 at 9:55 AM, Thomas Munro wrote: > On Wed, Jan 24, 2018 at 3:45 PM, Amit Kapila wrote: >> On Wed, Jan 24, 2018 at 2:35 AM, Robert Haas wrote: >>> In the case of Gather, for example, we won't >>>

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Amit Kapila
On Wed, Jan 24, 2018 at 10:03 AM, Peter Geoghegan wrote: > On Tue, Jan 23, 2018 at 8:25 PM, Thomas Munro > wrote: >>> Hmm, I think that case will be addressed because tuple queues can >>> detect if the leader is not attached. It does in code path >>>

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Peter Geoghegan
On Tue, Jan 23, 2018 at 8:25 PM, Thomas Munro wrote: >> Hmm, I think that case will be addressed because tuple queues can >> detect if the leader is not attached. It does in code path >> shm_mq_receive->shm_mq_counterparty_gone. In >> shm_mq_counterparty_gone, it

Regarding ambulkdelete, amvacuumcleanup index methods

2018-01-23 Thread Abinaya k
Hai all, We are building In-memory index extension for postgres. We would capture table inserts, updates, deletes using triggers. During vacuum operation, postgres would give calls to ambulkdelete, amvacuumcleanup (as part of index cleanup). As we handle all updates, deletes using triggers,

Re: PATCH: Configurable file mode mask

2018-01-23 Thread Michael Paquier
On Tue, Jan 23, 2018 at 10:48:07PM -0500, David Steele wrote: > Sorry - it means "level of effort". I was trying to get an idea if it > is something that could be available in the PG11 development cycle, or > if I should be looking at other ways to get the testing for this patch > completed. I

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Thomas Munro
On Wed, Jan 24, 2018 at 3:45 PM, Amit Kapila wrote: > On Wed, Jan 24, 2018 at 2:35 AM, Robert Haas wrote: >> In the case of Gather, for example, we won't >> WaitForParallelWorkersToFinish() until we've read all the tuples. If >> there's a tuple

Re: documentation is now XML

2018-01-23 Thread Tom Lane
"David G. Johnston" writes: > On Tuesday, January 23, 2018, Bruce Momjian wrote: >> All agreed, but what alternatives are being developed? > I seem to recall a proposal a while back to gain margin on some of the > limits by pruning the release notes

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-23 Thread Simon Riggs
On 24 January 2018 at 01:35, Peter Geoghegan wrote: > On Tue, Jan 23, 2018 at 5:51 AM, Simon Riggs wrote: >>> Not yet working >>> * Partitioning >>> * RLS >>> >>> Based on this successful progress I imagine I'll be looking to commit >>> this by the end of the

Re: [PATCH] fix for C4141 warning on MSVC

2018-01-23 Thread Tom Lane
Thomas Munro writes: > Here's one like that. Pushed; we'll soon see if the buildfarm likes it. I added a tweak to prevent forced inlining at -O0, as discussed in the other thread; and worked on the comments a bit. regards, tom lane

Re: Incorrect comments in partition.c

2018-01-23 Thread Amit Langote
On 2018/01/23 20:43, Etsuro Fujita wrote: > Here is a comment for get_qual_for_list in partition.c: > > * get_qual_for_list > * > * Returns an implicit-AND list of expressions to use as a list partition's > - * constraint, given the partition key and bound structures. > > I don't think the

Re: PATCH: Configurable file mode mask

2018-01-23 Thread David Steele
On 1/23/18 9:22 PM, Michael Paquier wrote: > On Tue, Jan 23, 2018 at 09:18:51AM -0500, David Steele wrote: >> On 1/20/18 5:47 PM, Michael Paquier wrote: >>> Making this possible would require first some >>> refactoring of PostgresNode.pm so as a node is aware of the binary paths >>> it uses to be

Re: documentation is now XML

2018-01-23 Thread David G. Johnston
On Tuesday, January 23, 2018, Bruce Momjian wrote: > On Tue, Jan 23, 2018 at 10:22:53PM -0500, Tom Lane wrote: > (a) it's got hard > > limits we're approaching, > All agreed, but what alternatives are being developed? > > I seem to recall a proposal a while back to gain

Re: documentation is now XML

2018-01-23 Thread Tom Lane
Bruce Momjian writes: > On Thu, Nov 23, 2017 at 03:39:24PM -0500, Tom Lane wrote: >> Also, we're way overdue for getting out from under the creaky TeX-based >> toolchain for producing PDFs. > I am coming in late here, but I am not aware of any open source > professional

Re: copy.c allocation constant

2018-01-23 Thread Bruce Momjian
On Tue, Nov 28, 2017 at 11:51:28AM -0500, Andrew Dunstan wrote: > > > While reading copy.c I noticed this line: > > > #define RAW_BUF_SIZE 65536        /* we palloc RAW_BUF_SIZE+1 bytes */ > > > Doesn't that seem rather odd? If we're adding 1 wouldn't it be better as > 65535 so we palloc a

Re: pgsql: Allow UPDATE to move rows between partitions.

2018-01-23 Thread Amit Kapila
On Tue, Jan 23, 2018 at 9:59 PM, Robert Haas wrote: > On Tue, Jan 23, 2018 at 8:44 AM, Amit Kapila wrote: >> On Sat, Jan 20, 2018 at 2:03 AM, Robert Haas wrote: >>> Allow UPDATE to move rows between partitions. >> >> +If

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Amit Kapila
On Wed, Jan 24, 2018 at 2:35 AM, Robert Haas wrote: > On Tue, Jan 23, 2018 at 11:15 AM, Robert Haas wrote: >> On Mon, Jan 22, 2018 at 10:56 PM, Amit Kapila >> wrote: >> >> OK. I've committed this patch and back-patched it

Re: Handling better supported channel binding types for SSL implementations

2018-01-23 Thread Michael Paquier
On Tue, Jan 23, 2018 at 12:08:37PM -0500, Peter Eisentraut wrote: > On 1/22/18 02:29, Michael Paquier wrote: >> However there is as well the argument that this list's contents are not >> directly used now, and based on what I saw from the MacOS SSL and GnuTLS >> patches that would not be the case

Re: PATCH: Configurable file mode mask

2018-01-23 Thread Michael Paquier
On Tue, Jan 23, 2018 at 09:18:51AM -0500, David Steele wrote: > On 1/20/18 5:47 PM, Michael Paquier wrote: >> Making this possible would require first some >> refactoring of PostgresNode.pm so as a node is aware of the binary paths >> it uses to be able to manipulate multiple instances with

Re: [PATCH] fix for C4141 warning on MSVC

2018-01-23 Thread Thomas Munro
On Wed, Jan 24, 2018 at 1:42 PM, Tom Lane wrote: > Thomas Munro writes: >> On Wed, Jan 24, 2018 at 1:16 PM, Michail Nikolaev >> wrote: >>> Just very small fix for C4141 warning > >> Thanks. This is similar to the

Re: Failed to request an autovacuum work-item in silence

2018-01-23 Thread Masahiko Sawada
On Tue, Jan 23, 2018 at 8:03 PM, Fabrízio de Royes Mello wrote: > > Em ter, 23 de jan de 2018 às 03:36, Masahiko Sawada > escreveu: >> >> Hi all, >> >> While reading the code, I realized that the requesting an autovacuum >> work-item could fail in

Re: pgsql: Add parallel-aware hash joins.

2018-01-23 Thread Thomas Munro
On Wed, Jan 24, 2018 at 12:10 PM, Tom Lane wrote: > There is a very clear secular trend up in the longer data series, > which indicates that we're testing more stuff, +1 > which doesn't bother > me in itself as long as the time is well spent. However, the trend > over the

Re: Remove utils/dsa.h from autovacuum.c

2018-01-23 Thread Masahiko Sawada
On Wed, Jan 24, 2018 at 3:39 AM, Alvaro Herrera wrote: > Masahiko Sawada wrote: >> Hi, >> >> Attached a patch for $subject. The implementation of autovacuum >> work-item has been changed by commit >> 31ae1638ce35c23979f9bcbb92c6bb51744dbccb but the loading of dsa.h >>

Re: Missing wal_receiver_status_interval in Subscribers section

2018-01-23 Thread Masahiko Sawada
On Wed, Jan 24, 2018 at 9:34 AM, Bruce Momjian wrote: > On Tue, Jan 23, 2018 at 06:36:04PM -0500, Bruce Momjian wrote: >> >> Can someone confirm this so I can apply this patch? > > Never mind. I see this was applied: > Thank you for your kind response. Regards, -- Masahiko

Re: [HACKERS] Refactoring identifier checks to consistently use strcmp

2018-01-23 Thread Michael Paquier
On Tue, Jan 23, 2018 at 04:22:22PM +0100, Daniel Gustafsson wrote: > On 23 Jan 2018, at 05:52, Michael Paquier wrote: >> 2) indexam.sgml mentions using pg_strcasecmp() for consistency with the >> core code when defining amproperty for an index AM. Well, with this >>

RE: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

2018-01-23 Thread Tsunakawa, Takayuki
From: Robert Haas [mailto:robertmh...@gmail.com] > Oh, incidentally -- in our internal testing, we found that > wal_sync_method=open_datasync was significantly faster than > wal_sync_method=fdatasync. You might find that open_datasync isn't much > different from pmem_drain, even though they're

Re: documentation is now XML

2018-01-23 Thread Bruce Momjian
On Thu, Nov 23, 2017 at 03:39:24PM -0500, Tom Lane wrote: > Also, we're way overdue for getting out from under the creaky TeX-based > toolchain for producing PDFs. Every time we make releases, I worry > whether we're going to get blindsided by its bugs with hotlinks that get > split across pages,

Re: [PATCH] fix for C4141 warning on MSVC

2018-01-23 Thread Thomas Munro
On Wed, Jan 24, 2018 at 1:16 PM, Michail Nikolaev wrote: > Just very small fix for C4141 warning > (https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-1-c4141). > > Also could be viewed on Github - >

Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

2018-01-23 Thread David Rowley
On 23 January 2018 at 23:22, Amit Langote wrote: > On 2018/01/23 15:44, David Rowley wrote: >> Attached is what I had in mind about how to do this. > > Thanks for the delta patch. I will start looking at it tomorrow. Thanks. I've been looking more at this and I've

Re: Missing wal_receiver_status_interval in Subscribers section

2018-01-23 Thread Bruce Momjian
Can someone confirm this so I can apply this patch? --- On Fri, Nov 17, 2017 at 06:34:29PM +0900, Masahiko Sawada wrote: > Hi, > > I found that the doc says in 19.6.4. Subscribers section that "Note > that

Re: [HACKERS] Planning counters in pg_stat_statements

2018-01-23 Thread Tom Lane
Thomas Munro writes: > One problem is that pgss_planner_hook doesn't have the source query > text. That problem could be solved by adding a query_string argument > to the planner hook function type and also planner(), > standard_planner(), pg_plan_query(),

Re: pgsql: Add parallel-aware hash joins.

2018-01-23 Thread Tom Lane
Robert Haas writes: > On Mon, Jan 22, 2018 at 6:53 PM, Tom Lane wrote: >> Here's a possibly more useful graph of regression test timings over >> the last year. I pulled this from the buildfarm database: it is the >> reported runtime for the

Re: [HACKERS] Planning counters in pg_stat_statements

2018-01-23 Thread Thomas Munro
On Fri, Jan 19, 2018 at 12:14 AM, Thomas Munro wrote: > On Sat, Jan 13, 2018 at 2:16 PM, Tom Lane wrote: >> What we could/should do instead, I think, is have pgss_planner_hook >> make its own pgss_store() call to log the planning time results >>

For consideration - clarification of multi-dimensional arrays in our docs.

2018-01-23 Thread David G. Johnston
Hey all! This doesn't come up that often but enough that it seems hammering home that multi-dimension <> array-of-array seems warranted. The first and last chuck cover definition and iteration respectively. The second chuck removes the mention of "subarray" since that's what we don't want

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Peter Geoghegan
On Tue, Jan 23, 2018 at 1:05 PM, Robert Haas wrote: > I was just talking to Thomas, and one or the other of us realized that > this doesn't completely fix the problem. I think that if a worker > fails, even by some unorthodox mechanism like an abrupt proc_exit(1), > after

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Thomas Munro
On Fri, Jan 19, 2018 at 6:22 AM, Robert Haas wrote: >> (3) >> erm, maybe it's a problem that errors occurring in workers while the >> leader is waiting at a barrier won't unblock the leader (we don't >> detach from barriers on abort/exit) -- I'll look into this. > > I think

Re: Libpq support to connect to standby server as priority

2018-01-23 Thread Jing Wang
Hi All, Recently I put a proposal to support 'prefer-read' parameter in target_session_attrs in libpq. Now I updated the patch with adding content in the sgml and regression test case. Some people may have noticed there is already another patch ( https://commitfest.postgresql.org/15/1148/ )

Tab complete after FROM ONLY

2018-01-23 Thread Vik Fearing
While reviewing the patch for tab completion after SELECT, I realized that psql doesn't know how to complete after FROM if the ONLY keyword is present. This trivial patch fixes that, as well as JOIN ONLY and TABLE ONLY. -- Vik Fearing +33 6 46 75 15 36

Re: Using ProcSignal to get memory context stats from a running backend

2018-01-23 Thread Alvaro Herrera
Did this go anywhere? -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: WIP: BRIN multi-range indexes

2018-01-23 Thread Tomas Vondra
On 01/23/2018 09:05 PM, Alvaro Herrera wrote: > This stuff sounds pretty nice. However, have a look at this report: > > https://codecov.io/gh/postgresql-cfbot/postgresql/commit/2aa632dae3066900e15d2d42a4aad811dec11f08 > > it seems to me that the new code is not tested at all. Shouldn't you >

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Robert Haas
On Tue, Jan 23, 2018 at 2:11 PM, Peter Geoghegan wrote: > Finally, it's still not clear to me why nodeGather.c's use of > parallel_leader_participation=off doesn't suffer from similar problems > [1]. Thomas and I just concluded that it does. See my email on the other thread just

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Robert Haas
On Tue, Jan 23, 2018 at 11:15 AM, Robert Haas wrote: > On Mon, Jan 22, 2018 at 10:56 PM, Amit Kapila wrote: >> It does turn such cases into error but only at the end when someone >> calls WaitForParallelWorkersToFinish. If one tries to rely on it

Re: [HACKERS] Support for Secure Transport SSL library on macOS as OpenSSL alternative

2018-01-23 Thread Peter Eisentraut
On 1/23/18 14:59, Daniel Gustafsson wrote: > It’s not specific to the implementation per se, but it increases the > likelyhood > of hitting it. In order to load certificates from Keychains the cert common > name must be specified in the connstr, when importing the testfiles into > keychains I

Re: WIP: BRIN multi-range indexes

2018-01-23 Thread Alvaro Herrera
This stuff sounds pretty nice. However, have a look at this report: https://codecov.io/gh/postgresql-cfbot/postgresql/commit/2aa632dae3066900e15d2d42a4aad811dec11f08 it seems to me that the new code is not tested at all. Shouldn't you add a few more tests? I think 0004 should apply to

Re: [HACKERS] Support for Secure Transport SSL library on macOS as OpenSSL alternative

2018-01-23 Thread Daniel Gustafsson
> On 23 Jan 2018, at 18:20, Peter Eisentraut > wrote: > > On 1/21/18 18:08, Daniel Gustafsson wrote: >> As per before, my patch for running tests against another set of binaries is >> included as well as a fix for connstrings with spaces, but with the recent >>

pg_rewind and replication slots

2018-01-23 Thread Laurenz Albe
When a primary with replication slots gets reset with pg_rewind, it keeps the replication slots. This does no harm per se, but when it gets promoted again, the replication slots are still there and are in the way. Won't they also hold back the xmin horizon? I think that pg_rewind should delete

Re: pgsql: Add parallel-aware hash joins.

2018-01-23 Thread Robert Haas
On Mon, Jan 22, 2018 at 6:53 PM, Tom Lane wrote: > Here's a possibly more useful graph of regression test timings over > the last year. I pulled this from the buildfarm database: it is the > reported runtime for the "installcheck-C" step in each successful > build of HEAD on

Re: Logical Decoding and HeapTupleSatisfiesVacuum assumptions

2018-01-23 Thread Petr Jelinek
On 23/01/18 16:01, Nikhil Sontakke wrote: >> I must be missing something in this discussion, cos I don't see any >> problems with this "other option". >> >> Surely we prepare the 2PCxact and then it is persistent. Yes, >> potentially for an unbounded period of time. And it is (usually) up to >>

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Peter Geoghegan
On Tue, Jan 23, 2018 at 10:50 AM, Peter Geoghegan wrote: > On Tue, Jan 23, 2018 at 10:36 AM, Robert Haas wrote: >> I am going to repeat my previous suggest that we use a Barrier here. >> Given the discussion subsequent to my original proposal, this can be a

Re: using index or check in ALTER TABLE SET NOT NULL

2018-01-23 Thread Sergei Kornilov
Hello Stephen I changed the suggested things and added some regression tests. Also i wrote few words to the documentation. New patch is attached. Thank you for feedback! regards, Sergeidiff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 286c7a8..cf2c56f

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Peter Geoghegan
On Tue, Jan 23, 2018 at 10:36 AM, Robert Haas wrote: > As Amit says, what remains is the case where fork() fails or the > worker dies before it reaches the line in ParallelWorkerMain that > reads shm_mq_set_sender(mq, MyProc). In those cases, no error will be > signaled

Re: [PATCH][PROPOSAL] Refuse setting toast.* reloptions when TOAST table does not exist

2018-01-23 Thread Nikolay Shaplov
В письме от 23 января 2018 12:03:50 пользователь Peter Eisentraut написал: > >>> This patch raises error if user tries o set or change toast.* option for > >>> a table that does not have a TOST relation. > > > > There might be a case for raising a warning in this situation, > > but I would

Re: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

2018-01-23 Thread Robert Haas
On Fri, Jan 19, 2018 at 9:42 AM, Robert Haas wrote: > That's not necessarily an argument against this patch, which by the > way I have not reviewed. Even a 5% speedup on this kind of workload > is potentially worthwhile; everyone likes it when things go faster. > I'm just

Re: Remove utils/dsa.h from autovacuum.c

2018-01-23 Thread Alvaro Herrera
Masahiko Sawada wrote: > Hi, > > Attached a patch for $subject. The implementation of autovacuum > work-item has been changed by commit > 31ae1638ce35c23979f9bcbb92c6bb51744dbccb but the loading of dsa.h > header file is remained. You're right -- pushed. -- Álvaro Herrera

Re: [PATCH][PROPOSAL] Refuse setting toast.* reloptions when TOAST table does not exist

2018-01-23 Thread Robert Haas
On Tue, Jan 23, 2018 at 12:03 PM, Peter Eisentraut wrote: > It might also be useful to set these reloptions before you add a new > column to a table. I don't understand. AAUI, it is currently the case that if you set the options before the TOAST table exists,

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Robert Haas
On Mon, Jan 22, 2018 at 10:13 PM, Peter Geoghegan wrote: > _bt_leader_heapscan() can detect when workers exit early, at least in > the vast majority of cases. It can do this simply by processing > interrupts and automatically propagating any error -- nothing special > about that. It

Possible performance regression in version 10.1 with pgbench read-write tests.

2018-01-23 Thread Mithun Cy
Hi all, When I was trying to do read-write pgbench bench-marking of PostgreSQL 9.6.6 vs 10.1 I found PostgreSQL 10.1 regresses against 9.6.6 in some cases. Non Default settings and test == Server: ./postgres -c shared_buffers=8GB -N 200 -c min_wal_size=15GB -c

Re: Doc tweak for huge_pages?

2018-01-23 Thread Catalin Iacob
On Mon, Jan 22, 2018 at 7:23 AM, Justin Pryzby wrote: > Consider this shorter, less-severe sounding alternative: > "... (but note that this feature can degrade performance of some > PostgreSQL workloads)." I think the patch looks good now. As Justin mentions, as far as I

Re: [PATCH] Logical decoding of TRUNCATE

2018-01-23 Thread Petr Jelinek
Hi, On 23/01/18 18:47, Marco Nenciarini wrote: > Il 23/01/18 18:25, Petr Jelinek ha scritto: >> On 23/01/18 18:19, Marco Nenciarini wrote: >>> Il 23/01/18 18:13, Petr Jelinek ha scritto: Hi, On 23/01/18 15:38, Marco Nenciarini wrote: > Il 22/01/18 23:18, Petr Jelinek ha

Re: pgsql: Improve scripting language in pgbench

2018-01-23 Thread Peter Eisentraut
On 1/10/18 07:36, Fabien COELHO wrote: > I just noticed while rebasing stuff that there is some crust in > "pgbench/t/001_pgbench_with_server.pl" coming from this patch: > > +=head > + > +} }); > + > +=cut > > I cannot find any use for these lines which are ignored by perl execution

Re: [PATCH] Logical decoding of TRUNCATE

2018-01-23 Thread Petr Jelinek
On 23/01/18 18:19, Marco Nenciarini wrote: > Il 23/01/18 18:13, Petr Jelinek ha scritto: >> Hi, >> >> On 23/01/18 15:38, Marco Nenciarini wrote: >>> Il 22/01/18 23:18, Petr Jelinek ha scritto: On 22/01/18 19:45, Petr Jelinek wrote: Actually on second look, I don't like the new

Re: [PATCH] Logical decoding of TRUNCATE

2018-01-23 Thread Marco Nenciarini
Il 23/01/18 18:13, Petr Jelinek ha scritto: > Hi, > > On 23/01/18 15:38, Marco Nenciarini wrote: >> Il 22/01/18 23:18, Petr Jelinek ha scritto: >>> On 22/01/18 19:45, Petr Jelinek wrote: >>> >>> Actually on second look, I don't like the new boolean parameter much. >>> I'd rather we didn't touch

Re: [HACKERS] Support for Secure Transport SSL library on macOS as OpenSSL alternative

2018-01-23 Thread Peter Eisentraut
On 1/21/18 18:08, Daniel Gustafsson wrote: > As per before, my patch for running tests against another set of binaries is > included as well as a fix for connstrings with spaces, but with the recent > hacking by Peter I assume this is superfluous. It was handy for development > so > I’ve kept it

Re: [PATCH] Logical decoding of TRUNCATE

2018-01-23 Thread Petr Jelinek
Hi, On 23/01/18 15:38, Marco Nenciarini wrote: > Il 22/01/18 23:18, Petr Jelinek ha scritto: >> On 22/01/18 19:45, Petr Jelinek wrote: >> >> Actually on second look, I don't like the new boolean parameter much. >> I'd rather we didn't touch the input list and always close only >> relations opened

Re: Handling better supported channel binding types for SSL implementations

2018-01-23 Thread Peter Eisentraut
On 1/22/18 02:29, Michael Paquier wrote: > However there is as well the argument that this list's contents are not > directly used now, and based on what I saw from the MacOS SSL and GnuTLS > patches that would not be the case after either. Right, there is no facility for negotiating the channel

Re: Test-cases for exclusion constraints is missing in alter_table.sql file

2018-01-23 Thread Peter Eisentraut
On 1/17/18 23:57, Ashutosh Sharma wrote: > By elsewhere do you mean in the contrib module 'sepgsql' > (sepgsql/sql/ddl.sql) because other than that i didn't find any other > places having ALTER TABLE ... ADD CONSTRAINT ... EXCLUDE ... sort of > sql queries other than the files which i mentioned in

Re: [PATCH][PROPOSAL] Refuse setting toast.* reloptions when TOAST table does not exist

2018-01-23 Thread Peter Eisentraut
On 1/18/18 18:42, Tom Lane wrote: > Robert Haas writes: >> On Wed, Jan 17, 2018 at 3:50 PM, Nikolay Shaplov wrote: >>> This patch raises error if user tries o set or change toast.* option for a >>> table that does not have a TOST relation. > There might

Re: PATCH: Configurable file mode mask

2018-01-23 Thread Peter Eisentraut
On 1/23/18 09:33, David Steele wrote: > What if I update pg_upgrade/test.sh to optionally allow group > permissions and we configure an animal to test it if it gets committed? > > It's not ideal, I know, but it would get the permissions patch over the > line and is consistent with how we

Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall

2018-01-23 Thread Tom Lane
Bruce Momjian writes: > Oh, I see what you mean. I was just worried that some code might expect > template1 to always have an oid of 1, but we can just call that code > broken. Ever since we invented template0, it's been possible to drop and recreate template1, so I'd say any

Re: [PATCH] Logical decoding of TRUNCATE

2018-01-23 Thread Marco Nenciarini
Il 22/01/18 23:18, Petr Jelinek ha scritto: > On 22/01/18 19:45, Petr Jelinek wrote: > > Actually on second look, I don't like the new boolean parameter much. > I'd rather we didn't touch the input list and always close only > relations opened inside the ExecuteTruncateGuts(). > > It may mean

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

2018-01-23 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> The most obvious hack is just to exclude PL objects with OIDs below 16384 >> from the dump. An issue with that is that it'd lose any nondefault >> changes to the ACL for plpgsql, which seems like a problem. On

Re: PATCH: Configurable file mode mask

2018-01-23 Thread Peter Eisentraut
On 1/19/18 16:54, Alvaro Herrera wrote: > If the only problem is that buildfarm would run tests twice, then I > think we should just press forward with this regardless of that: it > seems a chicken-and-egg problem, because buildfarm cannot be upgraded to > use some new test method if the method

Fwd: Regarding ambulkdelete, amvacuumcleanup index methods

2018-01-23 Thread Abinaya k
-- Forwarded message -- From: "Abinaya k" Date: Jan 23, 2018 1:43 PM Subject: Regarding ambulkdelete, amvacuumcleanup index methods To: Cc: Hai all, We are building In-memory index extension for postgres. We would

Re: [HACKERS] taking stdbool.h into use

2018-01-23 Thread Peter Eisentraut
On 1/16/18 01:14, Michael Paquier wrote: >> I don't see it either. I think the actually useful parts of that patch >> were to declare record_image_cmp's result correctly as int rather than >> bool, and to cope with varlena fields of unequal size. Unfortunately >> there seems to be no

Re: [HACKERS] PoC: full merge join on comparison clause

2018-01-23 Thread Stephen Frost
Greetings Alexander, * Alexander Kuzmenkov (a.kuzmen...@postgrespro.ru) wrote: > Here is the patch rebased to a852cfe9. Thanks for updating it. This would definitely be nice to have. Ashutosh, thanks for your previous review, would you have a chance to look at it again? Would be great to at

Re: pgsql: Allow UPDATE to move rows between partitions.

2018-01-23 Thread Robert Haas
On Tue, Jan 23, 2018 at 8:44 AM, Amit Kapila wrote: > On Sat, Jan 20, 2018 at 2:03 AM, Robert Haas wrote: >> Allow UPDATE to move rows between partitions. > > +If an UPDATE on a partitioned table causes a row to > move > +to another

Re: pgsql: Move handling of database properties from pg_dumpall into pg_dum

2018-01-23 Thread Tom Lane
Haribabu Kommi writes: > On Tue, Jan 23, 2018 at 8:56 AM, Tom Lane wrote: >> I'm thinking we'd still need to do what I said in the previous message, >> and change pg_dump so that the restore session will absorb ALTER DATABASE >> settings before

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Robert Haas
On Mon, Jan 22, 2018 at 10:56 PM, Amit Kapila wrote: > It does turn such cases into error but only at the end when someone > calls WaitForParallelWorkersToFinish. If one tries to rely on it at > any other time, it won't work as I think is the case for Parallel > Create

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2018-01-23 Thread Pavel Stehule
2018-01-22 23:15 GMT+01:00 Stephen Frost : > Pavel, > > * Pavel Stehule (pavel.steh...@gmail.com) wrote: > > here is a GUC based patch for plancache controlling. Looks so this code > is > > working. > > This really could use a new thread, imv. This thread is a year old and >

Re: Invalidation pg catalog cache in trigger functions

2018-01-23 Thread Константин Евтеев
Sorry, I forgot to show version of PostgreSQL: https://www.postgresql.org/message-id/CAAqA9PQXEmG%3Dk3WpDTmHZL-VKcMpDEA3ZC06Qr0ASO3oTA7bdw%40mail.gmail.com "Invalidation pg catalog cache in trigger functions" was detected on "PostgreSQL 9.2.18 on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu

Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall

2018-01-23 Thread Bruce Momjian
On Sun, Jan 21, 2018 at 11:02:29AM -0500, Tom Lane wrote: > Bruce Momjian writes: > > On Fri, Jan 19, 2018 at 02:54:25PM -0500, Tom Lane wrote: > >> Command was: DROP DATABASE "template1"; > > > Uh, the oid of the template1 database is 1, and I assume we would want > > to

  1   2   >