[HACKERS] Latches API on windows

2017-10-09 Thread Abbas Butt
Hi, I am working on a contrib module that uses RegisterDynamicBackgroundWorker API to create a couple of worker processes. For synchronization between the background worker processes I am using InitSharedLatch, SetLatch, WaitLatch APIs. One of the processes is supposed to wait for the latch, the

Re: [HACKERS] Discussion on missing optimizations

2017-10-09 Thread Ants Aasma
On Sun, Oct 8, 2017 at 7:24 PM, Tom Lane wrote: > Petr Jelinek writes: >> Okay, that makes sense, thanks for explanation. Your patch is the way to >> go then. > > Hearing no further comment, pushed. Thanks for reviewing it. The tautological col

[HACKERS] 10.0: Logical replication doesn't execute BEFORE UPDATE OF trigger

2017-10-09 Thread Aleksander Alekseev
Hi hackers, I've found something that looks like a bug. Steps to reproduce -- There are 3 instances of PostgreSQL 10.0 - inst1, inst2 and inst3. There is a table `test` on every instance: ``` CREATE TABLE test(k TEXT PRIMARY KEY, v TEXT); ``` Both inst1 and inst2 have `allpub`

[HACKERS] oversight in EphemeralNamedRelation support

2017-10-09 Thread Julien Rouhaud
Hi, Hugo Mercier (in Cc) reported me an error in a query, failing since pg10. Simple test case to reproduce: CREATE TABLE public.test (id integer); WITH test AS (select 42) INSERT INTO public.test SELECT * FROM test; which will fail with "relation "test" cannot be the target of a modifying

Re: [HACKERS] Pluggable storage

2017-10-09 Thread Alexander Korotkov
On Mon, Oct 9, 2017 at 5:32 PM, Tom Lane wrote: > Alexander Korotkov writes: > > For me, it's crucial point that pluggable storages should be able to have > > different MVCC implementation, and correspondingly have full control over > > its

Re: [HACKERS] Latches API on windows

2017-10-09 Thread Andres Freund
On October 9, 2017 6:56:10 AM PDT, Tom Lane wrote: >Craig Ringer writes: >> On 9 October 2017 at 21:26, Abbas Butt >wrote: >>> In my case this is not true, I am calling InitSharedLatch in >_PG_init >>> which gets called

Re: [HACKERS] GSoC 2017 : Patch for predicate locking in Gist index

2017-10-09 Thread Alexander Korotkov
On Thu, Oct 5, 2017 at 9:48 PM, Shubham Barai wrote: > On 3 October 2017 at 00:32, Alexander Korotkov > wrote: > >> On Mon, Oct 2, 2017 at 9:11 PM, Andrew Borodin >> wrote: >> >>> On Mon, Oct 2, 2017 at 8:00 PM,

Re: [HACKERS] Latches API on windows

2017-10-09 Thread Tom Lane
Craig Ringer writes: > On 9 October 2017 at 21:26, Abbas Butt wrote: >> In my case this is not true, I am calling InitSharedLatch in _PG_init >> which gets called at CREATE EXTENSION time. >> My question : Is there a way to get the latches API

Re: [HACKERS] Pluggable storage

2017-10-09 Thread Tom Lane
Alexander Korotkov writes: > For me, it's crucial point that pluggable storages should be able to have > different MVCC implementation, and correspondingly have full control over > its interactions with indexes. > Thus, it would be good if we would get consensus on that

Re: [HACKERS] Latches API on windows

2017-10-09 Thread Craig Ringer
On 9 October 2017 at 21:26, Abbas Butt wrote: > Hi, > I am working on a contrib module that uses RegisterDynamicBackgroundWorker > API > to create a couple of worker processes. For synchronization between the > background worker processes I am using InitSharedLatch,

Re: [HACKERS] Pluggable storage

2017-10-09 Thread Alexander Korotkov
On Wed, Sep 27, 2017 at 7:51 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > I took a look on this patch. I've following notes for now. > > tuple_insert_hook tuple_insert; /* heap_insert */ >> tuple_update_hook tuple_update; /* heap_update */ >> tuple_delete_hook tuple_delete; /*

Re: [HACKERS] Latches API on windows

2017-10-09 Thread Abbas Butt
Thanks for the suggestion. On Mon, Oct 9, 2017 at 6:56 PM, Tom Lane wrote: > Craig Ringer writes: > > On 9 October 2017 at 21:26, Abbas Butt > wrote: > >> In my case this is not true, I am calling InitSharedLatch in

Re: [HACKERS] On markers of changed data

2017-10-09 Thread Andrey Borodin
Hi Michael! > 9 окт. 2017 г., в 17:28, Michael Paquier > написал(а): >> VM is WAL-logged [0] >> FSM is not [1] > > If you are willing to go down this road, here are my takes on the matter: > - Any LSN map should be in a different file than FSM and VM. The VM > uses 2

[HACKERS] Unnecessary complexity around CurrentResourceOwner changes?

2017-10-09 Thread Tom Lane
We have some dozens of places that transiently change CurrentResourceOwner and then set it back to what it had been. I happened to notice that, while many of these places have a PG_TRY block to ensure that the restore happens even if they lose control to an error, not all of them have one. My

[HACKERS] Columnar storage support

2017-10-09 Thread legrand legrand
Is there a chance that pluggable storage permits creation of a columnar rdbms as monetDB in PostgreSQL ? Thanks un advance for thé answer -- Sent from: http://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Columnar storage support

2017-10-09 Thread Joshua D. Drake
On 10/09/2017 01:03 PM, legrand legrand wrote: Is there a chance that pluggable storage permits creation of a columnar rdbms as monetDB in PostgreSQL ? Thanks un advance for thé answer The extension C-Store from Citus is probably what you are looking for. jD -- Sent from:

Re: [HACKERS] Columnar storage support

2017-10-09 Thread Jaime Casanova
On 9 October 2017 at 15:03, legrand legrand wrote: > Is there a chance that pluggable storage permits creation of a columnar rdbms > as monetDB in PostgreSQL ? pluggable storage is one of the pieces for it, yes... but is not as simple as that, IIUC -- Jaime

Re: [HACKERS] oversight in EphemeralNamedRelation support

2017-10-09 Thread Thomas Munro
On Tue, Oct 10, 2017 at 2:35 AM, Julien Rouhaud wrote: > Hugo Mercier (in Cc) reported me an error in a query, failing since pg10. > > Simple test case to reproduce: > > CREATE TABLE public.test (id integer); > WITH test AS (select 42) INSERT INTO public.test SELECT * FROM

Re: [HACKERS] oversight in EphemeralNamedRelation support

2017-10-09 Thread Julien Rouhaud
On Mon, Oct 9, 2017 at 10:43 PM, Thomas Munro wrote: > > I suppose we could consider moving the schemaname check into > getRTEForSpecialRelationType(), since otherwise both callers need to > do that (and as you discovered, one forgot). Thanks for the feedback.

Re: [HACKERS] Partition-wise aggregation/grouping

2017-10-09 Thread David Rowley
On 10 October 2017 at 01:10, Jeevan Chalke wrote: > Attached new patch set having HEAD at 84ad4b0 with all these review points > fixed. Let me know if I missed any thanks. I've only really skimmed over this thread and only opened the code enough to extract the

Re: [HACKERS] Support to COMMENT ON DATABASE CURRENT_DATABASE

2017-10-09 Thread Jing Wang
Hi I don't know why the previous email can't be linked with the original email webpage. It is weird. So supplementing following information for understanding: The original email link: https://www.postgresql.org/message-id/CAF3%2BxM%2BxSswcWQZMP1cjj12gPz8DXHcM9_fT1y-0fVzxi9pmOw%40mail.gmail.com

[HACKERS] heap/SLRU verification, relfrozenxid cut-off, and freeze-the-dead bug (Was: amcheck (B-Tree integrity checking tool))

2017-10-09 Thread Peter Geoghegan
On Sun, Oct 16, 2016 at 6:46 PM, Noah Misch wrote: >> Noah and I discussed possible future directions for amcheck in person >> recently. I would like to get Noah's thoughts again here on how a tool >> like amcheck might reasonably target other access methods for >>

Re: [HACKERS] search path security issue?

2017-10-09 Thread Fabrízio de Royes Mello
On Sun, Oct 8, 2017 at 3:34 PM, Joe Conway wrote: > > On 10/06/2017 12:52 AM, Magnus Hagander wrote: > > It would be a nice feature to have in general, like a "basic guc > > permissions" thing. At least allowing a superuser to prevent exactly > > this. You could argue the same

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-10-09 Thread Michael Paquier
On Mon, Oct 9, 2017 at 2:29 AM, Peter Geoghegan wrote: > On Sat, Oct 7, 2017 at 4:25 PM, Alvaro Herrera > wrote: >> Hmm, I think I added a random sleep (max. 100ms) right after the >> HeapTupleSatisfiesVacuum call in vacuumlazy.c (lazy_scan_heap), and that

Re: [HACKERS] Issues with logical replication

2017-10-09 Thread Stas Kelvich
> On 2 Oct 2017, at 19:59, Petr Jelinek wrote: > >> >> Program terminated with signal SIGABRT, Aborted. >> #0 0x7f3608f8ec37 in __GI_raise (sig=sig@entry=6) at >> ../nptl/sysdeps/unix/sysv/linux/raise.c:56 >> 56../nptl/sysdeps/unix/sysv/linux/raise.c: No

Re: [HACKERS] On markers of changed data

2017-10-09 Thread Peter Geoghegan
On Sat, Oct 7, 2017 at 6:34 AM, Stephen Frost wrote: >> > That’s actually what pg_rman is doing for what it calls incremental >> > backups (perhaps that would be differential backup in PG >> > terminology?), and the performance is bad as you can imagine. We could >> > have a

Re: [HACKERS] Slow synchronous logical replication

2017-10-09 Thread Konstantin Knizhnik
Thank you for explanations. On 08.10.2017 16:00, Craig Ringer wrote: I think it'd be helpful if you provided reproduction instructions, test programs, etc, making it very clear when things are / aren't related to your changes. It will be not so easy to provide some reproducing scenario,

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-10-09 Thread Ashutosh Bapat
On Sat, Oct 7, 2017 at 1:04 AM, Robert Haas wrote: > > Committed. I hope that makes things less red rather than more, > because I'm going to be AFK for a few hours anyway. > Here's the last patch, dealing with the dummy relations, rebased. With this fix every join order

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-10-09 Thread Wood, Dan
I’m unclear on what is being repro’d in 9.6. Are you getting the duplicate rows problem or just the reindex problem? Are you testing with asserts enabled(I’m not)? If you are getting the dup rows consider the code in the block in heapam.c that starts with the comment “replace multi by update

Re: [HACKERS] Partition-wise aggregation/grouping

2017-10-09 Thread Ashutosh Bapat
On Tue, Oct 10, 2017 at 3:15 AM, David Rowley wrote: > On 10 October 2017 at 01:10, Jeevan Chalke > wrote: >> Attached new patch set having HEAD at 84ad4b0 with all these review points >> fixed. Let me know if I missed any thanks. > >

[HACKERS] Extended statistics is not working on Vars hidden under a RelabelType

2017-10-09 Thread David Rowley
Basically, $subject is causing us not to properly find matching extended stats in this case. The attached patch fixes it. The following test cases is an example of the misbehaviour. Note rows=1 vs rows=98 in the Gather node. create table ab (a varchar, b varchar); insert into ab select

[HACKERS] Re: [BUGS] 10.0: Logical replication doesn't execute BEFORE UPDATE OF trigger

2017-10-09 Thread Masahiko Sawada
On Mon, Oct 9, 2017 at 11:13 PM, Aleksander Alekseev wrote: > Hi hackers, > > I've found something that looks like a bug. > > Steps to reproduce > -- > > There are 3 instances of PostgreSQL 10.0 - inst1, inst2 and inst3. There > is a table `test` on

[HACKERS] Fix a typo in execReplication.c

2017-10-09 Thread Masahiko Sawada
Hi, Attached a patch for $subject. ISTM these are mistakes of copy-and-paste. Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center fix_comment_in_execReplication_c.patch Description: Binary data -- Sent via pgsql-hackers mailing list

Re: [HACKERS] [POC] hash partitioning

2017-10-09 Thread amul sul
On Sat, Oct 7, 2017 at 5:22 PM, amul sul wrote: > On Fri, Oct 6, 2017 at 5:35 PM, Jesper Pedersen > wrote: >> Hi Amul, >> >> Could you rebase on latest master ? >> > > Sure will post that soon, but before that, I need to test hash partitioning >

Re: [HACKERS] Predicate Locks for writes?

2017-10-09 Thread Alexander Korotkov
On Sat, Oct 7, 2017 at 2:26 PM, Simon Riggs wrote: > SERIALIZABLE looks for chains of rw cases. > > When we perform UPDATEs and DELETEs we search for rows and then modify > them. The current implementation views that as a read followed by a > write because we issue

Re: [HACKERS] Parallel Append implementation

2017-10-09 Thread Amit Kapila
On Fri, Oct 6, 2017 at 12:03 PM, Amit Khandekar wrote: > On 6 October 2017 at 08:49, Amit Kapila wrote: >> >> Okay, but why not cheapest partial path? > > I gave some thought on this point. Overall I feel it does not matter > which partial path it

Re: [HACKERS] On markers of changed data

2017-10-09 Thread Michael Paquier
On Mon, Oct 9, 2017 at 2:23 PM, Andrey Borodin wrote: >> I haven't gone and audited it myself, but I would certainly expect you >> to be able to use the LSN for everything which is WAL'd. If you have >> cases where that's not the case, it'd be useful to see them. > >

Re: [HACKERS] PoC: full merge join on comparison clause

2017-10-09 Thread Ashutosh Bapat
Hi Alexander, Commit c7a9fa399 has added another test on mergeopfamilies. I think your patch will need to take care of that test. On Wed, Oct 4, 2017 at 6:38 PM, Alexander Kuzmenkov wrote: > As discussed earlier, I changed the way we work with mergeopfamilies. I use >

Re: [HACKERS] Partition-wise aggregation/grouping

2017-10-09 Thread Jeevan Chalke
On Thu, Sep 28, 2017 at 3:12 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > > Here are comments on 0004 from last patch set. But most the comments > still apply. > Thank you, Ashutosh for reviewing. > > Patch 0001 adds functions create_hash_agg_path() and >

Re: [HACKERS] [POC] hash partitioning

2017-10-09 Thread Ashutosh Bapat
On Mon, Oct 9, 2017 at 4:44 PM, amul sul wrote: > 0002 few changes in partition-wise join code to support > hash-partitioned table as well & regression tests. +switch (key->strategy) +{ +case PARTITION_STRATEGY_HASH: +/* + * Indexes

Re: [HACKERS] separate serial_schedule useful?

2017-10-09 Thread Ashutosh Bapat
On Sat, Oct 7, 2017 at 10:19 PM, Tom Lane wrote: > I wrote: >> Robert Haas writes: Sorry, my bad. I wasn't aware of this rule. I should have looked at the beginning of the file for any rules. >>> There's no reason why pg_regress couldn't have a >>>

Re: [HACKERS] parallelize queries containing initplans

2017-10-09 Thread Amit Kapila
On Sat, Oct 7, 2017 at 7:22 PM, Robert Haas wrote: > On Fri, Oct 6, 2017 at 7:08 AM, Amit Kapila wrote: >> I have fixed the other review comment related to using safe_param_list >> in the attached patch. I think I have fixed all comments given by

Re: [HACKERS] Proposal: Local indexes for partitioned table

2017-10-09 Thread Ashutosh Bapat
On Sat, Oct 7, 2017 at 7:04 PM, Robert Haas wrote: > On Fri, Oct 6, 2017 at 12:37 PM, Alvaro Herrera > wrote: >> 2. create one index for each existing partition. These would be >>identical to what would happen if you created the index