Is PREPARE of ecpglib thread safe?

2019-03-14 Thread Matsumura, Ryo
Hi I'm afraid that PREPARE of ecpglib is not thread safe. The following global variables are modified without any locking system. Is it unnecessary worry? [interfaces/ecpg/ecpglib/prepare.c] static int nextStmtID = 1; static stmtCacheEntry *stmtCacheEntries = NULL; static struct

Re: Tid scan improvements

2019-03-14 Thread Edmund Horner
On Thu, 14 Mar 2019 at 16:46, David Rowley wrote: > > The only possible risk I can foresee is that it may be more likely we > underestimate the selectivity and that causes something like a nested > loop join due to the estimation being, say 1 row. > > It could happen in a case like: > > SELECT *

Re: Implementing Incremental View Maintenance

2019-03-14 Thread Mitar
Hi! On Thu, Jan 31, 2019 at 6:20 AM Yugo Nagata wrote: > BTW, what is uecase of reactive/live queries? (just curious) It allows syncing the state between client and server. Client can then have a subset of data and server can push changes as they are happening to the client. Client can in a

RE: Timeout parameters

2019-03-14 Thread Tsunakawa, Takayuki
From: Kyotaro HORIGUCHI [mailto:horiguchi.kyot...@lab.ntt.co.jp] > If so, in turn the socket_timeout doesn't work as expected? I > understand that what is proposed here is to disconnect after that > time of waiting for *the first tuple* of a query, regardless of > it is a long query or network

Re: jsonpath

2019-03-14 Thread Alexander Korotkov
On Sun, Mar 10, 2019 at 1:51 PM Alexander Korotkov wrote: > On Wed, Mar 6, 2019 at 12:40 AM Nikita Glukhov > wrote: > > Attached 36th version of the patches. > > Thank yo for the revision! > > In the attached revision following changes are made: > > > "unknown" refers here to ordinary

Re: Special role for subscriptions

2019-03-14 Thread Andrey Borodin
> 14 марта 2019 г., в 12:56, Evgeniy Efimkin > написал(а): > > Hi! >> I view that as the first step towards building a more granular privilege >> system for subscription creation, and that was the second half of what I >> was trying to say before- I do think there's value in having something

RE: Timeout parameters

2019-03-14 Thread MikalaiKeida
Hello, all. The main subject of discussion in this thread relates to the 'socket_timeout'. As I understand there is no any hesitation about applying TCP_USER_TIMEOUT into the PostgreSQL. We have been waiting for applying TCP_USER_TIMEOUT in PostgreSQL for about 6 moths. Fabien, I was wondering

Re: Is PREPARE of ecpglib thread safe?

2019-03-14 Thread Kyotaro HORIGUCHI
At Thu, 14 Mar 2019 09:49:11 +, "Matsumura, Ryo" wrote in <03040DFF97E6E54E88D3BFEE5F5480F737AC3AD8@G01JPEXMBYT04> > Horiguchi-san > > Thank you for your comment. > > > A connection cannot be concurrently used by multiple threads so > > the programmer must guard connections using mutex

Re: WIP: Avoid creation of the free space map for small tables

2019-03-14 Thread Amit Kapila
On Thu, Mar 14, 2019 at 7:08 AM John Naylor wrote: > > > [segfault problems] > > This now seems spurious. I ran make distclean, git pull, reapplied the > patch (leaving out the gettimeofday() calls), and now my upgrade perf > test works with default compiler settings. Not sure what happened, but

Re: [HACKERS] Block level parallel vacuum

2019-03-14 Thread Masahiko Sawada
On Thu, Mar 14, 2019 at 6:41 AM Robert Haas wrote: > > On Wed, Mar 13, 2019 at 1:56 AM Masahiko Sawada wrote: > > I don't have a strong opinion but the using a Node would be more > > suitable in the future when we add more options to vacuum. And it > > seems to me that it's unlikely to change a

Re: Re: Feature: triggers on materialized views

2019-03-14 Thread Mitar
Hi! On Thu, Mar 7, 2019 at 12:13 AM David Steele wrote: > There doesn't seem to be consensus on whether or not we want this patch. > Peter has issues with the way it works and Andres [1] thinks it should > be pushed to PG13 or possibly rejected. > > I'll push this to PG13 for now. Sorry, I am

Re: Is PREPARE of ecpglib thread safe?

2019-03-14 Thread Kyotaro HORIGUCHI
Hello. At Thu, 14 Mar 2019 07:17:08 +, "Matsumura, Ryo" wrote in <03040DFF97E6E54E88D3BFEE5F5480F737AC390D@G01JPEXMBYT04> > Hi > > I'm afraid that PREPARE of ecpglib is not thread safe. > The following global variables are modified without any locking system. > Is it unnecessary worry? >

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Amit Langote
On 2019/03/14 16:46, Amit Langote wrote: > On 2019/03/14 16:32, Kyotaro HORIGUCHI wrote: >> At Thu, 14 Mar 2019 16:13:23 +0900, Amit Langote >> wrote in >> <3ad792cd-0805-858e-595c-c09e9d1ce...@lab.ntt.co.jp> >>> Hi, >>> >>> I'm curious why DestroyPartitionDirectory doesn't do >>>

RE: speeding up planning with partitions

2019-03-14 Thread Imai, Yoshikazu
Amit-san, I have done code review of v31 patches from 0004 to 0008. 0004: * s/childern/children 0005: * This seems reasonable for not using a lot of memory in specific case, although it needs special looking of planner experts. 0006: * The codes initializing/setting RelOptInfo's part_rels

Re: Tid scan improvements

2019-03-14 Thread David Rowley
On Thu, 14 Mar 2019 at 21:12, Edmund Horner wrote: > I'm not sure how an unreasonable underestimation would occur here. If > you have a table bloated to say 10x its minimal size, the estimator > still assumes an even distribution of tuples (I don't think we can do > much better than that). So

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-03-14 Thread Pavan Deolasee
On Wed, Mar 13, 2019 at 11:37 AM Masahiko Sawada wrote: > > > I think that since COPY FREEZE can be executed only when the table is > created or truncated within the transaction other users cannot insert > any rows during COPY FREEZE. > > Right. But the truncating transaction can insert unfrozen

Re: Sparse bit set data structure

2019-03-14 Thread Heikki Linnakangas
On 14/03/2019 07:15, Andrey Borodin wrote: 14 марта 2019 г., в 0:18, Heikki Linnakangas написал(а): <0001-Add-SparseBitset-to-hold-a-large-set-of-64-bit-ints-.patch><0002-Andrey-Borodin-s-test_blockset-tool-adapted-for-Spar.patch> That is very interesting idea. Basically, B-tree and radix

RE: Is PREPARE of ecpglib thread safe?

2019-03-14 Thread Matsumura, Ryo
Horiguchi-san Thank you for your comment. > A connection cannot be concurrently used by multiple threads so > the programmer must guard connections using mutex [1] or > friends. If it is done by a single mutex (I suppose it is > common.), there's no race condition also on the prepared >

Re: Special role for subscriptions

2019-03-14 Thread Evgeniy Efimkin
Hi! > I view that as the first step towards building a more granular privilege > system for subscription creation, and that was the second half of what I > was trying to say before- I do think there's value in having something > more granular than just "this role can create ANY subscription". As

Re: Minimal logical decoding on standbys

2019-03-14 Thread Amit Khandekar
On Fri, 8 Mar 2019 at 20:59, Amit Khandekar wrote: > > On Mon, 4 Mar 2019 at 14:09, Amit Khandekar wrote: > > > > On Fri, 14 Dec 2018 at 06:25, Andres Freund wrote: > > > I've a prototype attached, but let's discuss the details in a separate > > > thread. This also needs to be changed for

Improve the generic plan mechanism

2019-03-14 Thread Kato, Sho
Hi, hackers I want to improve the generic plan mechanism and speed up the UPDATE/DELETE planning of a table partitioned into thousands. However, I am not sure if this is realistic, I would like advice. The current generic plan creates access plans for all child tables without using the

outdated reference to tuple header OIDs

2019-03-14 Thread John Naylor
It seems this is a leftover from commit 578b229718e8. Patch attached. -- John Naylorhttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services remove-ref-to-sys-oid-col.patch Description: Binary data

Re: pgsql: Add support for hyperbolic functions, as well as log10().

2019-03-14 Thread Dean Rasheed
On Thu, 14 Mar 2019 at 04:41, Tom Lane wrote: > > Dean Rasheed writes: > > I'm amazed that jacana's asinh() returned -0 for an input of +0. > > Even more amusingly, it returns NaN for acosh('infinity'), cf > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jacana=2019-03-14%2003%3A00%3A34

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Kyotaro HORIGUCHI
At Thu, 14 Mar 2019 17:18:29 +0900, Amit Langote wrote in > On 2019/03/14 16:46, Amit Langote wrote: > > On 2019/03/14 16:32, Kyotaro HORIGUCHI wrote: > >> At Thu, 14 Mar 2019 16:13:23 +0900, Amit Langote > >> wrote in > >> <3ad792cd-0805-858e-595c-c09e9d1ce...@lab.ntt.co.jp> > >>> Hi, >

RE: Timeout parameters

2019-03-14 Thread Tsunakawa, Takayuki
From: Fabien COELHO [mailto:coe...@cri.ensmp.fr] > I think that the typical use-case of \c is to connect to another database > on the same host, at least that what I do pretty often. The natural > expectation is that the same "other" connection parameters are used, > otherwise it does not make

Re: Inadequate executor locking of indexes

2019-03-14 Thread David Rowley
On Thu, 14 Mar 2019 at 21:52, Amit Langote wrote: > If the correct lock is taken in both cases by the current code, then maybe > there's no need to change anything? What does idxlockmode improve about > the existing situation? One thing I can imagine it improves is that we > don't need the

Re: Timeout parameters

2019-03-14 Thread Kyotaro HORIGUCHI
Hello. At Thu, 14 Mar 2019 12:33:38 +0300, mikalaike...@ibagroup.eu wrote in > Hello, all. > > The main subject of discussion in this thread relates to the > 'socket_timeout'. As I understand there is no any hesitation about > applying TCP_USER_TIMEOUT into the PostgreSQL. > We have been

Re: Tid scan improvements

2019-03-14 Thread Edmund Horner
On Thu, 14 Mar 2019 at 23:06, David Rowley wrote: > On Thu, 14 Mar 2019 at 21:12, Edmund Horner wrote: > > I'm not sure how an unreasonable underestimation would occur here. If > > you have a table bloated to say 10x its minimal size, the estimator > > still assumes an even distribution of

Re: WIP: Avoid creation of the free space map for small tables

2019-03-14 Thread John Naylor
On Thu, Mar 14, 2019 at 2:17 PM Amit Kapila wrote: > > 1. Added an Assert in new_cluster_needs_fsm() that old cluster version > should be >= 804 as that is where fsm support has been added. There is already an explicit check for 804 in the caller, and the HEAD code is already resilient to FSMs

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Kyotaro HORIGUCHI
At Thu, 14 Mar 2019 16:13:23 +0900, Amit Langote wrote in <3ad792cd-0805-858e-595c-c09e9d1ce...@lab.ntt.co.jp> > Hi, > > I'm curious why DestroyPartitionDirectory doesn't do > hash_destroy(pdir->pdir_hash)? Maybe it is trashed involved in destruction of es_query_cxt or planner_cxt? --

Re: speeding up planning with partitions

2019-03-14 Thread Amit Langote
Imai-san, On 2019/03/13 19:34, Imai, Yoshikazu wrote: > I have done code review of v31 patches from 0001 to 0004. > I described below what I confirmed or thoughts. Thanks for checking. > 0001: This seems ok. > > 0002: > * I don't really know that delaying adding resjunk output columns to the

Re: speeding up planning with partitions

2019-03-14 Thread David Rowley
On Thu, 14 Mar 2019 at 21:35, Imai, Yoshikazu wrote: > 0007: > * This changes some processes using "for loop" to using > "while(bms_next_member())" which speeds up processing when we scan few > partitions in one statement, but when we scan a lot of partitions in one > statement, its

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-14 Thread Heikki Linnakangas
On 13/03/2019 03:28, Peter Geoghegan wrote: On Wed, Mar 6, 2019 at 10:15 PM Heikki Linnakangas wrote: I made a copy of the _bt_binsrch, _bt_binsrch_insert. It does the binary search like _bt_binsrch does, but the bounds caching is only done in _bt_binsrch_insert. Seems more clear to have

RE: Timeout parameters

2019-03-14 Thread Tsunakawa, Takayuki
From: mikalaike...@ibagroup.eu [mailto:mikalaike...@ibagroup.eu] > > For example, OS issues such as abnormally (buggy) slow process scheduling > or paging/swapping that prevent control from being passed to postgres. Or, > abnormally long waits on lwlocks in postgres. statement_timeout doesn't >

why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Amit Langote
Hi, I'm curious why DestroyPartitionDirectory doesn't do hash_destroy(pdir->pdir_hash)? Thanks, Amit

Re: Timeout parameters

2019-03-14 Thread Kyotaro HORIGUCHI
Hello. At Thu, 14 Mar 2019 09:42:44 +, "Tsunakawa, Takayuki" wrote in <0A3221C70F24FB45833433255569204D1FBC7626@G01JPEXMBYT05> > From: mikalaike...@ibagroup.eu [mailto:mikalaike...@ibagroup.eu] > > > For example, OS issues such as abnormally (buggy) slow process scheduling > > or

Re: Fix handling of unlogged tables in FOR ALL TABLES publications

2019-03-14 Thread Kyotaro HORIGUCHI
At Thu, 14 Mar 2019 11:30:12 +0900, Amit Langote wrote in <59e5a734-9e06-1035-385b-626717581...@lab.ntt.co.jp> > On 2019/03/13 21:03, Peter Eisentraut wrote: > > If a FOR ALL TABLES publication exists, unlogged tables are ignored > > for publishing changes. But CheckCmdReplicaIdentity() would

RE: Timeout parameters

2019-03-14 Thread Fabien COELHO
HI think that your patch is responsible for the added option at least. I agree that the underlying issue that other parameters should probably also be reused, which would be a bug fix, does not belong to this thread. This doesn't seem to be a bug. \connect just establishes a new connection,

Re: Fix handling of unlogged tables in FOR ALL TABLES publications

2019-03-14 Thread Amit Langote
On 2019/03/14 15:03, Kyotaro HORIGUCHI wrote: > At Thu, 14 Mar 2019 11:30:12 +0900, Amit Langote > wrote in > <59e5a734-9e06-1035-385b-626717581...@lab.ntt.co.jp> >> On 2019/03/13 21:03, Peter Eisentraut wrote: >>> If a FOR ALL TABLES publication exists, unlogged tables are ignored >>> for

Re: Fix handling of unlogged tables in FOR ALL TABLES publications

2019-03-14 Thread Kyotaro HORIGUCHI
At Thu, 14 Mar 2019 15:31:03 +0900, Amit Langote wrote in <26bfa053-3fb2-ad1d-efbb-7c930b41c...@lab.ntt.co.jp> > On 2019/03/14 15:03, Kyotaro HORIGUCHI wrote: > > Is it the right thing that GetRelationPublicationsActions sets > > wrong rd_publicatons for the relations? > > Actually, after

Re: Fix volatile vs. pointer confusion

2019-03-14 Thread Peter Eisentraut
On 2019-03-11 08:23, Peter Eisentraut wrote: > Variables used after a longjmp() need to be declared volatile. In > case of a pointer, it's the pointer itself that needs to be declared > volatile, not the pointed-to value. So we need > > PyObject *volatile items; > > instead of > >

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Amit Langote
On 2019/03/14 16:32, Kyotaro HORIGUCHI wrote: > At Thu, 14 Mar 2019 16:13:23 +0900, Amit Langote > wrote in > <3ad792cd-0805-858e-595c-c09e9d1ce...@lab.ntt.co.jp> >> Hi, >> >> I'm curious why DestroyPartitionDirectory doesn't do >> hash_destroy(pdir->pdir_hash)? > > Maybe it is trashed

Re: Adding a concept of TEMPORARY TABLESPACE for the use in temp_tablespaces

2019-03-14 Thread Mitar
Hi! On Fri, Jan 25, 2019 at 2:32 PM Bruce Momjian wrote: > I wrote a blog entry about this: > > https://momjian.us/main/blogs/pgblog/2017.html#June_2_2017 > > This is certainly an area we can improve, but it would require changes > in several parts of the system to handle cases where the

Re: Feature: temporary materialized views

2019-03-14 Thread Mitar
Hi! I just want to make sure if I understand correctly. But my initial proposal/patch is currently waiting first for all patches for the refactoring to happen, which are done by amazing Andreas? This sounds good to me and I see a lot of progress/work has been done and I am OK with waiting. Please

Re: Inadequate executor locking of indexes

2019-03-14 Thread Amit Langote
On 2019/03/14 7:12, David Rowley wrote: > Thanks for having a look at this. > > On Wed, 13 Mar 2019 at 22:45, Amit Langote > wrote: >> I have one question about the relation between idxlockmode and >> rellockmode? From skimming the patch, it appears that they're almost >> always set to the same

Re: log bind parameter values on error

2019-03-14 Thread Peter Eisentraut
On 2019-02-15 15:02, Alexey Bashtanov wrote: > Hello Anders and Peter, > > Thanks for your messages. > Please see the new patch version attached. In my testing, I couldn't get this patch to do anything. Could you please share your testing steps? I did postgres -D data

Re: Using the return value of strlcpy() and strlcat()

2019-03-14 Thread Tom Lane
David Rowley writes: > We should have maybe thought a bit harder when we put that strlcpy > code into the codebase and considered if we might have been better off > inventing our own function that just returns what it did copy instead > of what it would have. Well, strlcpy is (somewhat)

Re: Offline enabling/disabling of data checksums

2019-03-14 Thread Christoph Berg
Re: Magnus Hagander 2019-03-14 > Are you suggesting we should support running with a master with checksums > on and a standby with checksums off in the same cluster? That seems.. Very > fragile. The case "shut down master and standby, run pg_checksums on both, and start them again" should be

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-14 Thread Kyotaro HORIGUCHI
At Wed, 13 Mar 2019 19:37:45 +1300, David Rowley wrote in > On Wed, 13 Mar 2019 at 17:20, Kyotaro HORIGUCHI > wrote: > > bms_member_index seems working differently than maybe expected. > > > > bms_member_index((2, 4), 0) => 0, (I think) should be -1 > > bms_member_index((2, 4), 1) => 0,

Re: Offline enabling/disabling of data checksums

2019-03-14 Thread Magnus Hagander
On Thu, Mar 14, 2019 at 1:23 AM Michael Paquier wrote: > On Wed, Mar 13, 2019 at 12:24:21PM +0100, Magnus Hagander wrote: > > Enabling or disabling the checksums offline on the master quite clearly > > requires a rebuild of the standby, there is no other way (this is one of > > the reasons for

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-14 Thread Kyotaro HORIGUCHI
At Wed, 13 Mar 2019 12:39:30 -0400, Robert Haas wrote in > On Wed, Mar 13, 2019 at 12:20 AM Kyotaro HORIGUCHI > wrote: > > +Multivariate MCV (most-common values) lists are a straightforward > > extension of > > > > "lists are *a*" is wrong? > > No, that's correct. Not sure exactly what

Re: Offline enabling/disabling of data checksums

2019-03-14 Thread Magnus Hagander
On Thu, Mar 14, 2019 at 5:39 AM Michael Paquier wrote: > On Wed, Mar 13, 2019 at 08:56:33PM +0900, Michael Paquier wrote: > > On Wed, Mar 13, 2019 at 12:09:24PM +0100, Michael Banck wrote: > > > The attached patch should do the above, on top of Michael's last > > > patchset. > > > > What you are

Re: libpq environment variables in the server

2019-03-14 Thread Peter Eisentraut
On 2019-03-03 09:06, Noah Misch wrote: > +1 for clearing PGAPPNAME before starting a test postmaster. > I think this qualifies: > env -i LANG=C "$(type -p pg_ctl)" -w restart -D "$PGDATA" OK, let's make this simpler then. Here is an updated patch that just unsets PGAPPNAME around pg_ctl

Re: Using the return value of strlcpy() and strlcat()

2019-03-14 Thread Dagfinn Ilmari Mannsåker
Tom Lane writes: > Ashwin Agrawal writes: >> On Wed, Mar 13, 2019 at 9:51 AM Tom Lane wrote: >>> I don't think that's a safe transformation: what strlcpy returns is >>> strlen(src), which might be different from what it was actually >>> able to fit into the destination. Yeah, Andrew Gierth

Re: Index Skip Scan

2019-03-14 Thread Dmitry Dolgov
> On Tue, Mar 5, 2019 at 4:05 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > Although there are still some rough edges, e.g. going forth, back and forth > again leads to a sutiation, when `_bt_first` is not applied anymore and the > first element is wrongly skipped. I'll try to fix it with

Re: Offline enabling/disabling of data checksums

2019-03-14 Thread Michael Banck
Hi, Am Mittwoch, den 13.03.2019, 17:54 +0100 schrieb Magnus Hagander: > On Wed, Mar 13, 2019 at 4:51 PM Michael Banck > wrote: > > Am Mittwoch, den 13.03.2019, 12:43 +0100 schrieb Magnus Hagander: > > > I think this is dangerous enough that it needs to be enforced and not > > > documented. > >

Re: Offline enabling/disabling of data checksums

2019-03-14 Thread Magnus Hagander
On Thu, Mar 14, 2019 at 3:28 PM Christoph Berg wrote: > Re: Magnus Hagander 2019-03-14 zmb8qck7ndmchey5...@mail.gmail.com> > > Are you suggesting we should support running with a master with checksums > > on and a standby with checksums off in the same cluster? That seems.. > Very > > fragile.

Re: Suggestions on message transfer among backends

2019-03-14 Thread Antonin Houska
Robert Haas wrote: > On Tue, Mar 12, 2019 at 4:34 AM Antonin Houska wrote: > > Andy Fan wrote: > > > I just don't know why shm_mq is designed to single-reader & single-writer. > > > > shm_mq was implemented as a part of infrastructure for parallel query > > processing. The leader backend

[sqlsmith] Failed assertion at relnode.c

2019-03-14 Thread tushar
Hi, While running sqlsmith against 12devel, got the the following assertion-  (issue is reproducible  on v10/v11 as well) TRAP: FailedAssertion("!(bms_is_subset(appendrel->lateral_relids, required_outer))", File: "relnode.c", Line: 1521) stack trace - #0  0x7f2a2f349277 in raise ()

RE: Timeout parameters

2019-03-14 Thread MikalaiKeida
Hello, Takayuki. > > > For example, OS issues such as abnormally (buggy) slow process scheduling > > or paging/swapping that prevent control from being passed to postgres. Or, > > abnormally long waits on lwlocks in postgres. statement_timeout doesn't > > take effect while waiting on a

Re: Removing unneeded self joins

2019-03-14 Thread David Rowley
On Fri, 1 Mar 2019 at 03:09, Alexander Kuzmenkov wrote: > New version is attached. I started looking over v11 and I'm wondering why you really need to know which unique index proved the join unique? I removed that check and I see it causes the following to remove the self-join: create unique

Re: Using the return value of strlcpy() and strlcat()

2019-03-14 Thread David Rowley
On Fri, 15 Mar 2019 at 00:11, Dagfinn Ilmari Mannsåker wrote: > Tom Lane writes: > > Mmm ... if there's a way to do it that's not messy and typo-prone, > > maybe. But I'm dubious that the potential gain is worth complicating > > the code. The strings involved aren't usually all that long. > >

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-03-14 Thread Masahiko Sawada
On Sat, Mar 9, 2019 at 3:08 AM Tomas Vondra wrote: > > On 3/8/19 5:38 PM, Antonin Houska wrote: > > Antonin Houska wrote: > > > >> Masahiko Sawada wrote: > >> > >>> Agreed. > >>> > >>> For the WAL encryption, I wonder if we can have a encryption key > >>> dedicated for WAL. Regardless of keys

Re: Timeout parameters

2019-03-14 Thread Robert Haas
One other thing -- I looked a bit into the pgsql-jdbc implementation of a similarly-named option, and it does seem to match what you are proposing here. I wonder what user experiences with that option have been like. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise

Re: WIP: Avoid creation of the free space map for small tables

2019-03-14 Thread Amit Kapila
On Thu, Mar 14, 2019 at 12:37 PM John Naylor wrote: > > On Thu, Mar 14, 2019 at 2:17 PM Amit Kapila wrote: > > > > 1. Added an Assert in new_cluster_needs_fsm() that old cluster version > > should be >= 804 as that is where fsm support has been added. > > There is already an explicit check for

Re: Feature: temporary materialized views

2019-03-14 Thread Andreas Karlsson
On 3/14/19 9:13 AM, Mitar wrote:> I just want to make sure if I understand correctly. But my initial proposal/patch is currently waiting first for all patches for the refactoring to happen, which are done by amazing Andreas? This sounds good to me and I see a lot of progress/work has been done

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-14 Thread Heikki Linnakangas
On 13/03/2019 03:28, Peter Geoghegan wrote: It would be great if you could take a look at the 'Add high key "continuescan" optimization' patch, which is the only one you haven't commented on so far (excluding the amcheck "relocate" patch, which is less important). I can put that one off for a

Google Summer of Code

2019-03-14 Thread Auguste Comte
Hello Moodle Team, I am a student who is currently take computer science classes at Glendale Community College, AZ who wants to take this new knowledge to the next. Since I’m so new to programming and software development I need some hands on experience beyond academic curriculum. So please

Re: pgsql: Add support for hyperbolic functions, as well as log10().

2019-03-14 Thread Andrew Dunstan
On 3/14/19 12:41 AM, Tom Lane wrote: > Dean Rasheed writes: >> I'm amazed that jacana's asinh() returned -0 for an input of +0. > Even more amusingly, it returns NaN for acosh('infinity'), cf > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jacana=2019-03-14%2003%3A00%3A34 > >

Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3

2019-03-14 Thread Antonin Houska
Thanks for feedback! Robert Haas wrote: > However, this is also quite invasive. It changes a lot of code and it > doesn't do so in a very predictable way. It's not like you went > through and replaced every call to write() with a call to > SpecialEncyptionMagicWrite(). Rather, there are new

Re: Problem with default partition pruning

2019-03-14 Thread Thibaut
Le 28/02/2019 à 09:26, Imai, Yoshikazu a écrit : > Hosoya-san > > On Wed, Feb 27, 2019 at 6:51 AM, Yuzuko Hosoya wrote: >>> From: Amit Langote [mailto:langote_amit...@lab.ntt.co.jp] >>> Sent: Wednesday, February 27, 2019 11:22 AM >>> >>> Hosoya-san, >>> >>> On 2019/02/22 17:14, Yuzuko Hosoya

Re: Offline enabling/disabling of data checksums

2019-03-14 Thread Michael Banck
Hi, Am Donnerstag, den 14.03.2019, 15:26 +0100 schrieb Magnus Hagander: > Given that the failure is data corruption, I don't think big fat > warning is enough. We should really make it impossible to start up the > postmaster by mistake during the checksum generation. People don't > read the

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Tom Lane
I wrote: > Agreed, but the comments in this area are crap. Actually, now that I've absorbed a bit more about 898e5e329, I don't like very much about it at all. I think having it try to hang onto pointers into the relcache is a completely wrongheaded design decision, and the right way for it to

Re: Offline enabling/disabling of data checksums

2019-03-14 Thread Michael Banck
Hi, Am Donnerstag, den 14.03.2019, 15:32 +0100 schrieb Magnus Hagander: > On Thu, Mar 14, 2019 at 3:28 PM Christoph Berg wrote: > > Re: Magnus Hagander 2019-03-14 > > > > > Are you suggesting we should support running with a master with checksums > > > on and a standby with checksums off in

Re: ATTACH/DETACH PARTITION CONCURRENTLY

2019-03-14 Thread Robert Haas
On Thu, Mar 14, 2019 at 6:12 AM David Rowley wrote: > On Wed, 6 Mar 2019 at 10:13, Robert Haas wrote: > > Would anyone like to argue that there is any other reason not to > > commit these patches? > > Hi Robert, > > Thanks for working on this. I'm sorry I didn't get a chance to > dedicate some

Re: hyrax vs. RelationBuildPartitionDesc

2019-03-14 Thread Tom Lane
Robert Haas writes: > I did not back-patch, because the code is in a different file in v11, > none of the hunks of the patch apply on v11, and v11 is not failing on > hyrax. Hmm, I wonder why not. I suppose the answer is that the leak is worse in HEAD than before, but how come? I followed your

Re: GIN indexes on an = ANY(array) clause

2019-03-14 Thread Andreas Karlsson
On 3/13/19 5:38 PM, Tom Lane wrote: regression=# select amopopr::regoperator from pg_amop where amopfamily = 2745; amopopr --- &&(anyarray,anyarray) @>(anyarray,anyarray) <@(anyarray,anyarray) =(anyarray,anyarray) (4 rows) and none of those are obviously

Re: Sparse bit set data structure

2019-03-14 Thread Julien Rouhaud
On Wed, Mar 13, 2019 at 8:18 PM Heikki Linnakangas wrote: > > I started to consider rewriting the data structure into something more > like B-tree. Then I remembered that I wrote a data structure pretty much > like that last year already! We discussed that on the "Vacuum: allow > usage of more

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Robert Haas
On Thu, Mar 14, 2019 at 1:40 PM Tom Lane wrote: > So here's my problem with that argument: you're effectively saying that > you needn't write any API spec for the PartitionDirectory functions > because you intend that every person calling them will read their code, > carefully and fully, before

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Robert Haas
On Thu, Mar 14, 2019 at 3:13 AM Amit Langote wrote: > I'm curious why DestroyPartitionDirectory doesn't do > hash_destroy(pdir->pdir_hash)? What would be the point? It's more efficient to let context teardown take care of it. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Robert Haas
On Thu, Mar 14, 2019 at 12:56 PM Tom Lane wrote: > Agreed, but the comments in this area are crap. Why doesn't > CreatePartitionDirectory say something like > > * The object lives inside the given memory context and will be > * freed when that context is destroyed. Nonetheless, the caller >

Re: partitioned tables referenced by FKs

2019-03-14 Thread Alvaro Herrera
On 2019-Feb-28, Amit Langote wrote: Hello > In the following case: > > create table pk (a int primary key) partition by list (a); > create table pk1 (a int primary key); > create table fk (a int references pk1); > > -- adds FK referencing pk1 via ATAddForeignKeyConstraint recursion > alter

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Tom Lane
Robert Haas writes: > On Thu, Mar 14, 2019 at 12:56 PM Tom Lane wrote: >> It's completely not acceptable that every reader of this code should >> have to reverse-engineer these design assumptions, especially given >> how shaky they are. > Well, one reason is that everything you just said is

Re: partitioned tables referenced by FKs

2019-03-14 Thread Robert Haas
On Thu, Mar 14, 2019 at 1:40 PM Alvaro Herrera wrote: > Once I was finished, fixed bugs and tested it, I realized that that was > a stupid thing to have done -- because THOSE ARE DIFFERENT CONSTRAINTS. This made me laugh. > When you say "fk (a) references pk1" you're saying that all the values

Re: [sqlsmith] Failed assertion at relnode.c

2019-03-14 Thread Amit Langote
On Thu, Mar 14, 2019 at 11:53 PM tushar wrote: > > Hi, > > While running sqlsmith against 12devel, got the the following > assertion- (issue is reproducible on v10/v11 as well) > > TRAP: FailedAssertion("!(bms_is_subset(appendrel->lateral_relids, > required_outer))", File: "relnode.c", Line:

Re: hyrax vs. RelationBuildPartitionDesc

2019-03-14 Thread Robert Haas
On Wed, Mar 13, 2019 at 2:59 PM Robert Haas wrote: > Yeah, that probably makes sense, though it might be polite to wait > another hour or two to see if anyone wants to argue with that approach > further. Hearing nobody, done. If someone wants to argue more we can always change it later. I did

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Tom Lane
Robert Haas writes: > On Thu, Mar 14, 2019 at 3:13 AM Amit Langote > wrote: >> I'm curious why DestroyPartitionDirectory doesn't do >> hash_destroy(pdir->pdir_hash)? > What would be the point? It's more efficient to let context teardown > take care of it. Agreed, but the comments in this area

Re: hyrax vs. RelationBuildPartitionDesc

2019-03-14 Thread Robert Haas
On Thu, Mar 14, 2019 at 12:36 PM Tom Lane wrote: > Hmm, I wonder why not. I suppose the answer is that > the leak is worse in HEAD than before, but how come? I'd like to know that, too, but right now I don't. > I followed your reference to 898e5e329, and I've got to say that > the hunk it adds

Re: why doesn't DestroyPartitionDirectory hash_destroy?

2019-03-14 Thread Robert Haas
On Thu, Mar 14, 2019 at 1:16 PM Tom Lane wrote: > Actually, now that I've absorbed a bit more about 898e5e329, > I don't like very much about it at all. I think having it > try to hang onto pointers into the relcache is a completely > wrongheaded design decision, and the right way for it to work

Re: [sqlsmith] Failed assertion at relnode.c

2019-03-14 Thread Tom Lane
tushar writes: > While running sqlsmith against 12devel, got the the following > assertion-  (issue is reproducible  on v10/v11 as well) > TRAP: FailedAssertion("!(bms_is_subset(appendrel->lateral_relids, > required_outer))", File: "relnode.c", Line: 1521) Doesn't crash for me, but that's

Re: Making all nbtree entries unique by having heap TIDs participate in comparisons

2019-03-14 Thread Peter Geoghegan
On Thu, Mar 14, 2019 at 4:00 AM Heikki Linnakangas wrote: > Oh yeah, that makes perfect sense. I wonder why we haven't done it like > that before? The new page split logic makes it more likely to help, but > even without that, I don't see any downside. The only downside is that we spend a few

Re: PostgreSQL vs SQL/XML Standards

2019-03-14 Thread Alvaro Herrera
On 2019-Mar-14, Pavel Stehule wrote: > I looking to code > > void(*nodefree) (xmlNodePtr) = NULL; > volatile xmlBufferPtr buf = NULL; > > should not be "nodefree" volatile too? Ah, good question. I remember I had it volatile and removed it for some reason, though I

Re: pgsql: Add support for hyperbolic functions, as well as log10().

2019-03-14 Thread Andrew Dunstan
On 3/14/19 3:08 PM, Tom Lane wrote: > Andrew Dunstan writes: >> On 3/14/19 12:41 AM, Tom Lane wrote: >>> So far, no other failures ... >> I have replicated this on my Msys2 test system. >> I assume it's a bug in the mingw math library. I think jacana is the >> only currently reporting mingw

Re: PostgreSQL vs SQL/XML Standards

2019-03-14 Thread Pavel Stehule
Hi pá 8. 3. 2019 v 19:44 odesílatel Alvaro Herrera napsal: > On 2019-Mar-08, Alvaro Herrera wrote: > > > > Maybe we can call explicitly xmlFreeDoc instead xmlFreeNode > > > > > > some like > > > > > > if (cur_copy->type == XML_DOCUMENT_NODE) > > > xmlFreeDoc((xmlDocPtr) cur_copy); > > > else

seems like a bug in pgbench -R

2019-03-14 Thread Tomas Vondra
Hi, There seems to be a bug in pgbench when used with '-R' option, resulting in stuck pgbench process. Reproducing it is pretty easy: echo 'select 1' > select.sql while /bin/true; do pgbench -n -f select.sql -R 1000 -j 8 -c 8 -T 1 > /dev/null 2>&1; date; done; I do get a stuck pgbench

Re: partitioned tables referenced by FKs

2019-03-14 Thread Alvaro Herrera
On 2019-Mar-14, Robert Haas wrote: > On Thu, Mar 14, 2019 at 1:40 PM Alvaro Herrera > wrote: > > When you say "fk (a) references pk1" you're saying that all the values > > in fk(a) must appear in pk1. OTOH when you say "fk references pk" you > > mean that the values might appear anywhere in

Re: pgsql: Add support for hyperbolic functions, as well as log10().

2019-03-14 Thread Tom Lane
Andrew Dunstan writes: > On 3/14/19 12:41 AM, Tom Lane wrote: >> So far, no other failures ... > I have replicated this on my Msys2 test system. > I assume it's a bug in the mingw math library. I think jacana is the > only currently reporting mingw member :-( The MSVC members appear to be >

Re: partitioned tables referenced by FKs

2019-03-14 Thread Robert Haas
On Thu, Mar 14, 2019 at 3:36 PM Alvaro Herrera wrote: > Well, I suppose that can be implemented as an optimization on top of > what we have, but I think that we should first get this feature right, > and later we can see about improving it. Sure, not arguing with that. > In any case, since the

Re: [PATCH] kNN for btree

2019-03-14 Thread Nikita Glukhov
Attached 10th versions of the patches. Fixed two bugs in patches 3 and 10 (see below). Patch 3 was extracted from the main patch 5 (patch 4 in v9). On 11.03.2019 20:42, Alexander Korotkov wrote: Hi! I've some questions regarding this patchset. 1) This comment needs to be revised. Now,

Re: pg_basebackup ignores the existing data directory permissions

2019-03-14 Thread Peter Eisentraut
On 2019-03-09 02:19, Haribabu Kommi wrote: > Yes, I agree that it may be a problem if the existing data directory > permissions > are 0700 to changing it to 0750. But it may not be a problem for the > scenarios, > where the existing data permissions  >=0750, to the upstream permissions. > Because

  1   2   >