WIP Patch: Add a function that returns binary JSONB as a bytea

2018-10-31 Thread Kevin Van
This patch adds a new function that allows callers to receive binary jsonb. This change was proposed in the discussion here: https://www.postgresql.org/message-id/CAOsiKEL7%2BKkV0C_hAJWxqwTg%2BPYVfiGPQ0yjFww7ECtqwBjb%2BQ%40mail.gmail.com and the primary motivation is to reduce database load by

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-10-31 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Kevin Van writes: > > This patch adds a new function that allows callers to receive binary jsonb. > > This change was proposed in the discussion here: > >

Re: PostgreSQL Limits and lack of documentation about them.

2018-10-31 Thread John Naylor
On 10/30/18, David Rowley wrote: > On 26 October 2018 at 11:40, Haribabu Kommi > wrote: >> On Fri, Oct 26, 2018 at 9:30 AM David Rowley >> >> wrote: >>> >>> For a long time, we documented our table size, max columns, max column >>> width limits, etc. in https://www.postgresql.org/about/ , but

Re: pg_dumpall --exclude-database option

2018-10-31 Thread Andrew Dunstan
On 10/13/2018 10:07 AM, Fabien COELHO wrote: Hello Andrew, A question: would it makes sense to have a symmetrical --include-database=PATTERN option as well? I don't think so. If you only want a few databases, just use pg_dump. The premise of pg_dumpall is that you want all of them and

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-10-31 Thread Andres Freund
Hi, On 2018-10-31 11:13:13 -0400, Andrew Dunstan wrote: > I agree that just sending a blob of the internal format isn't a great idea. It's entirely unacceptable afaict. Besides the whole "exposing internals" issue, it's also at least not endianess safe, depends on the local alignment

Re: [HACKERS] generated columns

2018-10-31 Thread Simon Riggs
On Wed, 31 Oct 2018 at 08:29, Erik Rijkers wrote: > On 2018-10-31 09:15, Simon Riggs wrote: > > On Wed, 31 Oct 2018 at 07:58, Erikjan Rijkers wrote: > > > > > >> I have also noticed that logical replication isn't possible on tables > >> with a generated column. That's a shame but I suppsoe

Re: WIP: Avoid creation of the free space map for small tables

2018-10-31 Thread Amit Kapila
On Wed, Oct 31, 2018 at 1:42 PM John Naylor wrote: > > Upthread I wrote: > > > -A possible TODO item is to teach pg_upgrade not to link FSMs for > > small heaps. I haven't look into the feasibility of that, however. > > This turned out to be relatively light weight (0002 attached). I had > to add

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-10-31 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> I dunno, I do not think it's a great idea to expose jsonb's internal >> format to the world. We intentionally did not do that when the type >> was first defined --- that's why its binary I/O format isn't already >> like this ---

Re: [HACKERS] generated columns

2018-10-31 Thread Sergei Kornilov
Hi > OK, so the problem is COPY. > > Which means we have an issue with restore. We need to be able to pg_dump a > table with generated columns, then restore it afterwards. More generally, we > need to be able to handle data that has already been generated - the > "generate" idea should apply

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-10-31 Thread Andrew Dunstan
On 10/31/2018 10:21 AM, Tom Lane wrote: Stephen Frost writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: I dunno, I do not think it's a great idea to expose jsonb's internal format to the world. We intentionally did not do that when the type was first defined --- that's why its binary I/O

Re: Super PathKeys (Allowing sort order through precision loss functions)

2018-10-31 Thread Tomas Vondra
On 10/31/2018 04:32 AM, David Rowley wrote: On 31 October 2018 at 14:23, Tomas Vondra wrote: The other thing likely affecting this is locale / collation. Probably not for date_trunc, but certainly for things like substr()/trim(), mentioned by Simon upthread. In some languages the rules are

Typo in xlog.c comment?

2018-10-31 Thread Daniel Gustafsson
Am I right in interpreting the below comment in xlog.c as documenting a new mode of operation, so “Not” should actually be “Note”? Or am I just not able to English today? diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 62fc418893..246869bba2 100644 ---

Re: Typo in xlog.c comment?

2018-10-31 Thread Andres Freund
Hi, On 2018-10-31 15:32:24 +0100, Daniel Gustafsson wrote: > Am I right in interpreting the below comment in xlog.c as documenting a new > mode of operation, so “Not” should actually be “Note”? Or am I just not able > to English today? No, you're right. Fixed! Greetings, Andres Freund

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-10-31 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> I dunno, I do not think it's a great idea to expose jsonb's internal > >> format to the world. We intentionally did not do that when the type > >> was first defined ---

Re: INSTALL file

2018-10-31 Thread Stephen Frost
* Andreas 'ads' Scherbaum (a...@pgug.de) wrote: > On 30.10.18 11:49, Andrew Dunstan wrote: > >On 10/30/2018 06:14 AM, Andreas 'ads' Scherbaum wrote: > >>On 30.10.18 04:11, Michael Paquier wrote: > >>>FWIW, I think that people depend too much on github and what github > >>>thinks projects should do

Re: Ordered Partitioned Table Scans

2018-10-31 Thread Antonin Houska
David Rowley wrote: > On 31 October 2018 at 13:05, David Rowley > wrote: > >>> On 28 October 2018 at 03:49, Julien Rouhaud wrote: > >> I've registered as a reviewer. I still didn't have a deep look at > >> the patch yet, but thanks a lot for working on it! > > > > Thanks for signing up to

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-10-31 Thread Robert Haas
On Wed, Aug 22, 2018 at 8:30 AM David Rowley wrote: > On 22 August 2018 at 19:08, Amit Langote > wrote: > > +#define PartitionTupRoutingGetToParentMap(p, i) \ > > +#define PartitionTupRoutingGetToChildMap(p, i) \ > > > > If the "Get" could be replaced by "Child" and "Parent", respectively, > >

Re: ToDo: show size of partitioned table

2018-10-31 Thread Pavel Stehule
st 31. 10. 2018 v 7:34 odesílatel Amit Langote < langote_amit...@lab.ntt.co.jp> napsal: > On 2018/10/31 15:30, Pavel Stehule wrote: > > st 31. 10. 2018 v 3:27 odesílatel Amit Langote < > > langote_amit...@lab.ntt.co.jp> napsal: > >> +appendPQExpBufferStr(, "\nWHERE c.relkind IN ('p')\n"); >

Re: WIP: Avoid creation of the free space map for small tables

2018-10-31 Thread Robert Haas
On Tue, Oct 23, 2018 at 9:42 AM John Naylor wrote: > A case could be made for setting the threshold to 4, since not having > 3 blocks of FSM in shared buffers exactly makes up for the 3 other > blocks of heap that are checked when free space runs out. That doesn't seem like an unreasonable

Re: FETCH FIRST clause WITH TIES option

2018-10-31 Thread Robert Haas
On Mon, Oct 29, 2018 at 12:48 PM Andrew Gierth wrote: > Then FETCH FIRST N WITH TIES becomes "stop when the expression > rank() over (order by ...) <= N > is no longer true" (where the ... is the existing top level order by) Wouldn't that be wicked expensive compared to something hard-coded

Re: pg_dumpall --exclude-database option

2018-10-31 Thread Fabien COELHO
:-( My fault, I just created a new one. Hmmm... so did I:-) We did it a few minutes apart. I did not find yours when I first searched, then I proceeded to try to move the previous CF entry which had been marked as "returned" but this was rejected, so I recreated the one without checking

Re: pg_promote not marked as parallel-restricted in pg_proc.dat

2018-10-31 Thread Robert Haas
On Mon, Oct 29, 2018 at 6:48 PM Michael Paquier wrote: > All the backup-related functions doing on-disk activity are marked as > parallel-restricted: > =# select proparallel, proname from pg_proc where proname ~ 'backup'; > proparallel | proname > -+-- > s

Re: pgbench doc fix

2018-10-31 Thread Robert Haas
On Tue, Oct 30, 2018 at 8:48 AM Tatsuo Ishii wrote: > Yes, you need to send params (thus send bind message) anyway. > Regarding re-parsing, maybe you mixed up parse-analythis with > planning? Re-parse-analythis can only be avoided if you can reuse > named (or unnamed) parepared statements. So

Parallel threads in query

2018-10-31 Thread Komяpa
Hi, I've tried porting some of PostGIS algorithms to utilize multiple cores via OpenMP to return faster. Question is, what's the best policy to allocate cores so we can play nice with rest of postgres? What I'd like to see is some function that I can call and get a number of threads I'm allowed

Re: pg_dumpall --exclude-database option

2018-10-31 Thread Fabien COELHO
Hello Andrew, This patch addresses all these concerns. Patch v4 applies cleanly, compiles, doc generation ok, global & local tests ok. Tiny comments: there is a useless added blank line at the beginning of the added varlistenry. I have recreated the CF entry and put the patch to

Re: pg_dumpall --exclude-database option

2018-10-31 Thread Andrew Dunstan
On 10/31/2018 12:44 PM, Fabien COELHO wrote: Hello Andrew, This patch addresses all these concerns. Patch v4 applies cleanly, compiles, doc generation ok, global & local tests ok. Tiny comments: there is a useless added blank line at the beginning of the added varlistenry. I have

Re: Super PathKeys (Allowing sort order through precision loss functions)

2018-10-31 Thread Tom Lane
Robert Haas writes: > This kinda reminds me of commit > 8f9fe6edce358f7904e0db119416b4d1080a83aa. We needed a way to provide > the planner with knowledge about the behavior of specific functions. > In that case, the specific need was to be able to tell the planner > that a certain function call

Re: Is there way to detect uncommitted 'new table' in pg_class?

2018-10-31 Thread Robert Haas
On Wed, Oct 31, 2018 at 6:05 AM Hubert Zhang wrote: > In PG READ UNCOMMITTED is treated as READ COMMITTED > But I have a requirement to read dirty table. Is there way to detect table > which is created in other uncommitted transaction? > > T1: > BEGIN; > create table a(i int); > > T2: > select *

How to properly use the Executor interface?

2018-10-31 Thread Kai Kratz
Hi Hackers, first time writing to the hackers list, so I hope this is the right place to ask. I recently joined Swarm64 and we are building a postgres extension with the fdw interface. I am trying to evaluate sql statements with ExecutorBeing, -Run, -End, -Finish calls during

Re: Continue work on changes to recovery.conf API

2018-10-31 Thread Robert Haas
On Fri, Sep 28, 2018 at 4:20 PM Peter Eisentraut wrote: > > - recovery_target (immediate), recovery_target_name, recovery_target_time, > > recovery_target_xid, recovery_target_lsn are replaced to > > recovery_target_type and recovery_target_value (was discussed and changed > > in previous

Re: Super PathKeys (Allowing sort order through precision loss functions)

2018-10-31 Thread David Rowley
On 1 November 2018 at 05:40, Robert Haas wrote: > This kinda reminds me of commit > 8f9fe6edce358f7904e0db119416b4d1080a83aa. We needed a way to provide > the planner with knowledge about the behavior of specific functions. > In that case, the specific need was to be able to tell the planner >

Re: PostgreSQL Limits and lack of documentation about them.

2018-10-31 Thread Nasby, Jim
> On Oct 31, 2018, at 5:22 PM, David Rowley > wrote: > > On 1 November 2018 at 04:40, John Naylor wrote: >> Thanks for doing this. I haven't looked at the rendered output yet, >> but I have some comments on the content. >> >> + Maximum Relation Size >> + 32 TB >> + Limited by

Re: Super PathKeys (Allowing sort order through precision loss functions)

2018-10-31 Thread Nasby, Jim
> On Oct 30, 2018, at 9:08 AM, Simon Riggs wrote: > > On Tue, 30 Oct 2018 at 07:58, David Rowley > wrote: > > I've started working on something I've ended up calling "Super > PathKeys". The idea here is to increase the likelihood of a Path with > PathKeys being used for a purpose that

Re: Parallel threads in query

2018-10-31 Thread Tom Lane
=?UTF-8?Q?Darafei_=22Kom=D1=8Fpa=22_Praliaskouski?= writes: > Question is, what's the best policy to allocate cores so we can play nice > with rest of postgres? > What I'd like to see is some function that I can call and get a number of > threads I'm allowed to run, that will also advise rest of

Re: Should pg 11 use a lot more memory building an spgist index?

2018-10-31 Thread Tom Lane
Amit Langote writes: > Maybe, we don't need to spoil the interface of index_beginscan with the > new memory context argument like my patch does if the simple following of > its contract by amendscan would suffice. Yeah, I'm not enamored of changing the API of index_beginscan for this; the

Re: Parallel threads in query

2018-10-31 Thread David Fetter
On Wed, Oct 31, 2018 at 09:07:43AM -1000, Darafei "Komяpa" Praliaskouski wrote: > Hi, > > I've tried porting some of PostGIS algorithms to utilize multiple cores via > OpenMP to return faster. Great! > Question is, what's the best policy to allocate cores so we can play nice > with rest of

Re: Super PathKeys (Allowing sort order through precision loss functions)

2018-10-31 Thread Tomas Vondra
On 10/31/2018 10:07 PM, David Rowley wrote: > On 1 November 2018 at 05:40, Robert Haas wrote: >> This kinda reminds me of commit >> 8f9fe6edce358f7904e0db119416b4d1080a83aa. We needed a way to provide >> the planner with knowledge about the behavior of specific functions. >> In that case, the

Re: replication_slots usability issue

2018-10-31 Thread Andres Freund
On 2018-10-30 10:52:54 -0700, Andres Freund wrote: > On 2018-10-30 11:51:09 +0900, Michael Paquier wrote: > > On Mon, Oct 29, 2018 at 12:13:04PM -0700, Andres Freund wrote: > > > I don't think this quite is the problem. ISTM the issue is rather that > > > StartupReplicationSlots() *needs* to check

Re: PostgreSQL Limits and lack of documentation about them.

2018-10-31 Thread David Rowley
On 1 November 2018 at 04:40, John Naylor wrote: > Thanks for doing this. I haven't looked at the rendered output yet, > but I have some comments on the content. > > + Maximum Relation Size > + 32 TB > + Limited by 2^32 pages per relation > > I prefer "limited to" or "limited by the

Re: COPY FROM WHEN condition

2018-10-31 Thread Nasby, Jim
On Oct 11, 2018, at 10:35 AM, David Fetter wrote: > >> It didn't get far, but you may want to take a look at a rejected patch for >> copy_srf() (set returning function) >> https://www.postgresql.org/message-id/CADkLM%3DdoeiWQX4AGtDNG4PsWfSXz3ai7kY%3DPZm3sUhsUeev9Bg%40mail.gmail.com >>

Re: replication_slots usability issue

2018-10-31 Thread Michael Paquier
HI Andres, On Wed, Oct 31, 2018 at 03:48:02PM -0700, Andres Freund wrote: > And done. Thanks for the report JD. Shouldn't we also switch the PANIC to a FATAL in RestoreSlotFromDisk()? I don't mind doing so myself if you agree with the change, only on HEAD as you seemed to disagree about

Re: PostgreSQL Limits and lack of documentation about them.

2018-10-31 Thread John Naylor
On 11/1/18, Nasby, Jim wrote: > Hmm… 18 bytes doesn’t sound right, at least not for the Datum. Offhand I’d > expect it to be the small (1 byte) varlena header + an OID (4 bytes). Even > then I don’t understand how 1600 text columns would work; the data area of a > tuple should be limited to ~2000

Re: Ordered Partitioned Table Scans

2018-10-31 Thread David Rowley
On 1 November 2018 at 04:01, Antonin Houska wrote: > * As for the logic, I found generate_mergeappend_paths() to be the most > interesting part: > > Imagine table partitioned by "i", so "partition_pathkeys" is {i}. > > partition 1: > > i | j > --+-- > 0 | 0 > 1 | 1 > 0 | 1 > 1 | 0 > > partition

move PartitionBoundInfo creation code

2018-10-31 Thread Amit Langote
Hi, Currently, the code that creates a PartitionBoundInfo struct from the PartitionBoundSpec nodes of constituent partitions read from the catalog is in RelationBuildPartitionDesc that's in partcache.c. I think that da6f3e45dd that moved around the partitioning code [1] really missed the

Re: Super PathKeys (Allowing sort order through precision loss functions)

2018-10-31 Thread David Rowley
On 1 November 2018 at 12:11, Tomas Vondra wrote: > I still have trouble imagining what exactly would the function do to > determine if the optimization can be applied to substr() and similar > collation-dependent cases. I guess the function would have to check for a Const offset of 0, and a

Re: INSTALL file

2018-10-31 Thread Andreas 'ads' Scherbaum
On 01.11.18 01:29, Michael Paquier wrote: On Wed, Oct 31, 2018 at 08:30:40AM -0400, Stephen Frost wrote: Agreed, we should really improve the README by merging the README.git into it and make the project, as a whole, more accessible to new developers. +1. I think as well that this approach

Re: pg_promote not marked as parallel-restricted in pg_proc.dat

2018-10-31 Thread Michael Paquier
On Wed, Oct 31, 2018 at 01:09:53PM -0400, Robert Haas wrote: > There's no rule whatsoever that a parallel worker can't write to the > disk. pg_start_backup and pg_stop_backup have to be > parallel-restricted because, when used in non-exclusive mode, they > establish backend-local state that

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-10-31 Thread Steve Singer
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:tested, failed --- a/doc/src/sgml/perform.sgml +++ b/doc/src/sgml/perform.sgml

Re: heap_sync seems rather oblivious to partitioned tables (wal_level=minimal)

2018-10-31 Thread David Rowley
Thanks for looking at this. On 1 November 2018 at 16:07, Steve Singer wrote: > --- a/doc/src/sgml/perform.sgml > +++ b/doc/src/sgml/perform.sgml > @@ -1534,9 +1534,10 @@ SELECT * FROM x, y, a, b, c WHERE something AND > somethingelse; > TRUNCATE command. In such cases no WAL > needs

Re: Is there way to detect uncommitted 'new table' in pg_class?

2018-10-31 Thread Michael Paquier
On Wed, Oct 31, 2018 at 01:30:52PM -0400, Robert Haas wrote: > In theory, at least, you could write C code to scan the catalog tables > with SnapshotDirty to find the catalog entries, but I don't think that > helps a whole lot. You couldn't necessarily rely on those catalog > entries to be in a

Re: Super PathKeys (Allowing sort order through precision loss functions)

2018-10-31 Thread David Rowley
On 1 November 2018 at 12:24, Andres Freund wrote: > FWIW, I kind of wonder if we built proper infrastructure to allow to > make such inferrences from function calls, whether it could also be made > to support the transformation of LIKEs into indexable <= >= clauses. Perhaps, but I doubt it would

Re: Getting ERROR: could not open file "base/13164/t3_16388" with partition table with ON COMMIT

2018-10-31 Thread Michael Paquier
On Thu, Nov 01, 2018 at 12:39:16PM +0900, Amit Langote wrote: > Rajkumar pointed out off-list that the patch still remains to be applied. > Considering that there is a planned point release on Nov 8, maybe we > should do something about this? Yes doing something about that very soon would be a

Re: COPY FROM WHEN condition

2018-10-31 Thread David Fetter
On Wed, Oct 31, 2018 at 11:21:33PM +, Nasby, Jim wrote: > On Oct 11, 2018, at 10:35 AM, David Fetter wrote: > > > >> It didn't get far, but you may want to take a look at a rejected patch for > >> copy_srf() (set returning function) > >>

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-10-31 Thread David Rowley
On 1 November 2018 at 13:35, Amit Langote wrote: > On 2018/11/01 8:58, David Rowley wrote: >> On 1 November 2018 at 06:45, Robert Haas wrote: >>> I think a better way to shorten the name would be to truncate the >>> PartitionTupRouting() prefix in some way, e.g. dropping TupRouting. >> >> Thanks

Re: Getting ERROR: could not open file "base/13164/t3_16388" with partition table with ON COMMIT

2018-10-31 Thread Amit Langote
On 2018/09/14 10:53, Amit Langote wrote: > On 2018/09/13 23:13, Tom Lane wrote: >> Amit Langote writes: >>> On 2018/09/13 1:14, Tom Lane wrote: That seems excessively restrictive. Anything that has storage (e.g. matviews) ought to be truncatable, no? >> >>> Not by heap_truncate it

Re: move PartitionBoundInfo creation code

2018-10-31 Thread Michael Paquier
On Thu, Nov 01, 2018 at 12:58:29PM +0900, Amit Langote wrote: > Attached find a patch that does such refactoring, along with making some > functions in partbounds.c that are not needed outside static. This looks like a very good idea to me. Thanks for digging into that. Please just make sure to

Re: move PartitionBoundInfo creation code

2018-10-31 Thread Amit Langote
On 2018/11/01 13:02, Michael Paquier wrote: > On Thu, Nov 01, 2018 at 12:58:29PM +0900, Amit Langote wrote: >> Attached find a patch that does such refactoring, along with making some >> functions in partbounds.c that are not needed outside static. > > This looks like a very good idea to me.

Re: Super PathKeys (Allowing sort order through precision loss functions)

2018-10-31 Thread Andres Freund
Hi, On 2018-11-01 12:19:32 +1300, David Rowley wrote: > On 1 November 2018 at 12:11, Tomas Vondra > wrote: > > I still have trouble imagining what exactly would the function do to > > determine if the optimization can be applied to substr() and similar > > collation-dependent cases. > > I

Re: row filtering for logical replication

2018-10-31 Thread Euler Taveira
Em qua, 28 de fev de 2018 às 20:03, Euler Taveira escreveu: > The attached patches add support for filtering rows in the publisher. > I rebased the patch. I added row filtering for initial synchronization, pg_dump support and psql support. 0001 removes unused code. 0002 reduces memory use. 0003

Re: INSTALL file

2018-10-31 Thread Andres Freund
Hi, On 2018-11-01 09:29:37 +0900, Michael Paquier wrote: > On Wed, Oct 31, 2018 at 08:30:40AM -0400, Stephen Frost wrote: > > Agreed, we should really improve the README by merging the README.git > > into it and make the project, as a whole, more accessible to new > > developers. > > +1. I

Re: INSTALL file

2018-10-31 Thread Michael Paquier
On Wed, Oct 31, 2018 at 08:30:40AM -0400, Stephen Frost wrote: > Agreed, we should really improve the README by merging the README.git > into it and make the project, as a whole, more accessible to new > developers. +1. I think as well that this approach would be a good thing. -- Michael

Re: PG vs macOS Mojave

2018-10-31 Thread Tom Lane
[ just when you thought it was safe to go back in the water ] I wrote: >> Jakob Egger writes: >>> I would assume that clang sets -isysroot automatically, but I have no idea >>> why that didn't work for you previously. >> [ experiments further ... ] It looks like clang does default to assuming

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-10-31 Thread David Rowley
On 1 November 2018 at 06:45, Robert Haas wrote: > On Wed, Aug 22, 2018 at 8:30 AM David Rowley > wrote: >> On 22 August 2018 at 19:08, Amit Langote >> wrote: >> > +#define PartitionTupRoutingGetToParentMap(p, i) \ >> > +#define PartitionTupRoutingGetToChildMap(p, i) \ >> > >> > If the "Get"

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-10-31 Thread Amit Langote
On 2018/11/01 8:58, David Rowley wrote: > On 1 November 2018 at 06:45, Robert Haas wrote: >> On Wed, Aug 22, 2018 at 8:30 AM David Rowley >> wrote: >>> On 22 August 2018 at 19:08, Amit Langote >>> wrote: +#define PartitionTupRoutingGetToParentMap(p, i) \ +#define

Re: Speeding up INSERTs and UPDATEs to partitioned tables

2018-10-31 Thread Gavin Flower
On 01/11/2018 14:30, David Rowley wrote: On 1 November 2018 at 13:35, Amit Langote wrote: On 2018/11/01 8:58, David Rowley wrote: [...] I agree. I don't think "TupRouting" really needs to be in the name. Probably "To" can also just become "2" and we can put back the Parent/Child before

Re: [HACKERS] Block level parallel vacuum

2018-10-31 Thread Yura Sokolov
Excuse me for being noisy. Increasing vacuum's ring buffer improves vacuum upto 6 times. https://www.postgresql.org/message-id/flat/20170720190405.GM1769%40tamriel.snowman.net This is one-line change. How much improvement parallel vacuum gives? 31.10.2018 3:23, Masahiko Sawada пишет: > On Tue,

Re: WIP Patch: Add a function that returns binary JSONB as a bytea

2018-10-31 Thread Tom Lane
Kevin Van writes: > This patch adds a new function that allows callers to receive binary jsonb. > This change was proposed in the discussion here: > https://www.postgresql.org/message-id/CAOsiKEL7%2BKkV0C_hAJWxqwTg%2BPYVfiGPQ0yjFww7ECtqwBjb%2BQ%40mail.gmail.com > and the primary motivation is to

Re: ToDo: show size of partitioned table

2018-10-31 Thread Pavel Stehule
st 31. 10. 2018 v 3:27 odesílatel Amit Langote < langote_amit...@lab.ntt.co.jp> napsal: > On 2018/10/30 20:03, Pavel Stehule wrote: > > út 30. 10. 2018 v 7:52 odesílatel Amit Langote < > > langote_amit...@lab.ntt.co.jp> napsal: > >> Could one of you please revise the patch to use that function to

Re: [HACKERS] generated columns

2018-10-31 Thread Michael Paquier
On Tue, Oct 30, 2018 at 09:35:18AM +0100, Peter Eisentraut wrote: > Attached is a new version of this patch. Thanks Peter for sending a new patch. I am still assigned as a reviewer, and still plan to look at it in more details. > It supports both computed-on-write and computed-on-read variants,

Re: jsonpath

2018-10-31 Thread Oleg Bartunov
On Mon, Oct 29, 2018 at 2:20 AM Tomas Vondra wrote: > > Hi, > > On 10/02/2018 04:33 AM, Michael Paquier wrote: > > On Sat, Sep 08, 2018 at 02:21:27AM +0300, Nikita Glukhov wrote: > >> Attached 18th version of the patches rebased onto the current master. > > > > Nikita, this version fails to

Re: ToDo: show size of partitioned table

2018-10-31 Thread Pavel Stehule
st 31. 10. 2018 v 8:38 odesílatel Michael Paquier napsal: > On Wed, Oct 31, 2018 at 03:34:02PM +0900, Amit Langote wrote: > > On 2018/10/31 15:30, Pavel Stehule wrote: > >> I am not sure. Has not sense run this test over empty database, and some > >> bigger database can increase running. > >> >

Re: [HACKERS] generated columns

2018-10-31 Thread Erikjan Rijkers
On 2018-10-30 16:14, Sergei Kornilov wrote: Hi I applied this patch on top 2fe42baf7c1ad96b5f9eb898161e258315298351 commit and found a bug while adding STORED column: postgres=# create table test(i int); CREATE TABLE postgres=# insert into test values (1),(2); INSERT 0 2 postgres=# alter table

Re: WIP: Avoid creation of the free space map for small tables

2018-10-31 Thread John Naylor
Upthread I wrote: > -A possible TODO item is to teach pg_upgrade not to link FSMs for > small heaps. I haven't look into the feasibility of that, however. This turned out to be relatively light weight (0002 attached). I had to add relkind to the RelInfo struct and save the size of each heap as

RE: Timeout parameters

2018-10-31 Thread Nagaura, Ryohei
Hi Andrei, First, I inform you that I may not contact for the following period: From November 1st to November 19th Second, I noticed my misunderstanding in previous mail. > > Nevertheless, it is necessary to take into account that the option > > TCP_USER_TIMEOUT is supported by Linux kernel

Re: Ordered Partitioned Table Scans

2018-10-31 Thread David Rowley
On 31 October 2018 at 13:05, David Rowley wrote: >>> On 28 October 2018 at 03:49, Julien Rouhaud wrote: >> I've registered as a reviewer. I still didn't have a deep look at >> the patch yet, but thanks a lot for working on it! > > Thanks for signing up to review. I need to send another

syntax error: VACUUM ANALYZE VERBOSE (PostgreSQL 11 regression)

2018-10-31 Thread Pavel Stehule
Hi I have report of one customer. Some scripts stopped on 11 because VACUUM ANALYZE VERBOSE doesn't work now. postgres=# vacuum analyze verbose; ERROR: syntax error at or near "verbose" LINE 1: vacuum analyze verbose; ^ vacuum verbose analyze is working. Regards Pavel

Re: ToDo: show size of partitioned table

2018-10-31 Thread Michael Paquier
On Wed, Oct 31, 2018 at 03:34:02PM +0900, Amit Langote wrote: > On 2018/10/31 15:30, Pavel Stehule wrote: >> I am not sure. Has not sense run this test over empty database, and some >> bigger database can increase running. >> >> More the size can be platform depend. > > Okay, sure. Well, that

Re: syntax error: VACUUM ANALYZE VERBOSE (PostgreSQL 11 regression)

2018-10-31 Thread Pavel Stehule
st 31. 10. 2018 v 8:34 odesílatel Sergei Kornilov napsal: > Hi > > At least this is documented behavior: > > When the option list is surrounded by parentheses, the options can be > written in any order. Without parentheses, options must be specified in > exactly the order shown above. >

Re: syntax error: VACUUM ANALYZE VERBOSE (PostgreSQL 11 regression)

2018-10-31 Thread Michael Paquier
On Wed, Oct 31, 2018 at 08:38:27AM +0100, Pavel Stehule wrote: > ok. Unfortunatelly it is not mentioned in release notes - in not compatible > changes. > > This change can hit lot of users. It is small nonsense, but lot of people > use it. Please just look at the order of the words in the

Re: COPY FROM WHEN condition

2018-10-31 Thread Masahiko Sawada
On Tue, Oct 30, 2018 at 11:47 PM Surafel Temesgen wrote: > > > Hi, > Thank you for looking at it . > On Sun, Oct 28, 2018 at 7:19 PM Tomas Vondra > wrote: >> >> >> 1) I think this deserves at least some regression tests. Plenty of tests >> already use COPY, but there's no coverage for the new

Re: ToDo: show size of partitioned table

2018-10-31 Thread Michael Paquier
On Wed, Oct 31, 2018 at 08:41:45AM +0100, Pavel Stehule wrote: > I am not sure - I remember one private test that we did on our patches, and > this tests fails sometimes on 32bits. So I afraid about stability. That could be possible as well. I think that you are right to be afraid of such

Re: [HACKERS] generated columns

2018-10-31 Thread Erik Rijkers
On 2018-10-31 09:15, Simon Riggs wrote: On Wed, 31 Oct 2018 at 07:58, Erikjan Rijkers wrote: I have also noticed that logical replication isn't possible on tables with a generated column. That's a shame but I suppsoe that is as expected. Couldn't see anything like that in the patch.

Re: syntax error: VACUUM ANALYZE VERBOSE (PostgreSQL 11 regression)

2018-10-31 Thread Pavel Stehule
st 31. 10. 2018 v 8:55 odesílatel Michael Paquier napsal: > On Wed, Oct 31, 2018 at 08:38:27AM +0100, Pavel Stehule wrote: > > ok. Unfortunatelly it is not mentioned in release notes - in not > compatible > > changes. > > > > This change can hit lot of users. It is small nonsense, but lot of

Re: ToDo: show size of partitioned table

2018-10-31 Thread Amit Langote
On 2018/10/31 15:30, Pavel Stehule wrote: > st 31. 10. 2018 v 3:27 odesílatel Amit Langote < > langote_amit...@lab.ntt.co.jp> napsal: >> +appendPQExpBufferStr(, "\nWHERE c.relkind IN ('p')\n"); >> >> I wonder if we should list partitioned indexes ('I') as well, because >> their size

Re: syntax error: VACUUM ANALYZE VERBOSE (PostgreSQL 11 regression)

2018-10-31 Thread Sergei Kornilov
Hi At least this is documented behavior: > When the option list is surrounded by parentheses, the options can be written > in any order. Without parentheses, options must be specified in exactly the > order shown above. https://www.postgresql.org/docs/current/static/sql-vacuum.html Previously

Re: syntax error: VACUUM ANALYZE VERBOSE (PostgreSQL 11 regression)

2018-10-31 Thread Michael Paquier
On Wed, Oct 31, 2018 at 08:19:35AM +0100, Pavel Stehule wrote: > I have report of one customer. Some scripts stopped on 11 because VACUUM > ANALYZE VERBOSE doesn't work now. > > postgres=# vacuum analyze verbose; > ERROR: syntax error at or near "verbose" > LINE 1: vacuum analyze verbose; >

Re: [HACKERS] generated columns

2018-10-31 Thread Simon Riggs
On Wed, 31 Oct 2018 at 07:58, Erikjan Rijkers wrote: > I have also noticed that logical replication isn't possible on tables > with a generated column. That's a shame but I suppsoe that is as > expected. > Couldn't see anything like that in the patch. Presumably unintended consequence. The

Is there way to detect uncommitted 'new table' in pg_class?

2018-10-31 Thread Hubert Zhang
Hi all, In PG READ UNCOMMITTED is treated as READ COMMITTED But I have a requirement to read dirty table. Is there way to detect table which is created in other uncommitted transaction? T1: BEGIN; create table a(i int); T2: select * from pg_class where relname='a'; could return table a? --

Re: Pluggable Storage - Andres's take

2018-10-31 Thread Dmitry Dolgov
> On Mon, 29 Oct 2018 at 05:56, Haribabu Kommi wrote: > >> This problem couldn't be reproduced on the master branch, so I've tried to >> investigate it. It comes from nodeModifyTable.c:1267, when we've got >> HeapTupleInvisible as a result, and this value in turn comes from >> table_lock_tuple.

Re: [PATCH][PROPOSAL] Add enum releation option type

2018-10-31 Thread Nikolay Shaplov
В письме от 12 сентября 2018 21:40:49 пользователь Nikolay Shaplov написал: > > As you mentioned in previous mail, you prefer to keep enum and > > relopt_enum_element_definition array in the same .h file. I'm not sure, > > but I think it is done to keep everything related to enum in one place > >