Re: New vacuum option to do only freezing

2019-03-31 Thread Masahiko Sawada
On Sat, Mar 30, 2019 at 5:04 AM Robert Haas wrote: > > On Fri, Mar 29, 2019 at 12:27 PM Masahiko Sawada > wrote: > > Yeah, but since multiple relations might be specified in VACUUM > > command we need to process index_cleanup option after opened each > > relations. Maybe we need to process all

Re: speeding up planning with partitions

2019-03-31 Thread Amit Langote
(I've closed the CF entry: https://commitfest.postgresql.org/22/1778/) On 2019/04/01 2:04, Tom Lane wrote: > Amit Langote writes: >> On Sun, Mar 31, 2019 at 11:45 AM Imai Yoshikazu >> wrote: >>> Certainly, using bitmapset contributes to the performance when scanning >>> one partition(few

Re: monitoring CREATE INDEX [CONCURRENTLY]

2019-03-31 Thread Rahila Syed
Hi Alvaro, Please see few comments below: 1. Makecheck fails currently as view definition of expected rules.out does not reflect latest changes in progress metrics numbering. 2. + + When creating an index on a partitioned, this column is set to the + total number of partitions

Re: Implementing Incremental View Maintenance

2019-03-31 Thread Yugo Nagata
On Thu, 27 Dec 2018 21:57:26 +0900 Yugo Nagata wrote: > Hi, > > I would like to implement Incremental View Maintenance (IVM) on PostgreSQL. I am now working on an initial patch for implementing IVM on PostgreSQL. This enables materialized views to be updated incrementally after one of their

GSoC proposal for pgAdmin 4 bytea support

2019-03-31 Thread Haoran Yu
Dear PostgreSQL community, I have submitted a proposal for the project pgAdmin 4 bytea support. The project discusses storing media content (images, audio, video) as bytea. However, I have a quick question. What does bytea data look like typically when storing media content? What I had in mind

Re: Protect syscache from bloating with negative cache entries

2019-03-31 Thread Kyotaro HORIGUCHI
At Fri, 29 Mar 2019 17:24:40 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20190329.172440.199616830.horiguchi.kyot...@lab.ntt.co.jp> > I ran three artificial test cases. The database is created by > gen_tbl.pl. Numbers are the average of the fastest five runs in > successive 15

RE: Timeout parameters

2019-03-31 Thread Nagaura, Ryohei
Hello, Fabien-san. > From: Fabien COELHO [mailto:coe...@cri.ensmp.fr] > I have further remarks after Kirk-san extensive review on these patches. Yes, I'm welcome. Thank you very much for your review. > * About TCP interface v18. > For homogeneity with the surrounding cases, ISTM that

Re: DWIM mode for psql

2019-03-31 Thread Amit Langote
Hi Thomas, Thanks for working on this. On Mon, Apr 1, 2019 at 5:53 Thomas Munro wrote: Hello, > > Building on the excellent work begun by commit e529cd4ffa60, I would > like to propose a do-what-I-mean mode for psql. Please find a POC > patch attached. It works like this: > > postgres=#

Re: speeding up planning with partitions

2019-03-31 Thread David Rowley
On Sun, 31 Mar 2019 at 05:50, Robert Haas wrote: > > On Sat, Mar 30, 2019 at 12:16 PM Amit Langote wrote: > > Fwiw, I had complained when reviewing the run-time pruning patch that > > creating those maps in the planner and putting them in > > PartitionPruneInfo might not be a good idea, but

Re: idle-in-transaction timeout error does not give a hint

2019-03-31 Thread Tatsuo Ishii
>>From: Tatsuo Ishii [mailto:is...@sraoss.co.jp] >> >>> Personally, I don't find this hint particularly necessary. The >>> session was terminated because nothing was happening, so the real fix >>> on the application side is probably more involved than just retrying. >>> This is different from

RE: idle-in-transaction timeout error does not give a hint

2019-03-31 Thread Ideriha, Takeshi
>From: Tatsuo Ishii [mailto:is...@sraoss.co.jp] > >> Personally, I don't find this hint particularly necessary. The >> session was terminated because nothing was happening, so the real fix >> on the application side is probably more involved than just retrying. >> This is different from some of

Re: [HACKERS] Weaker shmem interlock w/o postmaster.pid

2019-03-31 Thread Noah Misch
On Fri, Mar 29, 2019 at 09:53:51AM +, Daniel Gustafsson wrote: > On Saturday, March 9, 2019 8:16 AM, Noah Misch wrote: > > I renamed IpcMemoryAnalyze() to PGSharedMemoryAttach() and deleted the old > > function of that name. Now, this function never calls shmdt(); the caller is > >

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-03-31 Thread Noah Misch
On Sun, Mar 10, 2019 at 07:27:08PM -0700, Noah Misch wrote: > I also liked the design in the https://postgr.es/m/559fa0ba.3080...@iki.fi > last paragraph, and I suspect it would have been no harder to back-patch. I > wonder if it would have been simpler and better, but I'm not asking anyone to >

Re: FETCH FIRST clause PERCENT option

2019-03-31 Thread Tom Lane
Andres Freund writes: >> offset_clause: >> @@ -15435,6 +15442,7 @@ reserved_keyword: >> | ONLY >> | OR >> | ORDER >> +| PERCENT >> | PLACING >> | PRIMARY >>

Re: DWIM mode for psql

2019-03-31 Thread Andreas Karlsson
On 3/31/19 10:52 PM, Thomas Munro wrote:> Building on the excellent work begun by commit e529cd4ffa60, I would like to propose a do-what-I-mean mode for psql. Please find a POC patch attached. It works like this: postgres=# select datnaam from pg_database where ooid = 12917; ERROR: column

Re: FETCH FIRST clause PERCENT option

2019-03-31 Thread Andres Freund
Hi, On 2019-03-29 12:04:50 +0300, Surafel Temesgen wrote: > + if (node->limitOption == PERCENTAGE) > + { > + while (node->position - node->offset < > node->count) > +

Re: DWIM mode for psql

2019-03-31 Thread Corey Huinker
On Sun, Mar 31, 2019 at 5:04 PM Andres Freund wrote: > On 2019-04-01 09:52:34 +1300, Thomas Munro wrote: > > +/* > > + * This program is free software: you can redistribute it and/or modify > > + * it under the terms of the GNU General Public License as published by > > + * the Free Software

Re: DWIM mode for psql

2019-03-31 Thread Andres Freund
On 2019-04-01 09:52:34 +1300, Thomas Munro wrote: > +/* > + * This program is free software: you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation, either version 3 of the License, or > + * (at your

DWIM mode for psql

2019-03-31 Thread Thomas Munro
Hello, Building on the excellent work begun by commit e529cd4ffa60, I would like to propose a do-what-I-mean mode for psql. Please find a POC patch attached. It works like this: postgres=# select datnaam from pg_database where ooid = 12917; ERROR: column "datnaam" does not exist LINE 1:

Re: COPY FROM WHEN condition

2019-03-31 Thread Andres Freund
Hi, On 2019-04-01 02:00:26 +1300, David Rowley wrote: > On Fri, 29 Mar 2019 at 01:15, Andres Freund wrote: > > On 2019-03-28 20:48:47 +1300, David Rowley wrote: > > > I had a look at this and performance has improved again, thanks. > > > However, I'm not sure if the patch is exactly what we

Re: Ltree syntax improvement

2019-03-31 Thread Nikolay Shaplov
В письме от четверг, 7 марта 2019 г. 13:09:49 MSK пользователь Chris Travers написал: > We maintain an extension (https://github.com/adjust/wltree) > which has a fixed separator (::) and allows any utf-8 character in the tree. > > In our case we currently use our extended tree to store

Re: speeding up planning with partitions

2019-03-31 Thread Tom Lane
One thing that I intentionally left out of the committed patch was changes to stop short of scanning the whole simple_rel_array when looking only for baserels. I thought that had been done in a rather piecemeal fashion and it'd be better to address it holistically, which I've now done in the

RE: jsonpath

2019-03-31 Thread Tom Turelinckx
Tom Lane wrote: > I assume this trace is from this run? > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skate=2019-03-31%2006%3A24%3A35 Yes. I did get a core file now, but it wasn't picked up by the buildfarm script, so I extracted the backtrace manually. > That looks a whole lot like

RE: jsonpath

2019-03-31 Thread Tom Turelinckx
Alexander Korotkov wrote: > Hmm... 550b9d26f just makes jsonpath_gram.y and jsonpath_scan.l > compile at once. I've re-read this commit and didn't find anything > suspicious. > I've asked Andrew for access to jacana in order to investigate this myself. Stack trace from skate: [New LWP 6614]

Re: jsonpath

2019-03-31 Thread Andrew Dunstan
On 3/31/19 12:21 PM, Tom Turelinckx wrote: > Tom Lane wrote: > >> I assume this trace is from this run? >> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skate=2019-03-31%2006%3A24%3A35 > Yes. I did get a core file now, but it wasn't picked up by the buildfarm > script, so I extracted

Re: Google Summer of Code: question about GiST API advancement project

2019-03-31 Thread Andrey Borodin
Hi! > 31 марта 2019 г., в 14:58, GUO Rui написал(а): > > I'm Rui Guo, a PhD student focusing on database at the University of > California, Irvine. I'm interested in the "GiST API advancement" project for > the Google Summer of Code 2019 which is listed at >

Re: speeding up planning with partitions

2019-03-31 Thread Tom Lane
Amit Langote writes: > On Sun, Mar 31, 2019 at 11:45 AM Imai Yoshikazu > wrote: >> Certainly, using bitmapset contributes to the performance when scanning >> one partition(few partitions) from large partitions. > Thanks Imai-san for testing. I tried to replicate these numbers with the code

Re: [HACKERS][Proposal] LZ4 Compressed Storage Manager

2019-03-31 Thread Nikolay Petrov
31.03.2019, 17:26, "Nikolay Petrov" : > Hello everyone! > Thank you for your interest to this topic. > > I would like to propose Compressed Storage Manager for PostgreSQL. Previous thread here https://www.postgresql.org/message-id/flat/op.ux8if71gcigqcu%40soyouz And the result of previous

Re: [PATCH] get rid of StdRdOptions, use individual binary reloptions representation for each relation kind instead

2019-03-31 Thread Nikolay Shaplov
В письме от среда, 20 марта 2019 г. 6:15:38 MSK пользователь Iwata, Aya написал: > You told us "big picture" about opclass around the beginning of this thread. > In my understanding, the purpose of this refactoring is to make reloptions > more flexible to add opclass. I understand this change

Re: [PATCH] get rid of StdRdOptions, use individual binary reloptions representation for each relation kind instead

2019-03-31 Thread Nikolay Shaplov
В письме от понедельник, 18 марта 2019 г. 17:00:24 MSK пользователь Kyotaro HORIGUCHI написал: > > So I change status to "Waiting for Author". > That seems to be a good oppotunity. I have some comments. > > rel.h: > -#define RelationGetToastTupleTarget(relation, defaulttarg) \ > -

Re: [PATCH] get rid of StdRdOptions, use individual binary reloptions representation for each relation kind instead

2019-03-31 Thread Nikolay Shaplov
В письме от понедельник, 18 марта 2019 г. 3:03:04 MSK пользователь Iwata, Aya написал: > Hi Nikolay, Hi! Sorry for long delay. Postgres is not my primary work, so sometimes it takes a while to get to it. > This patch does not apply. Oh... Sorry... here goes new version > Please refer to

Re: jsonpath

2019-03-31 Thread Tom Lane
"Tom Turelinckx" writes: > Stack trace from skate: Huh ... so that's nowhere near the jsonpath-syntax-error crash that we saw before. I assume this trace is from this run? https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skate=2019-03-31%2006%3A24%3A35 That looks a whole lot like the

Re: Problem during Windows service start

2019-03-31 Thread Ramanarayana
Hi, If wait_for_postmaster returns POSTMASTER_STILL_STARTING will it be correct to set the status of windows service to SERVICE_START_PENDING ? I would like to take this up if no one is working on this. Regards, Ram.

[HACKERS][Proposal] LZ4 Compressed Storage Manager

2019-03-31 Thread Николай Петров
Hello everyone! Thank you for your interest to this topic. I would like to propose Compressed Storage Manager for PostgreSQL. The problem: In cases when you store some log-like data in your tables, or when you store time-series data you may face with high disk space consumption because of a

Re: pgsql: Improve autovacuum logging for aggressive and anti-wraparound ru

2019-03-31 Thread Michael Paquier
On Sat, Mar 30, 2019 at 10:12:33PM +0900, Michael Paquier wrote: > Okay, I'll use that then. And finally committed. I have changed the debug1 message so as "to prevent wraparound" is used instead of "anti-wraparound". I have noticed something which was also missing from all the patches proposed

Re: [HACKERS] generated columns

2019-03-31 Thread Jeff Janes
On Sat, Mar 30, 2019 at 4:03 AM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2019-03-26 20:50, Pavel Stehule wrote: > > It is great feature and I'll mark this feature as ready for commit > > Committed, thanks. > I can't do a same-major-version pg_upgrade across this commit,

Re: COPY FROM WHEN condition

2019-03-31 Thread David Rowley
On Fri, 29 Mar 2019 at 01:15, Andres Freund wrote: > On 2019-03-28 20:48:47 +1300, David Rowley wrote: > > I had a look at this and performance has improved again, thanks. > > However, I'm not sure if the patch is exactly what we need, let me > > explain. > > I'm not entirely sure either, I just

Re: Berserk Autovacuum (let's save next Mandrill)

2019-03-31 Thread Komяpa
> > > > Idea: look not on dead tuples, but on changes, just like ANALYZE does. > > It's my first patch on Postgres, it's probably all wrong but I hope it > > helps you get the idea. > > This was suggested and rejected years ago: > >

Re: Berserk Autovacuum (let's save next Mandrill)

2019-03-31 Thread Komяpa
> > If it's months, we probably want limit vacuum to working at a pretty > slow rate, say 1% of the table size per hour or something. If it's in > hours, we need to be a lot more aggressive. Right now we have no > information to tell us which of those things is the case, so we'd just > be

Re: Berserk Autovacuum (let's save next Mandrill)

2019-03-31 Thread Komяpa
> > By the way, the Routine Vacuuming chapter of the documentation says: > > "The sole disadvantage of increasing autovacuum_freeze_max_age (and > vacuum_freeze_table_age along with it) is that the pg_xact and > pg_commit_ts subdirectories of the database cluster will take more space > > [...] > >

Re: Berserk Autovacuum (let's save next Mandrill)

2019-03-31 Thread Komяpa
On Thu, Mar 28, 2019 at 6:43 PM Masahiko Sawada wrote: > >> 1. heap vacuum > >> > >> 2. HOT pruning > > > > Is it worth skipping it if we're writing a page anyway for the sake of > hint bits and new xids? This will all be no-op anyway on append-only tables > and happen only when we actually need

Google Summer of Code: question about GiST API advancement project

2019-03-31 Thread GUO Rui
Hi, hackers, I'm Rui Guo, a PhD student focusing on database at the University of California, Irvine. I'm interested in the "GiST API advancement" project for the Google Summer of Code 2019 which is listed at https://wiki.postgresql.org/wiki/GSoC_2019#GiST_API_advancement_.282019.29 . I'm still