Re: reducing the footprint of ScanKeyword (was Re: Large writable variables)

2019-01-06 Thread David Rowley
On Sat, 5 Jan 2019 at 09:20, John Naylor wrote: > > On 1/3/19, Joerg Sonnenberger wrote: > > Hello John, > > I was pointed at your patch on IRC and decided to look into adding my > > own pieces. What I can provide you is a fast perfect hash function > > generator. I've attached a sample hash

Re: FETCH FIRST clause PERCENT option

2019-01-06 Thread Tomas Vondra
On 1/6/19 12:40 PM, Surafel Temesgen wrote: > > > On Fri, Jan 4, 2019 at 5:27 PM Tomas Vondra > mailto:tomas.von...@2ndquadrant.com>> wrote: > > > What formula? All the math remains exactly the same, you just need to > update the number of rows to return and track how many rows are

Re: Thinking about EXPLAIN ALTER TABLE

2019-01-06 Thread Greg Stark
On Mon, 10 Dec 2018 at 11:14, Alvaro Herrera wrote: > > Hi Greg > > On 2018-Dec-07, Greg Stark wrote: > > > I'm thinking I should try to move all these decisions to phase 1 as > > much as possible but I'm not sure how feasible it will be to get the > > results exactly correct. Of course the cases

Re: FETCH FIRST clause PERCENT option

2019-01-06 Thread Surafel Temesgen
On Fri, Jan 4, 2019 at 5:27 PM Tomas Vondra wrote: > > What formula? All the math remains exactly the same, you just need to > update the number of rows to return and track how many rows are already > returned. > > I haven't tried doing that, but AFAICS you'd need to tweak how/when > node->count

Re: chained transactions

2019-01-06 Thread Fabien COELHO
Hello Peter, Sure. Within a read-only tx, it could check that transaction_read_only is on, and still on when chained, though. I think the tests prove the point that the values are set and unset and reset in various scenarios. We don't need to test every single combination, that's not the

Re: START/END line number for COPY FROM

2019-01-06 Thread Surafel Temesgen
Hi, On Sat, Jan 5, 2019 at 1:10 PM David Rowley wrote: > On Fri, 21 Dec 2018 at 02:02, Surafel Temesgen > wrote: > > Currently we can skip header line on COPY FROM but having the ability to > skip and stop copying at any line can use to divide long copy operation and > enable to copy a subset

Re: chained transactions

2019-01-06 Thread Fabien COELHO
Hello Peter, I'm wary of changing the SPI_commit and SPI_rollback interfaces which are certainly being used outside the source tree and could break countless code, and it seems quite unclean that commit and rollback would do anything else but committing or rollbacking. These are new as of

Re: reducing the footprint of ScanKeyword (was Re: Large writable variables)

2019-01-06 Thread Joerg Sonnenberger
On Mon, Jan 07, 2019 at 03:11:55AM +1300, David Rowley wrote: > What I'm most interested in is how long it took to generate the hash > function in hash2.c? It's within the noise floor of time(1) on my laptop, e.g. ~1ms. Joerg

Re: Ordered Partitioned Table Scans

2019-01-06 Thread Julien Rouhaud
On Sun, Jan 6, 2019 at 4:24 AM David Rowley wrote: > > On Thu, 20 Dec 2018 at 18:20, Julien Rouhaud wrote: > > > > > > No, what I'm proposing is to store if the partitions are naturally > > ordered or not, *and* recheck after pruning if that property could > > have changed (so if some partitions

Re: [PATCH] check for ctags utility in make_ctags

2019-01-06 Thread Nikolay Shaplov
В письме от четверг, 3 января 2019 г. 12:52:36 MSK пользователь Peter Eisentraut написал: > >> +1, let's keep it simple. I would just use "ctags/etags not found" > >> as error message. > > > > Actually I was trying to say "Please install 'ctags' [utility] to run > > make_ctags". But if all of

Re: Remove Deprecated Exclusive Backup Mode

2019-01-06 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Dec 13, 2018 at 2:29 PM David Steele wrote: > > We could leave the third parameter (changing the default to false) and > > error if it has any value but false. It's a bit ugly but it does > > maintain compatibility with the

Re: [PATCH] check for ctags utility in make_ctags

2019-01-06 Thread Tom Lane
Nikolay Shaplov writes: > В письме от четверг, 3 января 2019 г. 12:52:36 MSK пользователь Peter > Eisentraut написал: >> I don't know how portable command -v is. Some systems have a /bin/sh >> that is pre-POSIX. Same with $(...). > Do you know how to obtain such a shell in Debian? TBH, when

Re: [PATCH][PROPOSAL] Add enum releation option type

2019-01-06 Thread Nikolay Shaplov
В письме от четверг, 3 января 2019 г. 18:12:05 MSK пользователь Alvaro Herrera написал: > Attached version 7, with some renaming and rewording of comments. > (I also pgindented. Some things are not pretty because of lack of > typedefs.list patching ... a minor issue at worst.) Thanks! Imported

Re: [HACKERS] pgbench - allow to store select results into variables

2019-01-06 Thread Fabien COELHO
Hello Alvaro, I revised this patch a bit. Here's v25, where some finishing touches are needed -- see below. I think with these changes the patch would become committable, at least for me. Thanks a lot for having a look a this patch, and improving it. The updated version did not work,

Re: [PATCH] kNN for btree

2019-01-06 Thread Alexander Korotkov
On Sun, Dec 30, 2018 at 1:19 AM Alexander Korotkov wrote: > On Thu, Dec 27, 2018 at 5:46 AM Alexander Korotkov > wrote: > > * 0006-Remove-distance-operators-from-btree_gist-v04.patch > > > > I see you provide btree_gist--1.6.sql and remove btree_gist--1.2.sql. > > Note, that in order to better

Re: Adding a concept of TEMPORARY TABLESPACE for the use in temp_tablespaces

2019-01-06 Thread Mitar
Hi! On Sun, Jan 6, 2019 at 11:01 AM Mitar wrote: > I have experimented a bit and performance really improves if /dev/shm > is used. I have experimented with creating temporary tables inside a > regular (SSD backed) tablespace /dev/shm and I have seen at least 2x > improvement in time it takes

Adding a concept of TEMPORARY TABLESPACE for the use in temp_tablespaces

2019-01-06 Thread Mitar
Hi! I have read around the Internet a lot about the idea of using /dev/shm for a tablespace to put tables in and issues with that. But I still have not managed to get a good grasp why would that be a bad idea for using it for temporary objects. I understand that for regular tables this might

Re: reducing the footprint of ScanKeyword (was Re: Large writable variables)

2019-01-06 Thread Tom Lane
Joerg Sonnenberger writes: > On Mon, Jan 07, 2019 at 03:11:55AM +1300, David Rowley wrote: >> What I'm most interested in is how long it took to generate the hash >> function in hash2.c? > It's within the noise floor of time(1) on my laptop, e.g. ~1ms. I decided to do some simple performance

Re: reducing the footprint of ScanKeyword (was Re: Large writable variables)

2019-01-06 Thread Tom Lane
Joerg Sonnenberger writes: > On Sun, Jan 06, 2019 at 02:29:05PM -0500, Tom Lane wrote: >> * We should extend the ScanKeywordList representation to include a >> field holding the longest keyword length in the table, which >> gen_keywordlist.pl would have no trouble providing. Then we could >>

Re: [PATCH] check for ctags utility in make_ctags

2019-01-06 Thread Andrew Dunstan
On 1/6/19 12:16 PM, Tom Lane wrote: > > The correct way to code this is to depend on the exit code, > not the text output: > > if command -v etags >/dev/null > then > : ok > else > echo etags not found > exit 1 > fi more succinctly, command -v etags >/dev/null || { echo etags not

Re: Python versions (was Re: RHEL 8.0 build)

2019-01-06 Thread Tom Lane
Peter Eisentraut writes: > I think this was the option with the most support. Here is a patch. BTW, this is a pre-existing problem not the fault of this patch, but while we're fooling with the behavior of python lookup would be a great time to fix it: we should add something like

Re: reducing the footprint of ScanKeyword (was Re: Large writable variables)

2019-01-06 Thread Tom Lane
Joerg Sonnenberger writes: > On Sun, Jan 06, 2019 at 02:29:05PM -0500, Tom Lane wrote: >> So we probably can't have inlined hashing code --- I imagine the >> hash generator needs the flexibility to pick different values of >> those multipliers. > Right now, only the initial values are

Re: reducing the footprint of ScanKeyword (was Re: Large writable variables)

2019-01-06 Thread Joerg Sonnenberger
On Sun, Jan 06, 2019 at 02:29:05PM -0500, Tom Lane wrote: > * It's too bad that the hash function doesn't have a return convention > that allows distinguishing "couldn't possibly match any keyword" from > "might match keyword 0". I imagine a lot of the zero entries in its > hashtable could be

RE: [Proposal] Add accumulated statistics

2019-01-06 Thread Tsunakawa, Takayuki
From: Yotsunaga, Naoki [mailto:yotsunaga.na...@jp.fujitsu.com] > By the way, you can see the number of wait events with "LWLOCK_STATS", right? > Is this function implemented because it is necessary to know the number > of waiting events for investigation? > If so, is not that the number of wait

Re: Implementing Incremental View Maintenance

2019-01-06 Thread Tatsuo Ishii
> Hi all, just wanted to say I am very happy to see progress made on this, > my codebase has multiple "materialized tables" which are maintained with > statement triggers (transition tables) and custom functions. They are ugly > and a pain to maintain, but they work because I have no other >

Prevent extension creation in temporary schemas

2019-01-06 Thread Michael Paquier
Hi all, While looking at another bug I have noticed that it is possible to create an extension directly using a temporary schema, which is crazy. A simple example: =# create extension pg_prewarm with schema pg_temp_3; CREATE EXTENSION =# \dx pg_prewarm List of installed extensions

Noria and Postgres

2019-01-06 Thread Darin Gordon
Hey Postgres Team! I wanted to bring to everyone's attention a very interesting database, called Noria, written in Rust. It offers a compelling alternative to common dev patterns intended to boost performance of relational database interactions. The author only implemented a mysql backend at

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-01-06 Thread Tomas Vondra
FWIW the main unsolved issue (at least on the MCV part) is how it decides which items to keep in the list. As explained in [1], in the multivariate case we can't simply look at the group frequency and compare it to the average frequency (of the non-MCV items), which is what analyze_mcv_list()

Re: Implementing Incremental View Maintenance

2019-01-06 Thread Nguyễn Trần Quốc Vinh
Dear All, The tool analyzes the input query and then generates triggers (trigger functions and pl/pgsql scripts as well) on all manipulating events (insert/updates/delete) for all underlying base tables. The triggers do incremental updates to the table that contains the query result (MV). You can

Doc client_min_messages patch vis. INFO message severity

2019-01-06 Thread Karl O. Pinc
Hi, Attached is documentation patch: doc_client_min_messages_v1.patch Document that INFO severity messages are always sent to the client. This also adds hyperlinks to the table of severity levels where those levels are referenced in the docs. The patch was discussed on the #postgresql IRC

RE: [Proposal] Add accumulated statistics

2019-01-06 Thread Tsunakawa, Takayuki
Hi all, I think sampling like Oracle ASH should work for the DBA to find probable bottlenecks in many cases (, so I hope PostgreSQL will incorporate it...) On the other hand, it seems to have the following disadvantages, some of which others have already pointed out: 1. Doesn't provide

Re: [Sender Address Forgery]Re: error message when subscription target is a partitioned table

2019-01-06 Thread Amit Langote
Thanks for reviewing. On 2018/12/31 20:23, Peter Eisentraut wrote: > On 06/12/2018 05:46, Amit Langote wrote: >> /* >> * We currently only support writing to regular tables. >> */ > > I think that comment should stay above the code you are adding. Do you mean to keep it at the

Re: btree.sgml typo?

2019-01-06 Thread Tatsuo Ishii
> On Sat, Jan 5, 2019 at 1:35 AM Tatsuo Ishii wrote: >> PostgreSQL includes an implementation of the >> standard btree (multi-way binary tree) index data >> structure. >> >> I think the term "btree" here means "multi-way balanced tree", rather >> than "multi-way binary tree". In fact in our

Re: A few new options for vacuumdb

2019-01-06 Thread Michael Paquier
On Fri, Jan 04, 2019 at 11:49:46PM +, Bossart, Nathan wrote: > Here's an updated set of patches with the following changes: > > - 0002 adds the parenthesized syntax for ANALYZE. > - 0003 adds DISABLE_PAGE_SKIPPING for VACUUM. > - 0003 also ensures SKIP_LOCKED is applied for --analyze-only.

Re: [Sender Address Forgery]Re: error message when subscription target is a partitioned table

2019-01-06 Thread Michael Paquier
On Mon, Jan 07, 2019 at 01:49:49PM +0900, Amit Langote wrote: > { > /* > - * We currently only support writing to regular tables. > + * We currently only support writing to regular tables. However, give > + * a more specific error for partitioned and foreign tables. >