Re: [HACKERS] WAL logging problem in 9.4.3?

2019-03-20 Thread Noah Misch
On Wed, Mar 20, 2019 at 05:17:54PM +0900, Kyotaro HORIGUCHI wrote: > At Sun, 10 Mar 2019 19:27:08 -0700, Noah Misch wrote in > <20190311022708.ga2189...@rfd.leadboat.com> > > On Mon, Mar 04, 2019 at 12:24:48PM +0900, Kyotaro HORIGUCHI wrote: > > > +/* > > > + * Sync to disk any relations that we

Re: Pluggable Storage - Andres's take

2019-03-20 Thread Haribabu Kommi
Hi, The psql \dA commands currently doesn't show the type of the access methods of type 'Table'. postgres=# \dA heap List of access methods Name | Type --+--- heap | (1 row) Attached a simple patch that fixes the problem and outputs as follows. postgres=# \dA heap List of access

Re: Pluggable Storage - Andres's take

2019-03-20 Thread Haribabu Kommi
On Sat, Mar 16, 2019 at 5:43 PM Haribabu Kommi wrote: > > > On Sat, Mar 9, 2019 at 2:13 PM Andres Freund wrote: > >> Hi, >> >> While 0001 is pretty bulky, the interesting bits concentrate on a >> comparatively small area. I'd appreciate if somebody could give the >> comments added in tableam.h

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Abhijit Menon-Sen
At 2019-03-20 23:22:44 +0100, tomas.von...@2ndquadrant.com wrote: > > I don't really understand what issue are we trying to solve here. > > Can someone describe a scenario where this (name of the binary not > clearly indicating it's related postgres) causes issues in practice? > On my system,

Re: MacPorts support for "extra" tests

2019-03-20 Thread Tom Lane
Thomas Munro writes: > Peter E added some nice tests for LDAP and Kerberos, but they assume > you have Homebrew when testing on a Mac. Here's a patch to make them > work with MacPorts too (a competing open source port/package > distribution that happens to be the one that I use). The third >

Re: Determine if FOR UPDATE or FOR SHARE was used?

2019-03-20 Thread Chapman Flack
On 03/18/19 00:45, Tom Lane wrote: > I think it would help to take two steps back and ask why you want > to know this, and what exactly is it that you want to know, anyhow. > What does it matter if there's FOR SHARE in the query? Does it I was looking at an old design decision in PL/Java, which

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Tom Lane
Tomas Vondra writes: > On 3/21/19 1:49 AM, Michael Paquier wrote: >> On Thu, Mar 21, 2019 at 08:41:32AM +0900, Tatsuo Ishii wrote: >>> Can someone describe a scenario where this (name of the binary not >>> clearly indicating it's related postgres) causes issues in practice? >> Naming conflict

Re: DNS SRV support for LDAP authentication

2019-03-20 Thread Thomas Munro
On Tue, Mar 19, 2019 at 9:01 PM Thomas Munro wrote: > I'd like to commit this soon. Done, after some more comment adjustments. Thanks Daniel and Graham for your feedback! -- Thomas Munro https://enterprisedb.com

Re: current_logfiles not following group access and instead follows log_file_mode permissions

2019-03-20 Thread Haribabu Kommi
On Thu, Mar 21, 2019 at 12:41 PM Haribabu Kommi wrote: > > On Wed, Mar 20, 2019 at 4:33 PM Michael Paquier > wrote: > >> And actually it seems to me that you have a race condition in that >> stuff. I think that you had better use umask(), then fopen, and then >> once again umask() to put back

Re: MSVC Build support with visual studio 2019

2019-03-20 Thread Haribabu Kommi
On Thu, Mar 21, 2019 at 12:31 PM Michael Paquier wrote: > On Thu, Mar 21, 2019 at 09:47:02AM +0900, Michael Paquier wrote: > > When it comes to support newer versions of MSVC, we have come up > > lately to backpatch that down to two stable versions but not further > > down (see f2ab389 for v10

Re: current_logfiles not following group access and instead follows log_file_mode permissions

2019-03-20 Thread Haribabu Kommi
On Wed, Mar 20, 2019 at 4:33 PM Michael Paquier wrote: > On Fri, Mar 15, 2019 at 06:51:37PM +1100, Haribabu Kommi wrote: > > IMO, this update is just a recommendation to the user, and sometimes it > is > > still possible that there may be strict permissions for the log file > > even the data

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Tomas Vondra
On 3/21/19 1:49 AM, Michael Paquier wrote: > On Thu, Mar 21, 2019 at 08:41:32AM +0900, Tatsuo Ishii wrote: >>> Can someone describe a scenario where this (name of the binary not >>> clearly indicating it's related postgres) causes issues in practice? On >>> my system, there are ~1400 binaries

Re: MSVC Build support with visual studio 2019

2019-03-20 Thread Michael Paquier
On Thu, Mar 21, 2019 at 09:47:02AM +0900, Michael Paquier wrote: > When it comes to support newer versions of MSVC, we have come up > lately to backpatch that down to two stable versions but not further > down (see f2ab389 for v10 and v9.6), so it looks sensible to target > v11 and v10 as well if

Re: Special role for subscriptions

2019-03-20 Thread Andrey Borodin
> 21 марта 2019 г., в 8:56, Michael Paquier написал(а): > > On Wed, Mar 20, 2019 at 11:58:04PM +0800, Andrey Borodin wrote: >>> 20 марта 2019 г., в 21:46, Robert Haas написал(а): >>> I think we should view this permission as "you can create >>> subscriptions, plain and simple". >> >> That

Re: performance issue in remove_from_unowned_list()

2019-03-20 Thread Tomas Vondra
On 3/12/19 11:54 PM, Tomas Vondra wrote: > > > On 3/10/19 9:09 PM, Alvaro Herrera wrote: >> On 2019-Feb-07, Tomas Vondra wrote: >> >>> Attached is a WIP patch removing the optimization from DropRelationFiles >>> and adding it to smgrDoPendingDeletes. This resolves the issue, at least >>> in the

RE: Best way to keep track of a sliced TOAST

2019-03-20 Thread Bruno Hass
I would like to optimize the jsonb key access operations. I could not find the discussion you've mentioned, but I am giving some thought to the idea. Instead of storing lengths, could we dedicate the first chunk of the TOASTed jsonb to store where each key is located? Would it be a good idea?

Re: Psql patch to show access methods info

2019-03-20 Thread Nikita Glukhov
Hi. On 08.03.2019 7:52, Kyotaro HORIGUCHI wrote: Hello. At Mon, 10 Dec 2018 19:38:39 +0300, s.cherkas...@postgrespro.ru wrote in <70e94e339dd0fa2be5d3eebec68da...@postgrespro.ru> Here are some fixes. But I'm not sure that the renaming of columns for the '\dAp' command is sufficiently

Re: Special role for subscriptions

2019-03-20 Thread Michael Paquier
On Wed, Mar 20, 2019 at 11:58:04PM +0800, Andrey Borodin wrote: >> 20 марта 2019 г., в 21:46, Robert Haas написал(а): >> I think we should view this permission as "you can create >> subscriptions, plain and simple". > > That sounds good. > From my POV, the purpose of the patch is to allow users

MacPorts support for "extra" tests

2019-03-20 Thread Thomas Munro
Hello hackers, Peter E added some nice tests for LDAP and Kerberos, but they assume you have Homebrew when testing on a Mac. Here's a patch to make them work with MacPorts too (a competing open source port/package distribution that happens to be the one that I use). The third "extra" test is

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Michael Paquier
On Thu, Mar 21, 2019 at 08:41:32AM +0900, Tatsuo Ishii wrote: >> Can someone describe a scenario where this (name of the binary not >> clearly indicating it's related postgres) causes issues in practice? On >> my system, there are ~1400 binaries in /usr/bin, and for the vast >> majority of them

Re: MSVC Build support with visual studio 2019

2019-03-20 Thread Michael Paquier
On Thu, Mar 21, 2019 at 11:36:42AM +1100, Haribabu Kommi wrote: > I can provide a separate back branches patch later once this patch comes to > a stage of commit. Currently all the supported branches are possible to > compile with VS 2017. When it comes to support newer versions of MSVC, we have

MSVC Build support with visual studio 2019

2019-03-20 Thread Haribabu Kommi
Hi Hackers, Here I attached a patch that supports building of PostgreSQL with VS 2019. VS 2019 is going to release on Apr 2nd 2019, it will be good if version 12 supports compiling. The attached for is for review, it may needs some updates once the final version is released. Commit

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Tatsuo Ishii
> I don't really understand what issue are we trying to solve here. > > Can someone describe a scenario where this (name of the binary not > clearly indicating it's related postgres) causes issues in practice? On > my system, there are ~1400 binaries in /usr/bin, and for the vast > majority of

Re: Offline enabling/disabling of data checksums

2019-03-20 Thread Michael Paquier
On Wed, Mar 20, 2019 at 05:46:32PM +0100, Fabien COELHO wrote: > I think that the motivation/risks should appear before the solution. "As xyz > ..., ...", or there at least the logical link should be outlined. > > It is not clear for me whether the following sentences, which seems specific > to

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Tatsuo Ishii
>> +1. As one of third party PostgreSQL tool developers, I am afraid >> changing names of PostgreSQL commands would give us lots of pain: for >> example checking PostgreSQL version to decide to use command "foo" not >> "pg_foo". >> > createdb, dropdb, createuser, dropuser, reindexdb are binaries

Re: Removing unneeded self joins

2019-03-20 Thread David Rowley
On Thu, 21 Mar 2019 at 01:20, Alexander Kuzmenkov wrote: > Let's recap the conditions when we can remove a self-join. It is when > for each outer row, 1) at most one inner row matches the join clauses, > and 2) it is the same row as the outer one. I'm not sure what (2) means > precisely in a

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread Julien Rouhaud
On Wed, Mar 20, 2019 at 11:10 PM legrand legrand wrote: > > Thank you Julien for the workaround, > It is not easy to build "cross tables" in excel to join metrics per query > text ... then keep only one queryid over all environments, that's easy enough in SQL: SELECT min(queryid) OVER

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Tomas Vondra
On 3/20/19 7:08 PM, Alvaro Herrera wrote: > On 2019-Mar-20, Euler Taveira wrote: > >> Em qua, 20 de mar de 2019 às 14:57, Tom Lane escreveu: >>> >>> We managed to get rid of createlang and droplang in v10, and there >>> hasn't been that much push-back about it. So maybe there could be >>> a

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread legrand legrand
Julien Rouhaud wrote > On Wed, Mar 20, 2019 at 10:30 PM legrand legrand > > legrand_legrand@ > wrote: >> >> maybe this patch (with a GUC) >> https://www.postgresql.org/message-id/ > 55E51C48.1060102@ >> would be enough for thoses actually using a text normalization function. > > The rest of

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread legrand legrand
Julien Rouhaud wrote > On Wed, Mar 20, 2019 at 10:18 PM legrand legrand > > legrand_legrand@ > wrote: >> >> On my personal point of view, I need to get the same Queryid between >> (OLAP) >> environments >> to be able to compare Production, Pre-production, Qualif performances >> (and I don't

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread Julien Rouhaud
On Wed, Mar 20, 2019 at 10:30 PM legrand legrand wrote: > > maybe this patch (with a GUC) > https://www.postgresql.org/message-id/55e51c48.1060...@uptime.jp > would be enough for thoses actually using a text normalization function. The rest of thread raise quite a lot of concerns about the

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread Julien Rouhaud
On Wed, Mar 20, 2019 at 10:18 PM legrand legrand wrote: > > On my personal point of view, I need to get the same Queryid between (OLAP) > environments > to be able to compare Production, Pre-production, Qualif performances > (and I don't need Fully qualified relation names). Today to do that, >

Re: Re: A separate table level option to control compression

2019-03-20 Thread Shaun Thomas
Jumping in here, please be gentle. :) Contents & Purpose == This appears to be a patch to add a new table storage option similar to `toast_tuple_target` but geared toward compression. As a result, it's been named `compress_tuple_target`, and allows modifying the threshold where

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread legrand legrand
maybe this patch (with a GUC) https://www.postgresql.org/message-id/55e51c48.1060...@uptime.jp would be enough for thoses actually using a text normalization function. -- Sent from: http://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread legrand legrand
Julien Rouhaud wrote > On Wed, Mar 20, 2019 at 8:39 PM legrand legrand > > legrand_legrand@ > wrote: >> >> Yes, I would like first to understand what are the main needs, > > I don't really see one implementation that suits every need, as > probably not everyone will agree on using relation

Re: propagating replica identity to partitions

2019-03-20 Thread Alvaro Herrera
Unless there are any objections to fixing the REPLICA IDENTITY bug, I intend to push that tomorrow. People can continue to discuss changing the behavior of other subcommands where reasonable (OWNER TO) or even for the cases others consider not reasonable (TABLESPACE), but there is no consensus of

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread Julien Rouhaud
On Wed, Mar 20, 2019 at 8:39 PM legrand legrand wrote: > > Yes, I would like first to understand what are the main needs, I don't really see one implementation that suits every need, as probably not everyone will agree on using relation name vs fully qualified relation name for starter. The

Re: reducing the footprint of ScanKeyword (was Re: Large writable variables)

2019-03-20 Thread Tom Lane
Joel Jacobson writes: > I've seen a performance trick in other hash functions [1] > to instead read multiple bytes in each iteration, > and then handle the remaining bytes after the loop. > [1] https://github.com/wangyi-fudan/wyhash/blob/master/wyhash.h#L29 I can't get very excited about this,

Re: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread legrand legrand
> From "Kyotaro HORIGUCHI-2" >>At Wed, 20 Mar 2019 00:23:30 +, "Tsunakawa, Takayuki" >>> From: legrand legrand [mailto:legrand_legrand@] >>> norm.9: comments aware >> Is this to distinguish queries that have different comments for optimizer >> hints? If yes, I agree. > Or, any means to give

Re: reducing the footprint of ScanKeyword (was Re: Large writable variables)

2019-03-20 Thread Joel Jacobson
Many thanks for working on this, amazing work, really nice you made it a separate reusable Perl-module. The generated hash functions reads one character at a time. I've seen a performance trick in other hash functions [1] to instead read multiple bytes in each iteration, and then handle the

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Alvaro Herrera
On 2019-Mar-20, Andres Freund wrote: > On 2019-03-20 15:15:02 -0400, Jonathan S. Katz wrote: > > If we are evaluating this whole symlink / renaming thing, there could be > > arguments for a "pgsql" alias to psql (or vice versa), but I don't think > > "pg_sql" makes any sense and could be fairly

RE: [survey] New "Stable" QueryId based on normalized query text

2019-03-20 Thread legrand legrand
> From: "Tsunakawa, Takayuki" >> From: legrand legrand [mailto:legrand_legrand@] >> There are many projects that use alternate QueryId >> distinct from the famous pg_stat_statements jumbling algorithm. >I'd like to welcome the standard QueryID that DBAs and extension developers can depend on.

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Jonathan S. Katz
On 3/20/19 3:19 PM, Andres Freund wrote: > Hi, > > On 2019-03-20 15:15:02 -0400, Jonathan S. Katz wrote: >> If we are evaluating this whole symlink / renaming thing, there could be >> arguments for a "pgsql" alias to psql (or vice versa), but I don't think >> "pg_sql" makes any sense and could be

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-03-20 Thread legrand legrand
Hi Jim, Robert, As this is a distinct subject from adding QueryId to pg_stat_activity, would it be possible to continue the discussion "new QueryId definition" (for postgres open source software) here: https://www.postgresql.org/message-id/1553029215728-0.p...@n3.nabble.com Thanks in advance.

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Andres Freund
Hi, On 2019-03-20 15:15:02 -0400, Jonathan S. Katz wrote: > If we are evaluating this whole symlink / renaming thing, there could be > arguments for a "pgsql" alias to psql (or vice versa), but I don't think > "pg_sql" makes any sense and could be fairly confusing. I don't care much about

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Jonathan S. Katz
On 3/20/19 2:11 PM, Tom Lane wrote: > "Fred .Flintstone" writes: >> Even just creating symlinks would be a welcome change. >> So the real binary is pg_foo and foo is a symoblic link that points to >> pg_foo. >> Then at least I can type pg_ and use tab auto-completion to find >> everything

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Jonathan S. Katz
On 3/20/19 2:08 PM, Alvaro Herrera wrote: > On 2019-Mar-20, Euler Taveira wrote: > >> Em qua, 20 de mar de 2019 às 14:57, Tom Lane escreveu: >>> >>> We managed to get rid of createlang and droplang in v10, and there >>> hasn't been that much push-back about it. So maybe there could be >>> a

Re: Feature: triggers on materialized views

2019-03-20 Thread David Steele
On 3/15/19 8:15 PM, Mitar wrote: The only pending/unaddressed comment is about the philosophical question of what it means to be a trigger. There it seems we simply disagree with the reviewer and I do not know how to address that. I just see this as a very pragmatical feature which provides

Re: PostgreSQL pollutes the file system

2019-03-20 Thread David Steele
On 3/20/19 9:32 PM, Alvaro Herrera wrote: On 2019-Mar-20, Fred .Flintstone wrote: Even just creating symlinks would be a welcome change. So the real binary is pg_foo and foo is a symoblic link that points to pg_foo. Then at least I can type pg_ and use tab auto-completion to find everything

Re: GiST VACUUM

2019-03-20 Thread Heikki Linnakangas
On 15/03/2019 20:25, Andrey Borodin wrote: 11 марта 2019 г., в 20:03, Heikki Linnakangas написал(а): On 10/03/2019 18:40, Andrey Borodin wrote: One thing still bothers me. Let's assume that we have internal page with 2 deletable leaves. We lock these leaves in order of items on internal page.

Re: GiST VACUUM

2019-03-20 Thread Heikki Linnakangas
On 15/03/2019 20:25, Andrey Borodin wrote: 11 марта 2019 г., в 20:03, Heikki Linnakangas написал(а): On 10/03/2019 18:40, Andrey Borodin wrote: One thing still bothers me. Let's assume that we have internal page with 2 deletable leaves. We lock these leaves in order of items on internal page.

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Tom Lane
"Fred .Flintstone" writes: > Even just creating symlinks would be a welcome change. > So the real binary is pg_foo and foo is a symoblic link that points to pg_foo. > Then at least I can type pg_ and use tab auto-completion to find > everything related to PostgreSQL. You'd miss psql. I think

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Jehan-Guillaume de Rorthais
On Wed, 20 Mar 2019 13:56:55 -0400 Tom Lane wrote: > Julien Rouhaud writes: > > On Wed, Mar 20, 2019 at 6:25 PM Euler Taveira > > wrote: > >> createdb, dropdb, createuser, dropuser, reindexdb are binaries that > >> confuse most newbies. Which tool is theses binaries from? The names > >> does

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Alvaro Herrera
On 2019-Mar-20, Euler Taveira wrote: > Em qua, 20 de mar de 2019 às 14:57, Tom Lane escreveu: > > > > We managed to get rid of createlang and droplang in v10, and there > > hasn't been that much push-back about it. So maybe there could be > > a move to remove createuser/dropuser? Or at least

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Euler Taveira
Em qua, 20 de mar de 2019 às 14:57, Tom Lane escreveu: > > We managed to get rid of createlang and droplang in v10, and there > hasn't been that much push-back about it. So maybe there could be > a move to remove createuser/dropuser? Or at least rename them to > pg_createuser and pg_dropuser.

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Tom Lane
Julien Rouhaud writes: > On Wed, Mar 20, 2019 at 6:25 PM Euler Taveira wrote: >> createdb, dropdb, createuser, dropuser, reindexdb are binaries that >> confuse most newbies. Which tool is theses binaries from? The names >> does not give a hint. How often those confusing name tools are used? >

Re: Sparse bit set data structure

2019-03-20 Thread Julien Rouhaud
On Wed, Mar 20, 2019 at 5:20 PM Julien Rouhaud wrote: > > On Wed, Mar 20, 2019 at 2:10 AM Heikki Linnakangas wrote: > > > I'm now pretty satisfied with this. Barring objections, I'll commit this > > in the next few days. Please review, if you have a chance. > > You're defining SIMPLE8B_MAX_VALUE

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Julien Rouhaud
On Wed, Mar 20, 2019 at 6:25 PM Euler Taveira wrote: > > createdb, dropdb, createuser, dropuser, reindexdb are binaries that > confuse most newbies. Which tool is theses binaries from? The names > does not give a hint. How often those confusing name tools are used? initdb is probably an order of

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Alvaro Herrera
On 2019-Mar-20, Fred .Flintstone wrote: > Even just creating symlinks would be a welcome change. > So the real binary is pg_foo and foo is a symoblic link that points to pg_foo. > Then at least I can type pg_ and use tab auto-completion to find > everything related to PostgreSQL. There is merit

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Euler Taveira
Em qua, 20 de mar de 2019 às 14:22, Fred .Flintstone escreveu: > > Even just creating symlinks would be a welcome change. > So the real binary is pg_foo and foo is a symoblic link that points to pg_foo. > Then at least I can type pg_ and use tab auto-completion to find > everything related to

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Euler Taveira
Em qua, 20 de mar de 2019 às 11:39, Tatsuo Ishii escreveu: > > +1. As one of third party PostgreSQL tool developers, I am afraid > changing names of PostgreSQL commands would give us lots of pain: for > example checking PostgreSQL version to decide to use command "foo" not > "pg_foo". > createdb,

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Fred .Flintstone
On Wed, Mar 20, 2019 at 3:19 PM Tom Lane wrote: > If we didn't pull the trigger twenty years ago, nor ten years ago, > we're not likely to do so now. Yeah, it's a mess and we'd certainly > do it differently if we were starting from scratch, but we're not > starting from scratch. There are

Re: Add exclusive backup deprecation notes to documentation

2019-03-20 Thread David Steele
Hi Robert, On 3/20/19 6:31 PM, Robert Haas wrote: On Wed, Mar 20, 2019 at 9:00 AM Michael Paquier wrote: On Wed, Mar 20, 2019 at 04:29:35PM +0400, David Steele wrote: Please note that there have been objections to the patch later in this thread by Peter and Robert. I'm not very interested

Re: Optimze usage of immutable functions as relation

2019-03-20 Thread Alexander Kuzmenkov
On 11/16/18 22:03, Tom Lane wrote: A possible fix for this is to do eval_const_expressions() on function RTE expressions at this stage (and then not need to do it later), and then pull up only when we find that the RTE expression has been reduced to a single Const. Attached is a patch that

Re: Offline enabling/disabling of data checksums

2019-03-20 Thread Fabien COELHO
Michaël-san, I think that a clear warning not to run any cluster command in parallel, under pain of possible cluster corruption, and possibly other caveats about replication, should appear in the documentation. I still have the following extra documentation in my notes: Ok, it should have

Re: Automated way to find actual COMMIT LSN of subxact LSN

2019-03-20 Thread Tom Lane
Jeremy Finzel writes: > A related problem kind of demonstrates the same odd behavior. If you put > in recovery_target_xid to a subtransaction_id, it just skips it and > continues recovering, which really seems to be undesirable behavior. It > would be nice if that also could roll up to the next

Re: Sparse bit set data structure

2019-03-20 Thread Julien Rouhaud
On Wed, Mar 20, 2019 at 2:10 AM Heikki Linnakangas wrote: > > On 14/03/2019 17:37, Julien Rouhaud wrote: > > > + if (newitem <= sbs->last_item) > > + elog(ERROR, "cannot insert to sparse bitset out of order"); > > > > Is there any reason to disallow inserting duplicates? AFAICT

Re: pg_basebackup ignores the existing data directory permissions

2019-03-20 Thread Peter Eisentraut
On 2019-03-19 08:34, Haribabu Kommi wrote: > How about the following change? > > pg_basebackup  --> copies the contents of the src directory (with group > access)  > and even the root directory permissions. > > pg_basebackup --no-group-access   --> copies the contents of the src > directory  >

Re: Special role for subscriptions

2019-03-20 Thread Andrey Borodin
> 20 марта 2019 г., в 21:46, Robert Haas написал(а): > > On Wed, Mar 20, 2019 at 5:39 AM Evgeniy Efimkin > wrote: >> Hi! >>> Currently, user with pg_subscription_users can create subscription into any >>> system table, can't they? >>> We certainly need to change it to more secure way. >>

Re: pg_basebackup ignores the existing data directory permissions

2019-03-20 Thread Peter Eisentraut
On 2019-03-18 16:45, Robert Haas wrote: >> I'm strongly in favor of keeping initdb and pg_basebackup options >> similar and consistent. They are both ways to initialize data directories. >> >> You'll note that initdb does not behave the way you describe. It's not >> unreasonable behavior, but

Re: Built-in connection pooler

2019-03-20 Thread Konstantin Knizhnik
New version of the patch (rebased + bug fixes) is attached to this mail. On 20.03.2019 18:32, Konstantin Knizhnik wrote: Attached please find results of benchmarking of different connection poolers. Hardware configuration:    Intel(R) Xeon(R) CPU   X5675  @ 3.07GHz    24 cores (12

Re: Automated way to find actual COMMIT LSN of subxact LSN

2019-03-20 Thread Jeremy Finzel
> > If recovery_target_inclusive were able to take the third value > "xact", is it exactly what you want? > > And is it acceptable? > Yes, that would be exactly what I would want. It would work to have a 3rd value for recovery_target_inclusive, although perhaps it's debatable that instead, it

Re: Add exclusive backup deprecation notes to documentation

2019-03-20 Thread Robert Haas
On Wed, Mar 20, 2019 at 9:00 AM Michael Paquier wrote: > On Wed, Mar 20, 2019 at 04:29:35PM +0400, David Steele wrote: > > Please note that there have been objections to the patch later in this > > thread by Peter and Robert. I'm not very interested in watering down the > > documentation changes

Re: speeding up planning with partitions

2019-03-20 Thread Jesper Pedersen
Hi, On 3/19/19 11:15 PM, Imai, Yoshikazu wrote: Here the details. [creating partitioned tables (with 1024 partitions)] drop table if exists rt; create table rt (a int, b int, c int) partition by range (a); \o /dev/null select 'create table rt' || x::text || ' partition of rt for values from ('

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Tom Lane
Chris Travers writes: > On Wed, Mar 20, 2019 at 11:06 AM Andreas Karlsson wrote: >> On 3/19/19 11:19 AM, Fred .Flintstone wrote: >>> It would be better if these files were renamed to be prefixed with >>> pg_, such as pg_createdb. >>> Or even better postgresql-createdb then be reachable by

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Chris Howard
Another pattern is to have a separate bin path for various software packages:  /opt/postgres/bin  for example. That doesn't directly answer "what is createdb?" but it does give a quicker indication via the 'which' command. On 3/20/19 5:43 AM, Fred .Flintstone wrote: It seems nothing came

Re: Special role for subscriptions

2019-03-20 Thread Robert Haas
On Wed, Mar 20, 2019 at 5:39 AM Evgeniy Efimkin wrote: > Hi! > > Currently, user with pg_subscription_users can create subscription into any > > system table, can't they? > > We certainly need to change it to more secure way. > No, you can't add system tables to publication. In new patch i add

Re: Add exclusive backup deprecation notes to documentation

2019-03-20 Thread Magnus Hagander
On Mon, Mar 18, 2019 at 1:33 PM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2019-03-07 10:33, David Steele wrote: > > On 3/1/19 3:14 PM, Laurenz Albe wrote: > I think it would be helpful to frame the documentation in a way to > suggest that the nonexclusive mode is more for

Re: [GSoC] application ideas

2019-03-20 Thread pantilimonov misha
Excuse me for the previous letter, should be fixed now by using simple html. --- Greetings, i am interested in databases and would like to make a contribution to the PostgreSQL by participating in GSoC 2019. Currently i am studying in HSE[1], doing last year of master's program that mostly

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Fred .Flintstone
It seems nothing came out of the discussion in 2008. I feel the topic should be revisited. I am in favor of doing so too. The deprecation cycle could involve symlinks for a brief period of time or a couple of versions. Yes, the wrapper script approach is used by Git as well as the "dotnet"

Re: Re: query logging of prepared statements

2019-03-20 Thread Justin Pryzby
Hi, On Wed, Mar 20, 2019 at 02:46:00PM +0400, David Steele wrote: > >I perfectly understand your use case. I agree, it is duplicated. But I > >think some people may want to see it at every EXECUTE, if they don't want > >to grep for the prepared statement body which was logged earlier. > > > >I

Re: Add exclusive backup deprecation notes to documentation

2019-03-20 Thread Michael Paquier
On Wed, Mar 20, 2019 at 04:29:35PM +0400, David Steele wrote: > Please note that there have been objections to the patch later in this > thread by Peter and Robert. I'm not very interested in watering down the > documentation changes as Peter suggests, but I think at the very least we > should

Re: Offline enabling/disabling of data checksums

2019-03-20 Thread Michael Paquier
On Wed, Mar 20, 2019 at 10:38:36AM +0100, Fabien COELHO wrote: > Hmmm… so nothing:-) The core of the feature is still here, fortunately. > I think that a clear warning not to run any cluster command in parallel, > under pain of possible cluster corruption, and possibly other caveats about >

Re: Add exclusive backup deprecation notes to documentation

2019-03-20 Thread David Steele
On 3/8/19 6:08 AM, Magnus Hagander wrote: On Thu, Mar 7, 2019 at 5:35 PM Michael Paquier > wrote: On Thu, Mar 07, 2019 at 11:33:20AM +0200, David Steele wrote: > OK, here's a new version that splits the deprecation notes from the > discussion of risks. 

Re: Removing unneeded self joins

2019-03-20 Thread Alexander Kuzmenkov
On 3/14/19 14:21, David Rowley wrote: What do you think? Let's recap the conditions when we can remove a self-join. It is when for each outer row, 1) at most one inner row matches the join clauses, and 2) it is the same row as the outer one. I'm not sure what (2) means precisely in a

Re: Should we add GUCs to allow partition pruning to be disabled?

2019-03-20 Thread David Rowley
On Thu, 14 Mar 2019 at 02:10, Robert Haas wrote: > > On Tue, Mar 12, 2019 at 7:28 PM David Rowley > wrote: > > I think I've done that in the attached patch. > > Cool, thanks. Just so I don't forget about this, I've added it to the July 'fest. https://commitfest.postgresql.org/23/2065/ --

Re: BUG #15572: Misleading message reported by "Drop function operation" on DB with functions having same name

2019-03-20 Thread David Rowley
Thanks for reviewing this. On Wed, 20 Mar 2019 at 04:31, Pavel Stehule wrote: > I propose maybe more strongly comment fact so noError is applied only on "not > found" event. In other cases, this flag is ignored and error is raised > immediately there. I think so it is not good enough commented

Re: Re: Psql patch to show access methods info

2019-03-20 Thread David Steele
Hi Sergey, On 3/8/19 8:52 AM, Kyotaro HORIGUCHI wrote: At Mon, 10 Dec 2018 19:38:39 +0300, s.cherkas...@postgrespro.ru wrote in <70e94e339dd0fa2be5d3eebec68da...@postgrespro.ru> Here are some fixes. But I'm not sure that the renaming of columns for the '\dAp' command is sufficiently laconic

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Chris Travers
On Wed, Mar 20, 2019 at 11:06 AM Andreas Karlsson wrote: > On 3/19/19 11:19 AM, Fred .Flintstone wrote: > > PostgreSQL pollutes the file system with lots of binaries that it is > > not obvious that they belong to PostgreSQL. > > > > Such as "/usr/bin/createdb", etc. > > > > It would be better if

Re: Re: A separate table level option to control compression

2019-03-20 Thread David Steele
Hi Pavan, On 3/12/19 4:38 PM, Andrew Dunstan wrote: On 3/11/19 2:23 AM, Masahiko Sawada wrote: I like this idea. The patch seems to need update the part describing on-disk toast storage in storage.sgml. Yeah. Meanwhile, here's a rebased version of the patch to keep the cfbot happy.

Re: Re: query logging of prepared statements

2019-03-20 Thread David Steele
Hi Justin, On 3/5/19 2:30 PM, Arthur Zakirov wrote: On 04.03.2019 21:31, Justin Pryzby wrote: It wasn't intentional.  Find attached v3 patch which handles that case, by removing the 2nd call to errdetail_execute() ; since it's otherwise unused, so remove that function entirely. Thank you.

Re: Re: Planning counters in pg_stat_statements (using pgss_store)

2019-03-20 Thread David Steele
Hi PAscal, On 2/15/19 11:32 AM, Sergei Kornilov wrote: Hi  +#define PG_STAT_STATEMENTS_COLS_V1_4 25 I thought it was needed when adding new columns, isn't it ? Yes, this is needed. I mean it should be PG_STAT_STATEMENTS_COLS_V1_8: because such change was made for 1.8 pg_stat_statements

Re: selecting from partitions and constraint exclusion

2019-03-20 Thread David Rowley
On Wed, 20 Mar 2019 at 17:37, Amit Langote wrote: > That's because get_relation_constraints() no longer (as of PG 11) includes > the partition constraint for SELECT queries. But that's based on an > assumption that partitions are always accessed via parent, so partition > pruning would make

Re: Re: [RFC] [PATCH] Flexible "partition pruning" hook

2019-03-20 Thread David Steele
Hi Peter, On 2/28/19 10:36 PM, Mike Palmiotto wrote: On Wed, Feb 27, 2019 at 12:36 PM Peter Eisentraut wrote: To rephrase this: You have a partitioned table, and you have a RLS policy that hides certain rows, and you know based on your business logic that under certain circumstances entire

Re: Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2019-03-20 Thread David Steele
Hi Pavan, On 3/14/19 2:20 PM, Masahiko Sawada wrote: On Thu, Mar 14, 2019 at 5:17 PM Pavan Deolasee wrote: Ok. I will run some tests. But please note that this patch is a bug fix to address the performance issue that is caused by having to rewrite the entire table when all-visible bit is

Re: Re: Reporting script runtimes in pg_regress

2019-03-20 Thread David Steele
Hi Christophe, On 3/8/19 5:12 PM, Alvaro Herrera wrote: On 2019-Mar-08, Christoph Berg wrote: Re: Peter Eisentraut 2019-03-08 <3eb194cf-b878-1f63-8623-6d6add0ed...@2ndquadrant.com> On 2019-02-21 10:37, Christoph Berg wrote: diff --git a/src/test/regress/pg_regress.c

Re: PostgreSQL pollutes the file system

2019-03-20 Thread Andreas Karlsson
On 3/19/19 11:19 AM, Fred .Flintstone wrote: PostgreSQL pollutes the file system with lots of binaries that it is not obvious that they belong to PostgreSQL. Such as "/usr/bin/createdb", etc. It would be better if these files were renamed to be prefixed with pg_, such as pg_createdb. Or even

RE: speeding up planning with partitions

2019-03-20 Thread Imai, Yoshikazu
Amit-san, On Wed, Mar 20, 2019 at 9:07 AM, Amit Langote wrote: > On 2019/03/20 17:36, Imai, Yoshikazu wrote: > > On Wed, Mar 20, 2019 at 8:21 AM, Amit Langote wrote: > >> On 2019/03/20 12:15, Imai, Yoshikazu wrote: > >>> [select1024.sql] > >>> \set a random (1, 1024) > >>> select * from rt where

Re: Special role for subscriptions

2019-03-20 Thread Evgeniy Efimkin
Hi! > Currently, user with pg_subscription_users can create subscription into any > system table, can't they? > We certainly need to change it to more secure way. No, you can't add system tables to publication. In new patch i add privileges checks on target table, non superuser can't

Re: Offline enabling/disabling of data checksums

2019-03-20 Thread Fabien COELHO
Michaël-san, In short, you keep the main feature with: - No tweaks with postmaster.pid. - Rely just on the control file indicating an instance shutdown cleanly. - No tweaks with the system ID. - No renaming of the control file. Hmmm… so nothing:-) I think that this feature is useful, in

  1   2   >