Re: [HACKERS] One process per session lack of sharing

2016-07-17 Thread Craig Ringer
On 16 July 2016 at 00:43, james wrote: > On 15/07/2016 09:28, Craig Ringer wrote: > >> I don't think anyone's considering moving from multi-processing to >> multi-threading in PostgreSQL. I really, really like the protection that >> the shared-nothing-by-default process model gives us, among othe

Re: [HACKERS] One process per session lack of sharing

2016-07-17 Thread Craig Ringer
On 15 July 2016 at 20:54, wrote: > Hi > > > > but parallel processing doesn't requires threading support - see > PostgreSQL 9.6 features. > > To share dynamic execution code between threads much more easy(If > sharing this code between process is possible). > There is many other interact

Re: [HACKERS] Version number for pg_control

2016-07-17 Thread Craig Ringer
On 15 July 2016 at 23:54, Tom Lane wrote: > While researching a pgsql-general question, I noticed that commit > 73c986adde5d73a5e2555da9b5c8facedb146dcd added several new fields > to pg_control without bothering to touch PG_CONTROL_VERSION. Thus, > PG_CONTROL_VERSION is still "942" even though t

[HACKERS] [PROPOSAL] timestamp informations to pg_stat_statements

2016-07-17 Thread Jun Cheol Gim
Hi hackers! Following is a proposal to add timestamp informations to `pg_stat_statements`. # Use case - If we want to gather list and stats for queries executed at least once last 1 hour, we had to reset a hours ago. There is no way if we didn't. - If we found some strange query from `pg_stat_sta

[HACKERS] One question about transformation ANY Sublinks into joins

2016-07-17 Thread Armor
Hi I run a simple SQL with latest PG?? postgres=# explain select * from t1 where id1 in (select id2 from t2 where c1=c2); QUERY PLAN Seq Scan on t1 (cost=0.00..43291.83 rows=1130 wi

Re: [HACKERS] sslmode=require fallback

2016-07-17 Thread Robert Haas
On Fri, Jul 15, 2016 at 4:14 AM, Magnus Hagander wrote: >> The original complaint was not actually that "prefer" is a bad default, >> but that in the presence of a root certificate on the client, a >> certificate validation failure falls back to plain text. That seems >> like a design flaw of the

Re: [HACKERS] Improving executor performance - tidbitmap

2016-07-17 Thread Robert Haas
On Wed, Jul 13, 2016 at 11:06 PM, Andres Freund wrote: > On 2016-06-24 16:29:53 -0700, Andres Freund wrote: >> 4) Various missing micro optimizations have to be performed, for more >>architectural issues to become visible. E.g. [2] causes such bad >>slowdowns in hash-agg workloads, that ot

Re: [HACKERS] rethinking dense_alloc (HashJoin) as a memory context

2016-07-17 Thread Robert Haas
On Wed, Jul 13, 2016 at 4:39 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Jul 13, 2016 at 1:10 PM, Tomas Vondra >> wrote: >> What's not clear to me is to what extent slowing down pfree is an >> acceptable price for improving the behavior in other ways. I wonder >> how many of the pfree

Re: [HACKERS] One process per session lack of sharing

2016-07-17 Thread Jan Wieck
On Sun, Jul 17, 2016 at 3:23 AM, Craig Ringer wrote: > > > Lots more could be shared, too. Cached plans, for example. > But the fact that PostgreSQL has transactional DDL complicates things like a shared plan cache and shared PL/pgSQL execution trees. Those things are much easier in a trivial da

Re: [HACKERS] One process per session lack of sharing

2016-07-17 Thread Simon Riggs
On 12 July 2016 at 09:57, wrote: > We have faced with some lack of sharing resources. > So in our test memory usage per session: > Oracle: about 5M > MSSqlServer: about 4M > postgreSql: about 160М > Using shared resources also has significant problems, so care must be taken. I think its clear

Re: [HACKERS] sslmode=require fallback

2016-07-17 Thread Peter Eisentraut
On 7/15/16 4:14 AM, Magnus Hagander wrote: > The entire "prefer" mode is a design flaw, that we unfortunately picked > as default mode. > > If it fails *for any reason*, it falls back to plaintext. Thus, you have > to assume it will make a plaintext connection. Thus, it gives you zero > guarantees

Re: [HACKERS] sslmode=require fallback

2016-07-17 Thread Peter Eisentraut
On 7/15/16 3:07 PM, Andrew Dunstan wrote: > Do those packagers who install dummy certificates and turn SSL on also > change their pg_hba.conf.sample files to use hostssl?. That could go a > long way towards encouraging people. Debian, which I guess sort of started this, does not, but there are a

Re: [HACKERS] application_name in process name?

2016-07-17 Thread Tom Lane
Jim Nasby writes: > On 7/13/16 12:07 PM, Tom Lane wrote: >> In a lot of situations ("top" for instance) only a limited number of >> characters can be displayed from a process title. I'm hesitant to add >> fields to that string that we don't really need. > Could we make this configurable, similar

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-17 Thread Jim Nasby
On 7/13/16 2:06 PM, Joshua D. Drake wrote: On 07/07/2016 01:01 PM, Robert Haas wrote: There was an unconference session on this topic at PGCon and quite a number of people there stated that they found DDL to be an ease-of-use feature and wanted to have it. Yeah, I haven't meet anyone yet that

Re: [HACKERS] One process per session lack of sharing

2016-07-17 Thread Robert Haas
On Fri, Jul 15, 2016 at 4:28 AM, Craig Ringer wrote: > I don't think anyone's considering moving from multi-processing to > multi-threading in PostgreSQL. I really, really like the protection that the > shared-nothing-by-default process model gives us, among other things. We get some very importa

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-17 Thread Jim Nasby
On 7/7/16 8:17 PM, Simon Riggs wrote: Simplicity is key, I agree. But that's just a user interface feature, not a comment on what's underneath the covers. pg_upgrade is not simple and is never likely to be so, under the covers. Right, and what I'd prefer effort put into is making managing repl

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-17 Thread Robert Haas
On Thu, Jul 14, 2016 at 2:29 AM, Craig Ringer wrote: > Yes, I'd like that too. I'd also like to have fully parallized writeable > queries right now. But we can't build everything all at once. I agree. > Before doing parallelized writes, things like dsm, dsm queues, group > locking, worker manage

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-17 Thread Jan Wieck
On Sun, Jul 17, 2016 at 2:08 PM, Jim Nasby wrote: > On 7/13/16 2:06 PM, Joshua D. Drake wrote: > >> On 07/07/2016 01:01 PM, Robert Haas wrote: >> >> There was an unconference session on this topic at PGCon and quite a >>> number of people there stated that they found DDL to be an ease-of-use >>>

Re: [HACKERS] One process per session lack of sharing

2016-07-17 Thread Jim Nasby
On 7/14/16 12:34 AM, Craig Ringer wrote: Starting with a narrow scope would help. Save/restore GUCs and the other easy stuff, and disallow sessions that are actively LISTENing, hold advisory locks, have open cursors, etc from being saved and restored. Along the lines of narrow scope... I wonder

Re: [HACKERS] One process per session lack of sharing

2016-07-17 Thread Tom Lane
Robert Haas writes: > On Fri, Jul 15, 2016 at 4:28 AM, Craig Ringer wrote: >> I don't think anyone's considering moving from multi-processing to >> multi-threading in PostgreSQL. I really, really like the protection that the >> shared-nothing-by-default process model gives us, among other things.

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-17 Thread Petr Jelinek
On 17/07/16 20:08, Jim Nasby wrote: On 7/13/16 2:06 PM, Joshua D. Drake wrote: On 07/07/2016 01:01 PM, Robert Haas wrote: There was an unconference session on this topic at PGCon and quite a number of people there stated that they found DDL to be an ease-of-use feature and wanted to have it.

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-17 Thread Petr Jelinek
On 17/07/16 20:50, Robert Haas wrote: It's the same with cluster-wide management, dump and restore of replication state to re-create a replication setup elsewhere, etc. We have to build the groundwork first. Trying to pour the top storey concrete when the bottom storey isn't even there yet isn'

Re: [HACKERS] sslmode=require fallback

2016-07-17 Thread Christoph Berg
Re: Peter Eisentraut 2016-07-17 > On 7/15/16 3:07 PM, Andrew Dunstan wrote: > > Do those packagers who install dummy certificates and turn SSL on also > > change their pg_hba.conf.sample files to use hostssl?. That could go a > > long way towards encouraging people. > > Debian, which I guess s

Re: [HACKERS] A Modest Upgrade Proposal

2016-07-17 Thread Joshua D. Drake
On 07/17/2016 11:55 AM, Jan Wieck wrote: Yeah, I haven't meet anyone yet that would like to have: select replicate_these_relations('['public']); vs: ALTER SCHEMA public ENABLE REPLICATION; (or something like that). I generally agree, but I think

Re: [HACKERS] Improving executor performance - tidbitmap

2016-07-17 Thread Andres Freund
On 2016-07-17 08:32:17 -0400, Robert Haas wrote: > On Wed, Jul 13, 2016 at 11:06 PM, Andres Freund wrote: > > The major issue with the simplehash implementation in the path is > > probably the deletion; which should rather move cells around, rather > > than use toombstones. But that was too comple

Re: [HACKERS] Obsolete comment within fmgr.c

2016-07-17 Thread Peter Geoghegan
On Thu, Apr 14, 2016 at 4:03 PM, Peter Geoghegan wrote: > Attached patch removes obsolete comment from fmgr.c. This patch seems to have been overlooked. It's a pretty straightforward case. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chang

Re: [HACKERS] [PROPOSAL] timestamp informations to pg_stat_statements

2016-07-17 Thread Peter Geoghegan
On Sun, Jul 17, 2016 at 12:22 AM, Jun Cheol Gim wrote: > If we have timestamp of first and last executed, we can easily gather thess > informations and there are tons of more use cases. -1 from me. I think that this is the job of a tool that aggregates things from pg_stat_statements. It's unfort

Re: [HACKERS] [PROPOSAL] timestamp informations to pg_stat_statements

2016-07-17 Thread Tom Lane
Peter Geoghegan writes: > On Sun, Jul 17, 2016 at 12:22 AM, Jun Cheol Gim wrote: >> If we have timestamp of first and last executed, we can easily gather thess >> informations and there are tons of more use cases. > -1 from me. > I think that this is the job of a tool that aggregates things fro

Re: [HACKERS] Obsolete comment within fmgr.c

2016-07-17 Thread Tom Lane
Peter Geoghegan writes: > On Thu, Apr 14, 2016 at 4:03 PM, Peter Geoghegan wrote: >> Attached patch removes obsolete comment from fmgr.c. > This patch seems to have been overlooked. It's a pretty straightforward case. Yeah --- pushed. regards, tom lane -- Sent via pg

Re: [HACKERS] [PROPOSAL] timestamp informations to pg_stat_statements

2016-07-17 Thread Peter Geoghegan
On Sun, Jul 17, 2016 at 4:15 PM, Tom Lane wrote: > The concern I've got about this proposal is that the results get very > questionable as soon as we start dropping statement entries for lack > of space. last_executed would be okay, perhaps, but first_executed > not so much. Agreed. Also, for w

Re: [HACKERS] DO with a large amount of statements get stuck with high memory consumption

2016-07-17 Thread Jan Wieck
BTW, here is the email thread about double-linking MemoryContext children patch, that Kevin at the end committed to master. https://www.postgresql.org/message-id/55F2D834.8040106%40wi3ck.info Regards, Jan On Sat, Jul 16, 2016 at 3:47 PM, Jan Wieck wrote: > > > On Tue, Jul 12, 2016 at 3:29 PM

Re: [HACKERS] [PROPOSAL] timestamp informations to pg_stat_statements

2016-07-17 Thread Julien Rouhaud
On 18/07/2016 01:06, Peter Geoghegan wrote: > On Sun, Jul 17, 2016 at 12:22 AM, Jun Cheol Gim wrote: >> If we have timestamp of first and last executed, we can easily gather thess >> informations and there are tons of more use cases. > > -1 from me. > > I think that this is the job of a tool tha

Re: [HACKERS] Regression tests vs existing users in an installation

2016-07-17 Thread Tom Lane
I've gone ahead and pushed a patch that does all of the cosmetic renamings needed to clean up the global-object-names situation. I've not done anything yet about those special cases in the rolenames test, since it's open for discussion exactly what to do there. I figured that this patch was bulky

Re: [HACKERS] One process per session lack of sharing

2016-07-17 Thread Robert Haas
On Sun, Jul 17, 2016 at 3:12 PM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Jul 15, 2016 at 4:28 AM, Craig Ringer wrote: >>> I don't think anyone's considering moving from multi-processing to >>> multi-threading in PostgreSQL. I really, really like the protection that the >>> shared-nothin

Re: [HACKERS] One process per session lack of sharing

2016-07-17 Thread Robert Haas
On Sun, Jul 17, 2016 at 3:04 PM, Jim Nasby wrote: > On 7/14/16 12:34 AM, Craig Ringer wrote: >> Starting with a narrow scope would help. Save/restore GUCs and the other >> easy stuff, and disallow sessions that are actively LISTENing, hold >> advisory locks, have open cursors, etc from being saved

Re: [HACKERS] Regression tests vs existing users in an installation

2016-07-17 Thread Robert Haas
On Sat, Jul 16, 2016 at 11:38 AM, Tom Lane wrote: > I'm coming to the conclusion that the only thing that will make this > materially better in the long run is automatic enforcement of a convention > about what role names may be created in the regression tests. See my > response to Stephen just n

Re: [HACKERS] One process per session lack of sharing

2016-07-17 Thread Jan Wieck
On Sun, Jul 17, 2016 at 9:28 PM, Robert Haas wrote: > On Sun, Jul 17, 2016 at 3:12 PM, Tom Lane wrote: > > Robert Haas writes: > >> On Fri, Jul 15, 2016 at 4:28 AM, Craig Ringer > wrote: > >>> I don't think anyone's considering moving from multi-processing to > >>> multi-threading in PostgreSQ

Re: [HACKERS] Reviewing freeze map code

2016-07-17 Thread Andres Freund
On 2016-07-16 10:45:26 -0700, Andres Freund wrote: > > > On July 16, 2016 8:49:06 AM PDT, Tom Lane wrote: > >Amit Kapila writes: > >> On Sat, Jul 16, 2016 at 7:02 AM, Andres Freund > >wrote: > >>> I think we have two choices how to deal with that: First, we can add > >a > >>> new flags variabl

Re: [HACKERS] Reviewing freeze map code

2016-07-17 Thread Robert Haas
On Sun, Jul 17, 2016 at 10:48 PM, Andres Freund wrote: > Took till today. Attached is a rather heavily revised version of > Sawada-san's patch. Most notably the recovery routines take care to > reset the vm in all cases, we don't perform visibilitymap_get_status > from inside a critical section an

Re: [HACKERS] Reviewing freeze map code

2016-07-17 Thread Amit Kapila
On Mon, Jul 18, 2016 at 8:18 AM, Andres Freund wrote: > On 2016-07-16 10:45:26 -0700, Andres Freund wrote: >> >> >> On July 16, 2016 8:49:06 AM PDT, Tom Lane wrote: >> >Amit Kapila writes: >> >> On Sat, Jul 16, 2016 at 7:02 AM, Andres Freund >> >wrote: >> >>> I think we have two choices how to

Re: [HACKERS] Reviewing freeze map code

2016-07-17 Thread Andres Freund
On 2016-07-17 23:34:01 -0400, Robert Haas wrote: > Thanks very much for working on this. Random suggestions after a quick look: > > + * Before locking the buffer, pin the visibility map page if it may be > + * necessary. > > s/necessary/needed/ > > More substantively, what happens if th

Re: [HACKERS] Reviewing freeze map code

2016-07-17 Thread Andres Freund
On 2016-07-18 09:07:19 +0530, Amit Kapila wrote: > + /* > + * Before locking the buffer, pin the visibility map page if it may be > + * necessary. > + */ > > + if (PageIsAllVisible(BufferGetPage(*buffer))) > + visibilitymap_pin(relation, block, &vmbuffer); > + > LockBuffer(*buffer, BUFFER_LOCK_E

Re: [HACKERS] Reviewing freeze map code

2016-07-17 Thread Amit Kapila
On Mon, Jul 18, 2016 at 9:13 AM, Andres Freund wrote: > On 2016-07-18 09:07:19 +0530, Amit Kapila wrote: >> + /* >> + * Before locking the buffer, pin the visibility map page if it may be >> + * necessary. >> + */ >> >> + if (PageIsAllVisible(BufferGetPage(*buffer))) >> + visibilitymap_pin(relatio

Re: [HACKERS] Regression tests vs existing users in an installation

2016-07-17 Thread Michael Paquier
On Mon, Jul 18, 2016 at 10:37 AM, Robert Haas wrote: > On Sat, Jul 16, 2016 at 11:38 AM, Tom Lane wrote: > We could also do this by loading a C module during the regression > tests, which seems maybe less ugly than adding a GUC. > I don't particularly like your suggestion of spooky action at a >

Re: [HACKERS] RecoveryTargetTLI dead variable in XLogCtlData

2016-07-17 Thread Michael Paquier
On Wed, Jul 13, 2016 at 12:29 PM, Michael Paquier wrote: > I just bumped into $subject, a variable that is never set and never used: > --- a/src/backend/access/transam/xlog.c > +++ b/src/backend/access/transam/xlog.c > @@ -631,8 +631,6 @@ typedef struct XLogCtlData > TimeLineID replayEndTLI;