Re: [HACKERS] Parallel bitmap heap scan

2017-02-19 Thread Dilip Kumar
On Sun, Feb 19, 2017 at 10:34 PM, Robert Haas wrote: > Yeah, but it looks like ExecReScanGather gets rid of the workers, but > reuses the existing DSM. I'm not quite sure what happens to the DSA. > It looks like it probably just hangs around from the previous > iteration, which means that any all

[HACKERS] 答复: [HACKERS] Adding new output parameter of pg_stat_statements toidentify operation of the query.(Internet mail)

2017-02-19 Thread 李跃森
Yes, it seems the pg_stat_sql function can fit the individual need of collecting tags of query. However the new function can not return other values of query at the same time, such as block number info, run time and so on. Returning these values at the same time are very important. So I thin

Re: [HACKERS] Replication vs. float timestamps is a disaster

2017-02-19 Thread Andres Freund
On 2017-02-19 10:49:29 -0500, Tom Lane wrote: > Robert Haas writes: > > On Sun, Feb 19, 2017 at 3:31 AM, Tom Lane wrote: > >> Thoughts? Should we double down on trying to make this work according > >> to the "all integer timestamps" protocol specs, or cut our losses and > >> change the specs? >

Re: [HACKERS] Gather Merge

2017-02-19 Thread Rushabh Lathia
Thanks Amit for raising this point. I was not at all aware of mark/restore. I tried to come up with the case, but haven't found such case. For now here is the patch with comment update. Thanks, On Sun, Feb 19, 2017 at 7:30 PM, Amit Kapila wrote: > On Sun, Feb 19, 2017 at 2:22 PM, Robert Haas

Re: [HACKERS] SCRAM authentication, take three

2017-02-19 Thread Michael Paquier
On Sun, Feb 19, 2017 at 10:07 PM, Michael Paquier wrote: > There is something that I think is still unwelcome in this patch: the > interface in pg_hba.conf. I mentioned that in the previous thread but > now if you want to match a user and a database with a scram password > you need to do that with

Re: [HACKERS] [Bug fix] PQsendQuery occurs error when target_session_attrs is set to read-write

2017-02-19 Thread Michael Paquier
On Thu, Feb 16, 2017 at 10:55 AM, Michael Paquier wrote: > It is possible to get a test easily in this area by abusing of the > fact that multiple -d switches defined in psql make it use only the > last value. By looking at psql() in PostgresNode.pm you would see what > I mean as -d is defined by

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-19 Thread Masahiko Sawada
On Mon, Feb 20, 2017 at 11:35 AM, Jim Nasby wrote: > On 2/19/17 7:56 PM, Masahiko Sawada wrote: >> >> The half-dead pages are never cleaned up if the ratio of pages >> containing garbage is always lower than threshold. Also in gin index >> the pending list is never cleared, which become big proble

Re: [HACKERS] Parallel Append implementation

2017-02-19 Thread Ashutosh Bapat
On Sun, Feb 19, 2017 at 2:33 PM, Robert Haas wrote: > On Fri, Feb 17, 2017 at 11:44 AM, Ashutosh Bapat > wrote: >> That's true for a partitioned table, but not necessarily for every >> append relation. Amit's patch is generic for all append relations. If >> the child plans are joins or subquery s

[HACKERS] fd,c just Assert()s that lseek() succeeds

2017-02-19 Thread Tom Lane
I noticed while researching bug #14555 that fd.c contains two separate cases like vfdP->seekPos = lseek(vfdP->fd, (off_t) 0, SEEK_CUR); Assert(vfdP->seekPos != (off_t) -1); This seems, um, unwise. It might somehow fail to fail in production builds, because elsewhere it's assumed

Re: [HACKERS] Passing query string to workers

2017-02-19 Thread Rafia Sabih
On Sun, Feb 19, 2017 at 10:11 PM, Robert Haas wrote: > On Thu, Feb 16, 2017 at 6:41 PM, Kuntal Ghosh > wrote: > > On Thu, Feb 16, 2017 at 5:47 PM, Rafia Sabih > > wrote: > >> Other that that I updated some comments and other cleanup things. Please > >> find the attached patch for the revised ve

Re: [HACKERS] drop support for Python 2.3

2017-02-19 Thread Devrim Gündüz
Hi, On Sun, 2017-02-19 at 16:20 -0500, Tom Lane wrote: > Well, that test is checking which week-of-the-year a Sunday midnight is > considered to fall into.  There could be an edge-case bug in Tcl itself, > or a problem with the time zone data, or maybe if you're setting LC_TIME > to tr_TR, that c

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

2017-02-19 Thread Amit Langote
Hi Stephen, On 2017/02/17 22:32, Stephen Frost wrote: > Amit, > > * Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: >> In certain cases, pg_dump's dumpTableSchema() emits a separate ALTER TABLE >> command for those schema elements of a table that could not be included >> directly in the CREAT

Re: [HACKERS] Adding new output parameter of pg_stat_statements to identify operation of the query.

2017-02-19 Thread Tom Lane
Jim Nasby writes: > Something that needs to be considered with doing this in > pg_stat_statement is that a query that's reported there can contain > multiple SQL statements. I don't remember offhand if all statements get > parsed as a whole before anything else happens; if that's the case th

Re: [HACKERS] tab completion for partitioning

2017-02-19 Thread Amit Langote
On 2017/02/20 1:22, Robert Haas wrote: > On Thu, Feb 16, 2017 at 7:15 AM, Amit Langote > wrote: >> Also attaching 0002 (unchanged) for tab-completion support for the new >> partitioning syntax. > > At one point you have this: > > +/* Limited completion support for partition bound specificati

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-19 Thread Jim Nasby
On 2/19/17 7:56 PM, Masahiko Sawada wrote: The half-dead pages are never cleaned up if the ratio of pages containing garbage is always lower than threshold. Also in gin index the pending list is never cleared, which become big problem. I guess that we should take action for each type of indexes.

Re: [HACKERS] Partitioned tables and relfilenode

2017-02-19 Thread Amit Langote
On 2017/02/20 5:31, Simon Riggs wrote: > On 16 February 2017 at 11:32, Simon Riggs wrote: >> On 10 February 2017 at 06:19, Amit Langote >> wrote: >> >>> the "right thing" here being that the >>> command's code either throws an error or warning (in some cases) if the >>> specified table is a part

Re: [HACKERS] Adding new output parameter of pg_stat_statements to identify operation of the query.

2017-02-19 Thread Jim Nasby
On 2/19/17 6:34 PM, Tsunakawa, Takayuki wrote: We have done the job and are willing to post a patch. I sent one through my work mail, but it seems that my mail didn't reach the maillist, so I try again by using my personal mail account. A view for counting the number of executions per operation

Re: [HACKERS] Documentation improvements for partitioning

2017-02-19 Thread Amit Langote
On 2017/02/20 1:04, Robert Haas wrote: > On Thu, Feb 16, 2017 at 12:43 PM, Amit Langote wrote: >> So I count more than a few votes saying that we should be able to DROP >> partitioned tables without specifying CASCADE. >> >> I tried to implement that using the attached patch by having >> StoreCatal

Re: [HACKERS] Instability in select_parallel regression test

2017-02-19 Thread Amit Kapila
On Sun, Feb 19, 2017 at 8:32 PM, Robert Haas wrote: > On Sun, Feb 19, 2017 at 6:50 PM, Amit Kapila wrote: >> To close the remaining gap, don't you think we can check slot->in_use >> flag when generation number for handle and slot are same. > > That doesn't completely fix it either, because > Forg

[HACKERS] dropping partitioned tables without CASCADE

2017-02-19 Thread Amit Langote
Re-posting the patch I posted in a nearby thread [0]. On 2017/02/16 2:08, Robert Haas wrote: > On Wed, Feb 15, 2017 at 11:34 AM, Alvaro Herrera > wrote: >> I think new-style partitioning is supposed to consider each partition as >> an implementation detail of the table; the fact that you can mani

Re: [HACKERS] [PATCH] Add pg_disable_checksums() and supporting infrastructure

2017-02-19 Thread Jim Nasby
On 2/19/17 11:02 AM, David Christensen wrote: My design notes for the patch were submitted to the list with little comment; see: https://www.postgresql.org/message-id/1E6E64E9-634B-43F4-8AA2-CD85AD92D2F8%40endpoint.com I have since added the WAL logging of checksum states, however I’d be glad

Re: [HACKERS] Adding new output parameter of pg_stat_statements to identify operation of the query.

2017-02-19 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of > husttrip...@vip.sina.com > When using pg_stat_statements to collect running SQL of PG, we > find it is hard for our program to get exact operation type of the SQL, > such as SELECT, DELETE

Re: [HACKERS] case_preservation_and_insensitivity = on

2017-02-19 Thread Tom Lane
Joel Jacobson writes: > I think a good general philosophy for the PostgreSQL project would be to > try to look at how to meed the needs for new users of new projects > in a way that don't impair things for existing users, Yeah, exactly, and the problem here is that claiming that something like th

Re: [HACKERS] Parallel bitmap heap scan

2017-02-19 Thread Tom Lane
Thomas Munro writes: > One practical problem that came up was the need for executor nodes to > get a chance to do that kind of cleanup before the DSM segment is > detached. In my patch series I introduced a new node API > ExecNodeDetach to allow for that. Andres objected that the need for > that

Re: [HACKERS] case_preservation_and_insensitivity = on

2017-02-19 Thread Jim Nasby
On 2/19/17 4:51 PM, Joel Jacobson wrote: But once you've already decided to have a hard-and-fast rule that the names must be unique after lower-casing, there's no obvious benefit to rejecting queries that mention the same name with different case. Exactly, that trade-off is necessary, otherwise

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-19 Thread Masahiko Sawada
On Fri, Feb 17, 2017 at 3:41 AM, Robert Haas wrote: > On Thu, Feb 16, 2017 at 6:17 AM, Simon Riggs wrote: >> On 15 February 2017 at 08:07, Masahiko Sawada wrote: >>> It's a bug. Attached latest version patch, which passed make check. >> >> In its current form, I'm not sure this is a good idea. P

Re: ParallelFinish-hook of FDW/CSP (Re: [HACKERS] Steps inside ExecEndGather)

2017-02-19 Thread Kouhei Kaigai
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Robert Haas > Sent: Monday, February 20, 2017 2:20 AM > To: Kaigai Kouhei(海外 浩平) > Cc: Claudio Freire ; Amit Kapila > ; pgsql-hackers > Subject: Re: ParallelFinish-ho

Re: [HACKERS] Parallel bitmap heap scan

2017-02-19 Thread Thomas Munro
On Sun, Feb 19, 2017 at 10:34 PM, Robert Haas wrote: > On Sun, Feb 19, 2017 at 9:59 PM, Dilip Kumar wrote: >> I can imagine it can get executed over and over if plan is something like >> below. >> >> NestLoopJoin >> -> SeqScan >> -> Gather >> -> Parallel Bitmap Heap Scan >> >> Bu

Re: [HACKERS] pg_get_object_address() doesn't support composites

2017-02-19 Thread Jim Nasby
On 2/18/17 4:26 PM, Jim Nasby wrote: On 2/17/17 9:53 PM, Alvaro Herrera wrote: Jim Nasby wrote: See below. ISTM that pg_get_object_address should support everything pg_identify_object_as_address can output, no? I'm guessing the answer here is to have pg_identify_object_as_address complain if y

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-02-19 Thread Gilles Darold
Le 16/02/2017 à 16:13, Robert Haas a écrit : > On Wed, Feb 15, 2017 at 4:03 PM, Alvaro Herrera > wrote: >> So what is going on here is that SysLogger_Start() wants to unlink the >> current-logfile file if the collector is not enabled. This should >> probably be split out into a separate new funct

Re: [HACKERS] case_preservation_and_insensitivity = on

2017-02-19 Thread Joel Jacobson
On Sun, Feb 19, 2017 at 5:58 PM, Robert Haas wrote: >> When case preservation by default is on, then simply enforce >> UNIQUE(LOWER(object_name)), to prevent ambiguity. > > That (1) breaks backward compatibility, because people might have > objects with names identical except for case in existing

[HACKERS] Resolved typo in a comment

2017-02-19 Thread neha khatri
Hi, Attached is a patch that fixes a comment typo in varlena.c Thanks, Neha typo_correction.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Reporting xmin from VACUUMs

2017-02-19 Thread Jim Nasby
On 2/19/17 3:43 AM, Robert Haas wrote: This is the kind of information that you really want to see once per autovac, though, not just the most recent autovac or some kind of cumulative total. Knowing that I've done 301 index scans in my last 300 vacuums is not nearly as useful as knowing which a

Re: [HACKERS] Logical replication existing data copy - comments snapbuild.c

2017-02-19 Thread Erik Rijkers
On 2017-02-19 23:24, Erik Rijkers wrote: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch 0004-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION-v2.pat

Re: [HACKERS] Logical replication existing data copy - comments snapbuild.c

2017-02-19 Thread Erik Rijkers
0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch 0004-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION-v2.patch 0001-Logical-replication-support-for-in

Re: [HACKERS] Provide list of subscriptions and publications in psql's completion

2017-02-19 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Sun, Feb 19, 2017 at 6:13 PM, Michael Paquier > wrote: > > On Sun, Feb 19, 2017 at 8:03 PM, Magnus Hagander > > wrote: > >> If password auth is used, we have to store the password in plaintext > >> equivalent somewhere. Meaning it's by definition

[HACKERS] Deprecate floating point timestamps.

2017-02-19 Thread Jim Nasby
Over in the "Keeping pg_recvlogical's "feTimestamp" separate from TimestampTz"... On 2/17/17 12:15 PM, Tom Lane wrote: > I am not sure that it was really a good design to pretend that the > replication protocol is independent of --disable-integer-datetimes > when the underlying WAL stream most c

[HACKERS] Adding new output parameter of pg_stat_statements to identify operation of the query.

2017-02-19 Thread husttripper
Hi PG hackers: When using pg_stat_statements to collect running SQL of PG, we find it is hard for our program to get exact operation type of the SQL, such as SELECT, DELETE, UPDATE, INSERT, and so on. So we modify the the source code of pg_stat_statements and add another output param

Re: [HACKERS] Allow pg_dumpall to work without pg_authid

2017-02-19 Thread Stephen Frost
Robins, * Robins Tharakan (thara...@gmail.com) wrote: > On 19 February 2017 at 17:02, Robins Tharakan wrote: > > On Sun, 19 Feb 2017 at 10:08 Stephen Frost wrote: > >> If anything, it should use pg_roles, not pg_user. > >> > >> I don't really like the "--avoid-pgauthid" option, but "--no-passwor

Re: [HACKERS] drop support for Python 2.3

2017-02-19 Thread Tom Lane
Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= writes: > On Sun, 2017-02-19 at 13:48 -0500, Tom Lane wrote: >> Or conceivably it's timezone dependent? > FWIW, the timezone of the server is GMT+3, if that is what you are asking. Well, that test is checking which week-of-the-year a Sunday midnight is consider

Re: [HACKERS] Documentation improvements for partitioning

2017-02-19 Thread Simon Riggs
On 15 February 2017 at 15:46, Robert Haas wrote: >> It leaves me asking what else is missing. > > There is certainly a lot of room for improvement here but I don't > understand your persistent negativity about what's been done thus far. > I think it's pretty clearly a huge step forward, and I thi

Re: [HACKERS] Partitioned tables and relfilenode

2017-02-19 Thread Simon Riggs
On 16 February 2017 at 11:32, Simon Riggs wrote: > On 10 February 2017 at 06:19, Amit Langote > wrote: > >> the "right thing" here being that the >> command's code either throws an error or warning (in some cases) if the >> specified table is a partitioned table or ignores any partitioned tables

Re: [HACKERS] drop support for Python 2.3

2017-02-19 Thread Devrim Gündüz
Hi Tom, On Sun, 2017-02-19 at 13:48 -0500, Tom Lane wrote: > Hmph.  I can't see any relevant-looking source changes between 8.4.13 > and 8.4.15, which I have laying about here and which works fine. > I wonder if Red Hat is carrying some distro-specific patch that > breaks this case?  Just downlo

Re: [HACKERS] drop support for Python 2.3

2017-02-19 Thread Tom Lane
Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= writes: > On Sun, 2017-02-19 at 10:42 -0500, Tom Lane wrote: >> Relevant question: what version of tcl is installed on those? > 8.4.13 is installed. Hmph. I can't see any relevant-looking source changes between 8.4.13 and 8.4.15, which I have laying about here

Re: [HACKERS] drop support for Python 2.3

2017-02-19 Thread Devrim Gündüz
Hi Tom, On Sun, 2017-02-19 at 10:42 -0500, Tom Lane wrote: > Relevant question: what version of tcl is installed on those? 8.4.13 is installed. Regards, -- Devrim Gündüz EnterpriseDB: http://www.enterprisedb.com PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer Twitter: @DevrimGunduz

Re: [HACKERS] COPY IN/BOTH vs. extended query mode

2017-02-19 Thread Corey Huinker
On Sun, Feb 19, 2017 at 9:04 AM, Robert Haas wrote: If you tried to write an SQL-callable function that internally started > and ended a copy from the client, then I think you would run into this > problem, and probably some others. > > That's it. I had a PoC patch submitted that allowed someone

Re: [pgsql-www] [HACKERS] Small issue in online devel documentation build

2017-02-19 Thread Fabien COELHO
-f filename The filename in the newer html appears much larger under chrome, seemingly because of the within a . Maybe a bug in chrome CSS interpretation, because CSS on code seems to indicate "font-size: 1.3em", but it seems to do 1.3**2 instead for "filename"... However it

Re: ParallelFinish-hook of FDW/CSP (Re: [HACKERS] Steps inside ExecEndGather)

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 12:46 PM, Kouhei Kaigai wrote: > The attached patch is revised one. > > Invocation of Exec(Foreign|Custom)ParallelFinish was moved to > ExecParallelRetrieveInstrumentation() not to walk on the plan- > state tree twice. > One (hypothetical) downside is, FDW/CSP can retrieve

Re: [HACKERS] Replication vs. float timestamps is a disaster

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 9:19 PM, Tom Lane wrote: > Robert Haas writes: >> On Sun, Feb 19, 2017 at 3:31 AM, Tom Lane wrote: >>> Thoughts? Should we double down on trying to make this work according >>> to the "all integer timestamps" protocol specs, or cut our losses and >>> change the specs? >

Re: [HACKERS] Parallel bitmap heap scan

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 9:59 PM, Dilip Kumar wrote: > I can imagine it can get executed over and over if plan is something like > below. > > NestLoopJoin > -> SeqScan > -> Gather > -> Parallel Bitmap Heap Scan > > But in such case every time the Inner node of the NLJ will be > res

Re: [HACKERS] [PATCH] Add pg_disable_checksums() and supporting infrastructure

2017-02-19 Thread David Christensen
> On Feb 19, 2017, at 5:05 AM, Robert Haas wrote: > > On Fri, Feb 17, 2017 at 2:28 AM, David Christensen wrote: >> - Change "data_checksums" from a simple boolean to "data_checksum_state", an >> enum type for all of >> the potentially-required states for this feature (as well as enabling). >

Re: [HACKERS] case_preservation_and_insensitivity = on

2017-02-19 Thread Robert Haas
On Thu, Feb 16, 2017 at 11:16 PM, Joel Jacobson wrote: >> The short answer is that nobody can see a way to modify the identifier >> case-folding rules that isn't going to add more pain than it subtracts. >> And much of the added pain will be felt by people who aren't getting >> any benefit, who wi

Re: [HACKERS] New CORRESPONDING clause design

2017-02-19 Thread Robert Haas
On Thu, Feb 16, 2017 at 8:28 PM, Surafel Temsgen wrote: > Here is the implementation of the clause with the slight change, instead of > doing column mapping for each side of leaf Queries in planner I make the > projection nodes output to corresponding column lists only. > > This patch compiles and

Re: [HACKERS] Passing query string to workers

2017-02-19 Thread Robert Haas
On Thu, Feb 16, 2017 at 6:41 PM, Kuntal Ghosh wrote: > On Thu, Feb 16, 2017 at 5:47 PM, Rafia Sabih > wrote: >> Other that that I updated some comments and other cleanup things. Please >> find the attached patch for the revised version. > Looks good. > > It has passed the regression tests (with a

Re: [HACKERS] Parallel bitmap heap scan

2017-02-19 Thread Dilip Kumar
On Sun, Feb 19, 2017 at 7:44 PM, Robert Haas wrote: > It's probably OK if tbm_free() doesn't free the memory allocated from > DSA, and we just let cleanup at end of query do it. However, that > could cause some trouble if the Parallel Bitmap Heap Scan gets > executed over and over and keeps alloc

Re: [HACKERS] Does having a NULL column automatically exclude the table from the tupleDesc cache?

2017-02-19 Thread Ryan Murphy
> > You'll probably want to do those at a C level, bypassing the executor. I > > would guess that executor overhead will completely swamp the effect of > the > > cache in most cases. > > That seems like it's kind of missing the point. If the tupleDesc > cache saves so little that it's irrelevant w

Re: [HACKERS] Logical replication existing data copy - comments origin.c

2017-02-19 Thread Erik Rijkers
On 2017-02-19 17:21, Erik Rijkers wrote: 0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch 0004-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION-v2.pat

[HACKERS] tab completion for partitioning

2017-02-19 Thread Robert Haas
On Thu, Feb 16, 2017 at 7:15 AM, Amit Langote wrote: > Also attaching 0002 (unchanged) for tab-completion support for the new > partitioning syntax. At one point you have this: +/* Limited completion support for partition bound specification */ +else if (TailMatches3("ATTACH", "PARTITION

Re: [HACKERS] Logical replication existing data copy - comments origin.c

2017-02-19 Thread Erik Rijkers
0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch 0003-Fix-after-trigger-execution-in-logical-replication-v2.patch 0004-Add-RENAME-support-for-PUBLICATIONs-and-SUBSCRIPTION-v2.patch 0001-Logical-replication-support-for-in

Re: [HACKERS] Documentation improvements for partitioning

2017-02-19 Thread Robert Haas
On Thu, Feb 16, 2017 at 12:43 PM, Amit Langote wrote: > On 2017/02/16 2:08, Robert Haas wrote: >> On Wed, Feb 15, 2017 at 11:34 AM, Alvaro Herrera >> wrote: >>> I think new-style partitioning is supposed to consider each partition as >>> an implementation detail of the table; the fact that you ca

Re: [HACKERS] Documentation improvements for partitioning

2017-02-19 Thread Robert Haas
On Thu, Feb 16, 2017 at 7:15 AM, Amit Langote wrote: >> I think 0001 is better than the status quo, but I'm wondering whether >> we should try to do something slightly different. Maybe it should >> always work for the child table to specify neither WITH OIDS nor >> WITHOUT OIDS, but if you do spe

Re: [HACKERS] Replication vs. float timestamps is a disaster

2017-02-19 Thread Tom Lane
Robert Haas writes: > On Sun, Feb 19, 2017 at 3:31 AM, Tom Lane wrote: >> Thoughts? Should we double down on trying to make this work according >> to the "all integer timestamps" protocol specs, or cut our losses and >> change the specs? > I vote for doubling down. It's bad enough that we have

Re: [HACKERS] drop support for Python 2.3

2017-02-19 Thread Tom Lane
I wrote: > Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= writes: >> arapaima(x86) and aholehole(x86_64) are the new animals. They are running the >> buildfarm script now. > ... and failing. I wonder what is wrong with tcl_date_week()? > Will look for the explanation in a bit. Relevant question: what versi

Re: [HACKERS] Documentation improvements for partitioning

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 11:25 AM, Ashutosh Bapat wrote: > Forgot to attach the patch. Thanks Rajkumar for notifying me. I think this is overexplaining what is anyway obvious. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers

Re: [HACKERS] Suppressing that pesky warning with older flex versions

2017-02-19 Thread Tom Lane
Robert Haas writes: > On Sat, Feb 18, 2017 at 10:40 PM, Tom Lane wrote: >> It seems like it would be quite simple and reliable to apply a patch >> that inserts "(void) yyg;" into this function. (Which, indeed, is >> essentially how flex 2.5.36 and later fixed it.) > Sounds fine as a master-only

Re: [HACKERS] drop support for Python 2.3

2017-02-19 Thread Tom Lane
Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= writes: > arapaima(x86) and aholehole(x86_64) are the new animals. They are running the > buildfarm script now. ... and failing. I wonder what is wrong with tcl_date_week()? Will look for the explanation in a bit. regards, tom lane --

Re: [HACKERS] Instability in select_parallel regression test

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 6:50 PM, Amit Kapila wrote: > To close the remaining gap, don't you think we can check slot->in_use > flag when generation number for handle and slot are same. That doesn't completely fix it either, because ForgetBackgroundWorker() also does BackgroundWorkerData->parallel_

Re: [HACKERS] Allow pg_dumpall to work without pg_authid

2017-02-19 Thread Robins Tharakan
On 19 February 2017 at 17:02, Robins Tharakan wrote: > > On Sun, 19 Feb 2017 at 10:08 Stephen Frost wrote: > >> If anything, it should use pg_roles, not pg_user. >> >> I don't really like the "--avoid-pgauthid" option, but "--no-passwords" >> would probably work. >> >> > '--no-passwords' is a goo

Re: [HACKERS] Parallel bitmap heap scan

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 7:18 PM, Dilip Kumar wrote: > I have observed one problem with 0002 and I though of sharing that > before fixing the same because we might have encountered the same > problem in some other patches i.e parallel shared hash and there might > be already a way to handle that. >

Re: [HACKERS] Parallel bitmap heap scan

2017-02-19 Thread Amit Kapila
On Sun, Feb 19, 2017 at 7:18 PM, Dilip Kumar wrote: > On Sat, Feb 18, 2017 at 10:45 PM, Dilip Kumar wrote: >> in 0002: >> - Improved comments. >> - Code refactoring in BitmapHeapNext. >> - Removed local tbm creation in BitmapHeapNext : as per new tidbitmap >> it's of no use. > > I have observed o

Re: [HACKERS] COPY IN/BOTH vs. extended query mode

2017-02-19 Thread Robert Haas
On Thu, Feb 16, 2017 at 1:58 PM, Corey Huinker wrote: > Forgive my ignorance, but is this issue related to the Catch-22 I had with > "COPY as a set returning function", wherein a function that invokes > BeginCopyFrom() basically starts a result set, but then ends it to do the > BeginCopyFrom() hav

Re: [HACKERS] Gather Merge

2017-02-19 Thread Amit Kapila
On Sun, Feb 19, 2017 at 2:22 PM, Robert Haas wrote: > On Sat, Feb 18, 2017 at 6:43 PM, Amit Kapila wrote: >> I think there is a value in supporting mark/restore position for any >> node which produces sorted results, however, if you don't want to >> support it, then I think we should update above

Re: [HACKERS] Parallel bitmap heap scan

2017-02-19 Thread Dilip Kumar
On Sat, Feb 18, 2017 at 10:45 PM, Dilip Kumar wrote: > in 0002: > - Improved comments. > - Code refactoring in BitmapHeapNext. > - Removed local tbm creation in BitmapHeapNext : as per new tidbitmap > it's of no use. I have observed one problem with 0002 and I though of sharing that before fixing

Re: [HACKERS] Avoiding OOM in a hash join with many duplicate inner keys

2017-02-19 Thread Robert Haas
On Thu, Feb 16, 2017 at 8:13 PM, Thomas Munro wrote: > Obviously there are vanishing returns here as we add more defences > making it increasingly unlikely that we hit "fail" mode. But it > bothers me that hash joins in general are not 100% guaranteed to be > able to complete unless you have infi

Re: [HACKERS] Instability in select_parallel regression test

2017-02-19 Thread Amit Kapila
On Sun, Feb 19, 2017 at 5:54 PM, Robert Haas wrote: > On Sun, Feb 19, 2017 at 2:17 PM, Robert Haas wrote: >> Such a change can be made, but as I pointed out in the part you didn't >> quote, there are reasons to wonder whether that will be a constructive >> change in real life even if it's better

Re: [HACKERS] SCRAM authentication, take three

2017-02-19 Thread Michael Paquier
On Wed, Feb 15, 2017 at 8:28 PM, Heikki Linnakangas wrote: > On 02/07/2017 04:20 AM, Michael Paquier wrote: >> --- a/src/backend/utils/errcodes.txt >> +++ b/src/backend/utils/errcodes.txt >> @@ -247,6 +247,7 @@ Section: Class 28 - Invalid Authorization >> Specification >> >> 28000EERRCODE

Re: [HACKERS] Provide list of subscriptions and publications in psql's completion

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 6:13 PM, Michael Paquier wrote: > On Sun, Feb 19, 2017 at 8:03 PM, Magnus Hagander wrote: >> If password auth is used, we have to store the password in plaintext >> equivalent somewhere. Meaning it's by definition going to be exposed to >> superusers and replication downst

Re: [HACKERS] Provide list of subscriptions and publications in psql's completion

2017-02-19 Thread Magnus Hagander
On Sun, Feb 19, 2017 at 1:43 PM, Michael Paquier wrote: > On Sun, Feb 19, 2017 at 8:03 PM, Magnus Hagander > wrote: > > If password auth is used, we have to store the password in plaintext > > equivalent somewhere. Meaning it's by definition going to be exposed to > > superusers and replication

Re: [HACKERS] Provide list of subscriptions and publications in psql's completion

2017-02-19 Thread Michael Paquier
On Sun, Feb 19, 2017 at 8:03 PM, Magnus Hagander wrote: > If password auth is used, we have to store the password in plaintext > equivalent somewhere. Meaning it's by definition going to be exposed to > superusers and replication downstreams. Another possibility is to mention the use of the new p

Re: [HACKERS] Instability in select_parallel regression test

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 5:54 PM, Robert Haas wrote: > However, it looks like there's a race condition here, because the slot > doesn't get freed up at the same time that the PID gets set to 0. > That actually happens later, when the postmaster calls > maybe_start_bgworker() or DetermineSleepTime()

Re: [HACKERS] Provide list of subscriptions and publications in psql's completion

2017-02-19 Thread Magnus Hagander
On Sun, Feb 19, 2017 at 1:03 PM, Petr Jelinek wrote: > On 19/02/17 12:03, Magnus Hagander wrote: > > > > > > On Sun, Feb 19, 2017 at 2:01 AM, Michael Paquier > > mailto:michael.paqu...@gmail.com>> wrote: > > > > On Sun, Feb 19, 2017 at 9:50 AM, Michael Paquier > > mailto:michael.paqu...@g

Re: [HACKERS] Instability in select_parallel regression test

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 2:17 PM, Robert Haas wrote: > Such a change can be made, but as I pointed out in the part you didn't > quote, there are reasons to wonder whether that will be a constructive > change in real life even if it's better for the regression tests. > Optimizing PostgreSQL for the

Re: [HACKERS] Provide list of subscriptions and publications in psql's completion

2017-02-19 Thread Petr Jelinek
On 19/02/17 12:03, Magnus Hagander wrote: > > > On Sun, Feb 19, 2017 at 2:01 AM, Michael Paquier > mailto:michael.paqu...@gmail.com>> wrote: > > On Sun, Feb 19, 2017 at 9:50 AM, Michael Paquier > mailto:michael.paqu...@gmail.com>> wrote: > > I have been poking at it, and yeah... I mi

Re: [HACKERS] SCRAM authentication, take three

2017-02-19 Thread Michael Paquier
On Sun, Feb 19, 2017 at 6:55 PM, Robert Haas wrote: > Gosh, this SCRAM stuff seems to be taking us pretty deeply into > dealing with encoding details which apparently we haven't formerly > needed to worry about. That is a little surprising and maybe > something we should try to avoid? The RFC of

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 3:52 PM, Pavan Deolasee wrote: > This particular case of corruption results in a heap tuple getting indexed > by a wrong key (or to be precise, indexed by its old value). So the only way > to detect the corruption is to look at each index key and check if it > matches with

Re: [HACKERS] Adding new output parameter of pg_stat_statements to identify operation of the query.

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 2:58 PM, jasonysli(李跃森) wrote: > Hi PG hackers: > > When using pg_stat_statements to collect running SQL of PG, we find > it is hard for our program to get exact operation type of the SQL, such as > SELECT, DELETE, UPDATE, INSERT, and so on. > >So we modify th

Re: [HACKERS] [PATCH] Incremental sort (was: PoC: Partial sort)

2017-02-19 Thread Robert Haas
On Sat, Feb 18, 2017 at 4:01 PM, Alexander Korotkov wrote: > I decided to start new thread for this patch for following two reasons. > * It's renamed from "Partial sort" to "Incremental sort" per suggestion by > Robert Haas [1]. New name much better characterizes the essence of > algorithm. > *

Re: [HACKERS] [PATCH] Add pg_disable_checksums() and supporting infrastructure

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 2:28 AM, David Christensen wrote: > - Change "data_checksums" from a simple boolean to "data_checksum_state", an > enum type for all of > the potentially-required states for this feature (as well as enabling). Color me skeptical. I don't know what CHECKSUMS_ENABLING, C

Re: [HACKERS] Provide list of subscriptions and publications in psql's completion

2017-02-19 Thread Magnus Hagander
On Sun, Feb 19, 2017 at 2:01 AM, Michael Paquier wrote: > On Sun, Feb 19, 2017 at 9:50 AM, Michael Paquier > wrote: > > I have been poking at it, and yeah... I missed the fact that > > pg_subcription is not a view. I thought that check_conninfo was being > > called in this context only.. > > Sti

Re: [HACKERS] Parallel Index-only scan

2017-02-19 Thread Robert Haas
On Sat, Feb 18, 2017 at 12:02 PM, Amit Kapila wrote: > + /* > +* If we are here to just update the scan keys, then don't > reset parallel > +* scan. For detailed reason behind this look in the comments for > +* ExecReScanIndexScan. > +*/ > > You can phrase the

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

2017-02-19 Thread Pavan Deolasee
On Sun, Feb 19, 2017 at 3:43 PM, Robert Haas wrote: > On Fri, Feb 17, 2017 at 11:15 PM, Tom Lane wrote: > > > Ah, nah, scratch that. If any post-index-build pruning had occurred on a > > page, the evidence would be gone --- the non-matching older tuples would > > be removed and what remained wo

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-19 Thread Robert Haas
On Sat, Feb 18, 2017 at 4:52 AM, Tomas Vondra wrote: > I have my doubts about this actually addressing gitlab-like mistakes, > though, because it's a helluva jump from "It's waiting and not doing > anything," to "We need to remove the datadir." (One of the reasons being > that non-empty directory

Re: [HACKERS] Does having a NULL column automatically exclude the table from the tupleDesc cache?

2017-02-19 Thread Robert Haas
On Sat, Feb 18, 2017 at 12:33 AM, Jim Nasby wrote: > On 2/15/17 1:37 PM, Ryan Murphy wrote: >> attcacheoff can only be set positive for fields preceding any varlena >> (typlen<0, but including the first such) or nullable values. I don't >> know how much faster it is with the cache; yo

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 11:15 PM, Tom Lane wrote: > I wrote: >> However, you might be able to find it without so much random I/O. >> I'm envisioning a seqscan over the table, in which you simply look for >> HOT chains in which the indexed columns aren't all the same. When you >> find one, you'd h

Re: [HACKERS] SUBSCRIPTIONS and pg_upgrade

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 7:34 AM, Stephen Frost wrote: > I'm not entirely sure about the reasoning behind requiring a flag to > include subscriptions in pg_dump output, as the documentation doesn't > actually provide one, but if we are going to require that, shouldn't > pg_upgrade use it, to make s

Re: [HACKERS] SCRAM authentication, take three

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 7:29 PM, Michael Paquier wrote: > On Wed, Feb 15, 2017 at 9:27 PM, Michael Paquier > wrote: >> On Wed, Feb 15, 2017 at 7:58 PM, Heikki Linnakangas wrote: >>> On 02/09/2017 09:33 AM, Michael Paquier wrote: Now regarding the shape of the implementation for SCRAM, we ne

Re: [HACKERS] Partitioned tables and relfilenode

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 1:12 PM, Amit Langote wrote: >> I agree. But, we need to be careful that a database-wide VACUUM or >> ANALYZE doesn't hit the partitions multiple times, once for the parent >> and again for each child. Actually, a database-wide VACUUM should hit >> each partition individu

Re: [HACKERS] Replication vs. float timestamps is a disaster

2017-02-19 Thread Robert Haas
On Sun, Feb 19, 2017 at 3:31 AM, Tom Lane wrote: > Thoughts? Should we double down on trying to make this work according > to the "all integer timestamps" protocol specs, or cut our losses and > change the specs? I vote for doubling down. It's bad enough that we have so many internal details th

Re: [HACKERS] Reporting xmin from VACUUMs

2017-02-19 Thread Robert Haas
On Sat, Feb 18, 2017 at 12:30 AM, Jim Nasby wrote: >> What? > > There's a bunch of information reported by vacuum logging but not in > pg_stat*, such as all-visible/frozen skipping, unable to get cleanup lock, > last freeze scan, times autovac has been interrupted. There's been > resistance in the

Re: [HACKERS] Parallel Append implementation

2017-02-19 Thread Robert Haas
On Fri, Feb 17, 2017 at 2:56 PM, Amit Khandekar wrote: > The log2(num_children)+1 formula which you proposed does not take into > account the number of workers for each of the subplans, that's why I > am a bit more inclined to look for some other logic. May be, treat the > children as if they belo

  1   2   >