Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-15 Thread Masahiko Sawada
On Wed, Feb 15, 2017 at 4:39 PM, Ideriha, Takeshi wrote: > Hi, I tried regression test and found some errors concerning brin and gin, > though I didn't look into this. > > Here's a log: > > *** /home/ideriha/postgres-master/src/test/regress/expected/brin.out > 2017-02-13 11:33:43.270942937 +09

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-02-15 Thread Simon Riggs
On 14 February 2017 at 06:19, Seki, Eiji wrote: > In our benchmark, we found that waiting an ANALYZE process created by > autovacuum daemon often has a significant impact to the performance although > the waited process do only reading as to the table. ... > I'm not sure that this feature is use

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-15 Thread Alexander Korotkov
Hi! On Mon, Feb 13, 2017 at 8:02 PM, Ashutosh Sharma wrote: > I too have performed benchmarking of this patch on a large machine > (with 128 CPU(s), 520GB RAM, intel x86-64 architecture) and would like > to share my observations for the same (Please note that, as I had to > reverify readings on

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-02-15 Thread Michael Paquier
On Wed, Feb 15, 2017 at 5:32 PM, Simon Riggs wrote: > Please persuade us with measurements that allowing this impact on > ANALYZE would really improve performance at least in your case, and > also examine the effect of this on the accuracy and usefulness of the > gathered statistics. FWIW, there

Re: [HACKERS] Documentation improvements for partitioning

2017-02-15 Thread Ashutosh Bapat
Noticed some typos in the documentation. Here's patch to correct those. Sorry, if it has been already taken care of. On Wed, Feb 15, 2017 at 10:01 AM, Robert Haas wrote: > On Mon, Feb 13, 2017 at 5:57 AM, Amit Langote > wrote: >> On 2017/02/13 14:21, Amit Langote wrote: >>> On 2017/02/10 19:19,

Re: [HACKERS] SERIALIZABLE with parallel query

2017-02-15 Thread Thomas Munro
On Wed, Nov 9, 2016 at 10:51 AM, Thomas Munro wrote: > Need to audit predicate.c for cases where > MySerializableXact might be modified without suitable locking, The only thing I see along those lines is that CheckForSerializableConflictOut() and CheckForSerializableConflictIn() access SxactIsDoo

Re: [HACKERS] SERIALIZABLE with parallel query

2017-02-15 Thread Thomas Munro
On Wed, Nov 9, 2016 at 12:34 PM, Peter Geoghegan wrote: > On Tue, Nov 8, 2016 at 1:51 PM, Thomas Munro > wrote: >> Currently we don't generate parallel plans in SERIALIZABLE. What >> problems need to be solved to be able to do that? > > FWIW, parallel CREATE INDEX works at SERIALIZABLE isolation

Re: [HACKERS] Parallel Append implementation

2017-02-15 Thread Amit Khandekar
> On 14 February 2017 at 22:35, Robert Haas wrote: >> For example, suppose that I have a scan of two children, one >> of which has parallel_workers of 4, and the other of which has >> parallel_workers of 3. If I pick parallel_workers of 7 for the >> Parallel Append, that's probably too high. In

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-02-15 Thread Alvaro Herrera
Michael Paquier wrote: > On Wed, Feb 15, 2017 at 5:32 PM, Simon Riggs wrote: > > Please persuade us with measurements that allowing this impact on > > ANALYZE would really improve performance at least in your case, and > > also examine the effect of this on the accuracy and usefulness of the > > g

Re: [HACKERS] SCRAM authentication, take three

2017-02-15 Thread Heikki Linnakangas
On 02/09/2017 09:33 AM, Michael Paquier wrote: On Tue, Feb 7, 2017 at 11:28 AM, Michael Paquier wrote: Yes, I am actively working on this one now. I am trying to come up first with something in the shape of an extension to begin with, and get a patch out of it. That will be more simple for test

Re: [HACKERS] SCRAM authentication, take three

2017-02-15 Thread Heikki Linnakangas
On 02/07/2017 04:20 AM, Michael Paquier wrote: On Tue, Feb 7, 2017 at 3:12 AM, Aleksander Alekseev wrote: No, I'm afraid `make distclean` doesn't help. I've re-checked twice. Hm. I can see the failure on macos and python2 builds as well with the set of patches applied. And the master branch i

Re: [HACKERS] Parallel Index Scans

2017-02-15 Thread Amit Kapila
On Wed, Feb 15, 2017 at 2:04 AM, Robert Haas wrote: > On Tue, Feb 14, 2017 at 12:48 PM, Robert Haas wrote: >> That sounds way better. > > Here's an updated patch. Please review my changes, which include: > > * Various comment updates. 1. + * BTPARALLEL_IDLE indicates that no backend is currentl

Re: [HACKERS] Parallel bitmap heap scan

2017-02-15 Thread Dilip Kumar
On Tue, Feb 14, 2017 at 10:18 PM, Robert Haas wrote: > Thanks, the external interface to this looks much cleaner now. > Scrutinizing the internals: Thanks for the comments, I am working on these. I have few doubts for some of the comments. > I suggest removing both "entry1" and "status" from > T

Re: [HACKERS] Bug in to_timestamp().

2017-02-15 Thread Amul Sul
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: tested, passed Spec compliant: not tested Documentation:tested, passed Review of v7 patch: - Patch applies to the top of master HEAD cleanl

Re: [HACKERS] SCRAM authentication, take three

2017-02-15 Thread Michael Paquier
On Wed, Feb 15, 2017 at 7:58 PM, Heikki Linnakangas wrote: > On 02/09/2017 09:33 AM, Michael Paquier wrote: >> Now regarding the shape of the implementation for SCRAM, we need one >> thing: a set of routines in src/common/ to build decompositions for a >> given UTF-8 string with conversion UTF8 st

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-02-15 Thread Amit Kapila
On Wed, Feb 15, 2017 at 12:03 PM, Seki, Eiji wrote: > Amit Kapila wrote: >> How will you decide just based on oldest xmin whether the tuple is visible >> or not? How will you take decisions about tuples which have xmax set? > > In our use case, GetOldestXmin is used by an original maintainer pro

Re: [HACKERS] Parallel Index Scans

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 7:11 AM, Amit Kapila wrote: > Here second part of the comment (but have not yet advanced ..) seems > to be slightly misleading because this state has nothing to do with > the advancement of scan keys. > > I have not changed this because I am not sure what you have in mind.

Re: [HACKERS] Sum aggregate calculation for single precsion real

2017-02-15 Thread Robert Haas
On Tue, Feb 14, 2017 at 11:45 PM, Tom Lane wrote: > You could perhaps make an argument that sum(float4) would have less risk > of overflow if it accumulated in and returned float8, but frankly that > seems a bit thin. I think that's more or less the argument Konstantin is in fact making. Whether

Re: [HACKERS] Documentation improvements for partitioning

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 4:26 AM, Ashutosh Bapat wrote: > Noticed some typos in the documentation. Here's patch to correct > those. Sorry, if it has been already taken care of. Thanks. That is indeed nonstandard capitalization. Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.co

Re: [HACKERS] Parallel Append implementation

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 2:33 AM, Amit Khandekar wrote: > The only reason I combined the soft limit and the hard limit is > because it is not necessary to have two different fields. But of > course this is again under the assumption that allocating more than > parallel_workers would never improve t

Re: [HACKERS] Parallel Append implementation

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 4:43 AM, Amit Khandekar wrote: >> On 14 February 2017 at 22:35, Robert Haas wrote: >>> For example, suppose that I have a scan of two children, one >>> of which has parallel_workers of 4, and the other of which has >>> parallel_workers of 3. If I pick parallel_workers of

Re: [HACKERS] CREATE TABLE with parallel workers, 10.0?

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 12:24 AM, Joshua Chamberlain wrote: > Hello, > > (I'm posting to hackers since I got no response on the general list.) > > I use Postgres + PostGIS quite heavily, and recently have been taking full > advantage of the new parallelism in 9.6. I'm now running queries in a few

Re: [HACKERS] SERIALIZABLE with parallel query

2017-02-15 Thread Robert Haas
On Tue, Nov 8, 2016 at 4:51 PM, Thomas Munro wrote: > Currently we don't generate parallel plans in SERIALIZABLE. What > problems need to be solved to be able to do that? I'm probably > steamrolling over a ton of subtleties and assumptions here, but it > occurred to me that a first step might be

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-02-15 Thread Kuntal Ghosh
Hello everyone, As discussed in this thread, I've attached a set of patches to show auxiliary processes, autovacuum launcher and bgworker along with other backends in pg_stat_activity. For now, I've extended BackendStatusArray to store auxiliary processes. Backends use slots indexed in the range f

Re: [HACKERS] Documentation improvements for partitioning

2017-02-15 Thread Simon Riggs
On 13 February 2017 at 05:21, Amit Langote wrote: >> Few points from tests >> >> * "ERROR: cannot create index on partitioned table "measurement_year_month"" >> is misleading because you can create indexes on partitions > > Do you mean that this should not cause an error at all, but create the >

Re: [HACKERS] Documentation improvements for partitioning

2017-02-15 Thread Joshua D. Drake
On 02/15/2017 06:10 AM, Simon Riggs wrote: On 13 February 2017 at 05:21, Amit Langote wrote: If I issue DROP TABLE elsewhere, it doesn't refuse to drop because it has indexes, sequences etc on it. So why should it just because it has partitions? Because partitions may have data. JD -- C

Re: [HACKERS] Documentation improvements for partitioning

2017-02-15 Thread Alvaro Herrera
Joshua D. Drake wrote: > On 02/15/2017 06:10 AM, Simon Riggs wrote: > > On 13 February 2017 at 05:21, Amit Langote > > wrote: > > > If I issue DROP TABLE elsewhere, it doesn't refuse to drop because it > > has indexes, sequences etc on it. So why should it just because it has > > partitions? > >

Re: [HACKERS] UPDATE of partition key

2017-02-15 Thread David Fetter
On Wed, Feb 15, 2017 at 01:06:32PM +0530, Amit Khandekar wrote: > On 14 February 2017 at 22:24, David Fetter wrote: > > On Mon, Feb 13, 2017 at 05:31:56PM +0530, Amit Khandekar wrote: > >> Currently, an update of a partition key of a partition is not > >> allowed, since it requires to move the row

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-15 Thread Alexander Korotkov
On Wed, Feb 15, 2017 at 11:49 AM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Mon, Feb 13, 2017 at 8:02 PM, Ashutosh Sharma > wrote: > >> I too have performed benchmarking of this patch on a large machine >> (with 128 CPU(s), 520GB RAM, intel x86-64 architecture) and would like >>

Re: [HACKERS] CREATE COLLATION IF NOT EXISTS

2017-02-15 Thread Peter Eisentraut
On 2/8/17 10:55 PM, Peter Eisentraut wrote: > Here is a patch to complete the implementation of CREATE COLLATION IF > NOT EXISTS. The meat of this was already implemented for > pg_import_system_collations; this just exposes it in the SQL command. committed -- Peter Eisentraut http:

Re: [HACKERS] renaming pg_resetxlog to pg_resetwal has broken pg_upgrade.

2017-02-15 Thread Robert Haas
On Tue, Feb 14, 2017 at 5:13 PM, Michael Paquier wrote: > On Wed, Feb 15, 2017 at 3:55 AM, Jeff Janes wrote: >> I thought people would object to checking the version number in two >> different places to make the same fundamental decision, and would want that >> refactored somehow. But if you are

Re: [HACKERS] Documentation improvements for partitioning

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 9:37 AM, Alvaro Herrera wrote: > Joshua D. Drake wrote: >> On 02/15/2017 06:10 AM, Simon Riggs wrote: >> > On 13 February 2017 at 05:21, Amit Langote >> > wrote: >> >> > If I issue DROP TABLE elsewhere, it doesn't refuse to drop because it >> > has indexes, sequences etc o

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-02-15 Thread Kuntal Ghosh
On Mon, Feb 13, 2017 at 8:52 PM, Amit Kapila wrote: > As discussed, attached are refactoring patches and a patch to enable > WAL for the hash index on top of them. 0006-Enable-WAL-for-Hash-Indexes.patch needs to be rebased after commit 8da9a226369e9ceec7cef1. -- Thanks & Regards, Kuntal Ghosh

Re: [HACKERS] Reporting xmin from VACUUMs

2017-02-15 Thread Jim Nasby
Accidentally sent o--list. On 2/14/17 12:57 PM, Jim Nasby wrote: On 2/13/17 12:36 AM, Masahiko Sawada wrote: It seems sensible to me to do something like this. We already report a lot of other fine details, so what's one more? And it could be useful. Also, I've been proposing to report the

[HACKERS] [PATCH] Teach Catalog.pm how many attributes there should be per DATA() line

2017-02-15 Thread David Christensen
Throws a build error if we encounter a different number of fields in a DATA() line than we expect for the catalog in question. Previously, it was possible to silently ignore any mismatches at build time which could result in symbol undefined errors at link time. Now we stop and identify the infri

[HACKERS] [PATCH] Fix pg_proc comment grammar

2017-02-15 Thread David Christensen
Fixes some DESCR() grammar mistakes introduced by the xlog -> wal changes. --- src/include/catalog/pg_proc.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h index 41c12af..bb7053a 100644 --- a/src/include/catalo

Re: [HACKERS] Documentation improvements for partitioning

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 9:10 AM, Simon Riggs wrote: >>> * "ERROR: cannot create index on partitioned table >>> "measurement_year_month"" >>> is misleading because you can create indexes on partitions >> >> Do you mean that this should not cause an error at all, but create the >> specified index

Re: [HACKERS] Add checklist item for psql completion to commitfest review

2017-02-15 Thread Robert Haas
On Tue, Feb 14, 2017 at 1:13 PM, Jim Nasby wrote: > After seeing Yet Another Missing Psql Tab Completion it occurred to me... > why not add a checklist item to the commitfest review page? I realize most > regular contributors don't use the form, but a fair number of people do. I > don't see how it

Re: [HACKERS] pg_basebackup -R

2017-02-15 Thread Robert Haas
On Tue, Feb 14, 2017 at 11:36 PM, Michael Paquier wrote: > On Wed, Feb 15, 2017 at 2:38 AM, Robert Haas wrote: >> On Thu, Feb 9, 2017 at 9:40 PM, Michael Paquier >> wrote: >>> In short, I'd like to think that we should just filter out those two >>> parameters by name and call it a day. Or introd

Re: [HACKERS] Add doc advice about systemd RemoveIPC

2017-02-15 Thread Peter Eisentraut
On 2/14/17 11:49 AM, Magnus Hagander wrote: > + > +If systemd is in use, some care must be > taken > +that IPC resources (shared memory and semaphores) are not prematurely > +removed by the operating system. This is especially of concern when > +installing PostgreSQL from source

Re: [HACKERS] [Bug fix] PQsendQuery occurs error when target_session_attrs is set to read-write

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 1:31 AM, Michael Paquier wrote: > On Wed, Feb 15, 2017 at 11:08 AM, Robert Haas wrote: >> I think the patch as presented probably isn't quite what we want, >> because it waits synchronously for the second result to be ready. >> Note that the wait for the first result is as

Re: [HACKERS] Partitioned tables and relfilenode

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 2:14 AM, Michael Paquier wrote: > This is a collection of checks on relkind == RELKIND_PARTITIONED_TABLE > to avoid interactions with partition tables. Did you consider doing > something in the executor instead? That seems inferior, because the planner really ought to know

Re: [HACKERS] CONNECTION LIMIT and Parallel Query don't play well together

2017-02-15 Thread Peter Eisentraut
On 1/11/17 5:51 PM, David Rowley wrote: > Now, since background workers > don't consume anything from max_connections, then I don't really feel > that a background worker should count towards "CONNECTION LIMIT". I'd > assume any CONNECTION LIMITs that are set for a user would be > calculated based

Re: [HACKERS] CONNECTION LIMIT and Parallel Query don't play well together

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 11:19 AM, Peter Eisentraut wrote: > On 1/11/17 5:51 PM, David Rowley wrote: >> Now, since background workers >> don't consume anything from max_connections, then I don't really feel >> that a background worker should count towards "CONNECTION LIMIT". I'd >> assume any CONNE

Re: [HACKERS] Parallel bitmap heap scan

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 7:17 AM, Dilip Kumar wrote: > This is easily doable and it will look much cleaner. While doing this > I am facing one problem related to > relptr_store and relptr_access. The problem is that when I call > "relptr_store (base, rp, val)" with both base and val as a same > ad

[HACKERS] error handling in RegisterBackgroundWorker

2017-02-15 Thread Peter Eisentraut
If RegisterBackgroundWorker() (the non-dynamic kind that is only loadable from shared_preload_libraries) fails to register the worker, it writes a log message and proceeds, ignoring the registration request. I think that is a mistake, it should be a hard error. The only way in practice to fix the

Re: [HACKERS] Documentation improvements for partitioning

2017-02-15 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Feb 15, 2017 at 9:37 AM, Alvaro Herrera > wrote: > > Joshua D. Drake wrote: > >> Because partitions may have data. > > > > So would the table, were it not partitioned. > > True. I think the question here is: do we want to view the dependency > between a partitioned

[HACKERS] Does having a NULL column automatically exclude the table from the tupleDesc cache?

2017-02-15 Thread Ryan Murphy
Hi all, I was looking through some of the implementation details of the heap/tuples, specifically src/include/access/htup_details.h - and I came across the big macro fastgetattr, and had a question about it. I've included the code here for clarity and convenience: #define fastgetattr(tup, at

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-15 Thread Robert Haas
On Mon, Feb 13, 2017 at 11:07 AM, Alvaro Herrera wrote: > It seems to me that Andres comments here were largely ignored: > https://www.postgresql.org/message-id/20160822021747.u5bqx2xwwjzac...@alap3.anarazel.de > He was suggesting to increase the struct size to 16 bytes rather than > going all the

[HACKERS] Help text for pg_basebackup -R

2017-02-15 Thread Magnus Hagander
The current help text for pg_basebackup -R is "write recovery.conf after backup". This says nothing about what it actually does. I've had a number of people ask me now why that's not default "because you need a recovery.conf to restore from backup". The point being that it doesn't say anything abo

Re: [HACKERS] Documentation improvements for partitioning

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 11:34 AM, Alvaro Herrera wrote: > I think new-style partitioning is supposed to consider each partition as > an implementation detail of the table; the fact that you can manipulate > partitions separately does not really mean that they are their own > independent object. Y

Re: [HACKERS] error handling in RegisterBackgroundWorker

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 11:30 AM, Peter Eisentraut wrote: > If RegisterBackgroundWorker() (the non-dynamic kind that is only > loadable from shared_preload_libraries) fails to register the worker, it > writes a log message and proceeds, ignoring the registration request. I > think that is a mista

Re: [HACKERS] Documentation improvements for partitioning

2017-02-15 Thread Tom Lane
Alvaro Herrera writes: > Robert Haas wrote: >> True. I think the question here is: do we want to view the dependency >> between a partitioned table and a partition of that table as >> DEPENDENCY_NORMAL or as DEPENDENCY_AUTO? With table inheritance, it's >> always been "normal" and I'm not sure t

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-15 Thread Tom Lane
Robert Haas writes: > On Mon, Feb 13, 2017 at 11:07 AM, Alvaro Herrera > wrote: >> It seems to me that Andres comments here were largely ignored: >> https://www.postgresql.org/message-id/20160822021747.u5bqx2xwwjzac...@alap3.anarazel.de >> He was suggesting to increase the struct size to 16 bytes

Re: [HACKERS] [PATCH] Fix pg_proc comment grammar

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 10:40 AM, David Christensen wrote: > Fixes some DESCR() grammar mistakes introduced by the xlog -> wal changes. Thanks, David! Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Does having a NULL column automatically exclude the table from the tupleDesc cache?

2017-02-15 Thread Tom Lane
Ryan Murphy writes: > My question is this: HeapTupleNoNulls() is run first to see if there are > any columns that can be NULL. It looks like the fetchatt() that uses the > cache in the tupleDesc can only be used if there are no NULLable columns in > the table. Is my understanding correct? No,

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 12:14 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Feb 13, 2017 at 11:07 AM, Alvaro Herrera >> wrote: >>> It seems to me that Andres comments here were largely ignored: >>> https://www.postgresql.org/message-id/20160822021747.u5bqx2xwwjzac...@alap3.anarazel.de >>>

Re: [HACKERS] Does having a NULL column automatically exclude the table from the tupleDesc cache?

2017-02-15 Thread Ryan Murphy
> No, that tests whether the particular tuple contains any null fields, not > whether the whole table is declared NOT NULL. > > regards, tom lane > Ok, thanks, that makes sense. My question kind of remains though - does that mean that having any nulls in the tuple loses th

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-02-15 Thread Robert Haas
On Tue, Feb 14, 2017 at 1:38 PM, Jim Nasby wrote: > On 2/14/17 3:13 AM, Seki, Eiji wrote: >> +extern TransactionId GetOldestXmin(Relation rel, uint8 ignoreFlags); > > > My impression is that most other places that do this sort of thing just call > the argument 'flags', so as not to "lock in" a s

Re: [HACKERS] bytea_output vs make installcheck

2017-02-15 Thread Jeff Janes
On Tue, Feb 14, 2017 at 9:17 PM, Andres Freund wrote: > > > On February 14, 2017 9:02:14 PM PST, neha khatri > wrote: > >On Wed, Feb 15, 2017 at 10:04 AM, neha khatri > > wrote:. > >> > >> > >>> Attached are two options for doing that. One overrides bytea_output > >>> locally where-ever needed

Re: [HACKERS] Sum aggregate calculation for single precsion real

2017-02-15 Thread Tom Lane
Robert Haas writes: > I think that's more or less the argument Konstantin is in fact making. > Whether it's a good argument or a thin one is a value judgement. > Personally, I find it somewhere in the middle: I think the way it > works now is reasonable, and I think what he wants would have been >

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-15 Thread Tom Lane
Robert Haas writes: > ... Maybe that difference matters to the memory prefetching > controller, I dunno, but it seems funny that we did the PGXACT work to > reduce the number of cache lines that had to be touched in order to > take a snapshot to improve performance, and now we're talking about > i

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-15 Thread Alvaro Herrera
Alexander Korotkov wrote: > Difference between master, pgxact-align-2 and pgxact-align-3 doesn't exceed > per run variation. FWIW this would be more visible if you added error bars to each data point. Should be simple enough in gnuplot ... -- Álvaro Herrerahttps://www.2ndQuadra

Re: [HACKERS] IF NOT EXISTS option for CREATE SERVER and CREATE USER MAPPING statements

2017-02-15 Thread Anastasia Lubennikova
13.02.2017 19:34, Andrew Dunstan: On 01/13/2017 08:36 AM, Anastasia Lubennikova wrote: I implemented IF NOT EXISTS option for CREATE SERVER and CREATE USER MAPPING statements for one of our customers. I think other users can also find it useful for scripting and automated tasks. The patches the

Re: [HACKERS] Help text for pg_basebackup -R

2017-02-15 Thread Alvaro Herrera
Magnus Hagander wrote: > printf(_(" -R, --write-recovery-conf\n" > - " write recovery.conf after > backup\n")); > + " write recovery.conf for > replication\n")); > printf(_(" -S, --slot=SLOTNA

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 12:48 PM, Tom Lane wrote: > Robert Haas writes: >> ... Maybe that difference matters to the memory prefetching >> controller, I dunno, but it seems funny that we did the PGXACT work to >> reduce the number of cache lines that had to be touched in order to >> take a snapsho

Re: [HACKERS] Reporting xmin from VACUUMs

2017-02-15 Thread Alvaro Herrera
Jim Nasby wrote: > ISTR previous discussion of allowing more stats files; if that happened > I think having stats that were dedicated to (auto)vacuum would be very > useful. That's clearly a lot more work though. What? -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Dev

Re: [HACKERS] Does having a NULL column automatically exclude the table from the tupleDesc cache?

2017-02-15 Thread Alvaro Herrera
Ryan Murphy wrote: > My question kind of remains though - does that mean that having any nulls > in the tuple loses the ability to use the tupleDesc cache? and how much of > a performance impact is this? I'm sure there's a good reason why you can't > really use the cache if you have a null column

Re: [HACKERS] Parallel Index Scans

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 7:11 AM, Amit Kapila wrote:> > support related patch. In anycase, to avoid confusion I am attaching > all the three patches with this e-mail. Committed guc_parallel_index_scan_v1.patch, with changes to the documentation and GUC descriptions. -- Robert Haas EnterpriseDB:

Re: [HACKERS] Parallel Index Scans

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 1:39 PM, Robert Haas wrote: > On Wed, Feb 15, 2017 at 7:11 AM, Amit Kapila wrote:> >> support related patch. In anycase, to avoid confusion I am attaching >> all the three patches with this e-mail. > > Committed guc_parallel_index_scan_v1.patch, with changes to the > docu

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2017-02-15 Thread Robert Haas
On Tue, Jan 24, 2017 at 4:32 AM, Peter Moser wrote: > NORMALIZE: splits all the ranges of one relation according to all the range > boundaries of another (but possibly the same) relation whenever some equality > condition over some given attributes is satisfied. > > When the two relations are the

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-15 Thread Andres Freund
Hi, On 2017-02-15 11:43:17 -0500, Robert Haas wrote: > On Mon, Feb 13, 2017 at 11:07 AM, Alvaro Herrera > wrote: > > It seems to me that Andres comments here were largely ignored: > > https://www.postgresql.org/message-id/20160822021747.u5bqx2xwwjzac...@alap3.anarazel.de > > He was suggesting to

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-15 Thread Andres Freund
Hi, On 2017-02-15 12:24:44 -0500, Robert Haas wrote: > If you pad PGXACT out to one cache line, you could likewise take a > snapshot by touching 1 cache line per backend, and they'd be > consecutive. Maybe that difference matters to the memory prefetching > controller, I dunno, Unfortunately it'

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2017-02-15 Thread Robert Haas
On Tue, Jan 24, 2017 at 4:32 AM, Peter Moser wrote: >> Using common terms such as ALIGN and NORMALIZE for such a specific >> functionality seems a bit wrong. > > Would ALIGN RANGES/RANGE ALIGN and NORMALIZE RANGES/RANGE NORMALIZE be better > options? We are also thankful for any suggestion or comm

Re: [HACKERS] Parallel Index-only scan

2017-02-15 Thread Robert Haas
On Thu, Jan 19, 2017 at 7:07 AM, Rafia Sabih wrote: > Please find the attached file rebased patch of parallel index-only > scan on the latest Parallel index-scan patch [1]. This again needs minor rebasing but basically looks fine. It's a pretty straightforward extension of the parallel index sca

Re: [HACKERS] Does having a NULL column automatically exclude the table from the tupleDesc cache?

2017-02-15 Thread Ryan Murphy
> attcacheoff can only be set positive for fields preceding any varlena > (typlen<0, but including the first such) or nullable values. I don't > know how much faster it is with the cache; you can measure it if your > curiosity is strong enough -- just set the first column to nullable. > > Thanks!

Re: [HACKERS] Should we cacheline align PGXACT?

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 2:15 PM, Andres Freund wrote: > I don't think that's true for several reasons. Separating out PGXACT > didn't just mean reducing the stride size of the access / preventing > sharing. It also meant that frequently changing fields in PGPROC aren't > on the same cache-line as

Re: [HACKERS] operator_precedence_warning vs make installcheck

2017-02-15 Thread Tom Lane
I wrote: > We could possibly prevent the difference by having exprLocation look > through such nodes. I'm not sure offhand if there are cases where > that would be worse than before. We've definitely made some other > hacks to hide the difference between operator_precedence_warning on > and off.

Re: [HACKERS] Sum aggregate calculation for single precsion real

2017-02-15 Thread Claudio Freire
On Wed, Feb 15, 2017 at 9:52 AM, Robert Haas wrote: > On Tue, Feb 14, 2017 at 11:45 PM, Tom Lane wrote: >> You could perhaps make an argument that sum(float4) would have less risk >> of overflow if it accumulated in and returned float8, but frankly that >> seems a bit thin. > > I think that's mor

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-02-15 Thread Robert Haas
On Fri, Jan 20, 2017 at 2:09 AM, Michael Paquier wrote: > Okay I just did it. At the same time the check for ferror is not > necessary as fgets() returns NULL on an error as well so that's dead > code. I have also removed the useless call to FreeFile(). diff --git a/src/backend/postmaster/postmas

Re: [HACKERS] operator_precedence_warning vs make installcheck

2017-02-15 Thread Jeff Janes
On Wed, Feb 15, 2017 at 11:47 AM, Tom Lane wrote: > I wrote: > > We could possibly prevent the difference by having exprLocation look > > through such nodes. I'm not sure offhand if there are cases where > > that would be worse than before. We've definitely made some other > > hacks to hide the

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2017-02-15 Thread David G. Johnston
On Wed, Feb 15, 2017 at 12:24 PM, Robert Haas wrote: > On Tue, Jan 24, 2017 at 4:32 AM, Peter Moser wrote: > >> Using common terms such as ALIGN and NORMALIZE for such a specific > >> functionality seems a bit wrong. > > > > Would ALIGN RANGES/RANGE ALIGN and NORMALIZE RANGES/RANGE NORMALIZE be

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-02-15 Thread Karl O. Pinc
On Wed, 15 Feb 2017 15:23:00 -0500 Robert Haas wrote: > + errhint("The supported log formats are > \"stderr\"" > +" and \"csvlog\"."))); > > I think our preferred style is not to break strings across lines like > this. How do you do that a

Re: [HACKERS] Missing CHECK_FOR_INTERRUPTS in hash joins

2017-02-15 Thread Tom Lane
Thomas Munro writes: > On Wed, Feb 15, 2017 at 2:22 PM, Tom Lane wrote: >> Adding a C.F.I. inside this loop is the most straightforward fix, but >> I am leaning towards adding one in ExecHashJoinGetSavedTuple instead, > Would it also make sense to put one in the loop in > ExecHashIncreaseNumBatc

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-02-15 Thread Alvaro Herrera
Robert Haas wrote: > On Fri, Jan 20, 2017 at 2:09 AM, Michael Paquier > wrote: > > Okay I just did it. At the same time the check for ferror is not > > necessary as fgets() returns NULL on an error as well so that's dead > > code. I have also removed the useless call to FreeFile(). > > diff --git

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-02-15 Thread Andres Freund
On 2017-02-15 12:27:11 -0500, Robert Haas wrote: > On Tue, Feb 14, 2017 at 1:38 PM, Jim Nasby wrote: > > On 2/14/17 3:13 AM, Seki, Eiji wrote: > >> +extern TransactionId GetOldestXmin(Relation rel, uint8 ignoreFlags); > > > > > > My impression is that most other places that do this sort of thing

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-02-15 Thread Tom Lane
"Karl O. Pinc" writes: > On Wed, 15 Feb 2017 15:23:00 -0500 > Robert Haas wrote: >> I think our preferred style is not to break strings across lines like >> this. > How do you do that and not exceed the 80 character line limit? > Just ignore the line length limit? Right. It depends on context,

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-02-15 Thread Tom Lane
Andres Freund writes: > On 2017-02-15 12:27:11 -0500, Robert Haas wrote: >> I agree; also, many years ago a guy named Tom Lane told me that flags >> argument should typically be declared as type "int". I've followed >> that advice ever since. > Why is that? I think uint makes a lot more sense f

Re: [HACKERS] bytea_output vs make installcheck

2017-02-15 Thread neha khatri
Agreed with Jeff, false alarms should be avoided, whenever it is easy to put the avoiding mechanism in place. Regards, Neha

Re: [HACKERS] DROP SUBSCRIPTION and ROLLBACK

2017-02-15 Thread Petr Jelinek
On 15/02/17 06:43, Masahiko Sawada wrote: > On Tue, Feb 14, 2017 at 1:13 AM, Fujii Masao wrote: >> On Mon, Feb 13, 2017 at 4:05 PM, Masahiko Sawada >> wrote: >>> On Sat, Feb 11, 2017 at 8:18 PM, Petr Jelinek >>> wrote: On 10/02/17 19:55, Masahiko Sawada wrote: > On Thu, Feb 9, 2017 at

Re: [HACKERS] Logical Replication and Character encoding

2017-02-15 Thread Petr Jelinek
On 14/02/17 03:23, Kyotaro HORIGUCHI wrote: > At Sat, 4 Feb 2017 21:27:32 +0100, Petr Jelinek > wrote in > >> Hmm I wonder if we should just make the subscriber send the >> client_encoding always (based on server encoding of the subscriber). >> That should solve the issue in combination with yo

Re: [HACKERS] bytea_output vs make installcheck

2017-02-15 Thread Andres Freund
Hi, On 2017-02-15 09:30:39 -0800, Jeff Janes wrote: > On Tue, Feb 14, 2017 at 9:17 PM, Andres Freund wrote: > > What's your reason to get this fixed? > > > > More testing is better than less testing, and a good way to get less > testing is requiring the tester to memorize a list of false positiv

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-15 Thread Jeff Janes
On Tue, Feb 14, 2017 at 9:06 AM, Magnus Hagander wrote: Yeah, that's my view as well. I'm all for including it in verbose mode. > > *Iff* we can get a progress indicator through the checkpoint we could > include that in --progress mode. But that's a different patch, of course, > but it shouldn't

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-02-15 Thread Thomas Munro
Out of archeological curiosity, I was digging around in the hash join code and RCS history from Postgres 4.2[1], and I was astounded to discover that it had a parallel executor for Sequent SMP systems and was capable of parallel hash joins as of 1991. At first glance, it seems to follow approximat

Re: [HACKERS] bytea_output vs make installcheck

2017-02-15 Thread Tom Lane
Andres Freund writes: > On 2017-02-15 09:30:39 -0800, Jeff Janes wrote: >> I don't really see the cost here. > Because that means we essentially need to make sure that our tests pass > with a combination of about ~20-30 behaviour changing gucs, and ~5 > different compilation settings that change

Re: [HACKERS] bytea_output vs make installcheck

2017-02-15 Thread Andres Freund
On 2017-02-15 18:30:30 -0500, Tom Lane wrote: > If we tried to lock that down it'd be counterproductive for the reason > Andres mentions: sometimes you *want* to see what you get for other > settings. We could kinda address that by doing it in a separate file early in the schedule, which could jus

Re: [HACKERS] Logical replication existing data copy

2017-02-15 Thread Petr Jelinek
On 13/02/17 14:51, Erik Rijkers wrote: > On 2017-02-11 11:16, Erik Rijkers wrote: >> On 2017-02-08 23:25, Petr Jelinek wrote: >> >>> 0001-Use-asynchronous-connect-API-in-libpqwalreceiver-v2.patch >>> 0002-Always-initialize-stringinfo-buffers-in-walsender-v2.patch >>> 0003-Fix-after-trigger-executio

Re: [HACKERS] bytea_output vs make installcheck

2017-02-15 Thread Tom Lane
Andres Freund writes: > On 2017-02-15 18:30:30 -0500, Tom Lane wrote: >> If we tried to lock that down it'd be counterproductive for the reason >> Andres mentions: sometimes you *want* to see what you get for other >> settings. > We could kinda address that by doing it in a separate file early in

Re: [HACKERS] AT detach partition is broken

2017-02-15 Thread Amit Langote
On 2017/02/15 2:37, Robert Haas wrote: > On Mon, Feb 13, 2017 at 2:30 AM, Amit Langote > wrote: >> I noticed that running ALTER TABLE table_name DETACH PARTITION crashes, if >> table_name is not a partitioned table. That's because of an Assert in >> ATExecDetachPartition(). We really should err

Re: [HACKERS] CREATE TABLE with parallel workers, 10.0?

2017-02-15 Thread Haribabu Kommi
On Thu, Feb 16, 2017 at 12:48 AM, Robert Haas wrote: > On Wed, Feb 15, 2017 at 12:24 AM, Joshua Chamberlain > wrote: > > Hello, > > > > (I'm posting to hackers since I got no response on the general list.) > > > > I use Postgres + PostGIS quite heavily, and recently have been taking > full > > a

  1   2   >