Re: [HACKERS] [sqlsmith] FailedAssertion("!(XLogCtl->Insert.exclusiveBackup)", File: "xlog.c", Line: 10200)

2016-08-04 Thread Michael Paquier
On Thu, Aug 4, 2016 at 2:19 AM, Andreas Seltenreich wrote: > testing with sqlsmith shows that the following assertion doesn't hold: > > FailedAssertion("!(XLogCtl->Insert.exclusiveBackup)", File: "xlog.c", > Line: 10200) > > The triggering statements always contain a call to pg_start_backup w

Re: [HACKERS] Reviewing freeze map code

2016-08-04 Thread Andres Freund
Hi, On 2016-08-02 10:55:18 -0400, Noah Misch wrote: > [Action required within 72 hours. This is a generic notification.] > > The above-described topic is currently a PostgreSQL 9.6 open item. Andres, > since you committed the patch believed to have created it, you own this open > item. Well ki

Re: [HACKERS] [sqlsmith] FailedAssertion("!(XLogCtl->Insert.exclusiveBackup)", File: "xlog.c", Line: 10200)

2016-08-04 Thread Michael Paquier
On Thu, Aug 4, 2016 at 4:03 PM, Michael Paquier wrote: > This window exists as well on back-branches btw, this is not new to > 9.6. Magnus, what do you think? Actually, a completely water-proof solution is to use an in-memory flag proper to exclusive backups during the time pg_start_backup() is c

[HACKERS] [RFC] Change the default of update_process_title to off

2016-08-04 Thread Tsunakawa, Takayuki
Hello, I'd like to propose changing the default value of update_process_title to off, at least on Windows. I'll submit a patch if we see no big problem. PROBLEM Our customer is trying to certify PostgreSQL with their packaged software product. Curren

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Craig Ringer
On 4 August 2016 at 12:45, Tom Lane wrote: > Craig Ringer writes: > > On 4 August 2016 at 02:15, Tom Lane wrote: > >> So it seems like fixing libpq's parsing of server_version_num is > >> something we definitely want to fix ASAP in all back branches. > > > Well, this seems like a good time to m

Re: [HACKERS] [RFC] Change the default of update_process_title to off

2016-08-04 Thread Michael Paquier
On Thu, Aug 4, 2016 at 4:41 PM, Tsunakawa, Takayuki wrote: > 1. The performance gain is huge. > 2. It's almost useless because we can only see the postgres command line with > Process Explorer, which the user must download from Microsoft and install. > 3. I don't see the benefit of update_process

Re: [HACKERS] Oddity in EXPLAIN for foreign/custom join pushdown plans

2016-08-04 Thread Etsuro Fujita
On 2016/08/02 22:02, Kouhei Kaigai wrote: I wrote: I removed the Relations line. Here is an updated version of the patch. * As I said upthread, I left the upper-relation handling for another patch. Currently, the patch prints "Foreign Scan" with no relations in that case. * I kept custom joi

Re: [HACKERS] [RFC] Change the default of update_process_title to off

2016-08-04 Thread Andres Freund
On 2016-08-04 16:48:11 +0900, Michael Paquier wrote: > Here is a different proposal: documenting instead that disabling that > parameter on Windows can improve performance, at the cost of losing > information verbosity for processes. The benefit on windows seems pretty marginal, given the way it h

Re: [HACKERS] Leaking memory in text_overlay function

2016-08-04 Thread Anastasia Lubennikova
I found this thread on the open CF. As I see, the discussion is ended with decision to reject patch. I've just changed the status to "Rejected". 02.07.2016 18:11, Dirk Rudolph: Well that's good to know. It was just curious that, in my case, I only saw this in this particular function. Anywa

Re: [HACKERS] Why we lost Uber as a user

2016-08-04 Thread Torsten Zuehlsdorff
On 03.08.2016 21:05, Robert Haas wrote: On Wed, Aug 3, 2016 at 2:23 PM, Tom Lane wrote: Robert Haas writes: I don't think they are saying that logical replication is more reliable than physical replication, nor do I believe that to be true. I think they are saying that if logical corruption

Re: [HACKERS] Oddity in EXPLAIN for foreign/custom join pushdown plans

2016-08-04 Thread Kouhei Kaigai
> -Original Message- > From: Etsuro Fujita [mailto:fujita.ets...@lab.ntt.co.jp] > Sent: Thursday, August 04, 2016 4:42 PM > To: Kaigai Kouhei(海外 浩平); pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] Oddity in EXPLAIN for foreign/custom join pushdown > plans > > On 2016/08/02 22:02, K

Re: [HACKERS] Oddity in EXPLAIN for foreign/custom join pushdown plans

2016-08-04 Thread Etsuro Fujita
On 2016/08/04 18:03, Kouhei Kaigai wrote: Kaigai-san wrote: Also, the logic to print "Foreign (Scan|Insert|Update|Delete)" is different from what I suggested. I'm suggesting to allow extension giving a label to fill up "Foreign %s" format. Please explain why your choice is better than my propos

[HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Pavan Deolasee
Write Amplification Reduction Method (WARM) A few years back, we developed HOT to address the problem associated with MVCC and frequent updates and it has served us very well. But in the light of Uber's recent technical blog highlighting some of the problems th

[HACKERS] Unused argument in PinBuffer function

2016-08-04 Thread Ildar Musin
Hi all, I was looking through the buffer manager's code and have noticed that function PinBuffer has an unused 'strategy' argument. It's seems that after refactoring made by Alexander Korotkov and Andres Freund (48354581a49c30f5757c203415aa8412d85b0f70) it became useless. The attached patch r

[HACKERS] Bug in WaitForBackgroundWorkerShutdown() [REL9_5_STABLE]

2016-08-04 Thread Dmitry Ivanov
Recently I've encountered a strange crash while calling elog(ERROR, "...") after the WaitForBackgroundWorkerShutdown() function. It turns out that _returns_ inside the PG_TRY()..PG_CATCH() block are *highly* undesirable, since they leave PG_exception_stack pointing to a local struct in a dead f

Re: [HACKERS] Bug in WaitForBackgroundWorkerShutdown() [REL9_5_STABLE]

2016-08-04 Thread Dmitry Ivanov
> Recently I've encountered a strange crash while calling elog(ERROR, "...") > after the WaitForBackgroundWorkerShutdown() function. It turns out that > _returns_ inside the PG_TRY()..PG_CATCH() block are *highly* undesirable, > since they leave PG_exception_stack pointing to a local struct in a de

Re: [HACKERS] Bug in WaitForBackgroundWorkerShutdown() [REL9_5_STABLE]

2016-08-04 Thread Yury Zhuravlev
Dmitry Ivanov wrote: Recently I've encountered a strange crash while calling elog(ERROR, "...") after the WaitForBackgroundWorkerShutdown() function. It turns out that _returns_ inside the PG_TRY()..PG_CATCH() block are *highly* undesirable, since they leave PG_exception_stack pointing to a local

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-04 Thread Aleksander Alekseev
Thanks everyone for your remarks and comments! Here is an improved version of a patch. Main changes: * Patch passes `make installcheck` * Code is fully commented, also no more TODO's I wish I sent this version of a patch last time. Now I realize it was really hard to read and understand. Hope I

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-04 Thread Geoff Winkless
On 30 July 2016 at 13:42, Tomas Vondra wrote: > I'd argue that if you mess with catalogs directly, you're on your own. Interesting. What would you suggest people use instead? Geoff -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] max_parallel_degree > 0 for 9.6 beta

2016-08-04 Thread Robert Haas
On Thu, Aug 4, 2016 at 12:56 AM, Tom Lane wrote: > Noah Misch writes: >> On Wed, Apr 20, 2016 at 02:28:15PM -0400, Tom Lane wrote: >>> +1, but let's put an entry on the 9.6 open-items page to remind us to >>> make that decision at the right time. > >> It's that time. Do we restore the max_parall

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-08-04 Thread Amit Kapila
On Wed, Aug 3, 2016 at 5:09 PM, Pavel Stehule wrote: > > > 2016-08-03 12:16 GMT+02:00 Rahila Syed : >> >> Should changing the value from OFF to ON automatically either commit or >> rollback transaction in progress? >> >> >> FWIW, running set autocommit through ecpg commits the ongoing transaction

Re: [HACKERS] handling unconvertible error messages

2016-08-04 Thread Tom Lane
Victor Wagner writes: > If error occurs during processing of StartMessage protocol message, > i.e. client request connection to unexisting database, > ErrorResponse would contain message in the server default locale, > despite of client encoding being specified in the StartMessage. Yeah. I'm inc

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Tom Lane
Craig Ringer writes: > On 4 August 2016 at 12:45, Tom Lane wrote: >> Craig Ringer writes: >>> Well, this seems like a good time to make server_version_num GUC_REPORT >>> as well... >> To what end? Existing versions of libpq wouldn't know about it, and new >> versions of libpq couldn't rely on

Re: [HACKERS] [RFC] Change the default of update_process_title to off

2016-08-04 Thread Tom Lane
Andres Freund writes: > On 2016-08-04 16:48:11 +0900, Michael Paquier wrote: >> Here is a different proposal: documenting instead that disabling that >> parameter on Windows can improve performance, at the cost of losing >> information verbosity for processes. > The benefit on windows seems prett

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Vladimir Sitnikov
> > Sorry, but I don't buy that. I think sending both server_version and > server_version_num would be silly, and we're certainly not going to stop > sending server_version. > What is wrong with sending machine-readable value? Vladimir

[HACKERS] "Some tests to cover hash_index"

2016-08-04 Thread Mithun Cy
I am attaching the patch to improve some coverage of hash index code [1]. I have added some basic tests, which mainly covers overflow pages. It took 2 sec extra time in my machine in parallel schedule. Hit Total Coverage old tests Line Coverage 780 1478 52.7 Function Coverage 63 85 74.1 improv

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Tom Lane
Vladimir Sitnikov writes: >> Sorry, but I don't buy that. I think sending both server_version and >> server_version_num would be silly, and we're certainly not going to stop >> sending server_version. > What is wrong with sending machine-readable value? [ shrug... ] What do you claim is not ma

Re: [HACKERS] max_parallel_degree > 0 for 9.6 beta

2016-08-04 Thread David G. Johnston
On Thu, Aug 4, 2016 at 9:25 AM, Robert Haas wrote: > On Thu, Aug 4, 2016 at 12:56 AM, Tom Lane wrote: > > Noah Misch writes: > >> On Wed, Apr 20, 2016 at 02:28:15PM -0400, Tom Lane wrote: > >>> +1, but let's put an entry on the 9.6 open-items page to remind us to > >>> make that decision at the

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-08-04 Thread Pavel Stehule
2016-08-04 15:37 GMT+02:00 Amit Kapila : > On Wed, Aug 3, 2016 at 5:09 PM, Pavel Stehule > wrote: > > > > > > 2016-08-03 12:16 GMT+02:00 Rahila Syed : > >> > >> Should changing the value from OFF to ON automatically either commit or > >> rollback transaction in progress? > >> > >> > >> FWIW, runn

Re: [HACKERS] Hash Indexes

2016-08-04 Thread Mithun Cy
I did some basic testing of same. In that I found one issue with cursor. +BEGIN; +SET enable_seqscan = OFF; +SET enable_bitmapscan = OFF; +CREATE FUNCTION declares_cursor(int) + RETURNS void + AS 'DECLARE c CURSOR FOR SELECT * from con_hash_index_table WHERE keycol = $1;' +LANGUAGE SQL; +

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Vladimir Sitnikov
Tom Lane : > [ shrug... ] What do you claim is not machine-readable about > server_version? > 0) server_version needs a dance to parse. For instance, recent "Stamp version 10devel" patch did touch "server_version" parsing in fe-exec.c: https://github.com/vlsi/postgres/pull/2/files#diff-2df5cad06

Re: [HACKERS] handling unconvertible error messages

2016-08-04 Thread Victor Wagner
On Thu, 04 Aug 2016 09:42:10 -0400 Tom Lane wrote: > Victor Wagner writes: > > If error occurs during processing of StartMessage protocol message, > > i.e. client request connection to unexisting database, > > ErrorResponse would contain message in the server default locale, > > despite of clien

Re: [HACKERS] regression test for extended query protocol

2016-08-04 Thread Alvaro Herrera
Michael Paquier wrote: > On Thu, Aug 4, 2016 at 12:14 AM, Dave Cramer wrote: > > We currently run tests every time a PR is created on github, but I don't > > think there are any animals running the JDBC test suite > > > > We can add tests, what exactly do we want to test. Then setting up an animal

[HACKERS] improved DefElem list processing

2016-08-04 Thread Peter Eisentraut
Here are two WIP patches to improve the DefElem list processing that is used by many utility commands. One factors out the duplicate checks, which are currently taking up a lot of space with duplicate code. I haven't applied this everywhere yet, but the patch shows how much boring code can be sav

Re: [HACKERS] Design for In-Core Logical Replication

2016-08-04 Thread Simon Riggs
On 29 July 2016 at 16:53, Robert Haas wrote: > I think that to really understand exactly what you and Petr have in > mind, we'd need a description of where publication and subscription > data is stored within the server, and exactly what gets stored. > Perhaps that will come in a later email. I'

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Andres Freund
Hi, On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > Indexes whose values do not change do not require new index pointers. Only > the index whose key is being changed will need a new index entry. The new > index entry will be set to the CTID of the root line pointer. That seems to require tr

Re: [HACKERS] Why we lost Uber as a user

2016-08-04 Thread Alfred Perlstein
On 8/4/16 2:00 AM, Torsten Zuehlsdorff wrote: On 03.08.2016 21:05, Robert Haas wrote: On Wed, Aug 3, 2016 at 2:23 PM, Tom Lane wrote: Robert Haas writes: I don't think they are saying that logical replication is more reliable than physical replication, nor do I believe that to be true. I

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 09:31:18AM -0700, Andres Freund wrote: > Hi, > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > > Indexes whose values do not change do not require new index pointers. Only > > the index whose key is being changed will need a new index entry. The new > > index entry

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Simon Riggs
On 3 August 2016 at 20:37, Claudio Freire wrote: > On Wed, Aug 3, 2016 at 4:20 AM, Simon Riggs wrote: >> == IndexScan == >> >> Note that the executor code for IndexScan appears identical between >> the two optimizations. The difference between duplicate and range LITE >> tuples is needed only at

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Andres Freund
On 2016-08-04 12:55:25 -0400, Bruce Momjian wrote: > On Thu, Aug 4, 2016 at 09:31:18AM -0700, Andres Freund wrote: > > Hi, > > > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > > > Indexes whose values do not change do not require new index pointers. Only > > > the index whose key is bein

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Simon Riggs
On 4 August 2016 at 02:13, Bruce Momjian wrote: > How do plan to clear the bitmask so it, over time, doesn't end up being > all-set? I don't have a plan, though thoughts welcome. Similar situation that our current indexes don't recover from bloat, a situation made worse by non-HOT updates. So,

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 04:29:09PM +0530, Pavan Deolasee wrote: > Write Amplification Reduction Method (WARM) > > > A few years back, we developed HOT to address the problem associated with MVCC > and frequent updates and it has served us very well. But in the

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Simon Riggs
On 4 August 2016 at 17:31, Andres Freund wrote: > Hi, > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: >> Indexes whose values do not change do not require new index pointers. Only >> the index whose key is being changed will need a new index entry. The new >> index entry will be set to the

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 09:58:29AM -0700, Andres Freund wrote: > On 2016-08-04 12:55:25 -0400, Bruce Momjian wrote: > > On Thu, Aug 4, 2016 at 09:31:18AM -0700, Andres Freund wrote: > > > Hi, > > > > > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > > > > Indexes whose values do not chang

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 7:59 AM, Pavan Deolasee wrote: > We don’t want to force a recheck for every index fetch because that will > slow everything down. So we would like a simple and efficient way of knowing > about the existence of a WARM tuple in a chain and do a recheck in only > those cases, i

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Simon Riggs
On 4 August 2016 at 18:05, Bruce Momjian wrote: >> Approach 2 seems more reasonable and simple. >> >> There are only 2 bits for lp_flags and all combinations are already used. But >> for LP_REDIRECT root line pointer, we could use the lp_len field to store >> this >> special flag, which is not u

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 09:58:29AM -0700, Andres Freund wrote: > On 2016-08-04 12:55:25 -0400, Bruce Momjian wrote: > > On Thu, Aug 4, 2016 at 09:31:18AM -0700, Andres Freund wrote: > > > Hi, > > > > > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > > > > Indexes whose values do not chang

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Andres Freund
On 2016-08-04 18:11:17 +0100, Simon Riggs wrote: > On 4 August 2016 at 17:31, Andres Freund wrote: > > Hi, > > > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > >> Indexes whose values do not change do not require new index pointers. Only > >> the index whose key is being changed will need

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Simon Riggs
On 4 August 2016 at 18:17, Andres Freund wrote: > On 2016-08-04 18:11:17 +0100, Simon Riggs wrote: >> On 4 August 2016 at 17:31, Andres Freund wrote: >> > Hi, >> > >> > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: >> >> Indexes whose values do not change do not require new index pointers.

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 06:16:02PM +0100, Simon Riggs wrote: > On 4 August 2016 at 18:05, Bruce Momjian wrote: > > >> Approach 2 seems more reasonable and simple. > >> > >> There are only 2 bits for lp_flags and all combinations are already used. > >> But > >> for LP_REDIRECT root line pointer,

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Andres Freund
On 2016-08-04 18:18:47 +0100, Simon Riggs wrote: > On 4 August 2016 at 18:17, Andres Freund wrote: > > On 2016-08-04 18:11:17 +0100, Simon Riggs wrote: > > I'm less afraid of the fiddlyness of finding the root tuple, than the > > cost. It's not cheap to walk through, potentially, all hot chains to

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 06:03:34PM +0100, Simon Riggs wrote: > On 4 August 2016 at 02:13, Bruce Momjian wrote: > > > How do plan to clear the bitmask so it, over time, doesn't end up being > > all-set? > > I don't have a plan, though thoughts welcome. > > Similar situation that our current inde

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 1:58 PM, Simon Riggs wrote: > On 3 August 2016 at 20:37, Claudio Freire wrote: >> On Wed, Aug 3, 2016 at 4:20 AM, Simon Riggs wrote: >>> == IndexScan == >>> >>> Note that the executor code for IndexScan appears identical between >>> the two optimizations. The difference be

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 06:18:47PM +0100, Simon Riggs wrote: > > Have you considered what it'd take to allow index pointers to allow to > > point to "intermediate" root tuples? I.e. have some indexes point into > > the middle of an update-chain, whereas others still point to the > > beginning? Ther

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Pavan Deolasee
On Thu, Aug 4, 2016 at 10:41 PM, Simon Riggs wrote: > On 4 August 2016 at 17:31, Andres Freund wrote: > > Hi, > > > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > >> Indexes whose values do not change do not require new index pointers. > Only > >> the index whose key is being changed wi

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Pavan Deolasee
On Thu, Aug 4, 2016 at 10:49 PM, Bruce Momjian wrote: > On Thu, Aug 4, 2016 at 06:16:02PM +0100, Simon Riggs wrote: > > On 4 August 2016 at 18:05, Bruce Momjian wrote: > > > > >> Approach 2 seems more reasonable and simple. > > >> > > >> There are only 2 bits for lp_flags and all combinations a

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 2:14 PM, Claudio Freire wrote: > To fix this, what I was pondering, and I believe it would be viable, > is to teach heap_hot_search_buffer about the WARM invariant. It will > make correctness heavily depend on the invariant being true, which > means either flagging item poin

[HACKERS] psql: Missing option to print current buffer to current output channel (i.e., \qprint)

2016-08-04 Thread David G. Johnston
"\echo" has "\qecho" - I'm basically looking for a "\qprint" "\write" doesn't apply The following doesn't work either... #bash (stock Ubuntu 14.04) psql --set=query_in_var='SELECT version();' > /dev/null <<'SQL' \o /tmp/psql-test.txt \set ECHO queries --still ends up on stdout, hence the redirec

[HACKERS] Pgbench performance tuning?

2016-08-04 Thread Greg Stark
I'm trying to run pgbench on a moderately beefy machine (4-core 3.4GHz with 32G of ram and md mirrored spinning rust drives) at scale 300 with 32 clients with duration of 15min. I'm getting TPS numbers between 60-150 which seems surprisingly low to me and also to several people on IRC. Now pg_test

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 11:04:49PM +0530, Pavan Deolasee wrote: > If the root tuple still exists, we store the WARM flag (or > HEAP_RECHECK_REQUIRED as used in the original post) in the tuple header > itself. > When the root tuple becomes dead and HOT prune decides to replace it with a > LP_REDIRE

Re: [HACKERS] Pgbench performance tuning?

2016-08-04 Thread Andres Freund
On 2016-08-04 18:48:31 +0100, Greg Stark wrote: > I'm trying to run pgbench on a moderately beefy machine (4-core 3.4GHz > with 32G of ram and md mirrored spinning rust drives) at scale 300 > with 32 clients with duration of 15min. I'm getting TPS numbers > between 60-150 which seems surprisingly l

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 01:05:53PM -0400, Bruce Momjian wrote: > > Approach 2 seems more reasonable and simple.  > > > > There are only 2 bits for lp_flags and all combinations are already used. > > But > > for LP_REDIRECT root line pointer, we could use the lp_len field to store > > this > > sp

Re: [HACKERS] [RFC] Change the default of update_process_title to off

2016-08-04 Thread Robert Haas
On Thu, Aug 4, 2016 at 3:52 AM, Andres Freund wrote: > On 2016-08-04 16:48:11 +0900, Michael Paquier wrote: >> Here is a different proposal: documenting instead that disabling that >> parameter on Windows can improve performance, at the cost of losing >> information verbosity for processes. > > Th

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 02:35:32PM -0300, Claudio Freire wrote: > > Vacuum will need to be taught not to break the invariant, but I > > believe this is viable and efficient. > > > And I didn't mention the invariant. > > The invariant would be that all WARM chains: > > * contain entire HOT chai

Re: [HACKERS] improved DefElem list processing

2016-08-04 Thread Tom Lane
Peter Eisentraut writes: > Here are two WIP patches to improve the DefElem list processing that is > used by many utility commands. > One factors out the duplicate checks, which are currently taking up a > lot of space with duplicate code. I haven't applied this everywhere > yet, but the patch s

Re: [HACKERS] Double invocation of InitPostmasterChild in bgworker with -DEXEC_BACKEND

2016-08-04 Thread Robert Haas
On Tue, Aug 2, 2016 at 6:40 PM, Tom Lane wrote: > Thomas Munro writes: >> I discovered that if you build with -DEXEC_BACKEND on a Unix system >> and then try to start a background worker, it dies in >> InitializeLatchSupport: > >> TRAP: FailedAssertion("!(selfpipe_readfd == -1)", File: "latch.c",

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Simon Riggs
On 4 August 2016 at 18:27, Bruce Momjian wrote: >> > Also, why not use this bitmap for all indexes, not just update chains? >> >> I don't understand where you get this update chains thing from. >> >> The bitmap can apply to multiple tuples on one page, which is described. > > I am asking if we co

Re: [HACKERS] Pgbench performance tuning?

2016-08-04 Thread Andres Freund
On 2016-08-04 19:15:43 +0100, Greg Stark wrote: > On Thu, Aug 4, 2016 at 6:53 PM, Andres Freund wrote: > > What's the config? Version? What activity does pidstat -d -l indicate? > > How much WAL was generated? > > I know the specifics matter but I was also trying to avoid dumping too > much into

Re: [HACKERS] improved DefElem list processing

2016-08-04 Thread Tom Lane
I wrote: > Peter Eisentraut writes: >> The other adds a location field to the DefElem node. > +1 for sure, lots of places where that would be a good thing > (the duplicate check itself, for starters). Forgot to mention: seems like you should have added a location argument to makeDefElem.

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 3:07 PM, Bruce Momjian wrote: > On Thu, Aug 4, 2016 at 02:35:32PM -0300, Claudio Freire wrote: >> > Vacuum will need to be taught not to break the invariant, but I >> > believe this is viable and efficient. >> >> >> And I didn't mention the invariant. >> >> The invariant wo

Re: [HACKERS] handling unconvertible error messages

2016-08-04 Thread Tom Lane
Victor Wagner writes: > On Thu, 04 Aug 2016 09:42:10 -0400 > Tom Lane wrote: >> Yeah. I'm inclined to think that we should reset the message locale >> to C as soon as we've forked away from the postmaster, and leave it >> that way until we've absorbed settings from the startup packet. > Really,

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 3:23 PM, Claudio Freire wrote: >> That is a new HOT chain given our current code, but >> would the new tuple addition realize it needs to create a new index >> tuple? The new tuple code needs to check the index's root HOT tid for a >> non-match. > > I'm not proposing to cha

Re: [HACKERS] Pgbench performance tuning?

2016-08-04 Thread Greg Stark
On Thu, Aug 4, 2016 at 6:53 PM, Andres Freund wrote: > What's the config? Version? What activity does pidstat -d -l indicate? > How much WAL was generated? I know the specifics matter but I was also trying to avoid dumping too much into the email. The shared buffers is set to 16384 (128MB). Othe

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Peter Eisentraut
On 8/3/16 1:20 PM, Tom Lane wrote: > I was just wondering whether it would be worth starting to use "git tag -a". One should always use annotated tags for public releases. That way, the tag is its own Git object that cannot be later moved around or easily faked (modulo GPG signatures -- a whole d

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Robert Haas
On Thu, Aug 4, 2016 at 9:57 AM, Tom Lane wrote: > Vladimir Sitnikov writes: >>> Sorry, but I don't buy that. I think sending both server_version and >>> server_version_num would be silly, and we're certainly not going to stop >>> sending server_version. > >> What is wrong with sending machine-re

Re: [HACKERS] Keeping CURRENT_DATE and similar constructs in original format

2016-08-04 Thread Peter Eisentraut
On 5/12/16 6:14 PM, Tom Lane wrote: > So what I've wanted to do for some time is invent a new expression node > type that represents any one of these functions and can be reverse-listed > in the same format that the input had. The attached proposed patch does > that. I was experimenting with this

Re: [HACKERS] Bug in WaitForBackgroundWorkerShutdown() [REL9_5_STABLE]

2016-08-04 Thread Tom Lane
Yury Zhuravlev writes: > Dmitry Ivanov wrote: >>> Recently I've encountered a strange crash while calling elog(ERROR, "...") >>> after the WaitForBackgroundWorkerShutdown() function. It turns out that >>> _returns_ inside the PG_TRY()..PG_CATCH() block are *highly* undesirable, >>> since they leav

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Tom Lane
Robert Haas writes: > On Thu, Aug 4, 2016 at 9:57 AM, Tom Lane wrote: >> [ shrug... ] What do you claim is not machine-readable about >> server_version? > Surely you can't have missed the connection between the issue at hand > and what Craig is talking about. If libpq were using the > machine-

Re: [HACKERS] Fwd: [BUGS] BUG #14247: COMMENT is restored on wrong database

2016-08-04 Thread Robert Haas
On Tue, Aug 2, 2016 at 5:42 PM, David G. Johnston wrote: > Moving to -hackers since this is getting into details > > Bug Report # 14247 > > On Tue, Aug 2, 2016 at 4:58 PM, Tom Lane wrote: >> >> "David G. Johnston" writes: >> > Do you have an opinion on this following? >> >> I think the real prob

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Tom Lane
Peter Eisentraut writes: > On 8/3/16 1:20 PM, Tom Lane wrote: >> I was just wondering whether it would be worth starting to use "git tag -a". > One should always use annotated tags for public releases. That way, the > tag is its own Git object that cannot be later moved around or easily > faked

Re: [HACKERS] [Patch] RBTree iteration interface improvement

2016-08-04 Thread Robert Haas
On Thu, Jul 28, 2016 at 1:57 PM, Tom Lane wrote: > Aleksander Alekseev writes: >>> Can you explain use case where you need it? > >> ... Or maybe you have different objects, e.g. IndexScanDesc's, that should >> iterate over some tree's independently somewhere in indexam.c >> procedures. Exact orde

Re: [HACKERS] Fwd: [BUGS] BUG #14247: COMMENT is restored on wrong database

2016-08-04 Thread Tom Lane
Robert Haas writes: > On Tue, Aug 2, 2016 at 5:42 PM, David G. Johnston > wrote: > The fact that pg_dump is emitting COMMENT ON DATABASE at all is > fundamentally wrong given the existing division-of-labor decisions, > namely that pg_dump is responsible for objects within a database > not for dat

Re: [HACKERS] Fwd: [BUGS] BUG #14247: COMMENT is restored on wrong database

2016-08-04 Thread David G. Johnston
On Thu, Aug 4, 2016 at 4:50 PM, Tom Lane wrote: > Robert Haas writes: > > On Tue, Aug 2, 2016 at 5:42 PM, David G. Johnston > > wrote: > > The fact that pg_dump is emitting COMMENT ON DATABASE at all is > > fundamentally wrong given the existing division-of-labor decisions, > > namely that pg_d

Re: [HACKERS] Pgbench performance tuning?

2016-08-04 Thread Jeff Janes
On Thu, Aug 4, 2016 at 10:48 AM, Greg Stark wrote: > I'm trying to run pgbench on a moderately beefy machine (4-core 3.4GHz > with 32G of ram and md mirrored spinning rust drives) at scale 300 > with 32 clients with duration of 15min. I'm getting TPS numbers > between 60-150 which seems surprising

[HACKERS] Oddity with NOT IN

2016-08-04 Thread Jim Nasby
I've got a customer that discovered something odd... SELECT f1 FROM v1 WHERE f2 not in (SELECT bad FROM v2 WHERE f3 = 1); does not error, even though bad doesn't exist, but SELECT bad FROM v2 WHERE f3 = 1; gives ERROR: column "bad" does not exist Is that expected? This is on 9.4.8, and both

Re: [HACKERS] Oddity with NOT IN

2016-08-04 Thread Marko Tiikkaja
On 2016-08-04 11:23 PM, Jim Nasby wrote: I've got a customer that discovered something odd... SELECT f1 FROM v1 WHERE f2 not in (SELECT bad FROM v2 WHERE f3 = 1); does not error, even though bad doesn't exist, but I'm guessing there's a v1.bad? This is a common mistake, and also why I recomm

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 03:23:10PM -0300, Claudio Freire wrote: > INSERT INTO TABLE t (id,val,dat) VALUES (32,'a','blabla'); > UPDATE t SET dat='blabla2' where id = 32; > UPDATE t SET dat='blabla3', id=33 where id = 32; > UPDATE t SET val='b' where id = 33; > UPDATE t SET dat='blabla4' where id = 3

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Andrew Dunstan
On 08/01/2016 04:25 PM, Tom Lane wrote: Andrew Dunstan writes: Somewhat related is how we name the git branches. It would help me from a buildfarm POV if we kept lexically them sortable, which could be done at least for the next 90 major releases :-) by adding an underscore after the REL piec

[HACKERS] Bogus ANALYZE results for an otherwise-unique column with many nulls

2016-08-04 Thread Tom Lane
I looked into the problem described at https://www.postgresql.org/message-id/flat/VisenaEmail.26.df42f82acae38a58.156463942b8%40tc7-visena and I believe I've reproduced it: the requirement is that the inner join column for the antijoin must contain a lot of NULL values, and what isn't NULL must be

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 03:33:16PM -0300, Claudio Freire wrote: > On Thu, Aug 4, 2016 at 3:23 PM, Claudio Freire wrote: > >> That is a new HOT chain given our current code, but > >> would the new tuple addition realize it needs to create a new index > >> tuple? The new tuple code needs to check t

Re: [HACKERS] Bogus ANALYZE results for an otherwise-unique column with many nulls

2016-08-04 Thread Tom Lane
I wrote: > Looking around, there are a couple of places outside commands/analyze.c > that are making the same mistake, so this patch isn't complete, but it > illustrates what needs to be done. Here's a more complete patch. regards, tom lane diff --git a/src/backend/comman

Re: [HACKERS] Possible duplicate release of buffer lock.

2016-08-04 Thread Peter Geoghegan
On Wed, Aug 3, 2016 at 1:31 AM, Kyotaro HORIGUCHI wrote: > The first line is emitted for simultaneous deletion of a index > page, which is impossible by design in a consistent state so the > complained situation should be the result of an index corruption > before upgading, specifically, inconsist

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 10:58:33PM +0530, Pavan Deolasee wrote: > 2. I also think that most often the tuple that will be updated will have its > t_ctid pointing to itself. This sounds more invasive, but can we somehow Agreed! I think it has to. Great idea. > utilise the t_ctid field to store th

Re: [HACKERS] Fwd: [BUGS] BUG #14247: COMMENT is restored on wrong database

2016-08-04 Thread Craig Ringer
On 5 August 2016 at 05:03, David G. Johnston wrote: > ​ > I'm all for an elegant solution here though at some point having a working > solution now beats waiting for someone to willingly dive more deeply into > pg_dump. I too seem to recall previous proposals for COMMON ON CURRENT > DATABASE ye

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Craig Ringer
On 5 August 2016 at 04:31, Tom Lane wrote: > In short: if we'd done it that way all along, it would've been nice. But encouraging people to change horses now isn't really going to > make things better. What is far more likely to happen, if we were > to do that, is that people will make client

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 7:21 PM, Bruce Momjian wrote: > On Thu, Aug 4, 2016 at 03:23:10PM -0300, Claudio Freire wrote: >> INSERT INTO TABLE t (id,val,dat) VALUES (32,'a','blabla'); >> UPDATE t SET dat='blabla2' where id = 32; >> UPDATE t SET dat='blabla3', id=33 where id = 32; >> UPDATE t SET val=

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
>On Thu, Aug 4, 2016 at 06:21:48PM -0400, Bruce Momjian wrote: > Another approach would be to keep updates on the same page in a single > WARM chain, even if all indexes have changed columns. We will already > have code to walk the HOT chain looking for matches, and at most one > tuple in the upd

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 7:59 AM, Pavan Deolasee wrote: > So marking the index would require us to mark both old and new index tuples > as requiring re-check. That requires an additional index scan to locate the > old row and then an additional write to force it to re-check, which is > algorithmical

Re: [HACKERS] max_parallel_degree > 0 for 9.6 beta

2016-08-04 Thread Michael Paquier
On Thu, Aug 4, 2016 at 10:52 PM, David G. Johnston wrote: > My initial reaction was +1 but now I'm leaning toward enabled by default. > > Those who would upgrade to 9.6 within a year of its release are most likely, > process and personality wise, to be those for whom the risks and rewards of > new

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 09:53:31PM -0300, Claudio Freire wrote: > It's the other way around. A WARM chain has to span whole HOT chains. > The WARM chain always begins in the head of a HOT chain (or non-HOT > tuple of course), and cannot end before the HOT chain ends. > > That is, all within a sing

  1   2   >