Re: [HACKERS] handling unconvertible error messages

2016-08-04 Thread Victor Wagner
On Mon, 25 Jul 2016 10:43:44 -0400 Peter Eisentraut wrote: > Example: I have a database cluster initialized with > --locale=ru_RU.UTF-8 (built with NLS). Let's say for some reason, I > have client encoding set to LATIN1. All error messages come back > like

Re: [HACKERS] [sqlsmith] FailedAssertion("!(XLogCtl->Insert.exclusiveBackup)", File: "xlog.c", Line: 10200)

2016-08-04 Thread Michael Paquier
On Thu, Aug 4, 2016 at 2:19 AM, Andreas Seltenreich wrote: > testing with sqlsmith shows that the following assertion doesn't hold: > > FailedAssertion("!(XLogCtl->Insert.exclusiveBackup)", File: "xlog.c", > Line: 10200) > > The triggering statements always contain a call

Re: [HACKERS] handling unconvertible error messages

2016-08-04 Thread Victor Wagner
On Mon, 25 Jul 2016 10:43:44 -0400 Peter Eisentraut wrote: e is > a real error message, but it could not be converted. > > I think ideally we could make this better in two ways: > > 1) Send the original error message untranslated. That would require > saving

Re: [HACKERS] Bug in WaitForBackgroundWorkerShutdown() [REL9_5_STABLE]

2016-08-04 Thread Yury Zhuravlev
Dmitry Ivanov wrote: Recently I've encountered a strange crash while calling elog(ERROR, "...") after the WaitForBackgroundWorkerShutdown() function. It turns out that _returns_ inside the PG_TRY()..PG_CATCH() block are *highly* undesirable, since they leave PG_exception_stack pointing to a

[HACKERS] Bug in WaitForBackgroundWorkerShutdown() [REL9_5_STABLE]

2016-08-04 Thread Dmitry Ivanov
Recently I've encountered a strange crash while calling elog(ERROR, "...") after the WaitForBackgroundWorkerShutdown() function. It turns out that _returns_ inside the PG_TRY()..PG_CATCH() block are *highly* undesirable, since they leave PG_exception_stack pointing to a local struct in a dead

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-04 Thread Aleksander Alekseev
Thanks everyone for your remarks and comments! Here is an improved version of a patch. Main changes: * Patch passes `make installcheck` * Code is fully commented, also no more TODO's I wish I sent this version of a patch last time. Now I realize it was really hard to read and understand. Hope I

[HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Pavan Deolasee
Write Amplification Reduction Method (WARM) A few years back, we developed HOT to address the problem associated with MVCC and frequent updates and it has served us very well. But in the light of Uber's recent technical blog highlighting some of the problems

Re: [HACKERS] [Patch] Temporary tables that do not bloat pg_catalog (a.k.a fast temp tables)

2016-08-04 Thread Geoff Winkless
On 30 July 2016 at 13:42, Tomas Vondra wrote: > I'd argue that if you mess with catalogs directly, you're on your own. Interesting. What would you suggest people use instead? Geoff -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Oddity in EXPLAIN for foreign/custom join pushdown plans

2016-08-04 Thread Etsuro Fujita
On 2016/08/02 22:02, Kouhei Kaigai wrote: I wrote: I removed the Relations line. Here is an updated version of the patch. * As I said upthread, I left the upper-relation handling for another patch. Currently, the patch prints "Foreign Scan" with no relations in that case. * I kept custom

Re: [HACKERS] [RFC] Change the default of update_process_title to off

2016-08-04 Thread Michael Paquier
On Thu, Aug 4, 2016 at 4:41 PM, Tsunakawa, Takayuki wrote: > 1. The performance gain is huge. > 2. It's almost useless because we can only see the postgres command line with > Process Explorer, which the user must download from Microsoft and install. > 3. I don't

Re: [HACKERS] Bug in WaitForBackgroundWorkerShutdown() [REL9_5_STABLE]

2016-08-04 Thread Dmitry Ivanov
> Recently I've encountered a strange crash while calling elog(ERROR, "...") > after the WaitForBackgroundWorkerShutdown() function. It turns out that > _returns_ inside the PG_TRY()..PG_CATCH() block are *highly* undesirable, > since they leave PG_exception_stack pointing to a local struct in a

[HACKERS] Unused argument in PinBuffer function

2016-08-04 Thread Ildar Musin
Hi all, I was looking through the buffer manager's code and have noticed that function PinBuffer has an unused 'strategy' argument. It's seems that after refactoring made by Alexander Korotkov and Andres Freund (48354581a49c30f5757c203415aa8412d85b0f70) it became useless. The attached patch

Re: [HACKERS] Oddity in EXPLAIN for foreign/custom join pushdown plans

2016-08-04 Thread Kouhei Kaigai
> -Original Message- > From: Etsuro Fujita [mailto:fujita.ets...@lab.ntt.co.jp] > Sent: Thursday, August 04, 2016 4:42 PM > To: Kaigai Kouhei(海外 浩平); pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] Oddity in EXPLAIN for foreign/custom join pushdown > plans > > On 2016/08/02 22:02,

Re: [HACKERS] Reviewing freeze map code

2016-08-04 Thread Andres Freund
Hi, On 2016-08-02 10:55:18 -0400, Noah Misch wrote: > [Action required within 72 hours. This is a generic notification.] > > The above-described topic is currently a PostgreSQL 9.6 open item. Andres, > since you committed the patch believed to have created it, you own this open > item. Well

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Craig Ringer
On 4 August 2016 at 12:45, Tom Lane wrote: > Craig Ringer writes: > > On 4 August 2016 at 02:15, Tom Lane wrote: > >> So it seems like fixing libpq's parsing of server_version_num is > >> something we definitely want to fix ASAP in

[HACKERS] [RFC] Change the default of update_process_title to off

2016-08-04 Thread Tsunakawa, Takayuki
Hello, I'd like to propose changing the default value of update_process_title to off, at least on Windows. I'll submit a patch if we see no big problem. PROBLEM Our customer is trying to certify PostgreSQL with their packaged software product.

Re: [HACKERS] [RFC] Change the default of update_process_title to off

2016-08-04 Thread Andres Freund
On 2016-08-04 16:48:11 +0900, Michael Paquier wrote: > Here is a different proposal: documenting instead that disabling that > parameter on Windows can improve performance, at the cost of losing > information verbosity for processes. The benefit on windows seems pretty marginal, given the way it

Re: [HACKERS] Why we lost Uber as a user

2016-08-04 Thread Torsten Zuehlsdorff
On 03.08.2016 21:05, Robert Haas wrote: On Wed, Aug 3, 2016 at 2:23 PM, Tom Lane wrote: Robert Haas writes: I don't think they are saying that logical replication is more reliable than physical replication, nor do I believe that to be true. I

Re: [HACKERS] [sqlsmith] FailedAssertion("!(XLogCtl->Insert.exclusiveBackup)", File: "xlog.c", Line: 10200)

2016-08-04 Thread Michael Paquier
On Thu, Aug 4, 2016 at 4:03 PM, Michael Paquier wrote: > This window exists as well on back-branches btw, this is not new to > 9.6. Magnus, what do you think? Actually, a completely water-proof solution is to use an in-memory flag proper to exclusive backups during the

Re: [HACKERS] Leaking memory in text_overlay function

2016-08-04 Thread Anastasia Lubennikova
I found this thread on the open CF. As I see, the discussion is ended with decision to reject patch. I've just changed the status to "Rejected". 02.07.2016 18:11, Dirk Rudolph: Well that's good to know. It was just curious that, in my case, I only saw this in this particular function.

Re: [HACKERS] Oddity in EXPLAIN for foreign/custom join pushdown plans

2016-08-04 Thread Etsuro Fujita
On 2016/08/04 18:03, Kouhei Kaigai wrote: Kaigai-san wrote: Also, the logic to print "Foreign (Scan|Insert|Update|Delete)" is different from what I suggested. I'm suggesting to allow extension giving a label to fill up "Foreign %s" format. Please explain why your choice is better than my

Re: [HACKERS] max_parallel_degree > 0 for 9.6 beta

2016-08-04 Thread David G. Johnston
On Thu, Aug 4, 2016 at 9:25 AM, Robert Haas wrote: > On Thu, Aug 4, 2016 at 12:56 AM, Tom Lane wrote: > > Noah Misch writes: > >> On Wed, Apr 20, 2016 at 02:28:15PM -0400, Tom Lane wrote: > >>> +1, but let's put an entry on the 9.6

Re: [HACKERS] Hash Indexes

2016-08-04 Thread Mithun Cy
I did some basic testing of same. In that I found one issue with cursor. +BEGIN; +SET enable_seqscan = OFF; +SET enable_bitmapscan = OFF; +CREATE FUNCTION declares_cursor(int) + RETURNS void + AS 'DECLARE c CURSOR FOR SELECT * from con_hash_index_table WHERE keycol = $1;' +LANGUAGE SQL; +

Re: [HACKERS] [RFC] Change the default of update_process_title to off

2016-08-04 Thread Tom Lane
Andres Freund writes: > On 2016-08-04 16:48:11 +0900, Michael Paquier wrote: >> Here is a different proposal: documenting instead that disabling that >> parameter on Windows can improve performance, at the cost of losing >> information verbosity for processes. > The benefit

[HACKERS] "Some tests to cover hash_index"

2016-08-04 Thread Mithun Cy
I am attaching the patch to improve some coverage of hash index code [1]. I have added some basic tests, which mainly covers overflow pages. It took 2 sec extra time in my machine in parallel schedule. Hit Total Coverage old tests Line Coverage 780 1478 52.7 Function Coverage 63 85 74.1

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Tom Lane
Craig Ringer writes: > On 4 August 2016 at 12:45, Tom Lane wrote: >> Craig Ringer writes: >>> Well, this seems like a good time to make server_version_num GUC_REPORT >>> as well... >> To what end? Existing versions of libpq

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Tom Lane
Vladimir Sitnikov writes: >> Sorry, but I don't buy that. I think sending both server_version and >> server_version_num would be silly, and we're certainly not going to stop >> sending server_version. > What is wrong with sending machine-readable value? [ shrug...

Re: [HACKERS] max_parallel_degree > 0 for 9.6 beta

2016-08-04 Thread Robert Haas
On Thu, Aug 4, 2016 at 12:56 AM, Tom Lane wrote: > Noah Misch writes: >> On Wed, Apr 20, 2016 at 02:28:15PM -0400, Tom Lane wrote: >>> +1, but let's put an entry on the 9.6 open-items page to remind us to >>> make that decision at the right time. > >> It's

Re: [HACKERS] handling unconvertible error messages

2016-08-04 Thread Tom Lane
Victor Wagner writes: > If error occurs during processing of StartMessage protocol message, > i.e. client request connection to unexisting database, > ErrorResponse would contain message in the server default locale, > despite of client encoding being specified in the

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Vladimir Sitnikov
> > Sorry, but I don't buy that. I think sending both server_version and > server_version_num would be silly, and we're certainly not going to stop > sending server_version. > What is wrong with sending machine-readable value? Vladimir

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-08-04 Thread Pavel Stehule
2016-08-04 15:37 GMT+02:00 Amit Kapila : > On Wed, Aug 3, 2016 at 5:09 PM, Pavel Stehule > wrote: > > > > > > 2016-08-03 12:16 GMT+02:00 Rahila Syed : > >> > >> Should changing the value from OFF to ON automatically either

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-08-04 Thread Amit Kapila
On Wed, Aug 3, 2016 at 5:09 PM, Pavel Stehule wrote: > > > 2016-08-03 12:16 GMT+02:00 Rahila Syed : >> >> Should changing the value from OFF to ON automatically either commit or >> rollback transaction in progress? >> >> >> FWIW, running set

Re: [HACKERS] handling unconvertible error messages

2016-08-04 Thread Victor Wagner
On Thu, 04 Aug 2016 09:42:10 -0400 Tom Lane wrote: > Victor Wagner writes: > > If error occurs during processing of StartMessage protocol message, > > i.e. client request connection to unexisting database, > > ErrorResponse would contain message in the

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Vladimir Sitnikov
Tom Lane : > [ shrug... ] What do you claim is not machine-readable about > server_version? > 0) server_version needs a dance to parse. For instance, recent "Stamp version 10devel" patch did touch "server_version" parsing in fe-exec.c:

Re: [HACKERS] regression test for extended query protocol

2016-08-04 Thread Alvaro Herrera
Michael Paquier wrote: > On Thu, Aug 4, 2016 at 12:14 AM, Dave Cramer wrote: > > We currently run tests every time a PR is created on github, but I don't > > think there are any animals running the JDBC test suite > > > > We can add tests, what exactly do we want to test. Then

Re: [HACKERS] Design for In-Core Logical Replication

2016-08-04 Thread Simon Riggs
On 29 July 2016 at 16:53, Robert Haas wrote: > I think that to really understand exactly what you and Petr have in > mind, we'd need a description of where publication and subscription > data is stored within the server, and exactly what gets stored. > Perhaps that will

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Andres Freund
Hi, On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > Indexes whose values do not change do not require new index pointers. Only > the index whose key is being changed will need a new index entry. The new > index entry will be set to the CTID of the root line pointer. That seems to require

Re: [HACKERS] Keeping CURRENT_DATE and similar constructs in original format

2016-08-04 Thread Peter Eisentraut
On 5/12/16 6:14 PM, Tom Lane wrote: > So what I've wanted to do for some time is invent a new expression node > type that represents any one of these functions and can be reverse-listed > in the same format that the input had. The attached proposed patch does > that. I was experimenting with

Re: [HACKERS] Fwd: [BUGS] BUG #14247: COMMENT is restored on wrong database

2016-08-04 Thread Tom Lane
Robert Haas writes: > On Tue, Aug 2, 2016 at 5:42 PM, David G. Johnston > wrote: > The fact that pg_dump is emitting COMMENT ON DATABASE at all is > fundamentally wrong given the existing division-of-labor decisions, > namely that pg_dump is

Re: [HACKERS] Bug in WaitForBackgroundWorkerShutdown() [REL9_5_STABLE]

2016-08-04 Thread Tom Lane
Yury Zhuravlev writes: > Dmitry Ivanov wrote: >>> Recently I've encountered a strange crash while calling elog(ERROR, "...") >>> after the WaitForBackgroundWorkerShutdown() function. It turns out that >>> _returns_ inside the PG_TRY()..PG_CATCH() block are *highly*

Re: [HACKERS] Pgbench performance tuning?

2016-08-04 Thread Jeff Janes
On Thu, Aug 4, 2016 at 10:48 AM, Greg Stark wrote: > I'm trying to run pgbench on a moderately beefy machine (4-core 3.4GHz > with 32G of ram and md mirrored spinning rust drives) at scale 300 > with 32 clients with duration of 15min. I'm getting TPS numbers > between 60-150 which

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 10:58:33PM +0530, Pavan Deolasee wrote: > 2. I also think that most often the tuple that will be updated will have its > t_ctid pointing to itself. This sounds more invasive, but can we somehow Agreed! I think it has to. Great idea. > utilise the t_ctid field to store

[HACKERS] Oddity with NOT IN

2016-08-04 Thread Jim Nasby
I've got a customer that discovered something odd... SELECT f1 FROM v1 WHERE f2 not in (SELECT bad FROM v2 WHERE f3 = 1); does not error, even though bad doesn't exist, but SELECT bad FROM v2 WHERE f3 = 1; gives ERROR: column "bad" does not exist Is that expected? This is on 9.4.8, and

Re: [HACKERS] Oddity with NOT IN

2016-08-04 Thread Marko Tiikkaja
On 2016-08-04 11:23 PM, Jim Nasby wrote: I've got a customer that discovered something odd... SELECT f1 FROM v1 WHERE f2 not in (SELECT bad FROM v2 WHERE f3 = 1); does not error, even though bad doesn't exist, but I'm guessing there's a v1.bad? This is a common mistake, and also why I

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Tom Lane
Peter Eisentraut writes: > On 8/3/16 1:20 PM, Tom Lane wrote: >> I was just wondering whether it would be worth starting to use "git tag -a". > One should always use annotated tags for public releases. That way, the > tag is its own Git object that cannot be

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Andrew Dunstan
On 08/01/2016 04:25 PM, Tom Lane wrote: Andrew Dunstan writes: Somewhat related is how we name the git branches. It would help me from a buildfarm POV if we kept lexically them sortable, which could be done at least for the next 90 major releases :-) by adding an

Re: [HACKERS] Possible duplicate release of buffer lock.

2016-08-04 Thread Peter Geoghegan
On Wed, Aug 3, 2016 at 1:31 AM, Kyotaro HORIGUCHI wrote: > The first line is emitted for simultaneous deletion of a index > page, which is impossible by design in a consistent state so the > complained situation should be the result of an index corruption > before

Re: [HACKERS] [Patch] RBTree iteration interface improvement

2016-08-04 Thread Robert Haas
On Thu, Jul 28, 2016 at 1:57 PM, Tom Lane wrote: > Aleksander Alekseev writes: >>> Can you explain use case where you need it? > >> ... Or maybe you have different objects, e.g. IndexScanDesc's, that should >> iterate over some tree's independently

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 03:33:16PM -0300, Claudio Freire wrote: > On Thu, Aug 4, 2016 at 3:23 PM, Claudio Freire wrote: > >> That is a new HOT chain given our current code, but > >> would the new tuple addition realize it needs to create a new index > >> tuple? The new

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Robert Haas
On Thu, Aug 4, 2016 at 9:57 AM, Tom Lane wrote: > Vladimir Sitnikov writes: >>> Sorry, but I don't buy that. I think sending both server_version and >>> server_version_num would be silly, and we're certainly not going to stop >>> sending

Re: [HACKERS] Fwd: [BUGS] BUG #14247: COMMENT is restored on wrong database

2016-08-04 Thread David G. Johnston
On Thu, Aug 4, 2016 at 4:50 PM, Tom Lane wrote: > Robert Haas writes: > > On Tue, Aug 2, 2016 at 5:42 PM, David G. Johnston > > wrote: > > The fact that pg_dump is emitting COMMENT ON DATABASE at all is > > fundamentally

[HACKERS] Bogus ANALYZE results for an otherwise-unique column with many nulls

2016-08-04 Thread Tom Lane
I looked into the problem described at https://www.postgresql.org/message-id/flat/VisenaEmail.26.df42f82acae38a58.156463942b8%40tc7-visena and I believe I've reproduced it: the requirement is that the inner join column for the antijoin must contain a lot of NULL values, and what isn't NULL must be

Re: [HACKERS] Bogus ANALYZE results for an otherwise-unique column with many nulls

2016-08-04 Thread Tom Lane
I wrote: > Looking around, there are a couple of places outside commands/analyze.c > that are making the same mistake, so this patch isn't complete, but it > illustrates what needs to be done. Here's a more complete patch. regards, tom lane diff --git

Re: [HACKERS] Fwd: [BUGS] BUG #14247: COMMENT is restored on wrong database

2016-08-04 Thread Craig Ringer
On 5 August 2016 at 05:03, David G. Johnston wrote: > ​ > I'm all for an elegant solution here though at some point having a working > solution now beats waiting for someone to willingly dive more deeply into > pg_dump. I too seem to recall previous proposals for

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Craig Ringer
On 5 August 2016 at 04:31, Tom Lane wrote: > In short: if we'd done it that way all along, it would've been nice. But encouraging people to change horses now isn't really going to > make things better. What is far more likely to happen, if we were > to do that, is that

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 11:15 PM, Bruce Momjian wrote: > On Thu, Aug 4, 2016 at 09:53:31PM -0300, Claudio Freire wrote: >> It's the other way around. A WARM chain has to span whole HOT chains. >> The WARM chain always begins in the head of a HOT chain (or non-HOT >> tuple of

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 7:59 AM, Pavan Deolasee wrote: > So marking the index would require us to mark both old and new index tuples > as requiring re-check. That requires an additional index scan to locate the > old row and then an additional write to force it to

Re: [HACKERS] [sqlsmith] FailedAssertion("!(XLogCtl->Insert.exclusiveBackup)", File: "xlog.c", Line: 10200)

2016-08-04 Thread Michael Paquier
On Thu, Aug 4, 2016 at 4:38 PM, Michael Paquier wrote: > Andreas, with the patch attached is the assertion still triggered? > > Thoughts from others are welcome as well. Self review: * of streaming base backups currently in progress. forcePageWrites is set

Re: [HACKERS] Detecting skipped data from logical slots (data silently skipped)

2016-08-04 Thread Robert Haas
On Wed, Aug 3, 2016 at 9:39 AM, Craig Ringer wrote: > I think we have a bit of a problem with the behaviour specified for logical > slots, one that makes it hard to prevent a outdated snapshot or backup of a > logical-slot-using downstream from knowing it's missing a chunk

Re: [HACKERS] multivariate statistics (v19)

2016-08-04 Thread Michael Paquier
On Wed, Aug 3, 2016 at 10:58 AM, Tomas Vondra wrote: > Attached is v19 of the "multivariate stats" patch series - essentially v18 > rebased on top of current master. Aside from a few bug fixes, the main > improvement is addition of SGML docs demonstrating the

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
>On Thu, Aug 4, 2016 at 06:21:48PM -0400, Bruce Momjian wrote: > Another approach would be to keep updates on the same page in a single > WARM chain, even if all indexes have changed columns. We will already > have code to walk the HOT chain looking for matches, and at most one > tuple in the

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 09:53:31PM -0300, Claudio Freire wrote: > It's the other way around. A WARM chain has to span whole HOT chains. > The WARM chain always begins in the head of a HOT chain (or non-HOT > tuple of course), and cannot end before the HOT chain ends. > > That is, all within a

Re: [HACKERS] pg_ctl promote wait

2016-08-04 Thread Michael Paquier
On Wed, Aug 3, 2016 at 9:35 PM, Peter Eisentraut wrote: > Updated patch set for pg_ctl promote -w, incorporating AFAICT all of the > previous reviews, in particular Michael Paquier's changes to the > StartupXLOG() ordering, and rebasing on top of >

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Pavan Deolasee
On Fri, Aug 5, 2016 at 8:23 AM, Claudio Freire wrote: > On Thu, Aug 4, 2016 at 11:15 PM, Bruce Momjian wrote: > > > > > OK, that's a lot of text, and I am confused. Please tell me the > > advantage of having an index point to a string of HOT chains,

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 7:21 PM, Bruce Momjian wrote: > On Thu, Aug 4, 2016 at 03:23:10PM -0300, Claudio Freire wrote: >> INSERT INTO TABLE t (id,val,dat) VALUES (32,'a','blabla'); >> UPDATE t SET dat='blabla2' where id = 32; >> UPDATE t SET dat='blabla3', id=33 where id = 32;

Re: [HACKERS] PostgreSQL 10 Roadmaps

2016-08-04 Thread Etsuro Fujita
On 2016/08/02 23:54, Simon Riggs wrote: https://wiki.postgresql.org/wiki/PostgreSQL10_Roadmap Thank you for creating the wiki page! I added a link to the NTT roadmap page on the links page. We hope that we can collaborate on the projects with people working at other companies or with

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 11:53:05PM -0300, Claudio Freire wrote: > The point is avoiding duplicate rows in the output of index scans. > > I don't think you can avoid it simply by applying index condition > rechecks as the original proposal implies, in this case: > > CREATE TABLE t (id integer not

Re: [HACKERS] max_parallel_degree > 0 for 9.6 beta

2016-08-04 Thread Michael Paquier
On Thu, Aug 4, 2016 at 10:52 PM, David G. Johnston wrote: > My initial reaction was +1 but now I'm leaning toward enabled by default. > > Those who would upgrade to 9.6 within a year of its release are most likely, > process and personality wise, to be those for whom

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Fri, Aug 5, 2016 at 12:44 AM, Bruce Momjian wrote: >> UPDATE t SET dat = dat + 1, id = 3, someid = 3 where id = 2; > > This is ends the WARM chain, and creates new index entries because all > indexes are changed. > >> UPDATE t SET dat = dat + 1, id = 1, someid = 2 where id =

[HACKERS] Re: [COMMITTERS] pgsql: Prevent "snapshot too old" from trying to return pruned TOAST tu

2016-08-04 Thread Robert Haas
On Thu, Aug 4, 2016 at 10:58 PM, Robert Haas wrote: > On Thu, Aug 4, 2016 at 7:23 PM, Tom Lane wrote: >> Robert Haas writes: >>> Prevent "snapshot too old" from trying to return pruned TOAST tuples. >> >> Looks like this patch

Re: [HACKERS] Possible duplicate release of buffer lock.

2016-08-04 Thread Kyotaro HORIGUCHI
At Wed, 03 Aug 2016 12:00:33 -0400, Tom Lane wrote in <26373.1470240...@sss.pgh.pa.us> > Kyotaro HORIGUCHI writes: > > My point here is that if concurrent deletion can't be perfomed by > > the current implement, this while loop could be

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Tom Lane
Robert Haas writes: > On Thu, Aug 4, 2016 at 9:57 AM, Tom Lane wrote: >> [ shrug... ] What do you claim is not machine-readable about >> server_version? > Surely you can't have missed the connection between the issue at hand > and what Craig is

Re: [HACKERS] Fwd: [BUGS] BUG #14247: COMMENT is restored on wrong database

2016-08-04 Thread Robert Haas
On Tue, Aug 2, 2016 at 5:42 PM, David G. Johnston wrote: > Moving to -hackers since this is getting into details > > Bug Report # 14247 > > On Tue, Aug 2, 2016 at 4:58 PM, Tom Lane wrote: >> >> "David G. Johnston"

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 03:23:10PM -0300, Claudio Freire wrote: > INSERT INTO TABLE t (id,val,dat) VALUES (32,'a','blabla'); > UPDATE t SET dat='blabla2' where id = 32; > UPDATE t SET dat='blabla3', id=33 where id = 32; > UPDATE t SET val='b' where id = 33; > UPDATE t SET dat='blabla4' where id =

Re: [HACKERS] Possible duplicate release of buffer lock.

2016-08-04 Thread Kyotaro HORIGUCHI
Hello, At Thu, 4 Aug 2016 16:09:17 -0700, Peter Geoghegan wrote in > On Wed, Aug 3, 2016 at 1:31 AM, Kyotaro HORIGUCHI > wrote: > > The first line is emitted for simultaneous

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 7:59 AM, Pavan Deolasee wrote: > We don’t want to force a recheck for every index fetch because that will > slow everything down. So we would like a simple and efficient way of knowing > about the existence of a WARM tuple in a chain and do a

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 01:05:53PM -0400, Bruce Momjian wrote: > > Approach 2 seems more reasonable and simple.  > > > > There are only 2 bits for lp_flags and all combinations are already used. > > But > > for LP_REDIRECT root line pointer, we could use the lp_len field to store > > this > >

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 3:07 PM, Bruce Momjian wrote: > On Thu, Aug 4, 2016 at 02:35:32PM -0300, Claudio Freire wrote: >> > Vacuum will need to be taught not to break the invariant, but I >> > believe this is viable and efficient. >> >> >> And I didn't mention the invariant. >>

Re: [HACKERS] New version numbering practices

2016-08-04 Thread Peter Eisentraut
On 8/3/16 1:20 PM, Tom Lane wrote: > I was just wondering whether it would be worth starting to use "git tag -a". One should always use annotated tags for public releases. That way, the tag is its own Git object that cannot be later moved around or easily faked (modulo GPG signatures -- a whole

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Andres Freund
On 2016-08-04 18:11:17 +0100, Simon Riggs wrote: > On 4 August 2016 at 17:31, Andres Freund wrote: > > Hi, > > > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > >> Indexes whose values do not change do not require new index pointers. Only > >> the index whose key is

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Pavan Deolasee
On Thu, Aug 4, 2016 at 10:49 PM, Bruce Momjian wrote: > On Thu, Aug 4, 2016 at 06:16:02PM +0100, Simon Riggs wrote: > > On 4 August 2016 at 18:05, Bruce Momjian wrote: > > > > >> Approach 2 seems more reasonable and simple. > > >> > > >> There are only 2

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 2:14 PM, Claudio Freire wrote: > To fix this, what I was pondering, and I believe it would be viable, > is to teach heap_hot_search_buffer about the WARM invariant. It will > make correctness heavily depend on the invariant being true, which > means

[HACKERS] psql: Missing option to print current buffer to current output channel (i.e., \qprint)

2016-08-04 Thread David G. Johnston
"\echo" has "\qecho" - I'm basically looking for a "\qprint" "\write" doesn't apply The following doesn't work either... #bash (stock Ubuntu 14.04) psql --set=query_in_var='SELECT version();' > /dev/null <<'SQL' \o /tmp/psql-test.txt \set ECHO queries --still ends up on stdout, hence the

Re: [HACKERS] [RFC] Change the default of update_process_title to off

2016-08-04 Thread Robert Haas
On Thu, Aug 4, 2016 at 3:52 AM, Andres Freund wrote: > On 2016-08-04 16:48:11 +0900, Michael Paquier wrote: >> Here is a different proposal: documenting instead that disabling that >> parameter on Windows can improve performance, at the cost of losing >> information verbosity

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 02:35:32PM -0300, Claudio Freire wrote: > > Vacuum will need to be taught not to break the invariant, but I > > believe this is viable and efficient. > > > And I didn't mention the invariant. > > The invariant would be that all WARM chains: > > * contain entire HOT

Re: [HACKERS] improved DefElem list processing

2016-08-04 Thread Tom Lane
I wrote: > Peter Eisentraut writes: >> The other adds a location field to the DefElem node. > +1 for sure, lots of places where that would be a good thing > (the duplicate check itself, for starters). Forgot to mention: seems like you should have added a

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Simon Riggs
On 4 August 2016 at 18:17, Andres Freund wrote: > On 2016-08-04 18:11:17 +0100, Simon Riggs wrote: >> On 4 August 2016 at 17:31, Andres Freund wrote: >> > Hi, >> > >> > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: >> >> Indexes whose values do not

[HACKERS] Pgbench performance tuning?

2016-08-04 Thread Greg Stark
I'm trying to run pgbench on a moderately beefy machine (4-core 3.4GHz with 32G of ram and md mirrored spinning rust drives) at scale 300 with 32 clients with duration of 15min. I'm getting TPS numbers between 60-150 which seems surprisingly low to me and also to several people on IRC. Now

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 11:04:49PM +0530, Pavan Deolasee wrote: > If the root tuple still exists, we store the WARM flag (or > HEAP_RECHECK_REQUIRED as used in the original post) in the tuple header > itself. > When the root tuple becomes dead and HOT prune decides to replace it with a >

Re: [HACKERS] Pgbench performance tuning?

2016-08-04 Thread Andres Freund
On 2016-08-04 19:15:43 +0100, Greg Stark wrote: > On Thu, Aug 4, 2016 at 6:53 PM, Andres Freund wrote: > > What's the config? Version? What activity does pidstat -d -l indicate? > > How much WAL was generated? > > I know the specifics matter but I was also trying to avoid

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 06:03:34PM +0100, Simon Riggs wrote: > On 4 August 2016 at 02:13, Bruce Momjian wrote: > > > How do plan to clear the bitmask so it, over time, doesn't end up being > > all-set? > > I don't have a plan, though thoughts welcome. > > Similar situation

Re: [HACKERS] Pgbench performance tuning?

2016-08-04 Thread Andres Freund
On 2016-08-04 18:48:31 +0100, Greg Stark wrote: > I'm trying to run pgbench on a moderately beefy machine (4-core 3.4GHz > with 32G of ram and md mirrored spinning rust drives) at scale 300 > with 32 clients with duration of 15min. I'm getting TPS numbers > between 60-150 which seems surprisingly

Re: [HACKERS] Pgbench performance tuning?

2016-08-04 Thread Greg Stark
On Thu, Aug 4, 2016 at 6:53 PM, Andres Freund wrote: > What's the config? Version? What activity does pidstat -d -l indicate? > How much WAL was generated? I know the specifics matter but I was also trying to avoid dumping too much into the email. The shared buffers is set

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 09:58:29AM -0700, Andres Freund wrote: > On 2016-08-04 12:55:25 -0400, Bruce Momjian wrote: > > On Thu, Aug 4, 2016 at 09:31:18AM -0700, Andres Freund wrote: > > > Hi, > > > > > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > > > > Indexes whose values do not

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 1:58 PM, Simon Riggs wrote: > On 3 August 2016 at 20:37, Claudio Freire wrote: >> On Wed, Aug 3, 2016 at 4:20 AM, Simon Riggs wrote: >>> == IndexScan == >>> >>> Note that the executor code for IndexScan

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Simon Riggs
On 4 August 2016 at 18:05, Bruce Momjian wrote: >> Approach 2 seems more reasonable and simple. >> >> There are only 2 bits for lp_flags and all combinations are already used. But >> for LP_REDIRECT root line pointer, we could use the lp_len field to store >> this >> special

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Simon Riggs
On 4 August 2016 at 18:27, Bruce Momjian wrote: >> > Also, why not use this bitmap for all indexes, not just update chains? >> >> I don't understand where you get this update chains thing from. >> >> The bitmap can apply to multiple tuples on one page, which is described. > > I

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Simon Riggs
On 4 August 2016 at 17:31, Andres Freund wrote: > Hi, > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: >> Indexes whose values do not change do not require new index pointers. Only >> the index whose key is being changed will need a new index entry. The new >> index

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Andres Freund
On 2016-08-04 18:18:47 +0100, Simon Riggs wrote: > On 4 August 2016 at 18:17, Andres Freund wrote: > > On 2016-08-04 18:11:17 +0100, Simon Riggs wrote: > > I'm less afraid of the fiddlyness of finding the root tuple, than the > > cost. It's not cheap to walk through,

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Pavan Deolasee
On Thu, Aug 4, 2016 at 10:41 PM, Simon Riggs wrote: > On 4 August 2016 at 17:31, Andres Freund wrote: > > Hi, > > > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > >> Indexes whose values do not change do not require new index pointers. > Only

  1   2   >