[HACKERS] pretty bad n_distinct estimate, causing HashAgg OOM on TPC-H

2015-06-17 Thread Tomas Vondra
Hi, I'm currently running some tests on a 3TB TPC-H data set, and I tripped over a pretty bad n_distinct underestimate, causing OOM in HashAgg (which somehow illustrates the importance of the memory-bounded hashagg patch Jeff Davis is working on). The problem is Q18, particularly this

Re: [HACKERS] On columnar storage

2015-06-17 Thread Alvaro Herrera
Jim Nasby wrote: Related to idea of an 'auto join', I do wish we had the ability to access columns in a referenced FK table from a referring key; something like SELECT customer_id.first_name FROM invoice (which would be translated to SELECT first_name FROM invoice JOIN customer USING(

Re: [HACKERS] On columnar storage

2015-06-17 Thread Jim Nasby
On 6/17/15 2:04 PM, Alvaro Herrera wrote: Jim Nasby wrote: Related to idea of an 'auto join', I do wish we had the ability to access columns in a referenced FK table from a referring key; something like SELECT customer_id.first_name FROM invoice (which would be translated to SELECT first_name

Re: [HACKERS] [PATCH] Function to get size of asynchronous notification queue

2015-06-17 Thread Brendan Jurd
On Thu, 18 Jun 2015 at 03:06 Gurjeet Singh gurj...@singh.im wrote: I don't see this in the CF app; can you please add it there? Done. I did try to add it when I posted the email, but for some reason I couldn't connect to commitfest.postgresql.org at all. Seems fine now, though. Cheers, BJ

Re: [HACKERS] WAL replay bugs

2015-06-17 Thread Alvaro Herrera
Michael Paquier wrote: From 077d675795b4907904fa4e85abed8c4528f4666f Mon Sep 17 00:00:00 2001 From: Michael Paquier mich...@otacoo.com Date: Sat, 19 Jul 2014 10:40:20 +0900 Subject: [PATCH 3/3] Buffer capture facility: check WAL replay consistency Is there a newer version of this tech? --

Re: [HACKERS] 9.5 make world failing due to sgml tools missing

2015-06-17 Thread Tom Lane
Keith Fiske ke...@omniti.com writes: The current HEAD of postgres in the git repo is not building when using make world. It's been like this for about a month or so that I've been aware of. I didn't really need the world build so been making due without it. At PGCon now, though, so asked Bruce

Re: [HACKERS] GIN function of pageinspect has inconsistency data type.

2015-06-17 Thread Jim Nasby
On 6/16/15 8:26 AM, Sawada Masahiko wrote: I noticed while using gin function of pageinspect that there are some inconsistency data types. For example, data type of GinMetaPageData.head, and tail is BlockNumber, i.g, uint32. But in ginfunc.c, we get that value as int64. You can't put a uint32

Re: [HACKERS] could not adopt C locale failure at startup on Windows

2015-06-17 Thread Noah Misch
On Wed, Jun 17, 2015 at 01:43:55PM -0400, Tom Lane wrote: Noah Misch n...@leadboat.com writes: On Mon, Jun 15, 2015 at 12:37:43PM -0400, Tom Lane wrote: It's mere chance that the order of calls to pg_perm_setlocale() is such that the code works now; and it's not hard to imagine future

Re: [HACKERS] 9.5 make world failing due to sgml tools missing

2015-06-17 Thread Joshua D. Drake
On 06/17/2015 01:07 PM, Tom Lane wrote: Keith Fiske ke...@omniti.com writes: The current HEAD of postgres in the git repo is not building when using make world. It's been like this for about a month or so that I've been aware of. I didn't really need the world build so been making due without

Re: [HACKERS] [PATCH] Function to get size of asynchronous notification queue

2015-06-17 Thread Brendan Jurd
Posting v2 of the patch, incorporating some helpful suggestions from Merlin. Cheers, BJ *** a/doc/src/sgml/func.sgml --- b/doc/src/sgml/func.sgml *** *** 14800,14805 SELECT * FROM pg_ls_dir('.') WITH ORDINALITY AS t(ls,n); --- 14800,14811 /row row +

Re: [HACKERS] last_analyze/last_vacuum not being updated

2015-06-17 Thread Peter Eisentraut
On 6/15/15 4:45 PM, Peter Eisentraut wrote: On 6/8/15 3:16 PM, Peter Eisentraut wrote: I'm looking at a case on 9.4.1 where the last_analyze and last_vacuum stats for a handful of tables seem stuck. They don't update after running an ANALYZE or VACUUM command, and they don't react to

Re: [HACKERS] [PATCH] Function to get size of asynchronous notification queue

2015-06-17 Thread Brendan Jurd
On Thu, 18 Jun 2015 at 08:19 Merlin Moncure mmonc...@gmail.com wrote: scratch that. that note already exists in sql-notify.html. Instead, I'd modify that section to note that you can check queue usage with your new function. I have already done so. Under the paragraph about the queue

Re: [HACKERS] Auto-vacuum is not running in 9.1.12

2015-06-17 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Yeah, the case is pretty weird and I'm not really sure that the server ought to be expected to behave. But if this is actually the only part of the server that misbehaves because of sudden gigantic time jumps, I think it's fair to patch it.

Re: [HACKERS] WAL replay bugs

2015-06-17 Thread Michael Paquier
On Thu, Jun 18, 2015 at 3:39 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Michael Paquier wrote: From 077d675795b4907904fa4e85abed8c4528f4666f Mon Sep 17 00:00:00 2001 From: Michael Paquier mich...@otacoo.com Date: Sat, 19 Jul 2014 10:40:20 +0900 Subject: [PATCH 3/3] Buffer capture

Re: [HACKERS] [GENERAL] psql weird behaviour with charset encodings

2015-06-17 Thread Michael Paquier
On Thu, Jun 18, 2015 at 9:47 AM, Noah Misch n...@leadboat.com wrote: On Wed, Jun 03, 2015 at 05:25:45PM +0900, Michael Paquier wrote: On Tue, Jun 2, 2015 at 4:19 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Sun, May 24, 2015 at 2:43 AM, Noah Misch n...@leadboat.com wrote: It

Re: [HACKERS] [GENERAL] psql weird behaviour with charset encodings

2015-06-17 Thread Noah Misch
On Wed, Jun 03, 2015 at 05:25:45PM +0900, Michael Paquier wrote: On Tue, Jun 2, 2015 at 4:19 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Sun, May 24, 2015 at 2:43 AM, Noah Misch n...@leadboat.com wrote: It would be good to purge the code of precisions on s conversion

Re: [HACKERS] 9.5 make world failing due to sgml tools missing

2015-06-17 Thread Peter Eisentraut
On 6/17/15 3:35 PM, Keith Fiske wrote: The current HEAD of postgres in the git repo is not building when using make world. It's been like this for about a month or so that I've been aware of. I didn't really need the world build so been making due without it. At PGCon now, though, so asked

Re: [HACKERS] could not adopt C locale failure at startup on Windows

2015-06-17 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Mon, Jun 15, 2015 at 12:37:43PM -0400, Tom Lane wrote: It's mere chance that the order of calls to pg_perm_setlocale() is such that the code works now; and it's not hard to imagine future changes in gettext, or reordering of our own code, that would

Re: [HACKERS] pg_rewind and xlogtemp files

2015-06-17 Thread Michael Paquier
On Wed, Jun 17, 2015 at 9:07 PM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Jun 17, 2015 at 4:57 PM, Vladimir Borodin r...@simply.name wrote: 17 июня 2015 г., в 9:48, Michael Paquier michael.paqu...@gmail.com написал(а): On Wed, Jun 17, 2015 at 3:17 PM, Michael Paquier

Re: [HACKERS] pg_rewind and xlogtemp files

2015-06-17 Thread Fujii Masao
On Wed, Jun 17, 2015 at 4:57 PM, Vladimir Borodin r...@simply.name wrote: 17 июня 2015 г., в 9:48, Michael Paquier michael.paqu...@gmail.com написал(а): On Wed, Jun 17, 2015 at 3:17 PM, Michael Paquier michael.paqu...@gmail.com wrote: As pointed by dev1ant on the original bug report,

Re: [HACKERS] could not adopt C locale failure at startup on Windows

2015-06-17 Thread Noah Misch
On Mon, Jun 15, 2015 at 12:37:43PM -0400, Tom Lane wrote: After further thought, ISTM that this bug is evidence that 5f538ad was badly designed, and the proposed fix has blinkers on. If pg_bind_textdomain_codeset() is looking at the locale environment, we should not be calling it from inside

Re: [HACKERS] Auto-vacuum is not running in 9.1.12

2015-06-17 Thread Haribabu Kommi
On Wed, Jun 17, 2015 at 2:17 PM, Prakash Itnal prakash...@gmail.com wrote: Hi, Currently the issue is easily reproducible. Steps to reproduce: * Set some aggressive values for auto-vacuuming. * Run a heavy database update/delete/insert queries. This leads to invoking auto-vacuuming in quick

Re: [HACKERS] pg_rewind and xlogtemp files

2015-06-17 Thread Michael Paquier
On Wed, Jun 17, 2015 at 3:17 PM, Michael Paquier michael.paqu...@gmail.com wrote: As pointed by dev1ant on the original bug report, process_remote_file should ignore files named as pg_xlog/xlogtemp.*, and I think that this is the right thing to do. Any objections for a patch that at the same

[HACKERS] Is Postgres database server works fine if there is a change in system time?

2015-06-17 Thread Prakash Itnal
Hi, Currently we observed that certain postgres child process, for eg. autovacuum worker, are not working as expected if there is a system time change. So I wanted to know if postgres already supports system time changes or not. Please confirm if postgres already handles system time changes or

Re: [HACKERS] pg_rewind and xlogtemp files

2015-06-17 Thread Vladimir Borodin
17 июня 2015 г., в 9:48, Michael Paquier michael.paqu...@gmail.com написал(а): On Wed, Jun 17, 2015 at 3:17 PM, Michael Paquier michael.paqu...@gmail.com wrote: As pointed by dev1ant on the original bug report, process_remote_file should ignore files named as pg_xlog/xlogtemp.*, and I

[HACKERS] pg_rewind and xlogtemp files

2015-06-17 Thread Michael Paquier
Hi all, I just bumped into this report regarding pg_rewind, that impacts as well the version shipped in src/bin/pg_rewind: https://github.com/vmware/pg_rewind/issues/45 In short, the issue refers to the fact that if the source server filemap includes xlogtemp files pg_rewind will surely fail

Re: [HACKERS] checkpointer continuous flushing

2015-06-17 Thread Fabien COELHO
Hello, Here is version 3, including many performance tests with various settings, representing about 100 hours of pgbench run. This patch aims at improving checkpoint I/O behavior so that tps throughput is improved, late transactions are less frequent, and overall performances are more

[HACKERS] [PATCH] Function to get size of asynchronous notification queue

2015-06-17 Thread Brendan Jurd
Hello hackers, I present a patch to add a new built-in function pg_notify_queue_saturation(). The purpose of the function is to allow users to monitor the health of their notification queue. In certain cases, a client connection listening for notifications might get stuck inside a transaction,

[HACKERS] Inheritance planner CPU and memory usage change since 9.3.2

2015-06-17 Thread Thomas Munro
Hi We saw a rather extreme performance problem in a cluster upgraded from 9.1 to 9.3. It uses a largish number of child tables (partitions) and many columns. Planning a simple UPDATE via the base table started using a very large amount of memory and CPU time. My colleague Rushabh Lathia

Re: [HACKERS] Inheritance planner CPU and memory usage change since 9.3.2

2015-06-17 Thread Robert Haas
On Wed, Jun 17, 2015 at 9:32 AM, Thomas Munro thomas.mu...@enterprisedb.com wrote: We saw a rather extreme performance problem in a cluster upgraded from 9.1 to 9.3. It uses a largish number of child tables (partitions) and many columns. Planning a simple UPDATE via the base table started

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

2015-06-17 Thread Tomas Vondra
Hi, On 05/13/15 23:07, Jeff Janes wrote: With the warning it is very hard to correlate the discrepancy you do see with which column is causing it, as the warnings don't include table or column names (Assuming of course that you run it on a substantial database--if you just run it on a few toy

Re: [HACKERS] Auto-vacuum is not running in 9.1.12

2015-06-17 Thread Alvaro Herrera
Prakash Itnal wrote: Currently the issue is easily reproducible. Steps to reproduce: * Set some aggressive values for auto-vacuuming. * Run a heavy database update/delete/insert queries. This leads to invoking auto-vacuuming in quick successions. * Change the system time to older for eg.

Re: [HACKERS] Auto-vacuum is not running in 9.1.12

2015-06-17 Thread Tom Lane
Haribabu Kommi kommi.harib...@gmail.com writes: I can think of a case where the launcher_determine_sleep function returns a big sleep value because of system time change. Because of that it is possible that the launcher is not generating workers to do the vacuum. May be I am wrong. I talked

Re: [HACKERS] [PATCH] Function to get size of asynchronous notification queue

2015-06-17 Thread Gurjeet Singh
I don't see this in the CF app; can you please add it there? Best regards, On Wed, Jun 17, 2015 at 3:31 AM, Brendan Jurd dire...@gmail.com wrote: Hello hackers, I present a patch to add a new built-in function pg_notify_queue_saturation(). The purpose of the function is to allow users to

Re: [HACKERS] Sequence Access Method WIP

2015-06-17 Thread Petr Jelinek
On 2015-06-15 11:32, Vik Fearing wrote: I've been looking at these patches a bit and here are some comments: Thanks for looking at this. Patch 1: seqam I would like to see an example in the docs for CREATE SEQUENCE. That's perhaps not possible (or desirable) with only the local seqam?