Re: [HACKERS] invalid magic number in log segment

2013-12-11 Thread Heikki Linnakangas
On 12/11/2013 09:44 AM, Erikjan Rijkers wrote: I don't know whether the below constitutes a bug, but: Daily (sometimes even more often) I recompile 9.4devel (after git pull) to run a large dev database (100 GB or so). To avoid frequent initdb and many-hour-restore of data, I do this only

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Mark Kirkwood
On 11/12/13 19:34, Simon Riggs wrote: Realistically, I never heard of an Oracle DBA doing advanced statistical mathematics before setting the sample size on ANALYZE. You use the default and bump it up if the sample is insufficient for the data. I'm not sure that Oracle's stats and optimizer

Re: [HACKERS] Time-Delayed Standbys

2013-12-11 Thread Simon Riggs
On 11 December 2013 06:36, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: I think this feature will be used in a lot of scenarios in which PITR is currently used. We have to judge which is better, we get something potential or to protect stupid. And we had better to wait author's

Re: [HACKERS] logical changeset generation v6.8

2013-12-11 Thread Andres Freund
On 2013-12-10 19:11:03 -0500, Robert Haas wrote: Committed #1 (again). Thanks! Regarding this: + /* XXX: we could also do this unconditionally, the space is used anyway + if (copy_oid) + HeapTupleSetOid(key_tuple, HeapTupleGetOid(tp)); I would like to put

[HACKERS] Question about sorting internals

2013-12-11 Thread hubert depesz lubaczewski
Hi, before I'll go any further - this is only thought-experiment. I do not plan to use such queries in real-life applications. I was just presented with a question that I can't answer in any logical way. There are two simple queries: #v+ with rok2005 (miesiac,wynik) as (VALUES (1,1),(2,2)

Re: [HACKERS] Question about sorting internals

2013-12-11 Thread Ashutosh Bapat
Hi deepesz, You might want to see their EXPLAIN VERBOSE outputs. Having one of them (2004 one) lesser number of rows, might be getting picked up as first relation being union and thus ends up having it's rows before the second one. Explain output would make it more clear. Also, try having same

Re: [HACKERS] Completing PL support for Event Triggers

2013-12-11 Thread Dimitri Fontaine
Peter Eisentraut pete...@gmx.net writes: I think you are mistaken. My patch includes all changes between your v1 and v2 patch. I mistakenly remembered that we did remove all the is_event_trigger business from the plperl patch too, when it's not the case. Sorry about this confusion. My vote is

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-12-11 Thread Haribabu kommi
On 10 December 2013 19:55 Alvaro Herrera wrote: Haribabu kommi escribió: To detect provided data and xlog directories are same or not, I reused the Existing make_absolute_path() code as follows. 1. Moved the make_absolute_path() function from miscinit.c to path.c and Changed all

Re: [HACKERS] Question about sorting internals

2013-12-11 Thread hubert depesz lubaczewski
On Wed, Dec 11, 2013 at 03:34:38PM +0530, Ashutosh Bapat wrote: Hi deepesz, You might want to see their EXPLAIN VERBOSE outputs. Having one of them (2004 one) lesser number of rows, might be getting picked up as first relation being union and thus ends up having it's rows before the second

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-11 Thread Florian Pflug
On Dec5, 2013, at 15:44 , Andres Freund and...@2ndquadrant.com wrote: There might be some ugly compiler dependent magic we could do. Depending on how we decide to declare offsets. Like (very, very roughly) #define relptr(type, struct_name, varname) union struct_name##_##varname{ \ type

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-11 Thread Andres Freund
On 2013-12-11 11:42:25 +0100, Florian Pflug wrote: On Dec5, 2013, at 15:44 , Andres Freund and...@2ndquadrant.com wrote: There might be some ugly compiler dependent magic we could do. Depending on how we decide to declare offsets. Like (very, very roughly) #define relptr(type,

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Greg Stark
On Wed, Dec 11, 2013 at 12:58 AM, Simon Riggs si...@2ndquadrant.com wrote: Yes, it is not a perfect statistical sample. All sampling is subject to an error that is data dependent. Well there's random variation due to the limitations of dealing with a sample. And then there's systemic biases

Re: [HACKERS] Why the buildfarm is all pink

2013-12-11 Thread Andres Freund
On 2013-12-10 19:55:12 -0500, Tom Lane wrote: I was surprised to see that my back-patches of the recent SubLink unpleasantness were failing on many of the buildfarm members, but only in the 9.1 and 9.0 branches. The difficulty appears to be that the EXPLAIN output for the new test query

Re: [HACKERS] Extension Templates S03E11

2013-12-11 Thread Dimitri Fontaine
Hi, Stephen Frost sfr...@snowman.net writes: * Jeff Davis (pg...@j-davis.com) wrote: What is stopping Extension Templates, as proposed, from being this special extension creation mode? What would be a better design? The extra catalog tables which store SQL scripts in text columns is one of

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-11 Thread Florian Pflug
On Dec11, 2013, at 11:47 , Andres Freund and...@2ndquadrant.com wrote: On 2013-12-11 11:42:25 +0100, Florian Pflug wrote: On Dec5, 2013, at 15:44 , Andres Freund and...@2ndquadrant.com wrote: There might be some ugly compiler dependent magic we could do. Depending on how we decide to declare

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-11 Thread Andres Freund
On 2013-12-11 12:37:56 +0100, Florian Pflug wrote: On Dec11, 2013, at 11:47 , Andres Freund and...@2ndquadrant.com wrote: On 2013-12-11 11:42:25 +0100, Florian Pflug wrote: Yes (although there's C11 stuff to do equivalent stuff afair) - I was thinking of only doing it for compilers we

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Greg Stark
On Wed, Dec 11, 2013 at 11:01 AM, Greg Stark st...@mit.edu wrote: I'm not actually sure there is any systemic bias here. The larger number of rows per block generate less precise results but from my thought experiments they seem to still be accurate? So I've done some empirical tests for a

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Greg Stark
On Wed, Dec 11, 2013 at 12:08 PM, Greg Stark st...@mit.edu wrote: The only thing I can think of is maybe the most common elements are being selected preferentially from the early part of the sample which is removing a substantial part of the lower end of the range. But even removing 100 from

Re: [HACKERS] [PATCH] Add transforms feature

2013-12-11 Thread Robert Haas
On Tue, Dec 10, 2013 at 10:35 PM, Peter Eisentraut pete...@gmx.net wrote: On Fri, 2013-12-06 at 11:28 +0100, Dimitri Fontaine wrote: Here is an idea. Add a GUC that basically says something like use_transforms = on|off. You can then attach that to individual functions, which is the right

[HACKERS] should we add a XLogRecPtr/LSN SQL type?

2013-12-11 Thread Andres Freund
Hi, There's already a couple of SQL function dealing with XLogRecPtrs and the logical replication work will add a couple of more. Currently each of those funtions taking/returning an LSN does sprintf/scanf to print/parse the strings. Which both is awkward and potentially noticeable

Re: [HACKERS] Extra functionality to createuser

2013-12-11 Thread Robert Haas
On Tue, Dec 10, 2013 at 9:55 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Tue, Dec 10, 2013 at 12:20 AM, Robert Haas robertmh...@gmail.com wrote: On Sat, Dec 7, 2013 at 11:39 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Fri, Dec 6, 2013 at 10:31 AM, Peter Eisentraut pete...@gmx.net

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Heikki Linnakangas
On 12/11/2013 02:08 PM, Greg Stark wrote: On Wed, Dec 11, 2013 at 11:01 AM, Greg Stark st...@mit.edu wrote: I'm not actually sure there is any systemic bias here. The larger number of rows per block generate less precise results but from my thought experiments they seem to still be accurate?

Re: [HACKERS] should we add a XLogRecPtr/LSN SQL type?

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 7:41 AM, Andres Freund and...@2ndquadrant.com wrote: There's already a couple of SQL function dealing with XLogRecPtrs and the logical replication work will add a couple of more. Currently each of those funtions taking/returning an LSN does sprintf/scanf to print/parse

Re: [HACKERS] should we add a XLogRecPtr/LSN SQL type?

2013-12-11 Thread Euler Taveira
On 11-12-2013 09:41, Andres Freund wrote: There's already a couple of SQL function dealing with XLogRecPtrs and the logical replication work will add a couple of more. Currently each of those funtions taking/returning an LSN does sprintf/scanf to print/parse the strings. Which both is awkward

Re: [HACKERS] -d option for pg_isready is broken

2013-12-11 Thread Robert Haas
On Tue, Dec 10, 2013 at 11:40 PM, Fujii Masao masao.fu...@gmail.com wrote: While I was investigaing PQhost() for that approach, I found several problems of PQhost(). (1) PQhost() can return Unix-domain socket directory path even in the platform that doesn't support Unix-domain socket.

Re: [HACKERS] -d option for pg_isready is broken

2013-12-11 Thread Fujii Masao
On Wed, Dec 11, 2013 at 10:26 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Dec 10, 2013 at 11:40 PM, Fujii Masao masao.fu...@gmail.com wrote: While I was investigaing PQhost() for that approach, I found several problems of PQhost(). (1) PQhost() can return Unix-domain socket directory

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Robert Haas
On Sun, Oct 20, 2013 at 7:21 AM, Magnus Hagander mag...@hagander.net wrote: It seemed neater to me to create a new flag, so that in principle any vacuum() code path can request autovacuum_work_mem, rather than having lazyvacuum.c code call IsAutoVacuumWorkerProcess() for the same purpose. To

Re: [HACKERS] -d option for pg_isready is broken

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 8:45 AM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Dec 11, 2013 at 10:26 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Dec 10, 2013 at 11:40 PM, Fujii Masao masao.fu...@gmail.com wrote: While I was investigaing PQhost() for that approach, I found several

Re: [HACKERS] [PATCH] Add transforms feature

2013-12-11 Thread Hannu Krosing
On 12/11/2013 01:40 PM, Robert Haas wrote: On Tue, Dec 10, 2013 at 10:35 PM, Peter Eisentraut pete...@gmx.net wrote: On Fri, 2013-12-06 at 11:28 +0100, Dimitri Fontaine wrote: Here is an idea. Add a GUC that basically says something like use_transforms = on|off. You can then attach that to

Re: [HACKERS] Question about sorting internals

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 4:56 AM, hubert depesz lubaczewski dep...@depesz.com wrote: before I'll go any further - this is only thought-experiment. I do not plan to use such queries in real-life applications. I was just presented with a question that I can't answer in any logical way. There are

Re: [HACKERS] Why the buildfarm is all pink

2013-12-11 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: I haven't touched matview.sql here; that seems like a distinct issue. I'll fix that. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Florian Pflug
On Dec10, 2013, at 15:32 , Claudio Freire klaussfre...@gmail.com wrote: On Tue, Dec 10, 2013 at 11:02 AM, Greg Stark st...@mit.edu wrote: On 10 Dec 2013 08:28, Albe Laurenz laurenz.a...@wien.gv.at wrote: Doesn't all that assume a normally distributed random variable? I don't think so

Re: [HACKERS] In-Memory Columnar Store

2013-12-11 Thread Merlin Moncure
On Mon, Dec 9, 2013 at 1:40 PM, knizhnik knizh...@garret.ru wrote: Hello! I want to annouce my implementation of In-Memory Columnar Store extension for PostgreSQL: Documentation: http://www.garret.ru/imcs/user_guide.html Sources: http://www.garret.ru/imcs-1.01.tar.gz Any

Re: [HACKERS] -d option for pg_isready is broken

2013-12-11 Thread Andres Freund
On 2013-12-11 08:56:43 -0500, Robert Haas wrote: $ psql -d hostaddr=127.0.0.1 =# \conninfo You are connected to database postgres as user postgres via socket in /tmp at port 5432. Yeah, that's true. But the whole point of having both host and hostaddr seems to be that you can lie

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Simon Riggs
On 25 November 2013 21:51, Peter Geoghegan p...@heroku.com wrote: On Sun, Nov 24, 2013 at 9:06 AM, Simon Riggs si...@2ndquadrant.com wrote: VACUUM uses 6 bytes per dead tuple. And autovacuum regularly removes dead tuples, limiting their numbers. In what circumstances will the memory usage

Re: [HACKERS] [PATCH] Add transforms feature

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 9:19 AM, Hannu Krosing ha...@2ndquadrant.com wrote: On 12/11/2013 01:40 PM, Robert Haas wrote: On Tue, Dec 10, 2013 at 10:35 PM, Peter Eisentraut pete...@gmx.net wrote: On Fri, 2013-12-06 at 11:28 +0100, Dimitri Fontaine wrote: Here is an idea. Add a GUC that basically

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 9:43 AM, Simon Riggs si...@2ndquadrant.com wrote: On 25 November 2013 21:51, Peter Geoghegan p...@heroku.com wrote: On Sun, Nov 24, 2013 at 9:06 AM, Simon Riggs si...@2ndquadrant.com wrote: VACUUM uses 6 bytes per dead tuple. And autovacuum regularly removes dead

Re: [HACKERS] -d option for pg_isready is broken

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 9:35 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-12-11 08:56:43 -0500, Robert Haas wrote: $ psql -d hostaddr=127.0.0.1 =# \conninfo You are connected to database postgres as user postgres via socket in /tmp at port 5432. Yeah, that's true. But the

Re: [HACKERS] Why the buildfarm is all pink

2013-12-11 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: Tom Lane t...@sss.pgh.pa.us wrote: I haven't touched matview.sql here; that seems like a distinct issue. I'll fix that. Done. -- Kevin Grittner EDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing

Re: [HACKERS] [bug fix] pg_ctl always uses the same event source

2013-12-11 Thread MauMau
From: Amit Kapila amit.kapil...@gmail.com I think it is better to keep it like what I suggested above, because in that case it will assign default name even if postgres -C fails due to some reason. 2. What will happen if user doesn't change the name in event_source or kept the same name,

Re: [HACKERS] Why the buildfarm is all pink

2013-12-11 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-12-10 19:55:12 -0500, Tom Lane wrote: We need a more consistent strategy for this :-( Agreed, although I have no clue how it should look like. As a further datapoint I'll add that installcheck already regularly fails in HEAD if you have a

Re: [HACKERS] In-Memory Columnar Store

2013-12-11 Thread k...@rice.edu
On Mon, Dec 09, 2013 at 11:40:41PM +0400, knizhnik wrote: Hello! I want to annouce my implementation of In-Memory Columnar Store extension for PostgreSQL: Documentation: http://www.garret.ru/imcs/user_guide.html Sources: http://www.garret.ru/imcs-1.01.tar.gz Any feedbacks,

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Simon Riggs
On 11 December 2013 12:08, Greg Stark st...@mit.edu wrote: So there is something clearly wonky in the histogram stats that's affected by the distribution of the sample. ...in the case where the avg width changes in a consistent manner across the table. Well spotted. ISTM we can have a

Re: [HACKERS] [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-11 Thread MauMau
From: Kevin Grittner kgri...@ymail.com It seems to be a fairly common term of art for a problem which requires a restart or reconnection. FATAL is used when the problem is severe enough that the process or connection must end. It seems to me to be what should consistently be used when a client

Re: [HACKERS] Question about sorting internals

2013-12-11 Thread Tom Lane
hubert depesz lubaczewski dep...@depesz.com writes: There are two simple queries: ... They differ only in order of queries in union all part. The thing is that they return the same result. Why isn't one of them returning 2005 for 6th miesiac? With such a small amount of data, you're getting

Re: [HACKERS] [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-11 Thread Andres Freund
On 2013-12-12 00:31:25 +0900, MauMau wrote: What do you think of #5 and #6 when matching the above criteria? 5. FATAL: terminating walreceiver process due to administrator command 6. FATAL: terminating background worker \%s\ due to administrator command Those are important if they happen

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Simon Riggs
On 11 December 2013 14:50, Robert Haas robertmh...@gmail.com wrote: On Wed, Dec 11, 2013 at 9:43 AM, Simon Riggs si...@2ndquadrant.com wrote: On 25 November 2013 21:51, Peter Geoghegan p...@heroku.com wrote: On Sun, Nov 24, 2013 at 9:06 AM, Simon Riggs si...@2ndquadrant.com wrote: VACUUM uses

Re: [HACKERS] Extension Templates S03E11

2013-12-11 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Stephen Frost sfr...@snowman.net writes: The extra catalog tables which store SQL scripts in text columns is one of my main objections to the as-proposed Extension Templates. I view those scripts as a poor man's definition of database objects

Re: [HACKERS] [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-11 Thread Kevin Grittner
MauMau maumau...@gmail.com wrote: From: Kevin Grittner kgri...@ymail.com FATAL is used when the problem is severe enough that the process or connection must end. It seems to me to be what should consistently be used when a client connection or its process must be terminated for a reason

Re: [HACKERS] Why the buildfarm is all pink

2013-12-11 Thread Andres Freund
On 2013-12-11 10:07:19 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-12-10 19:55:12 -0500, Tom Lane wrote: We need a more consistent strategy for this :-( Agreed, although I have no clue how it should look like. As a further datapoint I'll add that

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Tom Lane
Greg Stark st...@mit.edu writes: So I've done some empirical tests for a table generated by: create table sizeskew as (select i,j,repeat('i',i) from generate_series(1,1000) as i, generate_series(1,1000) as j); I find that using the whole block doesn't cause any problem with the avg_width

Re: [HACKERS] [PATCH] Add transforms feature

2013-12-11 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Dec 10, 2013 at 10:35 PM, Peter Eisentraut pete...@gmx.net wrote: Here is an updated patch that implements this, makes some of the documentation improvements that you suggested, and rebases everything. I'm still kinda unimpressed by this.

Re: [HACKERS] In-Memory Columnar Store

2013-12-11 Thread knizhnik
Hello! Implementation of IMCS itself took me about two months (with testing and writing documentation). But huge part of the code was previously written by me for other projects, so I have reused them. Most of the time I have spent in integration of this code with PostgreSQL (I was not so

Re: [HACKERS] [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-11 Thread MauMau
From: Tom Lane t...@sss.pgh.pa.us Jim Nasby j...@nasby.net writes: On 12/9/13 5:56 PM, Tom Lane wrote: How so? FATAL means an error that terminates your session, which is exactly what these are. Except in these cases the user never actually got a working session; their request was denied.

Re: [HACKERS] -d option for pg_isready is broken

2013-12-11 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Dec 11, 2013 at 9:35 AM, Andres Freund and...@2ndquadrant.com wrote: One use case is accessing a particular host when using DNS round robin to standbys in combination with SSL. Ah, interesting point. And it's not inconceivable that some

Re: [HACKERS] logical changeset generation v6.8

2013-12-11 Thread Andres Freund
On 2013-12-10 19:11:03 -0500, Robert Haas wrote: Committed #1 (again). Regarding this: + /* XXX: we could also do this unconditionally, the space is used anyway + if (copy_oid) + HeapTupleSetOid(key_tuple, HeapTupleGetOid(tp)); I would like to put in a big +1

Re: [HACKERS] In-Memory Columnar Store

2013-12-11 Thread knizhnik
Hi, I depends on what you mean by transparently substitute. I f you want to be able to execute standard SQL queries using columnar store, then it seems to be impossible without rewriting of executor. I provided another approach based on calling standard functions which perform manipulations

Re: [HACKERS] Why the buildfarm is all pink

2013-12-11 Thread Tom Lane
Kevin Grittner kgri...@ymail.com writes: Kevin Grittner kgri...@ymail.com wrote: Tom Lane t...@sss.pgh.pa.us wrote: I haven't touched matview.sql here; that seems like a distinct issue. I'll fix that. Done. Thanks. regards, tom lane -- Sent via pgsql-hackers

Re: [HACKERS] [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-11 Thread Tom Lane
MauMau maumau...@gmail.com writes: I agree that #1-#3 are of course reasonable when there's any client the user runs. The problem is that #1 (The database system is starting up) is output in the server log by pg_ctl. In that case, there's no client the user is responsible for. Why does a

Re: [HACKERS] [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-11 Thread MauMau
From: Andres Freund and...@2ndquadrant.com On 2013-12-12 00:31:25 +0900, MauMau wrote: 5. FATAL: terminating walreceiver process due to administrator command 6. FATAL: terminating background worker \%s\ due to administrator command Those are important if they happen outside a shutdown. So,

Re: [HACKERS] Why the buildfarm is all pink

2013-12-11 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-12-11 10:07:19 -0500, Tom Lane wrote: Do you remember offhand where the failures are? No, but they are easy enough to reproduce. Out of 10 runs, I've attached the one with the most failures and checked that it seems to contain all the

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Tom Lane
I wrote: Hm. You can only take N rows from a block if there actually are at least N rows in the block. So the sampling rule I suppose you are using is select up to N rows from each sampled block --- and that is going to favor the contents of blocks containing narrower-than-average rows. Oh,

Re: [HACKERS] [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-11 Thread MauMau
From: Kevin Grittner kgri...@ymail.com 5. FATAL: terminating walreceiver process due to administrator command 6. FATAL: terminating background worker \%s\ due to administrator command Those are client connections and their backends terminated for a reason other than the client side of the

Re: [HACKERS] [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-11 Thread Josh Berkus
On 12/11/2013 08:48 AM, Tom Lane wrote: The fundamental problem IMO is that you want to complicate the definition of what these things mean as a substitute for DBAs learning something about Postgres. That seems like a fool's errand from here. They're going to have to learn what FATAL means

Re: [HACKERS] stats for network traffic WIP

2013-12-11 Thread Peter Eisentraut
On 12/10/13, 5:08 PM, Tom Lane wrote: Having said that, I can't get very excited about this feature anyway, so I'm fine with rejecting the patch. I'm not sure that enough people care to justify any added overhead at all. The long and the short of it is that network traffic generally is what

Re: [HACKERS] Problem with displaying wide tables in psql

2013-12-11 Thread Sergey Muraviov
Hi. I've improved the patch. It works in expanded mode when either format option is set to wrapped (\pset format wrapped), or we have no pager, or pager doesn't chop long lines (so you can still use the trick). Target output width is taken from either columns option (\pset columns 70), or

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 10:41 AM, Simon Riggs si...@2ndquadrant.com wrote: It looks fairly easy to estimate the memory needed for an auto vacuum, since we know the scale factor and the tuple estimate. We can then use the memory estimate to alter the scheduling of work. And/or we can use actual

Re: [HACKERS] stats for network traffic WIP

2013-12-11 Thread Atri Sharma
On Wed, Dec 11, 2013 at 11:12 PM, Peter Eisentraut pete...@gmx.net wrote: On 12/10/13, 5:08 PM, Tom Lane wrote: Having said that, I can't get very excited about this feature anyway, so I'm fine with rejecting the patch. I'm not sure that enough people care to justify any added overhead at

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Gavin Flower
On 12/12/13 06:22, Tom Lane wrote: I wrote: Hm. You can only take N rows from a block if there actually are at least N rows in the block. So the sampling rule I suppose you are using is select up to N rows from each sampled block --- and that is going to favor the contents of blocks

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Gavin Flower
On 12/12/13 06:22, Tom Lane wrote: I wrote: Hm. You can only take N rows from a block if there actually are at least N rows in the block. So the sampling rule I suppose you are using is select up to N rows from each sampled block --- and that is going to favor the contents of blocks

Re: [HACKERS] In-Memory Columnar Store

2013-12-11 Thread Merlin Moncure
On Wed, Dec 11, 2013 at 10:08 AM, knizhnik knizh...@garret.ru wrote: 1. Calls in PL/pgSQL are very slow - about 1-2 micsroseconds at my computer. Just defining insertion per-row trigger with empty procedure increase time of insertion of 6 million records twice - from 7 till 15 seconds. If

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Josh Berkus
On 12/11/2013 09:57 AM, Robert Haas wrote: I don't agree with that assessment. Anything that involves changing the scheduling of autovacuum is a major project that will legitimately provoke much controversy. Extensive testing will be needed to prove that the new algorithm doesn't perform

Re: [HACKERS] Time-Delayed Standbys

2013-12-11 Thread Fabrízio de Royes Mello
On Wed, Dec 11, 2013 at 6:27 AM, Simon Riggs si...@2ndquadrant.com wrote: On 11 December 2013 06:36, KONDO Mitsumasa kondo.mitsum...@lab.ntt.co.jp wrote: I think this feature will be used in a lot of scenarios in which PITR is currently used. We have to judge which is better, we get

Re: [HACKERS] logical changeset generation v6.8

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 11:25 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-12-10 19:11:03 -0500, Robert Haas wrote: Committed #1 (again). Regarding this: + /* XXX: we could also do this unconditionally, the space is used anyway + if (copy_oid) +

Re: [HACKERS] -d option for pg_isready is broken

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 11:24 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Dec 11, 2013 at 9:35 AM, Andres Freund and...@2ndquadrant.com wrote: One use case is accessing a particular host when using DNS round robin to standbys in combination with

Re: [HACKERS] [RFC] Shouldn't we remove annoying FATAL messages from server log?

2013-12-11 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: However, it would really be useful to have an extra tag (in addition to the ERROR or FATAL) for If you're seeing this message, something has gone seriously wrong on the server. Just stuff like corruption messages, backend crashes, etc. Right, we've

Re: [HACKERS] Completing PL support for Event Triggers

2013-12-11 Thread Peter Eisentraut
On 12/11/13, 5:06 AM, Dimitri Fontaine wrote: Peter Eisentraut pete...@gmx.net writes: I think you are mistaken. My patch includes all changes between your v1 and v2 patch. I mistakenly remembered that we did remove all the is_event_trigger business from the plperl patch too, when it's not

Re: [HACKERS] stats for network traffic WIP

2013-12-11 Thread Tom Lane
Atri Sharma atri.j...@gmail.com writes: On Wed, Dec 11, 2013 at 11:12 PM, Peter Eisentraut pete...@gmx.net wrote: Is there a reason why you can't get this directly from the OS? I would say that its more of a convenience to track the usage directly from the database instead of setting up OS

Re: [HACKERS] In-Memory Columnar Store

2013-12-11 Thread Kevin Grittner
k...@rice.edu k...@rice.edu wrote: The question I have, which applies to the matview support as well, is How can we transparently substitute usage of the in-memory columnar store/matview in a SQL query?. My take on that regarding matviews is: (1)  It makes no sense to start work on this

Re: [HACKERS] Extension Templates S03E11

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 10:43 AM, Tom Lane t...@sss.pgh.pa.us wrote: So while I hear your objection to the script in catalog idea Stephen, I think we should move forward. We don't have the luxury of only applying patches where no compromise has to be made, where everyone is fully happy with

Re: [HACKERS] -d option for pg_isready is broken

2013-12-11 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Dec 11, 2013 at 11:24 AM, Tom Lane t...@sss.pgh.pa.us wrote: In general, I think the definition of these query functions ought to be what was the value of this parameter when the connection was made. As such, I'm not even sure that the

Re: [HACKERS] preserving forensic information when we freeze

2013-12-11 Thread Robert Haas
On Thu, Nov 21, 2013 at 4:51 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-11-21 15:59:35 -0500, Robert Haas wrote: * Should HeapTupleHeaderXminFrozen also check for FrozenTransactionId? It seems quite possible that people think they've delt with frozen xmin entirely after

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Gavin Flower
On 12/12/13 07:22, Gavin Flower wrote: On 12/12/13 06:22, Tom Lane wrote: I wrote: Hm. You can only take N rows from a block if there actually are at least N rows in the block. So the sampling rule I suppose you are using is select up to N rows from each sampled block --- and that is going

Re: [HACKERS] -d option for pg_isready is broken

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 2:10 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Dec 11, 2013 at 11:24 AM, Tom Lane t...@sss.pgh.pa.us wrote: In general, I think the definition of these query functions ought to be what was the value of this parameter when

Re: [HACKERS] preserving forensic information when we freeze

2013-12-11 Thread Heikki Linnakangas
On 12/11/2013 09:17 PM, Robert Haas wrote: On Thu, Nov 21, 2013 at 4:51 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-11-21 15:59:35 -0500, Robert Haas wrote: Separate patch, but yeah, something like that. If we have to mark the page all-visible, we might as well freeze it while

Re: [HACKERS] -d option for pg_isready is broken

2013-12-11 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Dec 11, 2013 at 2:10 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Well, returning /tmp on Windows is just stupid. I don't see why we should feel bad about changing that. A bug is a bug. What I was

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Kevin Grittner
Gavin Flower gavinflo...@archidevsys.co.nz wrote: For example, assume 1000 rows of 200 bytes and 1000 rows of 20 bytes, using 400 byte pages.  In the pathologically worst case, assuming maximum packing density and no page has both types: the large rows would occupy  500 pages and the smaller

[HACKERS] 9.3 reference constraint regression

2013-12-11 Thread Daniel Wood
In 9.3 I can delete the parent of a parent-child relation if the child row is an uncommitted insert and I first update the parent. USER1: drop table child; drop table parent; create table parent (i int, c char(3)); create unique index parent_idx on parent (i); insert into parent values (1,

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Peter Geoghegan
On Wed, Dec 11, 2013 at 7:41 AM, Simon Riggs si...@2ndquadrant.com wrote: That's about 2-3 days work and I know Peter can hack it. So the situation is not perfection-sought-blocking-good, this is more like fairly poor solution being driven through when a better solution is available within the

Re: [HACKERS] -d option for pg_isready is broken

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 2:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Dec 11, 2013 at 2:10 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Well, returning /tmp on Windows is just stupid. I don't see why we

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Simon Riggs
On 11 December 2013 17:57, Robert Haas robertmh...@gmail.com wrote: Extensive testing will be needed to prove that the new algorithm doesn't perform worse than the current algorithm in any important cases. Agreed, but the amount of testing seems equivalent in both cases, assuming we weren't

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Gavin Flower
On 12/12/13 08:14, Gavin Flower wrote: On 12/12/13 07:22, Gavin Flower wrote: On 12/12/13 06:22, Tom Lane wrote: I wrote: Hm. You can only take N rows from a block if there actually are at least N rows in the block. So the sampling rule I suppose you are using is select up to N rows from

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Peter Geoghegan
On Tue, Dec 10, 2013 at 4:48 PM, Peter Geoghegan p...@heroku.com wrote: Why would I even mention that to a statistician? We want guidance. But yes, I bet I could give a statistician an explanation of statistics target that they'd understand without too much trouble. Actually, I think that if

Re: [HACKERS] ANALYZE sampling is too good

2013-12-11 Thread Gavin Flower
On 12/12/13 08:31, Kevin Grittner wrote: Gavin Flower gavinflo...@archidevsys.co.nz wrote: For example, assume 1000 rows of 200 bytes and 1000 rows of 20 bytes, using 400 byte pages. In the pathologically worst case, assuming maximum packing density and no page has both types: the large rows

Re: [HACKERS] -d option for pg_isready is broken

2013-12-11 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Dec 11, 2013 at 2:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: More generally, if we do go over in 9.4 to the position that PQhost reports the host parameter and nothing but, I'm not sure that introducing a third behavior into the back branches is

Re: [HACKERS] Extension Templates S03E11

2013-12-11 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: You've got that backwards. We do have the luxury of rejecting new features until people are generally satisfied that the basic design is right. There's no overlord decreeing that this must be in 9.4. I strongly agree. PostgreSQL has succeeded

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Robert Haas
On Wed, Dec 11, 2013 at 2:37 PM, Simon Riggs si...@2ndquadrant.com wrote: On 11 December 2013 17:57, Robert Haas robertmh...@gmail.com wrote: Extensive testing will be needed to prove that the new algorithm doesn't perform worse than the current algorithm in any important cases. Agreed, but

Re: [HACKERS] autovacuum_work_mem

2013-12-11 Thread Josh Berkus
On 12/11/2013 11:37 AM, Simon Riggs wrote: On 11 December 2013 17:57, Robert Haas robertmh...@gmail.com wrote: Extensive testing will be needed to prove that the new algorithm doesn't perform worse than the current algorithm in any important cases. Agreed, but the amount of testing seems

Re: [HACKERS] Extension Templates S03E11

2013-12-11 Thread Stephen Frost
Dimitri, * Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: Stephen Frost sfr...@snowman.net writes: The extra catalog tables which store SQL scripts in text columns is one of my main objections to the as-proposed Extension Templates. I view those scripts as a poor man's definition of

Re: [HACKERS] Reference to parent query from ANY sublink

2013-12-11 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: I applied it to master and ran the regression tests, and one of the subselect tests failed. This query: SELECT '' AS six, f1 AS Correlated Field, f3 AS Second Field    FROM SUBSELECT_TBL upper    WHERE f1 IN (SELECT f2 FROM SUBSELECT_TBL WHERE

  1   2   >