Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-19 Thread Tom Lane
Fujii Masao writes: > What I'm worried about is the case where a tablespace is created > under the $PGDATA directory. What would be the sense of that? If you're concerned about whether the code handles it correctly, maybe the right solution is to add code to CREATE TABLESPACE to disallow it.

Re: [HACKERS] pl/python refactoring

2011-01-19 Thread Tom Lane
Alvaro Herrera writes: > Excerpts from Peter Eisentraut's message of mar ene 18 19:22:50 -0300 2011: >> #16: This is probably pointless because pgindent will reformat this. > pgindent used to remove useless braces around single-statement blocks, > but this behavior was removed years ago because i

Re: [HACKERS] Replication logging

2011-01-19 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > Simon Riggs wrote: > >> I'm particularly concerned that people make such changes too quickly. > >> There are many things in this area of code that need changing, but also > >> many more that do not. If we are to move forwards we need to avoid going > >>

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 18, 2011 at 5:22 PM, Pavel Stehule > wrote: >> opinion isn't strong in this topic. One or twenty useless detoasting >> isn't really significant in almost use cases (problem is thousands >> detoasting). > Yeah. Many-times-repeated detoasting is really bad, and

Re: [HACKERS] Replication logging

2011-01-19 Thread Tom Lane
Bruce Momjian writes: > Simon Riggs wrote: >> I'm particularly concerned that people make such changes too quickly. >> There are many things in this area of code that need changing, but also >> many more that do not. If we are to move forwards we need to avoid going >> one step forwards, one step

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Pavel Stehule
2011/1/19 Robert Haas : > On Tue, Jan 18, 2011 at 5:22 PM, Pavel Stehule > wrote: >> opinion isn't strong in this topic. One or twenty useless detoasting >> isn't really significant in almost use cases (problem is thousands >> detoasting). > > Yeah.  Many-times-repeated detoasting is really bad,

Re: [HACKERS] Replication logging

2011-01-19 Thread Bruce Momjian
Simon Riggs wrote: > On Tue, 2011-01-18 at 10:57 -0500, Tom Lane wrote: > > Magnus Hagander writes: > > > Is there *any* usecase for setting them differently though? > > > > I can't believe we're still engaged in painting this bikeshed. Let's > > just control it off log_connections and have done

Re: [HACKERS] limiting hint bit I/O

2011-01-19 Thread Tom Lane
Robert Haas writes: > ... So what we > want to do is write a percentage of them, in a way that guarantees > that they'll all eventually get written if people continue to access > the same data. The word "guarantee" seems quite inappropriate here, since as far as I can see this approach provides n

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Robert Haas
On Tue, Jan 18, 2011 at 5:22 PM, Pavel Stehule wrote: > opinion isn't strong in this topic. One or twenty useless detoasting > isn't really significant in almost use cases (problem is thousands > detoasting). Yeah. Many-times-repeated detoasting is really bad, and this is not the only place in t

Re: [HACKERS] limiting hint bit I/O

2011-01-19 Thread Robert Haas
On Wed, Jan 19, 2011 at 11:18 AM, Merlin Moncure wrote: > On Wed, Jan 19, 2011 at 10:44 AM, Robert Haas wrote: >> Here's a new version of the patch based on some experimentation with >> ideas I posted yesterday.  At least on my Mac laptop, this is pretty >> effective at blunting the response time

Re: [HACKERS] limiting hint bit I/O

2011-01-19 Thread Merlin Moncure
On Wed, Jan 19, 2011 at 9:13 AM, Heikki Linnakangas wrote: > On 19.01.2011 15:56, Merlin Moncure wrote: >> >> On Wed, Jan 19, 2011 at 7:57 AM, Merlin Moncure >>  wrote: >>> >>> On Tue, Jan 18, 2011 at 12:44 PM, Robert Haas >>>  wrote: On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-19 Thread Simone Aiken
> Robert > > I think the first > thing to do would be to try to come up with a reproducible test case > where clustering the tables improves performance. > On that note, is there any standard way you guys do benchmarks? > Bruce > >I think CLUSTER is a win when you are looking up multiple

Re: [HACKERS] Re: [COMMITTERS] pgsql: Log replication connections only when log_connections is on

2011-01-19 Thread Robert Haas
On Wed, Jan 19, 2011 at 11:05 AM, Simon Riggs wrote: > The question we should have asked is "Why is removing those log entries > helpful?". I shouldn't have to justify putting something back, when the > good reason for its existence was previously explained and there was no > consensus to remove i

Re: [HACKERS] limiting hint bit I/O

2011-01-19 Thread Merlin Moncure
On Wed, Jan 19, 2011 at 10:44 AM, Robert Haas wrote: > Here's a new version of the patch based on some experimentation with > ideas I posted yesterday.  At least on my Mac laptop, this is pretty > effective at blunting the response time spike for the first table > scan, and it converges to steady-

Re: [HACKERS] pl/python refactoring

2011-01-19 Thread Jan Urbański
On 19/01/11 16:35, David Fetter wrote: > On Wed, Jan 19, 2011 at 11:09:56AM +0100, Jan Urbański wrote: >> On 19/01/11 02:06, Hitoshi Harada wrote: >>> - PLy_(input|output)_tuple_funcs() in PLy_trigger_handler() is added. >>> The comment says it should check for the possibility that the >>> relation

Re: [HACKERS] Re: [COMMITTERS] pgsql: Log replication connections only when log_connections is on

2011-01-19 Thread Simon Riggs
On Wed, 2011-01-19 at 16:42 +0100, Florian Pflug wrote: > > Could you explain the failure condition you do have in mind where > logging replication connections unconditionally is beneficial? Sure. Replication drops and immediately reconnects during night. When did that happen? How many times d

Re: [HACKERS] Re: [COMMITTERS] pgsql: Log replication connections only when log_connections is on

2011-01-19 Thread Tom Lane
Simon Riggs writes: > So now we have to check *all* of the standby logs in order to check that > replication on the master is working without problems. There will be no > default capability to tie up events on the master with failures of > replication. Events occurring on multiple standbys will no

Re: [HACKERS] limiting hint bit I/O

2011-01-19 Thread Robert Haas
On Wed, Jan 19, 2011 at 8:56 AM, Merlin Moncure wrote: > Ah, I tested your patch vs stock postgres vs my patch, basically your > results are unhappily correct (mine was just a hair faster than yours > which you'd expect).  The differential was even wider on my laptop > class hardware, maybe 26%.  

Re: [HACKERS] Re: [COMMITTERS] pgsql: Log replication connections only when log_connections is on

2011-01-19 Thread Florian Pflug
On Jan19, 2011, at 16:16 , Simon Riggs wrote: > On Wed, 2011-01-19 at 09:08 -0500, Robert Haas wrote: >> On Wed, Jan 19, 2011 at 7:55 AM, Simon Riggs wrote: >>> How will we diagnose erratic connection problems now? >> >> As Heikki pointed out, the slave still logs this information, so we >> can l

Re: [HACKERS] pl/python refactoring

2011-01-19 Thread David Fetter
On Wed, Jan 19, 2011 at 11:09:56AM +0100, Jan Urbański wrote: > On 19/01/11 02:06, Hitoshi Harada wrote: > > 2011/1/19 Peter Eisentraut : > >> On mån, 2011-01-17 at 21:49 +0200, Peter Eisentraut wrote: > >>> On sön, 2011-01-02 at 12:41 +0100, Jan Urbański wrote: > Here they are. There are 16 p

Re: [HACKERS] Re: [COMMITTERS] pgsql: Log replication connections only when log_connections is on

2011-01-19 Thread Simon Riggs
On Wed, 2011-01-19 at 09:08 -0500, Robert Haas wrote: > On Wed, Jan 19, 2011 at 7:55 AM, Simon Riggs wrote: > > How will we diagnose erratic connection problems now? > > As Heikki pointed out, the slave still logs this information, so we > can look there. If that's enough, yeah, you'll have to t

Re: [HACKERS] limiting hint bit I/O

2011-01-19 Thread Heikki Linnakangas
On 19.01.2011 15:56, Merlin Moncure wrote: On Wed, Jan 19, 2011 at 7:57 AM, Merlin Moncure wrote: On Tue, Jan 18, 2011 at 12:44 PM, Robert Haas wrote: On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure wrote: a few weeks back I hacked an experimental patch that removed the hint bit action comp

Re: [HACKERS] Re: [COMMITTERS] pgsql: Log replication connections only when log_connections is on

2011-01-19 Thread Robert Haas
On Wed, Jan 19, 2011 at 7:55 AM, Simon Riggs wrote: > How will we diagnose erratic connection problems now? As Heikki pointed out, the slave still logs this information, so we can look there. If that's enough, yeah, you'll have to turn log_connections on on the master, but I don't think that wil

Re: [HACKERS] pg_dump directory archive format / parallel pg_dump

2011-01-19 Thread Joachim Wieland
On Wed, Jan 19, 2011 at 7:47 AM, Heikki Linnakangas wrote: >> Here are the latest patches all of them also rebased to current HEAD. >> Will update the commitfest app as well. > > What's the idea of storing the file sizes in the toc file? It looks like > it's not used for anything. It's part of th

Re: [HACKERS] limiting hint bit I/O

2011-01-19 Thread Merlin Moncure
On Wed, Jan 19, 2011 at 7:57 AM, Merlin Moncure wrote: > On Tue, Jan 18, 2011 at 12:44 PM, Robert Haas wrote: >> On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure wrote: >>> a few weeks back I hacked an experimental patch that removed the hint >>> bit action completely.  the results were very prem

[HACKERS] Re: [COMMITTERS] pgsql: Log replication connections only when log_connections is on

2011-01-19 Thread Simon Riggs
On Wed, 2011-01-19 at 12:55 +, Simon Riggs wrote: > How will we diagnose erratic connection problems now? The point here is that your effort to *remove* pointless log lines now means that we cannot diagnose production problems with replication unless we now *add* hundreds of pointless log lin

Re: [HACKERS] limiting hint bit I/O

2011-01-19 Thread Merlin Moncure
On Tue, Jan 18, 2011 at 12:44 PM, Robert Haas wrote: > On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure wrote: >> a few weeks back I hacked an experimental patch that removed the hint >> bit action completely.  the results were very premature and/or >> incorrect, but my initial findings suggested

[HACKERS] Re: [COMMITTERS] pgsql: Log replication connections only when log_connections is on

2011-01-19 Thread Simon Riggs
On Wed, 2011-01-19 at 13:44 +0100, Magnus Hagander wrote: > On Wed, Jan 19, 2011 at 13:36, Simon Riggs wrote: > > On Tue, 2011-01-18 at 19:04 +, Magnus Hagander wrote: > >> Log replication connections only when log_connections is on > >> > >> Previously we'd always log replication connections,

Re: [HACKERS] pg_dump directory archive format / parallel pg_dump

2011-01-19 Thread Heikki Linnakangas
On 19.01.2011 07:45, Joachim Wieland wrote: On Mon, Jan 17, 2011 at 5:38 PM, Jaime Casanova wrote: This one is the last version of this patch? if so, commitfest app should be updated to reflect that Here are the latest patches all of them also rebased to current HEAD. Will update the commitfe

[HACKERS] Re: [COMMITTERS] pgsql: Log replication connections only when log_connections is on

2011-01-19 Thread Magnus Hagander
On Wed, Jan 19, 2011 at 13:36, Simon Riggs wrote: > On Tue, 2011-01-18 at 19:04 +, Magnus Hagander wrote: >> Log replication connections only when log_connections is on >> >> Previously we'd always log replication connections, with no >> way to turn them off. > > You noted that the code was th

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-19 Thread Magnus Hagander
On Wed, Jan 19, 2011 at 06:14, Fujii Masao wrote: > On Wed, Jan 19, 2011 at 4:12 AM, Magnus Hagander wrote: >> Ah, ok. I've added the errorcode now, PFA. I also fixed an error in >> the change for result codes I broke in the last patch. github branch >> updated as usual. > > Great. Thanks for the

[HACKERS] Re: [COMMITTERS] pgsql: Log replication connections only when log_connections is on

2011-01-19 Thread Simon Riggs
On Tue, 2011-01-18 at 19:04 +, Magnus Hagander wrote: > Log replication connections only when log_connections is on > > Previously we'd always log replication connections, with no > way to turn them off. You noted that the code was there intentionally, yet you also couldn't see the reason. Th

Re: [HACKERS] ToDo: support for parameters in EXECUTE statement

2011-01-19 Thread Pavel Stehule
2011/1/19 Heikki Linnakangas : > On 19.01.2011 12:53, Pavel Stehule wrote: >> >> The EXECUTE statement doesn't support a parametrization via >> SPI_execute_with_args call and PQexecParams too. It can be a security >> issue. If somebody use a prepared statement as protection to sql >> injection, the

Re: [HACKERS] ToDo: support for parameters in EXECUTE statement

2011-01-19 Thread Heikki Linnakangas
On 19.01.2011 12:53, Pavel Stehule wrote: The EXECUTE statement doesn't support a parametrization via SPI_execute_with_args call and PQexecParams too. It can be a security issue. If somebody use a prepared statement as protection to sql injection, then all security goes out, because he has to cal

Re: [HACKERS] pl/python refactoring

2011-01-19 Thread Hitoshi Harada
2011/1/19 Jan Urbański : > On 19/01/11 02:06, Hitoshi Harada wrote: >> - -1 is used as the initial value of PLyTypeInfo.is_rowtype. Why not 0? > > See the comments in struct PLyTypeInfo: > > is_rowtype can be: -1 = not known yet (initial state); 0 = scalar > datatype; 1 = rowtype; 2 = rowtype, but

[HACKERS] ToDo: support for parameters in EXECUTE statement

2011-01-19 Thread Pavel Stehule
Hello The EXECUTE statement doesn't support a parametrization via SPI_execute_with_args call and PQexecParams too. It can be a security issue. If somebody use a prepared statement as protection to sql injection, then all security goes out, because he has to call EXECUTE without parametrization. R

Re: [HACKERS] Replication logging

2011-01-19 Thread Simon Riggs
On Tue, 2011-01-18 at 10:57 -0500, Tom Lane wrote: > Magnus Hagander writes: > > Is there *any* usecase for setting them differently though? > > I can't believe we're still engaged in painting this bikeshed. Let's > just control it off log_connections and have done. Yes, this is a waste of time

Re: [HACKERS] pl/python refactoring

2011-01-19 Thread Jan Urbański
On 19/01/11 02:06, Hitoshi Harada wrote: > 2011/1/19 Peter Eisentraut : >> On mån, 2011-01-17 at 21:49 +0200, Peter Eisentraut wrote: >>> On sön, 2011-01-02 at 12:41 +0100, Jan Urbański wrote: Here they are. There are 16 patches in total. They amount to what's currently in my refactor bra

Re: [HACKERS] Confusing comment in TransactionIdIsInProgress

2011-01-19 Thread Simon Riggs
On Tue, 2011-01-18 at 10:51 +0200, Heikki Linnakangas wrote: > On 18.01.2011 07:15, Jim Nasby wrote: > > Shouldn't the comment read "If first time through"? > > > > /* > > * If not first time through, get workspace to remember main XIDs in. We > > * malloc it permanently to avoid repe

Re: [HACKERS] pl/python refactoring

2011-01-19 Thread Jan Urbański
On 18/01/11 23:22, Peter Eisentraut wrote: > On mån, 2011-01-17 at 21:49 +0200, Peter Eisentraut wrote: >> On sön, 2011-01-02 at 12:41 +0100, Jan Urbański wrote: >>> Here they are. There are 16 patches in total. They amount to what's >>> currently in my refactor branch (and almost to what I've sent

Re: [HACKERS] Extending opfamilies for GIN indexes

2011-01-19 Thread Dimitri Fontaine
Tom Lane writes: > Oh, wait a minute: there's a bad restriction there, namely that a > contrib module could only add "loose" operators that had different > declared input types from the ones known to the core opclass. Otherwise > there'd be a conflict with the contrib module and core needing to i

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-19 Thread Martijn van Oosterhout
On Tue, Jan 18, 2011 at 10:03:01AM +0200, Heikki Linnakangas wrote: >> That isn't ever going to happen, unless you'd like to give up hash joins >> and hash aggregation on text values. > > You could canonicalize the string first in the hash function. I'm not > sure if we have all the necessary inf

Re: [HACKERS] limiting hint bit I/O

2011-01-19 Thread Andrea Suisani
On 01/19/2011 09:03 AM, Andrea Suisani wrote: On 01/18/2011 06:44 PM, Robert Haas wrote: On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure wrote: a few weeks back I hacked an experimental patch that removed the hint bit action completely. the results were very premature and/or incorrect, but my i

Re: [HACKERS] limiting hint bit I/O

2011-01-19 Thread Andrea Suisani
On 01/18/2011 06:44 PM, Robert Haas wrote: On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure wrote: a few weeks back I hacked an experimental patch that removed the hint bit action completely. the results were very premature and/or incorrect, but my initial findings suggested that hint bits migh

<    1   2