Re: [HACKERS] Synchronous replication - patch status inquiry

2010-09-03 Thread Fujii Masao
On Fri, Sep 3, 2010 at 3:36 PM, Simon Riggs wrote: >> The former seems to be more useful, but triples the number of ACK >> from the standby. I'm not sure whether its overhead is ignorable, >> especially when the distance between the master and the standby is >> very long. > > No, it doesn't. There

Re: [HACKERS] Synchronous replication - patch status inquiry

2010-09-03 Thread Simon Riggs
On Fri, 2010-09-03 at 09:55 +0300, Heikki Linnakangas wrote: > On 03/09/10 09:36, Simon Riggs wrote: > > On Fri, 2010-09-03 at 12:50 +0900, Fujii Masao wrote: > >> That design would affect what the standby should reply. If we choose > >> async/recv/fsync/replay on a per-transaction basis, the stand

Re: [HACKERS] thousand unrelated data files in pg_default tablespace

2010-09-03 Thread Pavel Stehule
hello 2010/8/31 Tom Lane : > Pavel Stehule writes: >> there is a dump from 8KB files > > Well, those certainly look like tables/indexes not temp files. > So we can rule out one theory. > > You're *certain* these aren't referenced from pg_class.relfilenode > of any of the databases in the server?

[HACKERS] regclass without error?

2010-09-03 Thread Tatsuo Ishii
Hi, Is there any way to use regclass without having ERROR? pgpool-II needs to find the oid from table name and for the purpose it issues something like "SELECT 'table_name'::regproc::oid". Problem is, if the table does not exist, an error occured and the transaction aborts. Ideally if the table d

Re: [HACKERS] Synchronous replication - patch status inquiry

2010-09-03 Thread Heikki Linnakangas
On 03/09/10 10:45, Simon Riggs wrote: On Fri, 2010-09-03 at 09:55 +0300, Heikki Linnakangas wrote: On 03/09/10 09:36, Simon Riggs wrote: On Fri, 2010-09-03 at 12:50 +0900, Fujii Masao wrote: That design would affect what the standby should reply. If we choose async/recv/fsync/replay on a per-t

[HACKERS] plan time of MASSIVE partitioning ...

2010-09-03 Thread PostgreSQL - Hans-Jürgen Schönig
hello everybody, we came across an issue which turned out to be more serious than previously expected. imagine a system with, say, 1000 partitions (heavily indexed) or so. the time taken by the planner is already fairly heavy in this case. i tried this one with 5000 unindexed tables (just one c

Re: [HACKERS] Synchronization levels in SR

2010-09-03 Thread Boszormenyi Zoltan
Hi, Dimitri Fontaine írta: > Simon Riggs writes: > >> Seems strange. If you have 2 standbys and you say you would like node1 >> to be the preferred candidate, then you load it so heavily that a remote >> server with by-definition much larger network delay responds first, then >> I say your pre

Re: [HACKERS] Synchronous replication - patch status inquiry

2010-09-03 Thread Simon Riggs
On Fri, 2010-09-03 at 12:33 +0300, Heikki Linnakangas wrote: > On 03/09/10 10:45, Simon Riggs wrote: > > On Fri, 2010-09-03 at 09:55 +0300, Heikki Linnakangas wrote: > >> On 03/09/10 09:36, Simon Riggs wrote: > >>> On Fri, 2010-09-03 at 12:50 +0900, Fujii Masao wrote: > That design would affec

Re: [HACKERS] Synchronous replication - patch status inquiry

2010-09-03 Thread Heikki Linnakangas
On 03/09/10 13:20, Simon Riggs wrote: On Fri, 2010-09-03 at 12:33 +0300, Heikki Linnakangas wrote: On 03/09/10 10:45, Simon Riggs wrote: On Fri, 2010-09-03 at 09:55 +0300, Heikki Linnakangas wrote: On 03/09/10 09:36, Simon Riggs wrote: On Fri, 2010-09-03 at 12:50 +0900, Fujii Masao wrote: Th

Re: Interruptible sleeps (was Re: [HACKERS] CommitFest 2009-07: Yay, Kevin! Thanks, reviewers!)

2010-09-03 Thread Heikki Linnakangas
On 02/09/10 23:13, Tom Lane wrote: The WaitLatch ...timeout API could use a bit of refinement. I'd suggest defining negative timeout as meaning wait forever, so that timeout = 0 can be used for "check but don't wait". Also, it seems like the function shouldn't just return void but should return

Re: [HACKERS] thousand unrelated data files in pg_default tablespace

2010-09-03 Thread Heikki Linnakangas
On 03/09/10 11:16, Pavel Stehule wrote: 2010/8/31 Tom Lane: Pavel Stehule writes: there is a dump from 8KB files Well, those certainly look like tables/indexes not temp files. So we can rule out one theory. You're *certain* these aren't referenced from pg_class.relfilenode of any of the dat

[HACKERS] Streaming a base backup from master

2010-09-03 Thread Heikki Linnakangas
It's been discussed before that it would be cool if you could stream a new base backup from the master server, via libpq. That way you would not need low-level filesystem access to initialize a new standby. Magnus mentioned today that he started hacking on that, and coincidentally I just start

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Thom Brown
On 3 September 2010 12:19, Heikki Linnakangas wrote: > TODO: > > * We need a smarter way to do pg_start/stop_backup() with this. At the > moment, you can only have one backup running at a time, but we shouldn't > have that limitation with this built-in mechanism. Would it be possible to not requi

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Dave Page
On Fri, Sep 3, 2010 at 12:19 PM, Heikki Linnakangas wrote: > Here's a WIP patch. It adds a new "TAKE_BACKUP" command to the replication > command set. Upon receiving that command, the master starts a COPY, and > streams a tarred copy of the data directory to the client. The patch > includes a simp

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Magnus Hagander
On Fri, Sep 3, 2010 at 13:19, Heikki Linnakangas wrote: > It's been discussed before that it would be cool if you could stream a new > base backup from the master server, via libpq. That way you would not need > low-level filesystem access to initialize a new standby. > > Magnus mentioned today th

Re: [HACKERS] Synchronization levels in SR

2010-09-03 Thread Fujii Masao
On Fri, Sep 3, 2010 at 6:43 PM, Boszormenyi Zoltan wrote: > In my patch, when the transactions were waiting for ack from > the standby, they have already released all their locks, the wait > happened at the latest possible point in CommitTransaction(). > > In Fujii's patch (I am looking at synch_r

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Magnus Hagander
On Fri, Sep 3, 2010 at 13:25, Thom Brown wrote: > On 3 September 2010 12:19, Heikki Linnakangas > wrote: >> TODO: >> >> * We need a smarter way to do pg_start/stop_backup() with this. At the >> moment, you can only have one backup running at a time, but we shouldn't >> have that limitation with t

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Thom Brown
On 3 September 2010 12:30, Magnus Hagander wrote: > On Fri, Sep 3, 2010 at 13:25, Thom Brown wrote: >> On 3 September 2010 12:19, Heikki Linnakangas >> wrote: >>> TODO: >>> >>> * We need a smarter way to do pg_start/stop_backup() with this. At the >>> moment, you can only have one backup running

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Heikki Linnakangas
On 03/09/10 14:25, Thom Brown wrote: On 3 September 2010 12:19, Heikki Linnakangas wrote: TODO: * We need a smarter way to do pg_start/stop_backup() with this. At the moment, you can only have one backup running at a time, but we shouldn't have that limitation with this built-in mechanism.

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Heikki Linnakangas
On 03/09/10 14:28, Dave Page wrote: On Fri, Sep 3, 2010 at 12:19 PM, Heikki Linnakangas wrote: Here's a WIP patch. It adds a new "TAKE_BACKUP" command to the replication command set. Upon receiving that command, the master starts a COPY, and streams a tarred copy of the data directory to the c

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Magnus Hagander
On Fri, Sep 3, 2010 at 13:48, Heikki Linnakangas wrote: > On 03/09/10 14:28, Dave Page wrote: >> >> On Fri, Sep 3, 2010 at 12:19 PM, Heikki Linnakangas >>  wrote: >>> >>> Here's a WIP patch. It adds a new "TAKE_BACKUP" command to the >>> replication >>> command set. Upon receiving that command, t

Re: [HACKERS] Cost estimates for parameterized paths

2010-09-03 Thread Robert Haas
On Thu, Sep 2, 2010 at 5:31 PM, Tom Lane wrote: > Awhile back I ranted about replacing the planner's concept of inner > indexscans with a more generalized notion of "parameterized paths": > http://archives.postgresql.org/pgsql-hackers/2009-10/msg00994.php > > The executor fixes for that are done,

Re: [HACKERS] plan time of MASSIVE partitioning ...

2010-09-03 Thread Stephen Frost
* PostgreSQL - Hans-Jürgen Schönig (postg...@cybertec.at) wrote: > did anybody think of a solution to this problem. > or more precisely: can there be a solution to this problem? Please post to the correct list (-performance) and provide information like PG version, postgresql.conf, the actual tabl

Re: [HACKERS] plan time of MASSIVE partitioning ...

2010-09-03 Thread PostgreSQL - Hans-Jürgen Schönig
On Sep 3, 2010, at 2:04 PM, Stephen Frost wrote: > * PostgreSQL - Hans-Jürgen Schönig (postg...@cybertec.at) wrote: >> did anybody think of a solution to this problem. >> or more precisely: can there be a solution to this problem? > > Please post to the correct list (-performance) and provide in

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Greg Stark
On Fri, Sep 3, 2010 at 12:19 PM, Heikki Linnakangas wrote: > * We need a smarter way to do pg_start/stop_backup() with this. At the > moment, you can only have one backup running at a time, but we shouldn't > have that limitation with this built-in mechanism. Well there's no particular reason we

Re: [HACKERS] plan time of MASSIVE partitioning ...

2010-09-03 Thread Stephen Frost
* PostgreSQL - Hans-Jürgen Schönig (postg...@cybertec.at) wrote: > this seems like more a developer question to me than a pre performance one. > it is not related to the table structure at all - it is basically an issue > with incredibly large inheritance lists. > it applies to postgres 9 and most

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Heikki Linnakangas
On 03/09/10 15:16, Greg Stark wrote: On Fri, Sep 3, 2010 at 12:19 PM, Heikki Linnakangas wrote: * We need a smarter way to do pg_start/stop_backup() with this. At the moment, you can only have one backup running at a time, but we shouldn't have that limitation with this built-in mechanism. W

Re: [HACKERS] thousand unrelated data files in pg_default tablespace

2010-09-03 Thread Pavel Stehule
2010/9/3 Heikki Linnakangas : > On 03/09/10 11:16, Pavel Stehule wrote: >> >> 2010/8/31 Tom Lane: >>> >>> Pavel Stehule  writes: there is a dump from 8KB files >>> >>> Well, those certainly look like tables/indexes not temp files. >>> So we can rule out one theory. >>> >>> You're *certain

Re: [HACKERS] Synchronization levels in SR

2010-09-03 Thread Boszormenyi Zoltan
Fujii Masao írta: > On Fri, Sep 3, 2010 at 6:43 PM, Boszormenyi Zoltan wrote: > >> In my patch, when the transactions were waiting for ack from >> the standby, they have already released all their locks, the wait >> happened at the latest possible point in CommitTransaction(). >> >> In Fujii's

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Robert Haas
On Fri, Sep 3, 2010 at 7:28 AM, Dave Page wrote: > On Fri, Sep 3, 2010 at 12:19 PM, Heikki Linnakangas > wrote: >> Here's a WIP patch. It adds a new "TAKE_BACKUP" command to the replication >> command set. Upon receiving that command, the master starts a COPY, and >> streams a tarred copy of the

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Magnus Hagander
On Fri, Sep 3, 2010 at 15:24, Robert Haas wrote: > On Fri, Sep 3, 2010 at 7:28 AM, Dave Page wrote: >> On Fri, Sep 3, 2010 at 12:19 PM, Heikki Linnakangas >> wrote: >>> Here's a WIP patch. It adds a new "TAKE_BACKUP" command to the replication >>> command set. Upon receiving that command, the ma

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Dave Page
On Fri, Sep 3, 2010 at 2:24 PM, Robert Haas wrote: > On Fri, Sep 3, 2010 at 7:28 AM, Dave Page wrote: >> On Fri, Sep 3, 2010 at 12:19 PM, Heikki Linnakangas >> wrote: >>> Here's a WIP patch. It adds a new "TAKE_BACKUP" command to the replication >>> command set. Upon receiving that command, the

Re: [HACKERS] plan time of MASSIVE partitioning ...

2010-09-03 Thread Robert Haas
2010/9/3 PostgreSQL - Hans-Jürgen Schönig : > i tried this one with 5000 unindexed tables (just one col): > > test=# \timing > Timing is on. > test=# prepare x(int4) AS select * from t_data order by id desc; > PREPARE > Time: 361.552 ms > > you will see similar or higher runtimes in case of 500 par

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Robert Haas
On Fri, Sep 3, 2010 at 9:26 AM, Dave Page wrote: > rsync? Might be easier to use that from day 1 (well, day 2) than to > retrofit later. I'm not sure we want to depend on an external utility like that, particularly one that users may not have installed. And I'm not sure if that can be made to wo

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Dave Page
On Fri, Sep 3, 2010 at 2:29 PM, Robert Haas wrote: > On Fri, Sep 3, 2010 at 9:26 AM, Dave Page wrote: >> rsync? Might be easier to use that from day 1 (well, day 2) than to >> retrofit later. > > I'm not sure we want to depend on an external utility like that, > particularly one that users may no

Re: [HACKERS] regclass without error?

2010-09-03 Thread Robert Haas
On Fri, Sep 3, 2010 at 4:28 AM, Tatsuo Ishii wrote: > Is there any way to use regclass without having ERROR? > > pgpool-II needs to find the oid from table name and for the purpose it > issues something like "SELECT 'table_name'::regproc::oid". Problem is, > if the table does not exist, an error o

Re: [HACKERS] Path question

2010-09-03 Thread Hans-Jürgen Schönig
On Sep 2, 2010, at 1:20 AM, Robert Haas wrote: > On Sep 1, 2010, at 10:21 AM, Greg Stark wrote: >> For what it's worth I disagree with Tom. I think this is a situation >> where we need *both* types of solution. Ideally we will be able to use >> a plain Append node for cases where we know the rel

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Robert Haas
On Fri, Sep 3, 2010 at 9:32 AM, Dave Page wrote: > No, I agree we don't want an external dependency (I was just bleating > about needing tar on Windows). I was assuming/hoping there's a > librsync somewhere... The rsync code itself is not modular, I believe. I think the author thereof kind of to

Re: [HACKERS] english parser in text search: support for multiple words in the same position

2010-09-03 Thread Robert Haas
On Wed, Sep 1, 2010 at 2:42 AM, Sushant Sinha wrote: > I have attached a patch that emits parts of a host token, a url token, > an email token and a file token. Further, it makes sure that a > host/url/email/file token and the first part-token are at the same > position in tsvector. You should pr

Re: [HACKERS] regclass without error?

2010-09-03 Thread Tatsuo Ishii
> You can write a query against the system catalog tables. That was pretty hard than I though(schema search path etc.). > Or you could > install a function that wraps the regclass cast in an exception > handler. That requires users to install the function. Annoying for users. -- Tatsuo Ishii SRA

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > The rsync code itself is not modular, I believe. I think the author > thereof kind of took the approach of placing efficiency before all. Yeah, I looked into this when discussing this same concept at PGCon with folks. There doesn't appear to be a go

Re: Interruptible sleeps (was Re: [HACKERS] CommitFest 2009-07: Yay, Kevin! Thanks, reviewers!)

2010-09-03 Thread Tom Lane
Fujii Masao writes: > On Fri, Sep 3, 2010 at 11:08 AM, Tom Lane wrote: >> elog(FATAL) is *certainly* not a better idea.  I think there's really >> nothing that can be done, you just have to silently ignore the error. > Hmm.. some functions called by a signal handler use elog(FATAL), e.g., > Reco

Re: [HACKERS] regclass without error?

2010-09-03 Thread Marko Tiikkaja
On 2010-09-03 4:46 PM, Tatsuo Ishii wrote: You can write a query against the system catalog tables. That was pretty hard than I though(schema search path etc.). Or you could install a function that wraps the regclass cast in an exception handler. That requires users to install the function.

Re: [HACKERS] regclass without error?

2010-09-03 Thread David Fetter
On Fri, Sep 03, 2010 at 05:10:44PM +0300, Marko Tiikkaja wrote: > On 2010-09-03 4:46 PM, Tatsuo Ishii wrote: > >>You can write a query against the system catalog tables. > > > >That was pretty hard than I though(schema search path etc.). > > > >>Or you could > >>install a function that wraps the re

Re: [HACKERS] regclass without error?

2010-09-03 Thread Tom Lane
Tatsuo Ishii writes: > Is there any way to use regclass without having ERROR? > pgpool-II needs to find the oid from table name and for the purpose it > issues something like "SELECT 'table_name'::regproc::oid". Problem is, > if the table does not exist, an error occured and the transaction > abo

Re: [HACKERS] regclass without error?

2010-09-03 Thread Pavel Golub
Hello, guys. You wrote: TI> Hi, TI> Is there any way to use regclass without having ERROR? TI> pgpool-II needs to find the oid from table name and for the purpose it TI> issues something like "SELECT 'table_name'::regproc::oid". Problem is, TI> if the table does not exist, an error occured and

Re: [HACKERS] plan time of MASSIVE partitioning ...

2010-09-03 Thread Tom Lane
=?iso-8859-1?Q?PostgreSQL_-_Hans-J=FCrgen_Sch=F6nig?= writes: > imagine a system with, say, 1000 partitions (heavily indexed) or so. the time > taken by the planner is already fairly heavy in this case. As the fine manual points out, the current scheme for managing partitioned tables isn't inte

Re: [HACKERS] plan time of MASSIVE partitioning ...

2010-09-03 Thread PostgreSQL - Hans-Jürgen Schönig
On Sep 3, 2010, at 4:40 PM, Tom Lane wrote: > =?iso-8859-1?Q?PostgreSQL_-_Hans-J=FCrgen_Sch=F6nig?= > writes: >> imagine a system with, say, 1000 partitions (heavily indexed) or so. the >> time taken by the planner is already fairly heavy in this case. > > As the fine manual points out, the c

Re: Interruptible sleeps (was Re: [HACKERS] CommitFest 2009-07: Yay, Kevin! Thanks, reviewers!)

2010-09-03 Thread Tom Lane
Heikki Linnakangas writes: > On 02/09/10 23:13, Tom Lane wrote: >>> (Yeah, I realize the caller >>> could look into the latch to find that out, but callers really ought to >>> treat latches as opaque structs.) > Hmm, maybe we need a TestLatch function to check if a latch is set. +1. It could be

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Tom Lane
Heikki Linnakangas writes: > On 03/09/10 15:16, Greg Stark wrote: >> On Fri, Sep 3, 2010 at 12:19 PM, Heikki Linnakangas >> wrote: >>> * We need a smarter way to do pg_start/stop_backup() with this. At the >>> moment, you can only have one backup running at a time, but we shouldn't >>> have that

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Kevin Grittner
Stephen Frost wrote: > there's a heck of alot of complexity there that we *don't* need. > rsync is a great tool, don't get me wrong, but let's not try to go > over our heads here. Right -- among other things, it checks for portions of a new file which match the old file at a different location

[HACKERS] Windows Tools

2010-09-03 Thread David Fetter
Hello, At work, I've been tasked with providing some Windows connection libraries, etc. Are people still using MinGW, or should I just recommend using a proprietary toolkit to do the builds? If so, which one(s)? Cheers, David (Windows n00b) -- David Fetter http://fetter.org/ Phone: +1 415 235

Re: [HACKERS] Windows Tools

2010-09-03 Thread Dave Page
On Fri, Sep 3, 2010 at 4:11 PM, David Fetter wrote: > Hello, > > At work, I've been tasked with providing some Windows connection > libraries, etc.  Are people still using MinGW, or should I just > recommend using a proprietary toolkit to do the builds?  If so, which > one(s)? Visual C++ 2005 for

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Stephen Frost
Kevin, * Kevin Grittner (kevin.gritt...@wicourts.gov) wrote: > While 1GB granularity would be OK, I doubt it's optimal; I think CRC > checks for smaller chunks might be worthwhile. My gut feel is that > somewhere in the 64kB to 1MB range would probably be optimal for us, > although the "sweet spo

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Thom Brown
On 3 September 2010 16:01, Tom Lane wrote: > Heikki Linnakangas writes: >> On 03/09/10 15:16, Greg Stark wrote: >>> On Fri, Sep 3, 2010 at 12:19 PM, Heikki Linnakangas >>>  wrote: * We need a smarter way to do pg_start/stop_backup() with this. At the moment, you can only have one backu

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Kevin Grittner
>Stephen Frost wrote: > We have something much better, called WAL. If people want to keep > their backup current, they should use that after getting the base > backup up and working. Unless you want to provide support for Point In Time Recovery without excessive recovery times. > We don't n

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Heikki Linnakangas
On 03/09/10 18:01, Tom Lane wrote: Heikki Linnakangas writes: On 03/09/10 15:16, Greg Stark wrote: On Fri, Sep 3, 2010 at 12:19 PM, Heikki Linnakangas wrote: * We need a smarter way to do pg_start/stop_backup() with this. At the moment, you can only have one backup running at a time, but w

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Robert Haas
On Fri, Sep 3, 2010 at 11:20 AM, Stephen Frost wrote: > Kevin, > > * Kevin Grittner (kevin.gritt...@wicourts.gov) wrote: >> While 1GB granularity would be OK, I doubt it's optimal; I think CRC >> checks for smaller chunks might be worthwhile.  My gut feel is that >> somewhere in the 64kB to 1MB ra

Re: Interruptible sleeps (was Re: [HACKERS] CommitFest 2009-07: Yay, Kevin! Thanks, reviewers!)

2010-09-03 Thread Robert Haas
On Fri, Sep 3, 2010 at 10:07 AM, Tom Lane wrote: > Fujii Masao writes: >> On Fri, Sep 3, 2010 at 11:08 AM, Tom Lane wrote: >>> elog(FATAL) is *certainly* not a better idea.  I think there's really >>> nothing that can be done, you just have to silently ignore the error. > >> Hmm.. some functions

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Tom Lane
"Kevin Grittner" writes: >> Stephen Frost wrote: >> In any case, it's certainly not something required for an initial >> implementation.. > No disagreement there; but sometimes it pays to know where you might > want to go, so you don't do something to make further development in > that directio

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Robert Haas
On Fri, Sep 3, 2010 at 11:47 AM, Tom Lane wrote: > "Kevin Grittner" writes: >>> Stephen Frost wrote: >>> In any case, it's certainly not something required for an initial >>> implementation.. > >> No disagreement there; but sometimes it pays to know where you might >> want to go, so you don't do

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread David Blewett
On Fri, Sep 3, 2010 at 11:47 AM, Tom Lane wrote: > IOW, what I'd like to see is protocol extensions that allow an external > copy of rsync to be invoked; not build in rsync, or tar, or anything > else that we could get off-the-shelf. Personally, I would love to see protocol-level compression adde

Re: [HACKERS] regclass without error?

2010-09-03 Thread David E. Wheeler
On Sep 3, 2010, at 7:31 AM, Tom Lane wrote: > I don't think the cast should act that way, but I could see providing a > separate conversion function that returns 0 ... or perhaps better NULL > ... if no match. +1 I could use this in pgTAP. David -- Sent via pgsql-hackers mailing list (pgsql-ha

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > IOW, what I'd like to see is protocol extensions that allow an external > copy of rsync to be invoked; not build in rsync, or tar, or anything > else that we could get off-the-shelf. I'd much rather use an existing library to implement it than call out to s

Re: Interruptible sleeps (was Re: [HACKERS] CommitFest 2009-07: Yay, Kevin! Thanks, reviewers!)

2010-09-03 Thread Tom Lane
Robert Haas writes: > On Fri, Sep 3, 2010 at 10:07 AM, Tom Lane wrote: >> [ shrug... ]  I stated before that the Hot Standby patch is doing >> utterly unsafe things in signal handlers.  Simon rejected that. >> I am waiting for irrefutable evidence to emerge from the field >> (and am very confiden

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Kevin Grittner
Tom Lane wrote: > what I'd like to see is protocol extensions that allow an external > copy of rsync to be invoked; not build in rsync, or tar, or > anything else that we could get off-the-shelf. The complexities of dealing with properly invoking rsync externally could well require more code a

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Heikki Linnakangas
On 03/09/10 19:09, Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: IOW, what I'd like to see is protocol extensions that allow an external copy of rsync to be invoked; not build in rsync, or tar, or anything else that we could get off-the-shelf. I'd much rather use an existing libr

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Heikki Linnakangas
On 03/09/10 18:53, David Blewett wrote: On Fri, Sep 3, 2010 at 11:47 AM, Tom Lane wrote: IOW, what I'd like to see is protocol extensions that allow an external copy of rsync to be invoked; not build in rsync, or tar, or anything else that we could get off-the-shelf. Personally, I would love

Re: [HACKERS] Windows Tools

2010-09-03 Thread Magnus Hagander
On Fri, Sep 3, 2010 at 17:18, Dave Page wrote: > On Fri, Sep 3, 2010 at 4:11 PM, David Fetter wrote: >> Hello, >> >> At work, I've been tasked with providing some Windows connection >> libraries, etc.  Are people still using MinGW, or should I just >> recommend using a proprietary toolkit to do t

Re: Interruptible sleeps (was Re: [HACKERS] CommitFest 2009-07: Yay, Kevin! Thanks, reviewers!)

2010-09-03 Thread Robert Haas
On Fri, Sep 3, 2010 at 12:10 PM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Sep 3, 2010 at 10:07 AM, Tom Lane wrote: >>> [ shrug... ]  I stated before that the Hot Standby patch is doing >>> utterly unsafe things in signal handlers.  Simon rejected that. >>> I am waiting for irrefutable ev

Re: [HACKERS] regclass without error?

2010-09-03 Thread Alvaro Herrera
Excerpts from David Fetter's message of vie sep 03 10:21:00 -0400 2010: > > How about using a SAVEPOINT before the cast? Wouldn't fail your > > transaction.. > > For unattended operation, there are some issues: > > * Generating appropriate SAVEPOINT names > * Keeping track of same > * Detecting

Re: [HACKERS] regclass without error?

2010-09-03 Thread David Fetter
On Fri, Sep 03, 2010 at 01:28:15PM -0400, Alvaro Herrera wrote: > Excerpts from David Fetter's message of vie sep 03 10:21:00 -0400 2010: > > > > How about using a SAVEPOINT before the cast? Wouldn't fail your > > > transaction.. > > > > For unattended operation, there are some issues: > > > >

Re: [HACKERS] regclass without error?

2010-09-03 Thread Alvaro Herrera
Excerpts from David Fetter's message of vie sep 03 13:31:00 -0400 2010: > We have no mechanism to do this, or any other check, during a bulk > load. It'd be a great feature to have :) I'm not sure what kind of bulk load you are talking about, nor what does it have to do with pgpool-II. -- Álva

Re: Interruptible sleeps (was Re: [HACKERS] CommitFest 2009-07: Yay, Kevin! Thanks, reviewers!)

2010-09-03 Thread Heikki Linnakangas
On 03/09/10 17:51, Tom Lane wrote: Heikki Linnakangas writes: On 02/09/10 23:13, Tom Lane wrote: Also, using sig_atomic_t for owner_pid is entirely not sane. Hmm, true, it doesn't need to be set from signal handler, but is there an atomicity problem if one process calls ReleaseLatch while a

Re: [HACKERS] Cost estimates for parameterized paths

2010-09-03 Thread Tom Lane
Robert Haas writes: > On Thu, Sep 2, 2010 at 5:31 PM, Tom Lane wrote: >> The best idea I can come up with at the moment is to compute "best case" >> and "worst case" costs for a parameterized path, > Interestingly, I previously proposed almost exactly this approach to > handle a couple of other

Re: Interruptible sleeps (was Re: [HACKERS] CommitFest 2009-07: Yay, Kevin! Thanks, reviewers!)

2010-09-03 Thread Tom Lane
Heikki Linnakangas writes: > On 03/09/10 17:51, Tom Lane wrote: >> If there is *any* possibility of that happening then you have far worse >> problems than whether the field is atomically readable or not: the >> behavior will be unpredictable at just slightly larger timescales. > Each Walsender n

[HACKERS] ps buffer is incorrectly padded on the (latest) OS X

2010-09-03 Thread Alexey Klyukin
Hi, I always wondered why ps ax|grep postgres shows several extra blank lines after the process name, i.e. 972 ?? Ss 0:00.69 postgres: writer process 973 ?? Ss 0:00.51 postgres: wal writer process (I put newlines instead of spaces there). By looking into the code I've found

Re: [HACKERS] Windows Tools

2010-09-03 Thread David Fetter
On Fri, Sep 03, 2010 at 06:24:06PM +0200, Magnus Hagander wrote: > On Fri, Sep 3, 2010 at 17:18, Dave Page wrote: > > On Fri, Sep 3, 2010 at 4:11 PM, David Fetter wrote: > >> Hello, > >> > >> At work, I've been tasked with providing some Windows connection > >> libraries, etc.  Are people still u

Re: Interruptible sleeps (was Re: [HACKERS] CommitFest 2009-07: Yay, Kevin! Thanks, reviewers!)

2010-09-03 Thread Heikki Linnakangas
On 03/09/10 21:16, Tom Lane wrote: Heikki Linnakangas writes: WaitLatch had to set the pid on the Latch struct to allow other processes to send the signal. Another process could call SetLatch and read the pid field, while WaitLatch is just setting it. I think we'll have to put a spinlock there,

Re: Interruptible sleeps (was Re: [HACKERS] CommitFest 2009-07: Yay, Kevin! Thanks, reviewers!)

2010-09-03 Thread Tom Lane
Heikki Linnakangas writes: > On 03/09/10 21:16, Tom Lane wrote: >> It's probably not too unreasonable to assume that pid_t assignment is >> atomic. But I'm still thinking that we have bigger problems than that >> if there are really cases where SetLatch can execute at approximately >> the same ti

Re: [HACKERS] ps buffer is incorrectly padded on the (latest) OS X

2010-09-03 Thread Tom Lane
Alexey Klyukin writes: > I always wondered why ps ax|grep postgres shows several extra blank lines > after the process name, i.e. AFAIR it's always done that on OSX. I thought we'd tried the '\0' padding way back when and it didn't work nicely, but maybe Apple fixed that.

Re: Interruptible sleeps (was Re: [HACKERS] CommitFest 2009-07: Yay, Kevin! Thanks, reviewers!)

2010-09-03 Thread Ron Mayer
Tom Lane wrote: > Robert Haas writes: >> On Fri, Sep 3, 2010 at 10:07 AM, Tom Lane wrote: >>> [ shrug... ] I stated before that the Hot Standby patch is doing >>> utterly unsafe things in signal handlers. Simon rejected that. >>> I am waiting for irrefutable evidence to emerge from the field >>

Re: [HACKERS] returning multiple result sets from a stored procedure

2010-09-03 Thread Merlin Moncure
On Thu, Sep 2, 2010 at 7:17 PM, Alvaro Herrera wrote: > Excerpts from John Adams's message of jue sep 02 18:25:45 -0400 2010: >> I noticed in postgres you cannot return multiple result sets from a stored >> procedure (surprisingly as it looks like a very good dbms). > > If you're really intent abo

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread Martijn van Oosterhout
On Fri, Sep 03, 2010 at 09:56:12AM -0400, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: > > The rsync code itself is not modular, I believe. I think the author > > thereof kind of took the approach of placing efficiency before all. > > Yeah, I looked into this when discussin

[HACKERS] The other major HS TODO: standby promotion

2010-09-03 Thread Josh Berkus
All, As far as I'm concerned, the other big "missing feature" for HS is the ability to promote standbys to become the new master. If we had that feature, then HS can be the backbone of a large-scale PostgreSQL "cloud"; if we don't have it, then HS does not contribute very much to scalability bey

Re: [HACKERS] returning multiple result sets from a stored procedure

2010-09-03 Thread Josh Berkus
> I noticed in postgres you cannot return multiple result sets from a > stored procedure (surprisingly as it looks like a very good dbms). That feature has been on the TODO list for years. However, nobody has stepped forward to either write it, or to fund working on it. If your company has prog

Re: Interruptible sleeps (was Re: [HACKERS] CommitFest 2009-07: Yay, Kevin! Thanks, reviewers!)

2010-09-03 Thread Robert Haas
On Fri, Sep 3, 2010 at 3:11 PM, Ron Mayer wrote: > Tom Lane wrote: >> Robert Haas writes: >>> On Fri, Sep 3, 2010 at 10:07 AM, Tom Lane wrote: [ shrug... ]  I stated before that the Hot Standby patch is doing utterly unsafe things in signal handlers.  Simon rejected that. I am wai

Re: [HACKERS] returning multiple result sets from a stored procedure

2010-09-03 Thread Tom Lane
Josh Berkus writes: >> I noticed in postgres you cannot return multiple result sets from a >> stored procedure (surprisingly as it looks like a very good dbms). > That feature has been on the TODO list for years. However, nobody has > stepped forward to either write it, or to fund working on it.

Re: Interruptible sleeps (was Re: [HACKERS] CommitFest 2009-07: Yay, Kevin! Thanks, reviewers!)

2010-09-03 Thread Heikki Linnakangas
On 03/09/10 19:38, Robert Haas wrote: On Fri, Sep 3, 2010 at 12:10 PM, Tom Lane wrote: Robert Haas writes: On Fri, Sep 3, 2010 at 10:07 AM, Tom Lane wrote: [ shrug... ] I stated before that the Hot Standby patch is doing utterly unsafe things in signal handlers. Simon rejected that. I am

Re: [HACKERS] Cost estimates for parameterized paths

2010-09-03 Thread Robert Haas
On Fri, Sep 3, 2010 at 2:04 PM, Tom Lane wrote: > On reflection I think that for parameterized paths the problem won't be > too bad, because (a) we'll ignore parameterized paths except when > considering a join to the right outer rel, so their presence in the > rel's pathlist won't cost much other

Re: [HACKERS] Streaming a base backup from master

2010-09-03 Thread David Blewett
On Fri, Sep 3, 2010 at 12:23 PM, Heikki Linnakangas wrote: > On 03/09/10 18:53, David Blewett wrote: >> >> On Fri, Sep 3, 2010 at 11:47 AM, Tom Lane  wrote: >>> >>> IOW, what I'd like to see is protocol extensions that allow an external >>> copy of rsync to be invoked; not build in rsync, or tar,

Re: OT: OFF TOPIC: [HACKERS] returning multiple result sets from a stored procedure

2010-09-03 Thread Josh Berkus
John, > I honestly do not mean any offence, just out of curiosity. > > If you guys care about money and time why would you spend the best years > of your life basically copying commercial products for free? We don't do it to copy commercial products. We do it to build something better than them

Re: OT: OFF TOPIC: [HACKERS] returning multiple result sets from a stored procedure

2010-09-03 Thread Josh Berkus
On 9/3/10 2:20 PM, Josh Berkus wrote: > However, if you look at any mature, large open > source project you will find that at least 1/4 of its code contributors > are paid to work on the project by *someone*, and that those paid > developers account for 70% to 95% of the code. Relevant link for th

Re: Interruptible sleeps (was Re: [HACKERS] CommitFest 2009-07: Yay, Kevin! Thanks, reviewers!)

2010-09-03 Thread Robert Haas
On Fri, Sep 3, 2010 at 4:20 PM, Heikki Linnakangas wrote: > Maybe that's ok, if I'm reading the deadlock checker code correctly, it also > calls semop() to increment the another process' semaphore, and the deadlock > checker can be invoked from a signal handler while in semop() to wait on our > pr

Re: Interruptible sleeps (was Re: [HACKERS] CommitFest 2009-07: Yay, Kevin! Thanks, reviewers!)

2010-09-03 Thread Tom Lane
Heikki Linnakangas writes: > A safer approach would be to just PGSemaphoreUnlock() in the signal > handler, and do all the other processing outside it. I don't see any particularly good reason to assume that PGSemaphoreUnlock is safe either: you're still talking about nested semop operations. T

Re: [HACKERS] Cost estimates for parameterized paths

2010-09-03 Thread Tom Lane
Robert Haas writes: > On Fri, Sep 3, 2010 at 2:04 PM, Tom Lane wrote: >> On reflection I think that for parameterized paths the problem won't be >> too bad, because (a) we'll ignore parameterized paths except when >> considering a join to the right outer rel, so their presence in the >> rel's pat

Re: Interruptible sleeps (was Re: [HACKERS] CommitFest 2009-07: Yay, Kevin! Thanks, reviewers!)

2010-09-03 Thread Tom Lane
Robert Haas writes: > Color me confused; I may need to backpedal rapidly here. I had > thought that what Tom was complaining about was the fact that the > signal handler was taking LWLocks, which I would have thought to be > totally unsafe. Well, it's unsafe if the signal could interrupt mainlin

Re: [HACKERS] Windows Tools

2010-09-03 Thread Peter Geoghegan
Hi David I seem to recall the last time that I built Postgres on windows using VC++, there was a whole load of compiler warnings, because MS have taken it upon themselves to deprecate various C std lib functions (in particular, string.h functions). You're supposed to use their non-standard variant

[HACKERS] lexing small ints as int2

2010-09-03 Thread Alvaro Herrera
Hi, I'm researching if smallint can be made a higher-class citizen of our type system than currently. Does anyone know where to find the discussion refered to here? http://archives.postgresql.org/pgsql-hackers/2008-10/msg01485.php I did some searches on the archives but no keywords I search for

Re: [HACKERS] "serializable" in comments and names

2010-09-03 Thread Kevin Grittner
Tom Lane wrote: > +1 for adding parens; we might want to make a function of it > someday. > How about IsolationUsesXactSnapshot Patch attached. Joe said that he'll review this weekend and probably commit in a day or two if there are no objections. -Kevin *** a/src/backend/catalog/index.c

  1   2   >