Re: [HACKERS] pgbench \for or similar loop

2011-04-22 Thread Greg Smith
Alvaro Herrera wrote: Why do we have pgbench at all in the first place? Surely we could rewrite it in plpgsql with proper stored procedures. pgbench gives you a driver program with the following useful properties: 1) Multiple processes are spawned and each gets its own connection 2) A

Re: [HACKERS] pgbench \for or similar loop

2011-04-22 Thread Greg Smith
Kevin Grittner wrote: I'm not clear on exactly what you're proposing there, but the thing I've considered doing is having threads to try to keep a FIFO queue populated with a configurable transaction mix, while a configurable number of worker threads pull those transactions off the queue and...

Re: [HACKERS] Re: database system identifier differs between the primary and standby

2011-04-22 Thread Deka, Rajib IN MAA SL
Thanks Robert. Rajib Deka SIEMENS Ltd. Robert V Chandran Tower, First Floor, West Wing, #149, Velechery Tambaram Main Road, Pallikaranai, Chennai-100, INDIA. www.siemens.com Mob: +91-9176780669 | E-Mail: rajib.d...@siemens.com -Original Message- From: Robert Haas

Re: [HACKERS] pgbench \for or similar loop

2011-04-22 Thread Pavel Stehule
Hello This isn't very well known because the whole MySQL community fracturing has impacted their ability to actually release this overhaul--seems like they spend all their time just trying to add support for each new engine of the month.  I don't even like Lua, yet this still seems like a

Re: [HACKERS] Re: database system identifier differs between the primary and standby

2011-04-22 Thread Deka, Rajib IN MAA SL
Thanks a lot. I got the idea now. Rajib Deka SIEMENS Ltd. Robert V Chandran Tower, First Floor, West Wing, #149, Velechery Tambaram Main Road, Pallikaranai, Chennai-100, INDIA. www.siemens.com Mob: +91-9176780669 | E-Mail: rajib.d...@siemens.com -Original Message- From: Cédric Villemain

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-04-22 Thread Greg Smith
Andrew Dunstan wrote: Personally, I want pgindent installed in /usr/local/ or similar. That way I can have multiple trees and it will work in all of them without my having to build it for each. What I don't want is for the installed patched BSD indent to conflict with the system's indent,

Re: [HACKERS] best way to test new index?

2011-04-22 Thread Yves Weißig
Ok, but I thought more like an automated test, or a test which checks if the interface is correctly implemented. By the way, tho broaden the topic, what would be the best way to debug in pg? elog? asserts? Am 22.04.2011 01:26, schrieb Josh Berkus: On 4/21/11 1:28 PM, Kevin Grittner wrote: That

Re: [HACKERS] switch UNLOGGED to LOGGED

2011-04-22 Thread Leonardo Francalanci
Maybe you should change xl_act_commit to have a separate list of rels to drop the init fork for (instead of mixing those with the list of files to drop as a whole). I tried to follow your suggestion, thank you very much. Here's a first attempt at the patch. I tested it with: create

Re: [HACKERS] Re: database system identifier differs between the primary and standby

2011-04-22 Thread Deka, Rajib IN MAA SL
Finally I setup two machines for primary and backup using the following link. http://wiki.postgresql.org/wiki/Streaming_Replication But after starting the backup I got the following messages and postgresql service doesn't start. 2011-04-22 14:37:54 ETC/GMT LOG: database system was shut down in

Re: [HACKERS] psql 9.1 alpha5: connection pointer is NULL

2011-04-22 Thread Christoph Berg
Re: Tom Lane 2011-04-03 1397.1301782...@sss.pgh.pa.us Yeah, that's clearly a bug --- fix committed, thanks for the patch! It could explain Devrim's report if the parameters passed by psql had some problem that was detectable by conninfo_array_parse(). That seems a bit unlikely, but I did

Re: [HACKERS] fsync reliability

2011-04-22 Thread Simon Riggs
On Fri, Apr 22, 2011 at 4:51 AM, Greg Smith g...@2ndquadrant.com wrote: On 04/21/2011 04:26 AM, Simon Riggs wrote: However, that begs the question of what happens with WAL. At present, we do nothing to ensure that the entry in the directory containing the file has also reached disk. Well,

Re: [HACKERS] fsync reliability

2011-04-22 Thread Greg Smith
Simon Riggs wrote: We do issue fsync and then close, but only when we switch log files. We don't do that as part of the normal commit path. Since all these files are zero-filled before use, the space is allocated for them, and the remaining important filesystem layout metadata gets

Re: [HACKERS] stored procedures

2011-04-22 Thread Merlin Moncure
On Thu, Apr 21, 2011 at 8:34 PM, Robert Haas robertmh...@gmail.com wrote: On Apr 21, 2011, at 3:51 PM, Merlin Moncure mmonc...@gmail.com wrote: If you do it that (base it on AT) way, then you can't: 1) call any utility command (vacuum, etc) 2) run for an arbitrary amount of time 3) discard

[HACKERS] What Index Access Method Functions are really needed?

2011-04-22 Thread Yves Weißig
Hi, another question regarding indexes. Sadly I can't find enough info in the documentation. Which of the functions are needed in order for a index to work? I am developing a prototype so it would be great if not all of the functions have to be implemented. E.g. are amcostestimate or amoptions

Re: [HACKERS] fsync reliability

2011-04-22 Thread Simon Riggs
On Fri, Apr 22, 2011 at 1:35 PM, Greg Smith g...@2ndquadrant.com wrote: Simon Riggs wrote: We do issue fsync and then close, but only when we switch log files. We don't do that as part of the normal commit path. Since all these files are zero-filled before use, the space is allocated for

Re: [HACKERS] stored procedures

2011-04-22 Thread Merlin Moncure
On Thu, Apr 21, 2011 at 5:07 PM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: On Thu, Apr 21, 2011 at 1:13 PM, Tom Lane t...@sss.pgh.pa.us wrote: 3. What sort of primitive operations do you expect the SP to be able to execute outside a transaction?  The plpgsql

Re: [HACKERS] What Index Access Method Functions are really needed?

2011-04-22 Thread Tom Lane
=?ISO-8859-15?Q?Yves_Wei=DFig?= weis...@rbg.informatik.tu-darmstadt.de writes: another question regarding indexes. Sadly I can't find enough info in the documentation. Which of the functions are needed in order for a index to work? All of them. regards, tom lane --

Re: [HACKERS] stored procedures

2011-04-22 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: It wouldn't bother me in the lest that if in plpgsql procedures if you had to set up and tear down a transaction on every line. It would once you noticed the performance impact ... regards, tom lane -- Sent via pgsql-hackers

Re: [HACKERS] fsync reliability

2011-04-22 Thread Greg Stark
On Thu, Apr 21, 2011 at 4:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: The traditional standard is that the filesystem is supposed to take care of its own metadata, and even Linux filesystems have pretty much figured that out.  I don't really see a need for us to be nursemaiding the filesystem.  

Re: [HACKERS] psql 9.1 alpha5: connection pointer is NULL

2011-04-22 Thread Tom Lane
Christoph Berg c...@df7cb.de writes: I'm still seeing that problem: 9.1 HEAD compiled in my $HOME, with Debian's 9.0.1-2 libpq5 in /usr/lib: $ LC_ALL=C bin/psql psql: connection pointer is NULL Upgrading to libpq5 9.0.4-1 makes things a bit better: $ LC_ALL=C bin/psql psql: invalid

Re: [HACKERS] stored procedures

2011-04-22 Thread Merlin Moncure
On Fri, Apr 22, 2011 at 9:29 AM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: It wouldn't bother me in the lest that if in plpgsql procedures if you had to set up and tear down a transaction on every line. It would once you noticed the performance impact ...

Re: [HACKERS] stored procedures

2011-04-22 Thread Kevin Grittner
Merlin Moncure mmonc...@gmail.com wrote: Tom Lane t...@sss.pgh.pa.us wrote: You can't have arithmetic, comparisons, or much of anything outside a transaction with plpgsql. That model just plain doesn't work for this purpose, I think. You really want a control language that's independent

Re: [HACKERS] What Index Access Method Functions are really needed?

2011-04-22 Thread Leonardo Francalanci
another question regarding indexes. Sadly I can't find enough info in the documentation. Which of the functions are needed in order for a index to work? All of them. Maybe I completely misunderstood the question, but some functions are optionals, such as amgetbitmap, right?

Re: [HACKERS] On-the-fly index tuple deletion vs. hot_standby

2011-04-22 Thread Noah Misch
On Tue, Mar 15, 2011 at 10:22:59PM -0400, Noah Misch wrote: On Mon, Mar 14, 2011 at 01:56:22PM +0200, Heikki Linnakangas wrote: On 12.03.2011 12:40, Noah Misch wrote: The installation that inspired my original report recently upgraded from 9.0.1 to 9.0.3, and your fix did significantly

Re: [HACKERS] stored procedures

2011-04-22 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Merlin Moncure mmonc...@gmail.com wrote: wouldn't it be better if the current crop of language handlers could run procedures without major changes? C functions with SPI? However it's internally implemented, the more userland mindspace

Re: [HACKERS] stored procedures

2011-04-22 Thread Merlin Moncure
On Fri, Apr 22, 2011 at 10:10 AM, Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: Merlin Moncure mmonc...@gmail.com wrote: wouldn't it be better if the current crop of language handlers could run procedures without major changes?  C functions with SPI?

Re: [HACKERS] stored procedures

2011-04-22 Thread Robert Haas
On Apr 22, 2011, at 11:10 AM, Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: Merlin Moncure mmonc...@gmail.com wrote: wouldn't it be better if the current crop of language handlers could run procedures without major changes? C functions with SPI?

Re: [HACKERS] What Index Access Method Functions are really needed?

2011-04-22 Thread Kevin Grittner
Leonardo Francalanci m_li...@yahoo.it wrote: another question regarding indexes. Sadly I can't find enough info in the documentation. Which of the functions are needed in order for a index to work? All of them. Maybe I completely misunderstood the question, but some functions are

Re: [HACKERS] stored procedures

2011-04-22 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: On Apr 22, 2011, at 11:10 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'd like a pony, too. No ponies for me; make mine an Arabian stallion. Let's be perfectly clear about this: there is no part of plpgsql that can run outside a transaction today, and

Re: [HACKERS] stored procedures

2011-04-22 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Apr 22, 2011, at 11:10 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'd like a pony, too. Let's be perfectly clear about this: there is no part of plpgsql that can run outside a transaction today, and probably no part of the other PLs either, and

Re: [HACKERS] Re: database system identifier differs between the primary and standby

2011-04-22 Thread Kevin Grittner
Deka, Rajib IN MAA SL rajib.d...@siemens.com wrote: Finally I setup two machines for primary and backup using the following link. http://wiki.postgresql.org/wiki/Streaming_Replication But after starting the backup I got the following messages and postgresql service doesn't start.

Re: [HACKERS] best way to test new index?

2011-04-22 Thread Kevin Grittner
Yves Weißigweis...@rbg.informatik.tu-darmstadt.de wrote: Ok, but I thought more like an automated test, or a test which checks if the interface is correctly implemented. I'm not aware of any automated tests which would test whether a new index type is correctly implemented. For starters,

Re: [HACKERS] stored procedures

2011-04-22 Thread Joshua Berkus
Tom, I'd like a pony, too. Let's be perfectly clear about this: there is no part of plpgsql that can run outside a transaction today, and probably no part of the other PLs either, and changing that without major changes is wishful thinking of the first order. I always thought that it

Re: [HACKERS] stored procedures

2011-04-22 Thread Merlin Moncure
On Fri, Apr 22, 2011 at 11:06 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Apr 22, 2011, at 11:10 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'd like a pony, too.  Let's be perfectly clear about this: there is no part of plpgsql that can run outside a

Re: [HACKERS] stored procedures

2011-04-22 Thread Andrew Dunstan
On 04/22/2011 12:06 PM, Tom Lane wrote: You could possibly lobotomize plpgsql down to a small number of datatypes and operators that are known not to ever do anything more interesting than palloc() and elog(), but IMO the usefulness would be low and the fragility high. It'd be better to give

Re: [HACKERS] stored procedures

2011-04-22 Thread Peter Eisentraut
On fre, 2011-04-22 at 08:37 -0500, Merlin Moncure wrote: It wouldn't bother me in the lest that if in plpgsql procedures if you had to set up and tear down a transaction on every line. It would probably be more reasonable and feasible to have a setup where you can end a transaction in plpgsql

Re: [HACKERS] stored procedures

2011-04-22 Thread Merlin Moncure
On Fri, Apr 22, 2011 at 1:28 PM, Peter Eisentraut pete...@gmx.net wrote: On fre, 2011-04-22 at 08:37 -0500, Merlin Moncure wrote: It wouldn't bother me in the lest that if in plpgsql procedures if you had to set up and tear down a transaction on every line. It would probably be more

[HACKERS] Collation patch's handling of wcstombs/mbstowcs is sheerest fantasy

2011-04-22 Thread Tom Lane
I just noticed that the collation patch has modified char2wchar and wchar2char to accept a collation OID as argument ... but it hasn't done anything to make those arguments actually work. Since those functions depend on wcstombs and mbstowcs, which respond to LC_CTYPE and nothing else, this flat

Re: [HACKERS] stored procedures

2011-04-22 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: On Fri, Apr 22, 2011 at 1:28 PM, Peter Eisentraut pete...@gmx.net wrote: It would probably be more reasonable and feasible to have a setup where you can end a transaction in plpgsql but a new one would start right away. ya, that's an idea. Yeah,

Re: [HACKERS] fsync reliability

2011-04-22 Thread Greg Smith
On 04/22/2011 09:32 AM, Simon Riggs wrote: OK, that's good, but ISTM we still have a hole during RemoveOldXlogFiles() where we don't fsync or open/close the file, just rename it. This is also something that many applications rely upon working as hoped for here, even though it's not

Re: [HACKERS] Patch for pg_upgrade to turn off autovacuum

2011-04-22 Thread Bruce Momjian
Robert Haas wrote: On Apr 21, 2011, at 6:22 PM, Bruce Momjian br...@momjian.us wrote: Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Tom Lane wrote: Huh? Why would that be? Seems like you've done something in the wrong place if that's an issue. Yeah, it is complicated. I

Re: [HACKERS] Patch for pg_upgrade to turn off autovacuum

2011-04-22 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: I thought some more about this and I don't want autovacuum to run on the old server. This is because pg_dumpall --binary-upgrade --schema-only grabs the datfrozenxid for all the databases at the start, then connects to each database to gets the

Re: [HACKERS] Patch for pg_upgrade to turn off autovacuum

2011-04-22 Thread Bruce Momjian
Bruce Momjian wrote: Well, consider that this also locks out non-super users so I figured it would be good to run the old and new in the same binary upgrade mode. Again, we can do just the new cluster for 9.1. I can also control the behavior based on the catalog version number, which

Re: [HACKERS] Patch for pg_upgrade to turn off autovacuum

2011-04-22 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: I thought some more about this and I don't want autovacuum to run on the old server. This is because pg_dumpall --binary-upgrade --schema-only grabs the datfrozenxid for all the databases at the start, then connects to each database

Re: [HACKERS] stored procedures

2011-04-22 Thread Merlin Moncure
On Fri, Apr 22, 2011 at 3:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: On Fri, Apr 22, 2011 at 1:28 PM, Peter Eisentraut pete...@gmx.net wrote: It would probably be more reasonable and feasible to have a setup where you can end a transaction in plpgsql

Re: [HACKERS] stored procedures

2011-04-22 Thread Kevin Grittner
Merlin Moncure mmonc...@gmail.com wrote: hm, another neat thing about this is that it skirts the unfortunate confusion between sql 'begin' and plpgsql 'begin'... I hadn't thought about that. There is the SQL-standard START TRANSACTION synonym, so there is a way to deal with it -- but since

[HACKERS] SSI non-serializalbe UPDATE performance (was: getting to beta)

2011-04-22 Thread Dan Ports
For background, the issue here is that there are three SSI calls that get invoked even on non-serializable transactions: - PredicateLockPageSplit/Combine, during B-tree page splits/combines - PredicateLockTupleRowVersionLink, from heap_update These have to update any matching SIREAD locks to

Re: [HACKERS] Patch for pg_upgrade to turn off autovacuum

2011-04-22 Thread Jeff Davis
On Fri, 2011-04-22 at 17:34 -0400, Bruce Momjian wrote: Tom Lane wrote: Bruce Momjian br...@momjian.us writes: I thought some more about this and I don't want autovacuum to run on the old server. This is because pg_dumpall --binary-upgrade --schema-only grabs the datfrozenxid for all

Re: [HACKERS] Patch for pg_upgrade to turn off autovacuum

2011-04-22 Thread Bruce Momjian
Jeff Davis wrote: On Fri, 2011-04-22 at 17:34 -0400, Bruce Momjian wrote: Tom Lane wrote: Bruce Momjian br...@momjian.us writes: I thought some more about this and I don't want autovacuum to run on the old server. This is because pg_dumpall --binary-upgrade --schema-only grabs

Re: [HACKERS] Patch for pg_upgrade to turn off autovacuum

2011-04-22 Thread Bruce Momjian
Bruce Momjian wrote: Tom Lane wrote: Bruce Momjian br...@momjian.us writes: I thought some more about this and I don't want autovacuum to run on the old server. This is because pg_dumpall --binary-upgrade --schema-only grabs the datfrozenxid for all the databases at the start, then

Re: [HACKERS] Collation patch's handling of wcstombs/mbstowcs is sheerest fantasy

2011-04-22 Thread Tom Lane
I wrote: I just noticed that the collation patch has modified char2wchar and wchar2char to accept a collation OID as argument ... but it hasn't done anything to make those arguments actually work. Since those functions depend on wcstombs and mbstowcs, which respond to LC_CTYPE and nothing

Re: [HACKERS] stored procedures

2011-04-22 Thread David Christensen
On Apr 22, 2011, at 3:50 PM, Tom Lane wrote: Merlin Moncure mmonc...@gmail.com writes: On Fri, Apr 22, 2011 at 1:28 PM, Peter Eisentraut pete...@gmx.net wrote: It would probably be more reasonable and feasible to have a setup where you can end a transaction in plpgsql but a new one would