Re: [HACKERS] On-the-fly index tuple deletion vs. hot_standby

2010-12-09 Thread Simon Riggs
On Fri, 2010-12-03 at 21:43 +0200, Heikki Linnakangas wrote: On 29.11.2010 08:10, Noah Misch wrote: I have a hot_standby system and use it to bear the load of various reporting queries that take 15-60 minutes each. In an effort to avoid long pauses in recovery, I set a

Re: [HACKERS] To Signal The postmaster

2010-12-09 Thread Fujii Masao
On Wed, Dec 8, 2010 at 6:22 PM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Dec 8, 2010 at 4:59 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: For 9.1, we should think of a better way to do this, perhaps using SIGUSR1 to wake up. Maybe we won't even need the trigger

Re: [HACKERS] Hot Standby tuning for btree_xlog_vacuum()

2010-12-09 Thread Simon Riggs
Just wanted to say thanks for the review, since I haven't yet managed to fix and commit this. I expect to later this month. On Mon, 2010-09-27 at 23:06 -0400, Robert Haas wrote: On Thu, Apr 29, 2010 at 4:12 PM, Simon Riggs si...@2ndquadrant.com wrote: Simple tuning of btree_xlog_vacuum() using

Re: [HACKERS] [PERFORM] Slow BLOBs restoring

2010-12-09 Thread Robert Haas
On Thu, Dec 9, 2010 at 12:28 AM, Tom Lane t...@sss.pgh.pa.us wrote: Vlad Arkhipov arhi...@dc.baikal.ru writes: 08.12.2010 22:46, Tom Lane writes: Are you by any chance restoring from an 8.3 or older pg_dump file made on Windows?  If so, it's a known issue. No, I tried Linux only. OK, then

[HACKERS] PS display and standby query conflict

2010-12-09 Thread Fujii Masao
Hi, When I created the conflict between recovery and many read-only transactions in the standby server for test purpose, I found that the keyword waiting disappeared from PS display for just a moment even though the conflict had not been resolved yet. This seems strange to me. This problem

[HACKERS] Why percent_rank is so slower than rank?

2010-12-09 Thread Jie Li
Hi all, I'm new to window functions. Recently I run some simple queries but surprised to find percent_rank is so slower than rank, could anybody tell me why? The table schema: test=# \d inventory1 Table public.inventory1 Column| Type | Modifiers

Re: [HACKERS] [PERFORM] Slow BLOBs restoring

2010-12-09 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Dec 9, 2010 at 12:28 AM, Tom Lane t...@sss.pgh.pa.us wrote: * Mark BLOB TOC entries as SECTION_DATA, or somehow otherwise make them parallelizable.  Also break the BLOBS data item apart into an item per BLOB, so that that part's parallelizable.

Re: [HACKERS] [PERFORM] Slow BLOBs restoring

2010-12-09 Thread Tom Lane
I wrote: One fairly simple, if ugly, thing we could do about this is skip calling reduce_dependencies during the first loop if the TOC object is a blob; effectively assuming that nothing could depend on a blob. But that does nothing about the point that we're failing to parallelize blob

Re: [HACKERS] Solving sudoku using SQL

2010-12-09 Thread Alvaro Herrera
Excerpts from Jan Urbański's message of mié dic 08 17:11:44 -0300 2010: I'm pleasantly surprised that the SA code as it stands today, setting the equlibrium factor to 8 and temperature reduction factor to 0.4, the query takes 1799.662 ms in total. That's 5x better than Oracle :-) -- Álvaro

Re: [HACKERS] [PERFORM] Slow BLOBs restoring

2010-12-09 Thread Robert Haas
On Thu, Dec 9, 2010 at 10:05 AM, Tom Lane t...@sss.pgh.pa.us wrote: I wrote: One fairly simple, if ugly, thing we could do about this is skip calling reduce_dependencies during the first loop if the TOC object is a blob; effectively assuming that nothing could depend on a blob.  But that does

Re: [HACKERS] [PERFORM] Slow BLOBs restoring

2010-12-09 Thread Andrew Dunstan
On 12/09/2010 10:05 AM, Tom Lane wrote: I think what we need to do is make fix_dependencies build a reverse lookup list of all the objects dependent on each TOC object, so that the searching behavior in reduce_dependencies can be eliminated outright. That will take O(N) time and O(N) extra

Re: [HACKERS] Solving sudoku using SQL

2010-12-09 Thread Dimitri Fontaine
Merlin Moncure mmonc...@gmail.com writes: On Wed, Dec 8, 2010 at 8:57 AM, Tatsuo Ishii is...@postgresql.org wrote: Then execute the huge SELECT: http://codezine.jp/static/images/article/1629/html/sql.html benchmark what you've got against this (ported to postgres by marcin mank):

Re: [HACKERS] Solving sudoku using SQL

2010-12-09 Thread Merlin Moncure
On Thu, Dec 9, 2010 at 11:56 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Merlin Moncure mmonc...@gmail.com writes: On Wed, Dec 8, 2010 at 8:57 AM, Tatsuo Ishii is...@postgresql.org wrote: Then execute the huge SELECT: http://codezine.jp/static/images/article/1629/html/sql.html

Re: [HACKERS] Optimize PL/Perl function argument passing [PATCH]

2010-12-09 Thread Tim Bunce
On Wed, Dec 08, 2010 at 09:21:05AM -0800, David E. Wheeler wrote: On Dec 8, 2010, at 9:14 AM, Tim Bunce wrote: Do you have any more improvements in the pipeline? I'd like to add $arrayref = decode_array_literal('{2,3}') and maybe $hashref = decode_hstore_literal('x=1, y=2'). I don't

[HACKERS] initdb failure with Postgres 8.4.4

2010-12-09 Thread BRUSSER Michael
Initdb fails for me when host machine has no access to the Postgres build location. LOG: could not open directory .../install/share/timezone: No such file or directory LOG: could not open directory .../install/share/timezone: No such file or directory WARNING: could not open directory

Re: XLog vs SSD [Was: Re: [HACKERS] random write in xlog?]

2010-12-09 Thread James Cloos
JJ == Jeff Janes jeff.ja...@gmail.com writes: JJ So PG always writing 8K at a time is unlikely to make a difference JJ than if it wrote a smaller amount. Ah. Somehow I was thinking of the xlog files' 16M filesize rather than the internal 8k block size If it is only writing 8k blocks then

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-09 Thread Gurjeet Singh
On Sun, Dec 5, 2010 at 2:09 PM, Peter Eisentraut pete...@gmx.net wrote: On fre, 2010-12-03 at 15:27 -0500, Robert Haas wrote: On Fri, Dec 3, 2010 at 2:56 PM, r t pg...@xzilla.net wrote: What exactly was the objection to the following -- ALTER TABLE table_name ADD PRIMARY KEY

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-09 Thread Tom Lane
Gurjeet Singh singh.gurj...@gmail.com writes: But I still hold a bias towards renaming the index to match constraint name (with a NOTICE), rather than require that the constraint name match the index name, because the constraint name is optional and when it is not provided system has to

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-09 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: If the constraint name is not specified, we should certainly use the existing index name, not randomly rename it. +1 -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] BufFreelistLock

2010-12-09 Thread Jim Nasby
On Dec 8, 2010, at 11:44 PM, Jeff Janes wrote: For the clock sweep algorithm, I think you could access nextVictimBuffer without any type of locking. This is wrong, mainly because you wouldn't have any security against two processes decrementing the usage count of the same buffer because

Re: [HACKERS] initdb failure with Postgres 8.4.4

2010-12-09 Thread Tom Lane
BRUSSER Michael michael.brus...@3ds.com writes: Initdb fails for me when host machine has no access to the Postgres build location. LOG: could not open directory .../install/share/timezone: No such file or directory Moving the install tree works for me. Did you do something odd with the

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-09 Thread Josh Berkus
On 12/6/10 6:13 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: OK, patch coming then. Right now test_fsync aborts when O_DIRECT fails. What should I have it do instead? Report that it fails, and keep testing the other methods. Patch attached. Includes a fair amount of comment

Re: [HACKERS] initdb failure with Postgres 8.4.4

2010-12-09 Thread BRUSSER Michael
No, we do not use --with-system-tzdata option. I looked at the makefile and at the output of pg_config. We may need to do some cleanup there, but I did not pick any clues. The problem occurs on all our UNIX platforms. Is there anything I could do to shed more light on it? I can post the output

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-09 Thread Robert Haas
On Thu, Dec 9, 2010 at 2:51 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Tom Lane t...@sss.pgh.pa.us wrote: If the constraint name is not specified, we should certainly use the existing index name, not randomly rename it. +1 +1 -- Robert Haas EnterpriseDB:

Re: [HACKERS] Why percent_rank is so slower than rank?

2010-12-09 Thread Tom Lane
Jie Li jay23j...@gmail.com writes: I'm new to window functions. Recently I run some simple queries but surprised to find percent_rank is so slower than rank, could anybody tell me why? Huh, interesting. I can reproduce this with toy data, such as create table inventory1 (inv_date_sk int,

Re: [HACKERS] initdb failure with Postgres 8.4.4

2010-12-09 Thread Andrew Dunstan
On 12/09/2010 03:36 PM, BRUSSER Michael wrote: No, we do not use --with-system-tzdata option. I looked at the makefile and at the output of pg_config. We may need to do some cleanup there, but I did not pick any clues. The problem occurs on all our UNIX platforms. Is there anything I could

Re: [HACKERS] Extensions, patch v16

2010-12-09 Thread David E. Wheeler
On Dec 9, 2010, at 12:34 PM, Dimitri Fontaine wrote: - add support for 'relocatable' boolean property in the control file, as discussed on list this controls what happens at create extension time, by doing a relocation of the extension objects when the extension is relocatable and

Re: [HACKERS] Why percent_rank is so slower than rank?

2010-12-09 Thread Tom Lane
I wrote: We're throwing away one tuple at a time as we advance forward through the tuplestore, and moving 10+ tuple pointers each time. Ugh. This code was all right when written, because (IIRC) the mergejoin case was actually the only caller. But it's not all right for WindowAgg's

Re: [HACKERS] Why percent_rank is so slower than rank?

2010-12-09 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: I'm not sure whether or not to back-patch this into 9.0 and 8.4. The code in tuplestore.c hasn't changed at all since 8.4, so there's not much risk of cross-version bugs, but if I did miss anything we could be shipping a buggy version next week.

Re: [HACKERS] Why percent_rank is so slower than rank?

2010-12-09 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Tom Lane t...@sss.pgh.pa.us wrote: I'm not sure whether or not to back-patch this into 9.0 and 8.4. The code in tuplestore.c hasn't changed at all since 8.4, so there's not much risk of cross-version bugs, but if I did miss anything we could

Re: [HACKERS] Why percent_rank is so slower than rank?

2010-12-09 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Do you have reason to think that anybody is likely to exercise window functions in HEAD, beyond what the regression tests do, in the next couple of months? Not specifically, no. From the description (not having read the patch) I was somewhat concerned

Re: [HACKERS] Why percent_rank is so slower than rank?

2010-12-09 Thread Kenneth Marshall
On Thu, Dec 09, 2010 at 05:18:57PM -0500, Tom Lane wrote: I wrote: We're throwing away one tuple at a time as we advance forward through the tuplestore, and moving 10+ tuple pointers each time. Ugh. This code was all right when written, because (IIRC) the mergejoin case was actually

Re: [HACKERS] Final(?) proposal for wal_sync_method changes

2010-12-09 Thread Greg Smith
Since any Windows refactoring has been postponed for now (I'll get back to performance checks on that platform later), during my testing time this week instead I did a round of pre-release review of the change Tom has now committed. All looks good to me, including the docs changes. I

Re: [HACKERS] Instrument checkpoint sync calls

2010-12-09 Thread Greg Smith
Jeff Janes wrote: In my test cases, the syncs that the backends were doing were almost always to the same file that the checkpoint writer was already choking on (so they are entangled simply by virtue of that). So very quickly all the backends hit the same wall and thunked to a halt. This is

Re: [HACKERS] [GENERAL] Postgresql 9.1 pg_last_xact_replay_timestamp limitations

2010-12-09 Thread Fujii Masao
On Fri, Dec 10, 2010 at 1:24 AM, Gabi Julien gabi.jul...@broadsign.com wrote: On Wednesday 08 December 2010 21:58:46 you wrote: On Thu, Dec 9, 2010 at 1:37 AM, Gabi Julien gabi.jul...@broadsign.com wrote: slave# /etc/init.d/postgresql start slave# psql -hlocalhost my_db -c select

Re: [HACKERS] Final(?) proposal for wal_sync_method changes

2010-12-09 Thread Josh Berkus
Greg, This is interesting, because test_fsync consistently reported a rate of about half this when using open_datasync instead of the equal performance I'm getting from the database. I'll see if I can reproduce that further, but it's no reason to be concerned about the change that's been

Re: [HACKERS] Final(?) proposal for wal_sync_method changes

2010-12-09 Thread Greg Smith
Josh Berkus wrote: Did you rerun test_sync with O_DIRECT entabled, using my patch? The figures you had from test_fsync earlier were without O_DIRECT. No--I was just focused on testing the changes that had already been committed. The use of O_DIRECT in the server but not test_fsync could

[HACKERS] SynchRep; wait-forever and shutdown

2010-12-09 Thread Fujii Masao
Hi, In previous discussion, some people wanted the wait-forever option which blocks all the transactions on the master until sync'd standby has appeared, in order to reduce the risk of data loss in synchronous replication. What I'm not clear is; How does smart or fast shudown advance while all

Re: [HACKERS] serializable read only deferrable

2010-12-09 Thread Dan Ports
On Tue, Dec 07, 2010 at 10:14:24AM -0600, Kevin Grittner wrote: Essentially, instead of adding dependencies as you go along and abort once you hit a conflict, SERIALIZABLE READ ONLY DEFERRED transactions would assume the worst case from the start and thus be able to bypass the more

[HACKERS] Anyone for SSDs?

2010-12-09 Thread Vaibhav Kaushal
Hi all, Most of you already know I am new to this list and newer to any OSS development. However, while browsing the source code (of 9.0.1) I find that there is only one way to store relations on disk - the magnetic disk. This came suddenly in my mind so I am asking the experts here.

Re: [HACKERS] To Signal The postmaster

2010-12-09 Thread aaliya zarrin
Hi All, Can anybody tell after finding the trigger file what steps does postgres follow? When and how it will set the postgres recovery mode to false? On Thu, Dec 9, 2010 at 3:51 PM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Dec 8, 2010 at 6:22 PM, Fujii Masao masao.fu...@gmail.com

[HACKERS] Fwd: Extended query protocol and exact types matches.

2010-12-09 Thread Dmitriy Igrishin
-- Forwarded message -- From: Dmitriy Igrishin dmit...@gmail.com Date: 2010/12/10 Subject: Fwd: Extended query protocol and exact types matches. To: postgres list pgsql-...@postgresql.org Hey sql@, -- Forwarded message -- From: Dmitriy Igrishin dmit...@gmail.com