Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-25 Thread Santhakumaran
Dkloskxe Steve Crawford wrote: >On 05/25/2011 11:45 AM, Reuven M. Lerner wrote: >> Hi, Alex. You wrote: >>> Have you tried something like: >>> SELECT encode(regexp_replace('141142143', '(\d{3})', '\\\1', >>> 'g')::bytea, 'escape'); >> Hmm, forgot about regexp_replace. It might do the trick, b

Re: [PERFORM] "error with invalid page header" while vacuuming pgbench data

2011-05-25 Thread Craig Ringer
On 05/26/2011 06:18 AM, Scott Marlowe wrote: On Wed, May 25, 2011 at 4:07 PM, John Rouillard wrote: Well I am also managing to panic the kernel on some runs as well. So my guess is this is not only a postgres bug (if it's a postgres issue at all). As gregg mentioned in another followup ext4 u

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-25 Thread Craig Ringer
On 05/26/2011 12:42 AM, panam wrote: So, would you like to further investigate my previous issue (I think it is still strange that performance suddenly dropped that dramatically)? It's a bit beyond me, but I suspect that it'd be best if you could hang onto the dump file in case someone has th

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-25 Thread Merlin Moncure
On Wed, May 25, 2011 at 8:03 PM, Steve Crawford wrote: > On 05/25/2011 11:45 AM, Reuven M. Lerner wrote: >> >> Hi, Alex.  You wrote: >>> >>> Have you tried something like: >>> SELECT  encode(regexp_replace('141142143', '(\d{3})', '\\\1', >>> 'g')::bytea, 'escape'); >> >> Hmm, forgot about regexp_r

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-25 Thread Steve Crawford
On 05/25/2011 11:45 AM, Reuven M. Lerner wrote: Hi, Alex. You wrote: Have you tried something like: SELECT encode(regexp_replace('141142143', '(\d{3})', '\\\1', 'g')::bytea, 'escape'); Hmm, forgot about regexp_replace. It might do the trick, but without a full-blown eval that I can run on th

Re: FW: [PERFORM] KVP table vs. hstore - hstore performance (Was: Postgres NoSQL emulation)

2011-05-25 Thread Stefan Keller
Hi all Thank you to all who answered: That worked: CREATE INDEX planet_osm_point_tags_amenity ON planet_osm_point ((tags->'amenity')) WHERE (tags->'amenity') IS NOT NULL; My problem is, that in fact I don't know which tag to index since I'm running a web admin application where users can enter a

Re: [PERFORM] serveRAID M5014 SAS

2011-05-25 Thread Mark Kirkwood
On 26/05/11 10:24, Mark Kirkwood wrote: Also I think the amount of ram on the card is upgradable (4G is the max for the M5105 I *think* - can't find the right doc to check this ATM sorry). Looking at the (very sparse) product docs, it looks I am mistaken above - and that the cache sizes

Re: [PERFORM] serveRAID M5014 SAS

2011-05-25 Thread Mark Kirkwood
On 25/05/11 19:33, Grzegorz Jaśkiewicz wrote: Does anyone here have any bad experiences with the RAID card in subject ? This is in an IBM server, with 2.5" 10k drives. But we seem to observe its poor performance in other configurations as well (with different drives, different settings) in compa

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-25 Thread Reuven M. Lerner
Hi, everyone. Merlin wrote: let's see the source. I bet we can get this figured out. Here you go... it looked nicer before I started to make optimizations; I've gotten it to run about 2x as fast as the previous version, but now I'm sorta stuck, looking for further optimizations, including

Re: [PERFORM] "error with invalid page header" while vacuuming pgbench data

2011-05-25 Thread Scott Marlowe
On Wed, May 25, 2011 at 4:07 PM, John Rouillard wrote: > Well I am also managing to panic the kernel on some runs as well.  So > my guess is this is not only a postgres bug (if it's a postgres issue > at all). > > As gregg mentioned in another followup ext4 under centos 5.x may be an > issue. I'll

Re: FW: [PERFORM] KVP table vs. hstore - hstore performance (Was: Postgres NoSQL emulation)

2011-05-25 Thread Merlin Moncure
On Wed, May 25, 2011 at 11:59 AM, Pierre C wrote: >> You wrote >>> >>> Try to create a btree index on "(bench_hstore->bench_id) WHERE >>> (bench_hstore->bench_id) IS NOT NULL". >> >> What  do you mean exactly? >> => CREATE INDEX myhstore_kps_gin_idx ON myhstore USING gin(kvps) WHERE >> ??? IS NOT

Re: [PERFORM] "error with invalid page header" while vacuuming pgbench data

2011-05-25 Thread John Rouillard
On Wed, May 25, 2011 at 03:19:59PM -0500, Kevin Grittner wrote: > John Rouillard wrote: > > On Mon, May 23, 2011 at 05:21:04PM -0500, Kevin Grittner wrote: > >> John Rouillard wrote: > >> > >> > I seem to be able to provoke this error: > >> > > >> >vacuum...ERROR: invalid page header in

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-25 Thread Merlin Moncure
On Wed, May 25, 2011 at 11:59 AM, Reuven M. Lerner wrote: > Hi, everyone.  I'm working on a project that's using PostgreSQL 8.3, that > requires me to translate strings of octal digits into strings of characters > -- so '141142143' should become 'abc', although the database column > containing thi

Re: [PERFORM] "error with invalid page header" while vacuuming pgbench data

2011-05-25 Thread Greg Smith
On 05/23/2011 06:16 PM, John Rouillard wrote: OS: centos 5.5 Filesystem: data - ext4 (note 4 not 3); 6.6T formatted wal - ext4; 1.5T formatted Raid: data - level 10, 8 disk wd2003; controller LSI MegaRAID SAS 9260-4i wal - level 1, 2 disk wd2003; controller LSI MegaRAID SAS

Re: [PERFORM] "error with invalid page header" while vacuuming pgbench data

2011-05-25 Thread Kevin Grittner
John Rouillard wrote: > On Mon, May 23, 2011 at 05:21:04PM -0500, Kevin Grittner wrote: >> John Rouillard wrote: >> >> > I seem to be able to provoke this error: >> > >> >vacuum...ERROR: invalid page header in >> > block 2128910 of relation base/16385/21476 >> >> What versi

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-25 Thread Alex Hunsaker
On Wed, May 25, 2011 at 10:59, Reuven M. Lerner wrote: > Hi, everyone.  I'm working on a project that's using PostgreSQL 8.3, that > requires me to translate strings of octal digits into strings of characters > -- so '141142143' should become 'abc', although the database column > containing this d

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-25 Thread Pavel Stehule
Hello > > (1) Are there any good guidelines for what operations in pl/pgsql are > optimized for which data structures?  For example, it turns out that a great > deal of time is being spent in the substring() function, which surprised me. >  I thought that by switching to an array, it might be fast

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-25 Thread Alex Hunsaker
On Wed, May 25, 2011 at 12:45, Reuven M. Lerner wrote: > Hi, Alex.  You wrote: >> I think select E'\XXX' is what you are looking for (per the fine >> manual: >> http://www.postgresql.org/docs/current/static/datatype-binary.html) > > I didn't think that I could (easily) build a string like that fr

Re: [PERFORM] Speeding up loops in pl/pgsql function

2011-05-25 Thread Reuven M. Lerner
Hi, Alex. You wrote: Have you tried something like: SELECT encode(regexp_replace('141142143', '(\d{3})', '\\\1', 'g')::bytea, 'escape'); Hmm, forgot about regexp_replace. It might do the trick, but without a full-blown eval that I can run on the replacement side, it'll be a bit more challeng

Re: FW: [PERFORM] KVP table vs. hstore - hstore performance (Was: Postgres NoSQL emulation)

2011-05-25 Thread Pierre C
You wrote Try to create a btree index on "(bench_hstore->bench_id) WHERE (bench_hstore->bench_id) IS NOT NULL". What do you mean exactly? => CREATE INDEX myhstore_kps_gin_idx ON myhstore USING gin(kvps) WHERE ??? IS NOT NULL; My table's def is: CREATE TABLE myhstore ( id bigint PRIMARY KEY,

Re: [PERFORM] "error with invalid page header" while vacuuming pgbench data

2011-05-25 Thread John Rouillard
On Mon, May 23, 2011 at 05:21:04PM -0500, Kevin Grittner wrote: > John Rouillard wrote: > > > I seem to be able to provoke this error: > > > >vacuum...ERROR: invalid page header in > > block 2128910 of relation base/16385/21476 > > What version of PostgreSQL? Hmm, I thought

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-25 Thread Kevin Grittner
panam wrote: > Is there a way to determine the values actually used? The pg_settings view. Try the query shown here: http://wiki.postgresql.org/wiki/Server_Configuration -Kevin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscrip

[PERFORM] Speeding up loops in pl/pgsql function

2011-05-25 Thread Reuven M. Lerner
Hi, everyone. I'm working on a project that's using PostgreSQL 8.3, that requires me to translate strings of octal digits into strings of characters -- so '141142143' should become 'abc', although the database column containing this data (both before and after) is a bytea. While the function

Re: [PERFORM] Hash Anti Join performance degradation

2011-05-25 Thread panam
Hi all, @Tom, > BTW, this query doesn't actually match the EXPLAIN outputs... You're right, it is actually just the "heavy" subquery of a larger query which can be found here: http://pastebin.com/fuGrt0tB > One other thing I'm not following is how come it's using hash temp files > at all, when y

Re: [PERFORM] serveRAID M5014 SAS

2011-05-25 Thread Kevin Grittner
Greg Smith wrote: > -Is the battery working, and the write cache set in write-back > mode? My bet is on this point. Our hardware tech says that the difference between an M5014 and an M5015 is that the former takes a maximum of 256MB RAM while the latter takes a maximum of 512MB RAM and that

Re: [PERFORM] serveRAID M5014 SAS

2011-05-25 Thread Greg Smith
Grzegorz Jaśkiewicz wrote: Does anyone here have any bad experiences with the RAID card in subject ? This is in an IBM server, with 2.5" 10k drives. But we seem to observe its poor performance in other configurations as well (with different drives, different settings) in comparison with - say, w

Re: [PERFORM] [PERFORMANCE] expanding to SAN: which portion best to move

2011-05-25 Thread Vitalii Tymchyshyn
24.05.11 21:48, Greg Smith написав(ла): Bitmap heap scan: Here, the exact list of blocks to fetch is known in advance, they're random, and it's quite possible for the kernel to schedule them more efficiently than serial access of them can do. This was added as the effective_io_concurrency fea

[PERFORM] serveRAID M5014 SAS

2011-05-25 Thread Grzegorz Jaśkiewicz
Does anyone here have any bad experiences with the RAID card in subject ? This is in an IBM server, with 2.5" 10k drives. But we seem to observe its poor performance in other configurations as well (with different drives, different settings) in comparison with - say, what dell provides. Any expe