Re: [PERFORM] Postgres and Ingres R3 / SAN

2006-03-08 Thread Jim Nasby
Adding -performance back; you should do a reply-all if you want to reply to list messages. From: Jeremy Haile [mailto:[EMAIL PROTECTED] Can you point us at more info about this? I can't even find a website for Ingres... Ingres is based off of the same original codebase that PostgreSQL

[PERFORM] 1 TB of memory

2006-03-16 Thread Jim Nasby
PostgreSQL tuned to the max and still too slow? Database too big to fit into memory? Here's the solution! http://www.superssd.com/ products/tera-ramsan/ Anyone purchasing one will be expected to post benchmarks! :) -- Jim C. Nasby, Sr. Engineering Consultant [EMAIL PROTECTED] Pervasive

Re: [PERFORM] Query using SeqScan instead of IndexScan

2006-04-04 Thread Jim Nasby
On Apr 2, 2006, at 6:30 PM, Josh Berkus wrote: But just as a follow up question to your #1 suggestion, I have 8 GB of ram in my production server. You're saying to set the effective_cache_size then to 5 GB roughly? Somewhere around 655360? Currently it is set to 65535. Is that something that's

Re: [PERFORM] Query using SeqScan instead of IndexScan

2006-04-04 Thread Jim Nasby
On Apr 1, 2006, at 12:51 PM, Brendan Duddridge wrote: from SELECT * FROM pg_stats WHERE tablename='table' AND attname='category_id' I find correlation on category_product for category_id is 0.643703 Would setting the index on category_id to be clustered help with this? It would absolutely

Re: [PERFORM] freebsd/softupdates for data dir

2006-04-05 Thread Jim Nasby
On Apr 4, 2006, at 10:41 AM, Vivek Khera wrote: On Apr 3, 2006, at 10:10 PM, Mark Kirkwood wrote: I've always left them on, and never had any issues...(even after unscheduled power loss - which happened here yesterday). As I understand it, the softupdate code reorders *metadata* operations,

Re: [PERFORM] Better index stategy for many fields with few values

2006-04-12 Thread Jim Nasby
Adding -performance back in -Original Message-From: Oscar Picasso [mailto:[EMAIL PROTECTED]Sent: Wednesday, April 12, 2006 5:51 PMTo: Jim NasbySubject: Re: [PERFORM] Better index stategy for many fields with few values I would like to try it.However in an other post I

Re: [PERFORM] multi column query

2006-04-13 Thread Jim Nasby
] Sent: Wednesday, April 12, 2006 7:48 PM To: Jim Nasby Subject: RE: [PERFORM] multi column query I executed enable_seqscan=off and then ran an explain plan on the query UPDATE chkpfw_tr_dy_dimension SET summcount = a.summcount + b.summcount

[PERFORM] Blocks read for index scans

2006-04-13 Thread Jim Nasby
While working on determining a good stripe size for a database, I realized it would be handy to know what the average request size is. Getting this info is a simple matter of joining pg_stat_all_tables and pg_statio_all_tables and doing some math, but there's one issue I've found; it

Re: [PERFORM] pgmemcache

2006-04-13 Thread Jim Nasby
On Apr 13, 2006, at 12:38 PM, Tom Lane wrote: Christian Storm [EMAIL PROTECTED] writes: Not sure if I follow why this is a problem. Seems like it would be beneficial to have both BEFORE and AFTER COMMIT triggers. With the BEFORE COMMIT trigger you would have the ability to 'un- commit'

Re: [PERFORM] Blocks read for index scans

2006-04-13 Thread Jim Nasby
(hopefully) have already been in shared_buffers. On Thu, 2006-04-13 at 13:00 -0500, Jim Nasby wrote: While working on determining a good stripe size for a database, I realized it would be handy to know what the average request size is. Getting this info is a simple matter of joining

Re: [PERFORM] slow variable against int??

2006-05-12 Thread Jim Nasby
Please cc the list so others can help. From: Witold Strzelczyk [mailto:[EMAIL PROTECTED] On Friday 12 May 2006 00:04, you wrote: Yes, thanks but method is not a point. Actually, it is a point. Databases don't like doing things procedurally. Using a stored procedure to operate on a set of

Re: [PERFORM] Optimizing a huge_table/tiny_table join

2006-05-25 Thread Jim Nasby
On May 25, 2006, at 12:07 PM, Dawid Kuroczko wrote: On 5/25/06, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Well, they're not my statistics; they're explain's. You mean there's a bug in explain? I agree that it makes no sense that the costs don't differ as much as one would expect, but you

FW: [PERFORM] pg_xlog on data partition with BBU RAID

2006-06-09 Thread Jim Nasby
Forwarding to -performance From: Alan Hodgson [mailto:[EMAIL PROTECTED] On Friday 09 June 2006 12:41, Jim C. Nasby [EMAIL PROTECTED] wrote: Has anyone actually done any testing on this? Specifically, I'm wondering if the benefit of adding 2 more drives to a RAID10 outweighs whatever penalties

Re: [PERFORM] Confirmation of bad query plan generated by 7.4

2006-06-14 Thread Jim Nasby
On Jun 13, 2006, at 8:50 PM, Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: On Tue, Jun 13, 2006 at 06:04:42PM -0400, Tom Lane wrote: It'd depend on the context, possibly, but it's easy to show that the current planner does fold now() - interval_constant when making estimates. Simple

Re: [PERFORM] Precomputed constants?

2006-06-17 Thread Jim Nasby
On Jun 15, 2006, at 1:19 PM, Zoltan Boszormenyi wrote: # select distinct provolatile from pg_proc; provolatile - i s v (3 sor) If I get this right, IMMUTABLE/STABLE/VOLATILE are indicated with their initials. That's probably correct. If the docs don't specify this then the code

Re: [PERFORM] Optimizer internals

2006-06-17 Thread Jim Nasby
On Jun 16, 2006, at 8:43 AM, Jonah H. Harris wrote: Yes, this is certainly the most noticible case. This is one reason I'm behind the freespace patch. Unfortunately, a lot of inexperienced people use VACUUM FULL and don't understand why VACUUM is *generally* better.(to free up block-level

Re: [PERFORM] SAN performance mystery

2006-06-17 Thread Jim Nasby
On Jun 16, 2006, at 6:28 AM, Greg Stark wrote: I never understood why disk caches on the order of megabytes are exciting. Why should disk manufacturers be any better about cache management than OS authors? In the case of RAID 5 this could actually work against you since the RAID controller

Re: [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL community

2006-06-17 Thread Jim Nasby
On Jun 16, 2006, at 12:01 PM, Josh Berkus wrote: Folks, I am thrill to inform you all that Sun has just donated a fully loaded T2000 system to the PostgreSQL community, and it's being setup by Corey Shields at OSL (osuosl.org) and should be online probably early next week. The system has

Re: [PERFORM] Tuning New Server (slow function)

2006-06-22 Thread Jim Nasby
On Jun 21, 2006, at 5:53 PM, Ron St-Pierre wrote: Jim C. Nasby wrote: On Wed, Jun 21, 2006 at 02:27:41PM -0700, Ron St-Pierre wrote: We just purchased a new Dell PowerEdge 2800 (dual xeon, 8GB RAM, raid 4, RHEL, postgres 8.1) and ported our old database over to it (single cpu, RAID *4*?

[PERFORM]

2006-08-03 Thread Jim Nasby
I'm at a client who's an ASP; they've written their app such that each customer gets their own database. Rigth now they're at nearly 200 databases, and were thinking that they must be the largest PostgreSQL install in the world. :) After taking them down a notch or two, I started wondering how

Re: [PERFORM] [BUGS] BUG #2567: High IOWAIT

2006-08-10 Thread Jim Nasby
[EMAIL PROTECTED] Pervasive Software http://pervasive.comwork: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461 -Original Message- From: Kumarselvan S [mailto:[EMAIL PROTECTED] Sent: Wed 8/9/2006 11:33 PM To: Jim Nasby Subject: RE: [BUGS] BUG

Re: [PERFORM] Inner Join of the same table

2006-08-15 Thread Jim Nasby
On Aug 15, 2006, at 1:53 PM, Sebastián Baioni wrote: 9 - Index Scan using uesapt001 on APORTES (cost=0.00..37301678.64 rows=9339331 width=25) (actual time=110.000..2520690.000 rows=9335892 loops=1) It's taking 2520 seconds to scan an index with 9M rows, which

Re: [PERFORM] Update on high concurrency OLTP application and Postgres

2006-09-26 Thread Jim Nasby
know an analyze is needed, or statistics should be set higher. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 3: Have you

Re: [PERFORM] IN not handled very well?

2006-09-26 Thread Jim Nasby
after all? Based on your initial post, it probably should know that it's only getting 15 rows (since it did in your initial plan), so it's unclear why it's not choosing the bitmap scan. Can you post the results of EXPLAIN ANALYZE? -- Jim Nasby

Re: [PERFORM] Decreasing BLKSZ

2006-09-26 Thread Jim Nasby
the resulting monster UNION ALL), but you might get lucky. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 4: Have you searched

Re: [PERFORM] Confusion and Questions about blocks read

2006-09-26 Thread Jim Nasby
what you're talking about. I don't know if it actually happened or not, though. If this is something that interests you, I recommend taking a look at the code; it's generally not too hard to read through thanks to all the comments. -- Jim Nasby[EMAIL

Re: [PERFORM] PostgreSQL and sql-bench

2006-09-26 Thread Jim Nasby
that you're not inadvertently disabling ACIDity in MySQL/ InnoDB; some options/performance tweaks will do that last I looked. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end

Re: [PERFORM] selecting data from information_schema.columns

2006-10-02 Thread Jim Nasby
be very helpful for doing that. There is no ability to put triggers on DDL, so the best you could do with your caching table is to just periodically update it. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461

Re: [PERFORM] any hope for my big query?

2006-10-05 Thread Jim Nasby
to look into what a subselect will return to us. ;) -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com

Re: [PERFORM] Unsubscribe

2006-10-05 Thread Jim Nasby
these emails, such as mailman. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore

Re: [PERFORM] Unsubscribe

2006-10-05 Thread Jim Nasby
On Oct 4, 2006, at 11:35 AM, Csaba Nagy wrote: On Wed, 2006-10-04 at 18:02, Csaba Nagy wrote: If we didn't want to add it for each list we could just add a link here: http://www.postgresql.org/community/lists/subscribe OK, now that I had a second look on that page, it does contain

Re: [PERFORM] slow queue-like empty table

2006-10-05 Thread Jim Nasby
the programmers ... too big geographical distances ;-) This warrants a plane ticket. Seriously. If your app programmers aren't versed in transaction management, you should probably be defining a database API that allows the use of autocommit. -- Jim Nasby

Re: [PERFORM] Simple join optimized badly?

2006-10-08 Thread Jim Nasby
On Oct 7, 2006, at 8:50 PM, Denis Lussier wrote: Wouldn't PG supporting simple optmizer hints get around this kinda problem? Seems to me that at least one customer posting per week would be solved via the use of simple hints. If the community is interested... EnterpriseDB has added support

Re: [PERFORM] Setting nice values

2006-11-05 Thread Jim Nasby
will eventually do what you want. Take a look at the BizGres mailing list archives for more info. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast

Re: [PERFORM] Help w/speeding up range queries?

2006-11-06 Thread Jim Nasby
the geometric functions built into plain PostgreSQL (though perhaps that's what he meant by PostGIS stuff). -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end

Re: [PERFORM] Context switch storm

2006-11-16 Thread Jim Nasby
won't benefit that much because you'll end up double- buffering written data. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 6

Re: [PERFORM] File Systems Compared

2006-12-13 Thread Jim Nasby
the platter. So I don't see how enabling the disk cache will help, unless of course it's ignoring fsync. Now, I have heard something about drives using their stored rotational energy to flush out the cache... but I tend to suspect urban legend there... -- Jim Nasby

Re: [HACKERS] [PERFORM] how to plan for vacuum?

2007-01-26 Thread Jim Nasby
vs. writes are a concern. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore

[PERFORM] Fragmentation of WAL files

2007-04-26 Thread Jim Nasby
the WAL files were 64MB instead of 16MB, but even having 500 fragments for a 16MB WAL file seems like it would definitely impact performance. Can anyone else confirm this? I don't know if this is a windows-only issue, but I don't know of a way to check fragmentation in unix. -- Jim Nasby

Re: [PERFORM] seeking advise on char vs text or varchar in search table

2007-04-26 Thread Jim Nasby
. The only reason I've ever used char in other databases is to save the overhead of the variable-length information, so I recommend to people to just steer clear of char in PostgreSQL. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http

Re: [PERFORM] What`s wrong with JFS configuration?

2007-04-26 Thread Jim Nasby
. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire

Re: [PERFORM] What`s wrong with JFS configuration?

2007-04-27 Thread Jim Nasby
Adding -performance back in so others can learn. On Apr 26, 2007, at 9:40 AM, Paweł Gruszczyński wrote: Jim Nasby napisał(a): On Apr 25, 2007, at 8:51 AM, Paweł Gruszczyński wrote: where u6 stores Fedora Core 6 operating system, and u0 stores 3 partitions with ext2, ext3 and jfs filesystem

Re: [PERFORM] Feature Request --- was: PostgreSQL Performance Tuning

2007-04-27 Thread Jim Nasby
stick a web page somewhere that would produce a postgresql.conf based simply on how much available RAM you had, since that's one of the biggest performance-hampering issues we run into (ie: shared_buffers left at the default of 32MB). -- Jim Nasby[EMAIL

Re: [PERFORM] Feature Request --- was: PostgreSQL Performance Tuning

2007-05-05 Thread Jim Nasby
could help. Uh... what GUCs are that exacting on the amount of memory? For a decent, base-line configuration, that is. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end

Re: [PERFORM] How to Find Cause of Long Vacuum Times - NOOB Question

2007-05-07 Thread Jim Nasby
during the day (try setting vacuum_cost_delay to 20 as a starting point). -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 2

Re: [PERFORM] Best OS for Postgres 8.2

2007-05-09 Thread Jim Nasby
a small filesystem for pg_xlog and mount that as ext2. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 6: explain analyze is your

Re: [PERFORM] How to Find Cause of Long Vacuum Times - NOOB Question

2007-05-09 Thread Jim Nasby
On May 7, 2007, at 11:10 PM, Yudhvir Singh Sidhu wrote: Jim Nasby wrote: On May 5, 2007, at 5:57 PM, Yudhvir Singh Sidhu wrote: Problem: Vacuum times jump up from 45 minutes, or 1:30 minutes to 6+ hours overnight, once every 1 to 3 months. Solutions tried: db truncate - brings vacuum times

Re: [PERFORM] Feature suggestion : FAST CLUSTER

2007-05-29 Thread Jim Nasby
table, in a different ordering. I don't know of a good way to address that; you can cluster the toast table, but you'll be clustering on an OID, which isn't going to help you. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com

Re: [PERFORM] dbt2 NOTPM numbers

2007-06-11 Thread Jim Nasby
actually need to deal with things at a stripe level, or if they can deal with smaller chunks of a stripe. In either case, the issue is still the number of extra reads going on. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com

Re: [PERFORM] Vacuum takes forever

2007-06-11 Thread Jim Nasby
think of any reason you'd want to do that. I do find vacuum_cost_delay to be an extremely useful tool, but typically I'll set it to between 10 and 20 and leave the other parameters alone. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http

Re: [PERFORM] How much ram is too much

2007-06-11 Thread Jim Nasby
and set them to anything remotely close to 128GB. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 4: Have you searched our list

Re: [PERFORM] dbt2 NOTPM numbers

2007-06-19 Thread Jim Nasby
-used -benchmarks list, but perhaps it would be better if we setup a wiki for this... -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast

Re: [pgsql-advocacy] [PERFORM] [ADMIN] Postgres VS Oracle

2007-06-19 Thread Jim Nasby
-- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 4: Have you searched our list archives? http

Re: [PERFORM] Database-wide VACUUM ANALYZE

2007-06-25 Thread Jim Nasby
tables, that could be a big problem, as autovac could get tied up on a large table for a long enough period that the table needing to be frozen doesn't get frozen in time. I suspect 1B is a much better setting. I probably wouldn't go past 1.5B. -- Jim Nasby

Re: [PERFORM] Volunteer to build a configuration tool

2007-06-25 Thread Jim Nasby
currently done. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [PERFORM] best use of an EMC SAN

2007-07-11 Thread Jim Nasby
at OSDL; the only reason that was the case is because he had somewhere around 70 data drives. I suppose an entirely in-memory database might be able to swamp a 2 drive WAL as well. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com

[PERFORM] 8.2 - 8.3 performance numbers

2007-07-19 Thread Jim Nasby
complete 8.2 - 8.3 numbers). -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 5: don't forget to increase your free space map

Re: [PERFORM] 8.2 - 8.3 performance numbers

2007-07-20 Thread Jim Nasby
. Anyone else have something better? -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 4: Have you searched our list archives

Re: [PERFORM] Affect of Reindexing on Vacuum Times

2007-07-26 Thread Jim Nasby
times. All other things remain the same. Which means the only change I am performing is re-indexing. Reindex will shrink index sizes, which will speed up vacuuming. But that alone doesn't explain what you're seeing, which is rather odd. -- Jim Nasby

Re: [PERFORM] Performance on writable views

2007-08-13 Thread Jim Nasby
to a new partition), you're looking at evaluating every input query twice. In this case, the rules presumably are just simply re-directing DML, so there'd only be one rule in play at a time. That means the only real overhead is in the rewrite engine. -- Decibel!, aka Jim Nasby

Re: [PERFORM] Help with bulk read performance

2010-12-13 Thread Jim Nasby
On Nov 1, 2010, at 9:15 AM, Dan Schaffer wrote: We have an application that needs to do bulk reads of ENTIRE Postgres tables very quickly (i.e. select * from table). We have observed that such sequential scans run two orders of magnitude slower than observed raw disk reads (5 MB/s versus

Re: [PERFORM] Help with bulk read performance

2010-12-14 Thread Jim Nasby
On Dec 14, 2010, at 9:27 AM, Andy Colson wrote: Is this the same thing Nick is working on? How'd he get along? http://archives.postgresql.org/message-id/4cd1853f.2010...@noaa.gov So it is. The one I replied to stood out because no one had replied to it; I didn't see the earlier email. --

Re: [PERFORM] Help with bulk read performance

2010-12-14 Thread Jim Nasby
more throughput so it may not happen. Cheers, Nick On 12/14/2010 9:41 AM, Jim Nasby wrote: On Dec 14, 2010, at 9:27 AM, Andy Colson wrote: Is this the same thing Nick is working on? How'd he get along? http://archives.postgresql.org/message-id/4cd1853f.2010...@noaa.gov So it is. The one

Re: [PERFORM] Strange optimization - xmin,xmax compression :)

2010-12-19 Thread Jim Nasby
On Dec 17, 2010, at 8:46 PM, Robert Haas wrote: 2010/12/6 pasman pasmański pasma...@gmail.com: hello. i tested how are distributed values xmin,xmax on pages. in my tables . typically there are no more than 80 records on pages. maybe its possible to compress xmin xmax values to 1

Re: [PERFORM] encourging bitmap AND

2011-01-02 Thread Jim Nasby
On Dec 26, 2010, at 11:24 AM, Tom Lane wrote: If you're doing interval queries enough to worry about having an index for them, you really want an indexing structure that is designed to do interval queries efficiently. BTW, one way to accomplish that is to transform your data into geometric

Re: [PERFORM] CPU bound

2011-01-02 Thread Jim Nasby
On Dec 20, 2010, at 12:47 AM, Mladen Gogala wrote: Good time accounting is the most compelling reason for having a wait event interface, like Oracle. Without the wait event interface, one cannot really tell where the time is spent, at least not without profiling the database code, which is

Re: [PERFORM] Table partitioning problem

2011-03-09 Thread Jim Nasby
On Mar 8, 2011, at 9:45 AM, Samba GUEYE wrote: I have a problem with table partitioning because i have a foreign key applied on the partionned table and it throw a constraint violation error during inserts. I saw on the manual

Re: [PERFORM] NULLS LAST performance

2011-03-09 Thread Jim Nasby
On Feb 24, 2011, at 3:47 AM, Mathieu De Zutter wrote: which will index optimize your sql. Interesting that 'null last' fools disallows index usage even when the index was created with nullls last as the default. The problem is that his query needs to scan the index in DESC order, which

Re: [PERFORM] Shouldn't we have a way to avoid risky plans?

2011-05-04 Thread Jim Nasby
On Mar 24, 2011, at 5:23 PM, Claudio Freire wrote: I routinely have to work around query inefficiencies because GEQO does something odd - and since postgres gives me too few tools to tweak plans (increase statistics, use subqueries, rephrase joins, no direct tool before CTEs which are rather

Re: [PERFORM] reducing random_page_cost from 4 to 2 to force index scan

2011-05-17 Thread Jim Nasby
On May 16, 2011, at 10:46 AM, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Mon, May 16, 2011 at 12:49 AM, Jesper Krogh jes...@krogh.cc wrote: Ok, it may not work as well with index'es, since having 1% in cache may very well mean that 90% of all requested blocks are there.. for

Re: [PERFORM] KVP table vs. hstore - hstore performance (Was: Postgres NoSQL emulation)

2011-05-17 Thread Jim Nasby
On May 16, 2011, at 8:47 AM, Merlin Moncure wrote: On Sat, May 14, 2011 at 5:10 AM, Stefan Keller sfkel...@gmail.com wrote: Hi, I am conducting a benchmark to compare KVP table vs. hstore and got bad hstore performance results when the no. of records is greater than about 500'000. CREATE

Re: [PERFORM] reducing random_page_cost from 4 to 2 to force index scan

2011-05-19 Thread Jim Nasby
On May 19, 2011, at 9:53 AM, Robert Haas wrote: On Wed, May 18, 2011 at 11:00 PM, Greg Smith g...@2ndquadrant.com wrote: Jim Nasby wrote: I think the challenge there would be how to define the scope of the hot-spot. Is it the last X pages? Last X serial values? Something like correlation

Re: [PERFORM] seq scan in the case of max() on the primary key column

2011-06-17 Thread Jim Nasby
On Jun 16, 2011, at 1:36 PM, Shaun Thomas wrote: /** * Return the Maximum INT Value for a Partitioned Table Column * * @param string Name of Schema of the base partition table. * @param string Name of the base partition table. * @param string Name of column to search. */ CREATE OR

Re: [PERFORM] Infinite Cache

2011-07-01 Thread Jim Nasby
On Jul 1, 2011, at 9:43 AM, Anthony Presley wrote: Was curious if there was some sort of Open Source version of Infinite Cache, and/or a memcache layer that can be dropped in front of PostgreSQL without application changes (which seems to be the key piece of Infinite Cache), or is this

Re: [PERFORM] index not being used when variable is sent

2011-08-17 Thread Jim Nasby
On Aug 17, 2011, at 1:49 AM, Eyal Wilde wrote: 1. is there any more elegant solution? Very possibly, but I'm having a heck of a time trying to figure out what your current code is actually doing. What's the actual problem you're trying to solve here? -- Jim C. Nasby, Database Architect

Re: [PERFORM] Need to tune for Heavy Write

2011-08-17 Thread Jim Nasby
On Aug 4, 2011, at 10:07 AM, Scott Marlowe wrote: On Thu, Aug 4, 2011 at 7:57 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: RAM : 16 GB effective_cache_size = 4096MB That should probably be more like 12GB to 15GB. It probably won't affect the load time here, but could affect

Re: [PERFORM] heavy load-high cpu itilization

2011-08-17 Thread Jim Nasby
On Jul 30, 2011, at 3:02 PM, Filippos wrote: thx a lot for your answer. i will provide some stats, so if you could help me figure out the source of the problem that would be great -*top -c* Tasks: 1220 total, 49 running, 1171 sleeping, 0 stopped, 0 zombie Cpu(s): *84.1%us*, 2.8%sy,

Re: [PERFORM] Summaries on SSD usage?

2011-09-01 Thread Jim Nasby
On Aug 30, 2011, at 12:23 PM, Stefan Keller wrote: I'm looking for summaries (or best practices) on SSD usage with PostgreSQL. My use case is mainly a read-only database. Are there any around? I'm not sure, but for read-only why not just put more memory in the server? It'll be a lot cheaper

Re: [PERFORM] How clustering for scale out works in PostgreSQL

2013-09-12 Thread Jim Nasby
On 8/31/13 9:44 AM, Kevin Grittner wrote: bsreejithin bsreejit...@gmail.com wrote: What I posted is about a new setup that's going to come up..Discussions are on whether to setup DB cluster to handle 1000 concurrent users. I previously worked for Wisconsin Courts, where we had a single

Re: [PERFORM] Optimising views

2013-09-12 Thread Jim Nasby
On 8/29/13 9:22 PM, Bastiaan Olij wrote: Work well enough.. But as I'm using the same data in different reports and I though a view might be smart. So I created a view: create view v_costs as select dcjobid as costjobid, sum(dcamount) as costamount from directcosts group by dcjobid

Re: [PERFORM] Varchar vs foreign key vs enumerator - table and index size

2013-09-12 Thread Jim Nasby
On 8/31/13 8:35 AM, Łukasz Walkowski wrote: 3. And this part is most interesting for me. Columns browser, eventsource, eventtype, devicetype, operatingsystem contain a small pool of strings - for example for devicetype this is set to Computer, Mobile, Tablet or Unknown. Browser is set to

[PERFORM] Problems with hash join over nested loop

2013-10-28 Thread Jim Nasby
= n.customer_id) Total runtime: 257838.571 ms (57 rows) -- Jim Nasby, Lead Data Architect (512) 569-9461 -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Problems with hash join over nested loop

2013-10-28 Thread Jim Nasby
On 10/28/13 6:13 PM, Tom Lane wrote: Jim Nasby jna...@enova.com writes: I've been working on trying to normalize a table that's got a bunch of text fields. Normalizing the first 4 has been a non-issue. But when I try and normalize 2 additional fields a bunch of query plans go belly-up. Try

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Jim Nasby
On 10/29/13 9:10 AM, Merlin Moncure wrote: On Mon, Oct 28, 2013 at 6:13 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jim Nasby jna...@enova.com writes: I've been working on trying to normalize a table that's got a bunch of text fields. Normalizing the first 4 has been a non-issue. But when I try

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Jim Nasby
On 10/29/13 11:45 AM, Tom Lane wrote: Jim Nasby jna...@enova.com writes: I'm also wondering if it's time to raise those limits. Yeah, possibly. The current default values were set on machines much smaller/slower than most current hardware. I think also that the collapse limits were invented

Re: [PERFORM] Problems with hash join over nested loop

2013-10-29 Thread Jim Nasby
On 10/29/13 1:20 PM, Tom Lane wrote: Jim Nasby jna...@enova.com writes: On 10/29/13 11:45 AM, Tom Lane wrote: Jim Nasby jna...@enova.com writes: I'm also wondering if it's time to raise those limits. Yeah, possibly. The current default values were set on machines much smaller/slower than

Re: [PERFORM] Problems with hash join over nested loop

2013-10-30 Thread Jim Nasby
On 10/29/13 3:36 PM, Tom Lane wrote: Jim Nasby jna...@enova.com writes: Is there a way to measure memory consumption during planning, short of something like strace? (I've got no dev tools available on our servers.) Nothing built-in, I'm pretty sure. You could probably add some

Re: [PERFORM] Problems with hash join over nested loop

2013-10-30 Thread Jim Nasby
On 10/30/13 5:27 PM, Jim Nasby wrote: On 10/29/13 3:36 PM, Tom Lane wrote: Jim Nasby jna...@enova.com writes: Is there a way to measure memory consumption during planning, short of something like strace? (I've got no dev tools available on our servers.) Nothing built-in, I'm pretty sure

[PERFORM] Horrific time for getting 1 record from an index?

2013-11-11 Thread Jim Nasby
a hard time believing it actually too 257 seconds to get 2 pages (one index, one heap) back from our SAN. Am I missing something here? -- Jim Nasby, Lead Data Architect (512) 569-9461 -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription

Re: [PERFORM] Horrific time for getting 1 record from an index?

2013-11-11 Thread Jim Nasby
On 11/11/13 3:51 PM, Daniel Farina wrote: On Mon, Nov 11, 2013 at 1:48 PM, Jim Nasby jna...@enova.com wrote: Postgres 9.1.9. explain analyze select min(insert_time) from cnu_stats.page_hits_raw ; I checked and there were no un-granted locks... but I have a hard time believing it actually too

Re: [PERFORM] Horrific time for getting 1 record from an index?

2013-11-11 Thread Jim Nasby
On 11/11/13 4:57 PM, Jeff Janes wrote: On Mon, Nov 11, 2013 at 1:57 PM, Jim Nasby jna...@enova.com mailto:jna...@enova.com wrote: Btree indexes have special code that kill index-tuples when the table-tuple is dead-to-all, so only the first such query after the mass deletion becomes vacuum

Re: [PERFORM] Horrific time for getting 1 record from an index?

2013-11-12 Thread Jim Nasby
enough to the vacuum that I don't think visibility of these tuples would have changed mid-stream). -- Jim Nasby, Lead Data Architect (512) 569-9461 -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [PERFORM] updating statistics on slow running query

2014-11-14 Thread Jim Nasby
by T.form_id; I hope it works have a nice day -- Matteo Durighetto - - - - - - - - - - - - - - - - - - - - - - - Italian PostgreSQL User Group http://www.itpug.org/index.it.html Italian Community for Geographic Free/Open-Source Software http://www.gfoss.it -- Jim

Re: [PERFORM] pgtune + configurations with 9.3

2014-11-14 Thread Jim Nasby
SSD storage) but probably still good for this test. Awesome! If there's possibility of developers getting direct access, I suspect folks on -hackers would be interested. If not but you're willing to run tests for folks, they'd still be interested. :) -- Jim Nasby, Data Architect, Blue Treble

Re: [PERFORM] Index order ignored after `is null` in query

2014-11-14 Thread Jim Nasby
be useful, though for different reasons. -- Jim Nasby, Data Architect, Blue Treble Consulting Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [PERFORM] why pg_class.relfrozenxid needs to be updated for frozen tables (where all rows have xmin=2)?

2015-02-02 Thread Jim Nasby
On 2/2/15 7:36 PM, Jim Nasby wrote: Currently the fact that it needs to go back to old tables and FTS them every 2B transactions (or rely on autovacuum for this) and you can't do anything about it (like permanently freeze the tables) seems like a big scalability issue. Does

Re: [PERFORM] why pg_class.relfrozenxid needs to be updated for frozen tables (where all rows have xmin=2)?

2015-02-02 Thread Jim Nasby
relfrozenxid as soon as someone inserts a new tuple in the relation. It might be possible to tie this into the visibility map, but no one has looked at this yet. Perhaps you'd be willing to investigate this, or sponsor the work? Thank you. On Fri, Jan 30, 2015 at 5:28 PM, Jim Nasby jim.na

Re: [PERFORM] working around JSONB's lack of stats?

2015-02-02 Thread Jim Nasby
On 2/1/15 3:08 PM, Josh Berkus wrote: On 01/30/2015 05:34 PM, Jim Nasby wrote: On 1/30/15 2:26 PM, Josh Berkus wrote: This would probably work because there aren't a lot of data structures where people would have the same key:value pair in different locations in the JSON, and care about

Re: [PERFORM] why pg_class.relfrozenxid needs to be updated for frozen tables (where all rows have xmin=2)?

2015-02-02 Thread Jim Nasby
On 2/2/15 9:37 PM, Slava Mudry wrote: On Mon, Feb 2, 2015 at 5:52 PM, Jim Nasby jim.na...@bluetreble.com mailto:jim.na...@bluetreble.com wrote: On 2/2/15 7:36 PM, Jim Nasby wrote: Currently the fact that it needs to go back to old tables and FTS them

Re: [PERFORM] working around JSONB's lack of stats?

2015-01-30 Thread Jim Nasby
performance and have any significant size of data you've also indexed parts of the JSON, yes? -- Jim Nasby, Data Architect, Blue Treble Consulting Data in Trouble? Get it in Treble! http://BlueTreble.com -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes

  1   2   >