Re: [PERFORM] SSD + RAID

2010-02-23 Thread david
On Mon, 22 Feb 2010, Ron Mayer wrote: Also worth noting - Linux's software raid stuff (MD and LVM) need to handle this right as well - and last I checked (sometime last year) the default setups didn't. I think I saw some stuff in the last few months on this issue on the kernel mailing

Re: [PERFORM] SSD + RAID

2010-02-23 Thread Pierre C
Note that's power draw per bit. dram is usually much more densely packed (it can be with fewer transistors per cell) so the individual chips for each may have similar power draws while the dram will be 10 times as densely packed as the sram. Differences between SRAM and DRAM : - price per

Re: [PERFORM] SSD + RAID

2010-02-23 Thread Nikolas Everett
On Tue, Feb 23, 2010 at 6:49 AM, Pierre C li...@peufeu.com wrote: Note that's power draw per bit. dram is usually much more densely packed (it can be with fewer transistors per cell) so the individual chips for each may have similar power draws while the dram will be 10 times as densely

Re: [PERFORM] Slow query: table iteration (8.3)

2010-02-23 Thread Yeb Havinga
Glenn Maynard wrote: CREATE FUNCTION high_score_for_user(user_id int) RETURNS SETOF INTEGER LANGUAGE SQL AS $$ SELECT t.id FROM test t WHERE t.user_id = $1 ORDER BY t.score DESC LIMIT 1 $$; SELECT high_score_for_user(u.id) FROM test_users u; runs in 100ms. Though it

[PERFORM] Internal operations when the planner makes a hash join.

2010-02-23 Thread negora
Hello: I'm an ignorant in what refers to performance analysis of PostgreSQL. I've a doubt about how the PostgreSQL planner makes a hash join. I've tried to dig into the archive of this mailing list but I haven't found what I'm looking for. So I'm explaining my doubt with an example to see if

Re: [PERFORM] Internal operations when the planner makes a hash join.

2010-02-23 Thread Kevin Grittner
negora neg...@negora.com wrote: I've a doubt about how the PostgreSQL planner makes a hash join. Let's suppose that I've 2 tables, one of students and the other one of parents in a many-to-one relation. I want to do something like this: SELECT s.complete_name, f.complete_name

Re: [PERFORM] Internal operations when the planner makes a hash join.

2010-02-23 Thread negora
First of all, thank you for your fast answer, Kevin :) . However I still wonder if on the search into the hashed table (stored in the RAM, as you're pointing out), it checks for fathers as many times as students are selected, or if the engine uses some kind of intelligent heuristic to avoid

Re: [PERFORM] Internal operations when the planner makes a hash join.

2010-02-23 Thread Alvaro Herrera
negora wrote: According to how I understood the process, the engine would get the name from the student with ID 1 and would look for the name of the father with ID 1 in the hashed table. It'd do exactly the same with the student #2 and father #2. But my big doubt is about the 3rd one

Re: [PERFORM] SSD + RAID

2010-02-23 Thread Scott Carey
On Feb 23, 2010, at 3:49 AM, Pierre C wrote: Now I wonder about something. SSDs use wear-leveling which means the information about which block was written where must be kept somewhere. Which means this information must be updated. I wonder how crash-safe and how atomic these updates

Re: [PERFORM] Internal operations when the planner makes a hash join.

2010-02-23 Thread Scott Carey
On Feb 23, 2010, at 8:53 AM, Alvaro Herrera wrote: negora wrote: According to how I understood the process, the engine would get the name from the student with ID 1 and would look for the name of the father with ID 1 in the hashed table. It'd do exactly the same with the student #2 and

Re: [PERFORM] SSD + RAID

2010-02-23 Thread david
On Tue, 23 Feb 2010, da...@lang.hm wrote: On Mon, 22 Feb 2010, Ron Mayer wrote: Also worth noting - Linux's software raid stuff (MD and LVM) need to handle this right as well - and last I checked (sometime last year) the default setups didn't. I think I saw some stuff in the last few

Re: [PERFORM] SSD + RAID

2010-02-23 Thread Aidan Van Dyk
* da...@lang.hm da...@lang.hm [100223 15:05]: However, one thing that you do not get protection against with software raid is the potential for the writes to hit some drives but not others. If this happens the software raid cannot know what the correct contents of the raid stripe are, and

Re: [PERFORM] SSD + RAID

2010-02-23 Thread david
On Tue, 23 Feb 2010, Aidan Van Dyk wrote: * da...@lang.hm da...@lang.hm [100223 15:05]: However, one thing that you do not get protection against with software raid is the potential for the writes to hit some drives but not others. If this happens the software raid cannot know what the

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-23 Thread Kevin Grittner
Greg Smith g...@2ndquadrant.com wrote: You can easily quantify if the BGW is aggressive enough. Buffers leave the cache three ways, and they each show up as separate counts in pg_stat_bgwriter: buffers_checkpoint, buffers_clean (the BGW), and buffers_backend (the queries). Cranking it up

Re: [PERFORM] SSD + RAID

2010-02-23 Thread Mark Mielke
On 02/23/2010 04:22 PM, da...@lang.hm wrote: On Tue, 23 Feb 2010, Aidan Van Dyk wrote: * da...@lang.hm da...@lang.hm [100223 15:05]: However, one thing that you do not get protection against with software raid is the potential for the writes to hit some drives but not others. If this happens

Re: [PERFORM] Internal operations when the planner makes a hash join.

2010-02-23 Thread negora
Thank you for explaining me the internal behaviour of the PostgreSQL engine. I'll try to look for more information about that hash tables. It sounds really really interesting. Your information was very useful. The origin of my doubt resides in the fact that I need to do a joint between 3 HUGE

Re: [PERFORM] Internal operations when the planner makes a hash join.

2010-02-23 Thread Kevin Grittner
negora neg...@negora.com wrote: The origin of my doubt resides in the fact that I need to do a joint between 3 HUGE tables (millions of registries) and do certain operations with the retrieved information. I was deciding whether to use one SELECT with 3 JOINs, as I've been doing since the

Re: [PERFORM] moving pg_xlog -- yeah, it's worth it!

2010-02-23 Thread Ben Chobot
On Feb 23, 2010, at 2:23 PM, Kevin Grittner wrote: Here are the values from our two largest and busiest systems (where we found the pg_xlog placement to matter so much). It looks to me like a more aggressive bgwriter would help, yes? cir= select * from pg_stat_bgwriter ; -[ RECORD 1

Re: [PERFORM] Internal operations when the planner makes a hash join.

2010-02-23 Thread Kevin Grittner
negora neg...@negora.com wrote: I even might return the entire result to my external Java application You are probably going to want to configure it to use a cursor, at least if the result set is large (i.e., too big to cache the entire result set in memory before you read the first row).

[PERFORM] Thx and additional Q's .....

2010-02-23 Thread Dave Crooke
Thanks Joe. 1. In my case, I'm erring on the side of not using the limited partitioning support in PG 8.3, which we're using because I'm generating new tables all the time, I need to dynamically generate the DML anyway, and it's actually less code to just do my own calculation on the

Re: [PERFORM] Planner question - bit data types

2010-02-23 Thread Scott Carey
On Sep 7, 2009, at 7:05 PM, Karl Denninger wrote: The individual boolean fields don't kill me and in terms of some of the application issues they're actually rather easy to code for. The problem with re-coding for them is extensibility (by those who install and administer the package); a mask