Re: [PERFORM] Horribly slow hash join

2004-04-18 Thread Dennis Bjorklund
On Sat, 17 Apr 2004, Tom Lane wrote: *some* set of inputs. (Also, I have been harboring some notions of supporting cross-type hash joins for integer types, which will not work unless small int8 values hash the same as int4 etc.) The simple solution would be to always extend integers to 64

Re: [PERFORM] Horribly slow hash join

2004-04-18 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: Greg Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: (Also, I have been harboring some notions of supporting cross-type hash joins for integer types, which will not work unless small int8 values hash the same as int4 etc.) The

Re: [PERFORM] Horribly slow hash join

2004-04-18 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: Eh? Oh, negative numbers? So low^high^sign. [ thinks about it... ] Yeah, that would work. We can't backpatch it without breaking existing hash indexes on int8, but it'd be reasonable to change for 7.5 (since at the rate things are going, we won't have

Re: [PERFORM] Horribly slow hash join

2004-04-18 Thread Tom Lane
Dennis Bjorklund [EMAIL PROTECTED] writes: On Sat, 17 Apr 2004, Tom Lane wrote: *some* set of inputs. (Also, I have been harboring some notions of supporting cross-type hash joins for integer types, which will not work unless small int8 values hash the same as int4 etc.) The simple solution

Re: [PERFORM] Horribly slow hash join

2004-04-18 Thread Tom Lane
Dennis Bjorklund [EMAIL PROTECTED] writes: What do you mean? int8 is supported on all platformas No it isn't. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend

Re: [PERFORM] Horribly slow hash join

2004-04-18 Thread Dennis Bjorklund
On Sun, 18 Apr 2004, Tom Lane wrote: That creates portability issues though. We do not depend on there being a 64-bit-int type for anything except int8 itself, and I don't want to start doing so. What do you mean? int8 is supported on all platformas and if the hasfunction would convert all

Re: [PERFORM] Horribly slow hash join

2004-04-18 Thread Dennis Bjorklund
On Sun, 18 Apr 2004, Tom Lane wrote: What do you mean? int8 is supported on all platformas No it isn't. So on platforms where it isn't you would use int4 as the biggest int then. I don't really see that as a problem. As long as you calculate the hash on the biggest int on that platform it

Re: [PERFORM] Toooo many context switches (maybe SLES8?)

2004-04-18 Thread Dave Cramer
Isn't this a linux kernel issue ? My understanding is that the scheduler doesn't know that 2 of the CPU's are actually the same underlying hardware and sometimes two contexts end up fighting for the same underlying chip? --dc-- On Thu, 2004-04-15 at 16:37, Josh Berkus wrote: Folks, I am

Re: [PERFORM] sunquery and estimated rows

2004-04-18 Thread Markus Bertheau
, 17.04.2004, 01:45, Tom Lane : The planner sees that as where scope = some complicated expression and falls back to a default estimate. It won't simplify a sub-select to a constant. (Some people consider that a feature ;-).) Why? Thanks -- Markus Bertheau [EMAIL PROTECTED]

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-18 Thread Tom Lane
After some further digging I think I'm starting to understand what's up here, and the really fundamental answer is that a multi-CPU Xeon MP box sucks for running Postgres. I did a bunch of oprofile measurements on a machine belonging to one of Josh's clients, using a test case that involved heavy

Re: [PERFORM] sunquery and estimated rows

2004-04-18 Thread Tom Lane
Markus Bertheau [EMAIL PROTECTED] writes: В Сбт, 17.04.2004, в 01:45, Tom Lane пишет: The planner sees that as where scope = some complicated expression and falls back to a default estimate. It won't simplify a sub-select to a constant. (Some people consider that a feature ;-).) Why? It's

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-18 Thread Dave Cramer
So the the kernel/OS is irrelevant here ? this happens on any dual xeon? What about hypterthreading does it still happen if HTT is turned off ? Dave On Sun, 2004-04-18 at 17:47, Tom Lane wrote: After some further digging I think I'm starting to understand what's up here, and the really

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-18 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: So in the short term I think we have to tell people that Xeon MP is not the most desirable SMP platform to run Postgres on. (Josh thinks that the specific motherboard chipset being used in these machines might share some of the blame too. I don't have any

Re: [PERFORM] sunquery and estimated rows

2004-04-18 Thread Tom Lane
Rod Taylor [EMAIL PROTECTED] writes: It's the only way to prevent it from simplifying when you don't want it to. I'm having a difficult time coming up with a circumstance where that is beneficial except when stats are out of whack. Try trawling the archives --- I recall several cases in

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-18 Thread Tom Lane
Dave Cramer [EMAIL PROTECTED] writes: So the the kernel/OS is irrelevant here ? this happens on any dual xeon? I believe so. The context-switch behavior might possibly be a little more pleasant on other kernels, but the underlying spinlock problem is not dependent on the kernel. What about

Re: [PERFORM] Horribly slow hash join

2004-04-18 Thread Bruno Wolff III
On Sun, Apr 18, 2004 at 18:27:09 +0200, Dennis Bjorklund [EMAIL PROTECTED] wrote: On Sun, 18 Apr 2004, Tom Lane wrote: What do you mean? int8 is supported on all platformas No it isn't. So on platforms where it isn't you would use int4 as the biggest int then. I don't really see

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-18 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: There's nothing about the way Postgres spinlocks are coded that affects this? No. AFAICS our spinlock sequences are pretty much equivalent to the way the Linux kernel codes its spinlocks, so there's no deep dark knowledge to be mined there. We could

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-18 Thread Tom Lane
What about hypterthreading does it still happen if HTT is turned off ? The problem comes from keeping the caches synchronized between multiple physical CPUs. AFAICS enabling HTT wouldn't make it worse, because a hyperthreaded processor still only has one cache. Also, I forgot to say that

Re: [PERFORM] Horribly slow hash join

2004-04-18 Thread Dennis Bjorklund
On Sun, 18 Apr 2004, Bruno Wolff III wrote: Another option would be to put the numbers into two int4s. For int4 or smaller types one of these would be zero. int8s would be split between the two. The hash function would then be defined on the two int4s. Sure, this is an internal calculation in