Re: [HACKERS] New style of hash join proposal

2008-04-07 Thread Gregory Stark
Bruce Momjian [EMAIL PROTECTED] writes: If the scanning of the inner side is a performance problem, why would we be choosing a nested loop in the first place, vs. another type of join? I clearly haven't done a good job explaining this as nobody seems to getting what I'm describing. I think I'm

Re: [HACKERS] New style of hash join proposal

2008-04-06 Thread Bruce Momjian
Gregory Stark wrote: We currently execute a lot of joins as Nested Loops which would be more efficient if we could batch together all the outer keys and execute a single inner bitmap index scan for all of them together. Essentially what I'm saying is that we're missing a trick with Hash

Re: [HACKERS] New style of hash join proposal

2008-03-17 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Gregory Stark [EMAIL PROTECTED] writes: We currently execute a lot of joins as Nested Loops which would be more efficient if we could batch together all the outer keys and execute a single inner bitmap index scan for all of them together. Please give an

Re: [HACKERS] New style of hash join proposal

2008-03-17 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: Please give an example of what you're talking about that you think we can't do now. Note that we're doing a full sequential scan of a even though we've already finished hashing b and know full well which keys we'll

Re: [HACKERS] New style of hash join proposal

2008-03-17 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: Please give an example of what you're talking about that you think we can't do now. Note that we're doing a full sequential scan of a even though we've already finished hashing b

Re: [HACKERS] New style of hash join proposal

2008-03-17 Thread Gregory Stark
Gregory Stark [EMAIL PROTECTED] writes: It would be ideal if it could scan the invoices using an index, toss them all in a hash, then do a bitmap index scan to pull out all the matching detail records. If there are multiple batches it can start a whole new index scan for the each of the

Re: [HACKERS] New style of hash join proposal

2008-03-17 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: It would be ideal if it could scan the invoices using an index, toss them all in a hash, then do a bitmap index scan to pull out all the matching detail records. If there are multiple batches it can start a whole new index scan for the each of the

Re: [HACKERS] New style of hash join proposal

2008-03-17 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Gregory Stark [EMAIL PROTECTED] writes: It would be ideal if it could scan the invoices using an index, toss them all in a hash, then do a bitmap index scan to pull out all the matching detail records. If there are multiple batches it can start a whole new

Re: [HACKERS] New style of hash join proposal

2008-03-17 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: I don't understand which part of we can do that now isn't clear to you. Uh, except we can't. I already demonstrated that we could. If the problem is that the planner is cutting over from one plan type to the other at

Re: [HACKERS] New style of hash join proposal

2008-03-17 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: I don't understand which part of we can do that now isn't clear to you. Uh, except we can't. I already demonstrated that we could. If the problem is that the planner is cutting

Re: [HACKERS] New style of hash join proposal

2008-03-17 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: I already demonstrated that we could. We seem to be talking past each other. The plan you showed is analogous but using a plain old index scan. That's only because that seemed like the appropriate thing for the given

Re: [HACKERS] New style of hash join proposal

2008-03-17 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: I already demonstrated that we could. We seem to be talking past each other. The plan you showed is analogous but using a plain old index scan. That's only because that seemed

Re: [HACKERS] New style of hash join proposal

2008-03-17 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: Nested Loop (cost=5.39..198.81 rows=51 width=244) - HashAggregate (cost=1.06..1.11 rows=5 width=4) - Seq Scan on int4_tbl b (cost=0.00..1.05 rows=5 width=4) - Bitmap Heap Scan on tenk1 a (cost=4.33..39.41

Re: [HACKERS] New style of hash join proposal

2008-03-17 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: Nested Loop (cost=5.39..198.81 rows=51 width=244) - HashAggregate (cost=1.06..1.11 rows=5 width=4) - Seq Scan on int4_tbl b (cost=0.00..1.05 rows=5 width=4) - Bitmap Heap

Re: [HACKERS] New style of hash join proposal

2008-03-16 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: We currently execute a lot of joins as Nested Loops which would be more efficient if we could batch together all the outer keys and execute a single inner bitmap index scan for all of them together. Please give an example of what you're talking about

Re: [HACKERS] New style of hash join proposal

2007-12-21 Thread Bruce Momjian
Decibel! wrote: I fear the real complexity would be (as always) in the planner rather than the executor. I haven't really looked into what it would take to arrange this or how to decide when to do it. TODO? This email was added to the 8.4 queue:

Re: [HACKERS] New style of hash join proposal

2007-12-19 Thread Decibel!
On Dec 13, 2007, at 7:13 AM, Gregory Stark wrote: We currently execute a lot of joins as Nested Loops which would be more efficient if we could batch together all the outer keys and execute a single inner bitmap index scan for all of them together. Essentially what I'm saying is that we're

[HACKERS] New style of hash join proposal

2007-12-13 Thread Gregory Stark
We currently execute a lot of joins as Nested Loops which would be more efficient if we could batch together all the outer keys and execute a single inner bitmap index scan for all of them together. Essentially what I'm saying is that we're missing a trick with Hash Joins which currently