[PERFORM] Strange workaround for slow query

2010-03-10 Thread sverhagen


Hi group,


We have two related tables with event types and events. We query for a join
between these two tables and experience that, when there is an
to-be-expected very small result set, this query performs particularly
poor. Understanding in this matter would be appreciated.

SELECT * from events_event_types WHERE id IN (71,999);
 id |  name  | severity
++--
 71 | Xenteo Payment handled |   20
(1 row)


Following original query returns zero rows (as to be expected on what I
showed above) and takes (relatively) a lot of time doing so:

SELECT * FROM events_events LEFT OUTER JOIN events_event_types ON
eventType_id=events_event_types.id WHERE severity=70 AND (eventType_id IN
(71)) ORDER BY datetime DESC LIMIT 50;
 id | carparkid | cleared | datetime | identity | generatedbystationid |
eventtype_id | relatedstationid | processingstatus | id | name | severity
+---+-+--+--+--+--+--+--++--+--
(0 rows)
Time: 397.564 ms

Following query is much alike the original query, but I changed the "WHERE
severity". It returns the number of rows are requested in LIMIT and takes
only little time doing so:

SELECT * FROM events_events LEFT OUTER JOIN events_event_types ON
eventType_id=events_event_types.id WHERE severity=20 AND (eventType_id IN
(71)) ORDER BY datetime DESC limit 50;
...
(50 rows)
Time: 1.604 ms

The latter much to prove that this is a problem related to small result
sets.

Following query is much alike the original query, although I've added a
dummy value (non-existent in event types table; "999") to the WHERE IN
clause. It returns the same zero rows and takes only little time doing so:

SELECT * FROM events_events LEFT OUTER JOIN events_event_types ON
eventType_id=events_event_types.id WHERE severity=70 AND (eventType_id IN
(71, 999)) ORDER BY datetime DESC LIMIT 50;
 id | carparkid | cleared | datetime | identity | generatedbystationid |
eventtype_id | relatedstationid | processingstatus | id | name | severity
+---+-+--+--+--+--+--+--++--+--
(0 rows)
Time: 1.340 ms

Now I have at least two possibilities:
- Implementing the dummy value as shown above in my source code to improve
query performance (dirty but effective)
- Further investigating what is going on, which at this point is something
I need help with
Thanks for your assistance in this matter!


Following are a number of details to describe the environment that this is
seen in.

SELECT version();
PostgreSQL 8.3.7 on i486-pc-linux-gnu, compiled by GCC cc (GCC) 4.2.4
(Ubuntu 4.2.4-1ubuntu3)

Postgres was installed as Debian package in Ubuntu 8.04 LTS.

SELECT count(*) FROM events_events;
7619991
SELECT count(*) FROM events_events WHERE eventtype_id=71;
50348
SELECT count(*) FROM events_event_types;
82

\d events_event_types
  Table "public.events_event_types"
  Column  |  Type  |Modifiers
--++-
 id   | bigint | not null default nextval
('events_event_types_id_seq'::regclass)
 name | character varying(255) | not null
 severity | bigint | not null
Indexes:
"events_event_types_pkey" PRIMARY KEY, btree (id)
"events_event_types_name_key" UNIQUE, btree (name)
"events_event_types_severity_ind" btree (severity)
"test_events_eventtypes_id_severity_ind" btree (id, severity)
"test_events_eventtypes_severity_id_ind" btree (severity, id)

\d events_events
 Table "public.events_events"
Column|   Type   |
Modifiers
--+--+
 id   | bigint   | not null default nextval
('events_events_id_seq'::regclass)
 carparkid| bigint   |
 cleared  | boolean  | not null
 datetime | timestamp with time zone |
 identity | character varying(255)   |
 generatedbystationid | bigint   |
 eventtype_id | bigint   | not null
 relatedstationid | bigint   |
 processingstatus | character varying(255)   | not null
Indexes:
"events_events_pkey" PRIMARY KEY, btree (id)
"events_events_cleared_ind" btree (cleared)
"events_events_datetime_eventtype_id_ind" btree (datetime,
eventtype_id)
"events_events_datetime_ind" btree (datetime)
"events_events_eventtype_id_datetime_ind" btree (eventtype_id,
datetime)
"events_events_eventtype_id_ind" btree (eventtype_id)
"events_events_identity_ind" btree (identity)
"events_eve

Re: [PERFORM] Strange workaround for slow query

2010-03-10 Thread sverhagen
Hi,


EXPLAIN SELECT * FROM events_events LEFT OUTER JOIN events_event_types ON
eventType_id=events_event_types.id WHERE severity=20 AND (eventType_id
IN (71)) ORDER BY datetime DESC limit 50;
QUERY PLAN
---
 Limit  (cost=3.23..200.31 rows=50 width=131)
   ->  Nested Loop  (cost=3.23..49139.16 rows=12466 width=131)
 ->  Index Scan Backward using
events_events_eventtype_id_datetime_ind on events_events
(cost=0.00..48886.61 rows=12466 width=93)
   Index Cond: (eventtype_id = 71)
 ->  Materialize  (cost=3.23..3.24 rows=1 width=38)
   ->  Seq Scan on events_event_types  (cost=0.00..3.23 rows=1
width=38)
 Filter: ((id = 71) AND (severity = 20))


EXPLAIN SELECT * FROM events_events LEFT OUTER JOIN events_event_types ON
eventType_id=events_event_types.id WHERE severity=70 AND (eventType_id
IN (71, 999)) ORDER BY datetime DESC LIMIT 50;

QUERY PLAN

 Limit  (cost=27290.24..27290.37 rows=50 width=131)
   ->  Sort  (cost=27290.24..27303.15 rows=5164 width=131)
 Sort Key: events_events.datetime
 ->  Nested Loop  (cost=22.95..27118.70 rows=5164 width=131)
   ->  Seq Scan on events_event_types  (cost=0.00..3.02 rows=17
width=38)
 Filter: (severity = 70)
   ->  Bitmap Heap Scan on events_events  (cost=22.95..1589.94
rows=408 width=93)
 Recheck Cond: ((events_events.eventtype_id = ANY
('{71,999}'::bigint[])) AND (events_events.eventtype_id =
events_event_types.id))
 ->  Bitmap Index Scan on
test_events_events_eventtype_id_severity_ind  (cost=0.00..22.85 rows=408
width=0)
   Index Cond: ((events_events.eventtype_id = ANY
('{71,999}'::bigint[])) AND (events_events.eventtype_id =
events_event_types.id))

By the way, sorry for my colleague Kees re-posting my message, but I was
under the assumption that my post did not make it into the group (as we
experienced in the past as well).

Groeten, best regards,


Sander Verhagen

Re: [PERFORM] Strange workaround for slow query

2010-03-10 Thread sverhagen

> Thanks - I'm sorry that I was not more specific earlier, but what would
> be *really* helpful is the output of explain analyze, since that also
> shows actual time, # rows and # loops of the inner nestloop.

No problem at all.


EXPLAIN ANALYZE SELECT * FROM events_events LEFT OUTER JOIN
events_event_types ON
eventType_id=events_event_types.id WHERE severity=20 AND (eventType_id
IN (71)) ORDER BY datetime DESC limit 50;
QUERY PLAN
--
 Limit  (cost=3.23..200.31 rows=50 width=131) (actual time=0.070..0.341
rows=50 loops=1)
   ->  Nested Loop  (cost=3.23..49139.16 rows=12466 width=131) (actual
time=0.069..0.309 rows=50 loops=1)
 ->  Index Scan Backward using
events_events_eventtype_id_datetime_ind on events_events
(cost=0.00..48886.61 rows=12466 width=93) (actual time=0.037..0.144 rows=50
loops=1)
   Index Cond: (eventtype_id = 71)
 ->  Materialize  (cost=3.23..3.24 rows=1 width=38) (actual
time=0.001..0.001 rows=1 loops=50)
   ->  Seq Scan on events_event_types  (cost=0.00..3.23 rows=1
width=38) (actual time=0.024..0.029 rows=1 loops=1)
 Filter: ((id = 71) AND (severity = 20))
 Total runtime: 0.415 ms
(8 rows)

Time: 1.290 ms


EXPLAIN ANALYZE SELECT * FROM events_events LEFT OUTER JOIN
events_event_types ON
eventType_id=events_event_types.id WHERE severity=70 AND (eventType_id
IN (71)) ORDER BY datetime DESC limit 50;
QUERY PLAN
-
 Limit  (cost=3.23..200.31 rows=50 width=131) (actual
time=11641.775..11641.775 rows=0 loops=1)
   ->  Nested Loop  (cost=3.23..49139.16 rows=12466 width=131) (actual
time=11641.773..11641.773 rows=0 loops=1)
 ->  Index Scan Backward using
events_events_eventtype_id_datetime_ind on events_events
(cost=0.00..48886.61 rows=12466 width=93) (actual time=0.035..11573.320
rows=50389 loops=1)
   Index Cond: (eventtype_id = 71)
 ->  Materialize  (cost=3.23..3.24 rows=1 width=38) (actual
time=0.000..0.000 rows=0 loops=50389)
   ->  Seq Scan on events_event_types  (cost=0.00..3.23 rows=1
width=38) (actual time=0.028..0.028 rows=0 loops=1)
 Filter: ((id = 71) AND (severity = 70))
 Total runtime: 11641.839 ms
(8 rows)

Time: 11642.902 ms


EXPLAIN ANALYZE SELECT * FROM events_events LEFT OUTER JOIN
events_event_types ON
eventType_id=events_event_types.id WHERE severity=70 AND (eventType_id
IN (71, 999)) ORDER BY datetime DESC LIMIT 50;
QUERY PLAN
---
 Limit  (cost=27290.26..27290.38 rows=50 width=131) (actual
time=0.118..0.118 rows=0 loops=1)
   ->  Sort  (cost=27290.26..27303.17 rows=5164 width=131) (actual
time=0.117..0.117 rows=0 loops=1)
 Sort Key: events_events.datetime
 Sort Method:  quicksort  Memory: 17kB
 ->  Nested Loop  (cost=22.95..27118.71 rows=5164 width=131)
(actual time=0.112..0.112 rows=0 loops=1)
   ->  Seq Scan on events_event_types  (cost=0.00..3.02 rows=17
width=38) (actual time=0.016..0.041 rows=16 loops=1)
 Filter: (severity = 70)
   ->  Bitmap Heap Scan on events_events  (cost=22.95..1589.94
rows=408 width=93) (actual time=0.002..0.002 rows=0 loops=16)
 Recheck Cond: ((events_events.eventtype_id = ANY
('{71,999}'::bigint[])) AND (events_events.eventtype_id =
events_event_types.id))
 ->  Bitmap Index Scan on
test_events_events_eventtype_id_severity_ind  (cost=0.00..22.85 rows=408
width=0) (actual time=0.001..0.001 rows=0 loops=16)
   Index Cond: ((events_events.eventtype_id = ANY
('{71,999}'::bigint[])) AND (events_events.eventtype_id =
events_event_types.id))
 Total runtime: 0.179 ms
(12 rows)

Time: 1.510 ms


> I'm
> wondering though why you do a left outer join. From the \d output in the
> previous mail, events_event.eventtype_id has a not null constraint and a
> fk to events_event_types.id, so an inner join would be appropriate.
> Outer joins limits the amount of join orders the planner considers, so a
> better plan might arise when the join is changed to inner.

I do agree with this assessment. I'm sort of improving the performance of
an existing implementation of ours, for which I'm not aware why they chose
for LEFT OUTER. I did, however, test things with INNER as well, with the
same results, so I decided to stick with what I encountered in the existing
implementation. But it's on my mind as well ;-)

[PERFORM] Performance trouble finding records through related records

2011-03-01 Thread sverhagen
Hi, appreciated mailing list. Thanks already for taking your time for my
performance question. Regards, Sander.


===POSTGRESQL VERSION AND ORIGIN===

PostgreSQL 8.3.9 on i486-pc-linux-gnu, compiled by GCC cc (GCC) 4.2.4
(Ubuntu 4.2.4-1ubuntu3)
Installed using "apt-get install postgresql-8.3"


===A DESCRIPTION OF WHAT YOU ARE TRYING TO ACHIEVE===

Query involving tables events_events and events_eventdetails. There is any
number of events_eventdetails records for each events_events record.

There may be multiple records in events_events that have the same value for
their transactionId, which is available in one of their events_eventdetails
records.

We want a total query that returns events_events records that match
condition I. or II., sorted by datetime descending, first 50.

Condition I.
All events_events records for which an events_eventdetails records that
matches the following conditions:
- Column keyname (in events_eventdetails) equals "customerId", and
- Column value (in events_eventdetails) equals 598124, or more precisely
substring(customerDetails.value,0,32)='598124'

Condition II.
All events_events records that have a same value for in one of their
events_eventdetails records with keyname 'transactionId' as any of the
resulting events_events records of condition I.

In other words: I want all events for a certain customerId, and all events
with the same transactionId as those.

The total query's time should be of the magnitude 100ms, but currently is of
the magnitude 1min.

JUST FOR THE PURPOSE OF EXPERIMENT I've now a denormalized copy of
transactionId as a column in the events_events records. Been trying queries
on those, with no improvements.

I am not seeking WHY my query is too slow, rather trying to find a way to
get it faster :-)


===THE EXACT TEXT OF THE QUERY YOU RAN===

The total query:

SELECT events1.id, events1.transactionId, events1.dateTime FROM
events_events events1
JOIN events_eventdetails customerDetails
ON events1.id = customerDetails.event_id
AND customerDetails.keyname='customer_id'
AND substring(customerDetails.value,0,32)='598124'
WHERE events1.eventtype_id IN
(100,103,105,106,45,34,14,87,58,78,7,76,11,25,57,98,30,35,33,49,52,28,85,59,23,22,51,48,36,65,66,18,13,86,75,44,38,43,94,56,95,96,71,50,81,90,89,16,17,88,79,77,68,97,92,67,72,53,2,10,31,32,80,24,93,26,9,8,61,5,73,70,63,20,60,40,41,39,101,104,107,99,64,62,55,69,19,46,47,15,21,27,54,12,102,108)
UNION
SELECT events2.id, events2.transactionId, events2.dateTime FROM
events_events events2
JOIN events_eventdetails details2_transKey
ON events2.id = details2_transKey.event_id
AND details2_transKey.keyname='transactionId'
JOIN events_eventdetails details2_transValue
ON substring(details2_transKey.value,0,32) =
substring(details2_transValue.value,0,32)
AND details2_transValue.keyname='transactionId'
JOIN events_eventdetails customerDetails
ON details2_transValue.event_id = customerDetails.event_id
AND customerDetails.keyname='customer_id'
AND substring(customerDetails.value,0,32)='598124'
WHERE events2.eventtype_id IN
(100,103,105,106,45,34,14,87,58,78,7,76,11,25,57,98,30,35,33,49,52,28,85,59,23,22,51,48,36,65,66,18,13,86,75,44,38,43,94,56,95,96,71,50,81,90,89,16,17,88,79,77,68,97,92,67,72,53,2,10,31,32,80,24,93,26,9,8,61,5,73,70,63,20,60,40,41,39,101,104,107,99,64,62,55,69,19,46,47,15,21,27,54,12,102,108)
ORDER BY dateTime DESC LIMIT 50


===THE EXACT OUTPUT OF THAT QUERY===

The exactly correct and desired output is as follows:

id|transactionid|  datetime
--+-+
 16336643 | | 2011-03-01 11:10:38.648+01
 16336642 | | 2011-03-01 11:10:35.629+01
 16336641 | | 2011-03-01 11:10:35.625+01
 16336637 | | 2011-03-01 11:09:53.306+01
 16336634 | | 2011-03-01 11:09:14.027+01
 16336633 | 26eaeb24-7a93-4c9a-99f9-bd3b77f9636 | 2011-03-01 11:09:14.004+01
 16336632 | 26eaeb24-7a93-4c9a-99f9-bd3b77f9636 | 2011-03-01 11:09:13.925+01
 16336631 | | 2011-03-01 11:09:13.873+01
 16336630 | | 2011-03-01 11:09:13.741+01
 16336626 | | 2011-03-01 11:09:08.931+01
 16336625 | | 2011-03-01 11:09:01.811+01
 16336624 | 2037f235-89d2-402a-90eb-3bcf40d633c | 2011-03-01 11:09:01.771+01
 16336623 | 2037f235-89d2-402a-90eb-3bcf40d633c | 2011-03-01 11:09:01.729+01
 16336611 | | 2011-03-01 11:08:08.63+01
 16336610 | | 2011-03-01 11:08:02.805+01
 16336609 | | 2011-03-01 11:08:02.801+01
 16336606 | | 2011-03-01 11:07:55.324+01
 16336602 |   

Re: [PERFORM] Performance trouble finding records through related records

2011-03-02 Thread sverhagen
Thanks for your help already!
Hope you're up for some more :-)


Andy Colson wrote:
> 
> First off, excellent detail.
> 
> Second, your explain analyze was hard to read... but since you are not
> really interested in your posted query, I wont worry about looking at
> it... but... have you seen:
> 
> http://explain.depesz.com/
> 

Thanks for that. Using it below :-)


Andy Colson wrote:
> 
> If you run the individual queries, without the union, are the part's slow
> too?
> 

Only problem is the second part. So that part can safely be isolated. Also
the following does not play a role at this point: WHERE events2.eventtype_id
IN
(100,103,105,...

Then I went ahead and denormalized the transactionId on both ends, so that
both events_events records and events_eventdetails records have the
transactionId (or NULL). That simplifies the query to this:

SELECT events_events.* FROM events_events WHERE transactionid IN (
SELECT transactionid FROM events_eventdetails customerDetails
WHERE customerDetails.keyname='customer_id'
AND substring(customerDetails.value,0,32)='1957'
AND transactionid IS NOT NULL
) ORDER BY id LIMIT 50;

To no avail. Also changing the above WHERE IN into implicit/explicit JOIN's
doesn't make more than a marginal difference. Should joining not be very
efficient somehow?

http://explain.depesz.com/s/Pnb

The above link nicely shows the hotspots, but I am at a loss now as how to
approach them.


Andy Colson wrote:
> 
> Looked like your row counts (the estimate vs the actual) were way off,
> have you analyzed lately?
> 

Note sure what that means.
Isn't all the maintenance nicely automated through my config?


Andy Colson wrote:
> 
> I could not tell from the explain analyze if an index was used, but I
> notice you have a ton of indexes on events_events table.
> 

Yes, a ton of indexes, but still not the right one :-)


--
View this message in context: 
http://postgresql.1045698.n5.nabble.com/Performance-trouble-finding-records-through-related-records-tp3405914p3407330.html
Sent from the PostgreSQL - performance mailing list archive at Nabble.com.

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] Performance trouble finding records through related records

2011-03-03 Thread sverhagen

Andy Colson wrote:
> 
> For your query, I think a join would be the best bet, can we see its
> explain analyze?
> 


Here is a few variations:


SELECT events_events.* FROM events_events WHERE transactionid IN (
SELECT transactionid FROM events_eventdetails customerDetails
WHERE customerDetails.keyname='customer_id'
AND substring(customerDetails.value,0,32)='1957'
AND transactionid IS NOT NULL
) ORDER BY id LIMIT 50; 

-- http://explain.depesz.com/s/Pnb


explain analyze SELECT events_events.* FROM events_events,
events_eventdetails customerDetails
WHERE events_events.transactionid = customerDetails.transactionid
AND customerDetails.keyname='customer_id'
AND substring(customerDetails.value,0,32)='1957'
AND customerDetails.transactionid IS NOT NULL
ORDER BY id LIMIT 50; 

-- http://explain.depesz.com/s/rDh


explain analyze SELECT events_events.* FROM events_events
JOIN events_eventdetails customerDetails
ON events_events.transactionid = customerDetails.transactionid
AND customerDetails.keyname='customer_id'
AND substring(customerDetails.value,0,32)='1957'
AND customerDetails.transactionid IS NOT NULL
ORDER BY id LIMIT 50; 

-- http://explain.depesz.com/s/6aB


Thanks for your efforts!

--
View this message in context: 
http://postgresql.1045698.n5.nabble.com/Performance-trouble-finding-records-through-related-records-tp3405914p3407689.html
Sent from the PostgreSQL - performance mailing list archive at Nabble.com.

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] Performance trouble finding records through related records

2011-03-08 Thread sverhagen
Hi. Thanks for your response.


Robert Haas wrote:
> 
> Join Filter: ((events_events.transactionid)::text =
> (customerdetails.transactionid)::text)
> 
> Now why is there a cast to text there on both sides?  Do those two
> columns have exactly the same datatype?  If not, you probably want to
> fix that, as it can make a big difference.
> 

Good question. I seem not able to get rid of that, even though these are
same type:

=# \d events_events
Table "public.events_events"
Column|   Type   | Modifiers
--+--+--
[snip]
 transactionid| character varying(36)| not null
[snip]

=# \d events_eventdetails
Table "public.events_eventdetails"
Column |  Type  | Modifiers
---++--
[snip]
 transactionid | character varying(36)  | not null
[snip]

(These columns allowing null or not is just something I've been playing with
to no avail too.)



Robert Haas wrote:
> 
> Also, how many rows are there in events_events and how many in
> events_eventdetails?
> 

select count(*) from events_events; --> 3910163
select count(*) from events_eventdetails; --> 30216033
select count(*) from events_eventdetails_customer_id; (single partition) -->
2976101


--
View this message in context: 
http://postgresql.1045698.n5.nabble.com/Performance-trouble-finding-records-through-related-records-tp3405914p3413801.html
Sent from the PostgreSQL - performance mailing list archive at Nabble.com.

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] Performance trouble finding records through related records

2011-03-08 Thread sverhagen

Merlin Moncure-2 wrote:
> 
> 
> 8.3? try converting the above to WHERE EXISTS or (even better) a JOIN...
> 
> 


Thanks for that. But in my Mar 03, 2011; 10:19am post I already broke it
down to the barebones with some variations, among which JOIN. The EXISTS IN
variation was so poor that I left that one out.

Best regards, Sander.

--
View this message in context: 
http://postgresql.1045698.n5.nabble.com/Performance-trouble-finding-records-through-related-records-tp3405914p3413814.html
Sent from the PostgreSQL - performance mailing list archive at Nabble.com.

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] Performance trouble finding records through related records

2011-03-10 Thread sverhagen
Hi, all. I've done some further analysis, found a form that works if I split
things over two separate queries (B1 and B2, below) but still trouble when
combining (B, below).

This is the full pseudo-query: SELECT FROM A UNION SELECT FROM B ORDER BY
dateTime DESC LIMIT 50
In that pseudo-query:
- A is fast (few ms). A is all events for the given customer
- B is slow (1 minute). B is all events for the same transactions as
all events for the given customer

Zooming in on B it looks originally as follows:

SELECT events2.id, events2.transactionId, events2.dateTime FROM
events_events events2
JOIN events_eventdetails details2_transKey
ON events2.id = details2_transKey.event_id
AND details2_transKey.keyname='transactionId'
JOIN events_eventdetails details2_transValue
ON substring(details2_transKey.value,0,32) =
substring(details2_transValue.value,0,32)
AND details2_transValue.keyname='transactionId'
JOIN events_eventdetails customerDetails
ON details2_transValue.event_id = customerDetails.event_id
AND customerDetails.keyname='customer_id'
AND substring(customerDetails.value,0,32)='598124'
WHERE events2.eventtype_id IN (100,103,105,... et cetera ...) 


The above version of B is tremendously slow.

The only fast version I've yet come to find is as follows:
- Do a sub-query B1
- Do a sub-query B2 with the results of B1

B1 looks as follows:
Works very fast (few ms)
http://explain.depesz.com/s/7JS

SELECT substring(details2_transValue.value,0,32)
FROM events_eventdetails_customer_id customerDetails
JOIN only events_eventdetails details2_transValue
USING (event_id)
WHERE customerDetails.keyname='customer_id'
AND substring(customerDetails.value,0,32)='49'
AND details2_transValue.keyname='transactionId'


B2 looks as follows:
Works very fast (few ms)
http://explain.depesz.com/s/jGO

SELECT events2.id, events2.dateTime
FROM events_events events2
JOIN events_eventdetails details2_transKey
ON events2.id = details2_transKey.event_id
AND details2_transKey.keyname='transactionId'
AND substring(details2_transKey.value,0,32) IN (... results of B1
...)
AND events2.eventtype_id IN
(100,103,105,106,45,34,14,87,58,78,7,76,11,25,57,98,30,35,33,49,52,28,85,59,23,22,51,48,36,65,66,18,13,86,75,44,38,43,94,56,95,96,71,50,81,90,89,16,17,88,79,77,68,97,92,67,72,53,2,10,31,32,80,24,93,26,9,8,61,5,73,70,63,20,60,40,41,39,101,104,107,99,64,62,55,69,19,46,47,15,21,27,54,12,102,108)

The combined version of B works slow again (3-10 seconds):
http://explain.depesz.com/s/9oM

SELECT events2.id, events2.dateTime
FROM events_events events2
JOIN events_eventdetails details2_transKey
ON events2.id = details2_transKey.event_id
AND details2_transKey.keyname='transactionId'
AND substring(details2_transKey.value,0,32) IN (
SELECT substring(details2_transValue.value,0,32)
FROM events_eventdetails_customer_id customerDetails
JOIN only events_eventdetails details2_transValue
USING (event_id)
WHERE customerDetails.keyname='customer_id'
AND substring(customerDetails.value,0,32)='49'
AND details2_transValue.keyname='transactionId')
AND events2.eventtype_id IN
(100,103,105,106,45,34,14,87,58,78,7,76,11,25,57,98,30,35,33,49,52,28,85,59,23,22,51,48,36,65,66,18,13,86,75,44,38,43,94,56,95,96,71,50,81,90,89,16,17,88,79,77,68,97,92,67,72,53,2,10,31,32,80,24,93,26,9,8,61,5,73,70,63,20,60,40,41,39,101,104,107,99,64,62,55,69,19,46,47,15,21,27,54,12,102,108)

At the moment I see not other conclusion than to offer B1 and B2 to the
database separately, but it feels like defeat :-|


--
View this message in context: 
http://postgresql.1045698.n5.nabble.com/Performance-trouble-finding-records-through-related-records-tp3405914p3423334.html
Sent from the PostgreSQL - performance mailing list archive at Nabble.com.

-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance