Re: [PERFORM] how to fix problem then when two queries run at the same time, it takes longer to complete then if run in sequence

2008-07-31 Thread Theo Kramer


On 31 Jul 2008, at 10:29AM, Miernik wrote:


Richard Huxton [EMAIL PROTECTED] wrote:
I just installed pgpool2 and whoaaa! Everything its like about 3  
times

faster! My application are bash scripts using psql -c UPDATE 


Probably spending most of their time setting up a new connection,  
then

clearing it down again.


If I do it in Python it could do all queries in the same connection,  
so

should be faster? Besides that 'psql' is written in perl, so its also
heavy, by not using psql I get rid of perl library in RAM. Also the
script uses wget to poll some external data sources a lot, also
needlessly opening new connection to the webserver, so I want to make
the script save the http connection, which means I must get rid of  
wget.

Maybe I should write some parts in C?

BTW, doesn't there exist any tool does what psql -c does, but is
written in plain C, not perl? I was looking for such psql replacement,
but couldn't find any.



?

file `which psql`
/usr/bin/psql: ELF 32-bit LSB executable, Intel 80386, version 1  
(SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.9,  
stripped


--
Regards
Theo


--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] how to fix problem then when two queries run at the same time, it takes longer to complete then if run in sequence

2008-07-31 Thread Theo Kramer


On 31 Jul 2008, at 11:17AM, Miernik wrote:


Theo Kramer [EMAIL PROTECTED] wrote:

file `which psql`
/usr/bin/psql: ELF 32-bit LSB executable, Intel 80386, version 1
(SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.9,
stripped


[EMAIL PROTECTED]:~$ file `which psql`
/usr/bin/psql: symbolic link to `../share/postgresql-common/ 
pg_wrapper'

[EMAIL PROTECTED]:~$ file /usr/share/postgresql-common/pg_wrapper
/usr/share/postgresql-common/pg_wrapper: a /usr/bin/perl -w script  
text executable

[EMAIL PROTECTED]:~$



Hmm - looks like you are on a debian or debian derivative. However,  
pg_wrapper is
not psql. It invokes psql which is written in C. Once psql is invoked  
pg_wrapper drops away.

--
Regards
Theo


--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] mysterious difference in speed when combining two queries with OR

2008-04-23 Thread Theo Kramer


On 23 Apr 2008, at 9:23AM, Hans Ekbrand wrote:

I cannot understand why the following two queries differ so much in  
execution time (almost ten times)


Query A (two queries)

select distinct moment.mid from moment,timecard where parent = 45  
and (pid=17 and timecard.mid = moment.mid) order by moment.mid;
select distinct moment.mid from moment,timecard where parent = 45  
and (pbar = 0) order by moment.mid;


Query B (combining the two with OR)

select distinct moment.mid from moment,timecard where parent = 45  
and ((pid=17 and timecard.mid = moment.mid) or (pbar = 0)) order by  
moment.mid;


$ time psql -o /dev/null -f query-a.sql fektest

real0m2.016s
user0m1.532s
sys 0m0.140s

$ time psql -o /dev/null -f query-b.sql fektest

real0m28.534s
user0m1.516s
sys 0m0.156s

I have tested this in two different computers with different amount of
RAM, fast or slow CPU, and the difference is persistent, almost ten
times.

I should say that this is on postgresql 7.4.16 (debian stable).

Can query B be rewritten so that it would execute faster?


Try
select distinct moment.mid from moment,timecard where parent = 45 and  
(pid=17 and timecard.mid = moment.mid) order by moment.mid

union all
select distinct moment.mid from moment,timecard where parent = 45 and  
(pbar = 0) order by moment.mid;

--
Regards
Theo


--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance


Re: [PERFORM] oid...any optimizations

2008-03-05 Thread Theo Kramer
On Thu, 2008-03-06 at 12:32 +0530, sathiya psql wrote:
 i had a table with 50 lakh record...
 
 it has a column called oid ( obviously all the tables will have
 this ), but while doing any operation it is getting slow because of
 the number of records...
 
 if i remove the oid column will i get any benefit, what are all the
 other default columns created without our knowledge..

Probably not

Also - do not remove oid if your sql operations require it. 

A 'create index x_oid_idx on table x (oid)' might help. 

Also see EXPLAIN in the manual.

-- 
Regards
Theo


--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://mail.postgresql.org/mj/mj_wwwusr?domain=postgresql.orgextra=pgsql-performance


Re: [PERFORM] Optimizer : query rewrite and execution plan ?

2008-02-06 Thread Theo Kramer
On Wed, 2008-02-06 at 11:53 +, Simon Riggs wrote:
 On Wed, 2008-02-06 at 09:42 +0100, SURANTYN Jean François wrote:
 
  That issue is very annoying because with generated SQL queries (from
  Business Objects for example) on big tables, it is possible that some
  queries have several times the same where condition (where n=1 and
  n=1 for example), and as the optimizer is under-estimating the number
  of returned rows, some bad execution plans can be chosen (nested loops
  instead of hash joins for example)
 
 I can see the annoyance there.
 
 There's a balance in the planner between time spent to optimize the
 query and time spent to correct mistakes. If we looked continually for
 mistakes then planning time would increase for everybody that didn't
 suffer from this problem.
 
 Since the SQL is not your fault and difficult to control, it is an
 argument in favour of an optional planner mode that would perform
 additional checks for redundant clauses of various kinds. The default
 for that would be off since most people don't suffer from this
 problem. BO isn't the only SQL generating-client out there, so I think
 this is a fairly wide problem.

I would have to disagree. I spend a lot of time writing code that
generates SQL from a business app and feel strongly that any
optimisation is my responsibility.

Having to re-configure PG to switch on a planner mode, as suggested
above, to address badly generated SQL is not a good idea.

This with experience on having to talk business application developers
through re-configuring a database.

-- 
Regards
Theo


---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [PERFORM] Performance problems with prepared statements

2007-10-12 Thread Theo Kramer
On Thu, 2007-10-11 at 16:04 -0400, Merlin Moncure wrote:
 On 10/11/07, Andrew - Supernews [EMAIL PROTECTED] wrote:
  On 2007-10-10, Theo Kramer [EMAIL PROTECTED] wrote:
   When doing a 'manual' prepare and explain analyze I get the following
  
   rascal=# prepare cq (char(12), smallint, integer) as SELECT oid,
   calllog_mainteng, calllog_phase, calllog_self FROM calllog
   WHERE calllog_mainteng = $1
   AND calllog_phase = $2
   AND calllog_self  $3
   OR calllog_mainteng = $1
   AND calllog_phase  $2
   ORDER BY calllog_mainteng DESC,
calllog_phase DESC,
calllog_self DESC limit 25;
   PREPARE
 
  When you do this from the application, are you passing it 3 parameters,
  or 5?  The plan is clearly taking advantage of the fact that the two
  occurrences of $1 and $2 are known to be the same value; if your app is
  using some interface that uses ? placeholders rather than numbered
  parameters, then the planner will not be able to make this assumption.
 
  Also, from the application, is the LIMIT 25 passed as a constant or is that
  also a parameter?
 
 also, this looks a bit like a drilldown query, which is ordering the
 table on 2+ fields.  if that's the case, row wise comparison is a
 better and faster approach.

Agreed - and having a look into that.

   is this a converted cobol app?

:) - on the right track - it is a conversion from an isam based package
where I have changed the backed to PostgreSQL. Unfortunately there is
way too much legacy design and application code to change things at a
higher level.

-- 
Regards
Theo


---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [PERFORM] Performance problems with prepared statements

2007-10-12 Thread Theo Kramer
On Thu, 2007-10-11 at 13:28 -0400, Merlin Moncure wrote:
 On 10/11/07, Theo Kramer [EMAIL PROTECTED] wrote:
  On Thu, 2007-10-11 at 10:12 +0100, Richard Huxton wrote:
   Theo Kramer wrote:
   
So I suspect that there is something more fundamental here...
  
   OK, so there must be something different between the two scenarios. It
   can only be one of:
  1. Query
   2. DB Environment (user, locale, settings)
  3. Network environment (server/client/network activity etc)
 
  I suspect that it could also be in the way the libpq PQprepare(), and
  PQexecPrepared() are handled... as opposed to the way PREPARE and
  EXECUTE are handled.
 
 PQexecPrepared is generally the fastest way to run queries from a C
 app as long as you get the right plan.   Some suggestions
 
 * you can explain/explain analyze executing prepared statements from
 psql shell...try that and see if you can reproduce results

Did that - see previous emails in this thread.

 * at worst case you can drop to execParams which is faster (and
 better) than PQexec, at least

Thanks - will keep that option open.

 * if problem is plan related, you can always disable certain plan
 types (seqscan), prepare, and re-enable those plan types
 * do as Jonah suggested, first step is to try and reproduce problem from psql

No success on that.
-- 
Regards
Theo


---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [PERFORM] Performance problems with prepared statements

2007-10-12 Thread Theo Kramer
On Thu, 2007-10-11 at 18:28 +, Andrew - Supernews wrote:
 On 2007-10-10, Theo Kramer [EMAIL PROTECTED] wrote:
  When doing a 'manual' prepare and explain analyze I get the following
 
  rascal=# prepare cq (char(12), smallint, integer) as SELECT oid,
  calllog_mainteng, calllog_phase, calllog_self FROM calllog
  WHERE calllog_mainteng = $1
  AND calllog_phase = $2
  AND calllog_self  $3
  OR calllog_mainteng = $1
  AND calllog_phase  $2
  ORDER BY calllog_mainteng DESC,
   calllog_phase DESC,
   calllog_self DESC limit 25;
  PREPARE
 
 When you do this from the application, are you passing it 3 parameters,
 or 5?  The plan is clearly taking advantage of the fact that the two
 occurrences of $1 and $2 are known to be the same value; if your app is
 using some interface that uses ? placeholders rather than numbered
 parameters, then the planner will not be able to make this assumption.

You may just have hit the nail on the head. I use numbered parameters
but have $1 to $5 ... let me take a look to see if I can change this.

 Also, from the application, is the LIMIT 25 passed as a constant or is that
 also a parameter?

A constant.

 
-- 
Regards
Theo


---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [PERFORM] Performance problems with prepared statements

2007-10-12 Thread Theo Kramer
On Fri, 2007-10-12 at 09:02 -0400, Merlin Moncure wrote:
 fwiw, I converted a pretty large cobol app (acucobol) to postgresql
 backend translating queries on the fly.  if this is a fresh effort,
 you definately want to use the row-wise comparison feature of 8.2.
 not only is it much simpler, it's much faster.  with some clever
 caching strategies i was able to get postgresql performance to exceed
 the isam backend.  btw, I used execprepared for virtually the entire
 system.
 
 example read next:
 select * from foo where (a,b,c)  (a1,b1,c1) order by a,b,c limit 25;
 
 example read previous:
 select * from foo where (a,b,c)  (a1,b1,c1) order by a desc, b desc,
 c desc limit 25;
 
 etc.  this will use complete index for a,b,c and is much cleaner to
 prepare, and parse for the planner (the best you can get with standard
 tactics is to get backend to use index on a).
 
 Another big tip i can give you (also 8.2) is to check into advisory
 locks for isam style pessimistic locking.  With some thin wrappers you
 can generate full row and table locking which is quite powerful.

Very interesting - I have largely done the same thing, creating tables
on the fly, translating isam calls, and creating, preparing and
executing queries on the fly using the libpq PQprepare() and
PQexecPrepared() statements... and it is running rather well at several
sites, however, the initial port I did was for 8.0 and 8.1 so could not,
at the time use, row  level comparison, although I do have it on the
latest version of my code working on 8.2 which is not yet released.

The problem I have on row level comparison is that we have orders that
are mixed, ie. a mixture of ascending and descending orders and do not
know if it is possible to use row level comparison on that... eg. I
haven't been able to transform the following it a row comparison query.

select * from foo where
  (a = a1 and b = b1 and c = c1) or
  (a = a1 and b  b1) or
  (a  a1)
order by a, b desc, c;

I have, however, found that transforming the above into a union based
query performs substantially better.

Also indexes containing mixed order columns will only be available on
8.3...

But many thanks for your advice.

-- 
Regards
Theo


---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [PERFORM] Performance problems with prepared statements

2007-10-11 Thread Theo Kramer
On Thu, 2007-10-11 at 10:12 +0100, Richard Huxton wrote: 
 Theo Kramer wrote:
  
  So I suspect that there is something more fundamental here...
 
 OK, so there must be something different between the two scenarios. It 
 can only be one of:
1. Query
 2. DB Environment (user, locale, settings)
3. Network environment (server/client/network activity etc)

I suspect that it could also be in the way the libpq PQprepare(), and
PQexecPrepared() are handled... as opposed to the way PREPARE and
EXECUTE are handled.

 
 Are you sure you have the parameter types correct in your long-running 
 query?

Yes - the problem surfaced during a going live session on an 80 user
system... and we had to roll back to the previous system in a hurry.
This was a part of the application that had missed testing, but I have
had other reports from some of my other systems where this appears to be
a problem but not of the magnitude that this one is.

In any case I have managed to reproduce it in my test environment with
configuration settings the same.

 Try setting log_min_duration_statement=9000 or so to capture 
 long-running queries.

Thanks - will give that a try.

 
 Make sure the user and any custom settings are the same. Compare SHOW 
 ALL for both ways.

 You've said elsewhere you've ruled out the network environment, so 
 there's not point worrying about that further.

It is definitely not a network problem - ie. the postgresql server load
goes way up when this query is run.

-- 
Regards
Theo


---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


[PERFORM] Performance problems with prepared statements

2007-10-10 Thread Theo Kramer
Hi

I have been having some serious performance issues when using prepared
statements which I can not re-produce when using a direct statement. Let
me try to explain

The query does an order by in descending order on several columns for
which an index exists. 

The explain output as follows

rascal=# explain SELECT oid, * FROM calllog
WHERE calllog_mainteng = '124 '
AND calllog_phase = 8
AND calllog_self  366942
OR calllog_mainteng = '124 '
AND calllog_phase  8
ORDER BY calllog_mainteng DESC,
 calllog_phase DESC,
 calllog_self DESC limit 25;
   QUERY PLAN
-
 Limit  (cost=0.00..111.62 rows=25 width=2164)
   -  Index Scan Backward using calllog_rmc_idx on calllog
(cost=0.00..53475.22 rows=11977 width=2164)
 Index Cond: (calllog_mainteng = '124 '::bpchar)
 Filter: (((calllog_phase = 8) AND (calllog_self  366942)) OR
(calllog_phase  8))

When running the query directly from psql it returns the required rows
in less than 100 milli-seconds.

However, when using a prepared statement from my C application on the
above query and executing it the query duration is as follows

SELECT oid, * FROM calllog
WHERE calllog_mainteng = '124 '
AND calllog_phase = 8
AND calllog_self  366942
OR calllog_mainteng = '124 '
AND calllog_phase  8
ORDER BY calllog_mainteng DESC,
 calllog_phase DESC,
 calllog_self DESC limit 25
Row[s] = 25, Duration = 435409.474 ms

The index as per the explain is defined as follows

calllog_rmc_idx UNIQUE, btree (calllog_mainteng, calllog_phase,
calllog_self)

VACUUM and all those good things done

Version of PostgreSQL 8.1 and 8.2

enable_seqscan = off
enable_sort = off

Any advice/suggestions/thoughts much appreciated

-- 
Regards
Theo


---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [PERFORM] Performance problems with prepared statements

2007-10-10 Thread Theo Kramer
On Wed, 2007-10-10 at 15:55 -0400, Jonah H. Harris wrote:
 On 10/10/07, Theo Kramer [EMAIL PROTECTED] wrote:
  When running the query directly from psql it returns the required rows
  in less than 100 milli-seconds.
 
  However, when using a prepared statement from my C application on the
  above query and executing it the query duration is as follows
  ...
  Row[s] = 25, Duration = 435409.474 ms
 
 
 How are you timing it?  Does it really take 435 seconds to complete?

Fraid so - and I am running postgresql on a separate machine from the
client machine - with the load going way up on the postgresql machine
and the client machine remains idle until the query returns.

Also the postgresql has only the one prepared statement executing during
my tests.

 Try the following in psql:

Did that - see my previous email.
-- 
Regards
Theo


---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [PERFORM] Multicolumn order by

2006-04-19 Thread Theo Kramer
On Wed, 2006-04-19 at 01:08, Tom Lane wrote:
 Theo Kramer [EMAIL PROTECTED] writes:
  select * from mytable where
(c1 = 'c1v' and c2 = 'c2v' and c3 = 'c3v') or
(c1 = 'c1v' and c2  'c2v') or
(c1  'c1v')
order by c1, c2, c3;
 
 Yeah ... what you really want is the SQL-spec row comparison operator
 
 select ... where (c1,c2,c3) = ('c1v','c2v','c3v') order by c1,c2,c3;
 
 This does not work properly in any current PG release :-( but it does
 work and is optimized well in CVS HEAD.  See eg this thread
 http://archives.postgresql.org/pgsql-hackers/2006-02/msg00209.php

That is awesome - been fighting with porting my isam based stuff onto
sql for a long time and the row comparison operator is exactly what I
have been looking for.

I tried this on my test system running 8.1.3 and appears to work fine.
Appreciate it if you could let me know in what cases it does not work
properly.

-- 
Regards
Theo


---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [PERFORM] Multicolumn order by

2006-04-19 Thread Theo Kramer
On Wed, 2006-04-19 at 08:00, Theo Kramer wrote:

 I tried this on my test system running 8.1.3 and appears to work fine.
 Appreciate it if you could let me know in what cases it does not work
 properly.

Please ignore - 'Explain is your friend' - got to look at the tips :)
-- 
Regards
Theo


---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


[PERFORM] Multicolumn order by

2006-04-18 Thread Theo Kramer
Hi

Apologies if this has already been raised...

PostgreSQL 8.1.3 and prior versions. Vacuum done.

Assuming a single table with columns named c1 to cn and a requirement to
select from a particular position in multiple column order. 

The column values in my simple example below denoted by 'cnv' a typical
query would look as follows

select * from mytable where
  (c1 = 'c1v' and c2 = 'c2v' and c3 = 'c3v') or
  (c1 = 'c1v' and c2  'c2v') or
  (c1  'c1v')
  order by c1, c2, c3;

In real life with the table containing many rows (9 Million) and
a single multicolumn index on the required columns existing I get the
following

explain analyse
 SELECT
 tran_subledger,
 tran_subaccount,
 tran_mtch,
 tran_self,
 tran_Rflg FROM tran
WHERE ((tran_subledger = 2 AND tran_subaccount = 'ARM '
AND tran_mtch = 0 AND tran_self = 0 )
OR (tran_subledger = 2 AND tran_subaccount = 'ARM ' AND
tran_mtch  0 )
OR (tran_subledger = 2 AND tran_subaccount  'ARM ' )
OR (tran_subledger  2 ))
ORDER BY tran_subledger,
 tran_subaccount,
 tran_mtch,
 tran_self
limit 10;

 Limit  (cost=0.00..25.21 rows=10 width=36) (actual
time=2390271.832..2390290.305 rows=10 loops=1)
   -  Index Scan using tran_mtc_idx on tran  (cost=0.00..13777295.04
rows=5465198 width=36) (actual time=2390271.823..2390290.252 rows=10
loops=1)
 Filter: (((tran_subledger = 2) AND (tran_subaccount = 'ARM
'::bpchar) AND (tran_mtch = 0) AND (tran_self = 0)) OR ((tran_subledger
= 2) AND (tran_subaccount = 'ARM '::bpchar) AND
(tran_mtch  0)) OR ((tran_subledger = 2) AND (tran_subaccount 
'ARM '::bpchar)) OR (tran_subledger  2))
 Total runtime: 2390290.417 ms

Any suggestions/comments/ideas appreciated.
-- 
Regards
Theo


---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [PERFORM] Indexes with descending date columns

2006-03-29 Thread Theo Kramer
On Fri, 2006-03-24 at 12:21, Jim C. Nasby wrote:
 On Thu, Mar 23, 2006 at 01:09:49PM +0200, Theo Kramer wrote:
  ii If no to i, is it feasible to extend PostgreSQL to allow traversing
 an index in column descending and column ascending order - assuming
 an order by on more than one column with column order not 
 in the same direction and indexes existing? ... if that makes sense.
 
 Yes.
 
 stats=# explain select * from email_contrib order by project_id desc, id 
 desc, date desc limit 10;
QUERY PLAN 
   
 
  Limit  (cost=0.00..31.76 rows=10 width=24)
-  Index Scan Backward using email_contrib_pkey on email_contrib  
 (cost=0.00..427716532.18 rows=134656656 width=24)
 (2 rows)

Not quite what I mean - redo the above as follows and then see what
explain returns

explain select * from email_contrib order by project_id, id, date desc
limit 10;

-- 
Regards
Theo


---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [PERFORM] Indexes with descending date columns

2006-03-23 Thread Theo Kramer
On Fri, 2006-03-17 at 08:25, [EMAIL PROTECTED] wrote:
  I have a performance problem when traversing a table in index order with
  multiple columns including a date column in date reverse order. Below
  follows a simplified description of the table, the index and the
  associated query
  
  \d prcdedit
   prcdedit_prcd   | character(20)   |
   prcdedit_date   | timestamp without time zone |
  
  Indexes:
  prcdedit_idx btree (prcdedit_prcd, prcdedit_date)
 
 Depending on how you use the table, there are three possible solutions.
 
 First, if it makes sense in the domain, using an ORDER BY where _both_ 
 columns are used descending will make PG search the index in reverse and will 
 be just as fast as when both as searched by the default ascending.
 
 Second possibility: Create a dummy column whose value depends on the negative 
 of prcdedit_date, e.g., -extract(epoch from prcdedit_date), keep the dummy 
 column in sync with the original column using triggers, and rewrite your 
 queries to use ORDER BY prcdedit_prod, dummy_column.
 
 Third: Create an index on a function which sorts in the order you want, and 
 then always sort using the function index (you could use the 
 -extract(epoch...) gimmick for that, among other possibilities.)
 
 HTH.

All good input - thanks, however, before I start messing with my stuff
which I know will be complex - some questions to any of the developers
on the list.

i  Is it feasible to extend index creation to support descending 
   columns? ... this is supported on other commercial and non
   commercial databases, but I do not know if this is a SQL standard.

ii If no to i, is it feasible to extend PostgreSQL to allow traversing
   an index in column descending and column ascending order - assuming
   an order by on more than one column with column order not 
   in the same direction and indexes existing? ... if that makes sense.

-- 
Regards
Theo


---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [PERFORM] Indexes with descending date columns

2006-03-23 Thread Theo Kramer
On Thu, 2006-03-23 at 16:16, Alvaro Herrera wrote:
 Theo Kramer wrote:
 
  All good input - thanks, however, before I start messing with my stuff
  which I know will be complex - some questions to any of the developers
  on the list.
  
  i  Is it feasible to extend index creation to support descending 
 columns? ... this is supported on other commercial and non
 commercial databases, but I do not know if this is a SQL standard.
 
 This can be done.  You need to create an operator class which specifies
 the reverse sort order (i.e. reverse the operators), and then use it in
 the new index.

Hmmm, would that then result in the following syntax  being valid?

  create index my_idx on my_table (c1, c2 desc, c3, c4 desc) ;

where my_table is defined as

  create table my_table (
c1 text,
c2 timestamp,
c3 integer,
c4 integer
  );

If so, I would appreciate any pointers on where to start on this -
already fumbling my way through Interfacing Extensions To Indexes in the
manual...

Regards
Theo
-- 
Regards
Theo


---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


[PERFORM] Indexes with descending date columns

2006-03-16 Thread Theo Kramer
Hi

I have a performance problem when traversing a table in index order with
multiple columns including a date column in date reverse order. Below
follows a simplified description of the table, the index and the
associated query

\d prcdedit
 prcdedit_prcd   | character(20)   |
 prcdedit_date   | timestamp without time zone |

Indexes:
prcdedit_idx btree (prcdedit_prcd, prcdedit_date)

When invoking a query such as 

select oid, prcdedit_prcd, prcdedit_date, 'dd/mm/ hh24:mi:ss') as
mydate where prcdedit_prcd  'somevalue' order by prcdedit_prcd,
prcdedit_date desc;

the peformance is dismal.

However removing the 'desc' qualifier as follows the query flys

select oid, prcdedit_prcd, prcdedit_date, 'dd/mm/ hh24:mi:ss') as
mydate where prcdedit_prcd  'somevalue' order by prcdedit_prcd,
prcdedit_date;

PostgreSQL Version = 8.1.2

Row count on the table is  30

Explain is as follows for desc
 Limit  (cost=81486.35..81486.41 rows=25 width=230) (actual
time=116619.652..116619.861 rows=25 loops=1)
   -  Sort  (cost=81486.35..82411.34 rows=369997 width=230) (actual
time=116619.646..116619.729 rows=25 loops=1)
 Sort Key: prcdedit_prcd, prcdedit_date, oid
 -  Bitmap Heap Scan on prcdedit  (cost=4645.99..23454.94
rows=369997 width=230) (actual time=376.952..11798.834 rows=369630
loops=1)
   Recheck Cond: (prcdedit_prcd  '063266 
'::bpchar)
   -  Bitmap Index Scan on prcdedit_idx 
(cost=0.00..4645.99 rows=369997 width=0) (actual time=366.048..366.048
rows=369630 loops=1)
 Index Cond: (prcdedit_prcd  '063266 
'::bpchar)
 Total runtime: 116950.175 ms

and as follows when I remove the 'desc'

 Limit  (cost=0.00..2.34 rows=25 width=230) (actual time=0.082..0.535
rows=25 loops=1)
   -  Index Scan using prcdedit_idx on prcdedit  (cost=0.00..34664.63
rows=369997 width=230) (actual time=0.075..0.405 rows=25 loops=1)
 Index Cond: (prcdedit_prcd  '063266  '::bpchar)
 Total runtime: 0.664 ms


Any assistance/advice much appreciated.

-- 
Regards
Theo


---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings