Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread John A Meinel
Matthew Schumacher wrote: > John A Meinel wrote: > > >>Surely this isn't what you have. You have *no* loop here, and you have >>stuff like: >> AND >>(bayes_token_tmp) NOT IN (SELECT token FROM bayes_token); >> >>I'm guessing this isn't your last version of the function. >> >>As far as putting

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread Matthew Schumacher
John A Meinel wrote: > Surely this isn't what you have. You have *no* loop here, and you have > stuff like: > AND > (bayes_token_tmp) NOT IN (SELECT token FROM bayes_token); > > I'm guessing this isn't your last version of the function. > > As far as putting the CREATE TEMP TABLE inside th

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread John A Meinel
Matthew Schumacher wrote: > Matthew Schumacher wrote: > >>Tom Lane wrote: >> >> >> >>>I don't really see why you think that this path is going to lead to >>>better performance than where you were before. Manipulation of the >>>temp table is never going to be free, and IN (sub-select) is always >>>

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread Matthew Schumacher
Matthew Schumacher wrote: > Tom Lane wrote: > > >>I don't really see why you think that this path is going to lead to >>better performance than where you were before. Manipulation of the >>temp table is never going to be free, and IN (sub-select) is always >>inherently not fast, and NOT IN (sub-

Re: [PERFORM] "nice"/low priority Query

2005-08-04 Thread Neil Conway
Jim C. Nasby wrote: Actually, from what I've read 4.2BSD actually took priority into account when scheduling I/O. FWIW, you can set I/O priority in recent versions of the Linux kernel using ionice, which is part of RML's schedutils package (which was recently merged into util-linux). -Neil

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread Matthew Schumacher
Tom Lane wrote: > I don't really see why you think that this path is going to lead to > better performance than where you were before. Manipulation of the > temp table is never going to be free, and IN (sub-select) is always > inherently not fast, and NOT IN (sub-select) is always inherently > aw

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread Tom Lane
John A Meinel <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> I don't really see why you think that this path is going to lead to >> better performance than where you were before. > So for an IN (sub-select), does it actually pull all of the rows from > the other table, or is the planner smart en

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread Matthew Schumacher
John A Meinel wrote: > Matthew Schumacher wrote: > > I recommend that you drop and re-create the temp table. There is no > reason to have it around, considering you delete and re-add everything. > That means you never have to vacuum it, since it always only contains > the latest rows. Whenever I

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread John A Meinel
Tom Lane wrote: > Matthew Schumacher <[EMAIL PROTECTED]> writes: > >> for i in array_lower(intokenary, 1) .. array_upper(intokenary, 1) >> LOOP >>_token := intokenary[i]; >>INSERT INTO bayes_token_tmp VALUES (_token); >> END LOOP; > > >> UPDATE >>bayes_token >> SET >>spam_count

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread Tom Lane
Matthew Schumacher <[EMAIL PROTECTED]> writes: > for i in array_lower(intokenary, 1) .. array_upper(intokenary, 1) > LOOP > _token := intokenary[i]; > INSERT INTO bayes_token_tmp VALUES (_token); > END LOOP; > UPDATE > bayes_token > SET > spam_count = greatest_int(spam_co

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread John A Meinel
Matthew Schumacher wrote: > Okay, > > Here is the status of the SA updates and a question: > > Michael got SA changed to pass an array of tokens to the proc so right > there we gained a ton of performance due to connections and transactions > being grouped into one per email instead of one per toke

Re: [PERFORM] Indexed views.

2005-08-04 Thread Laszlo Hornyak
prasanna s wrote: Does postgres support indexed views/materialised views that some of the other databases support? Thanks Prasanna S Hi! It is not supported, but perhaps this will help you: http://www.varlena.com/varlena/GeneralBits/Tidbits/matviews.html ---(en

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread PFC
What I really want to do is have the token array available as a record so that I can query against it, but not have it take up the resources of a real table. If I could copy from an array into a record then I can even get rid of the loop. Anyone have any thoughts on how to do this? You coul

Re: [PERFORM] Indexed views.

2005-08-04 Thread Christopher Kings-Lynne
No, unless you use some custom triggers. prasanna s wrote: Does postgres support indexed views/materialised views that some of the other databases support? Thanks Prasanna S ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster

[PERFORM] Indexed views.

2005-08-04 Thread prasanna s
Does postgres support indexed views/materialised views that some of the other databases support? Thanks Prasanna S

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread Matthew Schumacher
Okay, Here is the status of the SA updates and a question: Michael got SA changed to pass an array of tokens to the proc so right there we gained a ton of performance due to connections and transactions being grouped into one per email instead of one per token. Now I am working on making the pro