Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread Matthew Schumacher
Okay, Here is the status of the SA updates and a question: Michael got SA changed to pass an array of tokens to the proc so right there we gained a ton of performance due to connections and transactions being grouped into one per email instead of one per token. Now I am working on making the

[PERFORM] Indexed views.

2005-08-04 Thread prasanna s
Does postgres support indexed views/materialised views that some of the other databases support? Thanks Prasanna S

Re: [PERFORM] Indexed views.

2005-08-04 Thread Christopher Kings-Lynne
No, unless you use some custom triggers. prasanna s wrote: Does postgres support indexed views/materialised views that some of the other databases support? Thanks Prasanna S ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread PFC
What I really want to do is have the token array available as a record so that I can query against it, but not have it take up the resources of a real table. If I could copy from an array into a record then I can even get rid of the loop. Anyone have any thoughts on how to do this? You

Re: [PERFORM] Indexed views.

2005-08-04 Thread Laszlo Hornyak
prasanna s wrote: Does postgres support indexed views/materialised views that some of the other databases support? Thanks Prasanna S Hi! It is not supported, but perhaps this will help you: http://www.varlena.com/varlena/GeneralBits/Tidbits/matviews.html

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread John A Meinel
Matthew Schumacher wrote: Okay, Here is the status of the SA updates and a question: Michael got SA changed to pass an array of tokens to the proc so right there we gained a ton of performance due to connections and transactions being grouped into one per email instead of one per token.

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread Tom Lane
Matthew Schumacher [EMAIL PROTECTED] writes: for i in array_lower(intokenary, 1) .. array_upper(intokenary, 1) LOOP _token := intokenary[i]; INSERT INTO bayes_token_tmp VALUES (_token); END LOOP; UPDATE bayes_token SET spam_count = greatest_int(spam_count +

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread John A Meinel
Tom Lane wrote: Matthew Schumacher [EMAIL PROTECTED] writes: for i in array_lower(intokenary, 1) .. array_upper(intokenary, 1) LOOP _token := intokenary[i]; INSERT INTO bayes_token_tmp VALUES (_token); END LOOP; UPDATE bayes_token SET spam_count =

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread Matthew Schumacher
John A Meinel wrote: Matthew Schumacher wrote: I recommend that you drop and re-create the temp table. There is no reason to have it around, considering you delete and re-add everything. That means you never have to vacuum it, since it always only contains the latest rows. Whenever I have

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread Tom Lane
John A Meinel [EMAIL PROTECTED] writes: Tom Lane wrote: I don't really see why you think that this path is going to lead to better performance than where you were before. So for an IN (sub-select), does it actually pull all of the rows from the other table, or is the planner smart enough to

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread Matthew Schumacher
Tom Lane wrote: I don't really see why you think that this path is going to lead to better performance than where you were before. Manipulation of the temp table is never going to be free, and IN (sub-select) is always inherently not fast, and NOT IN (sub-select) is always inherently awful.

Re: [PERFORM] nice/low priority Query

2005-08-04 Thread Neil Conway
Jim C. Nasby wrote: Actually, from what I've read 4.2BSD actually took priority into account when scheduling I/O. FWIW, you can set I/O priority in recent versions of the Linux kernel using ionice, which is part of RML's schedutils package (which was recently merged into util-linux). -Neil

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread Matthew Schumacher
Matthew Schumacher wrote: Tom Lane wrote: I don't really see why you think that this path is going to lead to better performance than where you were before. Manipulation of the temp table is never going to be free, and IN (sub-select) is always inherently not fast, and NOT IN (sub-select) is

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread John A Meinel
Matthew Schumacher wrote: Matthew Schumacher wrote: Tom Lane wrote: I don't really see why you think that this path is going to lead to better performance than where you were before. Manipulation of the temp table is never going to be free, and IN (sub-select) is always inherently not fast,

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread Matthew Schumacher
John A Meinel wrote: Surely this isn't what you have. You have *no* loop here, and you have stuff like: AND (bayes_token_tmp) NOT IN (SELECT token FROM bayes_token); I'm guessing this isn't your last version of the function. As far as putting the CREATE TEMP TABLE inside the

Re: [PERFORM] Performance problems testing with Spamassassin 3.1.0

2005-08-04 Thread John A Meinel
Matthew Schumacher wrote: John A Meinel wrote: Surely this isn't what you have. You have *no* loop here, and you have stuff like: AND (bayes_token_tmp) NOT IN (SELECT token FROM bayes_token); I'm guessing this isn't your last version of the function. As far as putting the CREATE TEMP