Re: [PERFORM] Index on VARCHAR with text_pattern_ops inside PL/PGSQL procedure.

2007-12-13 Thread Piotr Gasidło
2007/12/10, Piotr Gasidło [EMAIL PROTECTED]: Why PL/PGSQL is unable to proper utilize users_user_name_unique_text_pattern_ops? I found solution, that satisfies me (EXECUTE is a bit ugly for me). I've replaced LIKE operator with ~=~ operator. Now PL/PGSQL function properly uses index on SELECT.

[PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
Yesterday we moved a 300 GB table containing document images (mostly raster-scanned from paper), into a 215 GB PostgreSQL 8.2.5 database which contains the related case management data. (This separation was never right, since there were links from one to the other, but was necessary under our

Re: [PERFORM] Limited performance on multi core server

2007-12-13 Thread Simon Riggs
On Wed, 2007-12-12 at 07:38 -0800, Joshua D. Drake wrote: Matthew Lunnon wrote: Ah I was afraid of that. Maybe I'll have to come out of the dark ages. Yes :) but ignore the comment about the 8.3 Beta series. It is Beta for a reason, that means testing only, no production. Matthew, Some

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
On Thu, Dec 13, 2007 at 10:11 AM, in message [EMAIL PROTECTED], Simon Riggs [EMAIL PROTECTED] wrote: On Thu, 2007-12-13 at 09:46 -0600, Kevin Grittner wrote: The data was inserted through a Java program using a prepared statement with no indexes on the table. The primary key was then

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 09:46 -0600, Kevin Grittner wrote: The data was inserted through a Java program using a prepared statement with no indexes on the table. The primary key was then added, and now I've started a vacuum. The new table wound up being the first big table vacuumed, and I

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
On Thu, Dec 13, 2007 at 10:35 AM, in message [EMAIL PROTECTED], Tom Lane [EMAIL PROTECTED] wrote: Kevin Grittner [EMAIL PROTECTED] writes: ... although to a naive user it's not clear what is known at vacuum time that the INSERT into the empty table couldn't have inferred. The fact that

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Tom Lane
Kevin Grittner [EMAIL PROTECTED] writes: ... although to a naive user it's not clear what is known at vacuum time that the INSERT into the empty table couldn't have inferred. The fact that the INSERT actually committed. regards, tom lane

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Alvaro Herrera
Simon Riggs wrote: We might be able to have bgwriter set hint bits on dirty blocks, but the success of that would depend upon the transit time of blocks through the cache, i.e. it might be totally ineffective. So might be just overhead for the bgwriter and worse, could divert bgwriter

Re: [PERFORM] Limited performance on multi core server

2007-12-13 Thread Joshua D. Drake
Simon Riggs wrote: On Wed, 2007-12-12 at 07:38 -0800, Joshua D. Drake wrote: Matthew Lunnon wrote: Ah I was afraid of that. Maybe I'll have to come out of the dark ages. Yes :) but ignore the comment about the 8.3 Beta series. It is Beta for a reason, that means testing only, no production.

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 10:39 -0600, Kevin Grittner wrote: On Thu, Dec 13, 2007 at 10:35 AM, in message [EMAIL PROTECTED], Tom Lane [EMAIL PROTECTED] wrote: Kevin Grittner [EMAIL PROTECTED] writes: ... although to a naive user it's not clear what is known at vacuum time that the INSERT

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
On Thu, Dec 13, 2007 at 10:11 AM, in message [EMAIL PROTECTED], Simon Riggs [EMAIL PROTECTED] wrote: On Thu, 2007-12-13 at 09:46 -0600, Kevin Grittner wrote: The data was inserted through a Java program using a prepared statement with no indexes on the table. The primary key was then

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 11:46 -0600, Kevin Grittner wrote: On Thu, Dec 13, 2007 at 10:11 AM, in message [EMAIL PROTECTED], Simon Riggs [EMAIL PROTECTED] wrote: On Thu, 2007-12-13 at 09:46 -0600, Kevin Grittner wrote: The data was inserted through a Java program using a prepared

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Simon Riggs wrote: We might be able to have bgwriter set hint bits on dirty blocks, I don't think that works, because the bgwriter has no access to the catalogs, therefore it cannot examine the page contents. To bgwriter, pages are opaque. Another

[PERFORM] Putting files into fields in a table

2007-12-13 Thread Campbell, Lance
PostgreSQL: 8.2 I am looking at the possibility of storing files in some of my database tables. My concern is obviously performance. I have configured PostgreSQL to take advantage of Linux file caching. So my PostgreSQL does not have a large setting for shared_buffers even though I have 24G of

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 13:52 -0500, Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Simon Riggs wrote: We might be able to have bgwriter set hint bits on dirty blocks, I don't think that works, because the bgwriter has no access to the catalogs, therefore it cannot examine the

Re: [PERFORM] Putting files into fields in a table

2007-12-13 Thread Alexander Staubo
On 12/13/07, Campbell, Lance [EMAIL PROTECTED] wrote: I am looking at the possibility of storing files in some of my database tables. My concern is obviously performance. I have configured PostgreSQL to take advantage of Linux file caching. So my PostgreSQL does not have a large setting for

Re: [PERFORM] Putting files into fields in a table

2007-12-13 Thread Campbell, Lance
I did not see much info in the 8.2 documentation on BLOB. I did ready about bytea or binary data type. It seems like it would work for storing files. I guess I could stick with the OS for file storage but it is a pain. It would be easier to use the DB. Thanks, Lance Campbell Project

Re: [PERFORM] Putting files into fields in a table

2007-12-13 Thread Alvaro Herrera
Campbell, Lance wrote: I did not see much info in the 8.2 documentation on BLOB. That's because we don't call them blobs. Search for large objects instead: http://www.postgresql.org/docs/current/static/largeobjects.html -- Alvaro Herrera

Re: [PERFORM] Putting files into fields in a table

2007-12-13 Thread Campbell, Lance
Erik, The advantage with storing things in the database verses the file system is the number of lines of code. I manage 18 software applications. I have developed an application that reads in an XML file and will generate database java code for inserting, updating, selecting and deleting data.

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
On Thu, Dec 13, 2007 at 12:12 PM, in message [EMAIL PROTECTED], Simon Riggs [EMAIL PROTECTED] wrote: On Thu, 2007-12-13 at 11:46 -0600, Kevin Grittner wrote: If the hint bit changes are written to the WAL ... They're not. So one would expect a write-intensive initial vacuum after a

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 15:19 -0600, Kevin Grittner wrote: On Thu, Dec 13, 2007 at 12:12 PM, in message [EMAIL PROTECTED], Simon Riggs [EMAIL PROTECTED] wrote: On Thu, 2007-12-13 at 11:46 -0600, Kevin Grittner wrote: If the hint bit changes are written to the WAL ... They're not.

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Russell Smith
Simon Riggs wrote: On Thu, 2007-12-13 at 15:19 -0600, Kevin Grittner wrote: What impact would lack of the hint bits have until a vacuum? Vacuum isn't important here. Its the first idiot to read the data that gets hit. Given vacuum must then touch every page, is there a win in only

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Tom Lane
Russell Smith [EMAIL PROTECTED] writes: Given vacuum must then touch every page, is there a win in only setting hint bits on pages where vacuum has to do some other work on the page? As vacuum is causing significant IO load for data that may not be accessed for some time. Well, if vacuum