Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Tom Lane
Russell Smith <[EMAIL PROTECTED]> writes: > Given vacuum must then touch every page, is there a win in only setting > hint bits on pages where vacuum has to do some other work on the page? > As vacuum is causing significant IO load for data that may not be > accessed for some time. Well, if va

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Russell Smith
Simon Riggs wrote: On Thu, 2007-12-13 at 15:19 -0600, Kevin Grittner wrote: What impact would lack of the hint bits have until a vacuum? Vacuum isn't important here. Its the first idiot to read the data that gets hit. Given vacuum must then touch every page, is there a win in only

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
>>> On Thu, Dec 13, 2007 at 3:40 PM, in message <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> wrote: > On Thu, 2007-12-13 at 15:19 -0600, Kevin Grittner wrote: >> >>> On Thu, Dec 13, 2007 at 12:12 PM, in message >> <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> >> wrote: >> > On Thu

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 15:19 -0600, Kevin Grittner wrote: > >>> On Thu, Dec 13, 2007 at 12:12 PM, in message > <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> > wrote: > > On Thu, 2007-12-13 at 11:46 -0600, Kevin Grittner wrote: > >> If the hint bit changes are written to the WAL ... > > > >

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
>>> On Thu, Dec 13, 2007 at 12:12 PM, in message <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> wrote: > On Thu, 2007-12-13 at 11:46 -0600, Kevin Grittner wrote: >> If the hint bit changes are written to the WAL ... > > They're not. So one would expect a write-intensive initial vacuum aft

Re: [PERFORM] Putting files into fields in a table

2007-12-13 Thread Campbell, Lance
Erik, The advantage with storing things in the database verses the file system is the number of lines of code. I manage 18 software applications. I have developed an application that reads in an XML file and will generate database java code for inserting, updating, selecting and deleting data. S

Re: [PERFORM] Putting files into fields in a table

2007-12-13 Thread Erik Jones
On Dec 13, 2007, at 2:09 PM, Campbell, Lance wrote: I did not see much info in the 8.2 documentation on BLOB. I did ready about "bytea" or binary data type. It seems like it would work for storing files. I guess I could stick with the OS for file storage but it is a pain. It would be easier

Re: [PERFORM] Putting files into fields in a table

2007-12-13 Thread Alvaro Herrera
Campbell, Lance wrote: > I did not see much info in the 8.2 documentation on BLOB. That's because we don't call them "blobs". Search for "large objects" instead: http://www.postgresql.org/docs/current/static/largeobjects.html -- Alvaro Herrera http://www.flickr.com/phot

Re: [PERFORM] Putting files into fields in a table

2007-12-13 Thread Campbell, Lance
I did not see much info in the 8.2 documentation on BLOB. I did ready about "bytea" or binary data type. It seems like it would work for storing files. I guess I could stick with the OS for file storage but it is a pain. It would be easier to use the DB. Thanks, Lance Campbell Project Manager

Re: [PERFORM] Putting files into fields in a table

2007-12-13 Thread Alexander Staubo
On 12/13/07, Campbell, Lance <[EMAIL PROTECTED]> wrote: > I am looking at the possibility of storing files in some of my database > tables. My concern is obviously performance. I have configured PostgreSQL > to take advantage of Linux file caching. So my PostgreSQL does not have a > large settin

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 13:52 -0500, Tom Lane wrote: > Alvaro Herrera <[EMAIL PROTECTED]> writes: > > Simon Riggs wrote: > >> We might be able to have bgwriter set hint bits on dirty blocks, > > > I don't think that works, because the bgwriter has no access to the > > catalogs, therefore it cannot e

[PERFORM] Putting files into fields in a table

2007-12-13 Thread Campbell, Lance
PostgreSQL: 8.2 I am looking at the possibility of storing files in some of my database tables. My concern is obviously performance. I have configured PostgreSQL to take advantage of Linux file caching. So my PostgreSQL does not have a large setting for shared_buffers even though I have 24G of

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Simon Riggs wrote: >> We might be able to have bgwriter set hint bits on dirty blocks, > I don't think that works, because the bgwriter has no access to the > catalogs, therefore it cannot examine the page contents. To bgwriter, > pages are opaque. An

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 11:46 -0600, Kevin Grittner wrote: > >>> On Thu, Dec 13, 2007 at 10:11 AM, in message > <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> > wrote: > > On Thu, 2007-12-13 at 09:46 -0600, Kevin Grittner wrote: > > > >> The data was inserted through a Java program using a p

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
>>> On Thu, Dec 13, 2007 at 10:11 AM, in message <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> wrote: > On Thu, 2007-12-13 at 09:46 -0600, Kevin Grittner wrote: > >> The data was inserted through a Java program using a prepared >> statement with no indexes on the table. The primary key w

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Alvaro Herrera
Simon Riggs wrote: > We might be able to have bgwriter set hint bits on dirty blocks, but the > success of that would depend upon the transit time of blocks through the > cache, i.e. it might be totally ineffective. So might be just overhead > for the bgwriter and worse, could divert bgwriter atte

Re: [PERFORM] Limited performance on multi core server

2007-12-13 Thread Joshua D. Drake
Simon Riggs wrote: On Wed, 2007-12-12 at 07:38 -0800, Joshua D. Drake wrote: Matthew Lunnon wrote: Ah I was afraid of that. Maybe I'll have to come out of the dark ages. Yes :) but ignore the comment about the 8.3 Beta series. It is Beta for a reason, that means testing only, no production.

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 10:39 -0600, Kevin Grittner wrote: > >>> On Thu, Dec 13, 2007 at 10:35 AM, in message <[EMAIL PROTECTED]>, > Tom Lane <[EMAIL PROTECTED]> wrote: > > "Kevin Grittner" <[EMAIL PROTECTED]> writes: > >> ... although to a naive user it's not clear what > >> is known at vacuum time

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
>>> On Thu, Dec 13, 2007 at 10:35 AM, in message <[EMAIL PROTECTED]>, Tom Lane <[EMAIL PROTECTED]> wrote: > "Kevin Grittner" <[EMAIL PROTECTED]> writes: >> ... although to a naive user it's not clear what >> is known at vacuum time that the INSERT into the empty table >> couldn't have inferred. >

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Tom Lane
"Kevin Grittner" <[EMAIL PROTECTED]> writes: > ... although to a naive user it's not clear what > is known at vacuum time that the INSERT into the empty table > couldn't have inferred. The fact that the INSERT actually committed. regards, tom lane

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
>>> On Thu, Dec 13, 2007 at 10:11 AM, in message <[EMAIL PROTECTED]>, Simon Riggs <[EMAIL PROTECTED]> wrote: > On Thu, 2007-12-13 at 09:46 -0600, Kevin Grittner wrote: > >> The data was inserted through a Java program using a prepared >> statement with no indexes on the table. The primary key w

Re: [PERFORM] Limited performance on multi core server

2007-12-13 Thread Simon Riggs
On Wed, 2007-12-12 at 07:38 -0800, Joshua D. Drake wrote: > Matthew Lunnon wrote: > > Ah I was afraid of that. Maybe I'll have to come out of the dark ages. > > Yes :) but ignore the comment about the 8.3 Beta series. It is Beta for > a reason, that means testing only, no production. Matthew,

Re: [PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Simon Riggs
On Thu, 2007-12-13 at 09:46 -0600, Kevin Grittner wrote: > The data was inserted through a Java program using a prepared > statement with no indexes on the table. The primary key was then > added, and now I've started a vacuum. The new table wound up being > the first big table vacuumed, and I

[PERFORM] Heavy write activity on first vacuum of fresh TOAST data

2007-12-13 Thread Kevin Grittner
Yesterday we moved a 300 GB table containing document images (mostly raster-scanned from paper), into a 215 GB PostgreSQL 8.2.5 database which contains the related case management data. (This separation was never "right", since there were links from one to the other, but was necessary under our

Re: [PERFORM] Index on VARCHAR with text_pattern_ops inside PL/PGSQL procedure.

2007-12-13 Thread Piotr Gasidło
2007/12/10, Piotr Gasidło <[EMAIL PROTECTED]>: > Why PL/PGSQL is unable to proper utilize > users_user_name_unique_text_pattern_ops? I found solution, that satisfies me (EXECUTE is a bit ugly for me). I've replaced LIKE operator with ~=~ operator. Now PL/PGSQL function properly uses index on SELE