Re: [PERFORM] Why is vacuum_freeze_min_age 100m?

2009-08-13 Thread Robert Haas
On Thu, Aug 13, 2009 at 5:15 PM, Josh Berkus wrote: > Robert, > >> Ah.  Yeah, I agree with Tom: how would it help to make this smaller? >> It seems like that could possibly increase I/O, if the old data is >> changing at all, but even if it doesn't it I don't see that it saves >> you anything to fr

Re: [PERFORM] Under the hood of views

2009-08-13 Thread Greg Stark
On Fri, Aug 14, 2009 at 12:04 AM, David Kerr wrote: > On Thu, Aug 13, 2009 at 05:28:01PM +0100, Richard Huxton wrote: > - David Kerr wrote: > - > > - >create view test as > - >select a,b,c,d,e,f,g from testtable; > - > > - >select a from test; > - > > - >(does the engine retrieve b-g?) > - > - Shou

Re: [PERFORM] Under the hood of views

2009-08-13 Thread David Kerr
On Thu, Aug 13, 2009 at 05:28:01PM +0100, Richard Huxton wrote: - David Kerr wrote: - > - >create view test as - >select a,b,c,d,e,f,g from testtable; - > - >select a from test; - > - >(does the engine retrieve b-g?) - - Shouldn't - the query just gets rewritten macro-style. I don't think it - el

Re: [PERFORM] Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Josh Berkus
>> Why aren't we more opportunistic about freezing tuples? For instance, if >> we already have a dirty buffer in cache, we should be more aggressive >> about freezing those tuples than freezing tuples on disk. > > The most widely cited reason is that you lose forensics data. Although > they are

[PERFORM] Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Robert Haas
On Thu, Aug 13, 2009 at 5:33 PM, Jeff Davis wrote: > Or, perhaps when the bgwriter is flushing dirty buffers, it can look for > opportunities to set hint bits or freeze tuples. One of the tricky things here is that the time you are mostly likely to want to do this is when you are loading a lot of

[PERFORM] Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age100m? )

2009-08-13 Thread Kevin Grittner
Alvaro Herrera wrote: > Jeff Davis wrote: > >> Why aren't we more opportunistic about freezing tuples? For >> instance, if we already have a dirty buffer in cache, we should be >> more aggressive about freezing those tuples than freezing tuples on >> disk. > > The most widely cited reason is th

[PERFORM] Re: [HACKERS] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Alvaro Herrera
Jeff Davis wrote: > Why aren't we more opportunistic about freezing tuples? For instance, if > we already have a dirty buffer in cache, we should be more aggressive > about freezing those tuples than freezing tuples on disk. The most widely cited reason is that you lose forensics data. Although

Re: [PERFORM] Memory usage of writer process

2009-08-13 Thread Scott Carey
On 8/12/09 9:44 PM, "Alex" wrote: > The writer process seems to be using inordinate amounts of memory: > > PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ > COMMAND > 11088 postgres 13 -2 3217m 2.9g 2.9g S0 38.7 0:10.46 postgres: > writer process > 20190 postgres 13 -2 3

[PERFORM] freezing tuples ( was: Why is vacuum_freeze_min_age 100m? )

2009-08-13 Thread Jeff Davis
[ moving to -hackers ] If this topic has been discussed previously, please point me to the earlier threads. Why aren't we more opportunistic about freezing tuples? For instance, if we already have a dirty buffer in cache, we should be more aggressive about freezing those tuples than freezing tupl

Re: [PERFORM] Why is vacuum_freeze_min_age 100m?

2009-08-13 Thread Josh Berkus
Robert, > Ah. Yeah, I agree with Tom: how would it help to make this smaller? > It seems like that could possibly increase I/O, if the old data is > changing at all, but even if it doesn't it I don't see that it saves > you anything to freeze it sooner. Before 8.4, it actually does on tables w

Re: [PERFORM] Memory usage of writer process

2009-08-13 Thread Alvaro Herrera
Alex wrote: > The writer process seems to be using inordinate amounts of memory: > > PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ > COMMAND > 11088 postgres 13 -2 3217m 2.9g 2.9g S0 38.7 0:10.46 postgres: > writer process > 20190 postgres 13 -2 3219m 71m 68m S0 0.9

[PERFORM] Memory usage of writer process

2009-08-13 Thread Alex
The writer process seems to be using inordinate amounts of memory: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 11088 postgres 13 -2 3217m 2.9g 2.9g S0 38.7 0:10.46 postgres: writer process 20190 postgres 13 -2 3219m 71m 68m S0 0.9 0:52.48 postgres: cribq

[PERFORM] Less expensive proprietary or Open source ETL tools

2009-08-13 Thread Rstat
Hi, As of today, we are still enjoying our Informatica tool but in a few months we will need to change. Basically we do not use our software at its full capacity and don't feel we need it anymore. So we are trying to find a less expensive solution that would have the same features (or almost...

Re: [PERFORM] Under the hood of views

2009-08-13 Thread Richard Huxton
David Kerr wrote: create view test as select a,b,c,d,e,f,g from testtable; select a from test; (does the engine retrieve b-g?) Shouldn't - the query just gets rewritten macro-style. I don't think it eliminates joins if you don't need any columns, but that's not possible without a bit of an

[PERFORM] Under the hood of views

2009-08-13 Thread David Kerr
developer came by and asked me an interesting question. If he has a view with 20 columns in it, and he selects a specific column from the view in his query. Does the engine when accessing the view return all columns? or is it smart enough to know to just retrive the one? example: create view

Re: [PERFORM] How to run this in reasonable time:

2009-08-13 Thread Matthew Wakeling
On Thu, 13 Aug 2009, Greg Stark wrote: On Thu, Aug 13, 2009 at 3:16 PM, Matthew Wakeling wrote: Now, I'd like to get this done this side of Christmas, so I was wondering if there's a neat trick I can use to get it to only consider the rows from s to e, instead of having to iterate through them a

Re: [PERFORM] Why is vacuum_freeze_min_age 100m?

2009-08-13 Thread Kevin Grittner
Robert Haas wrote: > Someone had the idea a while back of pre-freezing inserted tuples in > the WAL-bypass case. I'm sure I'm not the one who thought up the idea and first posted about it, but I'm certainly an advocate for it. > It seems like in theory you could have a background process th

Re: [PERFORM] How to run this in reasonable time:

2009-08-13 Thread Greg Stark
On Thu, Aug 13, 2009 at 3:16 PM, Matthew Wakeling wrote: > Now, I'd like to get this done this side of Christmas, so I was wondering if > there's a neat trick I can use to get it to only consider the rows from s to > e, instead of having to iterate through them all. I tried this, but got an > error

[PERFORM] How to run this in reasonable time:

2009-08-13 Thread Matthew Wakeling
I'm trying to execute a query to take a row from a table, and return multiple rows, one per integer in the range between two of the fields in that row, for all rows in the table. Perhaps a better explanation would be the query: SELECT id, objectid, bin FROM locationbintemp, generate_series(0

Re: [PERFORM] transaction delays to apply

2009-08-13 Thread Nickolay
Tom Lane wrote: Nickolay writes: BUT it seems that rarely this transaction is being delayed to apply and log entry is being inserted in wrong order: ID timestamp 1 2009-08-08 00:00:00.111 2 2009-08-08 00:00:30.311 3 2009-08-08 00:00:00.211 Yep, that's right - sometimes for 30

Re: [PERFORM] transaction delays to apply

2009-08-13 Thread Nickolay
Tom Lane wrote: Nickolay writes: BUT it seems that rarely this transaction is being delayed to apply and log entry is being inserted in wrong order: ID timestamp 1 2009-08-08 00:00:00.111 2 2009-08-08 00:00:30.311 3 2009-08-08 00:00:00.211 Yep, that's right - sometimes for 30