Re: [HACKERS] Improving the Performance of Full Table Updates

2007-10-08 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > "Heikki Linnakangas" <[EMAIL PROTECTED]> writes: >> * mmap the WAL segments, instead of using the slru buffers. > > This one's almost certainly a nonstarter, for lack of any portable way > to control when writes happen. I think mlock and msync(MS_SYNC) oug

Re: [HACKERS] Improving the Performance of Full Table Updates

2007-10-08 Thread Tom Lane
"Heikki Linnakangas" <[EMAIL PROTECTED]> writes: > We mustn't write data pages before the WAL hits the disk, but we don't > have any such limitation for WAL. Um, you're right, I was remembering discussions about trying to mmap data files. Still, googling "msync performance" leaves me full of conc

Re: [HACKERS] Improving the Performance of Full Table Updates

2007-10-08 Thread Heikki Linnakangas
Tom Lane wrote: > "Heikki Linnakangas" <[EMAIL PROTECTED]> writes: >> * mmap the WAL segments, instead of using the slru buffers. > > This one's almost certainly a nonstarter, for lack of any portable way > to control when writes happen. For flushing, there's msync, which I believe is quite porta

Re: [HACKERS] Improving the Performance of Full Table Updates

2007-10-08 Thread Tom Lane
"Heikki Linnakangas" <[EMAIL PROTECTED]> writes: > * mmap the WAL segments, instead of using the slru buffers. This one's almost certainly a nonstarter, for lack of any portable way to control when writes happen. regards, tom lane ---(end of broadc

Re: [HACKERS] Improving the Performance of Full Table Updates

2007-10-08 Thread Heikki Linnakangas
Gokulakannan Somasundaram wrote: >Finally i found some time to look more into the CRC code. The > time is mostly taken when there is a back-up block in the XLOG structure. > when it calculates the CRC for the entire block(there is some optimization > already for the holes), the time is

Re: [HACKERS] Improving the Performance of Full Table Updates

2007-10-07 Thread Gokulakannan Somasundaram
Hi Heikki, Finally i found some time to look more into the CRC code. The time is mostly taken when there is a back-up block in the XLOG structure. when it calculates the CRC for the entire block(there is some optimization already for the holes), the time is spent on the CRC macro. I trie

Re: [HACKERS] Improving the Performance of Full Table Updates

2007-09-26 Thread Heikki Linnakangas
Gokulakannan Somasundaram wrote: > Hi Tom/ Heikki, >Thanks for the suggestion. After profiling i got similar results. > So i am thinking of a design like this to get the performance improvement. > > a) We can get one page for insert(during update) and we will hold the write > lock on

Re: [HACKERS] Improving the Performance of Full Table Updates

2007-09-26 Thread Gokulakannan Somasundaram
Hi Tom/ Heikki, Thanks for the suggestion. After profiling i got similar results. So i am thinking of a design like this to get the performance improvement. a) We can get one page for insert(during update) and we will hold the write lock on it, till the page gets filled. In this way, R

Re: [HACKERS] Improving the Performance of Full Table Updates

2007-09-21 Thread Heikki Linnakangas
Gokulakannan Somsundaram wrote: > Again if this full table updates are thought with the OLTP applications in > mind, then this is not at all a suitable option. This will only benefit the > people with Data Warehouses. > > Expecting some more replies Start with profiling. I just ran a quick o

Re: [HACKERS] Improving the Performance of Full Table Updates

2007-09-21 Thread Gokulakannan Somsundaram
Hi Tom, Thanks for the feedback. Let me clarify my intention again. This was thought for improving the performance of the Data Warehousing applications Full table updates similar to "Update dd set n2=n2+1" When you talked about firing triggers, i looked into the implementation of triggers a

Re: [HACKERS] Improving the Performance of Full Table Updates

2007-09-20 Thread Tom Lane
"Gokulakannan Somsundaram" <[EMAIL PROTECTED]> writes: > I propose to change this row-by-row approach, when it is a full table > update. I plan to send a extra flag(which will be set for Full table > Deletes/Updates). this would make the access method directly acquire the > exclusive lock and updat

Re: [HACKERS] Improving the Performance of Full Table Updates

2007-09-20 Thread Gokulakannan Somsundaram
The obvious advantages are a) Avoidance of one read lock per page b) One Big write lock instead of multiple write locks. But as you said, i will do some initial profiling and get back. Thanks, Gokul. On 9/20/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: > > Gokulakannan Somsundaram wrote: >

Re: [HACKERS] Improving the Performance of Full Table Updates

2007-09-20 Thread Heikki Linnakangas
Gokulakannan Somsundaram wrote: > Hi, >The Current architecture of Updates in PostgreSQL is > 1) Make a select query out of update. It involves a READ lock/BUFFER_SHARE > 2) Get the tupleid > 3) Goto the buffer containing the tupleid, make a BUFFER_EXCLUSIVE lock on > it > 4) update it > 5) Rep