Re: [HACKERS] Hot standby and b-tree killed items

ncellations and unlike VACUUM there is no way user can control the frequency of prune operations. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www

Re: [HACKERS] Hot standby and b-tree killed items

dated tables which receive very few queries on the standby. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hot standby and b-tree killed items

ce the query was started, it cancels itself automatically, Happy X'mas to all of you! Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Warning about the 8.4 release

elpful. > Should we dump your list to a separate Wiki page so that people can directly edit/comment/remove the items which they are sure about ? Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@p

Re: [HACKERS] visibility maps and heap_prune

On Thu, Jan 15, 2009 at 7:10 AM, Bruce Momjian wrote: > > Is this something for 8.4 CVS? > I worked out the patch as per Heikki's suggestion. So I think he needs to review and decide it's fate. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] pg_dump additional options for performance

n identify what can be restored in parallel and pg_restore using that information during restore. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will i

[HACKERS] CREATE TABLE, load and freezing

x27;t know if this is a real problem for anybody, but I could think of its use case, at least in theory. Is it worth doing ? Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 3: Have you checke

Re: [HACKERS] CREATE TABLE, load and freezing

d-only table. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org

Re: [HACKERS] CREATE TABLE, load and freezing

freezed table and even if the table is subsequently updated, hopefully DSM (or something of that sort) will help us reduce the vacuum freeze time whenever its required. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)---

Re: [HACKERS] CREATE TABLE, load and freezing

. > Understood. But if we consider a special case of creation and loading of a table in a single transaction, we can possibly save the information that the table was loaded with pre-frozen tuples with xmin equals to the transaction creating the table. Thanks, Pavan -- Pav

[HACKERS] Read-ahead and parallelism in redo recovery

Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq

Re: [HACKERS] Read-ahead and parallelism in redo recovery

On Fri, Feb 29, 2008 at 8:19 PM, Florian G. Pflug <[EMAIL PROTECTED]> wrote: > Pavan Deolasee wrote: > > What I am thinking is if we can read ahead these blocks in the shared > > buffers and then apply redo changes to them, it can potentially > > improve things a

Re: [HACKERS] Read-ahead and parallelism in redo recovery

e amount of redo work to be done at the recovery time. If we can significantly improve the recovery logic, we can then think of reducing the work done at the checkpoint time (either through lazy checkpoints or less frequent hard checkpoints) which would benefit the normal database operation. Thanks

Re: [HACKERS] 8.3.0 Core with concurrent vacuum fulls

e lock on the respective catalog relation. Then each try to initialize its own catalog cache. But to do that they need AccessShareLock on each other's table leading to a deadlock. Why not just unconditionally finish the phase 2 as part of InitPostgres ? I understand that we may end up initializing caches t

Re: [HACKERS] 8.3.0 Core with concurrent vacuum fulls

On Wed, Mar 5, 2008 at 3:41 PM, Pavan Deolasee <[EMAIL PROTECTED]> wrote: > > > > Two backends try to vacuum full two different catalog tables. Each acquires > an > exclusive lock on the respective catalog relation. Then each try to > initialize its > own cata

Re: [HACKERS] Nasty bug in heap_page_prune

t to run concurrent INSERTs / UPDATEs / VACUUMs / VACUUM FULL and CREATE/DROP INDEXes, and VACUUM FULL used to once in a while complain about tuple mismatch. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgr

Re: [HACKERS] 8.3.0 Core with concurrent vacuum fulls

when we collapse the redirected line pointers and that we can do at the end, on the original page. The last step which we run inside a critical section would then be just like invoking heap_xlog_clean with the information collected in the first pass. Thanks, Pavan -- Pavan Deolasee EnterpriseDB htt

Re: [HACKERS] 8.3.0 Core with concurrent vacuum fulls

metic caught my eye though. ! nunused = (end - nowunused); Shouldn't we typecast them to (char *) first ? Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your su

Re: [HACKERS] Nasty bug in heap_page_prune

n CTID and fail when the cached tuple is accessed. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] count(*) performance improvement ideas

e "counter" table tuple. This still doesn't solve the serializable transaction problem though. But I am sure we can figure out some solution for that case as well if we agree on the general approach. I am sure this must have been discussed before. So what are the objections ? Thanks

Re: [HACKERS] count(*) performance improvement ideas

On Wed, Mar 12, 2008 at 9:01 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > "Pavan Deolasee" <[EMAIL PROTECTED]> writes: > > I am sure this must have been discussed before. > > Indeed. Apparently you didn't find the threads in which the idea of >

Re: [HACKERS] count(*) performance improvement ideas

that the counter table may be completely cached in memory and won't bloat much. Also, we can always have a GUC (like pgstats) to control the overhead. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@post

Re: [HACKERS] count(*) performance improvement ideas

about now, nobody would have complained :-) Anyways, your point is taken and it would be great if can make it configurable, if not table level then at least globally. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] Nasty bug in heap_page_prune

; True :-) And I would personally prefer any hack than playing with left over redirected line pointers in VACUUM FULL. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Nasty bug in heap_page_prune

me. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PROC_VACUUM_FOR_WRAPAROUND doesn't work expectedly

pTransactionContext) and restore it back after vacuum() returns. But vacuum() might have started a new transaction invalidating the saved context. Do we see any problem here ? Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] Proposal for db level triggers

ombine > rows into a BLOb in an audit table. > Another use case of BEFORE COMMIT trigger is to update the row counts for fast select count(*) operations (of course with some additional infrastructure) Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via

Re: [HACKERS] count(*) performance improvement ideas

and merge them with the summary rows and delete those temporary rows I guess we can write the generic triggers as contrib module. What needs to done is to let user specify the tables and the conditions on which they want to track count(*) and then apply those conditions in the generic

Re: [HACKERS] MemoryContextSwitchTo() confusion

ally attached to the lifespan of the object itself. You may need to choose one of them if you know that what you are allocating can not or should not outlive that object. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hack

Re: [HACKERS] [GENERAL] ANALYZE getting dead tuple count hopelessly wrong

so update the FSM information of a page when its pruned/defragged so that the page can also be used for subsequent INSERTs or non-HOT UPDATEs in other pages. This might be easier said than done. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-h

Re: [HACKERS] [GENERAL] ANALYZE getting dead tuple count hopelessly wrong

h anyway --- it always examines all line pointers > on each selected page, so we might as well rewrite it to use a simple > loop more like vacuum uses. > I agree. I would write a patch on these lines, unless you are already on to it. Thanks, Pavan -- Pavan Deolasee Enterpris

Re: [HACKERS] [GENERAL] ANALYZE getting dead tuple count hopelessly wrong

can make heap_page_prune() to only return number of HOT tuples pruned and then explicitly count the DEAD line pointers in tups_vacuumed. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com Analyze-fix.patch.gz Description: GNU Zip compressed data -- Sent via

Re: [HACKERS] build multiple indexes in single table pass?

ated with the heap scan. So it might be tricky to separate out the scan and the index building activity. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: h

Re: [HACKERS] [GENERAL] ANALYZE getting dead tuple count hopelessly wrong

t_analyze() > can handle the latter part by groveling through the backend's pending > statistics data. > Seems like a right approach to me. I assume we shall do the same for DELETE_IN_PROGRESS tuples. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sen

Re: [HACKERS] [GENERAL] ANALYZE getting dead tuple count hopelessly wrong

On Thu, Apr 3, 2008 at 10:02 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > > I've applied a modified/extended form of this patch for 8.3.2. > Thanks. I had another concern about VACUUM not reporting DEAD line pointers (please see up thread). Any comments on that ? Thanks, Pavan

Re: [HACKERS] [GENERAL] ANALYZE getting dead tuple count hopelessly wrong

On Thu, Apr 3, 2008 at 10:39 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > "Pavan Deolasee" <[EMAIL PROTECTED]> writes: > > > Thanks. I had another concern about VACUUM not reporting DEAD line > > pointers (please see up thread). Any comments on that ? > &

Re: [HACKERS] [GENERAL] ANALYZE getting dead tuple count hopelessly wrong

On Fri, Apr 4, 2008 at 11:10 AM, Tom Lane <[EMAIL PROTECTED]> wrote: > The > policy of this project is that we only put nontrivial bug fixes into > back branches, and I don't think this item qualifies ... > Got it. I will submit a patch for HEAD. Thanks, Pav

Re: [HACKERS] Free Space Map data structure

hat level we would run out of maximum supported file size anyways. Well, this could be completely orthogonal to suggestions you are seeking, but nevertheless I had this thought for quite some time. So just wanted to speak about it :-) Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enter

Re: [HACKERS] Free Space Map data structure

go wrong often. And periodically, VACUUM would correct any mistakes in FSM info. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Segfault using heap_form_tuple

On Wed, Apr 9, 2008 at 10:48 PM, Claudio Rossi <[EMAIL PROTECTED]> wrote: > nulls = (bool *)palloc(natts*sizeof(bool *)); > May not be related to segfault you are seeing, but this looks completely wrong. You want array of bool and not (bool *). Thanks, Pavan -- Pavan Deolasee

Re: [HACKERS] Regression test fails when BLCKSZ is 1kB

hat we are lucky or our regression suite don't have long enough running tests to give autovacuum chance to recycle some of the dead tuples. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Regression test fails when BLCKSZ is 1kB

lock 1 .. 10 Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Regression test fails when BLCKSZ is 1kB

On Mon, Apr 21, 2008 at 10:54 PM, Pavan Deolasee <[EMAIL PROTECTED]> wrote: > Case 1. > > Insert 100 records --- goes into block 1 .. 10 > Delete 100 records > Insert 100 more records --- goes into 11 .. 20 > > > Case 2. > > Insert 100 records --- g

Re: [HACKERS] Can't t compile current HEAD

w include/portability directory added yesterday. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] deadlock while doing VACUUM and DROP

r similar deadlocks waiting to happen. Also I am not sure if the issue is big enough to demand the change. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www

Re: [HACKERS] deadlock while doing VACUUM and DROP

l" :( > Yeah. I think we better fix this, especially given the above mentioned scenario. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Avoiding second heap scan in VACUUM

ed successfully, but that won't have any correctness implication, but would only delay reclaiming DEAD_RECLAIMED line pointers. Comments ? Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] Avoiding second heap scan in VACUUM

t; will need only a single scan. Smaller tables that require almost > continual VACUUMing will probably do two scans, but who cares? > Yeah, I think we need to target the large table case. The second pass is obviously much more costly for large tables. I think the timed-wait answers your c

Re: [HACKERS] Avoiding second heap scan in VACUUM

. But still the second pass would be required and it would re-dirty all the pages again, Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Avoiding second heap scan in VACUUM

stoppers or the benefits of avoiding a second scan on a large table is not worth. I personally have a strong feeling that it's worth the efforts. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] Avoiding second heap scan in VACUUM

the initial blocks to remove the DEAD line pointers. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] No answers on CommitFest procedures?

ed, either at the code level or design level. I expect that feedback during this commit fest. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] initdb in current cvs head broken?

ing a clean build at your end once again. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Database corruption help

intermediate tuples in the HOT chain may get removed (because we handle aborted heap-only tuples separately) and break the HOT chain. I am also looking at the pruning logic to see if I can spot something unusual. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent

Re: [HACKERS] visibility maps and heap_prune

On Tue, Jul 21, 2009 at 10:38 AM, Robert Haas wrote: > > Pavan, are you planning to respond to Alex's comments and/or update this > patch? > Yes, I will. Hopefully by end of this week. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com -- Sent

Re: [HACKERS] Frequent Update Project: Design Overview of HOT Updates

On 11/10/06, Josh Berkus wrote: Tom,> Actually, you omitted to mention the locking aspects of moving tuples> around --- exactly how are you going to make that work without breaking> concurrent scans?I believe that's the "unsolved technical issue" in the prototype, unless Pavan h

Re: [HACKERS] Frequent Update Project: Design Overview of HOT Updates

On 11/10/06, Tom Lane <[EMAIL PROTECTED]> wrote: "Pavan Deolasee" <[EMAIL PROTECTED]> writes:> On 11/10/06, Josh Berkus < josh@agliodbs.com> wrote: >> I believe that's the "unsolved technical issue" in the prototype, unless>> Pavan has sol

Re: [HACKERS] Frequent Update Project: Design Overview of HOT Updates

On 11/10/06, Tom Lane <[EMAIL PROTECTED] > wrote: "Pavan Deolasee" <[EMAIL PROTECTED]> writes:> Yes. The last bit in the t_infomask is used up to mark presence of overflow > tuple header. But I believe there are few more bits that can be reused. > There are thr

Re: [HACKERS] Frequent Update Project: Design Overview of HOT Updates

On 11/10/06, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: Tom Lane wrote:> (Actually, the assumption that you can throw an additional back-pointer> into overflow tuple headers is the worst feature of this proposal in> that regard --- it's really not that easy to support multiple header > formats.)

Re: [HACKERS] Frequent Update Project: Design Overview of HOTUpdates

On 11/10/06, Simon Riggs <[EMAIL PROTECTED]> wrote: On Fri, 2006-11-10 at 12:32 +0100, Zeugswetter Andreas ADI SD wrote:> e.g. a different header seems no easier in overflow than in heap True. The idea there is that we can turn frequent update on/off fairlyeasily for normal tables since there are n

Re: [HACKERS] Frequent Update Project: Design Overview of HOT Updates

On 11/10/06, Tom Lane <[EMAIL PROTECTED]> wrote: "Pavan Deolasee" <[EMAIL PROTECTED]> writes:> On 11/10/06, Tom Lane <[EMAIL PROTECTED]> wrote:> (2) Isn't this full of race conditions? > I agree, there  could be race  conditions. But IMO we can handl

Re: [HACKERS] [GENERAL] Autovacuum Improvements

Simon Riggs wrote: On Fri, 2006-12-29 at 20:25 -0300, Alvaro Herrera wrote: Christopher Browne wrote: Seems to me that you could get ~80% of the way by having the simplest "2 queue" implementation, where tables with size < some threshold get thrown at the "little table" queue, and tables above

Re: [HACKERS] [GENERAL] Autovacuum Improvements

Simon Riggs wrote: > On Fri, 2006-12-29 at 20:25 -0300, Alvaro Herrera wrote: >> Christopher Browne wrote: >> >>> Seems to me that you could get ~80% of the way by having the simplest >>> "2 queue" implementation, where tables with size < some threshold get >>> thrown at the "little table" queue,

[HACKERS] Free space management within heap page

I am thinking that maintaining fragmented free space within a heap page might be a good idea. It would help us to reuse the free space ASAP without waiting for a vacuum run on the page. This in turn will lead to lesser heap bloats and also increase the probability of placing updated tuple in the s

Re: [HACKERS] Free space management within heap page

On 1/23/07, Martijn van Oosterhout wrote: On Tue, Jan 23, 2007 at 01:48:08PM +0530, Pavan Deolasee wrote: > We might not be able to reuse the line pointers because indexes may have > references to it. All such line pointers will be freed when the page is > vacuumed during the regul

Re: [HACKERS] Free space management within heap page

On 1/23/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: ITAGAKI Takahiro wrote: > Keeping only line pointers itself is not a problem, but it might lead > bloating of line pointers. If a particular tuple in a page is replaced > repeatedly, the line pointers area bloats up to 1/4 of the page.

Re: [HACKERS] Free space management within heap page

On 1/23/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: Pavan Deolasee wrote: > So during a sequential or index scan, if a tuple is found to be dead, the > corresponding line pointer is marked "unused" and the space is returned > to a > free list. This free list i

Re: [HACKERS] 10 weeks to feature freeze (Pending Work)

On 1/23/07, Joshua D. Drake <[EMAIL PROTECTED]> wrote: Or so... :) I am sure there are more, the ones with question marks are unknowns but heard of in the ether somewhere. Any additions or confirmations? I have the first phase of Frequent Update Optimizations (HOT) patch ready. But I held it

Re: [HACKERS] Piggybacking vacuum I/O

On 1/22/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: I've been looking at the way we do vacuums. The fundamental performance issue is that a vacuum generates nheapblocks+nindexblocks+ndirtyblocks I/Os. Vacuum cost delay helps to spread the cost like part payment, but the total is the same.

Re: [HACKERS] Free space management within heap page

On 1/23/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: ITAGAKI Takahiro wrote: > BLCKSZ is typically 8192 bytes and sizeof(ItemPointerData) is 4 bytes. > 1/4 comes from 8192 / 4 = 2048. If we allow zero-size tuples, the line > pointers area can bloat up to the ratio. We have tuples no less th

Re: [HACKERS] Piggybacking vacuum I/O

On 1/23/07, Tom Lane <[EMAIL PROTECTED]> wrote: "Pavan Deolasee" <[EMAIL PROTECTED]> writes: > Would it help to set the status of the XMIN/XMAX of tuples early enough such > that the heap page is still in the buffer cache, but late enough such that > the XMIN

Re: [HACKERS] Piggybacking vacuum I/O

On 1/23/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: Pavan Deolasee wrote: > Another source of I/O is perhaps the CLOG read/writes for checking > transaction status. If we are talking about large tables like accounts in > pgbench or customer/stock in DBT2, the tables ar

Re: [HACKERS] Free space management within heap page

On 1/23/07, Tom Lane <[EMAIL PROTECTED]> wrote: "Pavan Deolasee" <[EMAIL PROTECTED]> writes: > I know it might break the ctid chain, but does that really matter ? Yes. You can't just decide that the tuple isn't needed anymore. As per other followup, you

Re: [HACKERS] Piggybacking vacuum I/O

On 1/24/07, Tom Lane <[EMAIL PROTECTED]> wrote: "Pavan Deolasee" <[EMAIL PROTECTED]> writes: > On a typical desktop class 2 CPU Dell machine, we have seen pgbench > clocking more than 1500 tps. Only if you had fsync off, or equivalently a disk drive that lies abou

Re: [HACKERS] Free space management within heap page

On 1/24/07, Martijn van Oosterhout wrote: On Wed, Jan 24, 2007 at 12:45:53PM +0530, Pavan Deolasee wrote: > My apologies if this has been discussed before. I went through the earlier > discussions, but its still very fuzzy to me. I am not able to construct a > case > where a tuple

Re: [HACKERS] Free space management within heap page

On 1/24/07, Gregory Stark <[EMAIL PROTECTED]> wrote: "Pavan Deolasee" <[EMAIL PROTECTED]> writes: > On 1/24/07, Martijn van Oosterhout wrote: >> >> I thought the classical example was a transaction that updated the same >> tuple multiple times befor

Re: [HACKERS] Piggybacking vacuum I/O

On 1/24/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: Pavan Deolasee wrote: > I have just counted the number of read/write calls on the CLOG blocks. As > you can > see the total number of CLOG reads jumped from 545323 to 1181851 i.e. > 1181851 - 545323 = 636528 CLOG block

Re: [HACKERS] Piggybacking vacuum I/O

On 1/25/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: Pavan Deolasee wrote: > > Also is it worth optimizing on the total read() system calls which might > not > cause physical I/O, but > still consume CPU ? I don't think it's worth it, but now that we'r

Re: [HACKERS] Piggybacking vacuum I/O

On 1/26/07, Alvaro Herrera <[EMAIL PROTECTED]> wrote: Heikki Linnakangas wrote: > I'd like to see still more evidence that it's a problem before we start > changing that piece of code. It has served us well for years. So the TODO could be "investigate whether caching pg_clog and/or pg_subtrans

Re: [HACKERS] Piggybacking vacuum I/O

On 1/26/07, Alvaro Herrera <[EMAIL PROTECTED]> wrote: Maybe have the bgwriter update hint bits as it evicts pages out of the cache? It could result in pg_clog read traffic for each page that needs eviction; not such a hot idea. I thought once we enhance clog so that there are no clog reads,

Re: [HACKERS] Piggybacking vacuum I/O

On 1/26/07, Tom Lane <[EMAIL PROTECTED]> wrote: I think what he's suggesting is deliberately not updating the hint bits during a SELECT ... No, I was suggesting doing it in bgwriter so that we may not need to that during a SELECT. Of course, we need to investigate more and have numbers to pr

[HACKERS] stack usage in toast_insert_or_update()

Not sure whether its worth optimizing, but had spotted this while browsing the code a while back. So thought would post it anyways. The stack usage for toast_insert_or_update() may run into several KBs since the MaxHeapAttributeNumber is set to a very large value of 1600. The usage could anywhere

Re: [HACKERS] stack usage in toast_insert_or_update()

On 1/30/07, Tom Lane <[EMAIL PROTECTED]> wrote: "Pavan Deolasee" <[EMAIL PROTECTED]> writes: > The stack usage for toast_insert_or_update() may run into several KBs since > the MaxHeapAttributeNumber is set to a very large value of 1600. The usage > could anywhe

Re: [HACKERS] stack usage in toast_insert_or_update()

On 1/31/07, Tom Lane <[EMAIL PROTECTED]> wrote: "Pavan Deolasee" <[EMAIL PROTECTED]> writes: > Btw, I noticed that the toast_insert_or_update() is re-entrant. > toast_save_datum() calls simple_heap_insert() which somewhere down the > line calls toast_insert_or_

Re: [HACKERS] stack usage in toast_insert_or_update()

On 1/31/07, Pavan Deolasee <[EMAIL PROTECTED]> wrote: Attached is a patch which would print the recursion depth for toast_insert_or_update() before PANICing the server to help us examine the core. Here is the attachment. Thanks, Pavan -- EnterpriseDB http://www.enterprise

Re: [HACKERS] stack usage in toast_insert_or_update()

On 1/31/07, Tom Lane <[EMAIL PROTECTED]> wrote: We can't change TOAST_MAX_CHUNK_SIZE without forcing an initdb, but I think that it would be safe to remove the MAXALIGN'ing of the tuple size in the tests in heapam.c, that is That would mean that the tuple size in the heap may exceed TOAST_TU

Re: [HACKERS] HOT for PostgreSQL 8.3

On 2/9/07, Simon Riggs <[EMAIL PROTECTED]> wrote: On Wed, 2007-02-07 at 14:17 -0500, Tom Lane wrote: > ISTM we could fix that by extending the index VACUUM interface to > include two concepts: aside from "remove these TIDs when you find them", > there could be "replace these TIDs with those TID

Re: [HACKERS] HOT for PostgreSQL 8.3

On 2/11/07, Hannu Krosing <[EMAIL PROTECTED]> wrote: Ühel kenal päeval, P, 2007-02-11 kell 12:35, kirjutas Tom Lane: > Hannu Krosing <[EMAIL PROTECTED]> writes: > > What if we would just reuse the root tuple directly instead of turning > > it into a stub ? > > This would create a cycle of ctid p

Re: [HACKERS] HOT for PostgreSQL 8.3

On 2/12/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: Hannu Krosing wrote: > Ühel kenal päeval, P, 2007-02-11 kell 12:35, kirjutas Tom Lane: >> Hannu Krosing <[EMAIL PROTECTED]> writes: >>> What if we would just reuse the root tuple directly instead of turning >>> it into a stub ? >>> This w

Re: [HACKERS] HOT for PostgreSQL 8.3

On 2/13/07, Tom Lane <[EMAIL PROTECTED]> wrote: Heikki Linnakangas <[EMAIL PROTECTED]> writes: > Hannu Krosing wrote: >> Are we actually doing that ? I.E are null bitmaps really allocated in 1 >> byte steps nowadays ? > Yes. Not really; we still have to MAXALIGN at the end of the bitmap. The

[HACKERS] HOT WIP Patch - version 1

This is a WIP patch based on the recent posting by Simon and discussions thereafter. We are trying to do one piece at a time and intention is to post the work ASAP so that we could get early and continuous feedback from the community. We could then incorporate those suggestions in the next WIP pat

Re: [HACKERS] HOT for PostgreSQL 8.3

On 2/14/07, Hannu Krosing <[EMAIL PROTECTED]> wrote: OTOH, for same page HOT tuples, we have the command and trx ids stored twice first as cmax,xmax of the old tuple and as cmin,xmin of the updated tuple. One of these could probably be used for in-page HOT tuple pointer. I think we recently

Re: [HACKERS] HOT WIP Patch - version 1

On 2/14/07, Tom Lane <[EMAIL PROTECTED]> wrote: Heikki Linnakangas <[EMAIL PROTECTED]> writes: > What's the verdict on relaxing the "live tuple's ctid doesn't change > rule"? I think that's unacceptable; it is known that that will break the ODBC and JDBC drivers, as well as any other programs t

Re: [HACKERS] HOT WIP Patch - version 1

On 2/15/07, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: Do we actually ever want to remove dead tuples from the middle of the chain? If a tuple in the middle of the chain is dead, surely every tuple before it in the chain is dead as well, and we want to remove them as well. I'm thinking, remo

Re: [HACKERS] HOT for PostgreSQL 8.3

On 2/16/07, Hannu Krosing <[EMAIL PROTECTED]> wrote: Ühel kenal päeval, K, 2007-02-14 kell 10:41, kirjutas Tom Lane: > Hannu Krosing <[EMAIL PROTECTED]> writes: > > OTOH, for same page HOT tuples, we have the command and trx ids stored > > twice first as cmax,xmax of the old tuple and as cmin,xm

Re: [HACKERS] HOT for PostgreSQL 8.3

On 2/16/07, Zeugswetter Andreas ADI SD <[EMAIL PROTECTED]> wrote: > > As described, you've made > > that problem worse because you're trying to say we don't know which of > > the chain entries is pointed at. > > There should be a flag, say HOT_CHAIN_ENTRY for the tuple the it's called HEAP_UPD

Re: [HACKERS] HOT for PostgreSQL 8.3

On 2/16/07, Zeugswetter Andreas ADI SD <[EMAIL PROTECTED]> wrote: Oh sorry. Thanks for the clarification. Imho HEAP_UPDATE_ROOT should be renamed for this meaning then (or what does ROOT mean here ?). Maybe HEAP_UPDATE_CHAIN ? Yes, you are right. There is some disconnect between what Simon h

Re: [HACKERS] wishlist items ..

On 2/17/07, Lukas Kahwe Smith <[EMAIL PROTECTED]> wrote: I have emailed Gregory, Pavan and Simon only 2 days ago, so I am not suprised to not haven gotten feedback yet. Oops, I haven't received the email you mentioned ? Can you resend me the same ? Thanks, Pavan -- EnterpriseDB http:/

Re: [HACKERS] Chatter on DROP SOMETHING IF EXISTS

On 2/19/07, Tom Lane <[EMAIL PROTECTED]> wrote: Peter Eisentraut <[EMAIL PROTECTED]> writes: > Am Montag, 19. Februar 2007 13:12 schrieb Alvaro Herrera: >> I don't understand -- what problem you got with "NO OPERATION"? It >> seemed a sound idea to me. > It seems nonorthogonal. What if only s

[HACKERS] HOT WIP Patch - version 2

Reposting - looks like the message did not get through in the first attempt. My apologies if multiple copies are received. This is the next version of the HOT WIP patch. Since the last patch that I sent out, I have implemented the HOT-update chain pruning mechanism. When following a HOT-update

<    2   3   4   5   6   7   8   >