Re: [HACKERS] 9.3 Beta1 status report

he code before committing the patch. Andreas reported and fixed a bug which was an oversight in the FK locks patch. I think he also discovered an old bug that the tableoid was not being properly checked for HOT conditions but that had very limited impact since its not common to have indexes on tableoids. Thanks, Pavan -- Pavan Deolasee http://www.linkedin.com/in/pavandeolasee

[HACKERS] Couple of issues with pg_xlogdump

nks, Pavan -- Pavan Deolasee http://www.linkedin.com/in/pavandeolasee

Re: [HACKERS] Couple of issues with pg_xlogdump

On Tue, Apr 23, 2013 at 3:00 PM, Andres Freund wrote: > On 2013-04-23 14:51:05 +0530, Pavan Deolasee wrote: > > Hello, > > > > I was playing with pg_xlogdump in the HEAD and found a few issues. > > > > 1. Tried compiling pg_xlogdump via PGXS mechanism and it f

[HACKERS] Confusing long option in pg_receivexlog/basebackup/dumpall

mean something else. Thanks, Pavan -- Pavan Deolasee http://www.linkedin.com/in/pavandeolasee

Re: [HACKERS] Confusing long option in pg_receivexlog/basebackup/dumpall

e-id/512520fe.6050...@vmware.com>. > If you have any better ideas, I'm all ears... > > No, I don't have better ideas. I was worried that it was an oversight. But now I know it was chosen this way after a careful consideration. Thanks, Pavan -- Pavan Deolasee http://www.linkedin.com/in/pavandeolasee

Re: [HACKERS] too much pgbench init output

On Wed, Sep 5, 2012 at 9:47 AM, Robert Haas wrote: > > Actually it'd be nice to even get a bit more output: say, a timestamp > on each line, and a completion percentage... but now I'm getting > greedy. > > May be we need a verbosity level and print a lot less or a lot more information than what w

[HACKERS] pg_dump transaction's read-only mode

I'm looking at the following code in pg_dump.c /* * Start transaction-snapshot mode transaction to dump consistent data. */ ExecuteSqlStatement(fout, "BEGIN"); if (fout->remoteVersion >= 90100) { if (serializable_deferrable) ExecuteSqlStatement(fout,

Re: [HACKERS] [BUGS] BUG #7521: Cannot disable WAL log while using pg_dump

(Adding -hackers. Did not realize it got dropped) On Fri, Sep 7, 2012 at 11:25 PM, Gezeala M. Bacuño II wrote: > On Fri, Sep 7, 2012 at 7:17 AM, Pavan Deolasee > wrote: > > > > > > On Fri, Sep 7, 2012 at 7:00 PM, Marie Bacuno II > wrote: > >> > >>

Re: [HACKERS] pg_dump transaction's read-only mode

On Fri, Sep 7, 2012 at 6:06 PM, Kevin Grittner wrote: > That makes sense to me. The reason I didn't make that change when I > added the serializable special case to pg_dump was that it seemed > like a separate question; I didn't want to complicate an already big > patch with unnecessary changes

Re: [HACKERS] Problem Observed in behavior of Create Index Concurrently and Hot Update

On Wed, Oct 31, 2012 at 11:41 AM, Amit Kapila wrote: > > > According to me, the problem happens at Step-4. As at Step-4, it does the > HOT update due to which validate_index() is not able to put an entry for > C2=5 > > Thanks for the report. I can reproduce this issue. As you rightly pointed out,

[HACKERS] Bug in ALTER COLUMN SET DATA TYPE ?

Hi, I noticed this behavior on master and it seems like a bug to me: postgres=# CREATE TABLE test (a float check (a > 10.2)); CREATE TABLE postgres=# CREATE TABLE test_child() INHERITS(test); CREATE TABLE postgres=# ALTER TABLE test ALTER COLUMN a SET DATA TYPE numeric;

Re: [HACKERS] Problem Observed in behavior of Create Index Concurrently and Hot Update

On Wed, Oct 31, 2012 at 2:40 PM, Simon Riggs wrote: > > > > > diff --git a/src/backend/utils/cache/relcache.c > > b/src/backend/utils/cache/relcache.c > > index a59950e..9cadb3f 100644 > > --- a/src/backend/utils/cache/relcache.c > > +++ b/src/backend/utils/cache/relcache.c > > @@ -3355,6 +3355,1

Re: [HACKERS] Bug in ALTER COLUMN SET DATA TYPE ?

On Fri, Nov 2, 2012 at 8:11 PM, Tom Lane wrote: > Pavan Deolasee writes: > > I noticed this behavior on master and it seems like a bug to me: > > > postgres=# CREATE TABLE test (a float check (a > 10.2)); > > CREATE TABLE > > postgres=# CREATE TABLE test_child()

Re: [HACKERS] Bug in ALTER COLUMN SET DATA TYPE ?

On Fri, Nov 2, 2012 at 9:00 PM, Tom Lane wrote: > > > In the 8.4 branch, the error is coming from here: > > regression=# \set VERBOSITY verbose > regression=# ALTER TABLE test ALTER COLUMN a TYPE numeric; > ERROR: 42P16: constraint must be added to child tables too > LOCATION: ATAddCheckConstr

Re: [HACKERS] Bug in ALTER COLUMN SET DATA TYPE ?

On Fri, Nov 2, 2012 at 9:09 PM, Pavan Deolasee wrote: > > > I also though that for a moment, but the commit that I mentioned did not > move that error message from somewhere else. I also tested by resetting > to commit 1f0363001166ef6a43619846e44cfb9dbe7335ed (previous to the >

Re: [HACKERS] Bug in ALTER COLUMN SET DATA TYPE ?

On Sat, Nov 3, 2012 at 10:21 AM, Nikhil Sontakke wrote: > > >> So coming back to the issue, do you think it's a good idea to teach >> ATAddCheckConstraint() that the call is coming from a late phase of ALTER >> TABLE ? >> >> +1 > > You mentioned AT_PASS_OLD_INDEX in your original mail, but I gues

Re: [HACKERS] Bug in ALTER COLUMN SET DATA TYPE ?

On Mon, Nov 5, 2012 at 4:41 AM, Tom Lane wrote: > > > It's clear that we need to pass down the information that this action is > coming from re-creation of a check constraint, but I think the above > proposal for how to do it is pretty wrong-headed. Yeah, I only meant that we need to teach ATEx

Re: [HACKERS] visibility maps and heap_prune

494 msec to 366 msec, I somehow kept waiting for Heikki's decision on the general direction of the patch and lost interest in between. If we are still interested in this, I can work out a patch and submit for next release if not this. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] should I post the patch as committed?

up in the git log. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] PostgreSQL Core Team

his spare time, he >> serves as President of the Board of PostgreSQL Europe. > Many congratulations Magnus! You fully deserve it. Thanks, Pavan -- Pavan Deolasee EnterpriseDB     http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] branching for 9.2devel

ent is run and the new patch is sent back to the submitter who can then submit it to the hackers for review. If the patch does not apply cleanly, the same can also be emailed back to the submitter. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] "full_page_writes" makes no difference?

fter a checkpoint. So yes, it can reduce the amount of WAL written to the disk. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] "full_page_writes" makes no difference?

On Wed, May 4, 2011 at 5:46 PM, Pavan Deolasee wrote: > > > On Wed, May 4, 2011 at 7:16 AM, Tian Luo wrote: > >> Hi guys, >> >> No matter I turn on or turn off the "full_page_writes", I always >> observe 8192-byte writes of log data for simple write

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

h influential opinions. Its very likely that if you submit an idea or a patch, you would get some comment/suggestion/criticism very early. Since HOT is mentioned often in these discussions, I thought I should share my experience. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] crash-safe visibility map, take five

step or just pass InvalidBuffer to clear() in which case, it would assume that the vm buffer is not pinned and do the needful. The comment at the top of visibilitymap_pin_ok says "On entry, *buf", but the function really takes just a buf. You can possibly fold visibilitymap_pin_ok() into a macro (and also name it slightly differently like visibilitymap_is_pinned ?). Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] SSI predicate locking on heap -- tuple or row?

uld be great if you can upload the slides on the conference website. To expect that the attendees would have understood the nitty-gritties of SSI just listening to the presentation is so unhuman :-) Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com

[HACKERS] Proposal: Another attempt at vacuum improvements

ave really not spent time on that, so it will be easy find holes and corner cases. That can be worked out if we believe something like this will be useful. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] Expression Evaluator used for creating the plan tree / stmt ?

Otherwise static functions may get optimized and you may not see them in gdb stack. CFLAGS="-O0" ./configure --enable-debug Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] Proposal: Another attempt at vacuum improvements

some further improvements. > > Thanks for bringing up the idea during PGCon. That helped me to get interested in this again. I hope we would be able to take this to a logical conclusion this time and do something to alleviate the pain. > On Tue, May 24, 2011 at 2:58 AM, Pavan Deolasee >

Re: [HACKERS] Proposal: Another attempt at vacuum improvements

On Wed, May 25, 2011 at 7:20 PM, Robert Haas wrote: > On Wed, May 25, 2011 at 7:07 AM, Pavan Deolasee > wrote: >>> But instead of allocating permanent space in the page header, which would >>> both reduce (admittedly only by 8 bytes) the amount of space available

Re: [HACKERS] Proposal: Another attempt at vacuum improvements

heap first to collect the dead line pointers and then remove the corresponding index pointers. We would need to also take into consideration the implications on visibility map for any such scheme to work correctly and efficiently. Thanks, Pavan -- Pavan Deolasee EnterpriseDB     http://www.enterp

Re: [HACKERS] Proposal: Another attempt at vacuum improvements

ing the pruning, otherwise its overwritten with the current LSN. Since we hold the buffer cleanup lock, the special LSN storage can be reclaimed by shuffling things around. Thanks, Pavan -- Pavan Deolasee EnterpriseDB     http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal: Another attempt at vacuum improvements

On Wed, May 25, 2011 at 11:39 PM, Robert Haas wrote: > On Wed, May 25, 2011 at 1:43 PM, Pavan Deolasee > Now, there >> is no way you can store is after the line pointer array without moving >> the live tuple somewhere else. > > So far I agree.  But don't we always

Re: [HACKERS] Proposal: Another attempt at vacuum improvements

On Thu, May 26, 2011 at 9:40 AM, Robert Haas wrote: > On Wed, May 25, 2011 at 11:51 PM, Pavan Deolasee > Having said that, it doesn't excite me too much because I >> think we should do the dead line pointer reclaim operation during page >> pruning and we are already holdi

[HACKERS] Database research papers

to PostgreSQL implementation ? http://www.almaden.ibm.com/u/mohan/RJ6649Rev.pdf Thanks, Pavan -- Pavan Deolasee EnterpriseDB     http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] Proposal: Another attempt at vacuum improvements

On Thu, May 26, 2011 at 4:10 PM, Pavan Deolasee wrote: > On Thu, May 26, 2011 at 9:40 AM, Robert Haas wrote: > >> Currently, I believe the only way a page can get marked all-visible is >> by vacuum.  But if we make this change, then it would be possible for >> a HO

[HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

been debated/tested already. Thanks, Pavan -- Pavan Deolasee EnterpriseDB     http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

On Fri, May 27, 2011 at 7:36 PM, Tom Lane wrote: > Pavan Deolasee writes: >> My statistical skills are limited, but wouldn't that mean that for a >> fairly well distributed write activity across a large table, if there >> are even 3-4% update/deletes, we would mo

Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

On Fri, May 27, 2011 at 7:41 PM, Heikki Linnakangas wrote: > On 27.05.2011 16:52, Pavan Deolasee wrote: >> >> On closer inspection, I realized that we have >> deliberately put in this hook to ensure that we use visibility maps >> only when we see at least SKIP_PAGES_THR

Re: [HACKERS] [ADMIN] pg_class reltuples/relpages not updated by autovacuum/vacuum

om the VACUUM, tks to the VM. > > That would require proof, not just suggestion.  Skipping pages will > defeat the OS read-ahead algorithm, and so could easily cost more than > reading them. > My worry is what we have right now is also based on just assumptions and gut feeling

Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

APIs to test if a chunk of pages are all visible or not. We can then use that API to test every N blocks (where N is the number where continuous sequential scans would still be better than sequential scans with gaps) and either read all of them sequentially or just skip all of them. Tha

Re: [HACKERS] [ADMIN] pg_class reltuples/relpages not updated by autovacuum/vacuum

On Sun, May 29, 2011 at 9:27 PM, Tom Lane wrote: > Pavan Deolasee writes: >> On Sun, May 29, 2011 at 8:10 PM, Tom Lane wrote: >>> That would require proof, not just suggestion.  Skipping pages will >>> defeat the OS read-ahead algorithm, and so could easily cost

Re: [HACKERS] [ADMIN] pg_class reltuples/relpages not updated by autovacuum/vacuum

On Sun, May 29, 2011 at 10:35 PM, Tom Lane wrote: > Pavan Deolasee writes: >> I am sorry if I sounded terse above. But my gripe is that sometimes we >> are too reluctant to listen to ideas and insist on producing some hard >> numbers first which might take significant ef

Re: [HACKERS] heap vacuum & cleanup locks

how that the cleanup lock is indeed a major bottleneck. Thanks, Pavan -- Pavan Deolasee EnterpriseDB     http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] gdb with postgres

305 > client_read_ended(). > Please compile with -O0 -g flags to see all the debug symbols. CFLAGS="-O0 -g" ./configure --enable-debug Thanks, Pavan -- Pavan Deolasee EnterpriseDB     http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hacker

Re: [HACKERS] Proposal: Another attempt at vacuum improvements

On Thu, May 26, 2011 at 4:10 PM, Pavan Deolasee wrote: > > So are there any other objections/suggestions ? Anyone else cares to > look at the brief design that we discussed above ? Otherwise, I would > go ahead and work on this in the coming days. Of course, I will keep > the li

[HACKERS] Autoanalyze and OldestXmin

to be using it anywhere. Since acquire_sample_rows() returns palloced tuples, can't we let OldestXmin advance after scanning a page by ignoring procs with the flag set, just like we do for PROC_IN_VACUUM ? Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com 7457:2006-0

Re: [HACKERS] Autoanalyze and OldestXmin

On Wed, Jun 8, 2011 at 9:03 PM, Greg Stark wrote: > > On Jun 8, 2011 1:49 PM, "Pavan Deolasee" wrote: > > > > > > Hi All, > > > There is a PROC_IN_ANALYZE flag, but we don't seem to be using it > anywhere. Since acquire_sample_rows() returns

Re: [HACKERS] Autoanalyze and OldestXmin

On Wed, Jun 8, 2011 at 10:45 PM, Tom Lane wrote: > Pavan Deolasee writes: > > I first thought that analyze and vacuum can not run concurrently on the > same > > table since they take a conflicting lock on the table. So even if we > ignore > > the analyze process while

Re: [HACKERS] Autoanalyze and OldestXmin

On Thu, Jun 9, 2011 at 11:50 AM, Pavan Deolasee wrote: > > > > Ah, I see. Would there will be benefits if we can do some special handling > for cases where we know that ANALYZE is running outside a transaction block > and that its not going to invoke any user-defined func

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

> > Can we make this the last post on this topic please? > +1 :) Thanks, Pavan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Autoanalyze and OldestXmin

> > I am wondering if we shouldn't be asking ourselves a different > question: why is ANALYZE running long enough on your tables for this > to become an issue? How long is it taking? > The log file attached in the first post has the details; it's taking around 5 mins for the accounts table w

Re: [HACKERS] Autoanalyze and OldestXmin

On 09-Jun-2011, at 8:29 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Jun 9, 2011 at 10:52 AM, Pavan Deolasee >> wrote: >>>> I am wondering if we shouldn't be asking ourselves a different >>>> question: why is ANALYZE running long enough on

Re: [HACKERS] [OT] why not keeping the original column name in catalog after a drop?

lks > about? > > May be when a column with the same name is added and again dropped ? Of course, we can have the attribute number and column name both to avoid conflict. Thanks, Pavan -- Pavan Deolasee http://www.linkedin.com/in/pavandeolasee

Re: [HACKERS] WARM and indirect indexes

s a separate IndexAM. But I re-read this thread and found that you'd in fact done it that way in the first version but then discarded it for performance reasons. Is there a merit in evaluating that path for indirect indexes again? Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] WARM and indirect indexes

M. Please share your thoughts on what more can be and should be done. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

that much > what we used to do, except perhaps as a secondary comment, but we do > care very much about what it actually does, which the current comment > doesn't really explain. > > I reworked this quite a bit and I believe the new code does what you suggested. The HeapTuple

[HACKERS] Failure in commit_ts tap tests

r. $ perl -v This is perl 5, version 18, subversion 2 (v5.18.2) built for darwin-thread-multi-2level (with 2 registered patches, see perl -V for more detail) Copyright 1987-2013, Larry Wall Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

Re: [HACKERS] Failure in commit_ts tap tests

7 PST". If I unset that, the result comes in a format such as "2017-01-20 21:31:47.766371-08". Looks like perl doesn't throw an error if it can parse the leading part of the string as a numeric. It still throws a warning, but the test passes. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Failure in commit_ts tap tests

Argument "Fri 2017-46" isn't numeric in numeric ne (!=) at -e line 1. not ok 1 - ok # Failed test 'ok' # at -e line 1. # Tests were run but no plan was declared and done_testing() was not seen. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Failure in commit_ts tap tests

On Sat, Jan 21, 2017 at 9:39 PM, Tom Lane wrote: > Pavan Deolasee writes: > > On Sat, Jan 21, 2017 at 9:09 PM, Tom Lane wrote: > >> Hm, but what of the "null" value? Also, I get > >> > >> $ perl -e 'use warnings; use Test::More; ok("201

[HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

ing the old attribute list. 2017-01-29 10:44:31.889 UTC [73091] [xid=4670908]LOG: heap_update (cic_tab_accounts), hot_attrs ((b 9)) Other session, which I included for comparison, sees the new index and recomputes its rd_indexattr in time and hence that update works as expected. Thanks, Pavan

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

lationPutHeapTuple(relation, buffer, heaptup, > false, > > + &root_offnum); > > becomes > > root_offnum = RelationPutHeapTuple(relation, buffer, heaptup, > false, > InvalidOffsetNumber); > > Mak

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

and heap_hot_search() should return > a tri-valued enum instead of boolean; that idea looks reasonable in > theory but callers have to do more work afterwards, so maybe not. > > Ok. I'll try to rearrange it a bit. May be we just have one API after all? There are only a very few callers of these APIs. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Jan 31, 2017 at 7:21 PM, Alvaro Herrera wrote: > Pavan Deolasee wrote: > > On Thu, Jan 26, 2017 at 2:38 AM, Alvaro Herrera < > alvhe...@2ndquadrant.com> > > wrote: > > > > The simple_heap_update + CatalogUpdateIndexes pattern is getting > > >

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Jan 31, 2017 at 7:37 PM, Alvaro Herrera wrote: > Pavan Deolasee wrote: > > > > > Sounds good. Should I submit that as a separate patch on current master? > > Yes, please. > > Attached. Two new APIs added. - CatalogInsertHeapAndIndex which does a simple_h

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

chanism in place to clear those bits. So may be we add something to do that. I'd some other ideas (and a patch too) to reuse bits from t_ctid.ip_pos given that offset numbers can be represented in just 13 bits, even with the maximum block size. Can look at that if it comes to finding more bit

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

t; in the places where this optimization had been applied. > > > This looks reasonable enough to me. > > Done. > > Thanks for taking care of this. Shame that I missed this because I'd specifically noted the special casing for large objects etc. But looks like while changin

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

On Mon, Jan 30, 2017 at 7:20 PM, Pavan Deolasee wrote: > > Based on my investigation so far and the evidence that I've collected, > what probably happens is that after a relcache invalidation arrives at the > new transaction, it recomputes the rd_indexattr but based on

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

*/ > > + relation->rd_indexattr = NULL; > + > > I think setting directly to NULL will leak the memory. > Good catch, thanks. Will fix. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

es. This also makes it safe 4750 * (deadlock-free) for us to take locks on the relation's indexes. I've attached a revised patch based on these thoughts. It looks a bit more invasive than the earlier one-liner, but looks better to me. Thanks, Pavan -- Pavan Deolasee

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

may be fix CREATE INDEX CONCURRENTLY so that at least the first phase which add the index entry to the catalog happens with a strong lock. This will lead to momentary blocking of write queries, but protect us from the race. I'm assuming this is the only code that can cause the problem, and I h

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

se? Or do you think it will be hard to come up with a proper fix for the issue and it will need some serious refactoring? Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

t I'm pretty sure it will be ok. I also like the fact that retry logic is now limited to only when index set changes, which fits in the situation we're dealing with. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

a bigger machine, with many users (on a decent AWS machine, I would find every third CIC to be corrupt). Moreover the setup is not doing anything extraordinary. Just concurrent updates which change between HOT to non-HOT and a CIC. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

On Mon, Feb 6, 2017 at 8:15 AM, Amit Kapila wrote: > On Mon, Feb 6, 2017 at 8:01 AM, Pavan Deolasee > wrote: > > > > > > On Mon, Feb 6, 2017 at 1:44 AM, Tom Lane wrote: > >> > >> > >> > >> > 2. In the second patch, we tried to reco

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

at we know this bug exists? They can't fully trust CIC and they don't have any mechanism to confirm if the newly built index is corruption free or not. I'm not suggesting that we should hastily refactor the code, but at the very least some sort of band-aid fix which helps the situa

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

above.). I don't quite get that. Since rd_idattr must be already cached at that point and we don't expect to process a relcache flush between successive calls to RelationGetIndexAttrBitmap(), we should return a consistent copy of rd_idattr. But may be I am missing something. Thanks,

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

On Mon, Feb 6, 2017 at 8:01 AM, Pavan Deolasee wrote: > >> > I like this approach. I'll run the patch on a build with > CACHE_CLOBBER_ALWAYS, but I'm pretty sure it will be ok. > While it looks certain that the fix will miss this point release, FWIW I ran this pat

Re: [HACKERS] Index corruption with CREATE INDEX CONCURRENTLY

the push, and barring negative reports from the buildfarm, > it will be in today's releases. > Thank you for taking care of it. Buildfarm is looking green until now. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] [COMMITTERS] pgsql: Release note updates.

eturn any result, "SELECT * FROM tab WHERE key = oldval" may actually return the updated (and wrong) tuple. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] [COMMITTERS] pgsql: Release note updates.

mmitted-good transactions. It's all theory and I haven't had time to try this out. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] UNDO and in-place update

s. If we could somehow address that situation, that may still go a long way in containing bloat. Of course, there may still be cases where bloat will be unavoidable, but it seems far lesser work to me that a complete overhaul of the MVCC working. Thanks, Pavan [1] https://www.postgresql.org/message

Re: [HACKERS] Indirect indexes

On Thu, Dec 8, 2016 at 3:11 AM, Robert Haas wrote: > On Thu, Oct 20, 2016 at 11:30 AM, Pavan Deolasee > wrote: > > We have a design to convert WARM chains back to HOT and that will > increase > > the percentage of WARM updates much beyond 50%. I was waiting for > feedbac

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

would require us to compare only those columns that any caller of the function might be interested in. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Mon, Dec 26, 2016 at 11:49 AM, Jaime Casanova < jaime.casan...@2ndquadrant.com> wrote: > On 2 December 2016 at 07:36, Pavan Deolasee > wrote: > > > > I've updated the patches after fixing the issue. Multiple rounds of > > regression passes for me without

Re: [HACKERS] rewrite HeapSatisfiesHOTAndKey

n, please rebase your WARM patch on top of this and let me know how > you like it. I'll post a new version of indirect indexes later this > week. > > I've rebased WARM on top of this patch and the proposed changes look fine from WARM's perspective too. I'll send rebased p

Re: [HACKERS] rewrite HeapSatisfiesHOTAndKey

On Mon, Jan 2, 2017 at 10:17 AM, Amit Kapila wrote: > On Mon, Jan 2, 2017 at 9:28 AM, Pavan Deolasee > wrote: > > > > > > On Mon, Jan 2, 2017 at 8:52 AM, Amit Kapila > wrote: > >> > >> > >> I think there is some chance that such a change

Re: [HACKERS] rewrite HeapSatisfiesHOTAndKey

time. I will leave it to Alvaro's judgement to decide whether it's worth to commit the patch now or later when he or other committer looks at committing WARM/indirect indexes because without either of those patches this change probably does not bring up much value, if we ignore the slight im

Re: [HACKERS] rewrite HeapSatisfiesHOTAndKey

sult on a high end m/c. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Patch to add functionality to specify ORDER BY in CREATE FUNCTION for SRFs

can choose the most optimal sorting algorithm using the hint that the dataset is probably already sorted. Actually why not just do that? Though AFAIK its against our stated policy of not adding any planner hints. Thanks, Pavan -- Pavan Deolasee http://www.linkedin.com/in/pavandeolasee

[HACKERS] Single pass vacuum - take 1

of it. May be we can get the exact current LSN and not store it in the pg_class if we don't do anything during the cycle. Comments ? Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.

Re: [HACKERS] Single pass vacuum - take 1

nd Phase 3 > if the number of rows to remove is too few, say < 1%. > If you have set the vacuum parameters such that it kicks in when there are say 5% updates/deletes, you would most likely have that much work to do anyways. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] Single pass vacuum - take 1

On Thu, Jul 14, 2011 at 12:43 PM, Heikki Linnakangas < heikki.linnakan...@enterprisedb.com> wrote: > On 14.07.2011 18:57, Pavan Deolasee wrote: > >> On Thu, Jul 14, 2011 at 11:46 AM, Simon Riggs >> wrote: >> >>> I'd say that seems way too complex for

Re: [HACKERS] Single pass vacuum - take 1

On Thu, Jul 14, 2011 at 6:22 PM, Simon Riggs wrote: > On Thu, Jul 14, 2011 at 4:57 PM, Pavan Deolasee > wrote: > > > Thanks Simon for looking at the patch. > > Sorry, I didn't notice there was a patch attached. Not reviewed it. I > thought we were still just ta

Re: [HACKERS] Single pass vacuum - take 1

main sections of the pageinspect module, and > yes, I care. :-) > > Yeah, we should probably teach pageinspect to see and report this additional information. Thanks, Pavan > > -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] Single pass vacuum - take 1

On Thu, Jul 21, 2011 at 11:51 AM, Robert Haas wrote: > On Tue, Jul 12, 2011 at 4:47 PM, Pavan Deolasee > wrote: > > Comments ? > > I was going to spend some time reviewing this, but I see that (1) it > has bit-rotted slightly - there is a failing hunk in pg_class.h

Re: [HACKERS] Single pass vacuum - take 1

xcited about adding that complexity to the code. Even if a backend does not have up-to-date value, it will fail to collect the dead-vacuum pointers, but soon either it will catch up or some other backend will remove them or the next vacuum will take care of it. Thanks, Pavan -- Pavan Deolasee EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] Single pass vacuum - take 1

ng with different ideas > about the correct counter value. > > I think thats fine. For example, every backend runs with a different RecentXmin today and that doesn't impact any functionality. It only limits how much they can prune at any given time. The same would happen

Re: [HACKERS] write scalability

small.  pgbench invocation is: > It might be worthwhile to test only with the accounts and history table and also increasing the number of statements in a transaction. Otherwise the tiny tables can quickly become a bottleneck. Thanks, Pavan -- Pavan Deolasee EnterpriseDB     http:

<    1   2   3   4   5   6   7   8   >