Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Mon, Mar 20, 2017 at 8:11 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Sun, Mar 19, 2017 at 3:05 AM, Pavan Deolasee > <pavan.deola...@gmail.com> wrote: > > On Thu, Mar 16, 2017 at 12:53 PM, Robert Haas <robertmh...@gmail.com> > wrote: > > >>

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Mar 15, 2017 at 12:46 AM, Alvaro Herrera <alvhe...@2ndquadrant.com> wrote: > Pavan Deolasee wrote: > > On Tue, Mar 14, 2017 at 7:17 AM, Alvaro Herrera < > alvhe...@2ndquadrant.com> > > wrote: > > > > I have already commented a

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Mar 14, 2017 at 7:17 AM, Alvaro Herrera wrote: > > @@ -234,6 +236,21 @@ index_beginscan(Relation heapRelation, > > scan->heapRelation = heapRelation; > > scan->xs_snapshot = snapshot; > > > > + /* > > + * If the index supports recheck, make

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

at the second idea would require, is that it can > easily mask bugs. Agree. That's probably one reason why Alvaro wrote the patch to start with. I'll give the first of those two options a try. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Mar 22, 2017 at 8:43 AM, Pavan Deolasee <pavan.deola...@gmail.com> wrote: > > > BTW may I request another test with the attached patch? In this patch, we > check if the PageIsFull() even before deciding which attributes to check > for modification. If the

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

uple is smaller than the tuple used in previous UPDATE and we have just enough space to do HOT update this time, but I can think that's too narrow). Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services 0001_

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

lame it on AWS instance completely, but the pattern in your tests looks very similar where the number slowly and steadily keeps going up. If you do complete retest but run v18/v19 first and then run master, may be we'll see a complete opposite picture? Thanks, Pavan -- Pavan Deolasee

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Mar 14, 2017 at 7:16 PM, Alvaro Herrera <alvhe...@2ndquadrant.com> wrote: > Pavan Deolasee wrote: > > On Tue, Mar 14, 2017 at 7:17 AM, Alvaro Herrera < > alvhe...@2ndquadrant.com> > > wrote: > > > > I have already commented about the executor invo

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

the same tuple is rechecked often. But if a tuple is rechecked only once then may be other overheads will kick-in, thus reducing the regression significantly. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

ly the code that needs to be written. > > > > Yes, but once it is written it will take years before those bits can be > > used on most installations. > > Actually, the 2 bits from old-style VACUUM FULL bits could be reused if > one of the WARM bits woul

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

and then access TOAST to handle that. Would that work? Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Apr 5, 2017 at 8:42 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Apr 4, 2017 at 10:21 PM, Pavan Deolasee > <pavan.deola...@gmail.com> wrote: > > On Thu, Mar 30, 2017 at 7:55 PM, Robert Haas <robertmh...@gmail.com> > wrote: > >> bu

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

is doing additional work, without providing any benefit, may be you can still find regression. I am willing to fix them as long as they are fixable and we are comfortable with the additional code complexity. IMHO certain trade-offs are good, but I understand that not everybody will agree with my

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Apr 12, 2017 at 9:23 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Tue, Apr 11, 2017 at 10:50 PM, Pavan Deolasee > > > > > And I fixed them as quickly as humanly possible. > > > > Yes, you have responded to them quickly, but I didn't get

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Apr 6, 2017 at 1:06 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Apr 5, 2017 at 2:32 PM, Pavan Deolasee <pavan.deola...@gmail.com> > wrote: > > > The other way is to pass old tuple values along with the new tuple > values to > > amwarmin

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

e insert (if everything matches) or set a PREWARM flag on the old pointer, and insert the new tuple with POSTWARM flag. Searching for old index pointer will be non-starter for non-unique indexes, unless they are also sorted by TID, something that Claudio's patch does. What I am not sur

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Apr 12, 2017 at 10:42 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Apr 11, 2017 at 1:20 PM, Pavan Deolasee > > > 5. Added code to set a CLEAR pointer to a WARM pointer when we know that > the > > entire chain is WARM. This should address the wor

Re: [HACKERS] PANIC in pg_commit_ts slru after crashes

again increment ShmemVariableCache->nextXid, when there are no prepared transactions in the system. I wonder if we should do as in attached patch. It's not entirely clear to me why only CommitTS fails and not other slrus. One possible reason could be that CommitTS is started before t

Re: [HACKERS] Cleanup: avoid direct use of ip_posid/ip_blkid

On Thu, Mar 2, 2017 at 9:55 PM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2/22/17 08:38, Pavan Deolasee wrote: > > One reason why these macros are not always used is because they > > typically do assert-validation to ensure ip_posid has a valid value

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

On Wed, Mar 8, 2017 at 7:33 AM, Robert Haas wrote: > On Tue, Mar 7, 2017 at 4:26 PM, Stephen Frost wrote: > > Right, that's what I thought he was getting at and my general thinking > > was that we would need a way to discover if a CIC is ongoing on the

Re: [HACKERS] Skip all-visible pages during second HeapScan of CIC

tched CIC (or any other operation which may not like concurrent VM set) is in progress. We could use what Stephen suggested upthread to find that state. But right now it's hard to think because there is nothing on either side so we don't know what gets impacted by aggressive VM set and how. Thanks, P

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

ce is between spending considerable amount of time trying to handle every case vs doing it incrementally and start delivering to majority of the users, yet keeping the patch at a manageable level. Even if we were to provide table level option, my preference would be keep it ON by default. Thanks

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

les and I know no known bugs, but if we don't want to risk system tables, we might want to turn it off (just prior to release may be). Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

nough time reviewing other patches. I know its critical, and I'll try to improve on that. Congratulations to all whose work got accepted and many thanks to all reviewers/committers/CF managers. I know how difficult and thankless that work is. Thanks, Pavan -- Pavan Deolasee ht

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

ge, thus failing to make to the storage. Can you please test with the attached patch and confirm it works? I was able to reproduce the exact same assertion on my end and the patch seems to fix it. But an additional check won't harm. I'll include the fix in the next set of pat

Re: [HACKERS] Assertion failure in REL9_5_STABLE

On Wed, Apr 19, 2017 at 11:19 AM, Andrew Gierth <and...@tao11.riddles.org.uk > wrote: > >>>>> "Pavan" == Pavan Deolasee <pavan.deola...@gmail.com> writes: > > Pavan> I am attaching a patch that throws a similar ERROR during > Pavan> plannin

[HACKERS] Assertion failure in REL9_5_STABLE

frame #19: 0x00010f3eb066 postgres`PostmasterMain(argc=3, argv=0x7fb151403740) + 5414 at postmaster.c:1306 frame #20: 0x00010f32bddf postgres`main(argc=3, argv=0x7fb151403740) + 751 at main.c:228 frame #21: 0x7fff9c0b1255 libdyld.dylib`start + 1 frame

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

e_prune_opt(). Do we need to worry about wal_log_hints or checksums producing WAL because of hint bit updates? While I haven't read the thread, I am assuming if HOT pruning can happen, surely hint bits can get set too. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Jul 26, 2017 at 6:26 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Apr 18, 2017 at 4:25 AM, Pavan Deolasee > <pavan.deola...@gmail.com> wrote: > > I'll include the fix in the next set of patches. > > I haven't see a new set of patches. Are you i

Re: [HACKERS] Assertion failure in REL9_5_STABLE

On Wed, Apr 19, 2017 at 11:19 AM, Andrew Gierth <and...@tao11.riddles.org.uk > wrote: > >>>>> "Pavan" == Pavan Deolasee <pavan.deola...@gmail.com> writes: > > Pavan> I am attaching a patch that throws a similar ERROR during > Pavan> plannin

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Fri, Jul 28, 2017 at 5:57 AM, Peter Geoghegan <p...@bowt.ie> wrote: > Pavan Deolasee <pavan.deola...@gmail.com> wrote: > > I'll be happy if someone wants to continue hacking the patch further and > > get it in a committable shape. I can stay actively involved. But T

Re: [HACKERS] pgbench stuck with 100% cpu usage

performance, but testing crash recovery and I use it very often. This particular test was run to catch another bug which will be reported separately. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] pgbench stuck with 100% cpu usage

laptop's battery started draining out much more quickly. Having seen the problem, it seems very obvious though. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

[HACKERS] pgbench stuck with 100% cpu usage

t sure what was the rationale for that decision, may be to deal with transient failures? The commit that introduced this code is 12788ae49e1933f463bc. So I am copying Heikki. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Tr

Re: [HACKERS] Parallel worker error

rallelWorkerMain call > SetCurrentRoleId using information passed via the FixedParallelState > (not sure of the precise details here). > > Seems like a reasonable approach to me. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

[HACKERS] Parallel worker error

stand the differences in "role" and "session authorization", but it still looks like a bug to me. May be SerializeGUCState() should check if SetRoleIsActive is true and only then save the role information? Thanks, Pavan -- Pavan Deolasee http://www.2ndQ

Re: [HACKERS] Parallel worker error

oncurrent role drop problem someday, the single session problem looks more pressing. Thanks, Pavan -- Pavan Deolasee http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

<    3   4   5   6   7   8