On Thu, Apr 15, 2021 at 5:12 PM Andres Freund wrote:
> > https://thebuild.com/blog/2019/02/08/do-not-change-autovacuum-age-settings/
>
> Not at all convinced. The issue of needing to modify a lot of
> all-visible pages again to freeze them is big enough to let it be a
> problem even after the
Hi,
On 2021-04-14 21:30:29 -0700, Peter Geoghegan wrote:
> I think that this was once true, but is now much less common, mostly
> due to the freeze map stuff in 9.6. And due a general recognition that
> the *risk* of increasing them is just too great (a risk that we can
> hope was diminished by
On Wed, Apr 14, 2021 at 8:38 PM Andres Freund wrote:
> The reason I didn't do further reviews for things in this thread was
> that I was trying really hard to get the shared memory stats patch into
> a committable shape - there were just not enough hours in the day. I
> think it's to be expected
Hi,
On 2021-04-14 19:53:29 -0700, Peter Geoghegan wrote:
> > Or at least
> > tests for it should be added (pg_resetwal + autovacuum_naptime=1s or
> > such should make that doable, or even just running a small test with
> > lower thresholds).
>
> You know what else doesn't have test coverage? Any
On Wed, Apr 14, 2021 at 6:53 PM Andres Freund wrote:
> > To a large degree the failsafe is something that is written in the
> > hope that it will never be needed. This is unlike most other things,
> > and has its own unique risks.
>
> Among them that the code is not covered by tests and is
Hi,
On 2021-04-14 14:55:36 -0700, Peter Geoghegan wrote:
> On Wed, Apr 14, 2021 at 12:33 PM Andres Freund wrote:
> > I'm getting a bit bothered by the speed at which you're pushing fairly
> > substantial behavioural for vacuum. In this case without even a warning
> > that you're about to do so.
Hi,
On 2021-04-14 20:08:10 -0400, Robert Haas wrote:
> On Wed, Apr 14, 2021 at 5:55 PM Peter Geoghegan wrote:
> > On Wed, Apr 14, 2021 at 12:33 PM Andres Freund wrote:
> > > I'm getting a bit bothered by the speed at which you're pushing fairly
> > > substantial behavioural for vacuum. In this
On Wed, Apr 14, 2021 at 5:08 PM Robert Haas wrote:
> I think this is largely missing the point Andres was making, which is
> that you made a significant behavior change after feature freeze
> without any real opportunity for discussion.
I don't believe that it was a significant behavior change,
On Wed, Apr 14, 2021 at 5:55 PM Peter Geoghegan wrote:
> On Wed, Apr 14, 2021 at 12:33 PM Andres Freund wrote:
> > I'm getting a bit bothered by the speed at which you're pushing fairly
> > substantial behavioural for vacuum. In this case without even a warning
> > that you're about to do so.
>
On Wed, Apr 14, 2021 at 12:33 PM Andres Freund wrote:
> I'm getting a bit bothered by the speed at which you're pushing fairly
> substantial behavioural for vacuum. In this case without even a warning
> that you're about to do so.
To a large degree the failsafe is something that is written in
and try to VACUUM. Skipping the truncation might now make it
harder to actually vacuum all the tables without running out of space.
FWIW, I also don't like that substantial behaviour changes to how vacuum
works were discussed only in a thread titled "New IndexAM API
controlling index vacuum
On Wed, Apr 14, 2021 at 4:59 AM Peter Geoghegan wrote:
>
> On Mon, Apr 12, 2021 at 11:05 PM Masahiko Sawada
> wrote:
> > I realized that when the failsafe is triggered, we don't bypass heap
> > truncation that is performed before updating relfrozenxid. I think
> > it's better to bypass it too.
On Mon, Apr 12, 2021 at 11:05 PM Masahiko Sawada wrote:
> I realized that when the failsafe is triggered, we don't bypass heap
> truncation that is performed before updating relfrozenxid. I think
> it's better to bypass it too. What do you think?
I agree. Bypassing heap truncation is exactly the
On Thu, Apr 8, 2021 at 11:42 AM Masahiko Sawada wrote:
>
> On Thu, Apr 8, 2021 at 8:41 AM Peter Geoghegan wrote:
> >
> > On Wed, Apr 7, 2021 at 8:52 AM Peter Geoghegan wrote:
> > > All of the changes from your fixup patch are clear improvements, and
> > > so I'll include them in the final
On Thu, Apr 8, 2021 at 8:41 AM Peter Geoghegan wrote:
>
> On Wed, Apr 7, 2021 at 8:52 AM Peter Geoghegan wrote:
> > All of the changes from your fixup patch are clear improvements, and
> > so I'll include them in the final commit. Thanks!
>
> I did change the defaults of the GUCs to 1.6 billion,
On Wed, Apr 7, 2021 at 8:52 AM Peter Geoghegan wrote:
> All of the changes from your fixup patch are clear improvements, and
> so I'll include them in the final commit. Thanks!
I did change the defaults of the GUCs to 1.6 billion, though.
All patches in the patch series have been pushed.
On Wed, Apr 7, 2021 at 12:50 AM Masahiko Sawada wrote:
> Thank you for updating the patches!
>
> I've done the final round of review:
All of the changes from your fixup patch are clear improvements, and
so I'll include them in the final commit. Thanks!
> For 0001 patch, we call
On Tue, Apr 6, 2021 at 5:49 AM Peter Geoghegan wrote:
>
> On Mon, Apr 5, 2021 at 5:09 PM Peter Geoghegan wrote:
> > Oh yeah. "static BufferAccessStrategy vac_strategy" is guaranteed to
> > be initialized to 0, simply because it's static and global. That
> > explains it.
>
> So do we need to
On Wed, Apr 7, 2021 at 12:16 PM Peter Geoghegan wrote:
>
> On Tue, Apr 6, 2021 at 7:05 AM Matthias van de Meent
> wrote:
> > If you have updated patches, I'll try to check them this evening (CEST).
>
> Here is v11, which is not too different from v10 as far as the
> truncation stuff goes.
>
>
On Tue, Apr 6, 2021 at 7:05 AM Matthias van de Meent
wrote:
> If you have updated patches, I'll try to check them this evening (CEST).
Here is v11, which is not too different from v10 as far as the
truncation stuff goes.
Masahiko should take a look at the last patch again. I renamed the
GUCs to
On Tue, 6 Apr 2021 at 05:13, Peter Geoghegan wrote:
>
> On Mon, Apr 5, 2021 at 2:44 PM Matthias van de Meent
> wrote:
> > This assertion is false, and should be a guarding if-statement. HOT
> > redirect pointers are not updated if the tuple they're pointing to is
> > vacuumed (i.e. when it was
On Mon, Apr 5, 2021 at 2:44 PM Matthias van de Meent
wrote:
> This assertion is false, and should be a guarding if-statement. HOT
> redirect pointers are not updated if the tuple they're pointing to is
> vacuumed (i.e. when it was never committed) so this nextoffnum might
> in a correctly working
Hi,
On 2021-04-05 17:18:37 -0700, Peter Geoghegan wrote:
> On Mon, Apr 5, 2021 at 5:09 PM Peter Geoghegan wrote:
> > Oh yeah. "static BufferAccessStrategy vac_strategy" is guaranteed to
> > be initialized to 0, simply because it's static and global. That
> > explains it.
>
> So do we need to
On Mon, Apr 5, 2021 at 5:09 PM Peter Geoghegan wrote:
> Oh yeah. "static BufferAccessStrategy vac_strategy" is guaranteed to
> be initialized to 0, simply because it's static and global. That
> explains it.
So do we need to allocate a strategy in workers now, or leave things
as they are/were?
On Mon, Apr 5, 2021 at 5:00 PM Andres Freund wrote:
> What do you mean with "appear to work"? Isn't, in 13, the only
> consequence of vac_strategy not being "propagated" that we'll not use a
> strategy in parallel workers? Presumably that was hard to notice
> because most people don't run manual
Hi,
On 2021-04-05 16:53:58 -0700, Peter Geoghegan wrote:
> REL_13_STABLE will need to be considered separately. I still haven't
> figured out how this ever appeared to work for this long. The
> vac_strategy/bstrategy state simply wasn't propagated at all.
What do you mean with "appear to work"?
On Mon, Apr 5, 2021 at 4:29 PM Tom Lane wrote:
> Peter Geoghegan writes:
> > Do you think that it's okay that we rely on the propagation of global
> > state to parallel workers on Postgres 13? Don't we need something like
> > my fixup commit 49f49def on Postgres 13 as well? At least for the
> >
On Tue, Apr 6, 2021 at 8:29 AM Tom Lane wrote:
>
> Peter Geoghegan writes:
> > Do you think that it's okay that we rely on the propagation of global
> > state to parallel workers on Postgres 13? Don't we need something like
> > my fixup commit 49f49def on Postgres 13 as well? At least for the
>
Peter Geoghegan writes:
> Do you think that it's okay that we rely on the propagation of global
> state to parallel workers on Postgres 13? Don't we need something like
> my fixup commit 49f49def on Postgres 13 as well? At least for the
> EXEC_BACKEND case, I think.
Uh ... *what* propagation of
On Mon, Apr 5, 2021 at 4:30 AM Masahiko Sawada wrote:
> Did you try the change around parallel_process_one_index() that I
> suggested in the previous reply[1]? If we don't change the logic, we
> need to update the above comment. Previously, we update stats[idx] in
> vacuum_one_index() (renamed to
On Sun, 4 Apr 2021 at 04:00, Peter Geoghegan wrote:
>
> On Thu, Apr 1, 2021 at 8:25 PM Peter Geoghegan wrote:
> > I've also found a way to further simplify the table-without-indexes
> > case: make it behave like a regular two-pass/has-indexes VACUUM with
> > regard to visibility map stuff when
On Sun, Apr 4, 2021 at 11:00 AM Peter Geoghegan wrote:
>
> On Thu, Apr 1, 2021 at 8:25 PM Peter Geoghegan wrote:
> > I've also found a way to further simplify the table-without-indexes
> > case: make it behave like a regular two-pass/has-indexes VACUUM with
> > regard to visibility map stuff
On Thu, Apr 1, 2021 at 6:14 AM Robert Haas wrote:
> Without offering an opinion on this particular implementation choice,
> +1 for the idea of trying to make the table-with-indexes and the
> table-without-indexes cases work in ways that will feel similar to the
> user. Tables without indexes are
On Wed, Mar 31, 2021 at 9:44 PM Masahiko Sawada wrote:
> > But let me think about it...I suppose we could do it when one-pass
> > VACUUM considers vacuuming a range of FSM pages every
> > VACUUM_FSM_EVERY_PAGES. That's kind of similar to index vacuuming, in
> > a way -- it wouldn't be too bad to
On Thu, Apr 1, 2021 at 9:58 AM Peter Geoghegan wrote:
>
> On Wed, Mar 31, 2021 at 4:45 AM Masahiko Sawada wrote:
> > Both 0001 and 0002 patch refactors the whole lazy vacuum code. Can we
> > merge them? I basically agree with the refactoring made by 0001 patch
> > but I'm concerned a bit that
On Wed, Mar 31, 2021 at 4:45 AM Masahiko Sawada wrote:
> Both 0001 and 0002 patch refactors the whole lazy vacuum code. Can we
> merge them? I basically agree with the refactoring made by 0001 patch
> but I'm concerned a bit that having such a large refactoring at very
> close to feature freeze
On Wed, Mar 31, 2021 at 9:29 AM Robert Haas wrote:
> I can't effectively review 0001 because it both changes the code for
> individual functions significantly and reorders them within the file.
> I think it needs to be separated into two patches, one of which makes
> the changes and the other of
On Mon, Mar 29, 2021 at 12:16 AM Peter Geoghegan wrote:
> And now here's v8, which has the following additional cleanup:
I can't effectively review 0001 because it both changes the code for
individual functions significantly and reorders them within the file.
I think it needs to be separated
On Wed, Mar 31, 2021 at 12:01 PM Peter Geoghegan wrote:
>
> On Sun, Mar 28, 2021 at 9:16 PM Peter Geoghegan wrote:
> > And now here's v8, which has the following additional cleanup:
>
> And here's v9, which has improved commit messages for the first 2
> patches, and many small tweaks within all
On Wed, Mar 24, 2021 at 11:44 AM Peter Geoghegan wrote:
>
> On Tue, Mar 23, 2021 at 4:02 AM Masahiko Sawada wrote:
> > Here are review comments on 0003 patch:
>
> Attached is a new revision, v5. It fixes bit rot caused by recent
> changes (your index autovacuum logging stuff). It has also been
>
On Mon, Mar 22, 2021 at 6:40 AM Masahiko Sawada wrote:
> I've looked at this 0001 patch and here are some review comments:
> + * Since we might have to prune a second time here, the code is structured to
> + * use a local per-page copy of the counters that caller accumulates. We add
> + * our
On Mon, Mar 22, 2021 at 10:39 PM Masahiko Sawada wrote:
>
> On Sat, Mar 20, 2021 at 11:05 AM Peter Geoghegan wrote:
> >
> > On Wed, Mar 17, 2021 at 7:55 PM Peter Geoghegan wrote:
> >
> > 2. A patch to remove the tupgone case.
> >
> > Severa new and interesting changes here -- see below.
> >
> >
On Tue, Mar 23, 2021 at 12:41 PM Peter Geoghegan wrote:
>
> On Mon, Mar 22, 2021 at 8:33 PM Peter Geoghegan wrote:
> > More concretely, maybe the new GUC is forced to be 1.05 of
> > vacuum_freeze_table_age. Of course that scheme is a bit arbitrary --
> > but so is the existing 0.95 scheme.
>
> I
On Tue, Mar 23, 2021 at 12:28 PM Peter Geoghegan wrote:
>
> On Mon, Mar 22, 2021 at 6:41 PM Masahiko Sawada wrote:
> > But we're not sure when the next anti-wraparound vacuum will take
> > place. Since the table is already vacuumed by a non-aggressive vacuum
> > with disabling index cleanup, an
On Mon, Mar 22, 2021 at 8:33 PM Peter Geoghegan wrote:
> More concretely, maybe the new GUC is forced to be 1.05 of
> vacuum_freeze_table_age. Of course that scheme is a bit arbitrary --
> but so is the existing 0.95 scheme.
I meant to write 1.05 of autovacuum_vacuum_max_age. So just as
On Mon, Mar 22, 2021 at 8:28 PM Peter Geoghegan wrote:
> You seem to be concerned about a similar contradiction. In fact it's
> *very* similar contradiction, because this new GUC is naturally a
> "sibling GUC" of both vacuum_freeze_table_age and
> autovacuum_vacuum_max_age (the "units" are the
On Mon, Mar 22, 2021 at 6:41 PM Masahiko Sawada wrote:
> But we're not sure when the next anti-wraparound vacuum will take
> place. Since the table is already vacuumed by a non-aggressive vacuum
> with disabling index cleanup, an autovacuum will process the table
> when the table gets modified
On Fri, Mar 19, 2021 at 3:36 AM Peter Geoghegan wrote:
>
> On Thu, Mar 18, 2021 at 3:32 AM Masahiko Sawada wrote:
> > If we have the constant threshold of 1 billion transactions, a vacuum
> > operation might not be an anti-wraparound vacuum and even not be an
> > aggressive vacuum, depending on
On Mon, Mar 22, 2021 at 7:05 AM Robert Haas wrote:
> I agree. I was having trouble before understanding exactly what you
> are proposing, but this makes sense to me and I agree it's a good
> idea.
Our ambition is for this to be one big multi-release umbrella project,
with several individual
On Thu, Mar 18, 2021 at 9:42 PM Peter Geoghegan wrote:
> The fact that we can *continually* reevaluate if an ongoing VACUUM is
> at risk of taking too long is entirely the point here. We can in
> principle end index vacuuming dynamically, whenever we feel like it
> and for whatever reasons occur
On Sat, Mar 20, 2021 at 11:05 AM Peter Geoghegan wrote:
>
> On Wed, Mar 17, 2021 at 7:55 PM Peter Geoghegan wrote:
> > Attached patch series splits everything up. There is now a large patch
> > that removes the tupgone special case, and a second patch that
> > actually adds code that dynamically
On Sun, Mar 21, 2021 at 1:24 AM Greg Stark wrote:
> What I've seen is an application that regularly ran ANALYZE on a
> table. This worked fine as long as vacuums took less than the interval
> between analyzes (in this case 1h) but once vacuum started taking
> longer than that interval autovacuum
On Thu, 18 Mar 2021 at 14:37, Peter Geoghegan wrote:
> They usually involve some *combination* of Postgres problems,
> application code problems, and DBA error. Not any one thing. I've seen
> problems with application code that runs DDL at scheduled intervals,
> which interacts badly with vacuum
On Thu, Mar 18, 2021 at 2:05 PM Robert Haas wrote:
> On Wed, Mar 17, 2021 at 11:23 PM Peter Geoghegan wrote:
> > Most anti-wraparound VACUUMs are really not emergencies, though.
>
> That's true, but it's equally true that most of the time it's not
> necessary to wear a seatbelt to avoid personal
On Wed, Mar 17, 2021 at 11:23 PM Peter Geoghegan wrote:
> Most anti-wraparound VACUUMs are really not emergencies, though.
That's true, but it's equally true that most of the time it's not
necessary to wear a seatbelt to avoid personal injury. The difficulty
is that it's hard to predict on which
On Thu, Mar 18, 2021 at 3:32 AM Masahiko Sawada wrote:
> If we have the constant threshold of 1 billion transactions, a vacuum
> operation might not be an anti-wraparound vacuum and even not be an
> aggressive vacuum, depending on autovacuum_freeze_max_age value. Given
> the purpose of skipping
On Thu, Mar 18, 2021 at 3:41 PM Peter Geoghegan wrote:
>
> On Wed, Mar 17, 2021 at 11:23 PM Masahiko Sawada
> wrote:
> > Attached the updated patch that can be applied on top of your v3 patches.
>
> Some feedback on this:
>
> * I think that we can afford to be very aggressive here, because
On Wed, Mar 17, 2021 at 11:23 PM Masahiko Sawada wrote:
> Attached the updated patch that can be applied on top of your v3 patches.
Some feedback on this:
* I think that we can afford to be very aggressive here, because we're
checking dynamically. And we're concerned about extremes only. So an
On Thu, Mar 18, 2021 at 12:23 PM Peter Geoghegan wrote:
>
> On Wed, Mar 17, 2021 at 7:16 PM Masahiko Sawada wrote:
> > Since I was thinking that always skipping index vacuuming on
> > anti-wraparound autovacuum is legitimate, skipping index vacuuming
> > only when we're really close to the point
On Wed, Mar 17, 2021 at 7:16 PM Masahiko Sawada wrote:
> Since I was thinking that always skipping index vacuuming on
> anti-wraparound autovacuum is legitimate, skipping index vacuuming
> only when we're really close to the point of going into read-only mode
> seems a bit conservative, but maybe
On Mon, Mar 15, 2021 at 4:11 PM Andres Freund wrote:
> I kinda wonder whether this case should just be handled by just gotoing
> back to the start of the blkno loop, and redoing the pruning. The only
> thing that makes that a bit more complicatd is that we've already
> incremented
On Wed, Mar 17, 2021 at 7:21 AM Peter Geoghegan wrote:
>
> On Tue, Mar 16, 2021 at 6:40 AM Masahiko Sawada wrote:
> > > Note that I've merged multiple existing functions in vacuumlazy.c into
> > > one: the patch merges lazy_vacuum_all_indexes() and lazy_vacuum_heap()
> > > into a single function
On Tue, Mar 16, 2021 at 6:40 AM Masahiko Sawada wrote:
> > Note that I've merged multiple existing functions in vacuumlazy.c into
> > one: the patch merges lazy_vacuum_all_indexes() and lazy_vacuum_heap()
> > into a single function named vacuum_indexes_mark_unused()
> I agree to create a
On Tue, Mar 16, 2021 at 10:39 PM Masahiko Sawada wrote:
>
> On Mon, Mar 15, 2021 at 11:04 AM Peter Geoghegan wrote:
> >
> > One consequence of my approach is that we now call
> > lazy_cleanup_all_indexes(), even when we've skipped index vacuuming
> > itself. We should at least "check-in" with
On Mon, Mar 15, 2021 at 11:04 AM Peter Geoghegan wrote:
>
> On Thu, Mar 11, 2021 at 8:31 AM Robert Haas wrote:
> > But even if not, I'm not sure this
> > helps much with the situation you're concerned about, which involves
> > non-HOT tuples.
>
> Attached is a POC-quality revision of Masahiko's
On Sun, Mar 14, 2021 at 12:23 PM Peter Geoghegan wrote:
>
> On Fri, Mar 12, 2021 at 9:34 PM Masahiko Sawada wrote:
> > I agreed that when we're close to overrunning the
> > maintnenance_work_mem space, the situation changes. If we skip it in
> > even that case, the next vacuum will be likely to
On Mon, Mar 15, 2021 at 4:11 PM Andres Freund wrote:
> > > I'm not comfortable with this change without adding more safety
> > > checks. If there's ever a case in which the HEAPTUPLE_DEAD case is hit
> > > and the xid needs to be frozen, we'll either cause errors or
> > > corruption. Yes, that's
Hi,
On 2021-03-15 13:58:02 -0700, Peter Geoghegan wrote:
> On Mon, Mar 15, 2021 at 12:58 PM Peter Geoghegan wrote:
> > > I'm not comfortable with this change without adding more safety
> > > checks. If there's ever a case in which the HEAPTUPLE_DEAD case is hit
> > > and the xid needs to be
Hi,
On 2021-03-15 12:58:33 -0700, Peter Geoghegan wrote:
> On Mon, Mar 15, 2021 at 12:21 PM Andres Freund wrote:
> > It's evil sorcery. Fragile sorcery. I think Robert, Tom and me all run
> > afoul of edge cases around it in the last few years.
>
> Right, which is why I thought that I might be
On Mon, Mar 15, 2021 at 12:58 PM Peter Geoghegan wrote:
> > I'm not comfortable with this change without adding more safety
> > checks. If there's ever a case in which the HEAPTUPLE_DEAD case is hit
> > and the xid needs to be frozen, we'll either cause errors or
> > corruption. Yes, that's
On Mon, Mar 15, 2021 at 12:21 PM Andres Freund wrote:
> It's evil sorcery. Fragile sorcery. I think Robert, Tom and me all run
> afoul of edge cases around it in the last few years.
Right, which is why I thought that I might be missing something; why
put up with that at all for so long?
> > But
Hi,
On 2021-03-14 19:04:34 -0700, Peter Geoghegan wrote:
> Attached is a POC-quality revision of Masahiko's
> skip_index_vacuum.patch [1]. There is an improved design for skipping
> index vacuuming (improved over the INDEX_CLEANUP stuff from Postgres
> 12). I'm particularly interested in your
On Thu, Mar 11, 2021 at 8:31 AM Robert Haas wrote:
> But even if not, I'm not sure this
> helps much with the situation you're concerned about, which involves
> non-HOT tuples.
Attached is a POC-quality revision of Masahiko's
skip_index_vacuum.patch [1]. There is an improved design for skipping
On Sat, Mar 13, 2021 at 7:23 PM Peter Geoghegan wrote:
> In other words, I am not worried about debt, exactly. Debt is normal
> in moderation. Healthy, even. I am worried about bankruptcy, perhaps
> following a rare and extreme event. It's okay to be imprecise, but all
> of the problems must be
On Fri, Mar 12, 2021 at 9:34 PM Masahiko Sawada wrote:
> I agreed that when we're close to overrunning the
> maintnenance_work_mem space, the situation changes. If we skip it in
> even that case, the next vacuum will be likely to use up
> maintenance_work_mem, leading to a second index scan.
On Tue, Mar 9, 2021 at 2:22 PM Peter Geoghegan wrote:
>
> On Tue, Mar 2, 2021 at 8:49 PM Masahiko Sawada wrote:
> > On Tue, Mar 2, 2021 at 2:34 PM Peter Geoghegan wrote:
> > > lazy_vacuum_table_and_indexes() should probably not skip index
> > > vacuuming when we're close to exceeding the space
On Thu, Mar 11, 2021 at 8:31 AM Robert Haas wrote:
> I agree, but all you need is one long-lived tuple toward the end of
> the array and you're stuck never being able to truncate it. It seems
> like a worthwhile improvement, but whether it actually helps will be
> workload-dependant.
When it
On Thu, 11 Mar 2021 at 17:31, Robert Haas wrote:
>
> On Tue, Mar 9, 2021 at 3:35 PM Peter Geoghegan wrote:
> > Speaking of line pointer bloat (and "irreversible" bloat), I came
> > across something relevant today. I believe that this recent patch from
> > Matthias van de Meent is a relatively
On Tue, Mar 9, 2021 at 3:35 PM Peter Geoghegan wrote:
> Speaking of line pointer bloat (and "irreversible" bloat), I came
> across something relevant today. I believe that this recent patch from
> Matthias van de Meent is a relatively easy way to improve the
> situation:
>
>
On Wed, Mar 3, 2021 at 12:40 PM Peter Geoghegan wrote:
>
> On Tue, Mar 2, 2021 at 6:44 PM Masahiko Sawada wrote:
> > A scale type parameter seems good to me but I wonder if how users can
> > tune that parameter. We already have tuple-based parameters such as
> >
On Mon, Mar 8, 2021 at 7:34 PM Peter Geoghegan wrote:
> > One possible
> > consequence that I'm concerned about is sequential scan performance.
> > For an index scan, you just jump to the line pointer you want and then
> > go get the tuple, but a sequential scan has to loop over all the line
> >
On Tue, Mar 2, 2021 at 8:49 PM Masahiko Sawada wrote:
> On Tue, Mar 2, 2021 at 2:34 PM Peter Geoghegan wrote:
> > lazy_vacuum_table_and_indexes() should probably not skip index
> > vacuuming when we're close to exceeding the space allocated for the
> > LVDeadTuples array. Maybe we should not
On Mon, Mar 8, 2021 at 10:57 AM Robert Haas wrote:
> Yes, I agree that it's good to postpone this to a future release, and
> that thinking through the consequences is not so easy.
The current plan is to commit something like Masahiko's
skip_index_vacuum.patch for Postgres 14. The latest version
On Mon, Feb 1, 2021 at 10:17 PM Peter Geoghegan wrote:
> * No need to change MaxHeapTuplesPerPage for now, since that only
> really makes sense in cases that heavily involve bottom-up deletion,
> where we care about the *concentration* of LP_DEAD line pointers in
> heap pages (and not just the
On Tue, Mar 2, 2021 at 2:34 PM Peter Geoghegan wrote:
>
> On Mon, Mar 1, 2021 at 7:00 PM Peter Geoghegan wrote:
> > I think that you're right. However, in practice it isn't harmful
> > because has_dead_tuples is only used when "all_visible = true", and
> > only to detect corruption (which should
On Tue, Mar 2, 2021 at 6:44 PM Masahiko Sawada wrote:
> A scale type parameter seems good to me but I wonder if how users can
> tune that parameter. We already have tuple-based parameters such as
> autovacuum_vacuum_scale_factor/threshold and I think that users
> basically don't pay attention to
On Tue, Mar 2, 2021 at 12:00 PM Peter Geoghegan wrote:
>
> On Sun, Feb 21, 2021 at 10:28 PM Masahiko Sawada
> wrote:
> > Sorry for the late response.
>
> Me too!
No problem, thank you for your comment!
>
> > 1. Whereas skipping index vacuum and heap vacuum is a very attractive
> >
On Mon, Mar 1, 2021 at 7:00 PM Peter Geoghegan wrote:
> I think that you're right. However, in practice it isn't harmful
> because has_dead_tuples is only used when "all_visible = true", and
> only to detect corruption (which should never happen). I think that it
> should be fixed as part of this
On Sun, Feb 21, 2021 at 10:28 PM Masahiko Sawada wrote:
> Sorry for the late response.
Me too!
> 1. Whereas skipping index vacuum and heap vacuum is a very attractive
> improvement, if we skip that by default I wonder if we need a way to
> disable it. Vacuum plays a role in cleaning and
On Wed, Feb 10, 2021 at 4:12 PM Peter Geoghegan wrote:
>
> On Tue, Feb 9, 2021 at 6:14 PM Masahiko Sawada wrote:
> > Thanks. I think that's very good if we resolve this recycling stuff
> > first then try the new approach to skip index vacuum in more cases.
> > That way, even if the vacuum
On Tue, Feb 9, 2021 at 6:14 PM Masahiko Sawada wrote:
> Thanks. I think that's very good if we resolve this recycling stuff
> first then try the new approach to skip index vacuum in more cases.
> That way, even if the vacuum strategy stuff took a very long time to
> get committed over several
On Sat, Feb 6, 2021 at 5:02 AM Peter Geoghegan wrote:
>
> On Wed, Feb 3, 2021 at 8:18 PM Masahiko Sawada wrote:
> > > I'm starting to think that the right short term goal should not
> > > directly involve bottom-up index deletion. We should instead return to
> > > the idea of "unifying" the
On Wed, Feb 3, 2021 at 8:18 PM Masahiko Sawada wrote:
> > I'm starting to think that the right short term goal should not
> > directly involve bottom-up index deletion. We should instead return to
> > the idea of "unifying" the vacuum_cleanup_index_scale_factor feature
> > with the INDEX_CLEANUP
On Tue, Feb 2, 2021 at 12:17 PM Peter Geoghegan wrote:
>
> On Fri, Jan 29, 2021 at 5:26 PM Peter Geoghegan wrote:
> > It'll be essential to have good instrumentation as we do more
> > benchmarking. We're probably going to have to make subjective
> > assessments of benchmark results, based on
On Tue, Feb 2, 2021 at 6:28 AM Victor Yegorov wrote:
> I really like this idea!
Cool!
> It resembles the approach used in bottom-up index deletion, block-based
> accounting provides a better estimate for the usefulness of the operation.
It does resemble bottom-up index deletion, in one
вт, 2 февр. 2021 г. в 05:27, Peter Geoghegan :
> And now here is the second thing I thought of, which is much better:
>
> Sometimes 1% of the dead tuples in a heap relation will be spread
> across 90%+ of the pages. With other workloads 1% of dead tuples might
> be highly concentrated, and appear
On Mon, Feb 1, 2021 at 7:17 PM Peter Geoghegan wrote:
> Here is how the triggering criteria could work: maybe skipping
> accessing all indexes during VACUUM happens when less than 1% or
> 10,000 of the items from the table are to be removed by VACUUM --
> whichever is greater. Of course this is
On Fri, Jan 29, 2021 at 5:26 PM Peter Geoghegan wrote:
> It'll be essential to have good instrumentation as we do more
> benchmarking. We're probably going to have to make subjective
> assessments of benchmark results, based on multiple factors. That will
> probably be the only practical way to
On Tue, Jan 26, 2021 at 10:59 PM Masahiko Sawada wrote:
> What value is set to fillfactor?
90, same as before.
> That's very good. I'm happy that this patch efficiently utilizes
> bottom-up index deletion feature.
Me too!
> Looking at the relation size growth, there is almost no difference
>
On Fri, Jan 22, 2021 at 2:00 PM Peter Geoghegan wrote:
>
> On Tue, Jan 19, 2021 at 2:57 PM Peter Geoghegan wrote:
> > Looks good. I'll give this version a review now. I will do a lot more
> > soon. I need to come up with a good benchmark for this, that I can
> > return to again and again during
1 - 100 of 123 matches
Mail list logo
