Re: Thoughts on "killed tuples" index hint bits support on standby

2021-02-10 Thread Michail Nikolaev
Hello, Peter. If you are interested, the possible patch (based on FPI mask during replay) was sent with some additional explanation and graphics to (1). At the moment I unable to find any "incorrectness" in it. Thanks again for your comments. Michail. [1]

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-02-02 Thread Michail Nikolaev
Hello, Peter. > AFAICT that's not true, at least not in any practical sense. See the > comment in the middle of MarkBufferDirtyHint() that begins with "If we > must not write WAL, due to a relfilenode-specific...", and see the > "Checksums" section at the end of src/backend/storage/page/README.

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-02-01 Thread Peter Geoghegan
On Mon, Feb 1, 2021 at 1:19 PM Michail Nikolaev wrote: > It is fine to receive a page to the standby from any source: `btpo_flags` > should have some kind “LP_DEAD safe for standby” bit set to allow new bits to > be set and old - read. > > > We can't really mask LP_DEAD bits from > > the

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-02-01 Thread Michail Nikolaev
Hello, Peter. Thanks a lot for your comments. There are some mine thoughts, related to the “masked bits” solution and your comments: > During recovery, we will probably always have to consider the > possibility that LP_DEAD bits that get set on the primary may be > received by a replica through

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-01-31 Thread Peter Geoghegan
On Sat, Jan 30, 2021 at 5:39 PM Peter Geoghegan wrote: > If you invent some entirely new category of standby-only hint bit at a > level below the access method code, then you can use it inside access > method code such as nbtree. Maybe you don't have to play games with > minRecoveryPoint in code

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-01-30 Thread Peter Geoghegan
On Sat, Jan 30, 2021 at 9:11 AM Michail Nikolaev wrote: > > Yeah, it would help a lot. But those bits are precious. So it makes > > sense to think about what to do with both of them in index AMs at the > > same time. Otherwise we risk missing some important opportunity. > > Hm. I was trying to

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-01-30 Thread Michail Nikolaev
Hello, Peter. > Yeah, it would help a lot. But those bits are precious. So it makes > sense to think about what to do with both of them in index AMs at the > same time. Otherwise we risk missing some important opportunity. Hm. I was trying to "expand the scope" as you said and got an idea...

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-01-29 Thread Peter Geoghegan
On Thu, Jan 28, 2021 at 10:16 AM Michail Nikolaev wrote: > > I wonder if it would help to not actually use the LP_DEAD bit for > > this. Instead, you could use the currently-unused-in-indexes > > LP_REDIRECT bit. > > Hm… Sound very promising - an additional bit is a lot in this situation. Yeah,

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-01-28 Thread Michail Nikolaev
Hello, Peter. > I wonder if it would help to not actually use the LP_DEAD bit for > this. Instead, you could use the currently-unused-in-indexes > LP_REDIRECT bit. Hm… Sound very promising - an additional bit is a lot in this situation. > Whether or not "recently dead" means "dead to my >

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-01-27 Thread Peter Geoghegan
On Wed, Jan 27, 2021 at 5:24 PM Peter Geoghegan wrote: > The issue here is that it would also be nice to use a "recently dead" > bit on the primary, but if you do that then maybe you're back to the > original problem. OTOH, I also think that we could repurpose the > LP_NORMAL bit in index AMs, so

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-01-27 Thread Peter Geoghegan
On Wed, Jan 27, 2021 at 11:30 AM Michail Nikolaev wrote: > Sorry for necroposting, but if someone is interested - I hope the patch is > ready now and available in the other thread (1). I wonder if it would help to not actually use the LP_DEAD bit for this. Instead, you could use the

Re: Thoughts on "killed tuples" index hint bits support on standby

2021-01-27 Thread Michail Nikolaev
Hello, hackers. Sorry for necroposting, but if someone is interested - I hope the patch is ready now and available in the other thread (1). Thanks, Michail. [1] https://www.postgresql.org/message-id/flat/CANtu0oiP18H31dSaEzn0B0rW6tA_q1G7%3D9Y92%2BUS_WHGOoQevg%40mail.gmail.com

Re: Thoughts on "killed tuples" index hint bits support on standby

2020-04-09 Thread Michail Nikolaev
Hello, Peter. > Let me make sure I understand your position: > You're particularly concerned about cases where there are relatively > few page splits, and the standby has to wait for VACUUM to run on the > primary before dead index tuples get cleaned up. The primary itself > probably has no

Re: Thoughts on "killed tuples" index hint bits support on standby

2020-04-08 Thread Peter Geoghegan
On Wed, Apr 8, 2020 at 5:23 AM Michail Nikolaev wrote: > Yes, it is a brilliant idea to use uniqueness to avoid bloating the index. I > am > not able to understand all the code now, but I’ll check the patch in more > detail later. The design is probably simpler than you imagine. The algorithm

Re: Thoughts on "killed tuples" index hint bits support on standby

2020-04-08 Thread Michail Nikolaev
Hello, Peter. Thanks for your feedback. > Attached is a very rough POC patch of my own, which makes item > deletion occur "non-opportunistically" in unique indexes. The idea is > that we exploit the uniqueness property of unique indexes to identify > "version churn" from non-HOT updates. If any

Re: Thoughts on "killed tuples" index hint bits support on standby

2020-04-05 Thread Peter Geoghegan
Hi Michail, On Fri, Jan 24, 2020 at 6:16 AM Michail Nikolaev wrote: > Some of issues your mentioned (reporting feedback to the another > cascade standby, processing queries after restart and newer xid > already reported) could be fixed in provided design, but your > intention to have

Re: Thoughts on "killed tuples" index hint bits support on standby

2020-01-24 Thread Michail Nikolaev
Hello again. Andres, Peter, thanks for your comments. Some of issues your mentioned (reporting feedback to the another cascade standby, processing queries after restart and newer xid already reported) could be fixed in provided design, but your intention to have "independent correctness

Re: Thoughts on "killed tuples" index hint bits support on standby

2020-01-16 Thread Peter Geoghegan
On Thu, Jan 16, 2020 at 9:54 AM Andres Freund wrote: > I don't think we can rely on hot_standby_feedback at all. We can to > avoid unnecessary cancellations, etc, and even assume it's setup up > reasonably for some configurations, but there always needs to be an > independent correctness

Re: Thoughts on "killed tuples" index hint bits support on standby

2020-01-16 Thread Andres Freund
Hi, On 2020-01-16 14:30:12 +0300, Michail Nikolaev wrote: > First thing we need to consider - checksums and wal_log_hints are > widely used these days. So, at any moment master could send FPW page > with new "killed tuples" hints and overwrite hints set by standby. > Moreover it is not possible