Re: Add LSN <-> time conversion functionality

2024-08-09 Thread Melanie Plageman
On Fri, Aug 9, 2024 at 1:03 PM Tomas Vondra wrote: > > On 8/9/24 17:48, Melanie Plageman wrote: > > On Fri, Aug 9, 2024 at 9:15 AM Melanie Plageman > > wrote: > >> > >> On Fri, Aug 9, 2024 at 9:09 AM Tomas Vondra wrote: > >>> > >>

Re: Add LSN <-> time conversion functionality

2024-08-09 Thread Melanie Plageman
On Fri, Aug 9, 2024 at 9:15 AM Melanie Plageman wrote: > > On Fri, Aug 9, 2024 at 9:09 AM Tomas Vondra wrote: > > > > I suggest we do the simplest and most obvious algorithm possible, at > > least for now. Focusing on this part seems like a distraction from the > &g

Re: Add LSN <-> time conversion functionality

2024-08-09 Thread Melanie Plageman
On Fri, Aug 9, 2024 at 9:09 AM Tomas Vondra wrote: > > > > On 8/9/24 03:02, Melanie Plageman wrote: > > On Thu, Aug 8, 2024 at 2:34 PM Tomas Vondra wrote: > >> each seconds. And we want to allow merging stuff nicely. The smallest > >> merges we could do is 1

Re: Add LSN <-> time conversion functionality

2024-08-09 Thread Melanie Plageman
On Thu, Aug 8, 2024 at 9:02 PM Melanie Plageman wrote: > > On Thu, Aug 8, 2024 at 2:34 PM Tomas Vondra wrote: > > > > Maybe it'd be good to approach this from the opposite direction, say > > what "accuracy guarantees" we want to provide, and then design

Re: Add LSN <-> time conversion functionality

2024-08-08 Thread Melanie Plageman
On Thu, Aug 8, 2024 at 3:00 PM Robert Haas wrote: > > On Thu, Aug 8, 2024 at 2:34 PM Tomas Vondra wrote: > > A-D is already enough to cover 30h, with A-E it'd be ~300h. Do we need > > (or want) to keep a longer history? > > I think there is a difference of opinion about this between Melanie > and

Re: Add LSN <-> time conversion functionality

2024-08-08 Thread Melanie Plageman
On Thu, Aug 8, 2024 at 2:34 PM Tomas Vondra wrote: > > On 8/7/24 21:39, Melanie Plageman wrote: > > On Wed, Aug 7, 2024 at 1:06 PM Robert Haas wrote: > >> > >> As I mentioned to you off-list, I feel like this needs some sort of > >> recency bias.

Re: Add LSN <-> time conversion functionality

2024-08-07 Thread Melanie Plageman
On Wed, Aug 7, 2024 at 1:06 PM Robert Haas wrote: > > As I mentioned to you off-list, I feel like this needs some sort of > recency bias. Certainly vacuum, and really almost any conceivable user > of this facility, is going to care more about accurate answers for new > data than for old data. If t

Re: Add LSN <-> time conversion functionality

2024-08-07 Thread Melanie Plageman
M. Borodin wrote: > > > On 1 Aug 2024, at 05:44, Melanie Plageman wrote: > > > > On Sat, Jul 6, 2024 at 1:36 PM Andrey M. Borodin > > wrote: > >> > >> PgStartLSN = GetXLogInsertRecPtr(); > >> Should this be kind of RecoveryEndPtr? How i

Re: Draft release notes for next week's releases are up

2024-08-03 Thread Melanie Plageman
On Fri, Aug 2, 2024 at 4:26 PM Tom Lane wrote: > > See > > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=ee219ee8c40d88e7a0ef52c3c1b76c90bbd0d164 > > As usual, please send corrections/comments by Sunday. For the "Prevent infinite loop in VACUUM" item, I noticed two things 1)

Re: Add LSN <-> time conversion functionality

2024-07-31 Thread Melanie Plageman
a relic of my old design. Even without your point about cache locality, the code is much harder to understand with the backwards looping. I've changed the array to fill forwards and be accessed with forward loops. > > On 27 Jun 2024, at 07:18, Melanie Plageman > > wrote: > >

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-31 Thread Melanie Plageman
On Wed, Jul 31, 2024 at 4:38 PM Peter Geoghegan wrote: > > On Thu, Jun 20, 2024 at 7:42 PM Melanie Plageman > wrote: > > In back branches starting with 14, failing to remove tuples older than > > OldestXmin during pruning caused vacuum to infinitely loop in > > lazy_sc

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-26 Thread Melanie Plageman
On Wed, Jul 24, 2024 at 3:43 AM John Naylor wrote: > > On Wed, Jul 24, 2024 at 5:40 AM Masahiko Sawada wrote: > > > Without MEMORY_CONTEXT_CHECK, if size is 16 bytes, required_size is > > also 16 bytes as it's already 8-byte aligned and Bump_CHUNKHDRSZ is 0. > > On the other hand with MEMORY_CONT

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-26 Thread Melanie Plageman
On Wed, Jul 24, 2024 at 8:19 AM John Naylor wrote: > > On Wed, Jul 24, 2024 at 2:42 PM John Naylor wrote: > > As for lowering the limit, we've experimented with 256kB here: > > > > https://www.postgresql.org/message-id/canwcazzutvz3lsypauyqvzcezxz7qe+9ntnhgyzdtwxpul+...@mail.gmail.com > > > > As

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-26 Thread Melanie Plageman
On Mon, Jul 22, 2024 at 9:26 PM Masahiko Sawada wrote: > > + CREATE TABLE ${table1}(col1 int) > + WITH (autovacuum_enabled=false, fillfactor=10); > + INSERT INTO $table1 VALUES(7); > + INSERT INTO $table1 SELECT generate_series(1, $nrows) % 3; > + CREATE INDEX on ${table1}(col1); > +

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-23 Thread Melanie Plageman
On Mon, Jul 22, 2024 at 10:54 PM Masahiko Sawada wrote: > > On Mon, Jul 22, 2024 at 6:26 PM Melanie Plageman > wrote: > > > > On Mon, Jul 22, 2024 at 6:36 PM Tom Lane wrote: > > > > > > Melanie Plageman writes: > > > > We've only run

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Melanie Plageman
On Mon, Jul 22, 2024 at 6:36 PM Tom Lane wrote: > > Melanie Plageman writes: > > We've only run tests with this commit on some of the back branches for > > some of these animals. Of those, I don't see any failures so far. So, > > it seems the test instabilit

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Melanie Plageman
On Mon, Jul 22, 2024 at 2:17 PM Melanie Plageman wrote: > > So, I've just gone through all the test failures on master and 17 for > mamba, gull, mereswine, and copperhead. I wanted to confirm that the > test was always failing for the same reason and also if it had any > f

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Melanie Plageman
On Mon, Jul 22, 2024 at 1:17 PM Robert Haas wrote: > > On Mon, Jul 22, 2024 at 11:48 AM Tom Lane wrote: > > I'm a little suspicious > > of using it for tests that merely take an unreasonable amount of > > time --- to me, that indicates laziness on the part of the test > > author. > > Laziness wou

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Melanie Plageman
On Sun, Jul 21, 2024 at 4:29 PM Peter Geoghegan wrote: > > On Sun, Jul 21, 2024 at 12:51 PM Tom Lane wrote: > > I do not think the answer to this is to nag the respective animal > > owners to raise PG_TEST_TIMEOUT_DEFAULT. IMV this test is simply > > not worth the cycles it takes, at least not f

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Melanie Plageman
On Sun, Jul 21, 2024 at 5:04 PM Tom Lane wrote: > > Peter Geoghegan writes: > > On Sun, Jul 21, 2024 at 12:51 PM Tom Lane wrote: > >> I do not think the answer to this is to nag the respective animal > >> owners to raise PG_TEST_TIMEOUT_DEFAULT. IMV this test is simply > >> not worth the cycles

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-22 Thread Melanie Plageman
On Sun, Jul 21, 2024 at 12:51 PM Tom Lane wrote: > > Melanie Plageman writes: > > When I run it on my machine with some added logging, the space taken > > by dead items is about 330 kB more than maintenance_work_mem (which is > > set to 1 MB). I could roughly double the

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-17 Thread Melanie Plageman
On Wed, Jul 17, 2024 at 12:11 PM Peter Geoghegan wrote: > > On Wed, Jul 17, 2024 at 12:07 PM Melanie Plageman > wrote: > > We didn't end up doing two index vacuum passes. Because it doesn't > > repro locally for me, I can only assume that the conditions for > &g

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-17 Thread Melanie Plageman
On Wed, Jul 17, 2024 at 11:07 AM Melanie Plageman wrote: > > On Mon, Jul 15, 2024 at 6:02 PM Peter Geoghegan wrote: > > > > I think that there is some chance that this test will break the build > > farm in whatever way, since there is a long history of VACUUM not > &

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-17 Thread Melanie Plageman
On Mon, Jul 15, 2024 at 6:02 PM Peter Geoghegan wrote: > > On Mon, Jul 8, 2024 at 2:25 PM Melanie Plageman > wrote: > > I could still use another pair of eyes on the test (looking out for > > stability enhancing measures I could take). > > First, the basics: I

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-08 Thread Melanie Plageman
On Tue, Jul 2, 2024 at 7:07 PM Melanie Plageman wrote: > > On Mon, Jun 24, 2024 at 4:27 AM Heikki Linnakangas wrote: > > > > Would it be possible to make it robust so that we could always run it > > with "make check"? This seems like an important corner cas

Re: Confine vacuum skip logic to lazy_scan_skip

2024-07-08 Thread Melanie Plageman
On Sun, Jul 7, 2024 at 10:49 AM Noah Misch wrote: > > On Fri, Jun 28, 2024 at 05:36:25PM -0400, Melanie Plageman wrote: > > I've attached a WIP v11 streaming vacuum patch set here that is > > rebased over master (by Thomas), so that I could add a CF entry for > > it.

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-07-02 Thread Melanie Plageman
gger recovery conflicts. It would be super helpful if someone could take a look at the test and point out any ways I could make it even more likely to be stable. - Melanie From 7ecc180c4762c258dab01c7929c90a9c8e48c95b Mon Sep 17 00:00:00 2001 From: Melanie Plageman Date: Thu, 20 Jun 2024 17:21:11 -

Re: Add LSN <-> time conversion functionality

2024-06-28 Thread Melanie Plageman
On Wed, Jun 26, 2024 at 10:04 PM Melanie Plageman wrote: > > I've implemented these review points in the attached v4. I realized the docs had a compilation error. Attached v5 fixes that as well as three bugs I found while using this patch set more intensely today. I see Michael has b

Re: Confine vacuum skip logic to lazy_scan_skip

2024-06-28 Thread Melanie Plageman
On Sun, Mar 17, 2024 at 2:53 AM Thomas Munro wrote: > > On Tue, Mar 12, 2024 at 10:03 AM Melanie Plageman > wrote: > > I've rebased the attached v10 over top of the changes to > > lazy_scan_heap() Heikki just committed and over the v6 streaming read > > patch set

Re: Doc: fix track_io_timing description to mention pg_stat_io

2024-06-27 Thread Melanie Plageman
On Thu, Jun 27, 2024 at 5:06 AM wrote: > > Hi, > > pg_stat_io has I/O statistics that are collected when track_io_timing is > enabled, but it is not mentioned in the description of track_io_timing. > I think it's better to add a description of pg_stat_io for easy reference. > What do you think? S

Re: Add LSN <-> time conversion functionality

2024-06-26 Thread Melanie Plageman
Thanks so much Bharath, Andrey, and Ilya for the review! I've posted a new version here [1] which addresses some of your concerns. I'll comment on those it does not address inline. On Thu, May 30, 2024 at 1:26 PM Andrey M. Borodin wrote: > > === Questions === > 1. The patch does not handle serve

Re: Add LSN <-> time conversion functionality

2024-06-26 Thread Melanie Plageman
to address some feedback from another reviewer about Timeline being already in use in Postgres as a concept. On Mon, Mar 18, 2024 at 10:02 AM Daniel Gustafsson wrote: > > > On 22 Feb 2024, at 03:45, Melanie Plageman > > wrote: > > On Fri, Feb 16, 2024 at 3:41 PM Tomas Von

Re: Add LSN <-> time conversion functionality

2024-06-26 Thread Melanie Plageman
On Mon, Mar 18, 2024 at 1:29 PM Tomas Vondra wrote: > > On 2/22/24 03:45, Melanie Plageman wrote: > > The attached v3 has a new algorithm. Now, LSNTimes are added from the > > end of the array forward until all array elements have at least one > > logical member (array

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-25 Thread Melanie Plageman
On Tue, Jun 25, 2024 at 10:31 AM Robert Haas wrote: > > On Tue, Jun 25, 2024 at 9:07 AM Andres Freund wrote: > > It's not hard - but it has downsides. It'll mean that - outside of vacuum - > > we'll much more often not react to horizons going backwards due to > > hot_standby_feedback. Which means

Re: Backporting BackgroundPsql

2024-06-25 Thread Melanie Plageman
On Tue, Jun 25, 2024 at 7:40 AM Andres Freund wrote: > > Hi, > > On 2024-06-25 13:26:23 +0300, Heikki Linnakangas wrote: > > While fixing a recent bug on visibility on a standby [1], I wrote a > > regression test that uses BackgroundPsql to run some queries in a > > long-running psql session. The

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Mon, Jun 24, 2024 at 4:51 PM Peter Geoghegan wrote: > > On Mon, Jun 24, 2024 at 4:36 PM Robert Haas wrote: > > I thought the idea was that the GlobalVisTest stuff would force a > > recalculation now and then, but maybe it doesn't actually do that? > > It definitely can do that. Just not in a w

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Mon, Jun 24, 2024 at 4:42 PM Peter Geoghegan wrote: > > On Mon, Jun 24, 2024 at 3:36 PM Melanie Plageman > wrote: > > One thing I don't understand is why it is okay to freeze the xmax of a > > dead tuple just because it is from an aborted update. > > We do

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Thu, Jun 20, 2024 at 7:42 PM Melanie Plageman wrote: > > If vacuum fails to remove a tuple with xmax older than > VacuumCutoffs->OldestXmin and younger than > GlobalVisState->maybe_needed, it will ERROR out when determining > whether or not to freeze the tuple with &qu

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Mon, Jun 24, 2024 at 1:05 PM Robert Haas wrote: > > On Mon, Jun 24, 2024 at 12:43 PM Peter Geoghegan wrote: > > The problem here is that OldestXmin is supposed to be more > > conservative than vistest, which it almost always is, except in this > > one edge case. I don't think that plugging tha

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Mon, Jun 24, 2024 at 4:27 AM Heikki Linnakangas wrote: > > On 21/06/2024 03:02, Peter Geoghegan wrote: > > On Thu, Jun 20, 2024 at 7:42 PM Melanie Plageman > > wrote: > > > >> The repro forces a round of index vacuuming after the standby > >> reconn

Re: Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-24 Thread Melanie Plageman
On Mon, Jun 24, 2024 at 4:10 AM Alena Rybakina wrote: > > We can fix this by always removing tuples considered dead before > VacuumCutoffs->OldestXmin. This is okay even if a reconnected standby > has a transaction that sees that tuple as alive, because it will > simply wait to replay the removal

Re: FreezeLimit underflows in pg14 and 15 causing incorrect behavior in heap_prepare_freeze_tuple

2024-06-22 Thread Melanie Plageman
On Sat, Jun 22, 2024 at 10:53 AM Peter Geoghegan wrote: > > On Sat, Jun 22, 2024 at 10:43 AM Melanie Plageman > wrote: > > Hmm. So perhaps this subtraction results in the desired behavior for > > freeze limit -- but by using FreezeLimit as the cutoff_xid for > > heap_

Re: FreezeLimit underflows in pg14 and 15 causing incorrect behavior in heap_prepare_freeze_tuple

2024-06-22 Thread Melanie Plageman
On Fri, Jun 21, 2024 at 4:22 PM Melanie Plageman wrote: > > While investigating a bug over on [1], I found that > vacuum_set_xid_limits() is calculating freezeLimit in an unsafe way on > at least Postgres 14 and 15. > > limit = *oldestXmin - freezemin; > safeLimit =

FreezeLimit underflows in pg14 and 15 causing incorrect behavior in heap_prepare_freeze_tuple

2024-06-21 Thread Melanie Plageman
While investigating a bug over on [1], I found that vacuum_set_xid_limits() is calculating freezeLimit in an unsafe way on at least Postgres 14 and 15. limit = *oldestXmin - freezemin; safeLimit = ReadNextTransactionId() - autovacuum_freeze_max_age; if (TransactionIdPrecedes(limit, saf

Vacuum ERRORs out considering freezing dead tuples from before OldestXmin

2024-06-20 Thread Melanie Plageman
x27;t work on 14 and 15 because of changes to background_psql. - Melanie [1] https://postgr.es/m/20240415173913.4zyyrwaftujxthf2%40awork3.anarazel.de#1b216b7768b5bd577a3d3d51bd5aadee [2] https://www.postgresql.org/message-id/CAAKRu_Y_NJzF4-8gzTTeaOuUL3CcGoXPjXcAHbTTygT8AyVqag%40mail.gmail.com From fb74

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-06-19 Thread Melanie Plageman
Thanks for taking a look at my patches, Álvaro! On Wed, Jun 19, 2024 at 12:51 PM Alvaro Herrera wrote: > > On 2024-Jun-14, Melanie Plageman wrote: > > > Subject: [PATCH v21 12/20] Update variable names in bitmap scan descriptors > > > > The previous commit which ad

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-06-19 Thread Melanie Plageman
On Wed, Jun 19, 2024 at 12:38 PM Tomas Vondra wrote: > > > > On 6/19/24 17:55, Melanie Plageman wrote: > > On Tue, Jun 18, 2024 at 6:02 PM Tomas Vondra > > wrote: > > > From your v22b-0017-review.patch > > > > diff --git a/src/include/access/relsc

Re: Separate HEAP WAL replay logic into its own file

2024-06-18 Thread Melanie Plageman
On Mon, Jun 17, 2024 at 9:12 PM Li, Yong wrote: > > As a newcomer, when I was walking through the code looking for WAL replay > related code, it was relatively easy for me to find them for the B-Tree > access method because of the “xlog” hint in the file names. It took me a > while to find the

Re: Separate HEAP WAL replay logic into its own file

2024-06-17 Thread Melanie Plageman
On Mon, Jun 17, 2024 at 2:20 AM Li, Yong wrote: > > Hi PostgreSQL hackers, > > For most access methods in PostgreSQL, the implementation of the access > method itself and the implementation of its WAL replay logic are organized in > separate source files. However, the HEAP access method is an e

Re: Show WAL write and fsync stats in pg_stat_io

2024-06-17 Thread Melanie Plageman
On Thu, Jun 13, 2024 at 5:24 AM Nazir Bilal Yavuz wrote: > > On Sun, 9 Jun 2024 at 18:05, Nitin Jadhav > wrote: > > > > > If possible, let's have all the I/O stats (even for WAL) in > > > pg_stat_io. Can't we show the WAL data we get from buffers in the hits > > > column and then have read_bytes

Re: RFC: adding pytest as a supported test framework

2024-06-15 Thread Melanie Plageman
On Sat, Jun 15, 2024 at 5:53 PM Greg Sabino Mullane wrote: > > On Sat, Jun 15, 2024 at 12:48 PM Jelte Fennema-Nio wrote: >> >> Afaict, there's a significant part of our current community who feel the >> same way (and I'm pretty sure every sub-30 year old person who >> newly joins the community w

Re: RFC: adding pytest as a supported test framework

2024-06-13 Thread Melanie Plageman
On Thu, Jun 13, 2024 at 7:27 AM Daniel Gustafsson wrote: > > > On 13 Jun 2024, at 00:34, Melanie Plageman > > wrote: > > > FWIW, I felt a lot of pain trying to write recovery TAP tests with > > IPC::Run's pumping functionality. It was especially painful (as &

Re: RFC: adding pytest as a supported test framework

2024-06-12 Thread Melanie Plageman
On Tue, Jun 11, 2024 at 8:05 AM Andrew Dunstan wrote: > > > On 2024-06-10 Mo 21:49, Andres Freund wrote: > > Hi, > > On 2024-06-10 16:46:56 -0400, Andrew Dunstan wrote: > > I'm not sure what part of the testing infrastructure you think is > unmaintained. For example, the last release of Test::Simp

Re: First draft of PG 17 release notes

2024-05-21 Thread Melanie Plageman
On Tue, May 21, 2024 at 1:51 PM Robert Haas wrote: > > On Tue, May 21, 2024 at 12:27 PM Andres Freund wrote: > > To me that's the "General Performance" section. If somebody reading the > > release notes doesn't care about performance, they can just skip that > > section > > ([1]). I don't see w

Re: Streaming read-ready sequential scan code

2024-05-20 Thread Melanie Plageman
Thank you to all of you for looking into this. On Sat, May 18, 2024 at 12:47 AM Thomas Munro wrote: > > On Sat, May 18, 2024 at 11:30 AM Thomas Munro wrote: > > Andres happened to have TPC-DS handy, and reproduced that regression > > in q15. We tried some stuff and figured out that it requires

Re: Use streaming read API in ANALYZE

2024-05-20 Thread Melanie Plageman
On Wed, May 15, 2024 at 2:18 PM Nazir Bilal Yavuz wrote: > > On Mon, 29 Apr 2024 at 18:41, Nazir Bilal Yavuz wrote: > > > > On Mon, 8 Apr 2024 at 04:21, Thomas Munro wrote: > > I wanted to discuss what will happen to this patch now that > > 27bc1772fc8 is reverted. I am continuing this thread bu

Re: First draft of PG 17 release notes

2024-05-20 Thread Melanie Plageman
d as a co-author on all of the commits that were part of this feature, but he made a substantial investment in it and should be listed, IMO. Patch attached. - Melanie From 6f8c775a45ef90a3882d3d2f3c484fc82f9c072a Mon Sep 17 00:00:00 2001 From: Melanie Plageman Date: Mon, 20 May 2024 14:44:41 -0400

Re: First draft of PG 17 release notes

2024-05-20 Thread Melanie Plageman
On Sat, May 18, 2024 at 11:13 AM Bruce Momjian wrote: > > On Thu, May 16, 2024 at 09:09:11AM -0400, Melanie Plageman wrote: > > On Wed, May 15, 2024 at 11:48 PM Andres Freund wrote: > > > > > > On 2024-05-15 10:38:20 +0200, Alvaro Herrera wrote: > > > >

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-17 Thread Melanie Plageman
On Fri, May 17, 2024 at 7:11 AM Tomas Vondra wrote: > > On 5/16/24 23:43, Peter Eisentraut wrote: > > On 16.05.24 23:06, Joe Conway wrote: > >> On 5/16/24 16:57, Jacob Champion wrote: > >>> On Thu, May 16, 2024 at 1:31 PM Joe Conway wrote: > Maybe we should just make it a policy that *nothin

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-16 Thread Melanie Plageman
On Thu, May 16, 2024 at 5:04 PM Tom Lane wrote: > > Jacob Champion writes: > > ... Similar to how people currently use the > > Reviewer field as a personal TODO list... it might be nice to > > officially separate the ideas a bit. > > Oh, that's an independent pet peeve of mine. Usually, if I'm >

Re: commitfest.postgresql.org is no longer fit for purpose

2024-05-16 Thread Melanie Plageman
On Thu, May 16, 2024 at 2:30 PM Robert Haas wrote: > > Hi, > > The original intent of CommitFests, and of commitfest.postgresql.org > by extension, was to provide a place where patches could be registered > to indicate that they needed to be reviewed, thus enabling patch > authors and patch review

Re: race condition when writing pg_control

2024-05-16 Thread Melanie Plageman
On Sun, Jun 7, 2020 at 10:49 PM Thomas Munro wrote: > > On Wed, Jun 3, 2020 at 2:03 PM Michael Paquier wrote: > > On Wed, Jun 03, 2020 at 10:56:13AM +1200, Thomas Munro wrote: > > > Sorry for my radio silence, I got tangled up with a couple of > > > conferences. I'm planning to look at 0001 and

Re: First draft of PG 17 release notes

2024-05-16 Thread Melanie Plageman
On Wed, May 15, 2024 at 11:48 PM Andres Freund wrote: > > On 2024-05-15 10:38:20 +0200, Alvaro Herrera wrote: > > I disagree with this. IMO the impact of the Sawada/Naylor change is > > likely to be enormous for people with large tables and large numbers of > > tuples to clean up (I know we've ha

Re: First draft of PG 17 release notes

2024-05-15 Thread Melanie Plageman
On Wed, May 15, 2024 at 4:38 AM Alvaro Herrera wrote: > > On 2024-May-14, Bruce Momjian wrote: > > > On Tue, May 14, 2024 at 03:39:26PM -0400, Melanie Plageman wrote: > > > > I think that we need to more clearly point out the implications of the > > > feature

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-05-14 Thread Melanie Plageman
sert in the future. It is not protection against someone deleting the line scan->rs_empty_tuples_pending = 0 That is, it doesn't verify that the empty unused tuples count is discarded. Do you think that is necessary? - Melanie From 031012be9d77d5f5888b306d546322e6176e527e Mon Sep 17 00:

Re: First draft of PG 17 release notes

2024-05-14 Thread Melanie Plageman
On Thu, May 9, 2024 at 12:04 AM Bruce Momjian wrote: > > I have committed the first draft of the PG 17 release notes; you can > see the results here: > > https://momjian.us/pgsql_docs/release-17.html I had two comments: I think the read stream item: "Allow the grouping of file

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-05-14 Thread Melanie Plageman
On Tue, May 14, 2024 at 2:44 PM Tomas Vondra wrote: > > On 5/14/24 20:40, Melanie Plageman wrote: > > On Tue, May 14, 2024 at 2:33 PM Tomas Vondra > > wrote: > >> > >> On 5/14/24 19:42, Melanie Plageman wrote: > >>> I've fixed this. I've

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-05-14 Thread Melanie Plageman
On Tue, May 14, 2024 at 2:33 PM Tomas Vondra wrote: > > On 5/14/24 19:42, Melanie Plageman wrote: > > I've fixed this. I've also set enable_material off as I mentioned I > > might in my earlier mail. > > > > I'm not sure this (setting more and more GU

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-05-14 Thread Melanie Plageman
On Tue, May 14, 2024 at 2:18 AM Michael Paquier wrote: > > On Mon, May 13, 2024 at 10:05:03AM -0400, Melanie Plageman wrote: > > Remove the assert and reset the field on which it previously asserted to > > avoid incorrectly emitting NULL-filled tuples from a previous

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-05-13 Thread Melanie Plageman
On Sat, May 11, 2024 at 3:18 PM Tomas Vondra wrote: > > On 5/10/24 21:48, Melanie Plageman wrote: > > Attached is v3. I didn't use your exact language because the test > > wouldn't actually verify that we properly discard the tuples. Whether > > or not the emp

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-05-10 Thread Melanie Plageman
On Thu, May 2, 2024 at 5:37 PM Tomas Vondra wrote: > > > > On 4/24/24 22:46, Melanie Plageman wrote: > > On Tue, Apr 23, 2024 at 6:43 PM Tomas Vondra > > wrote: > >> > >> On 4/23/24 18:05, Melanie Plageman wrote: > >>> The patch with a fix

Re: open items

2024-05-10 Thread Melanie Plageman
On Thu, May 9, 2024 at 3:28 PM Robert Haas wrote: > > Just a few reminders about the open items list at > https://wiki.postgresql.org/wiki/PostgreSQL_17_Open_Items -- > > * Incorrect Assert in heap_end/rescan for BHS. Either the description > of this item is inaccurate, or we've been unable to fix

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-26 Thread Melanie Plageman
On Thu, Apr 25, 2024 at 7:57 PM Tom Lane wrote: > > I wrote: > > Hmm, is that actually true? There's no more reason to think a tuple > > in a temp table is old enough to be visible to all other sessions > > than one in any other table. It could be all right if we had a > > special-case rule for

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-25 Thread Melanie Plageman
On Wed, Apr 24, 2024 at 4:46 PM Melanie Plageman wrote: > > On Tue, Apr 23, 2024 at 6:43 PM Tomas Vondra > wrote: > > > > On 4/23/24 18:05, Melanie Plageman wrote: > > > One other note: there is some concurrency effect in the parallel > > > schedule group

Re: New GUC autovacuum_max_threshold ?

2024-04-25 Thread Melanie Plageman
On Thu, Apr 25, 2024 at 2:52 AM Frédéric Yhuel wrote: > > Le 24/04/2024 à 21:10, Melanie Plageman a écrit : > > On Wed, Apr 24, 2024 at 8:08 AM Frédéric Yhuel > > wrote: > >> > >> Hello, > >> > >> I would like to suggest a new parameter, a

Re: broken reading on standby (PostgreSQL 16.2)

2024-04-25 Thread Melanie Plageman
On Thu, Apr 25, 2024 at 2:13 AM Pavel Stehule wrote: > > Hi > > yesterday, I had to fix strange issue on standby server > > The query to freshly updated data fails > > select * from seller_success_rate where create_time::date = '2024-04-23'; > ERROR: 58P01: could not access status of transaction

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-24 Thread Melanie Plageman
On Tue, Apr 23, 2024 at 6:43 PM Tomas Vondra wrote: > > On 4/23/24 18:05, Melanie Plageman wrote: > > The patch with a fix is attached. I put the test in > > src/test/regress/sql/join.sql. It isn't the perfect location because > > it is testing something exercisable

Re: New GUC autovacuum_max_threshold ?

2024-04-24 Thread Melanie Plageman
On Wed, Apr 24, 2024 at 8:08 AM Frédéric Yhuel wrote: > > Hello, > > I would like to suggest a new parameter, autovacuum_max_threshold, which > would set an upper limit on the number of tuples to delete/update/insert > prior to vacuum/analyze. Hi Frédéric, thanks for the proposal! You are tacklin

Re: soliciting patches to review

2024-04-23 Thread Melanie Plageman
On Tue, Apr 23, 2024 at 1:27 PM Robert Haas wrote: > > Hi, > > Just a quick update. We have so far had 8 suggested patches from 6 > people, if I haven't missed anything. I'm fairly certain that not all > of those patches are going to be good candidates for this session, so > it would be great if a

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-23 Thread Melanie Plageman
On Mon, Apr 22, 2024 at 1:01 PM Melanie Plageman wrote: > > On Thu, Apr 18, 2024 at 5:39 AM Tomas Vondra > wrote: > > > > On 4/18/24 09:10, Michael Paquier wrote: > > > On Sun, Apr 07, 2024 at 10:54:56AM -0400, Melanie Plageman wrote: > > >> I've

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-22 Thread Melanie Plageman
On Thu, Apr 18, 2024 at 5:39 AM Tomas Vondra wrote: > > On 4/18/24 09:10, Michael Paquier wrote: > > On Sun, Apr 07, 2024 at 10:54:56AM -0400, Melanie Plageman wrote: > >> I've added an open item [1], because what's one open item when you can > >> have two

Re: Table AM Interface Enhancements

2024-04-12 Thread Melanie Plageman
On Thu, Apr 11, 2024 at 6:04 PM Alexander Korotkov wrote: > > On Fri, Apr 12, 2024 at 12:04 AM Andres Freund wrote: > > On 2024-04-11 20:46:02 +0300, Alexander Korotkov wrote: > > > I hope this work is targeting pg18. > > > > I think anything of the scope discussed by Melanie would be very clearl

Re: Table AM Interface Enhancements

2024-04-11 Thread Melanie Plageman
On Thu, Apr 11, 2024 at 12:19 PM Melanie Plageman wrote: > > On Wed, Apr 10, 2024 at 5:21 PM Andres Freund wrote: > > > > Hi, > > > > On 2024-04-10 16:50:44 -0400, Melanie Plageman wrote: > > > This brings up a question about the prefetching. We neve

Re: Table AM Interface Enhancements

2024-04-11 Thread Melanie Plageman
On Wed, Apr 10, 2024 at 5:21 PM Andres Freund wrote: > > Hi, > > On 2024-04-10 16:50:44 -0400, Melanie Plageman wrote: > > This brings up a question about the prefetching. We never had to have > > this discussion for sequential scan streaming read because it didn'

Re: Table AM Interface Enhancements

2024-04-10 Thread Melanie Plageman
On Wed, Apr 10, 2024 at 4:33 PM Andres Freund wrote: > > Hi, > > On 2024-04-10 16:24:40 -0400, Melanie Plageman wrote: > > This thread has been moving pretty fast, so could someone point out > > which version of the patch has the modifications to > > acquire_sample

Re: Table AM Interface Enhancements

2024-04-10 Thread Melanie Plageman
On Wed, Apr 10, 2024 at 4:03 PM Andres Freund wrote: > > Hi, > > On 2024-04-10 15:19:47 +0300, Alexander Korotkov wrote: > > On Mon, Apr 8, 2024 at 9:54 PM Robert Haas wrote: > > > On Mon, Apr 8, 2024 at 12:33 PM Alexander Korotkov > > > wrote: > > > > Yes, it was my mistake. I got rushing tryi

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Melanie Plageman
On Mon, Apr 8, 2024 at 10:41 AM Robert Treat wrote: > > On Mon, Apr 8, 2024 at 10:27 AM Melanie Plageman > wrote: > > On Mon, Apr 8, 2024 at 9:26 AM Robert Haas wrote: > > > On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier > > > wrote: > > > > And,

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Melanie Plageman
On Mon, Apr 8, 2024 at 9:26 AM Robert Haas wrote: > > On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier wrote: > > And, as of the moment of typing this email, I get: > > =# select '2024-04-08 00:00-12:00' - now() as time_remaining; > > time_remaining > > - > > 13:10:35.688134 > > (

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-07 Thread Melanie Plageman
On Sat, Apr 6, 2024 at 7:08 PM Thomas Munro wrote: > > On second thoughts, I think the original "invalidate" terminology was > fine, no need to invent a new term. > > I thought of a better name for the bufmgr.c function though: > InvalidateUnpinnedBuffer(). That name seemed better to me after I >

Re: Use streaming read API in ANALYZE

2024-04-07 Thread Melanie Plageman
On Sun, Apr 07, 2024 at 03:00:00PM -0700, Andres Freund wrote: > Hi, > > On 2024-04-07 16:59:26 -0400, Melanie Plageman wrote: > > From 1dc2343661f3edb3b1bc4307afb0e956397eb76c Mon Sep 17 00:00:00 2001 > > From: Melanie Plageman > > Date: Sun, 7 Apr 2024 14:55:22 -040

Re: Use streaming read API in ANALYZE

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 3:57 PM Melanie Plageman wrote: > > On Thu, Apr 04, 2024 at 02:03:30PM +0300, Nazir Bilal Yavuz wrote: > > > > On Wed, 3 Apr 2024 at 23:44, Melanie Plageman > > wrote: > > > > > > I don't see the point of BlockSampler_HasMor

Re: Use streaming read API in ANALYZE

2024-04-07 Thread Melanie Plageman
On Thu, Apr 04, 2024 at 02:03:30PM +0300, Nazir Bilal Yavuz wrote: > > On Wed, 3 Apr 2024 at 23:44, Melanie Plageman > wrote: > > > > I don't see the point of BlockSampler_HasMore() anymore. I removed it in > > the attached and made BlockSampler_Next() return

Re: Streaming I/O, vectored I/O (WIP)

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 1:33 PM Nazir Bilal Yavuz wrote: > > Hi, > > On Tue, 2 Apr 2024 at 11:40, Thomas Munro wrote: > > > > I had been planning to commit v14 this morning but got cold feet with > > the BMR-based interface. Heikki didn't like it much, and in the end, > > neither did I. I have n

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 10:42 AM Tomas Vondra wrote: > > > > On 4/7/24 16:24, Melanie Plageman wrote: > >>> Thanks! I thought about it a bit more, and I got worried about the > >>> > >>> Assert(scan->rs_empty_tuples_pending =

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 10:10 AM Tomas Vondra wrote: > > On 4/7/24 15:11, Melanie Plageman wrote: > > > Also, the iterators in the TableScanDescData might be something I > > could live with in the source code for a couple months before we make > > the rest of the changes

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 7:38 AM Tomas Vondra wrote: > > > > On 4/7/24 06:17, Melanie Plageman wrote: > > On Sun, Apr 07, 2024 at 02:27:43AM +0200, Tomas Vondra wrote: > >> On 4/6/24 23:34, Melanie Plageman wrote: > >>> ... > >>>> > >&

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 7:38 AM Tomas Vondra wrote: > > On 4/7/24 06:17, Melanie Plageman wrote: > >> What bothers me on 0006-0008 is that the justification in the commit > >> messages is "future commit will do something". I think it's fine to have > >

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-06 Thread Melanie Plageman
On Sun, Apr 07, 2024 at 02:27:43AM +0200, Tomas Vondra wrote: > On 4/6/24 23:34, Melanie Plageman wrote: > > ... > >> > >> I realized it makes more sense to add a FIXME (I used XXX. I'm not when > >> to use what) with a link to the message where Andres des

Re: Streaming read-ready sequential scan code

2024-04-06 Thread Melanie Plageman
ffer() for each block, heap sequential scans and TID range scans now use the streaming read API introduced in b5a9b18cd0. Author: Melanie Plageman Reviewed-by: Andres Freund Discussion: https://postgr.es/m/flat/CAAKRu_YtXJiYKQvb5JsA2SkwrsizYLugs4sSOZh3EAjKUg%3DgEQ%40

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-06 Thread Melanie Plageman
On Sat, Apr 06, 2024 at 04:57:51PM +0200, Tomas Vondra wrote: > On 4/6/24 15:40, Melanie Plageman wrote: > > On Sat, Apr 06, 2024 at 02:51:45AM +0200, Tomas Vondra wrote: > >> > >> > >> On 4/6/24 01:53, Melanie Plageman wrote: > >>> On Fri, Apr 05,

  1   2   3   4   5   6   7   >