Hi,
On Thu, Nov 25, 2021 at 10:00 PM Antonin Houska wrote:
>
> So it should be ok if the temporary undo is managed and discarded by
> individual backends. Patch 0005 of the new series tries to do that.
The cfbot reports that at least the 001 patch doesn't apply anymore:
On Tue, Sep 28, 2021 at 7:36 PM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
> > On Mon, Sep 20, 2021 at 10:55 AM Antonin Houska wrote:
> > >
> > > Amit Kapila wrote:
> > >
> > > > On Thu, Sep 9, 2021 at 8:33 PM Antonin Houska wrote:
> > >
> > > > > The problem of the temporary undo log is
Thomas Munro wrote:
> On Wed, Sep 29, 2021 at 8:18 AM Antonin Houska wrote:
> > I'm just trying to use the existing infrastructure: the effect of DROP TABLE
> > also appear to be performed by the checkpointer. However I don't know why
> > the
> > unlinks need to be performed by the
On Wed, Sep 29, 2021 at 8:18 AM Antonin Houska wrote:
> I'm just trying to use the existing infrastructure: the effect of DROP TABLE
> also appear to be performed by the checkpointer. However I don't know why the
> unlinks need to be performed by the checkpointer.
For DROP TABLE, we leave an
Dmitry Dolgov <9erthali...@gmail.com> wrote:
> > On Tue, Sep 21, 2021 at 10:07:55AM +0200, Dmitry Dolgov wrote:
> > On Tue, 21 Sep 2021 09:00 Antonin Houska, wrote:
> >
> > > Dmitry Dolgov <9erthali...@gmail.com> wrote:
> > >
> > > > Yep, makes sense, thanks. I have few more questions:
> > > >
>
> On Tue, Sep 21, 2021 at 10:07:55AM +0200, Dmitry Dolgov wrote:
> On Tue, 21 Sep 2021 09:00 Antonin Houska, wrote:
>
> > Dmitry Dolgov <9erthali...@gmail.com> wrote:
> >
> > > Yep, makes sense, thanks. I have few more questions:
> > >
> > > * The use case with orphaned files is working somewhat
Amit Kapila wrote:
> On Mon, Sep 20, 2021 at 10:55 AM Antonin Houska wrote:
> >
> > Amit Kapila wrote:
> >
> > > On Thu, Sep 9, 2021 at 8:33 PM Antonin Houska wrote:
> >
> > > > The problem of the temporary undo log is that it's loaded into local
> > > > buffers
> > > > and that backend can
On Mon, Sep 27, 2021 at 7:43 PM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
> Although the postgres core probably does not raise FATAL errors too often (OOM
> conditions seem to be the typical cause), I'm still not enthusiastic about
> idea that the undo feature turns such errors into PANIC.
Amit Kapila wrote:
> On Fri, Sep 24, 2021 at 4:44 PM Antonin Houska wrote:
> >
> > Amit Kapila wrote:
> >
> > > On Mon, Sep 20, 2021 at 10:24 AM Antonin Houska wrote:
> > > >
> > > > Amit Kapila wrote:
> > > >
> > > > > On Fri, Sep 17, 2021 at 9:50 PM Dmitry Dolgov <9erthali...@gmail.com>
>
On Fri, Sep 24, 2021 at 4:44 PM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
> > On Mon, Sep 20, 2021 at 10:24 AM Antonin Houska wrote:
> > >
> > > Amit Kapila wrote:
> > >
> > > > On Fri, Sep 17, 2021 at 9:50 PM Dmitry Dolgov <9erthali...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > On
Amit Kapila wrote:
> On Mon, Sep 20, 2021 at 10:24 AM Antonin Houska wrote:
> >
> > Amit Kapila wrote:
> >
> > > On Fri, Sep 17, 2021 at 9:50 PM Dmitry Dolgov <9erthali...@gmail.com>
> > > wrote:
> > > >
> > > > > On Tue, Sep 14, 2021 at 10:51:42AM +0200, Antonin Houska wrote:
> > > >
> > > >
On Mon, Sep 20, 2021 at 10:55 AM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
> > On Thu, Sep 9, 2021 at 8:33 PM Antonin Houska wrote:
>
> > > The problem of the temporary undo log is that it's loaded into local
> > > buffers
> > > and that backend can exit w/o flushing local buffers to
On Mon, Sep 20, 2021 at 10:24 AM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
> > On Fri, Sep 17, 2021 at 9:50 PM Dmitry Dolgov <9erthali...@gmail.com> wrote:
> > >
> > > > On Tue, Sep 14, 2021 at 10:51:42AM +0200, Antonin Houska wrote:
> > >
> > > * What happened with the idea of abandoning
On Tue, 21 Sep 2021 09:00 Antonin Houska, wrote:
> Dmitry Dolgov <9erthali...@gmail.com> wrote:
>
> > Yep, makes sense, thanks. I have few more questions:
> >
> > * The use case with orphaned files is working somewhat differently after
> > the rebase on the latest master, do you observe it as
Dmitry Dolgov <9erthali...@gmail.com> wrote:
> > On Tue, Sep 14, 2021 at 10:51:42AM +0200, Antonin Houska wrote:
> > Antonin Houska wrote:
> >
> > > Dmitry Dolgov <9erthali...@gmail.com> wrote:
> >
> > > > * By throwing at the patchset `make installcheck` I'm getting from time
> > > > to time
>
Amit Kapila wrote:
> On Thu, Sep 9, 2021 at 8:33 PM Antonin Houska wrote:
> > The problem of the temporary undo log is that it's loaded into local buffers
> > and that backend can exit w/o flushing local buffers to disk, and thus we
> > are
> > not guaranteed to find enough information when
Amit Kapila wrote:
> On Fri, Sep 17, 2021 at 9:50 PM Dmitry Dolgov <9erthali...@gmail.com> wrote:
> >
> > > On Tue, Sep 14, 2021 at 10:51:42AM +0200, Antonin Houska wrote:
> >
> > * What happened with the idea of abandoning discard worker for the sake
> > of simplicity? From what I see
On Fri, Sep 17, 2021 at 9:50 PM Dmitry Dolgov <9erthali...@gmail.com> wrote:
>
> > On Tue, Sep 14, 2021 at 10:51:42AM +0200, Antonin Houska wrote:
>
> * What happened with the idea of abandoning discard worker for the sake
> of simplicity? From what I see limiting everything to foreground undo
>
> On Tue, Sep 14, 2021 at 10:51:42AM +0200, Antonin Houska wrote:
> Antonin Houska wrote:
>
> > Dmitry Dolgov <9erthali...@gmail.com> wrote:
>
> > > * By throwing at the patchset `make installcheck` I'm getting from time
> > > to time
> > > and error on the restart:
> > >
> > > TRAP:
On Thu, Sep 9, 2021 at 8:33 PM Antonin Houska wrote:
>
> The cfbot complained that the patch series no longer applies, so I've rebased
> it and also tried to make sure that the other flags become green.
>
> One particular problem was that pg_upgrade complained that "live undo data"
> remains in
> On Wed, Jun 30, 2021 at 07:41:16PM +0200, Antonin Houska wrote:
> Antonin Houska wrote:
>
> > tsunakawa.ta...@fujitsu.com wrote:
> >
> > > I'm crawling like a snail to read the patch set. Below are my first set
> > > of review comments, which are all minor.
> >
> > Thanks.
>
> I've added the
On Wed, Jun 30, 2021 at 11:10 PM Antonin Houska wrote:
>
> Antonin Houska wrote:
>
> > tsunakawa.ta...@fujitsu.com wrote:
> >
> > > I'm crawling like a snail to read the patch set. Below are my first set
> > > of review comments, which are all minor.
> >
> > Thanks.
>
> I've added the patch
Antonin Houska wrote:
> tsunakawa.ta...@fujitsu.com wrote:
>
> > I'm crawling like a snail to read the patch set. Below are my first set of
> > review comments, which are all minor.
>
> Thanks.
I've added the patch to the upcoming CF [1], so it possibly gets more review
and makes some
tsunakawa.ta...@fujitsu.com wrote:
> I'm crawling like a snail to read the patch set. Below are my first set of
> review comments, which are all minor.
Thanks. I will check your comments when I'll be preparing the next version of
the patch.
--
Antonin Houska
Web:
I'm crawling like a snail to read the patch set. Below are my first set of
review comments, which are all minor.
(1)
+
tablespacetemporary
temporary -> undo
(2)
undo_tablespaces (string)
+
...
+The value is a list of names of tablespaces. When there is more than
+
On Wed, Feb 3, 2021 at 2:45 PM tsunakawa.ta...@fujitsu.com
wrote:
>
> From: Antonin Houska
> > not really an "ordinary patch".
> >
> > [1]
> > https://www.postgresql.org/message-id/CA%2BhUKG%2BMpzRsZFE7ChhR
> > q-Br5VYYi6mafVQ73Af7ahioWo5o8w%40mail.gmail.com
>
> I'm a bit interested in
From: Antonin Houska
> not really an "ordinary patch".
>
> [1]
> https://www.postgresql.org/message-id/CA%2BhUKG%2BMpzRsZFE7ChhR
> q-Br5VYYi6mafVQ73Af7ahioWo5o8w%40mail.gmail.com
I'm a bit interested in zheap-related topics. I'm reading this discussion to
see what I can do. (But this thread
Bruce Momjian wrote:
> On Fri, Jan 29, 2021 at 06:30:15PM +0100, Antonin Houska wrote:
> > Antonin Houska wrote:
> > > Well, on repeated run of the test I could also hit the first one. I could
> > > fix
> > > it and will post a new version of the patch (along with some other small
> > >
On Fri, Jan 29, 2021 at 06:30:15PM +0100, Antonin Houska wrote:
> Antonin Houska wrote:
> > Well, on repeated run of the test I could also hit the first one. I could
> > fix
> > it and will post a new version of the patch (along with some other small
> > changes) this week.
>
> Attached is the
Dmitry Dolgov <9erthali...@gmail.com> wrote:
> Thanks for the updated patch. As I've mentioned off the list I'm slowly
> looking through it with the intent to concentrate on undo progress
> tracking. But before I will post anything I want to mention couple of
> strange issues I see, otherwise I
> On Fri, Dec 04, 2020 at 10:22:42AM +0100, Antonin Houska wrote:
> Amit Kapila wrote:
>
> > On Fri, Nov 13, 2020 at 6:02 PM Antonin Houska wrote:
> > >
> > > Amit Kapila wrote:
> > >
> > > > On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska wrote:
> > > >
> > > > If you want to track at undo
On Fri, Dec 4, 2020 at 1:50 PM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
>
> > The earlier version of the patch having all these ideas
> > implemented is attached
> > (Infrastructure-to-execute-pending-undo-actions and
> > Provide-interfaces-to-store-and-fetch-undo-records). The second one
Amit Kapila wrote:
> On Wed, Nov 25, 2020 at 8:00 PM Antonin Houska wrote:
> > I meant that AbortOutOfAnyTransaction should PANIC itself if it sees that
> > there is unapplied undo, so nothing changes for its callers. Do I still miss
> > something?
> >
>
> Adding PANIC in some generic
Amit Kapila wrote:
> On Wed, Nov 25, 2020 at 7:47 PM Antonin Houska wrote:
> >
> > Antonin Houska wrote:
> >
> > > Amit Kapila wrote:
> >
> > > > I think we also need to maintain oldestXidHavingUndo for CLOG
> > > > truncation and
> > > > transaction-wraparound. We can't allow CLOG
On Wed, Nov 25, 2020 at 7:47 PM Antonin Houska wrote:
>
> Antonin Houska wrote:
>
> > Amit Kapila wrote:
>
> > > I think we also need to maintain oldestXidHavingUndo for CLOG truncation
> > > and
> > > transaction-wraparound. We can't allow CLOG truncation for the transaction
> > > whose undo
On Wed, Nov 25, 2020 at 8:00 PM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
> > On Wed, Nov 18, 2020 at 4:03 PM Antonin Houska wrote:
> > >
> > > Amit Kapila wrote:
> > >
> > > > On Fri, Nov 13, 2020 at 6:02 PM Antonin Houska wrote:
> > > > >
> > > > > Amit Kapila wrote:
> > > > >
> > > >
Amit Kapila wrote:
> On Wed, Nov 18, 2020 at 4:03 PM Antonin Houska wrote:
> >
> > Amit Kapila wrote:
> >
> > > On Fri, Nov 13, 2020 at 6:02 PM Antonin Houska wrote:
> > > >
> > > > Amit Kapila wrote:
> > > >
> > > > > On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska
> > > > > wrote:
> > > >
Antonin Houska wrote:
> Amit Kapila wrote:
> > I think we also need to maintain oldestXidHavingUndo for CLOG truncation and
> > transaction-wraparound. We can't allow CLOG truncation for the transaction
> > whose undo is not discarded as that could be required by some other
> > transaction.
>
On Wed, Nov 18, 2020 at 4:03 PM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
> > On Fri, Nov 13, 2020 at 6:02 PM Antonin Houska wrote:
> > >
> > > Amit Kapila wrote:
> > >
> > > > On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska wrote:
> > > > >
> > > > >
> > > > > No background undo
> > > >
Amit Kapila wrote:
> On Fri, Nov 13, 2020 at 6:02 PM Antonin Houska wrote:
> >
> > Amit Kapila wrote:
> >
> > > On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska wrote:
> > > >
> > > >
> > > > No background undo
> > > > --
> > > >
> > > > Reduced complexity of the patch seems to
On Sun, Nov 15, 2020 at 11:24 AM Amit Kapila wrote:
>
> On Fri, Nov 13, 2020 at 6:02 PM Antonin Houska wrote:
> >
> > Amit Kapila wrote:
> >
> > > On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska wrote:
> > > >
> > > >
> > > > No background undo
> > > > --
> > > >
> > > > Reduced
On Fri, Nov 13, 2020 at 6:02 PM Antonin Houska wrote:
>
> Amit Kapila wrote:
>
> > On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska wrote:
> > >
> > >
> > > No background undo
> > > --
> > >
> > > Reduced complexity of the patch seems to be the priority at the moment.
> > > Amit
Amit Kapila wrote:
> On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska wrote:
> >
> >
> > No background undo
> > --
> >
> > Reduced complexity of the patch seems to be the priority at the moment. Amit
> > suggested that cleanup of an orphaned relation file is simple enough to be
>
Thomas Munro wrote:
> On Thu, Nov 12, 2020 at 10:15 PM Antonin Houska wrote:
> I saw that -- great news! -- and have been meaning to write for a
> while. I think I am nearly ready to talk about it again.
I'm looking forward to it :-)
> 100% that it's worth trying to do something much
On Thu, Nov 12, 2020 at 2:45 PM Antonin Houska wrote:
>
>
> No background undo
> --
>
> Reduced complexity of the patch seems to be the priority at the moment. Amit
> suggested that cleanup of an orphaned relation file is simple enough to be
> done on foreground and I agree.
>
On Thu, Nov 12, 2020 at 10:15 PM Antonin Houska wrote:
> Thomas Munro wrote:
> > Thanks. We decided to redesign a couple of aspects of the undo
> > storage and record layers that this patch was intended to demonstrate,
> > and work on that is underway. More on that soon.
>
> As my boss
On Thu, Nov 28, 2019 at 3:45 PM Michael Paquier wrote:
> On Tue, Sep 17, 2019 at 10:03:20AM +1200, Thomas Munro wrote:
> > Oops, right. So it should just be added to the if condition. Will do.
>
> It's been a couple of months and the discussion has stale. It seems
> also that the patch was
On Tue, Sep 17, 2019 at 10:03:20AM +1200, Thomas Munro wrote:
> Oops, right. So it should just be added to the if condition. Will do.
It's been a couple of months and the discussion has stale. It seems
also that the patch was waiting for an update. So I am marking it as
RwF for now. Please
On Mon, Sep 16, 2019 at 10:37 PM Robert Haas wrote:
>
> It seems to me that zheap went wrong in ending up with separate undo
> types for in-place and non-in-place updates. Why not just have ONE
> kind of undo record that describes an update, and allow that update to
> have either one TID or two
On Tue, Sep 17, 2019 at 3:09 AM Kuntal Ghosh wrote:
> On Mon, Sep 16, 2019 at 11:23 AM Thomas Munro wrote:
> > Agreed. I added a line to break out of that loop if !block->in_use.
> >
> I think we should skip the block if !block->in_use. Because, the undo
> buffer can be registered in a
On Mon, Sep 16, 2019 at 11:09 AM Kuntal Ghosh
wrote:
> Okay. In that case, we need to rethink the cases for multi-inserts and
> non-inlace updates both of which currently inserts multiple undo
> record corresponding to a single WAL record. For multi-inserts, it can
> be solved easily by moving
Hello Thomas,
On Mon, Sep 16, 2019 at 11:23 AM Thomas Munro wrote:
>
> > 1. In UndoLogAllocateInRecovery, when we find the current log number
> > from the list of registered blocks, we don't check whether the
> > block->in_use flag is true or not. In XLogResetInsertion, we just
> > reset in_use
On Mon, Sep 16, 2019 at 5:27 AM Kuntal Ghosh wrote:
> While testing zheap over undo apis, we've found the following
> issues/scenarios that might need some fixes/discussions:
Thanks!
> 1. In UndoLogAllocateInRecovery, when we find the current log number
> from the list of registered blocks, we
Hello Thomas,
While testing zheap over undo apis, we've found the following
issues/scenarios that might need some fixes/discussions:
1. In UndoLogAllocateInRecovery, when we find the current log number
from the list of registered blocks, we don't check whether the
block->in_use flag is true or
On 2019-Sep-06, vignesh C wrote:
> Hi Thomas,
>
> While testing one of the recovery scenarios I found one issue:
> FailedAssertion("!(logno == context->recovery_logno)
I marked this patch Waiting on Author.
--
Álvaro Herrerahttps://www.2ndQuadrant.com/
PostgreSQL Development,
Hi Thomas,
While testing one of the recovery scenarios I found one issue:
FailedAssertion("!(logno == context->recovery_logno)
The details of the same is mentioned below:
The context's try_location was not updated in
UndoLogAllocateInRecovery, in PrepareUndoInsert the try_location was
updated
On Fri, Aug 30, 2019 at 8:27 PM Kuntal Ghosh wrote:
> I'm getting the following assert failure while performing the recovery
> with the same.
> "TRAP: FailedAssertion("slot->meta.status == UNDO_LOG_STATUS_FULL",
> File: "undolog.c", Line: 997)"
>
> I found that we don't emit an WAL record when we
Hello Thomas,
I was doing some testing for the scenario where the undo written by a
transaction overflows to multiple undo logs. For that I've modified
the following macro:
#define UndoLogMaxSize (1024 * 1024) /* 1MB undo log size */
(I should have used the provided pg_force_switch_undo though..)
On Wed, Aug 21, 2019 at 4:34 PM Robert Haas wrote:
> ReleaseResourcesAndProcessUndo() is only supposed to be called after
> AbortTransaction(), and the additional steps it performs --
> AtCleanup_Portals() and AtEOXact_Snapshot() or alternatively
> AtSubCleanup_Portals -- are taken from
On Fri, Aug 23, 2019 at 2:04 AM Thomas Munro wrote:
> 2. Strict self-update-only: We could update it as part of
> transaction cleanup. That is, when you commit or abort, probably at
> some time when your transaction is still advertised as running, you go
> and update your own transaction
On Wed, Aug 21, 2019 at 4:44 AM Andres Freund wrote:
> On 2019-08-20 21:02:18 +1200, Thomas Munro wrote:
> > 1. Anyone is allowed to try to read or write data at any UndoRecPtr
> > that has been allocated, through the buffer pool (though you'd usually
> > want to check it with
On Thu, Aug 22, 2019 at 9:55 PM Andres Freund wrote:
>
> Hi
>
> On August 22, 2019 9:14:10 AM PDT, Dilip Kumar wrote:
> > But, those requests will
> >ultimately be used for collecting the record by the bulk fetch. So if
> >we are planning to change the bulk fetch to read forward then maybe we
>
On Thu, Aug 22, 2019 at 7:34 PM Robert Haas wrote:
>
> On Thu, Aug 22, 2019 at 1:34 AM Dilip Kumar wrote:
> > Yeah, we can handle the bulk fetch as you suggested and it will make
> > it a lot easier. But, currently while registering the undo request
> > (especially during the first pass) we
On Thu, Aug 22, 2019 at 12:54 AM Andres Freund wrote:
> But why? It makes a *lot* more sense to have it in the beginning. I
> don't think bulk-fetch really requires it to be in the end - we can
> still process records forward on a page-by-page basis.
There are two separate needs here: to be able
Hi
On August 22, 2019 9:14:10 AM PDT, Dilip Kumar wrote:
> But, those requests will
>ultimately be used for collecting the record by the bulk fetch. So if
>we are planning to change the bulk fetch to read forward then maybe we
>don't need the valid last undo record pointer because that we will
On Thu, Aug 22, 2019 at 9:21 PM Dilip Kumar wrote:
>
> On Thu, Aug 22, 2019 at 7:34 PM Robert Haas wrote:
> >
> > On Thu, Aug 22, 2019 at 1:34 AM Dilip Kumar wrote:
> > > Yeah, we can handle the bulk fetch as you suggested and it will make
> > > it a lot easier. But, currently while
On Thu, Aug 22, 2019 at 1:34 AM Dilip Kumar wrote:
> Yeah, we can handle the bulk fetch as you suggested and it will make
> it a lot easier. But, currently while registering the undo request
> (especially during the first pass) we need to compute the from_urecptr
> and the to_urecptr. And, for
On Thu, Aug 22, 2019 at 11:04 AM Dilip Kumar wrote:
>
> On Thu, Aug 22, 2019 at 10:24 AM Andres Freund wrote:
> >
> > Hi,
> >
> > On 2019-08-22 10:19:04 +0530, Dilip Kumar wrote:
> > > On Thu, Aug 22, 2019 at 9:58 AM Andres Freund wrote:
> > > >
> > > > Hi,
> > > >
> > > > On 2019-08-22
On Thu, Aug 22, 2019 at 10:24 AM Andres Freund wrote:
>
> Hi,
>
> On 2019-08-22 10:19:04 +0530, Dilip Kumar wrote:
> > On Thu, Aug 22, 2019 at 9:58 AM Andres Freund wrote:
> > >
> > > Hi,
> > >
> > > On 2019-08-22 09:51:22 +0530, Dilip Kumar wrote:
> > > > We can not know the complete size of
On Thu, Aug 22, 2019 at 9:58 AM Andres Freund wrote:
>
> Hi,
>
> On 2019-08-22 09:51:22 +0530, Dilip Kumar wrote:
> > We can not know the complete size of the record even by reading the
> > header because we have a payload that is variable part and payload
> > length are stored in the payload
Hi,
On 2019-08-22 10:19:04 +0530, Dilip Kumar wrote:
> On Thu, Aug 22, 2019 at 9:58 AM Andres Freund wrote:
> >
> > Hi,
> >
> > On 2019-08-22 09:51:22 +0530, Dilip Kumar wrote:
> > > We can not know the complete size of the record even by reading the
> > > header because we have a payload that
Hi,
On 2019-08-22 09:51:22 +0530, Dilip Kumar wrote:
> We can not know the complete size of the record even by reading the
> header because we have a payload that is variable part and payload
> length are stored in the payload header which again can be at random
> offset.
Wait, but that's just
On Wed, Aug 21, 2019 at 9:04 PM Robert Haas wrote:
>
> On Wed, Aug 21, 2019 at 3:55 AM Dilip Kumar wrote:
> > I have already attempted that part and I feel it is not making code
> > any simpler than what we have today. For packing, it's fine because I
> > can process all the member once and
On Wed, Aug 14, 2019 at 2:57 AM Andres Freund wrote:
> - My reading of the current xact.c integration is that it's not workable
> as is. Undo is executed outside of a valid transaction state,
> exceptions aren't properly undone, logic would need to be duplicated
> to a significant degree,
On August 21, 2019 8:36:34 AM PDT, Robert Haas wrote:
> We treat LWLockAcquire() as a no-fail operation in many
>places; in my opinion, that elog(ERROR) that we have for too many
>LWLocks should be changed to elog(PANIC) precisely because we do treat
>LWLockAcquire() as no-fail in lots of
On Wed, Aug 21, 2019 at 3:55 AM Dilip Kumar wrote:
> I have already attempted that part and I feel it is not making code
> any simpler than what we have today. For packing, it's fine because I
> can process all the member once and directly pack it into one memory
> chunk and I can insert that to
On Wed, Aug 21, 2019 at 6:38 AM Amit Kapila wrote:
> > FWIW, although I also thought of doing what you are describing here, I
> > think Andres's proposal is probably preferable, because it's simpler.
> > There's not really any reason why we can't take the buffer locks from
> > within the critical
On Tue, Aug 20, 2019 at 8:10 PM Robert Haas wrote:
>
> On Tue, Aug 20, 2019 at 2:42 AM Amit Kapila wrote:
> > > Well, my main point, which so far has largely been ignored, was that we
> > > may not acquire page locks when we still need to search for victim
> > > buffers later. If we don't need
On Tue, Aug 20, 2019 at 7:57 PM Robert Haas wrote:
>
> On Mon, Aug 19, 2019 at 2:04 AM Dilip Kumar wrote:
> > Currently, In UnpackedUndoRecord we store all members directly which
> > are set by the caller. We store pointers to some header which are
> > allocated internally by the undo layer and
On Tue, Aug 13, 2019 at 8:11 AM Robert Haas wrote:
> > We can probably check the fxid queue and error queue to get that
> > value. However, I am not sure if that is sufficient because incase we
> > perform the request in the foreground, it won't be present in queues.
>
> Oh, I forgot about that
On Mon, Aug 19, 2019 at 2:04 AM Dilip Kumar wrote:
> Currently, In UnpackedUndoRecord we store all members directly which
> are set by the caller. We store pointers to some header which are
> allocated internally by the undo layer and the caller need not worry
> about setting them. So now you
On Tue, Aug 20, 2019 at 5:02 AM Thomas Munro wrote:
> 3. UndoLogDiscard() uses DiscardBuffer() to invalidate any currently
> unpinned buffers, and marks as BM_DISCARDED any that happen to be
> pinned right now, so they can't be immediately invalidated. Such
> buffers are never written back and
On 2019-08-20 09:44:23 -0700, Andres Freund wrote:
> On 2019-08-20 21:02:18 +1200, Thomas Munro wrote:
> > Aside from code changes based on review (and I have more to come of
> > those), the attached experimental patchset (also at
> > https://github.com/EnterpriseDB/zheap/tree/undo) has a new
Hi,
On 2019-08-20 17:11:38 +0530, Dilip Kumar wrote:
> On Wed, Aug 14, 2019 at 10:35 PM Andres Freund wrote:
> > On 2019-08-14 14:48:07 +0530, Dilip Kumar wrote:
> > > On Wed, Aug 14, 2019 at 12:27 PM Andres Freund wrote:
>
> > I don't think we can normally pin the undo buffers properly at
Hi,
On 2019-08-20 21:02:18 +1200, Thomas Munro wrote:
> Aside from code changes based on review (and I have more to come of
> those), the attached experimental patchset (also at
> https://github.com/EnterpriseDB/zheap/tree/undo) has a new protocol
> that, I hope, allows for better concurrency,
Hi,
On 2019-08-20 09:08:29 -0400, Robert Haas wrote:
> On Sat, Aug 17, 2019 at 1:28 PM Andres Freund wrote:
> > The primary one in the context here is that if we do *not* have to lock
> > the buffers all ahead of time, we can simplify the interface. We
> > certainly can't lock the buffers over
On Mon, Aug 19, 2019 at 8:22 AM Amit Kapila wrote:
> One point to remember in this regard is that we do need to modify the
> LSN in undo pages after writing WAL, so all the undo pages need to be
> locked by that time or we again need to take the lock on them.
Uh, but a big part of the point of
On Tue, Aug 20, 2019 at 2:42 AM Amit Kapila wrote:
> > Well, my main point, which so far has largely been ignored, was that we
> > may not acquire page locks when we still need to search for victim
> > buffers later. If we don't need to lock the pages up-front, but only do
> > so once we're
On Mon, Aug 19, 2019 at 5:16 PM Andres Freund wrote:
> Well, my main point, which so far has largely been ignored, was that we
> may not acquire page locks when we still need to search for victim
> buffers later. If we don't need to lock the pages up-front, but only do
> so once we're actually
On Sat, Aug 17, 2019 at 1:28 PM Andres Freund wrote:
> The primary one in the context here is that if we do *not* have to lock
> the buffers all ahead of time, we can simplify the interface. We
> certainly can't lock the buffers over IO (due to buffer reclaim) as
> we're doing right now, so we'd
On Wed, Aug 14, 2019 at 10:35 PM Andres Freund wrote:
>
> Hi,
>
> On 2019-08-14 14:48:07 +0530, Dilip Kumar wrote:
> > On Wed, Aug 14, 2019 at 12:27 PM Andres Freund wrote:
> I don't think we can normally pin the undo buffers properly at that
> stage. Without knowing the correct contents of the
On Tue, Aug 20, 2019 at 2:46 AM Andres Freund wrote:
>
> On 2019-08-19 17:52:24 +0530, Amit Kapila wrote:
> > On Sat, Aug 17, 2019 at 10:58 PM Andres Freund wrote:
> > >
> > > > Well, I don't understand why you're on about this. We've discussed it
> > > > a number of times but I'm still
On 2019-08-19 17:52:24 +0530, Amit Kapila wrote:
> On Sat, Aug 17, 2019 at 10:58 PM Andres Freund wrote:
> >
> > Hi,
> >
> > On 2019-08-17 12:05:21 -0400, Robert Haas wrote:
> > > On Wed, Aug 14, 2019 at 12:39 PM Andres Freund wrote:
> > > > > > Again, I think it's not ok to just assume you can
On Sat, Aug 17, 2019 at 10:58 PM Andres Freund wrote:
>
> Hi,
>
> On 2019-08-17 12:05:21 -0400, Robert Haas wrote:
> > On Wed, Aug 14, 2019 at 12:39 PM Andres Freund wrote:
> > > > > Again, I think it's not ok to just assume you can lock an essentially
> > > > > unbounded number of buffers. This
On Wed, Aug 14, 2019 at 10:35 PM Andres Freund wrote:
>
>
> > > - When reading an undo record, the whole stage of UnpackUndoData()
> > > reading data into a the UndoPackContext is omitted, reading directly
> > > into the UnpackedUndoRecord. That removes one further copy of the
> > >
On Sat, Aug 17, 2019 at 9:35 PM Robert Haas wrote:
>
> On Wed, Aug 14, 2019 at 12:39 PM Andres Freund wrote:
> > > > Again, I think it's not ok to just assume you can lock an essentially
> > > > unbounded number of buffers. This seems almost guaranteed to result in
> > > > deadlocks. And there's
Hi,
On 2019-08-17 12:05:21 -0400, Robert Haas wrote:
> On Wed, Aug 14, 2019 at 12:39 PM Andres Freund wrote:
> > > > Again, I think it's not ok to just assume you can lock an essentially
> > > > unbounded number of buffers. This seems almost guaranteed to result in
> > > > deadlocks. And there's
On Wed, Aug 14, 2019 at 12:39 PM Andres Freund wrote:
> > > Again, I think it's not ok to just assume you can lock an essentially
> > > unbounded number of buffers. This seems almost guaranteed to result in
> > > deadlocks. And there's limits on how many lwlocks one can hold etc.
> >
> > I think
On Fri, Aug 16, 2019 at 10:56 AM Andres Freund wrote:
>
> Hi,
>
> On 2019-08-16 09:44:25 +0530, Dilip Kumar wrote:
> > On Wed, Aug 14, 2019 at 2:48 PM Dilip Kumar wrote:
> > >
> > > On Wed, Aug 14, 2019 at 12:27 PM Andres Freund wrote:
> >
> > > > I think that batch reading should just copy
Hi,
On 2019-08-16 09:44:25 +0530, Dilip Kumar wrote:
> On Wed, Aug 14, 2019 at 2:48 PM Dilip Kumar wrote:
> >
> > On Wed, Aug 14, 2019 at 12:27 PM Andres Freund wrote:
>
> > > I think that batch reading should just copy the underlying data into a
> > > char* buffer. Only the records that
1 - 100 of 305 matches
Mail list logo
