On Tue, May 25, 2021 at 1:59 PM Pavan Deolasee <pavan.deola...@gmail.com> wrote:
> > On Tue, May 25, 2021 at 1:49 PM Dilip Kumar <dilipbal...@gmail.com> wrote: >> >> On Tue, May 25, 2021 at 1:45 PM Pavan Deolasee <pavan.deola...@gmail.com> >> wrote: >> > >> > I am not entirely sure if it works correctly. I'd tried something similar, >> > but the downstream node using >> > my output plugin gets NULL values for the toast columns. It's a bit hard >> > to demonstrate that with the >> > test_decoding plugin, but if you have some other mechanism to test that >> > change with an actual downstream >> > node receiving and applying changes, it will be useful to test with that. >> >> Okay, I will test that. Thanks. >> > > I modified test_decoding to print the tuples (like in the non-streamed case) > and included your proposed fix. PFA > > When the transaction is streamed, I see: > ``` > + opening a streamed block for transaction > + table public.toasted: INSERT: id[integer]:9001 other[text]:'bbb' > data[text]:'ccc' > + table public.toasted: INSERT: id[integer]:9002 other[text]:'ddd' > data[text]:'eee' > + table public.toasted: INSERT: id[integer]:9003 other[text]:'bar' > data[text]:unchanged-toast-datum > <snipped> > ``` > > For a non-streamed case, the `data[text]` column shows the actual data. That > probably manifests into NULL data when downstream handles it. Yes, I am able to reproduce this, basically, until we get the last tuple of the multi insert we can not clear the toast data otherwise we can never form a complete tuple. So the only possible fix I can think of is to consider the multi-insert WAL without the final multi-insert tuple as partial data then we will avoid streaming until we get the complete WAL of one multi-insert. I am working on the patch to fix this, I will share that in some time. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com
