Hi,
On 2014-07-06 14:01:11 +0000, Andres Freund wrote:
> Fix decoding of MULTI_INSERTs when rows other than the last are toasted.
>
> When decoding the results of a HEAP2_MULTI_INSERT (currently only
> generated by COPY FROM) toast columns for all but the last tuple
> weren't replaced by their actual contents before being handed to the
> output plugin. The reassembled toast datums where disregarded after
> every REORDER_BUFFER_CHANGE_(INSERT|UPDATE|DELETE) which is correct
> for plain inserts, updates, deletes, but not multi inserts - there we
> generate several REORDER_BUFFER_CHANGE_INSERTs for a single
> xl_heap_multi_insert record.
>
> To solve the problem add a clear_toast_afterwards boolean to
> ReorderBufferChange's union member that's used by modifications. All
> row changes but multi_inserts always set that to true, but
> multi_insert sets it only for the last change generated.
>
> Add a regression test covering decoding of multi_inserts - there was
> none at all before.
>
> Backpatch to 9.4 where logical decoding was introduced.
>
> Bug found by Petr Jelinek.
Further testing unfortuantely shows that this isn't sufficient:
Commit 1b86c81d2d fixed the decoding of toasted columns for the rows
contained in one xl_heap_multi_insert record. But that's not actually
enough because heap_multi_insert() will actually first toast all
passed in rows and then emit several *_multi_insert records; one for
each page it fills with tuples.
I've attached a preliminary patch that:
Add a XLOG_HEAP_LAST_MULTI_INSERT flag which is set in
xl_heap_multi_insert->flag denoting that this multi_insert record is
the last emitted by one heap_multi_insert() call. Then use that flag
in decode.c to only set clear_toast_afterwards in the right situation.
But since I am not exactly happy about that solution I'm wondering if
somebody can think of a better approach. Alternatives I though of
included:
* Add a boolean to xl_heap_multi_insert instead of adding the
XLOG_HEAP_LAST_MULTI_INSERT flag. I don't have an opinion either way
about that one.
* Only toast rows in heap_multi_insert for every page. That doesn't
really work without major reworks though as the loop over all the
tuples is done in a critical section.
* Don't clean up toast chunks for multi insert at all. Instead do so after
the next !multi insert record. I think that's out because of the
amount of multi_insert calls COPY can produce.
The patch also adds 200 lines of COPY FROM STDIN in the regression tests
to create a long COPY that does a multi insert with toast covering two
pages. Does anybody object to that?
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
From b154d9732918de190dc55ce8fecdd790ac3657e3 Mon Sep 17 00:00:00 2001
From: Andres Freund <and...@anarazel.de>
Date: Sun, 6 Jul 2014 18:44:54 +0200
Subject: [PATCH] Fix decoding of consecutive MULTI_INSERTs emitted by one
heap_multi_insert().
Commit 1b86c81d2d fixed the decoding of toasted columns for the rows
contained in one xl_heap_multi_insert record. But that's not actually
enough because heap_multi_insert() will actually first toast all
passed in rows and then emit several *_multi_insert records; one for
each page it fills with tuples.
Add a XLOG_HEAP_LAST_MULTI_INSERT flag which is set in
xl_heap_multi_insert->flag denoting that this multi_insert record is
the last emitted by one heap_multi_insert() call. Then use that flag
in decode.c to only set clear_toast_afterwards in the right situation.
Backpatch to 9.4, just like the previous commit.
---
contrib/test_decoding/expected/toast.out | 201 ++++++++++++++++++++++++++++++-
contrib/test_decoding/sql/toast.sql | 199 ++++++++++++++++++++++++++++++
src/backend/access/heap/heapam.c | 8 ++
src/backend/access/rmgrdesc/heapdesc.c | 2 +
src/backend/replication/logical/decode.c | 12 +-
src/include/access/heapam_xlog.h | 2 +
6 files changed, 421 insertions(+), 3 deletions(-)
diff --git a/contrib/test_decoding/expected/toast.out b/contrib/test_decoding/expected/toast.out
index 322afdb..53442e0 100644
--- a/contrib/test_decoding/expected/toast.out
+++ b/contrib/test_decoding/expected/toast.out
@@ -97,8 +97,207 @@ SELECT substr(data, 1, 200) FROM pg_logical_slot_get_changes('regression_slot',
table public.toasted_copy: INSERT: id[integer]:2 data[text]:'toasted1-1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
table public.toasted_copy: INSERT: id[integer]:3 data[text]:'untoasted2'
table public.toasted_copy: INSERT: id[integer]:4 data[text]:'toasted2-1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
+ table public.toasted_copy: INSERT: id[integer]:5 data[text]:'untoasted3'
+ table public.toasted_copy: INSERT: id[integer]:6 data[text]:'untoasted4'
+ table public.toasted_copy: INSERT: id[integer]:7 data[text]:'untoasted5'
+ table public.toasted_copy: INSERT: id[integer]:8 data[text]:'untoasted6'
+ table public.toasted_copy: INSERT: id[integer]:9 data[text]:'untoasted7'
+ table public.toasted_copy: INSERT: id[integer]:10 data[text]:'untoasted8'
+ table public.toasted_copy: INSERT: id[integer]:11 data[text]:'untoasted9'
+ table public.toasted_copy: INSERT: id[integer]:12 data[text]:'untoasted10'
+ table public.toasted_copy: INSERT: id[integer]:13 data[text]:'untoasted11'
+ table public.toasted_copy: INSERT: id[integer]:14 data[text]:'untoasted12'
+ table public.toasted_copy: INSERT: id[integer]:15 data[text]:'untoasted13'
+ table public.toasted_copy: INSERT: id[integer]:16 data[text]:'untoasted14'
+ table public.toasted_copy: INSERT: id[integer]:17 data[text]:'untoasted15'
+ table public.toasted_copy: INSERT: id[integer]:18 data[text]:'untoasted16'
+ table public.toasted_copy: INSERT: id[integer]:19 data[text]:'untoasted17'
+ table public.toasted_copy: INSERT: id[integer]:20 data[text]:'untoasted18'
+ table public.toasted_copy: INSERT: id[integer]:21 data[text]:'untoasted19'
+ table public.toasted_copy: INSERT: id[integer]:22 data[text]:'untoasted20'
+ table public.toasted_copy: INSERT: id[integer]:23 data[text]:'untoasted21'
+ table public.toasted_copy: INSERT: id[integer]:24 data[text]:'untoasted22'
+ table public.toasted_copy: INSERT: id[integer]:25 data[text]:'untoasted23'
+ table public.toasted_copy: INSERT: id[integer]:26 data[text]:'untoasted24'
+ table public.toasted_copy: INSERT: id[integer]:27 data[text]:'untoasted25'
+ table public.toasted_copy: INSERT: id[integer]:28 data[text]:'untoasted26'
+ table public.toasted_copy: INSERT: id[integer]:29 data[text]:'untoasted27'
+ table public.toasted_copy: INSERT: id[integer]:30 data[text]:'untoasted28'
+ table public.toasted_copy: INSERT: id[integer]:31 data[text]:'untoasted29'
+ table public.toasted_copy: INSERT: id[integer]:32 data[text]:'untoasted30'
+ table public.toasted_copy: INSERT: id[integer]:33 data[text]:'untoasted31'
+ table public.toasted_copy: INSERT: id[integer]:34 data[text]:'untoasted32'
+ table public.toasted_copy: INSERT: id[integer]:35 data[text]:'untoasted33'
+ table public.toasted_copy: INSERT: id[integer]:36 data[text]:'untoasted34'
+ table public.toasted_copy: INSERT: id[integer]:37 data[text]:'untoasted35'
+ table public.toasted_copy: INSERT: id[integer]:38 data[text]:'untoasted36'
+ table public.toasted_copy: INSERT: id[integer]:39 data[text]:'untoasted37'
+ table public.toasted_copy: INSERT: id[integer]:40 data[text]:'untoasted38'
+ table public.toasted_copy: INSERT: id[integer]:41 data[text]:'untoasted39'
+ table public.toasted_copy: INSERT: id[integer]:42 data[text]:'untoasted40'
+ table public.toasted_copy: INSERT: id[integer]:43 data[text]:'untoasted41'
+ table public.toasted_copy: INSERT: id[integer]:44 data[text]:'untoasted42'
+ table public.toasted_copy: INSERT: id[integer]:45 data[text]:'untoasted43'
+ table public.toasted_copy: INSERT: id[integer]:46 data[text]:'untoasted44'
+ table public.toasted_copy: INSERT: id[integer]:47 data[text]:'untoasted45'
+ table public.toasted_copy: INSERT: id[integer]:48 data[text]:'untoasted46'
+ table public.toasted_copy: INSERT: id[integer]:49 data[text]:'untoasted47'
+ table public.toasted_copy: INSERT: id[integer]:50 data[text]:'untoasted48'
+ table public.toasted_copy: INSERT: id[integer]:51 data[text]:'untoasted49'
+ table public.toasted_copy: INSERT: id[integer]:52 data[text]:'untoasted50'
+ table public.toasted_copy: INSERT: id[integer]:53 data[text]:'untoasted51'
+ table public.toasted_copy: INSERT: id[integer]:54 data[text]:'untoasted52'
+ table public.toasted_copy: INSERT: id[integer]:55 data[text]:'untoasted53'
+ table public.toasted_copy: INSERT: id[integer]:56 data[text]:'untoasted54'
+ table public.toasted_copy: INSERT: id[integer]:57 data[text]:'untoasted55'
+ table public.toasted_copy: INSERT: id[integer]:58 data[text]:'untoasted56'
+ table public.toasted_copy: INSERT: id[integer]:59 data[text]:'untoasted57'
+ table public.toasted_copy: INSERT: id[integer]:60 data[text]:'untoasted58'
+ table public.toasted_copy: INSERT: id[integer]:61 data[text]:'untoasted59'
+ table public.toasted_copy: INSERT: id[integer]:62 data[text]:'untoasted60'
+ table public.toasted_copy: INSERT: id[integer]:63 data[text]:'untoasted61'
+ table public.toasted_copy: INSERT: id[integer]:64 data[text]:'untoasted62'
+ table public.toasted_copy: INSERT: id[integer]:65 data[text]:'untoasted63'
+ table public.toasted_copy: INSERT: id[integer]:66 data[text]:'untoasted64'
+ table public.toasted_copy: INSERT: id[integer]:67 data[text]:'untoasted65'
+ table public.toasted_copy: INSERT: id[integer]:68 data[text]:'untoasted66'
+ table public.toasted_copy: INSERT: id[integer]:69 data[text]:'untoasted67'
+ table public.toasted_copy: INSERT: id[integer]:70 data[text]:'untoasted68'
+ table public.toasted_copy: INSERT: id[integer]:71 data[text]:'untoasted69'
+ table public.toasted_copy: INSERT: id[integer]:72 data[text]:'untoasted70'
+ table public.toasted_copy: INSERT: id[integer]:73 data[text]:'untoasted71'
+ table public.toasted_copy: INSERT: id[integer]:74 data[text]:'untoasted72'
+ table public.toasted_copy: INSERT: id[integer]:75 data[text]:'untoasted73'
+ table public.toasted_copy: INSERT: id[integer]:76 data[text]:'untoasted74'
+ table public.toasted_copy: INSERT: id[integer]:77 data[text]:'untoasted75'
+ table public.toasted_copy: INSERT: id[integer]:78 data[text]:'untoasted76'
+ table public.toasted_copy: INSERT: id[integer]:79 data[text]:'untoasted77'
+ table public.toasted_copy: INSERT: id[integer]:80 data[text]:'untoasted78'
+ table public.toasted_copy: INSERT: id[integer]:81 data[text]:'untoasted79'
+ table public.toasted_copy: INSERT: id[integer]:82 data[text]:'untoasted80'
+ table public.toasted_copy: INSERT: id[integer]:83 data[text]:'untoasted81'
+ table public.toasted_copy: INSERT: id[integer]:84 data[text]:'untoasted82'
+ table public.toasted_copy: INSERT: id[integer]:85 data[text]:'untoasted83'
+ table public.toasted_copy: INSERT: id[integer]:86 data[text]:'untoasted84'
+ table public.toasted_copy: INSERT: id[integer]:87 data[text]:'untoasted85'
+ table public.toasted_copy: INSERT: id[integer]:88 data[text]:'untoasted86'
+ table public.toasted_copy: INSERT: id[integer]:89 data[text]:'untoasted87'
+ table public.toasted_copy: INSERT: id[integer]:90 data[text]:'untoasted88'
+ table public.toasted_copy: INSERT: id[integer]:91 data[text]:'untoasted89'
+ table public.toasted_copy: INSERT: id[integer]:92 data[text]:'untoasted90'
+ table public.toasted_copy: INSERT: id[integer]:93 data[text]:'untoasted91'
+ table public.toasted_copy: INSERT: id[integer]:94 data[text]:'untoasted92'
+ table public.toasted_copy: INSERT: id[integer]:95 data[text]:'untoasted93'
+ table public.toasted_copy: INSERT: id[integer]:96 data[text]:'untoasted94'
+ table public.toasted_copy: INSERT: id[integer]:97 data[text]:'untoasted95'
+ table public.toasted_copy: INSERT: id[integer]:98 data[text]:'untoasted96'
+ table public.toasted_copy: INSERT: id[integer]:99 data[text]:'untoasted97'
+ table public.toasted_copy: INSERT: id[integer]:100 data[text]:'untoasted98'
+ table public.toasted_copy: INSERT: id[integer]:101 data[text]:'untoasted99'
+ table public.toasted_copy: INSERT: id[integer]:102 data[text]:'untoasted100'
+ table public.toasted_copy: INSERT: id[integer]:103 data[text]:'untoasted101'
+ table public.toasted_copy: INSERT: id[integer]:104 data[text]:'untoasted102'
+ table public.toasted_copy: INSERT: id[integer]:105 data[text]:'untoasted103'
+ table public.toasted_copy: INSERT: id[integer]:106 data[text]:'untoasted104'
+ table public.toasted_copy: INSERT: id[integer]:107 data[text]:'untoasted105'
+ table public.toasted_copy: INSERT: id[integer]:108 data[text]:'untoasted106'
+ table public.toasted_copy: INSERT: id[integer]:109 data[text]:'untoasted107'
+ table public.toasted_copy: INSERT: id[integer]:110 data[text]:'untoasted108'
+ table public.toasted_copy: INSERT: id[integer]:111 data[text]:'untoasted109'
+ table public.toasted_copy: INSERT: id[integer]:112 data[text]:'untoasted110'
+ table public.toasted_copy: INSERT: id[integer]:113 data[text]:'untoasted111'
+ table public.toasted_copy: INSERT: id[integer]:114 data[text]:'untoasted112'
+ table public.toasted_copy: INSERT: id[integer]:115 data[text]:'untoasted113'
+ table public.toasted_copy: INSERT: id[integer]:116 data[text]:'untoasted114'
+ table public.toasted_copy: INSERT: id[integer]:117 data[text]:'untoasted115'
+ table public.toasted_copy: INSERT: id[integer]:118 data[text]:'untoasted116'
+ table public.toasted_copy: INSERT: id[integer]:119 data[text]:'untoasted117'
+ table public.toasted_copy: INSERT: id[integer]:120 data[text]:'untoasted118'
+ table public.toasted_copy: INSERT: id[integer]:121 data[text]:'untoasted119'
+ table public.toasted_copy: INSERT: id[integer]:122 data[text]:'untoasted120'
+ table public.toasted_copy: INSERT: id[integer]:123 data[text]:'untoasted121'
+ table public.toasted_copy: INSERT: id[integer]:124 data[text]:'untoasted122'
+ table public.toasted_copy: INSERT: id[integer]:125 data[text]:'untoasted123'
+ table public.toasted_copy: INSERT: id[integer]:126 data[text]:'untoasted124'
+ table public.toasted_copy: INSERT: id[integer]:127 data[text]:'untoasted125'
+ table public.toasted_copy: INSERT: id[integer]:128 data[text]:'untoasted126'
+ table public.toasted_copy: INSERT: id[integer]:129 data[text]:'untoasted127'
+ table public.toasted_copy: INSERT: id[integer]:130 data[text]:'untoasted128'
+ table public.toasted_copy: INSERT: id[integer]:131 data[text]:'untoasted129'
+ table public.toasted_copy: INSERT: id[integer]:132 data[text]:'untoasted130'
+ table public.toasted_copy: INSERT: id[integer]:133 data[text]:'untoasted131'
+ table public.toasted_copy: INSERT: id[integer]:134 data[text]:'untoasted132'
+ table public.toasted_copy: INSERT: id[integer]:135 data[text]:'untoasted133'
+ table public.toasted_copy: INSERT: id[integer]:136 data[text]:'untoasted134'
+ table public.toasted_copy: INSERT: id[integer]:137 data[text]:'untoasted135'
+ table public.toasted_copy: INSERT: id[integer]:138 data[text]:'untoasted136'
+ table public.toasted_copy: INSERT: id[integer]:139 data[text]:'untoasted137'
+ table public.toasted_copy: INSERT: id[integer]:140 data[text]:'untoasted138'
+ table public.toasted_copy: INSERT: id[integer]:141 data[text]:'untoasted139'
+ table public.toasted_copy: INSERT: id[integer]:142 data[text]:'untoasted140'
+ table public.toasted_copy: INSERT: id[integer]:143 data[text]:'untoasted141'
+ table public.toasted_copy: INSERT: id[integer]:144 data[text]:'untoasted142'
+ table public.toasted_copy: INSERT: id[integer]:145 data[text]:'untoasted143'
+ table public.toasted_copy: INSERT: id[integer]:146 data[text]:'untoasted144'
+ table public.toasted_copy: INSERT: id[integer]:147 data[text]:'untoasted145'
+ table public.toasted_copy: INSERT: id[integer]:148 data[text]:'untoasted146'
+ table public.toasted_copy: INSERT: id[integer]:149 data[text]:'untoasted147'
+ table public.toasted_copy: INSERT: id[integer]:150 data[text]:'untoasted148'
+ table public.toasted_copy: INSERT: id[integer]:151 data[text]:'untoasted149'
+ table public.toasted_copy: INSERT: id[integer]:152 data[text]:'untoasted150'
+ table public.toasted_copy: INSERT: id[integer]:153 data[text]:'untoasted151'
+ table public.toasted_copy: INSERT: id[integer]:154 data[text]:'untoasted152'
+ table public.toasted_copy: INSERT: id[integer]:155 data[text]:'untoasted153'
+ table public.toasted_copy: INSERT: id[integer]:156 data[text]:'untoasted154'
+ table public.toasted_copy: INSERT: id[integer]:157 data[text]:'untoasted155'
+ table public.toasted_copy: INSERT: id[integer]:158 data[text]:'untoasted156'
+ table public.toasted_copy: INSERT: id[integer]:159 data[text]:'untoasted157'
+ table public.toasted_copy: INSERT: id[integer]:160 data[text]:'untoasted158'
+ table public.toasted_copy: INSERT: id[integer]:161 data[text]:'untoasted159'
+ table public.toasted_copy: INSERT: id[integer]:162 data[text]:'untoasted160'
+ table public.toasted_copy: INSERT: id[integer]:163 data[text]:'untoasted161'
+ table public.toasted_copy: INSERT: id[integer]:164 data[text]:'untoasted162'
+ table public.toasted_copy: INSERT: id[integer]:165 data[text]:'untoasted163'
+ table public.toasted_copy: INSERT: id[integer]:166 data[text]:'untoasted164'
+ table public.toasted_copy: INSERT: id[integer]:167 data[text]:'untoasted165'
+ table public.toasted_copy: INSERT: id[integer]:168 data[text]:'untoasted166'
+ table public.toasted_copy: INSERT: id[integer]:169 data[text]:'untoasted167'
+ table public.toasted_copy: INSERT: id[integer]:170 data[text]:'untoasted168'
+ table public.toasted_copy: INSERT: id[integer]:171 data[text]:'untoasted169'
+ table public.toasted_copy: INSERT: id[integer]:172 data[text]:'untoasted170'
+ table public.toasted_copy: INSERT: id[integer]:173 data[text]:'untoasted171'
+ table public.toasted_copy: INSERT: id[integer]:174 data[text]:'untoasted172'
+ table public.toasted_copy: INSERT: id[integer]:175 data[text]:'untoasted173'
+ table public.toasted_copy: INSERT: id[integer]:176 data[text]:'untoasted174'
+ table public.toasted_copy: INSERT: id[integer]:177 data[text]:'untoasted175'
+ table public.toasted_copy: INSERT: id[integer]:178 data[text]:'untoasted176'
+ table public.toasted_copy: INSERT: id[integer]:179 data[text]:'untoasted177'
+ table public.toasted_copy: INSERT: id[integer]:180 data[text]:'untoasted178'
+ table public.toasted_copy: INSERT: id[integer]:181 data[text]:'untoasted179'
+ table public.toasted_copy: INSERT: id[integer]:182 data[text]:'untoasted180'
+ table public.toasted_copy: INSERT: id[integer]:183 data[text]:'untoasted181'
+ table public.toasted_copy: INSERT: id[integer]:184 data[text]:'untoasted182'
+ table public.toasted_copy: INSERT: id[integer]:185 data[text]:'untoasted183'
+ table public.toasted_copy: INSERT: id[integer]:186 data[text]:'untoasted184'
+ table public.toasted_copy: INSERT: id[integer]:187 data[text]:'untoasted185'
+ table public.toasted_copy: INSERT: id[integer]:188 data[text]:'untoasted186'
+ table public.toasted_copy: INSERT: id[integer]:189 data[text]:'untoasted187'
+ table public.toasted_copy: INSERT: id[integer]:190 data[text]:'untoasted188'
+ table public.toasted_copy: INSERT: id[integer]:191 data[text]:'untoasted189'
+ table public.toasted_copy: INSERT: id[integer]:192 data[text]:'untoasted190'
+ table public.toasted_copy: INSERT: id[integer]:193 data[text]:'untoasted191'
+ table public.toasted_copy: INSERT: id[integer]:194 data[text]:'untoasted192'
+ table public.toasted_copy: INSERT: id[integer]:195 data[text]:'untoasted193'
+ table public.toasted_copy: INSERT: id[integer]:196 data[text]:'untoasted194'
+ table public.toasted_copy: INSERT: id[integer]:197 data[text]:'untoasted195'
+ table public.toasted_copy: INSERT: id[integer]:198 data[text]:'untoasted196'
+ table public.toasted_copy: INSERT: id[integer]:199 data[text]:'untoasted197'
+ table public.toasted_copy: INSERT: id[integer]:200 data[text]:'untoasted198'
+ table public.toasted_copy: INSERT: id[integer]:201 data[text]:'toasted3-12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678
+ table public.toasted_copy: INSERT: id[integer]:202 data[text]:'untoasted199'
+ table public.toasted_copy: INSERT: id[integer]:203 data[text]:'untoasted200'
COMMIT
-(47 rows)
+(246 rows)
SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
diff --git a/contrib/test_decoding/sql/toast.sql b/contrib/test_decoding/sql/toast.sql
index a5f9a5f..03146b0 100644
--- a/contrib/test_decoding/sql/toast.sql
+++ b/contrib/test_decoding/sql/toast.sql
@@ -59,6 +59,205 @@ ALTER TABLE toasted_copy ALTER COLUMN data SET STORAGE EXTERNAL;
2 toasted1-12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
3 untoasted2
4 toasted2-12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
+5 untoasted3
+6 untoasted4
+7 untoasted5
+8 untoasted6
+9 untoasted7
+10 untoasted8
+11 untoasted9
+12 untoasted10
+13 untoasted11
+14 untoasted12
+15 untoasted13
+16 untoasted14
+17 untoasted15
+18 untoasted16
+19 untoasted17
+20 untoasted18
+21 untoasted19
+22 untoasted20
+23 untoasted21
+24 untoasted22
+25 untoasted23
+26 untoasted24
+27 untoasted25
+28 untoasted26
+29 untoasted27
+30 untoasted28
+31 untoasted29
+32 untoasted30
+33 untoasted31
+34 untoasted32
+35 untoasted33
+36 untoasted34
+37 untoasted35
+38 untoasted36
+39 untoasted37
+40 untoasted38
+41 untoasted39
+42 untoasted40
+43 untoasted41
+44 untoasted42
+45 untoasted43
+46 untoasted44
+47 untoasted45
+48 untoasted46
+49 untoasted47
+50 untoasted48
+51 untoasted49
+52 untoasted50
+53 untoasted51
+54 untoasted52
+55 untoasted53
+56 untoasted54
+57 untoasted55
+58 untoasted56
+59 untoasted57
+60 untoasted58
+61 untoasted59
+62 untoasted60
+63 untoasted61
+64 untoasted62
+65 untoasted63
+66 untoasted64
+67 untoasted65
+68 untoasted66
+69 untoasted67
+70 untoasted68
+71 untoasted69
+72 untoasted70
+73 untoasted71
+74 untoasted72
+75 untoasted73
+76 untoasted74
+77 untoasted75
+78 untoasted76
+79 untoasted77
+80 untoasted78
+81 untoasted79
+82 untoasted80
+83 untoasted81
+84 untoasted82
+85 untoasted83
+86 untoasted84
+87 untoasted85
+88 untoasted86
+89 untoasted87
+90 untoasted88
+91 untoasted89
+92 untoasted90
+93 untoasted91
+94 untoasted92
+95 untoasted93
+96 untoasted94
+97 untoasted95
+98 untoasted96
+99 untoasted97
+100 untoasted98
+101 untoasted99
+102 untoasted100
+103 untoasted101
+104 untoasted102
+105 untoasted103
+106 untoasted104
+107 untoasted105
+108 untoasted106
+109 untoasted107
+110 untoasted108
+111 untoasted109
+112 untoasted110
+113 untoasted111
+114 untoasted112
+115 untoasted113
+116 untoasted114
+117 untoasted115
+118 untoasted116
+119 untoasted117
+120 untoasted118
+121 untoasted119
+122 untoasted120
+123 untoasted121
+124 untoasted122
+125 untoasted123
+126 untoasted124
+127 untoasted125
+128 untoasted126
+129 untoasted127
+130 untoasted128
+131 untoasted129
+132 untoasted130
+133 untoasted131
+134 untoasted132
+135 untoasted133
+136 untoasted134
+137 untoasted135
+138 untoasted136
+139 untoasted137
+140 untoasted138
+141 untoasted139
+142 untoasted140
+143 untoasted141
+144 untoasted142
+145 untoasted143
+146 untoasted144
+147 untoasted145
+148 untoasted146
+149 untoasted147
+150 untoasted148
+151 untoasted149
+152 untoasted150
+153 untoasted151
+154 untoasted152
+155 untoasted153
+156 untoasted154
+157 untoasted155
+158 untoasted156
+159 untoasted157
+160 untoasted158
+161 untoasted159
+162 untoasted160
+163 untoasted161
+164 untoasted162
+165 untoasted163
+166 untoasted164
+167 untoasted165
+168 untoasted166
+169 untoasted167
+170 untoasted168
+171 untoasted169
+172 untoasted170
+173 untoasted171
+174 untoasted172
+175 untoasted173
+176 untoasted174
+177 untoasted175
+178 untoasted176
+179 untoasted177
+180 untoasted178
+181 untoasted179
+182 untoasted180
+183 untoasted181
+184 untoasted182
+185 untoasted183
+186 untoasted184
+187 untoasted185
+188 untoasted186
+189 untoasted187
+190 untoasted188
+191 untoasted189
+192 untoasted190
+193 untoasted191
+194 untoasted192
+195 untoasted193
+196 untoasted194
+197 untoasted195
+198 untoasted196
+199 untoasted197
+200 untoasted198
+201 toasted3-12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
+202 untoasted199
+203 untoasted200
\.
SELECT substr(data, 1, 200) FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0');
SELECT pg_drop_replication_slot('regression_slot');
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 6861ae0..35f9404 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2515,6 +2515,14 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
info |= XLOG_HEAP_INIT_PAGE;
}
+ /*
+ * Signal that this is the last xl_heap_multi_insert record
+ * emitted by this call to heap_multi_insert(). Needed for logical
+ * decoding so it knows when to cleanup temporary data.
+ */
+ if (ndone + nthispage == ntuples)
+ xlrec->flags |= XLOG_HEAP_LAST_MULTI_INSERT;
+
recptr = XLogInsert(RM_HEAP2_ID, info, rdata);
PageSetLSN(page, recptr);
diff --git a/src/backend/access/rmgrdesc/heapdesc.c b/src/backend/access/rmgrdesc/heapdesc.c
index 7df18fa..b786955f 100644
--- a/src/backend/access/rmgrdesc/heapdesc.c
+++ b/src/backend/access/rmgrdesc/heapdesc.c
@@ -181,6 +181,8 @@ heap2_desc(StringInfo buf, XLogRecord *record)
appendStringInfo(buf, "rel %u/%u/%u; blk %u; %d tuples",
xlrec->node.spcNode, xlrec->node.dbNode, xlrec->node.relNode,
xlrec->blkno, xlrec->ntuples);
+ if (xlrec->flags & XLOG_HEAP_LAST_MULTI_INSERT)
+ appendStringInfoString(buf, " (last)");
}
else if (info == XLOG_HEAP2_LOCK_UPDATED)
{
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index 1734ec9..8f8732a 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -802,8 +802,16 @@ DecodeMultiInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
tuple->header.t_hoff = xlhdr->t_hoff;
}
- /* reset toast reassembly only after the last chunk */
- change->data.tp.clear_toast_afterwards = (i + 1) == xlrec->ntuples;
+ /*
+ * Reset toast reassembly state only after the last row in the last
+ * xl_multi_insert_tuple record emitted by one heap_multi_insert()
+ * call.
+ */
+ if (xlrec->flags & XLOG_HEAP_LAST_MULTI_INSERT &&
+ (i + 1) == xlrec->ntuples)
+ change->data.tp.clear_toast_afterwards = true;
+ else
+ change->data.tp.clear_toast_afterwards = false;
ReorderBufferQueueChange(ctx->reorder, r->xl_xid,
buf->origptr, change);
diff --git a/src/include/access/heapam_xlog.h b/src/include/access/heapam_xlog.h
index 05beb00..e964ecc 100644
--- a/src/include/access/heapam_xlog.h
+++ b/src/include/access/heapam_xlog.h
@@ -69,6 +69,8 @@
#define XLOG_HEAP_CONTAINS_NEW_TUPLE (1<<4)
#define XLOG_HEAP_PREFIX_FROM_OLD (1<<5)
#define XLOG_HEAP_SUFFIX_FROM_OLD (1<<6)
+/* last xl_heap_multi_insert record for one heap_multi_insert() call */
+#define XLOG_HEAP_LAST_MULTI_INSERT (1<<7)
/* convenience macro for checking whether any form of old tuple was logged */
#define XLOG_HEAP_CONTAINS_OLD \
--
2.0.0.rc2.4.g1dc51c6.dirty
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
