Hi,
On 2014-03-11 15:57:39 -0400, Peter Eisentraut wrote:
Where, if anywhere, is the current documentation for writing or using a
logical decoding output plugin consumer thingy?
There's a pending patch for it. The corresponding commit is
http://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git;a=commit;h=5eeedd55b2d7e53b5fdcdab6a8e74bb666d75bcc
I welcome feedback about it. I've spent a fair bit of time immersed in
this stuff, and I am not really sure anymore what's understandable and
whatnot ;)
It's referencing pg_recvlogical which isn't committed yet (that's the
commit just before), that's why the docs weren't committed with the
feature itself.
src/backend/replication/logical/logical.c, which textually contains most
of the functions that appear to interact with the test_decoding module,
contains this in the header comment:
The idea is that a consumer provides three callbacks, one to read WAL,
one to prepare a data write, and a final one for actually writing since
their implementation depends on the type of consumer. Check
logicalfunc.c for an example implementations of a fairly simple consumer
and a implementation of a WAL reading callback that's suitable for
simpler consumers.
There is no file logicalfunc.c.
Hrmpf: There's a missing 's', it's logicalfuncs.c.
And test_decoding actually uses five callbacks, not three.
The callbacks logicalfuncs.c header comment is talking about adding a
new method to output data. Currently you can stream out changes via
walsender and via the SQL SRFs. But it might be interesting to
e.g. consume the changes in a bgworker, without going through either SQL
or walsender. To do that you need the three callbacks referenced above.
Is a consumer the same as a decoder?
A consumer is just the recipient of the changestream. I.e the walsender
that streams out the changestream, or the SRF that spills the data into
a tuplestore.
I don't think the term decoder is used anywhere, but if it is, it'd
be the output plugin.
test_decoding.c contains this:
/* These must be available to pg_dlsym() */
static void pg_decode_startup(LogicalDecodingContext *ctx,
OutputPluginOptions *opt, bool is_init);
...
which is surely wrong.
Hm, these days the comment should be above _PG_init() and
_PG_output_plugin_init(). That changed around several times...
Could you perhaps commit the attached patch fixing the issues you
mentioned?
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training Services
diff --git a/contrib/test_decoding/test_decoding.c b/contrib/test_decoding/test_decoding.c
index e356c7c..31aa012 100644
--- a/contrib/test_decoding/test_decoding.c
+++ b/contrib/test_decoding/test_decoding.c
@@ -33,6 +33,7 @@
PG_MODULE_MAGIC;
+/* These must be available to pg_dlsym() */
extern void _PG_init(void);
extern void _PG_output_plugin_init(OutputPluginCallbacks *cb);
@@ -43,7 +44,6 @@ typedef struct
bool include_timestamp;
} TestDecodingData;
-/* These must be available to pg_dlsym() */
static void pg_decode_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
bool is_init);
static void pg_decode_shutdown(LogicalDecodingContext *ctx);
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 13a22d4..04e407a 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -12,14 +12,17 @@
*together provide logical decoding, primarily by providing so
*called LogicalDecodingContexts. The goal is to encapsulate most of the
*internal complexity for consumers of logical decoding, so they can
- *create and consume a changestream with a low amount of code.
+ *create and consume a changestream with a low amount of code. Builtin
+ *consumers are the walsender and SQL SRF interface, but it's possible to
+ *add further ones without changing core code, e.g. to consume changes in
+ *a bgworker.
*
*The idea is that a consumer provides three callbacks, one to read WAL,
*one to prepare a data write, and a final one for actually writing since
*their implementation depends on the type of consumer. Check
- *logicalfunc.c for an example implementations of a fairly simple consumer
+ *logicalfuncs.c for an example implementation of a fairly simple consumer
*and a implementation of a WAL reading callback that's suitable for
- *simpler consumers.
+ *simple consumers.
*-
*/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
