On 26/09/17 10:03, Craig Ringer wrote:
On 26 September 2017 at 14:08, Alvaro Hernandez <a...@ongres.com
<mailto:a...@ongres.com>> wrote:
OK, let me try to do that. I believe data integration is a
priority.
Definitely agree so far.
- If you want to develop your own output plugin, then your market
is reduced as you have to exclude all the managed solutions
(until, and only if, you would convince them to include your
plugin... highly unlikely, very difficult). And probably another %
of enterprise environments which will hesitate to link your own
plugin to the running production database. Last but not least, you
need to compile and test (and testing this is far from easy) on
multiple OSes/architectures.
Right. You probably don't want one output plugin per application.
This doesn't mean it has to be in-core, just flexible and share-able
by many, and adopted by cloud vendors. Like say PostGIS.
That would be nice. But this is chicken-and-egg: an out-of-core
plugin won't probably be used by many if applications like the ones I
was mentioning if they do not exist. And developing such an application
is so much less interesting if a significant part of your market is
excluded from your app.
However, it could work the other way around: a sufficiently good
enough in-core base plugin could foster applications/ecosystem, which
once adopted by users could push much more easily for other more
advanced out-of-core plugins, that would be more easily accepted by
pressure as those tools would already be with significant traction. But
I don't see it the other way around.
- If you stick to in-core plugins, then you need to support at
least three different output formats if you want to support 9.4+:
test_decoding (and pray it works!), pgoutput, and the "new"
in-core plugin that was proposed at the beginning of this thread,
if that would see the light.
The only practical way will IMO be to have whatever new plugin it also
have an out-of-core version maintained for older Pg versions, where it
can be installed.
But only in-core plugins help for general-purpose solutions.
I still don't agree there. If there's enough need/interest/adoption
you can get cloud vendors on board, they'll feel the customer
pressure. It's not our job to create that pressure and do their work
for them.
Don't want to get into a loop, but as I said before it's
chicken-and-egg. But nobody is asking core to do their work. As much as
I love it, I think logical decoding is a bit half-baked until there is a
single, quality, in-core plugin, as it discourages its usage, because of
the reasons I stated.
I see nothing wrong with a plugin starting off out of core and being
adopted+adapted later, assuming it's done well.
That said, I'm all in favour of a generic json output plugin that
shares infrastructure with logical replication, so people who are on
inflexible environments have a fallback option. I just don't care to
write it.
That's better than nothing. But as much as interoperable json may
be, people still need to talk the (binary) replication protocol to use
it. So once you talk binary protocol, why not talk binary also for the
output plugin and have a much more efficient output? Again, nothing
against json, but if a new plugin would be included in-core, I'd say
json + binary also. Or just document pgoutput, as it could be good enough.
Cheers,
Álvaro
--
Alvaro Hernandez
-----------
OnGres