On 14/04/16 20:46, Robert Haas wrote:
I believe
logical replication is a fundamental database technology that should
be considered just as much within the score of the core product as
physical replication, parallel query, or UPSERT.

Agreed, I believed we need this for very long time as well (pglogical is not my first or even second logical replication project).

At present, I don't understand why we would do sharding via FDWs, i.e. an
out-of-core solution and yet replication as an in-core solution. Sharding
desires/requires a single system image, so tight coupling is sensible (for
example, defining a distribution key column on a distributed table). For
replication between disparate loosely coupled systems, tight coupling is
exactly what you do not want. So doing it that way round would give an an
out-of-core solution for something that is best done in-core and an in-core
solution for something best done out-of-core.

First, I think that replication can be either loosely-coupled or
tightly-coupled.  There are interesting cases with intermittently
connected networks where you really don't want too much coupling, and
then there are cases where you are doing load-balancing across a
cluster and tight coupling is fine, even desirable.   Similarly,
although I agree that a sharding solution intrinsically requires
fairly tight coupling, I think that one of the strengths of FDWs is
that they do not.  I'm not very interested in seeing a sharding
solution in PostgreSQL that limits what you can do to a particular
network topology and enforces tight coupling whether you want it or
not.  I'm more interested in seeing how we can build something that
*permits* a tightly-coupled system but also lets people build other
kinds of systems if they wish.

I think we need both tightly and loosely coupled and I think we should aim to solution that will make it reasonably possible to do both, ie without reimplementing whole thing again to get the other variant. In pglogical I was more concerned with loosely-coupled use-cases btw. Mainly because I've seen more of them in the wild and also they are more interesting for me personally (partially because many of the tightly coupled use-cases can be solved by physical replication). I don't think there is anything there that would fundamentally prevent us building something that can work for both scenarios though.

About doing things with SQL. I think some stuff would be better done using SQL personally, like the nodes table, since I dislike the fact that it's not shared catalog currently. But many other things I'd prefer to manage using functions at least in the first iteration. I think it's preferable to have simple function interface and make things work correctly and then maybe try to replace it with DDL where it makes sense because DDL just needs much more boiler plate but otherwise is mostly mechanical. One thing that worries me with SQL is that we do have some history of bike-shedding any SQL syntax that's not part of standard to death halting the actual feature development for prolonged time periods as a result.

Finally a side note about sharding - I have strong believe that sharding needs to be tightly coupled to be effective and maintainable in production.

  Petr Jelinek                  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services

Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:

Reply via email to