This is not entirely "pie in the sky", but feel free to tell me why this is 
crazy.  I
have had this idea for several years, but have not seen anyone else suggest it,
nor any arguments why it would not work.

If we had 64-bit Oids, we could reserve the top 16 bits (for instance) to 
indicate a
server ID.  Each server in a cluster could be configured with a unique serverID 
[0..65535].  Every table, index, etc that is created on a server could be 
an Oid congruent to this serverID modulus 65536.  Each server would still have a
total of 2^48 Oids that it could assign before reaching Oid exhaustion, which is
64k times more Oids than the current design.  (The 16bit / 48bit split is 
and could be 8bit / 56bit or whatever.)

Any INSERT, UPDATE, DELETE, INDEX, REINDEX, DROP that was run on a server
could be rejected if the object has the serverId of a different server in the 
The serverId could be kept at all times in memory, and Oids for database objects
are always looked up when handling these SQL operations anyway, so there
should be effectively zero overhead for rejecting invalid operations that 
to change on server X a table/index/whatever which belongs to server Y.
I don't see how to run such a setup in serializable mode, but with read 
it should be ok for server X to change its own tables based on what it perceives
to be the current state of data on server X and Y (reading local copies of 
from Y) at the same time that server Y changes its own tables based on what it
perceives to be the current state of data on server Y and X (reading local 
of tables from X).  The inconsistencies possible from these actions seem to me
the same as the inconsistencies possible when the two operations are happening
on the same server simultaneously in read committed mode.

WAL records from each server should only ever change objects belonging to that
server, so with some effort it might be possible for every server in the 
cluster to
replay the WAL records from every other server without collisions.  (This is 
the point
where my idea might be crazy -- I'm not sure how difficult/impossible it would
be to merge WAL information from multiple sources).

The total amount of WAL data that needs to be replayed on any given server would
be proportion to the total amount of data changes cluster-wide, which is no 
than the current state of affairs; it just happens to all come from a single 
master in
the current design.

The performance problems that at first seem inevitable in this design owing to 
need for each server to wait for wal replay from other servers to avoid data 
would only actually happen if updates were frequently based on reads from other
servers.  If the vast majority of data changes were not based on the contents of
remotely owned tables/indexes/sequences and such, then changes to local data
could proceed without stalling.

Therefore, this approach would probably be most appropriate for highly 
data, with each partition being owned by one server in the cluster, and 
based on data from more than one partition being rare.  (SELECTs based on data
from multiple partitions should be just fine.)

If you think about it, that's just an extension of the current architecture.  
all data is partitioned into a single partition belonging to the master, with 
zero partitions
on the slaves, and all updates are performed based on a single partition, that 
the one on the master.  It just isn't obvious that this is the design, because 
it is a
degenerate case of the more general case that I am describing.

Application software that wants to run INSERTS, UPDATE, etc. would have to 
to the appropriate server for the table in question, but that might not be too 
if the data is partitioned anyway -- just connect to the server with the 
partition you
want to change.

Thanks in advance for having even read this far....

Mark Dilger

Reply via email to