On 1.12.11 15:34, Jukka Zitting wrote:
But I don't think we should try to increase concurrency of
write operations within the *same* repository [...]
That's a valid solution, but it means that we explicitly drop the idea
of being able to scale concurrent writes linearly by adding new
cluster nodes. See also http://markmail.org/message/nuxclhqsbihanbjg
for the original discussion about this.
- in a cluster, writes do not scale as writing is basically synchronized
This I think is the key issue here. Do we want to break the
synchronization requirement and thus eliminate this bottleneck
entirely, or are we happy with more localized performance improvements
for the concurrent write case?
Ultimately the question boils down to whether we want to support
massively parallel systems where the number of concurrent writers
(measured as writes happening within the communication latency across
the entire cluster) starts at thousands and can ultimately scale go up
millions. Such systems are practically impossible to implement with
any non-zero level of global synchronization, which is why I think
this is such a crucial design decision to be made.
I'm fine if we explicitly decide not to target such use cases. Then we
can avoid having to worry about the complexities of an eventual
consistency model. With good optimization it's still possible to
achieve pretty impressive concurrent performance even with a
synchronized cluster.
One of the main goals of the Microkernel architecture was to get true
write scalability. If we go down that route, we have to accept that we
are trading this for consistency.
Michael
