keith-turner commented on issue #3559:
URL: https://github.com/apache/accumulo/issues/3559#issuecomment-1654606194
> This issue and that one probably overlap a lot, as they are both very high
level and not clear on details.
I think this issue is very narrowly defined, its about making FATEs storage
more scalable so that FATE can be used more frequently than it is currently
used. Below are the details and background.
* All writes to zookeeper must go through a single server with a single
write ahead log and then be pushed to a quorum of other servers. Therefore
writes to ZK are not scalable (ZK is more scalable on the read side).
* FATE operations can be write heavy ZK operations. This could negatively
impact the general health of ZK on which accumulo depends for ZKs more scalable
read functionality.
* The metadata table is much more scalable for writes as each tabletserver
has its own walog and metadata tablets scan span many tablet servers.
* Using the metadata table for storage of FATE would increase the number of
FATE operations that could execute concurrently and reduce the general write
load on ZK.
* Making FATE storage more scalable would open it up to being used for
operations that it is currently not suitable for like per tablet compaction
commit. Making the fate storage more scalable may help improve user operations
like bulk import at scale also.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
