Hey all,
I would like to create a plugin, and I need a hand. Below are the
requirements I have.
- Our documents are immutable. They are only ever created or deleted,
updates do not apply.
- We want mirrors of our ES cluster in multiple AWS regions. This way
if the WAN between regions is severed for any reason, we do not suffer an
outage, just a delay in consistency.
- As documents are added or removed they are rolled up then shipped in
batch to the other AWS Regions. This can be a fast as a few milliseconds,
or as slow as minutes, and will be user configurable. Note that a full
backup+load is too slow, this is more of a near realtime operation.
- This will sync the following operations.
- Index creation/deletion
- Alias creation/deletion
- Document creation/deletion
What I'm thinking architecturally.
- The plugin is installed on each node in our cluster in all regions
- The plugin will only gather changes for the primary shards on the
local node
- After the timeout elapses, the plugin will ship the changelog to the
other AWS regions, where the plugin will receive it and process it
Are there any api's I can look at that are a good starting point for
developing this? I'd like to do a simple prototype with 2 1 node clusters
reasonably soon. I found several plugin tutorials, but I'm more concerned
with what part of the ES api I can call to receive events, if any.
Thanks,
Todd
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dff53da5-8a0c-4805-8f97-72844019a79e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
