More detail below, but the the crux of my question is: What's the best way to spin up/down "on demand" an ES cluster on EC2 that uses ephermal local storage? Essentially, I want to run the cluster during the week and spin down over the weekend. Other than brute force snapshot/restore, is there any more creative way to do this, like mirroring local storage to EBS or similar?
Some more background: We run multiple ES clusters on ec2 (we use opsworks for deployment automation). We started out several years back using EBS because we didn't know any better, and have switched over to using SSD based local storage. The performance improvements have been unbelievable. Obviously, using ephermal local storage comes at a cost: we use replication, take frequent snapshots, and store all source data to mitigate the risk of data loss. the other thing that local storage means is that our cluster essentially needs to be up and running 24/7, which I think is a fairly normal. I'm investigating some ways to save on cost for a large-ish cluster, and one of the things is that we don't need it to necessarily run 24/7; specifically, we want to turn the cluster off over the weekend. That said, restoring terabytes from snapshot doesn't seem like a very efficient way to do this, so I want to consider options, and was hoping the community could help me in identifying options that I am missing. thanks in advance for any thoughts you may have. --paul -- *Important Notice:* The information contained in or attached to this email message is confidential and proprietary information of RedOwl Analytics, Inc., and by opening this email or any attachment the recipient agrees to keep such information strictly confidential and not to use or disclose the information other than as expressly authorized by RedOwl Analytics, Inc. If you are not the intended recipient, please be aware that any use, printing, copying, disclosure, dissemination, or the taking of any act in reliance on this communication or the information contained herein is strictly prohibited. If you think that you have received this email message in error, please delete it and notify the sender. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/588e485c-6029-4ded-a3ce-a8dd01213510%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
