Supposedly doing RAID0 EBS volumes helps to mitigate some I/O issues. You could go that route and avoid having to refresh, assuming the performance would be acceptable.
On Tuesday, February 24, 2015 at 10:27:16 AM UTC-5, Paul Sanwald wrote: > > More detail below, but the the crux of my question is: What's the best way > to spin up/down "on demand" an ES cluster on EC2 that uses ephermal local > storage? Essentially, I want to run the cluster during the week and spin > down over the weekend. Other than brute force snapshot/restore, is there > any more creative way to do this, like mirroring local storage to EBS or > similar? > > Some more background: > We run multiple ES clusters on ec2 (we use opsworks for deployment > automation). We started out several years back using EBS because we didn't > know any better, and have switched over to using SSD based local storage. > The performance improvements have been unbelievable. > > Obviously, using ephermal local storage comes at a cost: we use > replication, take frequent snapshots, and store all source data to mitigate > the risk of data loss. the other thing that local storage means is that our > cluster essentially needs to be up and running 24/7, which I think is a > fairly normal. > > I'm investigating some ways to save on cost for a large-ish cluster, and > one of the things is that we don't need it to necessarily run 24/7; > specifically, we want to turn the cluster off over the weekend. That said, > restoring terabytes from snapshot doesn't seem like a very efficient way to > do this, so I want to consider options, and was hoping the community could > help me in identifying options that I am missing. > > thanks in advance for any thoughts you may have. > > --paul > > *Important Notice:* The information contained in or attached to this > email message is confidential and proprietary information of RedOwl > Analytics, Inc., and by opening this email or any attachment the recipient > agrees to keep such information strictly confidential and not to use or > disclose the information other than as expressly authorized by RedOwl > Analytics, Inc. If you are not the intended recipient, please be aware > that any use, printing, copying, disclosure, dissemination, or the taking > of any act in reliance on this communication or the information contained > herein is strictly prohibited. If you think that you have received this > email message in error, please delete it and notify the sender. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6513247e-5c4e-4fd6-9486-ba8b2245c575%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
