There are some new options in the latest builds of ElasticSearch as I understand it that replace the old S3 Gateway.
However, neither the S3 Gateway nor those others are requirements for setting up ElasticSearch on EC2. They are only disaster recovery options that will help you to get back up and running in the event of a failure. Similar capability can be had using EBS and periodic snapshots. Running ElasticSearch on EC2 feels natural and works well. The Elastic capabilities of both systems come together very nicely. It is easy to add and remove nodes to grow and shrink your capacity. Probably the biggest pitfall is making sure that you give your cluster a unique name, or you may end up with your nodes joining someone else's ElasticSearch cluster. I saw this when I brought my first ES node online on EC2. The traditional Zen auto discovery will work on EC2, especially if using VPC where you can control your own subnets. There is also a plugin that allows using the Amazon instance tags to facilitate auto discovery. http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-ec2.html#modules-discovery-ec2. Don't get so hung up that the S3 Gateway has been dropped. You can easily deploy using EBS volumes. If you have large indexes that will not fit in memory or high indexing rates, you will need to use an EBS Optimized instance and provisioned IOPS. You can also use multiple EBS volumes and software raid(mdadm) in order to increase IO, with or without provisioned IOPS. On Sunday, July 27, 2014 4:47:20 AM UTC-6, vjbangis wrote: > > Actually I'm following that tutorials but according to this link > <https://groups.google.com/forum/#!topic/elasticsearch/x8BjtiMTSt4>. > The "S3 Gateway" has been dropped, so you'll either need to use EBS, or > set up some mechanism to do snapshots to S3. Other than that, no major > changes. > > > On Sunday, July 27, 2014 6:43:08 PM UTC+8, Mark Walkom wrote: >> >> There's a bit more to it, but yes that is the general idea. >> >> I'm not familiar with ES on EC2 though so I can't give you any >> directions, though something like >> http://www.elasticsearch.org/tutorials/elasticsearch-on-ec2/ may help. >> >> Regards, >> Mark Walkom >> >> Infrastructure Engineer >> Campaign Monitor >> email: [email protected] >> web: www.campaignmonitor.com >> >> >> On 27 July 2014 20:32, vjbangis <[email protected]> wrote: >> >>> Yup, I have one node acting as ELK stack running on EC2 and I want to >>> add more nodes. >>> >>> With that single node (ELK), if I run another EC2 instance and installed >>> ES only would that make a cluster? >>> >>> >>> On Sunday, July 27, 2014 5:47:53 PM UTC+8, Mark Walkom wrote: >>>> >>>> Can you elaborate a bit more, do you only have one node now? >>>> >>>> It is simple to expand a single node to multiple. >>>> >>>> Regards, >>>> Mark Walkom >>>> >>>> Infrastructure Engineer >>>> Campaign Monitor >>>> email: [email protected] >>>> web: www.campaignmonitor.com >>>> >>>> >>>> On 27 July 2014 19:30, arshpreet singh <[email protected]> wrote: >>>> >>>>> On Sun, Jul 27, 2014 at 2:58 PM, vjbangis <[email protected]> wrote: >>>>> > If I have ELK stack running on EC2. How can I make the ES as a >>>>> cluster? >>>>> >>>>> They say it is super easy to do/find. >>>>> http://stackoverflow.com/questions/16821101/how-to-set-up-es-cluster >>>>> >>>>> -- >>>>> >>>>> Thanks >>>>> Arshpreet singh >>>>> http://arshpreetsingh.wordpress.com/ >>>>> >>>>> GOD,Nature,Mathematics,Programming. >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "elasticsearch" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit https://groups.google.com/d/ >>>>> msgid/elasticsearch/CAAstK2GrS%3DnsPXXkOg% >>>>> 3DHMWqU4MibdGaULqD0psx3RQtr1qw_SA%40mail.gmail.com. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/b5e04682-f1a1-4799-a977-fd19984c3530%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/elasticsearch/b5e04682-f1a1-4799-a977-fd19984c3530%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/964816a3-0b65-4ddc-b0b5-92e933dd0b24%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
