Almost certainly as Wayne suggests your bottleneck is IO. The default storage is EBS which is a type of network SAN. Some instance types have "EBS Optimized" which you should try. This gives a dedicated network channel to EBS. Then add RAID across the EBS for extra fun.
Even better as Wayne suggests is instances with "Provisioned IOPS" or some of the truly amazing DB oriented instances with tons of local storage. Also you could consider using Ephemerial Storage, however as the name suggests it will not last beyond the instance life. ----------------------------------------------------------------------------- David Lee Lead Engineer MarkLogic Corporation [email protected] Phone: +1 812-482-5224 Cell: +1 812-630-7622 www.marklogic.com -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Wayne Feick Sent: Tuesday, January 08, 2013 7:20 AM To: General Mark Logic Developer Discussion Subject: Re: [MarkLogic Dev General] MarkLogic in AWS Cloud I don't have a lot of experience with it, but EBS volumes have limited bandwidth. Some people have had success striping across multiple EBS volumes from within Linux instances. You could also look at the more recent guaranteed IOPs capability Amazon now offers. Wayne Ron Hitchens <[email protected]> wrote: Has anyone had any experience configuring and running non-trivial MarkLogic clusters in the cloud? Specifically Amazon EC2 VMs? I've got a test cluster of three nodes setup in AWS and am trying to figure out the best configuration for it. The system seems to be quite slow at some things, but reasonably fast at others. Bumping the VM up to bigger instances (more ram, more cores) doesn't seem to have a significant impact on speed or throughput. I suspect I/O bandwidth may be the culprit, but that's just a hunch. Does anyone have any experience with tuning EC2 VMs? The test environment I'm working with now is three m2.xlarge instances (32gb RAM, 4 cores, "high" network speed). The OS is Windows (groan, I don't have a choice there). Production cluster(s) are likely to be similar, but probably six nodes or so. Any advice//war stories/dire warnings greatly appreciated. Thanks. --- Ron Hitchens {mailto:[email protected]} Ronsoft Technologies +44 7879 358 212 (voice) http://www.ronsoft.com +1 707 924 3878 (fax) Bit Twiddling At Its Finest "No amount of belief establishes any fact." -Unknown _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
