I'm trying to work out the best way to deploy a system
I'm designing into the cloud on AWS. We've been through
various permutations of AWS configurations and the main
thing we've learned is that there is a lot of uncertainty
and unpredictability around I/O performance in AWS.
It's relatively expensive to provision guaranteed, high
performance I/O. We're testing an SSD solution at the
moment, but that is ephemeral (lost if the VM shuts down)
and very expensive. That's not a deal-killer for our
architecture, but makes it more complicated to deploy
and strains the ops budget.
RAM, on the other hand, is relatively cheap to add to
and AWS instance. The total database size, at present, is
under 20GB and will grow relatively slowly. Provisioning
an AWS instance with ~64GB of RAM is fairly cost effective,
but the persistent EBS storage is sloooow.
So, I have two questions:
1) Is there a best practice to tune MarkLogic where
RAM is plentiful (twice the size of the data or more) so
as to maximize caching of data. Ideally, we'd like the
whole database loaded into RAM. This system will run as
a read-only replica of a master database located elsewhere.
The goal is to maximize query performance, but updates of
relatively low frequency will be coming in from the master.
The client is a Windows shop, but Linux is an approved
solution if need be. Are there exploitable differences at
the OS level that can improve filesystem caching? Are there
RAM disk or configuration tricks that would maximize RAM
usage without affecting update persistence?
2) Given #1 could lead to a mostly RAM-based configuration,
does it make sense to go with a single high-RAM, high-CPU
E+D-node that serves all requests with little or no actual I/O?
Or would it be an overall win to cluster E-nodes in front of
the big-RAM D-node to offload query evaluation and pay the
(10-gb) network latency penalty for inter-node comms?
We do have the option of deploying multiple standalone
big-RAM E+D-nodes, each of which is a full replica of the data
from the master. This would basically give us the equivalent
of failover redundancy, but at the load balancer level rather
than within the cluster. This would also let us disperse
them across AZs and regions without worrying about split-brain
cluster issues.
Thoughts? Recommendations?
---
Ron Hitchens {mailto:[email protected]} Ronsoft Technologies
+44 7879 358 212 (voice) http://www.ronsoft.com
+1 707 924 3878 (fax) Bit Twiddling At Its Finest
"No amount of belief establishes any fact." -Unknown
_______________________________________________
General mailing list
[email protected]
http://developer.marklogic.com/mailman/listinfo/general
