This may or may not help, but the following worked well for me. Just as any database-backed application, the business logic (such as what you described) is best implemented outside of the database. Since ES is a first-class Java citizen and its Java API is clean and superb (documentation shortcomings are more than made up for by the excellent advice from this newsgroup), this is what I did:
1. Create my business logic, and use the TransportClient to contact Elasticsearch. It's easy to test it in a stand-alone driver. 2. Create a server using Netty and the LMAX Disruptor. Netty is awesome, and I could craft the interface to accept the questions that made sense to the business, not to Elasticsearch. Then I instantiated an LMAX Disruptor in multi-producer mode, and each Netty thread simply created an event that contained 2 objects: The decoded HTTP request parameter, and the reference to the response output channel. 3. The Disruptor contained a fixed number of Worker handler threads (to match those in Elasticsearch) that actually converted the request into one or more Elasticsearch requests, performing whatever functions the business logic required, and then writing the responses back to the response channel. Even if my server got a huge burst of requests, they were quickly queued which means that the Netty handler thread count stays low. And then the back-end Disruptor thread max will ensure that Elasticsearch is never overloaded. This is kind of the hard way using Java, but it suited me fine. Other solutions could be similarly implemented with comperable performance but perhaps a lot less development time in Python with Django, or Node.js, or Ruby on Rails. But Java worked well for me; Netty is fast and easy to work with, and the LMAX Disruptor offers a near-zero latency nearly lockless queuing, and together they keep the thread count low while still handling many many more requests at one time. This sounds like a lot of work, but the hard work is a one-off kind of thing, and then the business logic can be easily tuned, fixed, and extended without much effort. Even things that Elasticsearch does not do and will never do (such as foreign keys and enforced consistency across types or even indices) are relatively easily done within the custom business logic of a front-end server. Again, I don't offer this as a strong suggestion but rather as a case study of what worked well for me. Brian -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a5011573-970d-4d4c-839d-a274e4d4cc34%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
