Re: ES-Hadoop data query liveness near real-time quantified?

2014-08-10 Thread Pierre WP
Hi Costin, thanks for the fast and detailed response. I don't have much direct experience with ES though, so I have basically no intuition about what the possible range of performance would be. Would you happen to have some useful case studies that could give me some kind of idea? What is

Re: ES-Hadoop data query liveness near real-time quantified?

2014-08-10 Thread Costin Leau
See http://www.elasticsearch.org/case-studies/ There are plenty of use case of organizations from all kind of industries. For IT audience, Github, Wikipedia or Stackoverflow give some insights into how ES is used at scale, on large volumes and data and what 'real-time' means. As for the

ES-Hadoop data query liveness near real-time quantified?

2014-08-01 Thread Pierre WP
I have a question about what near real-time means exactly, in a quantified way, when described this way on the ES-hadoop home page: We are happy to report that es-hadoop is being used in multiple data-intensive environments; in a recent example, a large financial institute that stores all of

Re: ES-Hadoop data query liveness near real-time quantified?

2014-08-01 Thread Costin Leau
Using the RTC definitions, ES, Hadoop, the JVM and the popular OS themselves are soft/near real-time systems - so if you are coming from a hard/firm RT system, you can safely assume that everything (and again not just ES) is soft. As a tangent, very few systems are hard RT (ES is neither a nuclear