I am quite new to ElasticSearch, most of what I need to seems to be easily supported however I have hit one problem - specifically searching by temporal proximity.
What do I mean by this? I'll explain a simplified scenario. I have indexed events each with an associated date-time. I need to discover those events that occur within a given time period *of each other*. Ideally this time period would be arbitrary and specified at search time. The "of each other" part is key here. I cannot just, say: - Aggregate by week or month, as an event that occurs at the end of one month is within a month of one which occurs at the start of the next month but would fall into a different bucket. - Do a date range based search, as I do not have a fixed range to search. To describe what I need in a different way, in SQL I'd do something like: SELECT eventid, eventdate from Events t1 where EXISTS( SELECT 1 FROM Events t2 WHERE t1.eventid <> t2.eventid AND t1.eventdate BETWEEN dateadd( day, -30, t2.eventdate ) AND dateadd( day, 30, t2.eventdate ) ); The actual scenario is more complex - each event has a type and what I need ultimately is to be able to answer questions of akin to: "Find events of type X that occur with in 2 days of an event of type Y" and even: "Find events of type X that occur with in 2 days of an event of type Y and of type Z" Each event will be nested in/a child of, a parent record and I am only interested in the temporal proximity of events with the same parent. The database has a total of 10^9 events, each parent may have the order of 10^3 associated events. The use case is search heavy with ingests of deltas approximately weekly. I can munge the data on import in any way that would help. I have had a couple of ideas on how to tackle the problem but neither are satisfactory. I wondered whether there is a standard way to tackle this kind of requirement in ElasticSearch and whether anyone else had run up against it. Thanks, Paul. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/31bc7955-2043-49dd-865d-1dbc048a6dde%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
