Thank you Ed, I'll remember that, about store index that way. 2015-01-13 17:29 GMT-04:30 Ed Kim <[email protected]>:
> Yea, since you mentioned real time, I thought you were concerned about > data coming in 'today.' > > For older data, the only aspect of real time that applies is how soon you > can search against ES once loaded. ES supports near real time so this isn't > really an issue, and considering you have 10 year old data, I had assumed > it's ok if the older data doesn't have to be readily available until the > entire data set is loaded, especially since the intent for these type of > things is to get an overall view of all the data points. > > Anyways, one thing I did in the past was to partition the historical data > (in oracle) and then index the data into ES in time intervals. For example, > store 1 year's worth of data in a single index. If doing it this way, you > will have 10 indexes, and for query purposes, you can set aliases on the > indexes and control your query using aliases. For instance, if most of your > queries for analysis are for 'the last year', you can query against the > index that contains data for 2014 and 2015. This way you don't query a > giant index that can go back 10 years. Also, by breaking down into chunks, > any indexing failures would not cause me to re-index the entire thing, > which could take a very long time. > > Best wishes on your project! > > On Tuesday, January 13, 2015 at 1:07:31 PM UTC-8, Marian Valero wrote: >> >> Thanks Ed. >> >> Yes, but I have data from 10 years ago and I have to input this data too >> for analyze that logs with the data that is insert every day. There are >> millions of logs. >> >> 2015-01-13 16:22 GMT-04:30 Ed Kim <[email protected]>: >> >>> Not sure if this is an option for you, but if an application is feeding >>> that log data into oracle, you could consider having that application also >>> index into ElasticSearch. >>> >>> On Wednesday, January 7, 2015 at 5:46:18 AM UTC-8, Marian Valero wrote: >>>> >>>> I'm reading data from my oracle database to java with JDBC but I want >>>> to know how can I input the data that I'm getting in Elasticsearch in real >>>> time. >>>> >>>> Thanks. >>>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "elasticsearch" group. >>> To unsubscribe from this topic, visit https://groups.google.com/d/ >>> topic/elasticsearch/cnkq84aom1M/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> To view this discussion on the web visit https://groups.google.com/d/ >>> msgid/elasticsearch/86e937ac-af3f-43f7-8c1f-60c7517b8cbf% >>> 40googlegroups.com >>> <https://groups.google.com/d/msgid/elasticsearch/86e937ac-af3f-43f7-8c1f-60c7517b8cbf%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/elasticsearch/cnkq84aom1M/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/1fd5e8fb-117e-419b-b5b7-cda96a55abec%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/1fd5e8fb-117e-419b-b5b7-cda96a55abec%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADT%2BR%3DZ2s4o%3De_Px8PzPEjgcftM3%2BxtKLSJ6EQPrtfRS_BiMaA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
