Best method to achieve this would be to implement this in front of ES so the bulk indexing client runs only at the time it should run.
For the gathering plugin which I am working on, I plan to separate the two phases of gathering documents and indexing documents. So, by giving a scheduling option, it will be possible to index (or even reindex) gathered documents at a later time, for example, documents are continuously collected from various sources, like JDBC, web, or file system, and then indexed at some later time (for example at night). Such collected documents will be stored in an archive format at each gatherer node, like the archive formats supported in the knapsack plugin. Jörg On Sun, Feb 23, 2014 at 6:52 AM, vineeth mohan <[email protected]>wrote: > Hi , > > I am doing a lot of bulk insert into Elasticsearch and at the same time > doing lots of read in another index. > > Because of the bulk insert my searches on other index are slow. > > It is not very urgent that these bulk indexes actually gets indexed and > are immediately searchable. > > Is there anyway , I can ask Elasticsearch to receive the bulk inserts but > do the actual indexing ( Which should be the CPU consuming part ) later. > > I figured out that Elasticsearch would wait for 1 second before making the > documents searchable. > Here , what is it waiting for ? Is it to index the document or reopening > the indexWriter ? > Will it help me if i can configure this 1 second to 1 hour ? > If so , which parameter should i tweak. > > Kindly let me know if there are any other similar features out there which > can be of any help. > > Thanks > Vineeth > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CAGdPd5kwxwB%2Bi%3DHZDS1y%2B6Ad-VTax8hLSpgSVaSNH7CbzagB3Q%40mail.gmail.com > . > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFFewVHjeoEyZVktYEEqtbBXoD4VH3K-Tx9KAh%3DTfj%3D1Q%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
