Hello Michael - Thanks for the configuration.
Hello Jörg - I was thinking more in lines of translog -
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-translog.html
I believe the index operation is first written to translog ( Which i am not
sure if is a part of lucene ) and then written to lucene later.
Here if we can ask ES , to accumulate a huge amount of feeds to index and
index it later , will that do the trick ?
Thanks
Vineeth
On Sun, Feb 23, 2014 at 7:03 PM, Michael Sick <
[email protected]> wrote:
> Also, if there are no other clients wanting a faster refresh, you can
> set index.refresh_interval to a higher value than the 1s default either in
> general for your index or just during the times when you're doing your bulk
> updates.
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules.html
>
>
> On Sun, Feb 23, 2014 at 8:28 AM, [email protected] <
> [email protected]> wrote:
>
>> Best method to achieve this would be to implement this in front of ES so
>> the bulk indexing client runs only at the time it should run.
>>
>> For the gathering plugin which I am working on, I plan to separate the
>> two phases of gathering documents and indexing documents. So, by giving a
>> scheduling option, it will be possible to index (or even reindex) gathered
>> documents at a later time, for example, documents are continuously
>> collected from various sources, like JDBC, web, or file system, and then
>> indexed at some later time (for example at night). Such collected documents
>> will be stored in an archive format at each gatherer node, like the archive
>> formats supported in the knapsack plugin.
>>
>> Jörg
>>
>>
>>
>> On Sun, Feb 23, 2014 at 6:52 AM, vineeth mohan <[email protected]
>> > wrote:
>>
>>> Hi ,
>>>
>>> I am doing a lot of bulk insert into Elasticsearch and at the same time
>>> doing lots of read in another index.
>>>
>>> Because of the bulk insert my searches on other index are slow.
>>>
>>> It is not very urgent that these bulk indexes actually gets indexed and
>>> are immediately searchable.
>>>
>>> Is there anyway , I can ask Elasticsearch to receive the bulk inserts
>>> but do the actual indexing ( Which should be the CPU consuming part ) later.
>>>
>>> I figured out that Elasticsearch would wait for 1 second before making
>>> the documents searchable.
>>> Here , what is it waiting for ? Is it to index the document or reopening
>>> the indexWriter ?
>>> Will it help me if i can configure this 1 second to 1 hour ?
>>> If so , which parameter should i tweak.
>>>
>>> Kindly let me know if there are any other similar features out there
>>> which can be of any help.
>>>
>>> Thanks
>>> Vineeth
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CAGdPd5kwxwB%2Bi%3DHZDS1y%2B6Ad-VTax8hLSpgSVaSNH7CbzagB3Q%40mail.gmail.com
>>> .
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFFewVHjeoEyZVktYEEqtbBXoD4VH3K-Tx9KAh%3DTfj%3D1Q%40mail.gmail.com
>> .
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAP8axnCq-PE%3Du0ZSC6d7rDxME%3DpkzpBo%3D9-tq_rT%2BCZjQgzFxg%40mail.gmail.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAGdPd5%3DMvTYj0amH46nkm%3DkAEZ6HS2yaAYX5fadS7vaY6cmRvw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
