How many of these 100M documents have been indexed with Elasticsearch 1.2.0? The tool would only send indexing requests for documents that have been misrouted, so if you only see 8 indexing requests per second, which is indeed low, that might mean that the bottleneck is searching for the mis-routed documents.
On Thu, Jun 12, 2014 at 8:53 AM, Ralph Meijer <[email protected]> wrote: > On 2014-06-11 10:50, Adrien Grand wrote: > > On Tue, Jun 10, 2014 at 11:44 PM, Ralph Meijer <[email protected]> wrote: > >> > >> * Does using explicit routing (in this cased defined in mapping to use > a > >> customer id field) have the same routing issue? > >> > > > > Yes. The issue is on the hashing function, so it occurs when using > explicit > > routing as well. > > [..] > > Ok, thanks! > > I've been running the tool, but looking at my graphs, it looks like its > copy_if_missing function processes about 8 documents a second, which > seems really slow compared to the index rates I normally have. Because > this is just one daily index with over a 100 million entries, completing > the sequence is going to take forever. Is there a way to speed this up? > > If not, I'll probably stop the process and reindex using the _source > field, taking the copies for granted. > > -- > ralphm > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/53994E4F.1090704%40ik.nu. > For more options, visit https://groups.google.com/d/optout. > -- Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j75UgRQPt4OqTFX0jFY85kNThVbxo5x_Ka9bFNc_zNaaQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
