Hi Jörg, It looks like you know something about scan/scroll I haven't found documented elsewhere -- how to scan 60 million docs with 1000 documents per fetch, at constant time per fetch. Other comments I've seen indicate that, the deeper you get into fetching the results, the slower each fetch gets. I'm looking at alternatives for implementing a feature which will require scan/scroll on a similar scale, and knowing that what you've done is possible is critical for my planning. Could you please share the key parts of your setup/retrieve code, in addition to the configuration / version information you've already shared?
Thanks in advance, -Mark On Wednesday, November 27, 2013 8:03:46 AM UTC-6, Jörg Prante wrote: > > I executed a scan/scroll over 60 million docs, size of the indices (folder > 'data' size) is 87G. > > java version "1.7.0_25" > Java(TM) SE Runtime Environment (build 1.7.0_25-b15) > Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode) > > Heap is 2G > > Red Hat Enterprise Linux Server release 6.4 (Santiago) > > Jörg > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/705ede01-a19e-4c40-94d4-2e592de72303%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
