You must initiate scan/scroll with search type SCAN. The scan/scroll
pattern is like this
SearchRequest searchRequest = new
SearchRequestBuilder(client).setQuery(QueryBuilders.matchAllQuery()).request();
searchRequest.searchType(SearchType.SCAN).scroll(request.getTimeout());
SearchResponse searchResponse = client.search(searchRequest).actionGet();
// get total hits here before entering the loop
while (searchResponse.getScrollId() != null) {
searchResponse =
client.prepareSearchScroll(searchResponse.getScrollId())
.setScroll(request.getTimeout()).execute().actionGet();
long hits = searchResponse.getHits().getHits().length;
// process hits of a scroll here
}
Jörg
On Mon, Nov 10, 2014 at 1:27 PM, Yarden Bar <[email protected]> wrote:
> One issue I identified is the heap size was too small for the query, I've
> increased the heap memory and the CircuitBreakerException stopped happening.
>
> But the scrolling still returning the SAME result.
>
> An updated code example is below:
> import org.elasticsearch.action.search.SearchType
> import org.elasticsearch.client.transport.TransportClient
> import org.elasticsearch.common.settings.ImmutableSettings
> import org.elasticsearch.common.transport.InetSocketTransportAddress
> import org.elasticsearch.common.unit.TimeValue
> import org.elasticsearch.index.query.{FilterBuilders, QueryBuilders}
> import org.elasticsearch.search.Scroll
> import org.elasticsearch.search.sort.SortOrder
>
> val es_settings = ImmutableSettings.settingsBuilder().put(
> "transport.sniff", true).put("cluster.name", "test_acm_es").build()
> var client = new TransportClient(es_settings).addTransportAddress(new
> InetSocketTransportAddress("myServer",9300))
> val query = QueryBuilders.filteredQuery(QueryBuilders.matchAllQuery(),
> FilterBuilders.queryFilter(
> QueryBuilders.queryString("((market:2 AND feed:55) OR (market:2 AND
> feed:32))")))
> var result = client.prepareSearch("orderbook-2014.11.03").setTypes(List(
> "level"):_*).setQuery(query).setSearchType(SearchType.DFS_QUERY_THEN_FETCH
> ).setSize(10000).addSort("updateNo", SortOrder.ASC).setScroll(new Scroll(
> TimeValue.timeValueMinutes(5))).get()
> var scrollId = ""
> var itr = 0
> do {
> scrollId = result.getScrollId
> result = client.prepareSearchScroll(scrollId).setScroll(TimeValue.
> timeValueMinutes(3)).get()
> println(s"Iteration=$itr, scrollResult=${result.getHits.getHits.length}")
> // println("------------------------------------")
> // result.getHits.getHits.foreach(h => println(h.getId))
> // println("------------------------------------")
> itr+=1
> } while (result.getHits.getHits.length != 0)
>
> enabling the print block reveals that the searchHit array is the same for
> each iteration...
>
> Thanks,
> Yarden
>
> On Wednesday, November 5, 2014 2:48:46 PM UTC+2, Yarden Bar wrote:
>
>> Hi all,
>>
>> I'm encountering a strange behavior when executing a search-scroll on a
>> single node of ES-1.3.4 with Java client.
>>
>> The scenario is as follows:
>>
>> 1. Start a single node of version 1.3.4
>> 2. Add snapshot repository pointing to version 1.1.1 snapshots
>> 3. Restore snapshots version 1.1.1 snapshot to 1.3.4 node
>> 4. Execute search on an index with
>> 5. client.prepareSearch("my_index*").setQuery(QueryBuilders.
>> filteredQuery(QueryBuilders.matchAllQuery(), FilterBuilders.
>> queryFilter(
>> QueryBuilders.queryString(s"$terms AND
>> snapshotNo:[${mdp.fromSnapshot}
>> TO ${mdp.toSnapshot}]") )) )
>> .addFields(OBFields.values.map(_.toString).toList: _*).setSize(
>> pageSize).addSort(OBFields.updateNo.toString, SortOrder.ASC)
>> .setScroll(TimeValue.timeValueMinutes(3)).execute().actionGet
>> ()
>>
>>
>> 6. Execute the following search scroll
>> client.prepareSearchScroll(scrollId).setScroll(TimeValue.tim
>> eValueMinutes(3)).execute().actionGet()
>>
>> I have a loop iterating over #6, providing the same scrollId and checking
>> for (result.getHits().getHits().legth == 0) to terminate.
>> I keep getting the same result 'page' with the same amount of results.
>>
>>
>> Any Idea??
>>
>>
>> Thanks,
>> Yarden
>>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/66e02775-17dd-4ea0-a8b3-39eb7e2a7aca%40googlegroups.com
> <https://groups.google.com/d/msgid/elasticsearch/66e02775-17dd-4ea0-a8b3-39eb7e2a7aca%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGgEhDf210fHVx%2BNqj-qFc5xu32zTp9FkK3W1Dtpi%3DJgg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
