[
https://issues.apache.org/jira/browse/METAMODEL-80?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14152882#comment-14152882
]
Kasper Sørensen commented on METAMODEL-80:
------------------------------------------
The atomic boolean is mostly just lazyness because I like the compareAndSet
method - ensures that we only close the dataset once, even in case of race
conditions. But it's probably over-engineered since I doubt anyone would try to
concurrently close it :-)
The part about the next() method is that the hits-array is only a page of the
total result. By default ES only provides up to 10 hits in a single page. So if
you query something that yields more than 10 documents/rows, then you need to
keep scrolling when you're finished with the page that is currently in memory.
> Change the ElasticSearch DataSet cursor implementation
> ------------------------------------------------------
>
> Key: METAMODEL-80
> URL: https://issues.apache.org/jira/browse/METAMODEL-80
> Project: Metamodel
> Issue Type: Improvement
> Reporter: Alberto Rodriguez
> Assignee: Alberto Rodriguez
> Priority: Minor
>
> The ElasticSearch DataSet implementation is currently eagerly loading all the
> results into an array. This is a potential problem for large datasets that
> could cause out of memory exceptions.
> We should get an active cursor instead the current one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
