Hi Thomas Corthals,
After experimenting with a page size of 100,000 records, I successfully
retrieved approximately 19 million records and saved the response to a PDF
file from our application. However, I noticed that the Solr heap memory did
not release after completing the activity. Subsequent
Hi Prasad,
Have you tried with a smaller page size? Just how many documents you can
fetch in one page with the given memory depends on the size of the
documents. You'll have to try out what works for you.
Regardless of page size a cursor will still be the way to go to page
through a large set of
Hello Thomas Corthals,
Thank you very much for your valuable reply. I am trying to use cursors,
but for the first query also its taking so much time to get the results,
and next query I am getting heap memory error in my java application.
On Fri, Apr 5, 2024 at 2:41 PM Thomas Corthals
wrote:
Hi Prasad,
This is expected with "deep paging":
https://solr.apache.org/guide/solr/latest/query-guide/pagination-of-results.html#performance-problems-with-deep-paging
Have a look at cursors instead, that should solve your problem:
Hi,
Solr usually fills the heap with various caches so I wouldn't worry much about
it consuming %90 of the heap, unless I get OutOfMemory errors.
Pagination using rows parameter is intended for when row count is very low and
page number is also small (eg. rows=10 page=2 etc.). It's problematic
