Hello all, To address this issue, I have created the Jira ticket OFBIZ-13366 <https://issues.apache.org/jira/browse/OFBIZ-13366>. All the relevant details, analysis, and the proposed patch are available in the Jira ticket itself. The issue occurs because the executeFind service fetches records using the maxRows value instead of applying proper pagination. As a result, when navigating to later pages, an increasingly large number of records are fetched from the database. For example, navigating to the last page of a dataset containing millions of records causes all records to be loaded into memory, eventually leading to an OutOfMemoryError.
To resolve this issue, we have updated the implementation to use LIMIT and OFFSET so that only the required records are fetched from the database for each page request instead of loading unnecessary records into memory. Thanks & regards, Pushpraj Singh Jadoun On Sat, 23 Nov 2024 at 14:10, Pushpraj Singh Jadoun < [email protected]> wrote: > Hello Deepak, > > I believe this issue occurs because all the records are being fetched from > the database, and handling such a large dataset in OFBiz is causing an > *OutOfMemoryError*. > > I also tried unchecking the "Ignore Case" option, but the error persists. > > The *executeFind* service is responsible for querying results from the > database. I noticed that it uses the *maxRows* method to limit the number > of records fetched. For example, on the first page, 20 records are shown by > default, so only 20 records are fetched from the database using the > *maxRows* method. However, when navigating to the next page, 40 records > are fetched instead of just the next 20, because the *maxRows* value > passed is 40. Similarly, when navigating to the last page of a dataset > containing 3 million records, the *maxRows* value passed is 3 million, > causing all 3 million records to be fetched from the database. > > Would it be possible to use *offset* and *limit* to fetch only the > necessary records from the database, rather than retrieving unnecessary > ones? > > Thanks & regards > Pushpraj SIngh Jadoun > > On Thu, 17 Oct 2024 at 12:40, Deepak Dixit <[email protected]> wrote: > >> Hi Puspraj, >> >> When enabled, the "Ignore Case" option converts field names to uppercase >> before querying. If there's any case difference in the field names, the >> database index may not function as expected, which causes the query to >> traverse the entire table to filter results. This can significantly impact >> performance. >> >> By default ignore cases should be unchecked!! >> >> >> Thanks & Regards >> -- >> Deepak Dixit >> ofbiz.apache.org >> >> >> On Wed, Jun 5, 2024 at 3:44 PM Pushpraj Singh Jadoun < >> [email protected]> wrote: >> >> > Hello Nicolas, >> > >> > I tested it with the following configuration: >> > org.gradle.jvmargs=-Xms128m -Xmx8192m -XX:+CMSClassUnloadingEnabled >> > >> > Kind regards, >> > Pushpraj SIngh Jadoun >> > >> > On Wed, 5 Jun 2024 at 13:32, Nicolas Malin <[email protected]> >> > wrote: >> > >> >> Hello Pushpraj, >> >> >> >> How memory have you set at XMX value ? >> >> >> >> Nicolas >> >> >> >> Le 17/05/2024 à 17:08, Pushpraj Singh Jadoun a écrit : >> >> >> >> Hello all, >> >> >> >> I am facing an issue while fetching a large number of datasets (over 20 >> >> million records) from an entity. Specifically, I receive the following >> >> error: >> >> >> >> "Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded" >> >> >> >> >> >> This error occurs when I attempt to navigate to the last page using the >> >> entity engine search page in Webtools. I have attached a screenshot >> for you >> >> to look over. >> >> [image: image-18- (1).png] >> >> >> >> >> >> >> >> [image: MAIL.png] >> >> >> >> >> >> Has anyone experienced a similar issue or can you offer advice on >> >> resolving this? >> >> >> >> Kind regards, >> >> Pushpraj SIngh Jadoun >> >> >> >> >> >> >> >
