Hi Wayne, Thank your for correcting me however I was just suggesting some options to improve performance after solving that issue as he said, he is only one forest in the DB with TB data.
Regards, Indy On Mon, May 11, 2015 at 11:57 PM, Wayne Feick <[email protected]> wrote: > Creating more forests won't help, because they'll all have the same > in-memory limits and the rebalancer will run into the same issue as the > reindexer if it tries to move the document to a different forest. > > Wayne. > > > > On 05/11/2015 11:18 AM, Indrajeet Verma wrote: > > Shashi, > > I am sure, this file size must be large. > > This is not good that you have created only one forest. There should be > more than that. I would recommend around 8-10 forests. > > However these should be based on core of CPUs. per 2 core 1 forest but > if your server is having 32 cores, 8-10 forest should be sufficient. Please > correct somebody if I am wrong. > > As per my understanding you should to do following things to solve your > problem, > > 1. Delete this large file size. I would not recommend to increase memory > size just for the heck. Your performance will be degraded. > > 2. Create more forest and attach to the database. if you are using 7+, > this automatically re-balance the data. > > 3. After re-balancing the content, you might perform manual merge to > claim memory immediately. > > 3. Split your files into smaller size > > Regards, > Indy > > > On Mon, May 11, 2015 at 11:30 PM, Wayne Feick <[email protected]> > wrote: > >> Looping in some additional information from private email. Since your >> list size is already configured to the maximum (32768), you could try to >> identify some index settings that you don't actually need and turn them off. >> >> If that isn't an option, you could try breaking it up into multiple >> documents, and then deleting the original document with >> xdmp:document-delete(). >> >> Wayne. >> >> >> >> On 05/11/2015 10:52 AM, Wayne Feick wrote: >> >> Hi Shashidhar, >> >> It sounds like the document was close to the limit when it was originally >> ingested, and that turning on additional index setting put it over the top. >> >> The error message says that your in-memory list storage is full, so if >> you go to the Admin UI and look at the database settings, you'll see an >> entry for "in memory list size". Configure a larger value there and you >> should be able to finish your reindex. >> >> Wayne. >> >> >> On 04/23/2015 01:28 AM, Shashidhar Rao wrote: >> >> Hi, >> >> Can somebody help me how to fix this issue >> >> There is currently an XDMP-FORESTERR: Error in reindex of forest >> PROD_DB_1: XDMP-REINDEX: Error reindexing >> fn:doc("/home/data/Folder2/US07625699-20091201-T00002.XML"): >> XDMP-FRAGTOOLARGE: Fragment of >> /home/data/Folder2/US07625699-20091201-T00002.XML too large for in-memory >> storage: XDMP-INMMLISTFULL: In-memory list storage full; list: table=100%, >> wordsused=50%, wordsfree=25%, overhead=25%; tree: table=0%, wordsused=6%, >> wordsfree=94%, overhead=0% exception. Information on this page may be >> missing. >> >> It says US07625699-20091201-T00002.XML too large. >> what are the other options any suggestions would be helpful. >> >> Is deleting this file an option as the last resort? >> >> Thanks >> >> >> >> _______________________________________________ >> General mailing [email protected] >> Manage your subscription at: >> http://developer.marklogic.com/mailman/listinfo/general >> >> >> -- >> Wayne Feick >> Principal Engineer >> MarkLogic [email protected] >> Phone: +1 650 655 2378www.marklogic.com >> >> This e-mail and any accompanying attachments are confidential. The >> information >> is intended solely for the use of the individual to whom it is addressed. Any >> review, disclosure, copying, distribution, or use of this e-mail >> communication >> by others is strictly prohibited. If you are not the intended recipient, >> please >> notify us immediately by returning this message to the sender and delete all >> copies. Thank you for your cooperation. >> >> >> >> _______________________________________________ >> General mailing [email protected] >> Manage your subscription at: >> http://developer.marklogic.com/mailman/listinfo/general >> >> >> -- >> Wayne Feick >> Principal Engineer >> MarkLogic [email protected] >> Phone: +1 650 655 2378www.marklogic.com >> >> This e-mail and any accompanying attachments are confidential. The >> information >> is intended solely for the use of the individual to whom it is addressed. Any >> review, disclosure, copying, distribution, or use of this e-mail >> communication >> by others is strictly prohibited. If you are not the intended recipient, >> please >> notify us immediately by returning this message to the sender and delete all >> copies. Thank you for your cooperation. >> >> >> _______________________________________________ >> General mailing list >> [email protected] >> Manage your subscription at: >> http://developer.marklogic.com/mailman/listinfo/general >> >> > > > _______________________________________________ > General mailing [email protected] > Manage your subscription at: > http://developer.marklogic.com/mailman/listinfo/general > > > -- > Wayne Feick > Principal Engineer > MarkLogic [email protected] > Phone: +1 650 655 2378www.marklogic.com > > This e-mail and any accompanying attachments are confidential. The information > is intended solely for the use of the individual to whom it is addressed. Any > review, disclosure, copying, distribution, or use of this e-mail communication > by others is strictly prohibited. If you are not the intended recipient, > please > notify us immediately by returning this message to the sender and delete all > copies. Thank you for your cooperation. > > > _______________________________________________ > General mailing list > [email protected] > Manage your subscription at: > http://developer.marklogic.com/mailman/listinfo/general > >
_______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
