Right, helix, but I mentioned OOM because people seem confused by it and anyways over allocating the Tomcat Java heap potentially starves the OS and other processes like the media filter.
Alan On Sun, Oct 4, 2015, 13:21 helix84 <[email protected]> wrote: > On Sun, Oct 4, 2015 at 10:37 AM, Alan Orth <[email protected]> wrote: > >> Out of memory errors only happen when the Linux kernel runs out of memory >> and—as a last-gasp effort before crashing—tries to reclaim some memory by >> killing a process that is deems is responsible for using a lot of memory. >> The problem with Java's heap is that it just "takes it" all at once, and >> that memory cannot be used by the OS or any other applications, including >> cron jobs like the media filter. >> > > Alan, the Linux OOM killer is not at play here. Were it to kill tomcat, > you wouldn't see it in tomcat logs as tomcat wouldn't get the chance to > even log a message. > > The issue here is, as correctly pointed out before, that command line > tools are a separate process with their own memory limit. It should be also > said that when settimg memory limits, swapping should be avoided at all > costs, therefore to set memory limits, look at a) how much RAM the machine > has, b) how much your system uses without tomcat running, c) -Xmx you > allocated to Tomcat d) -Xmx you allocated to command line tools. Then > change them while keeping in mind that a > b + c + d. > > Regarding the specific original problem - raising the -Xmx for command > line tools should help, but it just happens that there are sometimes larger > PDFs than you have memory available to process them. The media filter will > happily skip these and process the rest. If you don't have enough memory to > extract fulltext from a particular PDF, you may consider extracting the > text in other way (an offline tool or maybe even manually copying the text > out of a PDF viewer) and adding a .txt bitstream with the same name as the > .pdf to the TEXT bundle of the pdf's item. I'm not suggesting it's the > recommended approach, but it will get the job done if you can't live > without the indexed fulltext. > > > Regards, > ~~helix84 > > Compulsory reading: DSpace Mailing List Etiquette > https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette > > -- You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/dspace-tech. For more options, visit https://groups.google.com/d/optout.
