You'll want to uncomment both of these lines actually: #java.arg.11=-XX:PermSize=128M #java.arg.12=-XX:MaxPermSize=128M
And if you like you can make them a bit larger. Thanks Joe On Tue, Jul 28, 2015 at 6:52 PM, Joe Witt <[email protected]> wrote: > Jeremiah, > > In this case it is talking about permgen (Java 7 and where classes go). > > If you take a look at the './conf/bootstap.conf' file there is a line that > says: > > #java.arg.11=-XX:PermSize=128M > > Uncomment that and you should be good to go. We need more than the > typical Java 7 default for PermGen space. > > Of note the MergeContent processor is extremely memory efficient and > pretty much a non-factor on the heap as it never actually loads > content into memory. Very different than most systems would be. > > Thanks > Joe > > On Tue, Jul 28, 2015 at 6:47 PM, Adams, Jeremiah > <[email protected]> wrote: >> I am running into memory issues when using MergeContent. I am seeing >> OutofMemoryErrors. I am using 0.1.0. >> >> I am processing a kafka queue via GetKafka, some custom Processors and >> PutKafka. I have a PutKafka Controller just prior to the MergeContent >> Processor. I am passing JSON messages through the FlowFiles. >> >> I am looking to merge the flow files until reaching a minimum size or a >> time period elapses, then write them to S3. I get the memory issue with >> just 15-20 messages. The MergeContent Processor continues to update the >> Tasks in Tasks/Time even when not sending messages to the flow. >> >> Here are my settings for the MergeContent Processor: >> Merge Strategy: Bin-Packing Algo >> Merge Format: Binary Concatenation >> Attribute Strategy: Keep only Common Attributes >> Correlation Attribute Name: No Value Set >> Minimum Number of Entries: 1 >> Maximum Number of Entries: No Value Set >> Minimum Group Size: 1KB >> Maximum Group Size: 2 MB >> Max Bin Age: 10 m >> Maximum number of Bins: 100 >> Header File: No Value Set >> Footer File: No Value Set >> Demarcator File: No Value Set >> Compression Level: 1 >> Keep Path: False >> >> 2015-07-28 14:25:17,580 WARN [NiFi Web Server-22] >> org.eclipse.jetty.servlet.ServletHandler Error for >> /nifi-api/controller/process-groups/caffa1c4-ef49-4f0c-b144-0b5d47264481/status >> java.lang.OutOfMemoryError: PermGen space >> 2015-07-28 14:25:17,811 WARN [NiFi Web Server-22] >> org.eclipse.jetty.server.HttpChannel >> /nifi-api/controller/process-groups/caffa1c4-ef49-4f0c-b144-0b5d47264481/status?recursive=false >> java.lang.OutOfMemoryError: PermGen space >> 2015-07-28 14:25:18,020 WARN [NiFi Web Server-22] >> o.e.jetty.util.thread.QueuedThreadPool >> java.lang.OutOfMemoryError: PermGen space >> 2015-07-28 14:25:18,020 WARN [NiFi Web Server-22] >> o.e.jetty.util.thread.QueuedThreadPool Unexpected thread death: >> org.eclipse.jetty.util.thread.QueuedThreadPool$3@1a4b0747 in NiFi Web >> Server{STARTED,8<=9<=200,i=0,q=4} >> 2015-07-28 14:25:18,775 WARN [NiFi Web Server-23] >> org.eclipse.jetty.servlet.ServletHandler Error for >> /nifi-api/controller/process-groups/caffa1c4-ef49-4f0c-b144-0b5d47264481/status >> java.lang.OutOfMemoryError: PermGen space >> 2015-07-28 14:25:19,005 WARN [NiFi Web Server-23] >> org.eclipse.jetty.server.HttpChannel >> /nifi-api/controller/process-groups/caffa1c4-ef49-4f0c-b144-0b5d47264481/status?recursive=false >> java.lang.OutOfMemoryError: PermGen space >> 2015-07-28 14:25:19,240 WARN [NiFi Web Server-23] >> o.e.jetty.util.thread.QueuedThreadPool >> java.lang.OutOfMemoryError: PermGen space >> 2015-07-28 14:25:19,240 WARN [NiFi Web Server-23] >> o.e.jetty.util.thread.QueuedThreadPool Unexpected thread death: >> org.eclipse.jetty.util.thread.QueuedThreadPool$3@1a4b0747 in NiFi Web >> Server{STARTED,8<=9<=200,i=0,q=4} >> >> *Jeremiah Adams* >> >> Senior Software Developer >> Pearson >> >> 2154 East Commons Ave. >> Suite 400 >> Centennial, CO 80122 >> >> >> Always Learning >> Learn more at www.pearson.com
