No, this will work for the master only. To pass this to agents you have to tweak the startup script after line 104 ($FLUME_HOME/bin/flume): https://github.com/cloudera/flume/blob/master/bin/flume#L104
JOPTS="$JOPTS -Xms1g -Xmx2g " as example. - Alex On Jan 17, 2013, at 10:40 AM, "Kumar, Deepak8" <[email protected]> wrote: > Hi Alex, > The memory is getting used at flume agent. Setting the variables > MASTER_WATCHDOG & MASTERI_WATCHDOG at flume master will work for agent memory > as well? Or do we have some other option as flume agent? > > Regards, > Deepak > > -----Original Message----- > From: Alexander Alten-Lorenz [mailto:[email protected]] > Sent: Thursday, January 17, 2013 2:18 AM > To: [email protected] > Subject: Re: Flume Service Takes Lot of Memory > > Hi, > > To add JVM options in flume we have to edit the watchdog line in bin/flume > into: > MASTER_WATCHDOG="java -XX:+HeapDumpOnOutOfMemoryError ${JOPTS} -Dpid=$$ > -Dpidfile=$MASTERPID $WATCHDOG_CLASS" > MASTERI_WATCHDOG="java -XX:+HeapDumpOnOutOfMemoryError ${JOPTS} -Dpid=$$ > -Dpidfile=$MASTERPID $WATCHDOG_CLASS" > > and restart Flume master. > > flume-env.sh will not be used, because bin/flume honors only the variables > FLUME_PID_DIR and FLUME_IDENT_STRING > > Ensure that you have provided the jvm with sufficient heap space. By default > Flume starts the jvm with it's default heap allocation, which differs > depending on the jvm version, the host type (os, 32/64 bit, etc...), total > host memory available, as well as other issues. > > The environment variable UOPTS can be used to pass additional jvm parameters > when running Flume. e.g. > > $ UOPTS="-Xms1g -Xmx2g" bin/flume node > > which starts a flume node with an initial heap of one gig and a max heap of > two gig. See "java -h" or "java -X" for more details on available jvm options. > > cheers, > Alex > > On Jan 17, 2013, at 8:10 AM, "Kumar, Deepak8" <[email protected]> wrote: > >> Hi Alex, >> Thanks for assisting. I do have the following settings in flume-env.sh >> file export UOPTS="-Xms256m -Xmx512m" >> >> But once I do top, it shows this much virtual memory >> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND >> 8571 root 21 0 1209m 424m 11m S 2.0 5.3 53:05.95 java >> -Dflume.log.dir=/var/log/flume 4957 root >> >> Regards, >> Deepak >> >> -----Original Message----- >> From: Alexander Alten-Lorenz [mailto:[email protected]] >> Sent: Thursday, January 17, 2013 1:58 AM >> To: [email protected] >> Subject: Re: Flume Service Takes Lot of Memory >> >> Hi Deepak, >> >> This is a normal behavior, you can tweak flume's (OG) settings by adding >> some Xmx params: >> >> UOPTS="-Xms1g -Xmx2g" bin/flume node >> >> Of course you can also tweak the bin/flume script. >> >> - Alex >> >> On Jan 17, 2013, at 7:34 AM, "Kumar, Deepak8" <[email protected]> wrote: >> >>> Hi, >>> Could you please guide? >>> >>> Regards, >>> Deepak >>> >>> -----Original Message----- >>> From: Kumar, Deepak8 [CCC-OT_IT NE] >>> Sent: Wednesday, January 16, 2013 1:29 PM >>> To: [email protected] >>> Subject: Flume Service Takes Lot of Memory >>> >>> Hi, >>> I am using flume OG 0.9, the flume agent seems to take almost 1Gig Virtual >>> Memory. Is it uses this much normally or do we have some configuration >>> settings? >>> >>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND >>> 8571 root 21 0 1209m 424m 11m S 2.0 5.3 53:05.95 java >>> -Dflume.log.dir=/var/log/flume 4957 root >>> >>> Regards, >>> Deepak >> >> -- >> Alexander Alten-Lorenz >> http://mapredit.blogspot.com >> German Hadoop LinkedIn Group: http://goo.gl/N8pCF >> > > -- > Alexander Alten-Lorenz > http://mapredit.blogspot.com > German Hadoop LinkedIn Group: http://goo.gl/N8pCF > -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF
