hi Alex What you got has been set with 4GB heap -Regards Denny Ye
2012/7/25 alo.alt <[email protected]> > Hey Denny, > > Please set XMX and XMS as the same (4GB here), because on high traffic > sinks it could be that the allocation process can cause a gc sweep. Also I > would recommend to define the gen sizes with: > -XX:NewSize=64m -XX:MaxNewSize=64m > > cheers, > Alex > > > > > On Mittwoch, 25. Juli 2012 11:48:22, Denny Ye wrote: > >> hi Alex, >> Attachment is preparing for you! >> Long term pause may be the critical problem for us. Do you agree me ? >> Wish your response, thanks! >> >> -Regards >> Denny Ye >> >> 2012/7/25 alo.alt <[email protected] <mailto:[email protected]>> >> >> >> Hey Denny, >> >> thanks for the report. >> >> Can you please try to rerun with: >> >> JAVA_OPTS="-Xms200m -Xmx200m -Xmn32m -XX:+UseParNewGC >> -XX:+UseConcMarkSweepGC -XX:**CMSInitiatingOccupancyFraction**=70 >> -Xss128k >> -XX:+UseMembar -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps >> -Xloggc:/var/log/flume/gc.log" >> >> Plese attach the gc.log after. >> >> cheers, >> Alex >> >> Am 25.07.2012 10 <tel:25.07.2012%2010>:35, schrieb Denny Ye: >> >> > hi all, >> > >> > I tested Flume in last week with ScribeSource( >> > >> https://issues.apache.org/**jira/browse/FLUME-1382<https://issues.apache.org/jira/browse/FLUME-1382>) >> and HDFS Sink. >> More >> > detailed conditions and deployment cases listed below. Too many >> 'Full GC' >> > impact the throughput and amount of events promoted into old >> generation. I >> > have applied some tuning methods, no much effect. >> > >> > Could someone give me your feedback or tip to reduce the GC >> problem? >> > Wish your attention. >> > >> > >> > PS: Using Mike's report template at >> > >> https://cwiki.apache.org/**FLUME/flume-ng-performance-** >> measurements.html<https://cwiki.apache.org/FLUME/flume-ng-performance-measurements.html> >> > >> > * >> > * >> > >> > *Flume Performance Test 2012-07-25* >> > >> > *Overview* >> > >> > The Flume agent was run on its own physical machine in a single >> JVM. A >> > separate client machine generated load against the Flume box in >> > List<LogEntry> format. Flume stored data onto a 4-node HDFS cluster >> > configured on its own separate hardware. No virtual machines >> were used in >> > this test. >> > >> > *Hardware specs* >> > >> > CPU: Inter Xeon L5640 2 x quad-core @ 2.27 GHz (12 physical cores) >> > >> > Memory: 16 GB >> > >> > OS: CentOS release 5.3 (Final) >> > >> > *Flume configuration* >> > >> > JAVA Version: 1.6.0_20 (Java HotSpot 64-Bit Server VM) >> > >> > JAVA OPTS: -Xms1024m -Xmx4096m -XX:PermSize=256m -XX:NewRatio=1 >> > -XX:SurvivorRatio=5 -XX:InitialTenuringThreshold=**15 >> > -XX:MaxTenuringThreshold=31 -XX:PretenureSizeThreshold=**4096 >> > >> > Num. agents: 1 >> > >> > Num. parallel flows: 5 >> > >> > Source: ScribeSource >> > >> > Channel: MemoryChannel >> > >> > Sink: HDFSEventSink >> > >> > Selector: RandomSelector >> > >> > *Config-file* >> > >> > # list sources, channels, sinks for the agent >> > >> > agent.sources = seqGenSrc >> > >> > agent.channels = mc1 mc2 mc3 mc4 mc5 >> > >> > agent.sinks = hdfsSin1 hdfsSin2 hdfsSin3 hdfsSin4 hdfsSin5 >> > >> > >> > >> > # define sources >> > >> > agent.sources.seqGenSrc.type = >> org.apache.flume.source.**scribe.ScribeSource >> > >> > agent.sources.seqGenSrc.**selector.type = io.flume.RandomSelector >> > >> > >> > >> > # define sinks >> > >> > agent.sinks.hdfsSin1.type = hdfs >> > >> > agent.sinks.hdfsSin1.hdfs.path = /flume_test/data1/ >> > >> > agent.sinks.hdfsSin1.hdfs.**rollInterval = 300 >> > >> > agent.sinks.hdfsSin1.hdfs.**rollSize = 0 >> > >> > agent.sinks.hdfsSin1.hdfs.**rollCount = 1000000 >> > >> > agent.sinks.hdfsSin1.hdfs.**batchSize = 10000 >> > >> > agent.sinks.hdfsSin1.hdfs.**fileType = DataStream >> > >> > agent.sinks.hdfsSin1.hdfs.**txnEventMax = 1000 >> > >> > # ... define sink #2 #3 #4 #5 ... >> > >> > >> > >> > # define channels >> > >> > agent.channels.mc1.type = memory >> > >> > agent.channels.mc1.capacity = 1000000 >> > >> > agent.channels.mc1.**transactionCapacity = 1000 >> > >> > # ... define channel #2 #3 #4 #5 ... >> > >> > >> > >> > # specify the channel each sink and source should use >> > >> > agent.sources.seqGenSrc.**channels = mc1 mc2 mc3 mc4 mc5 >> > >> > agent.sinks.hdfsSin1.channel = mc1 >> > >> > # ... specify sink #2 #3 #4 #5 ... >> > >> > *Hadoop configuration* >> > >> > The HDFS sink was connected to a 4-node Hadoop cluster running >> CDH3u1. For >> > different HDFS sink, HDFS wrote data into different path. >> > >> > *Visualization of test setup* >> > >> > >> https://lh3.googleusercontent.**com/**dGumq1pu1Wr3Bj8WJmRHOoLWmUlGqx* >> *C4wW7_**XCNO9R1wuh15LRXaKKxGoccpjBXtgq**cdSVW-vtg<https://lh3.googleusercontent.com/dGumq1pu1Wr3Bj8WJmRHOoLWmUlGqxC4wW7_XCNO9R1wuh15LRXaKKxGoccpjBXtgqcdSVW-vtg> >> > >> > There are 10 Scribe Clients and each client send 20 million LogEntry >> > objects to ScribleSource. >> > >> > *Data description* >> > >> > List<LogEntry> entries containing a string category and a >> ByteArray body. >> > The ByteArray body size is 500 bytes. >> > >> > *Results* >> > >> > Throughput: >> > >> > Average: Source: 46.4 MB/s, Sink: 45.2 MB/s >> > >> > Maximum: Source: 67.1 MB/s, Sink: 88.3 MB/s >> > >> > >> > >> > CPU: Average: 196%, Maximum: 440% >> > >> > >> > >> > GC: Young GC: 1636 times, Full GC: 384 times >> > >> > >> > No data loss. >> > >> > *Heap and GC* >> > >> > By analyzing JVM Heap, we found that there are many LogEntry >> objects in >> > OldGen. We have tried to carry out some optimizations, but the >> results are >> > not satisfactory. We will continue to track this limitation. >> > >> > >> > >> > FullGC Log examples: >> > >> > [Full GC [PSYoungGen: 1497984K->0K(1797568K)] [PSOldGen: >> > 1720643K->1693741K(2097152K)] 3218627K->1693741K(3894720K) >> [PSPermGen: >> > 14566K->14566K(262144K)], 5.0027700 secs] [Times: user=5.01 >> sys=0.00, >> > real=5.00 secs] >> > >> > [Full GC [PSYoungGen: 1497960K->0K(1797568K)] [PSOldGen: >> > 1693805K->1752540K(2097152K)] 3191765K->1752540K(3894720K) >> [PSPermGen: >> > 14571K->14571K(262144K)], 5.0732570 secs] [Times: user=5.07 >> sys=0.00, >> > real=5.07 secs] >> > >> > [Full GC [PSYoungGen: 1497984K->0K(1797568K)] [PSOldGen: >> > 1752540K->1642553K(2097152K)] 3250524K->1642553K(3894720K) >> [PSPermGen: >> > 14572K->14568K(262144K)], 5.0710730 secs] [Times: user=5.07 >> sys=0.01, >> > real=5.08 secs] >> > >> > >> > >> > －Regards >> > >> > Denny Ye >> > >> >> >>
