I was going to reply earlier that GC typically does very well with many allocations of the same size, since (typically) no compactions are necessary. So having many variable size packets there might be overall less garbage produced in size, but the collection takes more time because it take more work to collect variable sized chunks.
-- Lars ________________________________ From: 宾莉金 <[email protected]> To: [email protected] Sent: Sunday, July 29, 2012 6:27 PM Subject: Re: DFSClient Packet‘is not fully used，introduced Packet Pool and reuse Packet. 2012/7/30 Stack <[email protected]> > On Sun, Jul 29, 2012 at 5:09 PM, 宾莉金 <[email protected]> wrote: > > I find every Packet will new a byte array with size is 65557(more > than > > 64K). but the pktLen is just a few hundreds or a few ten hunderds . So > new > > a byte array with big size but just use a very little of it. So the > > utilization > > rate is very low. > > That makes sense. If sync enabled and we are syncing every write, if > the write is small, we are likely over allocating if 64k is default > buffer size. > > Nice work. > > Out of interest, is this buffer size configurable? (I've not looked). > > Yes，the buffer size can be configurable，using dfs.write.packet.size. this.writePacketSize = conf.getInt("dfs.write.packet.size", 64*1024); > I rewrite the logic of new Packet with a PacketPool，I find the > > hregionserver's gc frequency and gc total time reduce a lot，but a single > gc > > time increase a lot，so i reduce the jvm New gen size，but the same result > > get，The average RT increase and TPS reduced. > > What is RT? > > RT = response time > If you enable GC logging, can you contrast the before and after and > see if the logs tell you anything about the changed character of the > GC'ing? (The pool objects are being promoted to tenured gen because > they stick around longer and the clean up of the content in tenured > gen is whats taking longer? Or the copying of retained objects now > you are using a packet pool -- where before they were being released > -- in young gen is whats making the GC take longer?) > > > I cann't see why? And wish > > some one can told me why? may be the reason is gc. > > Although average RT increase，but more request has little rt and also > more > > request has a big rt，so the average RT increase. > > > > Thanks, > St.Ack > GC take longer ，may be because of there are more small objects. -- *Best Regards,* lijin bin
