-----Original Message----- From: wireshark-dev-boun...@wireshark.org [mailto:wireshark-dev-boun...@wireshark.org] On Behalf Of Bill Meier Sent: den 6 oktober 2009 00:49 To: Developer support list for Wireshark Subject: Re: [Wireshark-dev] New packet list - out of memory?
Bill Meier wrote: >> A data point: A few days ago while working on bug #2375 I noticed that >> on Windows that if I defined the env logicals to not use se_alloc & >> etc that *much* less memory was required to load a capture. (20% of >> that used with se_alloc & etc ? I don't quite remember). >> > >Details: > >Env: Windows Vista > >Actions: > >1. Load Wireshark > Determine "Private Working Set", "Working Set", "Commit Size" > >2. Load a 40 Meg capture file. > Determine .... again > >The above was done for: > >a. Wireshark with new_packet_list: > >b. Wireshark w/new_packet_list: > set WIRESHARK_DEBUG_EP_NO_CHUNKS=T > set WIRESHARK_DEBUG_SE_NO_CHUNKS=T > >c. Wireshark without new packet list > >d. Wireshark without new packet list > set WIRESHARK_DEBUG_EP_NO_CHUNKS=T > set WIRESHARK_DEBUG_SE_NO_CHUNKS=T > >Measurements > >Note: The Working Set was always about 20 Meg larger than the > Private Working Set(PWS) so only the PWS is shown. > >(All values in Megs) > EP/SE Chunks No Chunks > PWS Commit PWS Commit > ---- ------- ----- ------- >Wireshark: current_packet_list > After startup: 42 62 42 52 > After 40M CF load: 180 399 195 205 > > >Wireshark: new_packet_list: > After startup: 42 62 42 52 > After 40M CF load: 136 564 165 177 > > >A conclusion: > >Commit Size (which I believe is the amount of VM address space used > by the process and which is presumably the limiting factor for > memory usage). > - Best for new_packet_list/no_chunks; > - worst (quite) for new_packet_list/chunks; > >Comments welcome... > >(I did these tests somewhat quickly so I could have missed something...) > >Bill Hi, I expanded on Bills measurements using the file that caused out of memory for the new packet list. (Thanks Bill I didn't know about the commit size option in the task manager). 146 MB 661417 pkts (TCP reassembly) Version 1.2.2 (SVN Rev 29910) PWS Commit 42 016 61 876 704 656 1 639 256 Loading time ~57s Old packet list SVN 30353 PWS Commit 43 060 63 384 492 456 1 427 480 Loading time 47.752 - 48.485 (2 Loading time measures) Old packet list SVN 30353 set WIRESHARK_DEBUG_SE_NO_CHUNKS=T PWS Commit 43 060 63 384 559 744 579 968 Loading time 48.797 Old packet list SVN 30353 set WIRESHARK_DEBUG_EP_NO_CHUNKS=T set WIRESHARK_DEBUG_SE_NO_CHUNKS=T 43 208 53 312 559 612 569 632 Loading time 55.162 Old packet list SVN 30353 #define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 64) PWS Commit 43 288 64 188 492 104 567 060 Loading time 47.736 - 48.391 (2 Loading time measures) (This file can't be loaded using 512) New packet list SVN 30353 #define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 64) PWS Commit 43 492 64 512 332 400 471 264 Loading time 55.895 - 57.018 (4 Loading time measures) New packet list SVN 30353 #define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 64) set WIRESHARK_DEBUG_SE_NO_CHUNKS=T PWS Commit 43 660 64 736 468 496 493 660 Loading time 53.666 New packet list SVN 30353 #define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 64) set WIRESHARK_DEBUG_EP_NO_CHUNKS=T set WIRESHARK_DEBUG_SE_NO_CHUNKS=T PWS Commit 43 660 64 736 468 428 481 516 Loading time 1:01.215 - 1:02.682 (2 Loading time measures) Compared with 1.2.2 memory usage has improved quite a bit. Loading time has improved with the old packet list but not so much with the new packet list!? Conclusions: - se alloced memory is a problem as the allocation "waists" memory. This is not a big problem for ep memory as it's given back to the pool after each packet. - Using the old packet list the file loads faster !? This was not the case some way along the line... - Surprisingly enough the file loads faster not using chunks for se_alloc ??? Whereas it is faster using ep memory. Increasing EMEM_ALLOCS_PER_CHUNK looks like a good thing - drawbacks? Should the use of se_alloc be discouraged/removed? Regards Anders ________________________________________________________________________ ___ Sent via: Wireshark-dev mailing list <wireshark-dev@wireshark.org> Archives: http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe ___________________________________________________________________________ Sent via: Wireshark-dev mailing list <wireshark-dev@wireshark.org> Archives: http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe
