2014-07-12 0:07 GMT+02:00 Anders Broman <[email protected]>: > > Den 11 jul 2014 23:13 skrev "Bálint Réczey" <[email protected]>: > > >> >> Hi All, >> >> Please provide the input data for letting others reproduce the results >> or perform the performance tests on pcap files already available to >> the public > > Ok I'll see if we can use something from the wiki instead. > >> >> I'm not a fan of implementing custom memory management methods because >> partly because I highly doubt we can beat jemalloc easily on >> performance and custom allocation methods can also have nasty bugs >> like the one observed in OpenSSL: >> http://www.tedunangst.com/flak/post/analysis-of-openssl-freelist-reuse >> > > We have gone through a set of memory allocation schemes already to try to > improve performance (g_slice, emen and now wmem) are you saying that you are > opposed to that? No, IMO there is need for wmem for being able to keep memory allocated during scopes, but I would prefer seeing it only tracking and one-by-one freeing the memory areas instead of doing bulk alloc()-s for optimizing for speed.
I tried to detail this in my answer to Evan's email. > > As wmem seems to be the faster scheme for packet scope memory allocation > /free, why not use it in all possible places where the scope is "packet"? > >> Please don't sacrifice protection for 2% speedup. Please keep wmem >> usage for cases where it is used for garbage collecting (free() after >> end of frame/capture file) not when the allocation and deallocation >> are already done properly. > > ? A slow scheme might be working well but that in it self does not warrant > to not replace it with a faster one. With this reasoning we shouldn't have > introduced ep memory in the first place. > > What percentage if improvement do you think makes a change worthwhile? > > The set of improvements Jacub and I have done lately has given a reduction > of 40-50 percent compared to 1.10 measuring with the sample file. The > problem is that each improvement only yeald a percent or 2. Agreed that we > shouldn't complicate the code for a small speed gain. 40-50% reduction is great and congratulations for such a speedup! I hope memory allocation handling is responsible for only small fraction of it because I would like to keep the possibility of detecting memory allocation related errors and I would prefer using tools implemented outside of Wireshark. For example I would avoid bulk allocations to make us able to use ASAN and Valgrind, even if we would implement our canaries. > > In your blog you say that people would accept double the execution time with > increased security - I'm not so sure. If say the reformatting of a video > takes one hour instead of 30 minutes. In Debian you would be able to pick a slow and secure video player for streaming from the untrusted Internet and a fast and less secure video encoder. I expect people to install Wireshark from the hardened-amd64 repository if they want to monitor a hostile network, while others pick the fast Wireshark for using it in their safe labs. So it depends and there will be good options to choose from. Cheers, Balint > > Just my 2 cents > Anders > >> >> Thanks, >> Balint >> >> 2014-07-11 8:58 GMT+02:00 Jakub Zawadzki <[email protected]>: >> > Hi, >> > >> > On Sat, Jun 21, 2014 at 10:12:48PM +0200, Jakub Zawadzki wrote: >> >> If we're in topic of optimizing 'slower' [de]allocations in common >> >> functions: >> >> >> >> - tvb allocation/deallocation (2.5%, or 3.4% when no filtering) >> >> >> >> 243,931,671 * ???:tvb_new >> >> [/tmp/wireshark/epan/.libs/libwireshark.so.0.0.0] >> >> 202,052,290 > ???:g_slice_alloc (2463493x) >> >> [/usr/lib64/libglib-2.0.so.0.3600.4] >> >> >> >> 291,765,126 * ???:tvb_free_chain >> >> [/tmp/wireshark/epan/.libs/libwireshark.so.0.0.0] >> >> 256,390,635 > ???:g_slice_free1 (2435843x) >> >> [/usr/lib64/libglib-2.0.so.0.3600.4] >> > >> >> This, or next week I'll try to do tvb. >> > >> > ... or maybe this week: >> > >> > ver0 | 18,055,719,820 (-----------) | Base version >> > 96f0585268f1cc4e820767c4038c10ed4915c12a >> > ver1 | 18,185,185,838 (0.6% slower) | Change tvb allocator g_slice_* to >> > wmem with file scope >> > ver2 | 17,809,433,204 (1.4% faster) | Change tvb allocator g_slice_* to >> > wmem with file/packet scope >> > ver3 | 17,812,128,887 (1.3% faster) | Change tvb allocator g_slice_* to >> > simple object allocator with epan scope >> > ver4 | 17,704,132,561 (2.0% faster) | Change tvb allocator g_slice_* to >> > simple object allocator with file scope >> > >> > I have uploaded patches & profiler outputs to: >> > http://www.wireshark.org/~darkjames/tvb-opt-allocator/ >> > >> > Please review, and check what version is OK to be applied. >> > >> > >> > P.S: I'll might find some time to do ver5 with slab allocator, but it'll >> > look like object allocator API with epan scope. >> > >> > Cheers, >> > Jakub. >> > >> > ___________________________________________________________________________ >> > Sent via: Wireshark-dev mailing list <[email protected]> >> > Archives: http://www.wireshark.org/lists/wireshark-dev >> > Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev >> > >> > mailto:[email protected]?subject=unsubscribe >> >> ___________________________________________________________________________ >> Sent via: Wireshark-dev mailing list <[email protected]> >> Archives: http://www.wireshark.org/lists/wireshark-dev >> Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev >> >> mailto:[email protected]?subject=unsubscribe ___________________________________________________________________________ Sent via: Wireshark-dev mailing list <[email protected]> Archives: http://www.wireshark.org/lists/wireshark-dev Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev mailto:[email protected]?subject=unsubscribe
