Re: [Valgrind-users] Trouble using more than 24Gb memory on 64 bit system with 512G
On 21-Jul-2016 07:17, Julian Seward wrote: > On 21/07/16 16:09, Julian Seward wrote: >> >>> - aspacem_maxAddr = (Addr) 0x7fff; >>> + aspacem_maxAddr = (Addr) 0x40 - 1; // 256G >> >> Are you sure this frag is right? It seems to have drastically >> reduced aspacem_maxAddr. It may be that this is a constant >> that shouldn't change. > > Yeah, that's not right. aspacem_maxAddr needs to increase, not > decrease. The one-step transformation is shown by > > svn diff -c13278 svn://svn.valgrind.org/valgrind/trunk > > so just do that twice and you should get what you want. Makes perfect sense EXCEPT that the patched 3.9.0 version which does work for large amounts of memory has: diff aspacemgr-linux.c.orig.20160720132943 aspacemgr-linux.c 271c271 < # define VG_N_SEGMENTS 5000 --- > # define VG_N_SEGMENTS 50 273c273 < # define VG_N_SEGMENTS 3 --- > # define VG_N_SEGMENTS 50 1636c1636aspacem_maxAddr = (Addr) 0x40ULL - 1; /* 256 GB */ 1642c1642 aspacem_maxAddr = (Addr) 0x40ULL - 1; /* 256 GB */ 1663c1663 < aspacem_maxAddr = (Addr)0x10ULL - 1; // 64G --- > aspacem_maxAddr = (Addr) 0x40ULL - 1; /* 256 GB */ Some of those are in 32 bit sections, some in 64 bit, and I don't understand how a 32 bit machine could access 256 GB of memory. Not that I care in this case since only the 64 bit one matters to me, and it does work. It isn't my code, but I can't help thinking that we wouldn't be scratching our heads about how to do this if somewhere up at the top of the main include there was: #define VALGRIND_MAX_USABLE_GBMEM64 256 #define VALGRIND_MAX_USABLE_GBMEM32 31 and all the rest of this was handled automatically! Regards, David Mathog mat...@caltech.edu Manager, Sequence Analysis Facility, Biology Division, Caltech -- What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports.http://sdm.link/zohodev2dev ___ Valgrind-users mailing list Valgrind-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/valgrind-users
Re: [Valgrind-users] Trouble using more than 24Gb memory on 64 bit system with 512G
> - aspacem_maxAddr = (Addr) 0x7fff; > + aspacem_maxAddr = (Addr) 0x40 - 1; // 256G Are you sure this frag is right? It seems to have drastically reduced aspacem_maxAddr. It may be that this is a constant that shouldn't change. You can see what the initial memory layout looks like by doing this valgrind -d -d /bin/date and looking at the "<<< SHOW_SEGMENTS: Initial layout (5 segments)" for both the original and patched versions. For the original version, I have --12650:2: aspacem 0: RSVN 00-0003ff 64m - SmFixed --12650:2: aspacem 1: 000400-0801ff 32736m --12650:2: aspacem 2: RSVN 080200-0802000fff4096 - SmFixed --12650:2: aspacem 3: 0802001000-0f 32735m --12650:2: aspacem 4: RSVN 10- 16383e - SmFixed that is .. 2 32G RSVNs with a 4k guard page in between. If you do it right, I *think* what you should get is 2 128G RSVNs with a 4k guard page in between. J -- What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports.http://sdm.link/zohodev2dev ___ Valgrind-users mailing list Valgrind-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/valgrind-users
Re: [Valgrind-users] Trouble using more than 24Gb memory on 64 bit system with 512G
On 21/07/16 16:09, Julian Seward wrote: > >> - aspacem_maxAddr = (Addr) 0x7fff; >> + aspacem_maxAddr = (Addr) 0x40 - 1; // 256G > > Are you sure this frag is right? It seems to have drastically > reduced aspacem_maxAddr. It may be that this is a constant > that shouldn't change. Yeah, that's not right. aspacem_maxAddr needs to increase, not decrease. The one-step transformation is shown by svn diff -c13278 svn://svn.valgrind.org/valgrind/trunk so just do that twice and you should get what you want. J -- What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports.http://sdm.link/zohodev2dev ___ Valgrind-users mailing list Valgrind-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/valgrind-users
Re: [Valgrind-users] Trouble using more than 24Gb memory on 64 bit system with 512G
On 20-Jul-2016 13:00, mathog wrote: > Greetings, > > I cannot seem to get valgrind to work with more than 24GB (or so) > memory. Others report > going much higher than that, so I must be doing something wrong. Just found this http://joelinoff.com/blog/?p=1549 which has a script for patching 3.09.0 so that it works on memory up to 256GB. It is running now and seems to work - "top" shows i holding 59G and still growing. I tried examining the diffs for what it changed and copying those parts over onto 3.11.0 but it still didn't work. If somebody knows how to make this go for 3.11.0 please post it (or a link to it) here. Thanks, David Mathog mat...@caltech.edu Manager, Sequence Analysis Facility, Biology Division, Caltech -- What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports.http://sdm.link/zohodev2dev ___ Valgrind-users mailing list Valgrind-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/valgrind-users
