Re: [Libmesh-devel] Metis memory usage

On Mon, Nov 4, 2013 at 11:56 AM, Kirk, Benjamin (JSC-EG311) < benjamin.k...@nasa.gov> wrote: > > On Nov 4, 2013, at 12:28 PM, John Peterson > wrote: > > > > Nicely done yet again! The new implementation ran faster, and cut .23 > Gb of memory. That is a pretty huge memory savings for a 2 Gb mes

Re: [Libmesh-devel] Metis memory usage

On Nov 4, 2013, at 12:28 PM, John Peterson wrote: > > Nicely done yet again! The new implementation ran faster, and cut .23 Gb of > memory. That is a pretty huge memory savings for a 2 Gb mesh, especially > considering the graph is only actually formed on one processor now. > > https://dri

Re: [Libmesh-devel] Metis memory usage

On Mon, Nov 4, 2013 at 10:52 AM, Kirk, Benjamin (JSC-EG311) < benjamin.k...@nasa.gov> wrote: > On Nov 2, 2013, at 2:42 PM, John Peterson wrote: > > > > >> I'll still work to refractor the adjacency build, but that'll take more > concentration than I can muster this morning! > > > > OK, but you mi

Re: [Libmesh-devel] Metis memory usage

On Nov 2, 2013, at 2:42 PM, John Peterson wrote: > >> I'll still work to refractor the adjacency build, but that'll take more >> concentration than I can muster this morning! > > OK, but you might not even want to bother! We were around 5.5-6 Gb total > memory (on 2 cores) on the SFC and Ce

Re: [Libmesh-devel] Metis memory usage

On Sat, Nov 2, 2013 at 12:42 PM, John Peterson wrote: > > > > On Sat, Nov 2, 2013 at 8:21 AM, Kirk, Benjamin (JSC-EG311) < > benjamin.k...@nasa.gov> wrote: > >> On Nov 2, 2013, at 9:03 AM, John Peterson >> wrote: >> >> > Damn... Yeah this would clearly help the most for the parallel runs. >> N

Re: [Libmesh-devel] Metis memory usage

On Sat, Nov 2, 2013 at 8:21 AM, Kirk, Benjamin (JSC-EG311) < benjamin.k...@nasa.gov> wrote: > On Nov 2, 2013, at 9:03 AM, John Peterson > wrote: > > > Damn... Yeah this would clearly help the most for the parallel runs. No > reason to really optimize anything else until that's implemented! > >

Re: [Libmesh-devel] Metis memory usage

On Nov 2, 2013, at 9:03 AM, John Peterson wrote: > Damn... Yeah this would clearly help the most for the parallel runs. No > reason to really optimize anything else until that's implemented! So that is done now in the branch... I'll still work to refractor the adjacency build, but that'll ta

Re: [Libmesh-devel] Metis memory usage

> On Nov 2, 2013, at 7:01 AM, "Kirk, Benjamin (JSC-EG311)" > wrote: > >> On Nov 2, 2013, at 7:42 AM, "Kirk, Benjamin (JSC-EG311)" >> wrote: >> >>> On Nov 1, 2013, at 11:15 PM, "John Peterson" wrote: >>> >>> Finally, note that we call adjncy.reserve(graph_size); where >>> graph_size=48M.

Re: [Libmesh-devel] Metis memory usage

On Nov 2, 2013, at 7:42 AM, "Kirk, Benjamin (JSC-EG311)" wrote: > On Nov 1, 2013, at 11:15 PM, "John Peterson" wrote: > >> Finally, note that we call adjncy.reserve(graph_size); where graph_size=48M. >> This was most likely done for speed, but it means that the full memory for >> both 'grap

Re: [Libmesh-devel] Metis memory usage

On Nov 1, 2013, at 11:15 PM, "John Peterson" wrote: > Finally, note that we call adjncy.reserve(graph_size); where graph_size=48M. > This was most likely done for speed, but it means that the full memory for > both 'graph' and 'adjncy' are required at the same time. We could instead > try le

Re: [Libmesh-devel] Metis memory usage

On Fri, Nov 1, 2013 at 7:46 PM, John Peterson wrote: > > I guess the next thing I'll look at is > > 400.a/b: Wraps the creation of the 'xadj', 'adjncy', and 'graph' objects. >> vwgt is also written to during this time, so it must finally be allocated. >> Note that 'graph' has been deallocated b

Re: [Libmesh-devel] Metis memory usage

On Fri, Nov 1, 2013 at 12:54 PM, John Peterson wrote: > > On Fri, Nov 1, 2013 at 11:59 AM, John Peterson wrote: > >> >> It also appears to be a good deal faster than before, which is a nice > side benefit! > Well, marginally faster once I put all the logging statements and "sleep" calls back in,

Re: [Libmesh-devel] Metis memory usage

On Fri, 1 Nov 2013, John Peterson wrote: > But maybe not quite as much less as I would've hoped... It shaved off half a gig; I was only expecting .6 gigs based on your preliminary container results. I am a bit saddened to see how massive the *rest* of that allocation gets... --- Roy --

Re: [Libmesh-devel] Metis memory usage

> On Nov 1, 2013, at 2:00 PM, Cody Permann wrote: > > John, Did you and David make any more headway on the "Judy Array" C++ > wrappers? > http://judy.sourceforge.net Yes they use approximately as much memory as unordered_map, so vector is still superior here. ---

Re: [Libmesh-devel] Metis memory usage

John, Did you and David make any more headway on the "Judy Array" C++ wrappers? http://judy.sourceforge.net On Fri, Nov 1, 2013 at 1:54 PM, John Peterson wrote: > > > > On Fri, Nov 1, 2013 at 11:59 AM, John Peterson wrote: > >> >> >> >> On Fri, Nov 1, 2013 at 11:54 AM, Kirk, Benjamin (JSC-EG311

Re: [Libmesh-devel] Metis memory usage

On Fri, Nov 1, 2013 at 11:59 AM, John Peterson wrote: > > > > On Fri, Nov 1, 2013 at 11:54 AM, Kirk, Benjamin (JSC-EG311) < > benjamin.k...@nasa.gov> wrote: > >> So I just pushed a 'vectormap' branch, which defines a very limited >> map-like implementation using vector >> >> I reimplemented metis

Re: [Libmesh-devel] Metis memory usage

On Fri, Nov 1, 2013 at 1:28 PM, John Peterson wrote: > > > > On Fri, Nov 1, 2013 at 11:50 AM, Roy Stogner wrote: > >> >> On Fri, 1 Nov 2013, John Peterson wrote: >> >> unordered_map ends up somewhere between vector and map in terms of >>> memory usage. >>> >>> https://drive.google.com/file/**d/0

Re: [Libmesh-devel] Metis memory usage

On Fri, Nov 1, 2013 at 11:50 AM, Roy Stogner wrote: > > On Fri, 1 Nov 2013, John Peterson wrote: > > unordered_map ends up somewhere between vector and map in terms of >> memory usage. >> >> https://drive.google.com/file/**d/0B9BK7pg8se_** >> iTFBBSTZzSm8xSTA/edit?usp=**sharing

Re: [Libmesh-devel] Metis memory usage

On Fri, Nov 1, 2013 at 11:54 AM, Kirk, Benjamin (JSC-EG311) < benjamin.k...@nasa.gov> wrote: > So I just pushed a 'vectormap' branch, which defines a very limited > map-like implementation using vector > > I reimplemented metis to use it. I have not benchmarked speed, but John > any chance you co

Re: [Libmesh-devel] Metis memory usage

So I just pushed a 'vectormap' branch, which defines a very limited map-like implementation using vector I reimplemented metis to use it. I have not benchmarked speed, but John any chance you could add this to the memory profile plot for the 200^3 case? -Ben On Nov 1, 2013, at 1:50 PM, Roy

Re: [Libmesh-devel] Metis memory usage

Oops! I'm sure the label on the left is supposed to be in Megabytes. I forgot to make that Kilobyte adjustment when I simplified the scaling logic yesterday. I'll fix that up and get it checked back in :-) Cody Sent from my iPhone On Nov 1, 2013, at 12:37 PM, John Peterson wrote: On Fri,

Re: [Libmesh-devel] Metis memory usage

On Fri, 1 Nov 2013, John Peterson wrote: unordered_map ends up somewhere between vector and map in terms of memory usage. https://drive.google.com/file/d/0B9BK7pg8se_iTFBBSTZzSm8xSTA/edit?usp=sharing  Interesting, thanks! I assume that y label should read "megabytes", not "kilobytes? --- R

Re: [Libmesh-devel] Metis memory usage

On Fri, Nov 1, 2013 at 7:31 AM, John Peterson wrote: > > > > On Nov 1, 2013, at 5:56 AM, Roy Stogner > wrote: > > > > > >> On Thu, 31 Oct 2013, John Peterson wrote: > >> > >> We build a std::map<> from Elem* to a unique sorted contiguous ID, > as Metis only considers the active elements and

Re: [Libmesh-devel] Metis memory usage

> On Nov 1, 2013, at 5:56 AM, Roy Stogner wrote: > > >> On Thu, 31 Oct 2013, John Peterson wrote: >> >> We build a std::map<> from Elem* to a unique sorted contiguous ID, as >> Metis only considers the active elements and needs >> some contiguous numbering. I expect that gets quite

Re: [Libmesh-devel] Metis memory usage

On Thu, 31 Oct 2013, John Peterson wrote: We build a std::map<> from Elem* to a unique sorted contiguous ID, as Metis only considers the active elements and needs some contiguous numbering. I expect that gets quite big, and maybe should be refactored to use a sorted std::vec

Re: [Libmesh-devel] Metis memory usage

> You are quite correct about this. I ran our memory logger to compare the > size of memory of a map and a vector of pair, both with 8M > elements. The results were pretty striking Wow, sounds like we need a MaplikeVector<> to borrow Roy's terminology from the ParallelMesh stuff. Something t

Re: [Libmesh-devel] Metis memory usage

> > We build a std::map<> from Elem* to a unique sorted contiguous ID, as > Metis only considers the active elements and needs some contiguous > numbering. I expect that gets quite big, and maybe should be refactored to > use a sorted std::vector > instead? You are quite correct about this. I ra

Re: [Libmesh-devel] Metis memory usage

On Wed, Oct 30, 2013 at 6:37 PM, Kirk, Benjamin (JSC-EG311) < benjamin.k...@nasa.gov> wrote: > That graph is pretty awesome - thanks! I'm gonna have to digest that, > but I think there could be some small room for improvement with a different > data structure - if I interpret the steep ramp up to

Re: [Libmesh-devel] Metis memory usage

That graph is pretty awesome - thanks! I'm gonna have to digest that, but I think there could be some small room for improvement with a different data structure - if I interpret the steep ramp up to (global_index_map end) as map construction. -Ben On Oct 30, 2013, at 6:55 PM, "John Peterson"

Re: [Libmesh-devel] Metis memory usage

On Wed, Oct 30, 2013 at 3:27 PM, John Peterson wrote: > > > > On Wed, Oct 30, 2013 at 3:09 PM, Kirk, Benjamin (JSC-EG311) < > benjamin.k...@nasa.gov> wrote: > >> Yeah, before I get too carried away I should probably just try running >> the existing code path twice: Once as-is, and again actually

Re: [Libmesh-devel] Metis memory usage

On Wed, Oct 30, 2013 at 3:09 PM, Kirk, Benjamin (JSC-EG311) < benjamin.k...@nasa.gov> wrote: > Yeah, before I get too carried away I should probably just try running the > existing code path twice: Once as-is, and again actually commenting out > the underlying Metis call, making the partitioner a

Re: [Libmesh-devel] Metis memory usage

Yeah, before I get too carried away I should probably just try running the existing code path twice: Once as-is, and again actually commenting out the underlying Metis call, making the partitioner a big, expensive no-op. Actually, John, if you have a chance could you rerun one of the cases you

Re: [Libmesh-devel] Metis memory usage

Wow, If this does indeed fix the issue, then I can think of a lot of memory hog areas in MOOSE that we might have to cleanup sooner rather than later. I hope the overhead of the tree doesn't dominate the value_type stored so much that it blows us up our total usage by 200%! On the other hand size