Greg, I agree with you Randall had contributed some nice patches, and should be inside.
Suhaib > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of > [EMAIL PROTECTED] > Sent: Friday, June 23, 2000 5:00 PM > To: [email protected] > Subject: Re: [opendx-dev] dxexec mem optimize patch - could use some > input > > > Wow. I'm *very* impressed. So when I call DXGetModuleId I'll still get a > string, but internally to the exec it holds the module id as a list of > integers, right? That means it should be transparent to anybody outside > the exec and that therefore uses the DXGetModuleId interface. I'll be > happy to look at it, but I suspect that the only way to be really > comfortable with it is to run it awhile. On the other hand, you're > apparently running it on the mother of all networks, and if its OK there, > well, then. I'm sure you have a ton of Routes, Switches and macros in > there that will test all the nasty cases. > > There are quite a lot of changes between 4.1.0 and the CVS head. > How about > running acvs update on your tree to square it up, try it out, and then do > your own checkin? I'd first copy your current state aside so > that any bugs > that have been introduced since 4.1.0 (who, me?) won't interfere with your > main path of progress. > > Anybody who can hack the exec ought to be on the inside. If you agree, > I'll set it up. > > Greg > > > > Randall Hopper <[EMAIL PROTECTED]>@opendx.watson.ibm.com on 06/23/2000 > 12:25:12 PM > > Please respond to [email protected] > > Sent by: [EMAIL PROTECTED] > > > To: [email protected] > cc: > Subject: [opendx-dev] dxexec mem optimize patch - could use some input > > > > This patch optimizes DX memory handling, primarily with regards to > module paths. With it applied to DX, I get a 25% speed boost on > cached net > executions. I'd appreciate it if the CVS heads would review this > patch and > see if there's commit potential here. > > Further dxexec profiling revealed that dxexec spends much of its time > on cached executions regrowing module path strings, one string path > component at a time. For the network I'm working with, dxexec > calls memory > allocation routines over 1,000,000 times each cached execution for module > path construction alone. > > This patch changes the representation of module paths to remove most > of this overhead. Module paths are now represented as a list of > Module/Instance number pairs in binary form. Building and augmenting > module paths in this form is fast and requires no dynamic memory, once the > module and macro names are cached. Converting these paths to string form > for debug prints, module ID strings, etc. is also fast. > > OVERVIEW: > > Sample module path: > /main:0/BoxAndWhiskerPlot:1/BoxAndWisker:2/Compute:5 > Old representation: > "/main:0/BoxAndWhiskerPlot:3/BoxAndWisker:4/Compute:5" > New representation: (0,0),(1,3),(2,4),(3,5) > > What DX uses module paths for: > - Cache tags > - Module paths on run-time gfuncs > - DX debug/warning/error prints > - Module "begin" messages to dxui > - Comparison of module names and macro execution paths > - Names of ASYNC_LOCAL variables > - Module ID strings > > Patch overview > - New ModPath struct defined in graph.h > - Replaces cpath and path in both graph.h's gfunc struct and > cache.h's > pathtag struct with associated code updates > - Also, pre-grows some frequently built lists in the graph execution > code > > What's left > - The Macro/Module Name symbol table is currently global. > That's probably not ideal; it should be associated with > the network. > > (What's a good spot in the data structures though? on Program?) > - The error handling calls (ExDie, etc.) might need changed > > This patch is against the 4.1.0 source. It applies cleanly to CVS except > for one small piece which is easy to hand-merge. However, I didn't send a > patch against CVS since it is a bit broken for IRIX now, and this patch > isn't completely ready for commit. > > Thanks, > > Randall > > -- > Randall Hopper (mailto:[EMAIL PROTECTED]) > EPA Scientific Visualization Center > US EPA MD/24 ERC-1A; RTP, NC 27711 > > (See attached file: dx-4.1.0-modpath-optimize.patch.gz) > >
