[opendx-dev] dxexec mem optimize patch - could use some input

[Randall Hopper]

     This patch optimizes DX memory handling, primarily with regards to
module paths.  With it applied to DX, I get a 25% speed boost on cached net
executions.  I'd appreciate it if the CVS heads would review this patch and
see if there's commit potential here.

     Further dxexec profiling revealed that dxexec spends much of its time
on cached executions regrowing module path strings, one string path
component at a time.  For the network I'm working with, dxexec calls memory
allocation routines over 1,000,000 times each cached execution for module
path construction alone.

     This patch changes the representation of module paths to remove most
of this overhead.  Module paths are now represented as a list of
Module/Instance number pairs in binary form.  Building and augmenting
module paths in this form is fast and requires no dynamic memory, once the
module and macro names are cached.  Converting these paths to string form
for debug prints, module ID strings, etc. is also fast.

OVERVIEW:

   Sample module path:  /main:0/BoxAndWhiskerPlot:1/BoxAndWisker:2/Compute:5
   Old representation: "/main:0/BoxAndWhiskerPlot:3/BoxAndWisker:4/Compute:5"
   New representation: (0,0),(1,3),(2,4),(3,5)

   What DX uses module paths for:
     - Cache tags
     - Module paths on run-time gfuncs
     - DX debug/warning/error prints
     - Module "begin" messages to dxui
     - Comparison of module names and macro execution paths
     - Names of ASYNC_LOCAL variables
     - Module ID strings

   Patch overview
     - New ModPath struct defined in graph.h
     - Replaces cpath and path in both graph.h's gfunc struct and cache.h's 
       pathtag struct with associated code updates
     - Also, pre-grows some frequently built lists in the graph execution code

   What's left
     - The Macro/Module Name symbol table is currently global.
       That's probably not ideal; it should be associated with the network.

       (What's a good spot in the data structures though?  on Program?)
     - The error handling calls (ExDie, etc.) might need changed

This patch is against the 4.1.0 source.  It applies cleanly to CVS except
for one small piece which is easy to hand-merge.  However, I didn't send a
patch against CVS since it is a bit broken for IRIX now, and this patch
isn't completely ready for commit.

Thanks,

Randall

--
Randall Hopper (mailto:[EMAIL PROTECTED])
EPA Scientific Visualization Center
US EPA MD/24 ERC-1A; RTP, NC 27711

dx-4.1.0-modpath-optimize.patch.gz
Description: Binary data