On Aug 17, 4:55 pm, Fred Drake <fdr...@acm.org> wrote: > On Tue, Aug 17, 2010 at 11:20 AM, Matt Williams <li...@milliams.com> wrote: > > The problem I'm now facing is that the parsing of the XML file can > > take quite some time (~6 seconds for openbabel, up to ~45 seconds for > > OGRE3D and obviously even more for larger projects). > > I'm not familiar with the doxygen tag file, but it looks like there's > a lot of XPath usage in your code that reads that an produces the > mapping. It might be worth looking at more incremental ways to handle > that, possibly using a SAX- or Expat-based parser. ElementTree is > generally good for parsing, but XPath handling is likely more > expensive. > > Also might be worth seeing how much time is being spent in your > parsing.normalize() function. > > If those don't produce enough potential speedups, then it's time to > cache the converted structure.
The XML structure is quite flat and simple so you're right that I could probably get some speed-up from using something like SAX (from what I remember SAX is a stream type thing so memory consumption should be slightly less). I've already done some profiling though and almost all the time (easily 90%) is spent in the normalise function. The reason it's so slow is that it's implementing a token parser (using pyparsing) to find the arguments for the C++ functions and consistently normalise them (see the tests in doxylink/test/test_parser.py) so that the user can request a specific version of an overloaded function. Since the version checked in on bitbucket, I've doubled the speed of the normalise function and so it's unlikely I'll get more than another 20%-30% speed-up without taking a different (i.e. non-pyparsing) approach. If you have any great ides, let me know :) In the meantime (since my original email) I've implemented pickling of the mapping dictionary which works just fine so to some extent, that's now a solved problem. However, it would still be very nice if I could trigger a rebuild of all .rst files which are using my role upon the rebuild of the mapping. Perhaps this is not possible in Sphinx in which case I'll simply have to add a note in the documentation but it would be a great feature I think. Thanks for the help, Matt -- You received this message because you are subscribed to the Google Groups "sphinx-dev" group. To post to this group, send email to sphinx-...@googlegroups.com. To unsubscribe from this group, send email to sphinx-dev+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sphinx-dev?hl=en.
