[ disclaimer: I am being funded to do this work ]

I've been looking into bottlenecks in a tight fetch loop with millions of
rows, e.g.

    while ($sth->fetch) {
        # do a small amount of work

In the particular case I've been investigating, the method lookup in
XS_DBI_dispatch() contributed a measurable part of the total loop time;
in particular, when as a test I added very crude and hacky caching for the
lookup of the inner fetch method, the execution time for the loop dropped
from 23.0s to 19.2s.

So, I would like to attempt to add a method cache to XS_DBI_dispatch()
(and possibly see if anything else can be tweaked).

My questions are:

* Would such a patch be welcome in principle?
* Any gotchas that I might not be aware of (I'm not familiar with DBI/DBD
  internals apart from what I've played with for the last few days).
* What's the earliest perl we have to maintain compatibility with?
* Does the test suite already have good coverage for method lookup
  (especially where methods are redefined or stashes modified, i.e. the
  sort of thing that would invalidate caches), or should I add some?
* can I rely on _install_method() being the only way that XS CVs can get
  to point to XS_DBI_dispatch()?
* To what extent (if any) is DBI allowed to cheat when it comes to
  bypassing the perl API (in particular I'm thinking of the method
  invalidation stuff which I don't think has always been part of the API).



Please note that ash-trays are provided for the use of smokers,
whereas the floor is provided for the use of all patrons.
    -- Bill Royston

Reply via email to