Re: use filetest anomaly

2009-03-31 Thread Dave Mitchell
On Mon, Dec 29, 2008 at 09:20:23PM -0500, Matthew Persico wrote: On Mon, Dec 29, 2008 at 2:35 AM, Rafael Garcia-Suarez rgarciasua...@gmail.com wrote: 2008/12/29 Matthew Persico matthew.pers...@gmail.com: Perl - 5.10.0 $ ls -la ~/foo -r-xr-xr-x 1 persicom develop0 Dec 28 21:15

speeding up XS_DBI_dispatch()

2012-01-22 Thread Dave Mitchell
[ disclaimer: I am being funded to do this work ] I've been looking into bottlenecks in a tight fetch loop with millions of rows, e.g. $sth-bind_columns(...); while ($sth-fetch) { # do a small amount of work } In the particular case I've been investigating, the method lookup

Re: speeding up XS_DBI_dispatch()

2012-01-25 Thread Dave Mitchell
On Tue, Jan 24, 2012 at 08:42:28PM +, Tim Bunce wrote: On Mon, Jan 23, 2012 at 12:51:12AM +, Dave Mitchell wrote: In the particular case I've been investigating, the method lookup in XS_DBI_dispatch() contributed a measurable part of the total loop time; in particular, when

Re: speeding up XS_DBI_dispatch()

2012-01-27 Thread Dave Mitchell
On Fri, Jan 27, 2012 at 02:53:47PM +, Tim Bunce wrote: On Wed, Jan 25, 2012 at 08:17:27PM +, Tim Bunce wrote: On Wed, Jan 25, 2012 at 04:14:07PM +, Dave Mitchell wrote: Just out of curiosity, why can't threaded perls use it? To be honest I can't remember now. It's disabled

Re: speeding up XS_DBI_dispatch()

2012-01-29 Thread Dave Mitchell
On Wed, Jan 25, 2012 at 04:14:07PM +, Dave Mitchell wrote: then the CPU usage of the while loop broke down as follows: 7.0% overhead of loop, i.e. while() {$c++} 19.2% handle the outer method call, i.e. $sth-fetch() of which half is method lookup

Re: speeding up XS_DBI_dispatch()

2012-01-30 Thread Dave Mitchell
On Sun, Jan 29, 2012 at 10:06:58PM +, Tim Bunce wrote: On Sun, Jan 29, 2012 at 06:12:50PM +, Dave Mitchell wrote: The code itself (see diff below) just attaches some magic to the outer CV that caches the GV of the corresponding inner method. Any reason for not using the existing

Re: speeding up XS_DBI_dispatch()

2012-02-07 Thread Dave Mitchell
On Tue, Feb 07, 2012 at 10:54:12PM +, Tim Bunce wrote: -if (!imp_msv) { +if (!imp_msv || !GvCV(imp_msv)) { What's the GvCV test for? One of the more vicious tests in t/31methcache.t does local $DBD::Sponge::st::{fetch}; which causes gv_fetchmeth(...,'fetch') to

Re: speeding up XS_DBI_dispatch()

2012-02-08 Thread Dave Mitchell
On Wed, Feb 08, 2012 at 10:58:37AM +, Tim Bunce wrote: PS - I'm currently working on bringing PERINTERP and DBIS into the Brave New World of MY_CXT. Wonderful! I presume that'll cause an API change that'll require compiled drivers to be recompiled (ie. we'd bump DBISTATE_VERSION so

Re: speeding up XS_DBI_dispatch()

2012-02-10 Thread Dave Mitchell
On Thu, Feb 09, 2012 at 11:10:17PM +0100, demerphq wrote: Well perls fork() relies on threaded perl so it could very easily be Dave's patch. Dave do you have access to a win32 build environment? I'm afraid not. -- All wight. I will give you one more chance. This time, I want to hear no

Re: dPERINTERP/MY_CXT (was: speeding up XS_DBI_dispatch())

2012-02-10 Thread Dave Mitchell
On Thu, Jan 26, 2012 at 12:26:44PM +, Tim Bunce wrote: On Wed, Jan 25, 2012 at 05:37:13PM -0800, Jan Dubois wrote: On Wed, 25 Jan 2012, Tim Bunce wrote: On Wed, Jan 25, 2012 at 04:14:07PM +, Dave Mitchell wrote: PS - I'm specifically being paid only to fix a performance issue

Re: dPERINTERP/MY_CXT (was: speeding up XS_DBI_dispatch())

2012-02-17 Thread Dave Mitchell
On Fri, Feb 10, 2012 at 10:03:06AM -0800, Jan Dubois wrote: On Fri, 10 Feb 2012, Tim Bunce wrote: On Fri, Feb 10, 2012 at 03:27:24PM +, Dave Mitchell wrote: If anyone knows of a more elegant way to make a function from DBI.xs available to DBD:: code, please let me know! I hope

Re: Problems with DBI 1.617_903 on MSWin32

2012-02-22 Thread Dave Mitchell
On Wed, Feb 22, 2012 at 10:04:45AM +, Martin J. Evans wrote: DBI.xs(4291) : error C2275: 'SV' : illegal use of this type as an expression C:\Perl\lib\CORE\perl.h(2243) : see declaration of 'SV' ... Lines 4291 and 4292 are: MY_CXT_INIT; (void)MY_CXT; /* avoid 'unused

Re: Problems with DBI 1.617_903 on MSWin32

2012-02-22 Thread Dave Mitchell
On Wed, Feb 22, 2012 at 02:50:46PM +, Martin J. Evans wrote: On 22/02/12 11:09, Dave Mitchell wrote: It didn't: It is possible the following is the relevant code after preprocessing: Hi, it turns out that it's a different section of code, but looking at the right bit of code, the BOOT

Misc DBI_dispatch speedup patches

2012-02-24 Thread Dave Mitchell
The 8 attached patches (to be applied in sequence) all provide tweaks, cleanup, or otherwise attempt to make DBI_dispatch go a little bit faster. This speedups are all relatively minor, and my timings show the cumulative speedup for my standard fetch() test loop is somewhere between 0% and 9%

why is imp_xxh stored in an SV?

2012-02-24 Thread Dave Mitchell
(I confess I haven't looked closely at this yet, but...) I seems that the imp_xxh_t structure is stored in the PVX of an SV pointed to by mg_obj. Is there any reason why it can't be just directly malloced and pointed to from mg_ptr instead? -- The warp engines start playing up a bit, but seem

Re: Changes to make DBIS more efficient

2012-03-02 Thread Dave Mitchell
On Fri, Mar 02, 2012 at 03:11:17PM +, Tim Bunce wrote: On Fri, Feb 10, 2012 at 03:27:24PM +, Dave Mitchell wrote: I'm concerned that changing the API, and thus forcing all compiled drivers to be recompiled at the same time the DBI is installed, isn't worth the gain. Especially

Re: Changes to make DBIS more efficient

2012-03-02 Thread Dave Mitchell
On Fri, Mar 02, 2012 at 03:23:30PM +, Dave Mitchell wrote: I'd be happy in principle, although some guidance would be welcome Also in particular, what needs to continue happening across an API change? Is it just that a driver, compiled and installed against an old DBI, must continue to work

Re: Changes to make DBIS more efficient

2012-04-18 Thread Dave Mitchell
On Fri, Mar 02, 2012 at 05:04:29PM +, Dave Mitchell wrote: On Fri, Mar 02, 2012 at 03:23:30PM +, Dave Mitchell wrote: I'd be happy in principle, although some guidance would be welcome Also in particular, what needs to continue happening across an API change? Is it just

Re: why is imp_xxh stored in an SV?

2012-04-18 Thread Dave Mitchell
On Mon, Apr 16, 2012 at 03:12:07PM +0100, Tim Bunce wrote: On Fri, Feb 24, 2012 at 05:18:14PM +, Dave Mitchell wrote: (I confess I haven't looked closely at this yet, but...) [And I confess I forgot about this message - sorry for the delay.] I seems that the imp_xxh_t structure

Re: why is imp_xxh stored in an SV?

2012-04-18 Thread Dave Mitchell
Note that this and the just-submitted DBIS patch represent the end of my work on improving performance on DBI, (not counting any remedial work that may be required). PS - here are some final timings, comparing my 'while ($sth-fetch()) {$c++}' test loop (with unchanged DBD-mysql-4.020),

Re: [rt.cpan.org #77137] DBI 1.620 causes segfault in Perl when used with module Thread::Pool::Simple.

2012-05-11 Thread Dave Mitchell
On Fri, May 11, 2012 at 04:22:18PM +0100, Tim Bunce wrote: Dave, I'd be grateful if you could take a look at this since it seems to involve our recent changes and standard perl modules. I can reproduce. Am looking into it. -- That he said that that that that is is is debatable, is debatable.

Re: [rt.cpan.org #77137] DBI 1.620 causes segfault in Perl when used with module Thread::Pool::Simple.

2012-05-12 Thread Dave Mitchell
On Fri, May 11, 2012 at 04:52:33PM +0100, Dave Mitchell wrote: On Fri, May 11, 2012 at 04:22:18PM +0100, Tim Bunce wrote: Dave, I'd be grateful if you could take a look at this since it seems to involve our recent changes and standard perl modules. I can reproduce. Am looking into it. now