Andre Bonhote wrote:
It's really a dbi-user's issue -- it is about a possible bug in DBI, DBD::Oracle or Oracle itself, rather than a design/implementation issue common to all driver writers. That isn't a hard'n'fast determination -- there's room to think that maybe it is partly an issue that faces each driver writer on occasion.thanks for your mail. I still think that dbi-dev is the right place for this. Anyone agrees?
That's irksome; slow leaks are hard to fix, both in tyres (tires for the American section of the audience) and in code.On Fri, Feb 07, 2003 at 06:22:12AM -0800, Jonathan Leffler wrote:I don't use DBD::Oracle, but DBD::Informix has a test harness module, and one of the functions in there, memory_leak_test, is specifically for running memory leak tests.There's something like that in DBD::Oracle, too. You can run test.pl with -m to run 100 loops. With those 100 iterations, nothing happens. I increased it to 10000, and still: no memory growth.
You may need to start looking rather hard at whether you need a memory allocation tracker. It is going to be painful, pretty much regardless. With a 16MB front-load and an average of about 1/20 KB (say 50 bytes - maybe 48 bytes data and 4 bytes administrative overhead) per iteration -- which gets quantized since memory is allocated in fairly large chunks -- it is going to be difficult to track down the source of the leak.There's an outside chance the bug is in DBI code [...]Well, that's my opinion. I don't think DBI is buggy.Step 1 is to establish that there is a leak [...]
Step 2 is to try and make a slow leak like your into a more serious (blatant) leak without changing the substance of the test. [...]
Haha! I read this and changed my test table to ~17k rows with 5 filled columns of varchar(200). The result? It grows almost as slow as test one. It starts with 16M, gets some more kb at iteration 50, stays like that until it. 200, and ends up at the 10000th iteration with only 500k more than at the beginning. ARGH!
Given that the behaviour doesn't change dramatically, the problem may be a small leakage for each statement that is executed. That is going to be a pain to all and sundry.
(40-16)*1024/0.05 = 0.5MStep 3 is to see whether you can change the blatantly leaky test so that it doesn't leak.Umh, it's not blatantly leaking yet, I suppose ... But the daemon I was speaking of grew of 40 Megs over the weekend.
Presumably, your daemon iterated about half a million times over the weekend? Does that sound plausible?
I looked at Devel::Size and wasn't convinced it would be of any help to me debugging a problem in Informix ESQL/C. I can see that it might be of relevance to the Perl core team, but not to an extension where the extension libraries are suspected of doing the leaking. Devel::Size might tell you if one of the variables in DBI or DBD::Oracle was stashing extra memory over time, but that's not the same as a leak where there is nothing referencing the memory that is leaked -- pretty much by definition, Devel::Size won't help you if the Oracle OCI code is losing memory because it is not tied to anything, least of all a Perl variable that you can monitor the size of.Once you've done that, you may be starting to get an idea about what
triggers the leak. And you may have a workaround. And the more
precise information may help the author(s) of DBI and the driver (in this case, the same person, of course) track down where the problem is.
Right. Well I am going to use -d again, and maybe Devel::Size. Any other hints on how to track down this bloody thing?
What I have done, in some other programs, is use a debugging malloc. It makes everything run glacially, but you can track exactly which memory is allocated, and which memory is freed, and you can add calls to an annotation function to identify places where you code is executed. This can reveal which part of the code is allocating the unfreed memory. Ultimately, you could hone in to the individual OCI call that is leaking - that is, allocating memory which is not subsequently freed. At that point, you can remove Perl and DBI and DBD::Oracle from the equation and write a simple, pure C + OCI program that demonstrates the same leakage -- and the job is done. You can send stroppy notes to Oracle and wait for the upgrade with the bug fix. But tracking that sort of leak is painful. Fortunately, Perl is quite good at processing data...
If you want to go down this route (which I do regard as a desparation measure), then contact me for a debugging malloc. No promises that it will do anything other than make you aware of how much memory is allocated in your program -- but it might, just conceivably, help.
However, I am not going to be listening to this mailing list for the rest of this week, so try to find another way to demonstrate the problem, and/or get someone else's debuggin malloc -- dmalloc from http://dmalloc.com springs to mind as a possibility worth investigating. And frankly, apart from possibly supplying a debugging malloc, there isn't a lot more I can offer in the way of help. You need Tim Bunce to chip in with any views he has on the subject, but if the problem is in the OCI itself, there isn't going to be very much he can do for you either.
--
Jonathan Leffler ([EMAIL PROTECTED], [EMAIL PROTECTED]) #include <disclaimer.h>
Guardian of DBD::Informix 1.04.PC1 -- http://dbi.perl.org/
