Hi Tim,
now I have done some new tests.
DBD::DB2 0.75 have only memory leaks with fetchrow_hashref() [16%].
DBD::DB2 0.76 have memory leaks with prepare() [27%] and with fetchrow_hashref() [16%].
DBI 1.37 and DBI 1.38 have no differences
Thanks
Ruediger
Complete test results:
----------------------
DBD::DB2 0.75 prepare
--------------------------------
0 SZ 26064 (100.00%) RSS 11532 (100.00%)
1000 SZ 26068 (100.02%) RSS 11600 (100.59%)
2000 SZ 26068 (100.02%) RSS 11600 (100.59%)
3000 SZ 26068 (100.02%) RSS 11600 (100.59%)
4000 SZ 26068 (100.02%) RSS 11600 (100.59%)
5000 SZ 26068 (100.02%) RSS 11600 (100.59%)
6000 SZ 26068 (100.02%) RSS 11600 (100.59%)
7000 SZ 26068 (100.02%) RSS 11600 (100.59%)
8000 SZ 26068 (100.02%) RSS 11600 (100.59%)
9000 SZ 26068 (100.02%) RSS 11600 (100.59%)
10000 SZ 26068 (100.02%) RSS 11600 (100.59%)
DBD::DB2 0.75 prepare + execute
--------------------------------
0 SZ 26064 (100.00%) RSS 11648 (100.00%)
1000 SZ 26068 (100.02%) RSS 11716 (100.58%)
2000 SZ 26068 (100.02%) RSS 11716 (100.58%)
3000 SZ 26068 (100.02%) RSS 11716 (100.58%)
4000 SZ 26068 (100.02%) RSS 11716 (100.58%)
5000 SZ 26068 (100.02%) RSS 11716 (100.58%)
6000 SZ 26068 (100.02%) RSS 11716 (100.58%)
7000 SZ 26068 (100.02%) RSS 11716 (100.58%)
8000 SZ 26068 (100.02%) RSS 11716 (100.58%)
9000 SZ 26068 (100.02%) RSS 11716 (100.58%)
10000 SZ 26068 (100.02%) RSS 11716 (100.58%)
DBD::DB2 0.75 prepare + execute + fetchrow_hashref
--------------------------------
0 SZ 26064 (100.00%) RSS 11760 (100.00%)
1000 SZ 26208 (100.55%) RSS 11928 (101.43%)
2000 SZ 26408 (101.32%) RSS 12128 (103.13%)
3000 SZ 26608 (102.09%) RSS 12328 (104.83%)
4000 SZ 26808 (102.85%) RSS 12528 (106.53%)
5000 SZ 27008 (103.62%) RSS 12728 (108.23%)
6000 SZ 27208 (104.39%) RSS 12928 (109.93%)
7000 SZ 27408 (105.16%) RSS 13128 (111.63%)
8000 SZ 27608 (105.92%) RSS 13328 (113.33%)
9000 SZ 27808 (106.69%) RSS 13528 (115.03%)
10000 SZ 28008 (107.46%) RSS 13728 (116.73%)
DBD::DB2 0.75 prepare + execute + fetchrow_arrayref
--------------------------------
0 SZ 26064 (100.00%) RSS 11760 (100.00%)
1000 SZ 26068 (100.02%) RSS 11788 (100.24%)
2000 SZ 26068 (100.02%) RSS 11788 (100.24%)
3000 SZ 26068 (100.02%) RSS 11788 (100.24%)
4000 SZ 26068 (100.02%) RSS 11788 (100.24%)
5000 SZ 26068 (100.02%) RSS 11788 (100.24%)
6000 SZ 26068 (100.02%) RSS 11788 (100.24%)
7000 SZ 26068 (100.02%) RSS 11788 (100.24%)
8000 SZ 26068 (100.02%) RSS 11788 (100.24%)
9000 SZ 26068 (100.02%) RSS 11788 (100.24%)
10000 SZ 26068 (100.02%) RSS 11788 (100.24%)
-----------------------------------------------------------------------
DBD::DB2 0.76 prepare
--------------------------------
0 SZ 26060 (100.00%) RSS 11528 (100.00%)
1000 SZ 26368 (101.18%) RSS 11904 (103.26%)
2000 SZ 26680 (102.38%) RSS 12216 (105.97%)
3000 SZ 26992 (103.58%) RSS 12528 (108.67%)
4000 SZ 27304 (104.77%) RSS 12840 (111.38%)
5000 SZ 27616 (105.97%) RSS 13152 (114.09%)
6000 SZ 27928 (107.17%) RSS 13464 (116.79%)
7000 SZ 28244 (108.38%) RSS 13780 (119.54%)
8000 SZ 28556 (109.58%) RSS 14092 (122.24%)
9000 SZ 28868 (110.78%) RSS 14404 (124.95%)
10000 SZ 29180 (111.97%) RSS 14716 (127.65%)
DBD::DB2 0.76 prepare + execute
--------------------------------
0 SZ 26060 (100.00%) RSS 11644 (100.00%)
1000 SZ 26368 (101.18%) RSS 12016 (103.19%)
2000 SZ 26680 (102.38%) RSS 12328 (105.87%)
3000 SZ 26992 (103.58%) RSS 12640 (108.55%)
4000 SZ 27304 (104.77%) RSS 12952 (111.23%)
5000 SZ 27616 (105.97%) RSS 13264 (113.91%)
6000 SZ 27928 (107.17%) RSS 13576 (116.59%)
7000 SZ 28244 (108.38%) RSS 13892 (119.31%)
8000 SZ 28556 (109.58%) RSS 14204 (121.99%)
9000 SZ 28868 (110.78%) RSS 14516 (124.67%)
10000 SZ 29180 (111.97%) RSS 14828 (127.34%)
DBD::DB2 0.76 prepare + execute + fetchrow_hashref
--------------------------------
0 SZ 26064 (100.00%) RSS 11772 (100.00%)
1000 SZ 26520 (101.75%) RSS 12256 (104.11%)
2000 SZ 27040 (103.74%) RSS 12776 (108.53%)
3000 SZ 27560 (105.74%) RSS 13296 (112.95%)
4000 SZ 28080 (107.73%) RSS 13816 (117.36%)
5000 SZ 28600 (109.73%) RSS 14336 (121.78%)
6000 SZ 29120 (111.72%) RSS 14856 (126.20%)
7000 SZ 29640 (113.72%) RSS 15376 (130.62%)
8000 SZ 30164 (115.73%) RSS 15900 (135.07%)
9000 SZ 30684 (117.73%) RSS 16420 (139.48%)
10000 SZ 31204 (119.72%) RSS 16940 (143.90%)
DBD::DB2 0.76 prepare + execute + fetchrow_arrayref
--------------------------------
0 SZ 26064 (100.00%) RSS 11764 (100.00%)
1000 SZ 26368 (101.17%) RSS 12096 (102.82%)
2000 SZ 26680 (102.36%) RSS 12408 (105.47%)
3000 SZ 26992 (103.56%) RSS 12720 (108.13%)
4000 SZ 27304 (104.76%) RSS 13032 (110.78%)
5000 SZ 27620 (105.97%) RSS 13348 (113.46%)
6000 SZ 27932 (107.17%) RSS 13660 (116.12%)
7000 SZ 28244 (108.36%) RSS 13972 (118.77%)
8000 SZ 28556 (109.56%) RSS 14284 (121.42%)
9000 SZ 28868 (110.76%) RSS 14596 (124.07%)
10000 SZ 29180 (111.96%) RSS 14908 (126.73%)
>
> On Wed, Sep 24, 2003 at 03:51:50PM +0200, [EMAIL PROTECTED] wrote:
> > Hi all,
> >
> > I'm working in a database migration project (Sybase ASE -> DB2 V8) with perl, DBI
> > and DBD::DB2 and we have a memory leak problem when we run a lot of select
> > statements during data import/export.
> >
> > We tested the System with different combinations of DBI (1.20 - 1.38), DBD::DB2
> > (0.75-0.76) and DB2 V8.1.3 Fixpacks (FP2 + FP3).
> >
> > All test results in a more or less large memory leak (see Script output).
> > Someone can help me or should we go back to Sybase ASE.
>
> It's more likely to get fixed if you can narrow down the cause.
> For example
> does just doing prepare() in a loop leak, or
> does just doing prepare() and execute() with no fetches leak, or
> does fetchrow_arrayref leak (eg avoid hashes), or
> etc etc
>
> Tim.
>
> > Rgds,
> >
> > Ruediger
> >
> >
> > Script:
> > ------
> > #!/usr/bin/perl
> >
> > use strict;
> > use locale;
> > use DBI;
> >
> > my $dbh = DBI->connect('dbi:DB2:xxx','xxx','xxx',{AutoCommit => 1,});
> >
> > sub getps {
> > my @a = split(/ +/,`ps auxww | grep _00_xx_test5.pl | grep -v grep`);
> > return ($a[4],$a[5],$a[1]);
> > }
> >
> > my @procorg;
> >
> > sub ps {
> > my @proc = getps();
> > my $d1 = sprintf("%3.2f",$proc[0]/$procorg[0]*100.0);
> > my $d2 = sprintf("%3.2f",$proc[1]/$procorg[1]*100.0);
> > return "SZ ".($proc[0])." ($d1%) RSS ".($proc[1])." ($d2%)";
> > }
> >
> > my $c = 0;
> > while (1) {
> > my @RESULT = ();
> > $c++;
> > # select 13 records (ID, NAME, ID_TYPE)
> > my $st = $dbh->prepare('select * from ut_attribute');
> >
> > $st->execute();
> >
> > # test 1
> > # ------
> > # while (my $ref = $st->fetchrow_hashref()) {
> > # push (@RESULT, $ref);
> > # }
> >
> > # test 2
> > # ------
> > # my $ref = $st->fetchall_hashref('ID');
> >
> > # test 3
> > # ------
> > while (my $hashref = $sth->fetchrow_hashref) {
> > }
> >
> > $st->finish();
> > if ($c==1) {
> > @procorg = getps();
> > print "0 ".ps()."\n";
> > }
> > elsif ($c % 1000 == 0) {
> > print "$c ".ps()."\n";
> > }
> > }
> > $dbh->disconnect;
> >
> >
> > Script output:
> > -------------
> > with DBD::DB2 0.76
> > ------------------
> > 0 SZ 25412 (100.00%) RSS 11300 (100.00%)
> > 1000 SZ 26196 (103.09%) RSS 12088 (106.97%)
> > 2000 SZ 27028 (106.36%) RSS 12920 (114.34%)
> > 3000 SZ 27860 (109.63%) RSS 13752 (121.70%)
> > 4000 SZ 28692 (112.91%) RSS 14584 (129.06%)
> > 5000 SZ 29524 (116.18%) RSS 15416 (136.42%)
> > 6000 SZ 30356 (119.46%) RSS 16248 (143.79%)
> > 7000 SZ 31188 (122.73%) RSS 17080 (151.15%)
> > 8000 SZ 32020 (126.00%) RSS 17912 (158.51%)
> > 9000 SZ 32852 (129.28%) RSS 18744 (165.88%)
> > 10000 SZ 33684 (132.55%) RSS 19576 (173.24%)
> > 11000 SZ 34512 (135.81%) RSS 20404 (180.57%)
> >
> >
> > with DBD::DB2 0.75
> > ------------------
> > 0 SZ 25416 (100.00%) RSS 11296 (100.00%)
> > 1000 SZ 25572 (100.61%) RSS 11456 (101.42%)
> > 2000 SZ 25776 (101.42%) RSS 11660 (103.22%)
> > 3000 SZ 25980 (102.22%) RSS 11864 (105.03%)
> > 4000 SZ 26184 (103.02%) RSS 12068 (106.83%)
> > 5000 SZ 26388 (103.82%) RSS 12272 (108.64%)
> > 6000 SZ 26592 (104.63%) RSS 12476 (110.45%)
> > 7000 SZ 26796 (105.43%) RSS 12680 (112.25%)
> > 8000 SZ 27000 (106.23%) RSS 12884 (114.06%)
> > 9000 SZ 27204 (107.03%) RSS 13088 (115.86%)
> > 10000 SZ 27408 (107.84%) RSS 13292 (117.67%)
> > 11000 SZ 27616 (108.66%) RSS 13500 (119.51%)
> > 12000 SZ 27816 (109.44%) RSS 13700 (121.28%)
> >
> >
> >
> > Environment:
> > ------------
> > SuSE SLES 8
> > Linux Kernel 2.4.19
> > DB2/LINUX 8.1.3 (also testet Fixpack 2 (8.1.2))
> > Perl 5.8.0
> > DBI 1.38 (also tested 1.20 - 1.37)
> > DBD::DB2 0.76 (also tested 0.75)
> >
> >
