Re: [RDBO] Too Many connections
On 1/25/08 4:54 PM, maxim wrote:
> How it may work if I am using Rose::DB::Object::Loader and creating my Manager
> subclasses in the runtime ?
You can pre-create the common base class for all your classes created by the
Loader, then tell the Loader to use your base class using the base_class
parameter:
http://search.cpan.org/dist/Rose-DB-Object/lib/Rose/DB/Object/Loader.pm#base
_class
> I ran into the same problem with looping over thousand of
> My::Manager->get_someTable() calls
Manager classes will call init_db() on their object_class if you do not pass
in a Rose::DB -derived object using the "db" parameter.
http://search.cpan.org/dist/Rose-DB-Object/lib/Rose/DB/Object/Manager.pm#obj
ect_class
http://search.cpan.org/dist/Rose-DB-Object/lib/Rose/DB/Object/Manager.pm#db
So either make sure your init_db() methods are set up correctly (e.g., by
defining your common base class up-front, as described above) or just pass
in a "db" parameter to every Manager call.
-John
>> -Original Message-
>> From: [EMAIL PROTECTED]
>> [mailto:[EMAIL PROTECTED] On
>> Behalf Of John Siracusa
>> Sent: Thursday, January 10, 2008 7:25 PM
>> To: Rose-DB-Object
>> Subject: Re: [RDBO] Too Many connections
>>
>> On 1/10/08 5:30 PM, James Masters wrote:
>>> Looking into this, I seem to simply be doing the following
>> in a "for"
>>> loop more than 100 times:
>>>
>>> my $shipobj = MGORD::Shipment->new(shipid => $shipid)->load(with
>>> =>['items', 'costs', 'problems', 'predictedcosts', 'toaddress',
>>> 'fromaddress']);
>>>
>>> So the implication is that each one of these is opening a
>> new client
>>> connection and not closing it.
>>>
>>> I know I can just increase the default connections limit
>> from 100 to
>>> say 200 to fix the problem. But is Rose really supposed to be
>>> creating all these client connections or have I written
>> something wrongly?
>>
>> When an RDBO-derived object needs a database connection, it
>> calls its db()
>> method:
>>
>>
>> http://search.cpan.org/dist/Rose-DB-Object/lib/Rose/DB/Object.pm#db
>>
>> If a db object does not yet exist, the init_db() method is
>> called to create
>> one:
>>
>>
>> http://search.cpan.org/dist/Rose-DB-Object/lib/Rose/DB/Object.
> pm#init_db
>>
>> The default implementation of init_db() returns a new
>> Rose::DB object, which in turn will create its own DBI
>> database connection.
>>
>> If you'd like one or more objects to share a single
>> Rose::DB-derived object (and therefore a single database
>> connection), a straightforward way to do it is to explicitly
>> code the sharing:
>>
>> my $db = My::DB->new; # share this among all objects created below
>>
>> for(...)
>> {
>> my $shipobj =
>> MGORD::Shipment->new(db => $db, ...)->load(...);
>> }
>>
>> If you'd rather not do this explicitly, or if you simply want
>> another default policy, then you should override the
>> init_db() method, either in your common base class or in each
>> individual RDBO-derived class, depending on the policy you want.
>>
>> For example, if you want all objects of or derived from a
>> given class to share a single database connection by default,
>> you could use the
>> new_or_cached() Rose::DB method:
>>
>> http://search.cpan.org/dist/Rose-DB/lib/Rose/DB.pm#new_or_cached
>>
>> writing an init_db() method like this:
>>
>> sub init_db { My::DB->new_or_cached }
>>
>> Remember that no matter what policy you implement in
>> init_db(), you can always explicitly create and pass a
>> Rose::DB-derived object as that value of a "db" parameter to
>> any individual object or Manager call.
>>
>> -John
>>
>>
>>
>>
>> --
>> ---
>> Check out the new SourceForge.net Marketplace.
>> It's the best place to buy or sell services for just about
>> anything Open Source.
>> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.n
> et/marketplace
>> ___
>> Rose-db-object mailing list
>> Rose-db-object@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/rose-db-object
>>
>
>
>
> -
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
> ___
> Rose-db-object mailing list
> Rose-db-object@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/rose-db-object
-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Rose-db-object mailing list
Rose-db-object@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rose-db-object
Re: [RDBO] Too Many connections
Hi John,
How it may work if I am using Rose::DB::Object::Loader and creating my
Manager
subclasses in the runtime ? I ran into the same problem with looping
over thousand of My::Manager->get_someTable() calls
Thanks,
Maxim Grigoriev.
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On
> Behalf Of John Siracusa
> Sent: Thursday, January 10, 2008 7:25 PM
> To: Rose-DB-Object
> Subject: Re: [RDBO] Too Many connections
>
> On 1/10/08 5:30 PM, James Masters wrote:
> > Looking into this, I seem to simply be doing the following
> in a "for"
> > loop more than 100 times:
> >
> > my $shipobj = MGORD::Shipment->new(shipid => $shipid)->load(with
> > =>['items', 'costs', 'problems', 'predictedcosts', 'toaddress',
> > 'fromaddress']);
> >
> > So the implication is that each one of these is opening a
> new client
> > connection and not closing it.
> >
> > I know I can just increase the default connections limit
> from 100 to
> > say 200 to fix the problem. But is Rose really supposed to be
> > creating all these client connections or have I written
> something wrongly?
>
> When an RDBO-derived object needs a database connection, it
> calls its db()
> method:
>
>
> http://search.cpan.org/dist/Rose-DB-Object/lib/Rose/DB/Object.pm#db
>
> If a db object does not yet exist, the init_db() method is
> called to create
> one:
>
>
> http://search.cpan.org/dist/Rose-DB-Object/lib/Rose/DB/Object.
pm#init_db
>
> The default implementation of init_db() returns a new
> Rose::DB object, which in turn will create its own DBI
> database connection.
>
> If you'd like one or more objects to share a single
> Rose::DB-derived object (and therefore a single database
> connection), a straightforward way to do it is to explicitly
> code the sharing:
>
> my $db = My::DB->new; # share this among all objects created below
>
> for(...)
> {
> my $shipobj =
> MGORD::Shipment->new(db => $db, ...)->load(...);
> }
>
> If you'd rather not do this explicitly, or if you simply want
> another default policy, then you should override the
> init_db() method, either in your common base class or in each
> individual RDBO-derived class, depending on the policy you want.
>
> For example, if you want all objects of or derived from a
> given class to share a single database connection by default,
> you could use the
> new_or_cached() Rose::DB method:
>
> http://search.cpan.org/dist/Rose-DB/lib/Rose/DB.pm#new_or_cached
>
> writing an init_db() method like this:
>
> sub init_db { My::DB->new_or_cached }
>
> Remember that no matter what policy you implement in
> init_db(), you can always explicitly create and pass a
> Rose::DB-derived object as that value of a "db" parameter to
> any individual object or Manager call.
>
> -John
>
>
>
>
> --
> ---
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for just about
> anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.n
et/marketplace
> ___
> Rose-db-object mailing list
> Rose-db-object@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/rose-db-object
>
-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Rose-db-object mailing list
Rose-db-object@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rose-db-object
Re: [RDBO] Too Many connections
On Fri, 11 Jan 2008 13:29:43 -0500 "John Siracusa" <[EMAIL PROTECTED]> wrote: JS> On Jan 11, 2008 1:10 PM, Ted Zlatanov <[EMAIL PROTECTED]> wrote: >> I think the automatic class builder should have a required option to set >> the default explicitly (it didn't, last time I looked). That would >> ensure the developer thought about it even in the path of least >> resistance. JS> Wat would the be the possible values for that option, an what code JS> would each option produce? I'd call the option "connection_creation_mode" but that may be too verbose. per_object: the current default, new DB each time init_db is called per_process: init_db as you showed earlier externally_managed: init_db will die(), user must write it himself apache_managed: works as you'd expect with Apache::DB (I don't know the details) Plus maybe some others? I think externally_managed covers all the strange cases you don't necessarily want to accomodate in the builder. The transaction issue certainly will bite people who don't know about it, so I think this is a good thing. Ted - Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ Rose-db-object mailing list Rose-db-object@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/rose-db-object
Re: [RDBO] Too Many connections
On Jan 11, 2008 1:10 PM, Ted Zlatanov <[EMAIL PROTECTED]> wrote: > I think the automatic class builder should have a required option to set > the default explicitly (it didn't, last time I looked). That would > ensure the developer thought about it even in the path of least > resistance. Wat would the be the possible values for that option, an what code would each option produce? -John - Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ Rose-db-object mailing list Rose-db-object@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/rose-db-object
Re: [RDBO] Too Many connections
On Fri, 11 Jan 2008 07:43:23 -0500 "John Siracusa" <[EMAIL PROTECTED]> wrote: JS> That's why I think the default unshared behavior is the "safest" in JS> that sharing now requires some action (and therefore some thought) on JS> the part of the developer. The required action is simple (usually a JS> line or two of code in one place, as you've seen) and I like the fact JS> that it forces the developer to actually formulate and stick to some JS> sort of overall policy for connection sharing, rather than just JS> winging it without really understanding what's going on. I think the automatic class builder should have a required option to set the default explicitly (it didn't, last time I looked). That would ensure the developer thought about it even in the path of least resistance. Ted - Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ Rose-db-object mailing list Rose-db-object@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/rose-db-object
Re: [RDBO] Too Many connections
On Jan 11, 2008 4:43 AM, James Masters <[EMAIL PROTECTED]> wrote: > In my naivety, it seems to me that conceptually a script would normally have > a single shared connection for all it's calls. Both from a tidiness point > of view and also because I guess it would be better performing not to keep > opening new connections (?). What is the reason for having the default > opening a new connection and not using an existing one? > > i.e. Are there any disadvantages in having a single shared connection for > multiple calls? If there were one obvious default behavior for managing db connections that everyone expected, I'd do it. Trouble is, there is no single "best" policy, and everyone's expectations differ. The existing "default unshared" behavior is intended to prevent situations where a user unfamiliar with transactions does a commit or rollback in one part of his code and is surprised to find that it committed (or undid) work done previously in another part of the code. This behavior is "obvious" if you a) understand transactions, and b) are fully aware that database connections are being shared. But you've seen, many people don't actually know what the default db connection behavior is in RDBO, so even if they understand transactions they still may be surprised by strange bugs that cause "spooky" effects in widely separated pieces of their own code. That's why I think the default unshared behavior is the "safest" in that sharing now requires some action (and therefore some thought) on the part of the developer. The required action is simple (usually a line or two of code in one place, as you've seen) and I like the fact that it forces the developer to actually formulate and stick to some sort of overall policy for connection sharing, rather than just winging it without really understanding what's going on. -John - Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ Rose-db-object mailing list Rose-db-object@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/rose-db-object
Re: [RDBO] Too Many connections
Well explained; I think I understand, thank you.
In my naivety, it seems to me that conceptually a script would normally have
a single shared connection for all it's calls. Both from a tidiness point
of view and also because I guess it would be better performing not to keep
opening new connections (?). What is the reason for having the default
opening a new connection and not using an existing one?
i.e. Are there any disadvantages in having a single shared connection for
multiple calls?
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Behalf Of John
Siracusa
Sent: 11 January 2008 01:25
To: Rose-DB-Object
Subject: Re: [RDBO] Too Many connections
On 1/10/08 5:30 PM, James Masters wrote:
> Looking into this, I seem to simply be doing the following in a "for" loop
> more than 100 times:
>
> my $shipobj = MGORD::Shipment->new(shipid => $shipid)->load(with
=>['items',
> 'costs', 'problems', 'predictedcosts', 'toaddress', 'fromaddress']);
>
> So the implication is that each one of these is opening a new client
> connection and not closing it.
>
> I know I can just increase the default connections limit from 100 to say
200
> to fix the problem. But is Rose really supposed to be creating all these
> client connections or have I written something wrongly?
When an RDBO-derived object needs a database connection, it calls its db()
method:
http://search.cpan.org/dist/Rose-DB-Object/lib/Rose/DB/Object.pm#db
If a db object does not yet exist, the init_db() method is called to create
one:
http://search.cpan.org/dist/Rose-DB-Object/lib/Rose/DB/Object.pm#init_db
The default implementation of init_db() returns a new Rose::DB object, which
in turn will create its own DBI database connection.
If you'd like one or more objects to share a single Rose::DB-derived object
(and therefore a single database connection), a straightforward way to do it
is to explicitly code the sharing:
my $db = My::DB->new; # share this among all objects created below
for(...)
{
my $shipobj =
MGORD::Shipment->new(db => $db, ...)->load(...);
}
If you'd rather not do this explicitly, or if you simply want another
default policy, then you should override the init_db() method, either in
your common base class or in each individual RDBO-derived class, depending
on the policy you want.
For example, if you want all objects of or derived from a given class to
share a single database connection by default, you could use the
new_or_cached() Rose::DB method:
http://search.cpan.org/dist/Rose-DB/lib/Rose/DB.pm#new_or_cached
writing an init_db() method like this:
sub init_db { My::DB->new_or_cached }
Remember that no matter what policy you implement in init_db(), you can
always explicitly create and pass a Rose::DB-derived object as that value of
a "db" parameter to any individual object or Manager call.
-John
-
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
___
Rose-db-object mailing list
Rose-db-object@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rose-db-object
-
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
___
Rose-db-object mailing list
Rose-db-object@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rose-db-object
Re: [RDBO] Too Many connections
On 1/10/08 5:30 PM, James Masters wrote:
> Looking into this, I seem to simply be doing the following in a "for" loop
> more than 100 times:
>
> my $shipobj = MGORD::Shipment->new(shipid => $shipid)->load(with =>['items',
> 'costs', 'problems', 'predictedcosts', 'toaddress', 'fromaddress']);
>
> So the implication is that each one of these is opening a new client
> connection and not closing it.
>
> I know I can just increase the default connections limit from 100 to say 200
> to fix the problem. But is Rose really supposed to be creating all these
> client connections or have I written something wrongly?
When an RDBO-derived object needs a database connection, it calls its db()
method:
http://search.cpan.org/dist/Rose-DB-Object/lib/Rose/DB/Object.pm#db
If a db object does not yet exist, the init_db() method is called to create
one:
http://search.cpan.org/dist/Rose-DB-Object/lib/Rose/DB/Object.pm#init_db
The default implementation of init_db() returns a new Rose::DB object, which
in turn will create its own DBI database connection.
If you'd like one or more objects to share a single Rose::DB-derived object
(and therefore a single database connection), a straightforward way to do it
is to explicitly code the sharing:
my $db = My::DB->new; # share this among all objects created below
for(...)
{
my $shipobj =
MGORD::Shipment->new(db => $db, ...)->load(...);
}
If you'd rather not do this explicitly, or if you simply want another
default policy, then you should override the init_db() method, either in
your common base class or in each individual RDBO-derived class, depending
on the policy you want.
For example, if you want all objects of or derived from a given class to
share a single database connection by default, you could use the
new_or_cached() Rose::DB method:
http://search.cpan.org/dist/Rose-DB/lib/Rose/DB.pm#new_or_cached
writing an init_db() method like this:
sub init_db { My::DB->new_or_cached }
Remember that no matter what policy you implement in init_db(), you can
always explicitly create and pass a Rose::DB-derived object as that value of
a "db" parameter to any individual object or Manager call.
-John
-
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
___
Rose-db-object mailing list
Rose-db-object@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rose-db-object
