Re: [Catalyst] DB values that change very rarely
On 2013-01-11 19:05, Francisco Obispo wrote:
You really don't want this in DBIx::Class, you want fine grain control of
when to fetch from the database, etc. Anyone can build their layer on top of
DBIx::Class to implement caching.
Using something like memcached would save you some database hits, you can
either set the values to expire at a specified time / number of seconds, or
you can set/delete keys as part of the procedures at insert/update/delete on
the database.
If you you Postgresql, you can actually take advantage of PgPool, which will
give you the option to enable a caching layer…
Francisco
Please don't top post!
I *do* want this in DBIC so that a prefetch specified in some query
doesn't hit the database if the table is cached schema-wide. Any other
caching technique would mean changing every piece of code whereas this
would be transparent. I do know DBIx::Class quite well and don't think
that's possible with a component.
On Jan 11, 2013, at 4:01 AM, Alexander Hartmaier
alexander.hartma...@t-systems.at wrote:
On 2013-01-10 16:15, Jesse Sheidlower wrote:
In one of my apps, I have a number of tables that contain values that
change very rarely. Think of something like a category table in an
e-commerce app--they're mostly static, but every now and then you need
to change or add or delete a category. These occasional changes do,
however, need to be made by a (privileged) user, rather than a
developer, so I can't just put them in the config file and edit this and
restart the app when necessary. Since I don't know when they might
change, I don't know how I could cache them, because when a change is
made it needs to take effect immediately.
These tables are used on almost every request. The result is that most
of my controllers end up looking something like this, or having
something like this at the end of them:
sub add : Local {
my ( $self, $c ) = @_;
$c-stash-{title} = 'Add a new lemma';
$c-stash-{batches} = $c-model('WordsDB::Batch')-search();
$c-stash-{statuses} = $c-model('WordsDB::Status')-search();
$c-stash-{regions} = $c-model('WordsDB::Region')-search();
$c-stash-{subjects} = $c-model('WordsDB::Subject')-search();
$c-stash-{template} = 'add.tt';
}
This means that almost every page generation hits the database a whole
bunch of
unnecessary times, and that all of my controllers are cluttered.
This must be a fairly common problem. What's the best way to deal with
it--both the desire to persist what is mostly static data, and to keep
my controllers clean?
Thanks.
Jesse Sheidlower
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
I'd LOVE to see that functionality in DBIx::Class::Schema so prefetches
use the cached values to populate the related data instead of hitting
the database.
--
Best regards, Alexander Hartmaier
***
T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
Handelsgericht Wien, FN 79340b
***
Notice: This e-mail contains information that is confidential and may be
privileged.
If you are not the intended recipient, please notify the sender and then
delete this e-mail immediately.
***
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Francisco Obispo
Director of Applications and Services - ISC
email: fobi...@isc.org
Phone: +1 650 423 1374 || INOC-DBA *3557* NOC
PGP KeyID = B38DB1BE
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
--
LG Alex
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] DB values that change very rarely
On 2013-01-10 16:15, Jesse Sheidlower wrote:
In one of my apps, I have a number of tables that contain values that
change very rarely. Think of something like a category table in an
e-commerce app--they're mostly static, but every now and then you need
to change or add or delete a category. These occasional changes do,
however, need to be made by a (privileged) user, rather than a
developer, so I can't just put them in the config file and edit this and
restart the app when necessary. Since I don't know when they might
change, I don't know how I could cache them, because when a change is
made it needs to take effect immediately.
These tables are used on almost every request. The result is that most
of my controllers end up looking something like this, or having
something like this at the end of them:
sub add : Local {
my ( $self, $c ) = @_;
$c-stash-{title} = 'Add a new lemma';
$c-stash-{batches} = $c-model('WordsDB::Batch')-search();
$c-stash-{statuses} = $c-model('WordsDB::Status')-search();
$c-stash-{regions} = $c-model('WordsDB::Region')-search();
$c-stash-{subjects} = $c-model('WordsDB::Subject')-search();
$c-stash-{template} = 'add.tt';
}
This means that almost every page generation hits the database a whole bunch
of
unnecessary times, and that all of my controllers are cluttered.
This must be a fairly common problem. What's the best way to deal with
it--both the desire to persist what is mostly static data, and to keep
my controllers clean?
Thanks.
Jesse Sheidlower
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
I'd LOVE to see that functionality in DBIx::Class::Schema so prefetches
use the cached values to populate the related data instead of hitting
the database.
--
Best regards, Alexander Hartmaier
***
T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
Handelsgericht Wien, FN 79340b
***
Notice: This e-mail contains information that is confidential and may be
privileged.
If you are not the intended recipient, please notify the sender and then
delete this e-mail immediately.
***
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] DB values that change very rarely
You really don't want this in DBIx::Class, you want fine grain control of when
to fetch from the database, etc. Anyone can build their layer on top of
DBIx::Class to implement caching.
Using something like memcached would save you some database hits, you can
either set the values to expire at a specified time / number of seconds, or you
can set/delete keys as part of the procedures at insert/update/delete on the
database.
If you you Postgresql, you can actually take advantage of PgPool, which will
give you the option to enable a caching layer…
Francisco
On Jan 11, 2013, at 4:01 AM, Alexander Hartmaier
alexander.hartma...@t-systems.at wrote:
On 2013-01-10 16:15, Jesse Sheidlower wrote:
In one of my apps, I have a number of tables that contain values that
change very rarely. Think of something like a category table in an
e-commerce app--they're mostly static, but every now and then you need
to change or add or delete a category. These occasional changes do,
however, need to be made by a (privileged) user, rather than a
developer, so I can't just put them in the config file and edit this and
restart the app when necessary. Since I don't know when they might
change, I don't know how I could cache them, because when a change is
made it needs to take effect immediately.
These tables are used on almost every request. The result is that most
of my controllers end up looking something like this, or having
something like this at the end of them:
sub add : Local {
my ( $self, $c ) = @_;
$c-stash-{title} = 'Add a new lemma';
$c-stash-{batches} = $c-model('WordsDB::Batch')-search();
$c-stash-{statuses} = $c-model('WordsDB::Status')-search();
$c-stash-{regions} = $c-model('WordsDB::Region')-search();
$c-stash-{subjects} = $c-model('WordsDB::Subject')-search();
$c-stash-{template} = 'add.tt';
}
This means that almost every page generation hits the database a whole bunch
of
unnecessary times, and that all of my controllers are cluttered.
This must be a fairly common problem. What's the best way to deal with
it--both the desire to persist what is mostly static data, and to keep
my controllers clean?
Thanks.
Jesse Sheidlower
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
I'd LOVE to see that functionality in DBIx::Class::Schema so prefetches
use the cached values to populate the related data instead of hitting
the database.
--
Best regards, Alexander Hartmaier
***
T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
Handelsgericht Wien, FN 79340b
***
Notice: This e-mail contains information that is confidential and may be
privileged.
If you are not the intended recipient, please notify the sender and then
delete this e-mail immediately.
***
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Francisco Obispo
Director of Applications and Services - ISC
email: fobi...@isc.org
Phone: +1 650 423 1374 || INOC-DBA *3557* NOC
PGP KeyID = B38DB1BE
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
[Catalyst] DB values that change very rarely
In one of my apps, I have a number of tables that contain values that
change very rarely. Think of something like a category table in an
e-commerce app--they're mostly static, but every now and then you need
to change or add or delete a category. These occasional changes do,
however, need to be made by a (privileged) user, rather than a
developer, so I can't just put them in the config file and edit this and
restart the app when necessary. Since I don't know when they might
change, I don't know how I could cache them, because when a change is
made it needs to take effect immediately.
These tables are used on almost every request. The result is that most
of my controllers end up looking something like this, or having
something like this at the end of them:
sub add : Local {
my ( $self, $c ) = @_;
$c-stash-{title} = 'Add a new lemma';
$c-stash-{batches} = $c-model('WordsDB::Batch')-search();
$c-stash-{statuses} = $c-model('WordsDB::Status')-search();
$c-stash-{regions} = $c-model('WordsDB::Region')-search();
$c-stash-{subjects} = $c-model('WordsDB::Subject')-search();
$c-stash-{template} = 'add.tt';
}
This means that almost every page generation hits the database a whole bunch of
unnecessary times, and that all of my controllers are cluttered.
This must be a fairly common problem. What's the best way to deal with
it--both the desire to persist what is mostly static data, and to keep
my controllers clean?
Thanks.
Jesse Sheidlower
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] DB values that change very rarely
On Thu, Jan 10, 2013 at 10:15 AM, Jesse Sheidlower jes...@panix.com wrote: Since I don't know when they might change, I don't know how I could cache them, because when a change is made it needs to take effect immediately. These tables are used on almost every request. Use a cache that allows you to give names to cached objects. Store your category list in the cache under a name like global-category-list'. When ever you change the categories, you expire the named object. The you either let the next lookup regenerate the list, or regenerate it when the update is made to the list. All access to the cached data must be performed through a machanism that wraps the cache. Len. Did I hear you on how to do everything, talking about Crash Blossoms? -- lenja...@jaffesystems.com 614-404-4214 www.lenjaffe.com Proprietor: http://www.theycomewithcheese.com/ - An Homage to Fromage Perl Advent Planet http://www.lenjaffe.com/PerlAdventPlanet/ - Advent Calendars: Perlish and otherwise. Greenbar http://www.greenbartraining.org/: Grubmaster: 2012-2009, Grub Asst: 2008, Trained: 2007. ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
RE: [Catalyst] DB values that change very rarely
-Message d'origine- De : Jesse Sheidlower [mailto:jes...@panix.com] Envoyé : jeudi, 10. janvier 2013 16:16 À : catalyst@lists.scsys.co.uk Objet : [Catalyst] DB values that change very rarely In one of my apps, I have a number of tables that contain values that change very rarely. [...] This means that almost every page generation hits the database a whole bunch of unnecessary times, and that all of my controllers are cluttered. This must be a fairly common problem. What's the best way to deal with it--both the desire to persist what is mostly static data, and to keep my controllers clean? Hi Jesse, First of all, you need a piece of information that tells you if the configuration data has changed. It can be a version number or a timestamp column, either associated to each table (if these tables change individually), or in a global version table that covers everything. Every time that a power user edits such data, your version number or timestamp must be updated. Or maybe all this configuration data that doesn't change very often could be stored, not in database, but in a config file (using YAML, XML, or Config::General). In that case you have the mtime of the file to tell you if the data has changed. Based on this, your controller can use any of the CPAN caching modules to keep the data in memory; you still need to go to the datasource to check if the version number or timestamp has changed, but that's a much smaller request. Another way would be to exploit the caching functionality of the browser itself. In that case, the configuration data should be at a separate URL, like /myApp/config.json. Every page should use client-side javascript to exploit the config.json data (for exemple for building the various OPTIONS of a SELECT). Each page of the application loads /myApp/config.json, either through a script src=..., or through an Ajax call.The controller that serves this config.json should implement conditional GET, i.e. read the If-Modified-Since or If-None-Match headers, ask the datasource for its timestamp, and return 302 NOT MODIFIED if the client tag is in sync with the server data. That's a client-server round trip, but it's very fast because there is almost no content to be exchanged. Furthermore, the advantage is that the /config.json data is shared among all pages of your app -- it only travelled once from the server to the client. Hope this helps -- good luck, Laurent Dami ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
