subject:"RE\: \[Catalyst\] DB values that change very rarely"

Re: [Catalyst] DB values that change very rarely

2013-01-14 Thread Robert Rothenberg

I have a caching plugin that would be useful. Cache::HTTP::Preempt. Will
upload to CPAN soon but I am moving house now.
On Jan 14, 2013 8:18 AM, "Alexander Hartmaier" <
alexander.hartma...@t-systems.at> wrote:

> On 2013-01-11 19:05, Francisco Obispo wrote:
> > You really don't want this in DBIx::Class, you want fine grain control
> of when to fetch from the database, etc. Anyone can build their layer on
> top of DBIx::Class to implement caching.
> >
> > Using something like memcached would save you some database hits, you
> can either set the values to expire at a specified time / number of
> seconds, or you can set/delete keys as part of the procedures at
> insert/update/delete on the database.
> >
> > If you you Postgresql, you can actually take advantage of PgPool, which
> will give you the option to enable a caching layer…
> >
> > Francisco
> Please don't top post!
> I *do* want this in DBIC so that a prefetch specified in some query
> doesn't hit the database if the table is cached schema-wide. Any other
> caching technique would mean changing every piece of code whereas this
> would be transparent. I do know DBIx::Class quite well and don't think
> that's possible with a component.
> >
> >
> >
> > On Jan 11, 2013, at 4:01 AM, Alexander Hartmaier <
> alexander.hartma...@t-systems.at> wrote:
> >
> >> On 2013-01-10 16:15, Jesse Sheidlower wrote:
> >>> In one of my apps, I have a number of tables that contain values that
> >>> change very rarely. Think of something like a "category" table in an
> >>> e-commerce app--they're mostly static, but every now and then you need
> >>> to change or add or delete a category. These occasional changes do,
> >>> however, need to be made by a (privileged) user, rather than a
> >>> developer, so I can't just put them in the config file and edit this
> and
> >>> restart the app when necessary. Since I don't know when they might
> >>> change, I don't know how I could cache them, because when a change is
> >>> made it needs to take effect immediately.
> >>>
> >>> These tables are used on almost every request. The result is that most
> >>> of my controllers end up looking something like this, or having
> >>> something like this at the end of them:
> >>>
> >>> sub add : Local {
> >>>   my ( $self, $c ) = @_;
> >>>
> >>>   $c->stash->{title} = 'Add a new lemma';
> >>>
> >>>   $c->stash->{batches} = $c->model('WordsDB::Batch')->search();
> >>>   $c->stash->{statuses} = $c->model('WordsDB::Status')->search();
> >>>   $c->stash->{regions} = $c->model('WordsDB::Region')->search();
> >>>   $c->stash->{subjects} = $c->model('WordsDB::Subject')->search();
> >>>
> >>>   $c->stash->{template} = 'add.tt';
> >>> }
> >>>
> >>> This means that almost every page generation hits the database a whole
> bunch of
> >>> unnecessary times, and that all of my controllers are cluttered.
> >>>
> >>> This must be a fairly common problem. What's the best way to deal with
> >>> it--both the desire to persist what is mostly static data, and to keep
> >>> my controllers clean?
> >>>
> >>> Thanks.
> >>>
> >>> Jesse Sheidlower
> >>>
> >>> ___
> >>> List: Catalyst@lists.scsys.co.uk
> >>> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> >>> Searchable archive:
> http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> >>> Dev site: http://dev.catalyst.perl.org/
> >> I'd LOVE to see that functionality in DBIx::Class::Schema so prefetches
> >> use the cached values to populate the related data instead of hitting
> >> the database.
> >>
> >> --
> >> Best regards, Alexander Hartmaier
> >>
> >>
> >>
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
> >> T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
> >> Handelsgericht Wien, FN 79340b
> >>
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
> >> Notice: This e-mail contains information that is confidential and may
> be privileged.
> >> If you are not the intended recipient, please notify the sender and then
> >> delete this e-mail immediately.
> >>
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
> >>
> >> ___
> >> List: Catalyst@lists.scsys.co.uk
> >> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> >> Searchable archive:
> http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> >> Dev site: http://dev.catalyst.perl.org/
> > Francisco Obispo
> > Director of Applications and Services - ISC
> > email: fobi...@isc.org
> > Phone: +1 650 423 1374 || INOC-DBA *3557* NOC
> > PGP KeyID = B38DB1BE
> >
> >
> > ___
> > List: Catalyst@lists.scsys.co.uk
> > Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> > Searchable archive:
> http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> > Dev site: http://dev.catalyst.perl.org/
>
> --
> LG Alex
>
>
>

Re: [Catalyst] DB values that change very rarely

2013-01-14 Thread Alexander Hartmaier

On 2013-01-11 19:05, Francisco Obispo wrote:
> You really don't want this in DBIx::Class, you want fine grain control of 
> when to fetch from the database, etc. Anyone can build their layer on top of 
> DBIx::Class to implement caching.
>
> Using something like memcached would save you some database hits, you can 
> either set the values to expire at a specified time / number of seconds, or 
> you can set/delete keys as part of the procedures at insert/update/delete on 
> the database.
>
> If you you Postgresql, you can actually take advantage of PgPool, which will 
> give you the option to enable a caching layer… 
>
> Francisco
Please don't top post!
I *do* want this in DBIC so that a prefetch specified in some query
doesn't hit the database if the table is cached schema-wide. Any other
caching technique would mean changing every piece of code whereas this
would be transparent. I do know DBIx::Class quite well and don't think
that's possible with a component.
>
>
>
> On Jan 11, 2013, at 4:01 AM, Alexander Hartmaier 
>  wrote:
>
>> On 2013-01-10 16:15, Jesse Sheidlower wrote:
>>> In one of my apps, I have a number of tables that contain values that
>>> change very rarely. Think of something like a "category" table in an
>>> e-commerce app--they're mostly static, but every now and then you need
>>> to change or add or delete a category. These occasional changes do,
>>> however, need to be made by a (privileged) user, rather than a
>>> developer, so I can't just put them in the config file and edit this and
>>> restart the app when necessary. Since I don't know when they might
>>> change, I don't know how I could cache them, because when a change is
>>> made it needs to take effect immediately.
>>>
>>> These tables are used on almost every request. The result is that most
>>> of my controllers end up looking something like this, or having
>>> something like this at the end of them:
>>>
>>> sub add : Local {
>>>   my ( $self, $c ) = @_;
>>>
>>>   $c->stash->{title} = 'Add a new lemma';
>>>
>>>   $c->stash->{batches} = $c->model('WordsDB::Batch')->search();
>>>   $c->stash->{statuses} = $c->model('WordsDB::Status')->search();
>>>   $c->stash->{regions} = $c->model('WordsDB::Region')->search();
>>>   $c->stash->{subjects} = $c->model('WordsDB::Subject')->search();
>>>
>>>   $c->stash->{template} = 'add.tt';
>>> }
>>>
>>> This means that almost every page generation hits the database a whole 
>>> bunch of
>>> unnecessary times, and that all of my controllers are cluttered.
>>>
>>> This must be a fairly common problem. What's the best way to deal with
>>> it--both the desire to persist what is mostly static data, and to keep
>>> my controllers clean?
>>>
>>> Thanks.
>>>
>>> Jesse Sheidlower
>>>
>>> ___
>>> List: Catalyst@lists.scsys.co.uk
>>> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
>>> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
>>> Dev site: http://dev.catalyst.perl.org/
>> I'd LOVE to see that functionality in DBIx::Class::Schema so prefetches
>> use the cached values to populate the related data instead of hitting
>> the database.
>>
>> --
>> Best regards, Alexander Hartmaier
>>
>>
>> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
>> T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
>> Handelsgericht Wien, FN 79340b
>> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
>> Notice: This e-mail contains information that is confidential and may be 
>> privileged.
>> If you are not the intended recipient, please notify the sender and then
>> delete this e-mail immediately.
>> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
>>
>> ___
>> List: Catalyst@lists.scsys.co.uk
>> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
>> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
>> Dev site: http://dev.catalyst.perl.org/
> Francisco Obispo 
> Director of Applications and Services - ISC
> email: fobi...@isc.org
> Phone: +1 650 423 1374 || INOC-DBA *3557* NOC
> PGP KeyID = B38DB1BE
>
>
> ___
> List: Catalyst@lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/

-- 
LG Alex


___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

Re: [Catalyst] DB values that change very rarely

2013-01-11 Thread Francisco Obispo

You really don't want this in DBIx::Class, you want fine grain control of when 
to fetch from the database, etc. Anyone can build their layer on top of 
DBIx::Class to implement caching.

Using something like memcached would save you some database hits, you can 
either set the values to expire at a specified time / number of seconds, or you 
can set/delete keys as part of the procedures at insert/update/delete on the 
database.

If you you Postgresql, you can actually take advantage of PgPool, which will 
give you the option to enable a caching layer… 

Francisco



On Jan 11, 2013, at 4:01 AM, Alexander Hartmaier 
 wrote:

> On 2013-01-10 16:15, Jesse Sheidlower wrote:
>> In one of my apps, I have a number of tables that contain values that
>> change very rarely. Think of something like a "category" table in an
>> e-commerce app--they're mostly static, but every now and then you need
>> to change or add or delete a category. These occasional changes do,
>> however, need to be made by a (privileged) user, rather than a
>> developer, so I can't just put them in the config file and edit this and
>> restart the app when necessary. Since I don't know when they might
>> change, I don't know how I could cache them, because when a change is
>> made it needs to take effect immediately.
>> 
>> These tables are used on almost every request. The result is that most
>> of my controllers end up looking something like this, or having
>> something like this at the end of them:
>> 
>> sub add : Local {
>>   my ( $self, $c ) = @_;
>> 
>>   $c->stash->{title} = 'Add a new lemma';
>> 
>>   $c->stash->{batches} = $c->model('WordsDB::Batch')->search();
>>   $c->stash->{statuses} = $c->model('WordsDB::Status')->search();
>>   $c->stash->{regions} = $c->model('WordsDB::Region')->search();
>>   $c->stash->{subjects} = $c->model('WordsDB::Subject')->search();
>> 
>>   $c->stash->{template} = 'add.tt';
>> }
>> 
>> This means that almost every page generation hits the database a whole bunch 
>> of
>> unnecessary times, and that all of my controllers are cluttered.
>> 
>> This must be a fairly common problem. What's the best way to deal with
>> it--both the desire to persist what is mostly static data, and to keep
>> my controllers clean?
>> 
>> Thanks.
>> 
>> Jesse Sheidlower
>> 
>> ___
>> List: Catalyst@lists.scsys.co.uk
>> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
>> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
>> Dev site: http://dev.catalyst.perl.org/
> I'd LOVE to see that functionality in DBIx::Class::Schema so prefetches
> use the cached values to populate the related data instead of hitting
> the database.
> 
> --
> Best regards, Alexander Hartmaier
> 
> 
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
> T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
> Handelsgericht Wien, FN 79340b
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
> Notice: This e-mail contains information that is confidential and may be 
> privileged.
> If you are not the intended recipient, please notify the sender and then
> delete this e-mail immediately.
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
> 
> ___
> List: Catalyst@lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/

Francisco Obispo 
Director of Applications and Services - ISC
email: fobi...@isc.org
Phone: +1 650 423 1374 || INOC-DBA *3557* NOC
PGP KeyID = B38DB1BE


___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

Re: [Catalyst] DB values that change very rarely

2013-01-11 Thread Alexander Hartmaier

On 2013-01-10 16:15, Jesse Sheidlower wrote:
> In one of my apps, I have a number of tables that contain values that
> change very rarely. Think of something like a "category" table in an
> e-commerce app--they're mostly static, but every now and then you need
> to change or add or delete a category. These occasional changes do,
> however, need to be made by a (privileged) user, rather than a
> developer, so I can't just put them in the config file and edit this and
> restart the app when necessary. Since I don't know when they might
> change, I don't know how I could cache them, because when a change is
> made it needs to take effect immediately.
>
> These tables are used on almost every request. The result is that most
> of my controllers end up looking something like this, or having
> something like this at the end of them:
>
> sub add : Local {
>my ( $self, $c ) = @_;
>
>$c->stash->{title} = 'Add a new lemma';
>
>$c->stash->{batches} = $c->model('WordsDB::Batch')->search();
>$c->stash->{statuses} = $c->model('WordsDB::Status')->search();
>$c->stash->{regions} = $c->model('WordsDB::Region')->search();
>$c->stash->{subjects} = $c->model('WordsDB::Subject')->search();
>
>$c->stash->{template} = 'add.tt';
> }
>
> This means that almost every page generation hits the database a whole bunch 
> of
> unnecessary times, and that all of my controllers are cluttered.
>
> This must be a fairly common problem. What's the best way to deal with
> it--both the desire to persist what is mostly static data, and to keep
> my controllers clean?
>
> Thanks.
>
> Jesse Sheidlower
>
> ___
> List: Catalyst@lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/
I'd LOVE to see that functionality in DBIx::Class::Schema so prefetches
use the cached values to populate the related data instead of hitting
the database.

--
Best regards, Alexander Hartmaier


*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
Handelsgericht Wien, FN 79340b
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
Notice: This e-mail contains information that is confidential and may be 
privileged.
If you are not the intended recipient, please notify the sender and then
delete this e-mail immediately.
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*

___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

RE: [Catalyst] DB values that change very rarely

2013-01-10 Thread Dami Laurent (PJ)

>-Message d'origine-
>De : Jesse Sheidlower [mailto:jes...@panix.com]
>Envoyé : jeudi, 10. janvier 2013 16:16
>À : catalyst@lists.scsys.co.uk
>Objet : [Catalyst] DB values that change very rarely
>
>
>In one of my apps, I have a number of tables that contain values that
>change very rarely. 
[...]
>This means that almost every page generation hits the database a whole bunch of
>unnecessary times, and that all of my controllers are cluttered.
>
>This must be a fairly common problem. What's the best way to deal with
>it--both the desire to persist what is mostly static data, and to keep
>my controllers clean?

Hi Jesse,

First of all, you need a piece of information that tells you if the 
configuration data has changed. It can be a version number or a timestamp 
column, either associated to each table (if these tables change individually), 
or in a global "version table" that covers everything. Every time that a power 
user edits such data, your version number or timestamp must be updated. 

Or maybe all this configuration data that doesn't change very often could be 
stored, not in database, but in a config file (using YAML, XML, or 
Config::General). In that case you have the mtime of the file to tell you if 
the data has changed.

Based on this, your controller can use any of the CPAN caching modules to keep 
the data in memory; you still need to go to the datasource to check if the 
version number or timestamp has changed, but that's a much smaller request.

Another way would be to exploit the caching functionality of the browser 
itself. In that case, the configuration data should be at a separate URL, like 
/myApp/config.json. Every page should use client-side javascript to exploit the 
config.json data (for exemple for building the various OPTIONS of a SELECT). 
Each page of the application loads /myApp/config.json, either through a

Re: [Catalyst] DB values that change very rarely

2013-01-10 Thread Len Jaffe

On Thu, Jan 10, 2013 at 10:15 AM, Jesse Sheidlower  wrote:

> Since I don't know when they might
> change, I don't know how I could cache them, because when a change is
> made it needs to take effect immediately.
>
> These tables are used on almost every request.

Use a cache that allows you to give names to cached objects.
Store your category list in the cache under a name like
"global-category-list'.

When ever you change the categories, you expire the named object.  The you
either let the next lookup regenerate the list, or regenerate it when the
update is made to the list.

All access to the cached data must be performed through a machanism that
wraps the cache.

Len.

Did I hear you on "how to do everything", talking about Crash Blossoms?

-- 
lenja...@jaffesystems.com   614-404-4214   www.lenjaffe.com
Proprietor: http://www.theycomewithcheese.com/ - An Homage to Fromage
Perl Advent Planet  - Advent
Calendars: Perlish and otherwise.
Greenbar : Grubmaster: 2012-2009, Grub
Asst: 2008, Trained: 2007.
___
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

Re: [Catalyst] DB values that change very rarely

Re: [Catalyst] DB values that change very rarely

Re: [Catalyst] DB values that change very rarely

Re: [Catalyst] DB values that change very rarely

RE: [Catalyst] DB values that change very rarely

Re: [Catalyst] DB values that change very rarely

6 matches

Site Navigation

Mail list logo

Footer information