Re: [Dbix-class] Caching a resultset?
On Thu, Jan 17, 2013 at 01:06:43AM +1100, Peter Rabbitson wrote: On Wed, Jan 16, 2013 at 02:47:19PM +0100, Alexander Hartmaier wrote: I had NO idea that's already possible, thanks for the great pointers! Do we want to include that in the Cookbook? There is no such thing as too much documentation. Please go for it. Also see if you can do something about [1] while you are at it would be great. [1] http://lists.scsys.co.uk/pipermail/dbix-class/2013-January/011040.html Any luck with that? :) ___ List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class IRC: irc.perl.org#dbix-class SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/ Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On 2013-01-15 21:49, Peter Rabbitson wrote:
On Tue, Jan 15, 2013 at 05:12:12PM +0100, Alexander Hartmaier wrote:
On 2013-01-15 16:51, Peter Rabbitson wrote:
On Tue, Jan 15, 2013 at 10:09:19AM -0500, Jesse Sheidlower wrote:
On Tue, Jan 15, 2013 at 05:18:06AM +1100, Peter Rabbitson wrote:
On Mon, Jan 14, 2013 at 11:07:34AM -0500, Jesse Sheidlower wrote:
So apart from the mystery around the nonsensical exception - does this
help?
local $DBIx::Class::ResultSourceHandle::thaw_schema =
$c-model(...)-schema;
Yes! Putting that before the unless seems to make everything work
correctly Will be testing further this afternoon. Thanks!
Please let me know if wider testing is succesful.
In one sense, wider testing was indeed successful. The RS was cached
properly, it worked in several different environments, etc.
However, it didn't work in one particularly crucial way, which in
retrospect I'm surprised I didn't think about: the resultset may have
been cached, but once I get it back, it's still just a resultset, so
when I use it again in the template, it hits the database again. Every
time.
This is exactly what I wanted to avoid by this whole escapade.
You need to combine the above with [1] and/or [2].
[1] https://metacpan.org/module/DBIx::Class::ResultSet#cache
[2] https://metacpan.org/module/DBIx::Class::ResultSet#set_cache
Cheers
You mean don't prefetch the relationships and populate it's cache after
fetching all results?
Um... prefetch does not touch the cache of the *main* rs. I am not sure
what you mean above - explain.
My schema has quite some 'side tables' like types, stati and so on. They
almost never change.
It would help performance if I could tell the schema to load all rows,
cache them and populate a relationship from the cache when prefetched.
For example
Interface-belongs_to('rel_interface_type', ::Interface_Type,
'fk_interface_type');
Schema-connect({
... ,
prefetch = [qw( Interface_Type )],
});
# this would not prefetch rel_interface_type like it would normally do
but populate the rel_interface_type cache from the cached Interface_Type's
my $rs_interfaces = Interface-search_rs(..., { prefetch =
'rel_interface_type' });
Does this make sense?
Cheers
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
***
T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
Handelsgericht Wien, FN 79340b
***
Notice: This e-mail contains information that is confidential and may be
privileged.
If you are not the intended recipient, please notify the sender and then
delete this e-mail immediately.
***
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On Wed, Jan 16, 2013 at 11:03:43AM +0100, Alexander Hartmaier wrote:
On 2013-01-15 21:49, Peter Rabbitson wrote:
On Tue, Jan 15, 2013 at 05:12:12PM +0100, Alexander Hartmaier wrote:
On 2013-01-15 16:51, Peter Rabbitson wrote:
On Tue, Jan 15, 2013 at 10:09:19AM -0500, Jesse Sheidlower wrote:
On Tue, Jan 15, 2013 at 05:18:06AM +1100, Peter Rabbitson wrote:
On Mon, Jan 14, 2013 at 11:07:34AM -0500, Jesse Sheidlower wrote:
So apart from the mystery around the nonsensical exception - does
this help?
local $DBIx::Class::ResultSourceHandle::thaw_schema =
$c-model(...)-schema;
Yes! Putting that before the unless seems to make everything work
correctly Will be testing further this afternoon. Thanks!
Please let me know if wider testing is succesful.
In one sense, wider testing was indeed successful. The RS was cached
properly, it worked in several different environments, etc.
However, it didn't work in one particularly crucial way, which in
retrospect I'm surprised I didn't think about: the resultset may have
been cached, but once I get it back, it's still just a resultset, so
when I use it again in the template, it hits the database again. Every
time.
This is exactly what I wanted to avoid by this whole escapade.
You need to combine the above with [1] and/or [2].
[1] https://metacpan.org/module/DBIx::Class::ResultSet#cache
[2] https://metacpan.org/module/DBIx::Class::ResultSet#set_cache
Cheers
You mean don't prefetch the relationships and populate it's cache after
fetching all results?
Um... prefetch does not touch the cache of the *main* rs. I am not sure
what you mean above - explain.
My schema has quite some 'side tables' like types, stati and so on. They
almost never change.
It would help performance if I could tell the schema to load all rows,
cache them and populate a relationship from the cache when prefetched.
For example
Interface-belongs_to('rel_interface_type', ::Interface_Type,
'fk_interface_type');
Schema-connect({
... ,
prefetch = [qw( Interface_Type )],
});
You are looking at the wrong layer.
1) At connection time tell the storage you *may* be using Cursor::Cached at
places: [1]
2) Add the appropriate cache_for attribute to the relationships you want to
add it to:
Interface-belongs_to('rel_interface_type', ::Interface_Type,
'fk_interface_type', {
cache_object = do {
# an instance of say Cache::FileCache, something that
# understands get()[2] set()[3] and remove()[4]
# sorry about the links to source, mst sucks at POD,
# docpatches more than welcome
},
cache_for = do { # sub-argument to set as in [3] }
);
3) Optionally register extra relationships )extra names) which never use
the cache
This way it is just granular enough to be useful, and won't require you
to change any of the actual app logic.
Cheers
[1]
https://metacpan.org/source/ARCANEZ/DBIx-Class-Cursor-Cached-1.001002/lib/DBIx/Class/Cursor/Cached.pm#L101
[2]
https://metacpan.org/source/ARCANEZ/DBIx-Class-Cursor-Cached-1.001002/lib/DBIx/Class/Cursor/Cached.pm#L77
[3]
https://metacpan.org/source/ARCANEZ/DBIx-Class-Cursor-Cached-1.001002/lib/DBIx/Class/Cursor/Cached.pm#L79
[4]
https://metacpan.org/source/ARCANEZ/DBIx-Class-Cursor-Cached-1.001002/lib/DBIx/Class/Cursor/Cached.pm#L86
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On Wed, Jan 16, 2013 at 02:51:59AM +1100, Peter Rabbitson wrote: On Tue, Jan 15, 2013 at 10:09:19AM -0500, Jesse Sheidlower wrote: On Tue, Jan 15, 2013 at 05:18:06AM +1100, Peter Rabbitson wrote: On Mon, Jan 14, 2013 at 11:07:34AM -0500, Jesse Sheidlower wrote: So apart from the mystery around the nonsensical exception - does this help? local $DBIx::Class::ResultSourceHandle::thaw_schema = $c-model(...)-schema; Yes! Putting that before the unless seems to make everything work correctly Will be testing further this afternoon. Thanks! Please let me know if wider testing is succesful. In one sense, wider testing was indeed successful. The RS was cached properly, it worked in several different environments, etc. However, it didn't work in one particularly crucial way, which in retrospect I'm surprised I didn't think about: the resultset may have been cached, but once I get it back, it's still just a resultset, so when I use it again in the template, it hits the database again. Every time. This is exactly what I wanted to avoid by this whole escapade. You need to combine the above with [1] and/or [2]. [1] https://metacpan.org/module/DBIx::Class::ResultSet#cache [2] https://metacpan.org/module/DBIx::Class::ResultSet#set_cache Did that help? ___ List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class IRC: irc.perl.org#dbix-class SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/ Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On 2013-01-16 13:20, Peter Rabbitson wrote:
On Wed, Jan 16, 2013 at 11:03:43AM +0100, Alexander Hartmaier wrote:
On 2013-01-15 21:49, Peter Rabbitson wrote:
On Tue, Jan 15, 2013 at 05:12:12PM +0100, Alexander Hartmaier wrote:
On 2013-01-15 16:51, Peter Rabbitson wrote:
On Tue, Jan 15, 2013 at 10:09:19AM -0500, Jesse Sheidlower wrote:
On Tue, Jan 15, 2013 at 05:18:06AM +1100, Peter Rabbitson wrote:
On Mon, Jan 14, 2013 at 11:07:34AM -0500, Jesse Sheidlower wrote:
So apart from the mystery around the nonsensical exception - does
this help?
local $DBIx::Class::ResultSourceHandle::thaw_schema =
$c-model(...)-schema;
Yes! Putting that before the unless seems to make everything work
correctly Will be testing further this afternoon. Thanks!
Please let me know if wider testing is succesful.
In one sense, wider testing was indeed successful. The RS was cached
properly, it worked in several different environments, etc.
However, it didn't work in one particularly crucial way, which in
retrospect I'm surprised I didn't think about: the resultset may have
been cached, but once I get it back, it's still just a resultset, so
when I use it again in the template, it hits the database again. Every
time.
This is exactly what I wanted to avoid by this whole escapade.
You need to combine the above with [1] and/or [2].
[1] https://metacpan.org/module/DBIx::Class::ResultSet#cache
[2] https://metacpan.org/module/DBIx::Class::ResultSet#set_cache
Cheers
You mean don't prefetch the relationships and populate it's cache after
fetching all results?
Um... prefetch does not touch the cache of the *main* rs. I am not sure
what you mean above - explain.
My schema has quite some 'side tables' like types, stati and so on. They
almost never change.
It would help performance if I could tell the schema to load all rows,
cache them and populate a relationship from the cache when prefetched.
For example
Interface-belongs_to('rel_interface_type', ::Interface_Type,
'fk_interface_type');
Schema-connect({
... ,
prefetch = [qw( Interface_Type )],
});
You are looking at the wrong layer.
1) At connection time tell the storage you *may* be using Cursor::Cached at
places: [1]
2) Add the appropriate cache_for attribute to the relationships you want to
add it to:
Interface-belongs_to('rel_interface_type', ::Interface_Type,
'fk_interface_type', {
cache_object = do {
# an instance of say Cache::FileCache, something that
# understands get()[2] set()[3] and remove()[4]
# sorry about the links to source, mst sucks at POD,
# docpatches more than welcome
},
cache_for = do { # sub-argument to set as in [3] }
);
3) Optionally register extra relationships )extra names) which never use
the cache
This way it is just granular enough to be useful, and won't require you
to change any of the actual app logic.
Cheers
[1]
https://metacpan.org/source/ARCANEZ/DBIx-Class-Cursor-Cached-1.001002/lib/DBIx/Class/Cursor/Cached.pm#L101
[2]
https://metacpan.org/source/ARCANEZ/DBIx-Class-Cursor-Cached-1.001002/lib/DBIx/Class/Cursor/Cached.pm#L77
[3]
https://metacpan.org/source/ARCANEZ/DBIx-Class-Cursor-Cached-1.001002/lib/DBIx/Class/Cursor/Cached.pm#L79
[4]
https://metacpan.org/source/ARCANEZ/DBIx-Class-Cursor-Cached-1.001002/lib/DBIx/Class/Cursor/Cached.pm#L86
I had NO idea that's already possible, thanks for the great pointers! Do
we want to include that in the Cookbook?
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
***
T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
Handelsgericht Wien, FN 79340b
***
Notice: This e-mail contains information that is confidential and may be
privileged.
If you are not the intended recipient, please notify the sender and then
delete this e-mail immediately.
***
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On Wed, Jan 16, 2013 at 02:47:19PM +0100, Alexander Hartmaier wrote: I had NO idea that's already possible, thanks for the great pointers! Do we want to include that in the Cookbook? There is no such thing as too much documentation. Please go for it. Also see if you can do something about [1] while you are at it would be great. Cheers [1] http://lists.scsys.co.uk/pipermail/dbix-class/2013-January/011040.html ___ List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class IRC: irc.perl.org#dbix-class SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/ Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On Tue, Jan 15, 2013 at 05:18:06AM +1100, Peter Rabbitson wrote: On Mon, Jan 14, 2013 at 11:07:34AM -0500, Jesse Sheidlower wrote: So apart from the mystery around the nonsensical exception - does this help? local $DBIx::Class::ResultSourceHandle::thaw_schema = $c-model(...)-schema; Yes! Putting that before the unless seems to make everything work correctly Will be testing further this afternoon. Thanks! Please let me know if wider testing is succesful. In one sense, wider testing was indeed successful. The RS was cached properly, it worked in several different environments, etc. However, it didn't work in one particularly crucial way, which in retrospect I'm surprised I didn't think about: the resultset may have been cached, but once I get it back, it's still just a resultset, so when I use it again in the template, it hits the database again. Every time. This is exactly what I wanted to avoid by this whole escapade. ___ List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class IRC: irc.perl.org#dbix-class SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/ Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On Tue, Jan 15, 2013 at 02:15:02PM +, David Cantrell wrote: On Sun, Jan 13, 2013 at 11:35:06PM -0500, Jesse Sheidlower wrote: I have a Catalyst app that very frequently (pretty much every request) requires several DBIC resultsets that return a small number of values, that very rarely change. I'm trying to cache this, so I can update the cache when the values change and not have to hit my DB a half-dozen times on every request for data that is effectively static. To solve this problem at the BBC in the guts of iPlayer, we used memcache. We *didn't* bother updating the cache if the DB contents changed though, as we didn't mind the user's view being a few minutes out of date. While it wouldn't matter much if things were a little out of date, my site is relatively low traffic, so caching something for a few minutes wouldn't make much difference. So I need to cache it long-term, and I do need to update on a change. I've spent quite a bit of time thinking about how to solve this in the general case, and not come up with anything that works yet - not even anything that works in theory and goes arse-over-tit in practice. Can you at least reduce it to one query with a left right in out shake it all about join? Not really. The problem is that I have a number of separate, unrelated tables, each having category-like data (category, status, region, other-status, etc.), and each being used in different ways, so I have to get each table separately. ___ List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class IRC: irc.perl.org#dbix-class SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/ Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On Tue, Jan 15, 2013 at 10:09:19AM -0500, Jesse Sheidlower wrote: On Tue, Jan 15, 2013 at 05:18:06AM +1100, Peter Rabbitson wrote: On Mon, Jan 14, 2013 at 11:07:34AM -0500, Jesse Sheidlower wrote: So apart from the mystery around the nonsensical exception - does this help? local $DBIx::Class::ResultSourceHandle::thaw_schema = $c-model(...)-schema; Yes! Putting that before the unless seems to make everything work correctly Will be testing further this afternoon. Thanks! Please let me know if wider testing is succesful. In one sense, wider testing was indeed successful. The RS was cached properly, it worked in several different environments, etc. However, it didn't work in one particularly crucial way, which in retrospect I'm surprised I didn't think about: the resultset may have been cached, but once I get it back, it's still just a resultset, so when I use it again in the template, it hits the database again. Every time. This is exactly what I wanted to avoid by this whole escapade. You need to combine the above with [1] and/or [2]. [1] https://metacpan.org/module/DBIx::Class::ResultSet#cache [2] https://metacpan.org/module/DBIx::Class::ResultSet#set_cache Cheers ___ List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class IRC: irc.perl.org#dbix-class SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/ Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On Tue, Jan 15, 2013 at 05:12:12PM +0100, Alexander Hartmaier wrote: On 2013-01-15 16:51, Peter Rabbitson wrote: On Tue, Jan 15, 2013 at 10:09:19AM -0500, Jesse Sheidlower wrote: On Tue, Jan 15, 2013 at 05:18:06AM +1100, Peter Rabbitson wrote: On Mon, Jan 14, 2013 at 11:07:34AM -0500, Jesse Sheidlower wrote: So apart from the mystery around the nonsensical exception - does this help? local $DBIx::Class::ResultSourceHandle::thaw_schema = $c-model(...)-schema; Yes! Putting that before the unless seems to make everything work correctly Will be testing further this afternoon. Thanks! Please let me know if wider testing is succesful. In one sense, wider testing was indeed successful. The RS was cached properly, it worked in several different environments, etc. However, it didn't work in one particularly crucial way, which in retrospect I'm surprised I didn't think about: the resultset may have been cached, but once I get it back, it's still just a resultset, so when I use it again in the template, it hits the database again. Every time. This is exactly what I wanted to avoid by this whole escapade. You need to combine the above with [1] and/or [2]. [1] https://metacpan.org/module/DBIx::Class::ResultSet#cache [2] https://metacpan.org/module/DBIx::Class::ResultSet#set_cache Cheers You mean don't prefetch the relationships and populate it's cache after fetching all results? Um... prefetch does not touch the cache of the *main* rs. I am not sure what you mean above - explain. Cheers ___ List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class IRC: irc.perl.org#dbix-class SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/ Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On Sun, Jan 13, 2013 at 11:35:06PM -0500, Jesse Sheidlower wrote:
I have a Catalyst app that very frequently (pretty much every request)
requires several DBIC resultsets that return a small number of values,
that very rarely change. I'm trying to cache this, so I can update the
cache when the values change and not have to hit my DB a half-dozen
times on every request for data that is effectively static.
Originally the relevant line was along the lines of:
$c-stash-{subjects} = $c-model('WordsDB::Subject')-search();
I replaced this, following the C::P::Cache docs, with
unless ( $c-stash-{subjects} = $cache-get( 'subjects' ) ) {
$c-stash-{subjects} =
$c-model('WordsDB::Subject')-search();
$cache-set( 'subject', $c-stash-{subjects} );
}
However, this dies (on a second run, when it's actually hitting the
cache) with undef error - Can't call method select on an undefined
value at /usr/share/perl5/DBIx/Class/ResultSet.pm line 957.
This is a crappy error message. Please tell us which DBIC version are
you running exactly so that we can figure out what is on this line.
So I'm assuming that I can't just stuff a RS into the cache and expect
it to work.
Things should just work, at least for Storable-based serialization. Need
more details to diagnose this.
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On Mon, Jan 14, 2013 at 09:26:04AM +0100, Alexander Hartmaier wrote: The resultset has a ref to the schema which in turn has one to its database connection which can't be serialized/cached. After unfreezing it you have to link the $rs back to a schema with a working database connection. I wasn't able to find this in the docs although I'm pretty sure I've read it sometimes. It is at a crappy place in the docs[1][2], but the error message that was supposed to print is pretty comprehensive[3]. Obviously improving the docs is never a bad idea. [1] https://metacpan.org/module/DBIx::Class::ResultSourceHandle#STORABLE_thaw [2] https://metacpan.org/module/DBIx::Class::Schema#thaw [3] https://metacpan.org/source/GETTY/DBIx-Class-0.08204/lib/DBIx/Class/ResultSource.pm#L1132 Cheers ___ List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class IRC: irc.perl.org#dbix-class SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/ Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On Mon, Jan 14, 2013 at 10:13:44PM +1100, Peter Rabbitson wrote:
On Sun, Jan 13, 2013 at 11:35:06PM -0500, Jesse Sheidlower wrote:
I have a Catalyst app that very frequently (pretty much every request)
requires several DBIC resultsets that return a small number of values,
that very rarely change. I'm trying to cache this, so I can update the
cache when the values change and not have to hit my DB a half-dozen
times on every request for data that is effectively static.
Originally the relevant line was along the lines of:
$c-stash-{subjects} = $c-model('WordsDB::Subject')-search();
I replaced this, following the C::P::Cache docs, with
unless ( $c-stash-{subjects} = $cache-get( 'subjects' ) ) {
$c-stash-{subjects} =
$c-model('WordsDB::Subject')-search();
$cache-set( 'subject', $c-stash-{subjects} );
}
However, this dies (on a second run, when it's actually hitting the
cache) with undef error - Can't call method select on an undefined
value at /usr/share/perl5/DBIx/Class/ResultSet.pm line 957.
This is a crappy error message. Please tell us which DBIC version are
you running exactly so that we can figure out what is on this line.
It's version 0.08196. This is in the cursor() method.
So I'm assuming that I can't just stuff a RS into the cache and expect
it to work.
Things should just work, at least for Storable-based serialization. Need
more details to diagnose this.
What would you like to know?
Jesse Sheidlower
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On Mon, Jan 14, 2013 at 06:30:42AM -0500, Jesse Sheidlower wrote:
On Mon, Jan 14, 2013 at 10:13:44PM +1100, Peter Rabbitson wrote:
On Sun, Jan 13, 2013 at 11:35:06PM -0500, Jesse Sheidlower wrote:
I have a Catalyst app that very frequently (pretty much every request)
requires several DBIC resultsets that return a small number of values,
that very rarely change. I'm trying to cache this, so I can update the
cache when the values change and not have to hit my DB a half-dozen
times on every request for data that is effectively static.
Originally the relevant line was along the lines of:
$c-stash-{subjects} = $c-model('WordsDB::Subject')-search();
I replaced this, following the C::P::Cache docs, with
unless ( $c-stash-{subjects} = $cache-get( 'subjects' ) ) {
$c-stash-{subjects} =
$c-model('WordsDB::Subject')-search();
$cache-set( 'subject', $c-stash-{subjects} );
}
What is $cache being serialized via? Storable? Or some other scheme?
However, this dies (on a second run, when it's actually hitting the
cache) with undef error - Can't call method select on an undefined
value at /usr/share/perl5/DBIx/Class/ResultSet.pm line 957.
What is the exact code from get()ing the cache value to the error (or
what is the smallest piece of code you can reduce it to)?
Cheers
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On Mon, Jan 14, 2013 at 11:06:56PM +1100, Peter Rabbitson wrote:
On Mon, Jan 14, 2013 at 06:30:42AM -0500, Jesse Sheidlower wrote:
On Mon, Jan 14, 2013 at 10:13:44PM +1100, Peter Rabbitson wrote:
On Sun, Jan 13, 2013 at 11:35:06PM -0500, Jesse Sheidlower wrote:
I have a Catalyst app that very frequently (pretty much every request)
requires several DBIC resultsets that return a small number of values,
that very rarely change. I'm trying to cache this, so I can update the
cache when the values change and not have to hit my DB a half-dozen
times on every request for data that is effectively static.
Originally the relevant line was along the lines of:
$c-stash-{subjects} = $c-model('WordsDB::Subject')-search();
I replaced this, following the C::P::Cache docs, with
unless ( $c-stash-{subjects} = $cache-get( 'subjects' ) ) {
$c-stash-{subjects} =
$c-model('WordsDB::Subject')-search();
$cache-set( 'subject', $c-stash-{subjects} );
}
What is $cache being serialized via? Storable? Or some other scheme?
I'm using Cache::FastMmap, which uses Storable (unless you flip a switch
to have it store values as raw binary data; I didn't flip this switch).
However, this dies (on a second run, when it's actually hitting the
cache) with undef error - Can't call method select on an undefined
value at /usr/share/perl5/DBIx/Class/ResultSet.pm line 957.
What is the exact code from get()ing the cache value to the error (or
what is the smallest piece of code you can reduce it to)?
It's in a Catalyst context, so there's a ton of stuff in between, but
the core is that I put the RS into the Catalyst stash as indicated
above, and then in a template I have:
[% WHILE ( subject = subjects.next ) %]
option value=[% subject.id %][% subject %]/option
[% END %]
Thanks.
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On Sun, Jan 13, 2013 at 11:35:06PM -0500, Jesse Sheidlower wrote:
I have a Catalyst app that very frequently (pretty much every request)
requires several DBIC resultsets that return a small number of values,
that very rarely change. I'm trying to cache this, so I can update the
cache when the values change and not have to hit my DB a half-dozen
times on every request for data that is effectively static.
Originally the relevant line was along the lines of:
$c-stash-{subjects} = $c-model('WordsDB::Subject')-search();
I replaced this, following the C::P::Cache docs, with
unless ( $c-stash-{subjects} = $cache-get( 'subjects' ) ) {
$c-stash-{subjects} =
$c-model('WordsDB::Subject')-search();
$cache-set( 'subject', $c-stash-{subjects} );
}
However, this dies (on a second run, when it's actually hitting the
cache) with undef error - Can't call method select on an undefined
value at /usr/share/perl5/DBIx/Class/ResultSet.pm line 957. So I'm
assuming that I can't just stuff a RS into the cache and expect it to
work. Is there an easy way around it? (I know I could retrieve the
actual data and put that into the cache, but then I'd have to rewrite a
whole bunch of templates, that are expecting a resultset.)
Thanks.
So apart from the mystery around the nonsensical exception - does this help?
local $DBIx::Class::ResultSourceHandle::thaw_schema = $c-model(...)-schema;
Cheers
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
One problem I see with the exact code posted here is a typo referring to
singular-vs-plural on the hash key:
On Sun, Jan 13, 2013 at 10:35 PM, Jesse Sheidlower jes...@panix.com wrote:
unless ( $c-stash-{subjects} = $cache-get( 'subjects' ) ) {
$c-stash-{subjects} =
$c-model('WordsDB::Subject')-search();
$cache-set( 'subject', $c-stash-{subjects} );
}
Note 'subject' vs 'subjects':
$cache-set( 'subject', $c-stash-{subjects} );
--
Will Trillich :: 812.454.6431
“Grading takes away all the fun from failing. And a huge part of education
is about failure.” -- Shimon Schocken
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On Mon, Jan 14, 2013 at 09:41:45AM -0600, will trillich wrote:
One problem I see with the exact code posted here is a typo referring to
singular-vs-plural on the hash key:
On Sun, Jan 13, 2013 at 10:35 PM, Jesse Sheidlower jes...@panix.com wrote:
unless ( $c-stash-{subjects} = $cache-get( 'subjects' ) ) {
$c-stash-{subjects} =
$c-model('WordsDB::Subject')-search();
$cache-set( 'subject', $c-stash-{subjects} );
}
Note 'subject' vs 'subjects':
$cache-set( 'subject', $c-stash-{subjects} );
That was my typo re-keying the actual code for the mailing list, the
original (which presents it differently, in a way not relevant for this
purpose) is correct. Sorry for the misleading example.
Jesse
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On Tue, Jan 15, 2013 at 01:30:55AM +1100, Peter Rabbitson wrote:
On Sun, Jan 13, 2013 at 11:35:06PM -0500, Jesse Sheidlower wrote:
I have a Catalyst app that very frequently (pretty much every request)
requires several DBIC resultsets that return a small number of values,
that very rarely change. I'm trying to cache this, so I can update the
cache when the values change and not have to hit my DB a half-dozen
times on every request for data that is effectively static.
Originally the relevant line was along the lines of:
$c-stash-{subjects} = $c-model('WordsDB::Subject')-search();
I replaced this, following the C::P::Cache docs, with
unless ( $c-stash-{subjects} = $cache-get( 'subjects' ) ) {
$c-stash-{subjects} =
$c-model('WordsDB::Subject')-search();
$cache-set( 'subject', $c-stash-{subjects} );
}
However, this dies (on a second run, when it's actually hitting the
cache) with undef error - Can't call method select on an undefined
value at /usr/share/perl5/DBIx/Class/ResultSet.pm line 957. So I'm
assuming that I can't just stuff a RS into the cache and expect it to
work. Is there an easy way around it? (I know I could retrieve the
actual data and put that into the cache, but then I'd have to rewrite a
whole bunch of templates, that are expecting a resultset.)
Thanks.
So apart from the mystery around the nonsensical exception - does this help?
local $DBIx::Class::ResultSourceHandle::thaw_schema = $c-model(...)-schema;
Yes! Putting that before the unless seems to make everything work
correctly Will be testing further this afternoon. Thanks!
Does this need to be documented somewhere, or does something need to be
fixed, or...?
Jesse
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
On Mon, Jan 14, 2013 at 11:07:34AM -0500, Jesse Sheidlower wrote:
On Tue, Jan 15, 2013 at 01:30:55AM +1100, Peter Rabbitson wrote:
On Sun, Jan 13, 2013 at 11:35:06PM -0500, Jesse Sheidlower wrote:
I have a Catalyst app that very frequently (pretty much every request)
requires several DBIC resultsets that return a small number of values,
that very rarely change. I'm trying to cache this, so I can update the
cache when the values change and not have to hit my DB a half-dozen
times on every request for data that is effectively static.
Originally the relevant line was along the lines of:
$c-stash-{subjects} = $c-model('WordsDB::Subject')-search();
I replaced this, following the C::P::Cache docs, with
unless ( $c-stash-{subjects} = $cache-get( 'subjects' ) ) {
$c-stash-{subjects} =
$c-model('WordsDB::Subject')-search();
$cache-set( 'subject', $c-stash-{subjects} );
}
However, this dies (on a second run, when it's actually hitting the
cache) with undef error - Can't call method select on an undefined
value at /usr/share/perl5/DBIx/Class/ResultSet.pm line 957. So I'm
assuming that I can't just stuff a RS into the cache and expect it to
work. Is there an easy way around it? (I know I could retrieve the
actual data and put that into the cache, but then I'd have to rewrite a
whole bunch of templates, that are expecting a resultset.)
Thanks.
So apart from the mystery around the nonsensical exception - does this help?
local $DBIx::Class::ResultSourceHandle::thaw_schema =
$c-model(...)-schema;
Yes! Putting that before the unless seems to make everything work
correctly Will be testing further this afternoon. Thanks!
Please let me know if wider testing is succesful.
Does this need to be documented somewhere, or does something need to be
fixed, or...?
Please see [1]. There are really two problems:
1) the exception thrown was supposed to be [2]. This part is a bug, and
I can't yet figure out what is causing it. The test at [3] is nearly
identical, and behaves correctly. Any thoughts on why yours failed
(perhaps a debugger walk) would be more than welcome.
2) The docs suck (again as per [1]) and would welcome imrpovements. Also
adding stuff to C::P::Cache won't hurt either.
Cheers
[1] http://lists.scsys.co.uk/pipermail/dbix-class/2013-January/011040.html
[2]
https://metacpan.org/source/GETTY/DBIx-Class-0.08204/lib/DBIx/Class/ResultSource.pm#L1132
[3] https://metacpan.org/source/GETTY/DBIx-Class-0.08204/t/84serialize.t#L199
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
[Dbix-class] Caching a resultset?
I have a Catalyst app that very frequently (pretty much every request)
requires several DBIC resultsets that return a small number of values,
that very rarely change. I'm trying to cache this, so I can update the
cache when the values change and not have to hit my DB a half-dozen
times on every request for data that is effectively static.
Originally the relevant line was along the lines of:
$c-stash-{subjects} = $c-model('WordsDB::Subject')-search();
I replaced this, following the C::P::Cache docs, with
unless ( $c-stash-{subjects} = $cache-get( 'subjects' ) ) {
$c-stash-{subjects} =
$c-model('WordsDB::Subject')-search();
$cache-set( 'subject', $c-stash-{subjects} );
}
However, this dies (on a second run, when it's actually hitting the
cache) with undef error - Can't call method select on an undefined
value at /usr/share/perl5/DBIx/Class/ResultSet.pm line 957. So I'm
assuming that I can't just stuff a RS into the cache and expect it to
work. Is there an easy way around it? (I know I could retrieve the
actual data and put that into the cache, but then I'd have to rewrite a
whole bunch of templates, that are expecting a resultset.)
Thanks.
Jesse Sheidlower
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
Re: [Dbix-class] Caching a resultset?
If you are using MySQL, I think query cache of MySQL works for you.
Hitting query cache is supposed fast enough for general purpose.
Is it really necessary to try to cache a RS, which is not a easy job,
I'm afraid.
On Mon, Jan 14, 2013 at 1:35 PM, Jesse Sheidlower jes...@panix.com wrote:
I have a Catalyst app that very frequently (pretty much every request)
requires several DBIC resultsets that return a small number of values,
that very rarely change. I'm trying to cache this, so I can update the
cache when the values change and not have to hit my DB a half-dozen
times on every request for data that is effectively static.
Originally the relevant line was along the lines of:
$c-stash-{subjects} = $c-model('WordsDB::Subject')-search();
I replaced this, following the C::P::Cache docs, with
unless ( $c-stash-{subjects} = $cache-get( 'subjects' ) ) {
$c-stash-{subjects} =
$c-model('WordsDB::Subject')-search();
$cache-set( 'subject', $c-stash-{subjects} );
}
However, this dies (on a second run, when it's actually hitting the
cache) with undef error - Can't call method select on an undefined
value at /usr/share/perl5/DBIx/Class/ResultSet.pm line 957. So I'm
assuming that I can't just stuff a RS into the cache and expect it to
work. Is there an easy way around it? (I know I could retrieve the
actual data and put that into the cache, but then I'd have to rewrite a
whole bunch of templates, that are expecting a resultset.)
Thanks.
Jesse Sheidlower
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk
