Re: DateTime.pm on a Diet
On Thu, Jul 21, 2005 at 01:54:30PM +1000, Rick Measham wrote: > I want to wrap this up and release so there's 24 hours to finalise the > name. Here's the names I like thus far: > > DateTime::LazyInit (from John Siracusa) > DateTime::Mock (from Joshua Hoblitt) > DateTime::Diet (original development name based on the subject) > > If anyone has any interest, please comment in the next 24 hours (based > on the timestamp of this message) I'm not trying to beat this thread to death but it just occurred to me that this is following the proxy design pattern. So DateTime::Proxy might be a good name as well. Of the three name in your list I think I like DateTime::LazyInit more than the others. Cheers, -J -- pgpRyOZFJZvo4.pgp Description: PGP signature
Re: DateTime.pm on a Diet
DateTime::LazyInit (from John Siracusa) has received the most votes by far, so I'll go with it. As one respondant put it, it has 'plain spokenness' DateTime::Diet, the other name to recieve some votes, made me think of 'DateTime::Lite', and it isn't -- it's full-blown DateTime, just not immediately. I'll release sometime over the weekend, once I get tests and docs writtem. Cheers! Rick Measham
Re: DateTime.pm on a Diet
On 21 Jul 2005, at 15:51, [EMAIL PROTECTED] wrote: Quoting Rick Measham <[EMAIL PROTECTED]>: Joshua Hoblitt wrote: What about DateTime::Mock? Since that would make it clear that this isn't /really/ a DT object. Thanks Joshua, I want to wrap this up and release so there's 24 hours to finalise the name. Here's the names I like thus far: DateTime::LazyInit (from John Siracusa) DateTime::Mock (from Joshua Hoblitt) DateTime::Diet (original development name based on the subject) +1 for LazyInit for plain-spokenness -1 on the Mock, since that implies it is for testing and needs you to stub out the code. [snip] Ditto. Adrian
Re: DateTime.pm on a Diet
Quoting Rick Measham <[EMAIL PROTECTED]>: Joshua Hoblitt wrote: What about DateTime::Mock? Since that would make it clear that this isn't /really/ a DT object. Thanks Joshua, I want to wrap this up and release so there's 24 hours to finalise the name. Here's the names I like thus far: DateTime::LazyInit (from John Siracusa) DateTime::Mock (from Joshua Hoblitt) DateTime::Diet (original development name based on the subject) +1 for LazyInit for plain-spokenness -1 on the Mock, since that implies it is for testing and needs you to stub out the code. +1 for Diet (it has more cheek)
Re: DateTime.pm on a Diet
What about DateTime::Mock? Since that would make it clear that this isn't /really/ a DT object. -J -- On Thu, Jul 07, 2005 at 02:12:41PM +1000, Rick Measham wrote: > Dave Rolsky wrote: > > DateTime::Shim > > DateTime::Trampoline > > DateTime::ThinShim > > DateTime::Proxy > > DateTime::Diet - not awful but a little cute, methinks ;) > > Of the above, I still like DT:Diet. > > If you want something less cute/more serious how about: > DateTime::FastConstructor > > The others listed above don't seems to convey the purpose of the module. > > Cheers! > Rick Measham pgp8TUzd5fLrA.pgp Description: PGP signature
Re: DateTime.pm on a Diet
On Thu, 21 Jul 2005, Rick Measham wrote: Joshua Hoblitt wrote: What about DateTime::Mock? Since that would make it clear that this isn't /really/ a DT object. Thanks Joshua, I want to wrap this up and release so there's 24 hours to finalise the name. Here's the names I like thus far: DateTime::LazyInit (from John Siracusa) I think I like this one best. -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
Re: DateTime.pm on a Diet
Joshua Hoblitt wrote: What about DateTime::Mock? Since that would make it clear that this isn't /really/ a DT object. Thanks Joshua, I want to wrap this up and release so there's 24 hours to finalise the name. Here's the names I like thus far: DateTime::LazyInit (from John Siracusa) DateTime::Mock (from Joshua Hoblitt) DateTime::Diet (original development name based on the subject) If anyone has any interest, please comment in the next 24 hours (based on the timestamp of this message) Cheers! Rick Measham
Re: DateTime.pm on a Diet
On 7/7/05 12:12 AM, Rick Measham wrote: > Dave Rolsky wrote: >> DateTime::Shim >> DateTime::Trampoline >> DateTime::ThinShim >> DateTime::Proxy >> DateTime::Diet - not awful but a little cute, methinks ;) > > Of the above, I still like DT:Diet. > > If you want something less cute/more serious how about: > DateTime::FastConstructor > > The others listed above don't seems to convey the purpose of the module. I'll throw a few names out: DateTime::Delayed DateTime::AutoLoaded DateTime::LazyInit -John
Re: DateTime.pm on a Diet
On Thu, 7 Jul 2005, Rick Measham wrote: Dave Rolsky wrote: It has a pretty different API, in that it's new() constructor accepts anything without validation. I suppose it could check for extra args and call DateTime::Fat->new() if needed. I think that'd be a possibility, but it'd have to pass the DateTime.pm test suite in that case. We'd need to bring complete validation back in though. Otherwise your object dies unexpectedly later on: $dt = DateTime::Diet->new( year => 2004, month => 232 ); print $dt->year; #2004 print $dt->month; #232 print $dt->monthname; # DIE: Illegal value passed to DateTime::new Yeah, this occurred to me after I wrote that. Having these sorts of "delayed" errors is very bad and surprising for users. Rick's code is well suited to instances where you have data that you've already checked, but not so good for general purpose usage. -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
Re: DateTime.pm on a Diet
Dave Rolsky wrote: It has a pretty different API, in that it's new() constructor accepts anything without validation. I suppose it could check for extra args and call DateTime::Fat->new() if needed. Can we split the existing new() into validation vs. object creation and share it between the two equally (no, I haven't looked at the source)? Thus, the lightweight new() could validate by default, but offer a quick no_validate => 1 option for the people creating objects out of databases (prevalidated). It may be once that you are slinging lightweight objects around most of the time, the performance drain due to validation won't be quite as noticible. I think that'd be a possibility, but it'd have to pass the DateTime.pm test suite in that case. That's certainly where I was considering starting (with the test suite). It might also be a good experiment to see where the performance issues really are. John -- John Peacock Director of Information Research and Technology Rowman & Littlefield Publishing Group 4720 Boston Way Lanham, MD 20706 301-459-3366 x.5010 fax 301-429-5747
Re: DateTime.pm on a Diet
Dave Rolsky wrote: DateTime::Shim DateTime::Trampoline DateTime::ThinShim DateTime::Proxy DateTime::Diet - not awful but a little cute, methinks ;) Of the above, I still like DT:Diet. If you want something less cute/more serious how about: DateTime::FastConstructor The others listed above don't seems to convey the purpose of the module. Cheers! Rick Measham
Re: DateTime.pm on a Diet
On Wed, 6 Jul 2005, John Peacock wrote: At the risk of sounding flippant, why not rename the existing DateTime class to DateTime::Fat (or the less inflamatory DateTime::Base) and put the proposed module in it's place as DateTime. If it is designed well, everything should Just Work(TM) with the shim class, which should always be faster than the existing class, even with the extra dispatch to plump up the object when needed. At first I thought this might be a good idea .. Dave Rolsky wrote: It has a pretty different API, in that it's new() constructor accepts anything without validation. I suppose it could check for extra args and call DateTime::Fat->new() if needed. I think that'd be a possibility, but it'd have to pass the DateTime.pm test suite in that case. We'd need to bring complete validation back in though. Otherwise your object dies unexpectedly later on: $dt = DateTime::Diet->new( year => 2004, month => 232 ); print $dt->year; #2004 print $dt->month; #232 print $dt->monthname; # DIE: Illegal value passed to DateTime::new This needs to be an *alternate* rather than standard way of creating DateTime objects, and the documentation will basically just tell you that you need to be very aware of the lack of immediate validation. Cheers! Rick Measham
Re: DateTime.pm on a Diet
On Wed, 6 Jul 2005, John Peacock wrote: At the risk of sounding flippant, why not rename the existing DateTime class to DateTime::Fat (or the less inflamatory DateTime::Base) and put the proposed module in it's place as DateTime. If it is designed well, everything should Just Work(TM) with the shim class, which should always be faster than the existing class, even with the extra dispatch to plump up the object when needed. It has a pretty different API, in that it's new() constructor accepts anything without validation. I suppose it could check for extra args and call DateTime::Fat->new() if needed. I think that'd be a possibility, but it'd have to pass the DateTime.pm test suite in that case. -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
Re: DateTime.pm on a Diet
Dave Rolsky wrote: It sounded like people were interested. And maybe it's a "if you build it they will come" thing ;) Anyway, go for it and let's brainstorm on a better namespace. Here's some thoughts: At the risk of sounding flippant, why not rename the existing DateTime class to DateTime::Fat (or the less inflamatory DateTime::Base) and put the proposed module in it's place as DateTime. If it is designed well, everything should Just Work(TM) with the shim class, which should always be faster than the existing class, even with the extra dispatch to plump up the object when needed. John -- John Peacock Director of Information Research and Technology Rowman & Littlefield Publishing Group 4720 Boston Way Lanham, MD 20706 301-459-3366 x.5010 fax 301-429-5747
Re: DateTime.pm on a Diet
On Thu, 7 Jul 2005, Rick Measham wrote: So .. would this module actually get used by anyone but me? If so I'll go ahead and polish it off. It sounded like people were interested. And maybe it's a "if you build it they will come" thing ;) Anyway, go for it and let's brainstorm on a better namespace. Here's some thoughts: DateTime::Shim DateTime::Trampoline DateTime::ThinShim DateTime::Proxy DateTime::Diet - not awful but a little cute, methinks ;) -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
Re: DateTime.pm on a Diet
On Wed, 6 Jul 2005, John Siracusa wrote:
If you're going for speed, there's more low-hanging fruit to be had.
Check
out my modifications in DateTime::Diet2 (attached).
Dave Rolsky wrote:
Yeah, I was thinking that we'd do something like this if we're going to
turn this into an officially blessed implementation. I was assuming
Rick just used AUTOLOAD for a quick proof of concept.
I was .. however, as Dave later suggests, I'd planned on actually
creating the subs as perl subs .. not even having the overhead of the
autocreation in BEGIN.
..
sub minute { return $_[0]->{minute} || 0 }
..
So .. would this module actually get used by anyone but me? If so I'll
go ahead and polish it off.
Cheers!
Rick Measham
Re: DateTime.pm on a Diet
On Wed, 6 Jul 2005, Eric Cholet wrote: Have you tried turning off the validation by setting the PERL_NO_VALIDATION environment variable? John, thanks for the pointer. I see this is documented in Params::Validate. I'd never thought of looking in Params::Validate docs to find out how to disable DateTime's use of it. It might be useful to include a pointer to this "feature" in DateTime's docs. If people want to make use of this it'd be nice to find a slightly smaller hammer than an env var. Unfortunately Perl doesn't make this easy, since just because you want to turn off validation doesn't mean that CPAN modules you use which also use DateTime want to turn it off. Sigh. -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
Re: DateTime.pm on a Diet
On Wed, 6 Jul 2005, John Siracusa wrote: On 7/5/05, Rick Measham <[EMAIL PROTECTED]> wrote: DateTime::Diet (attached) is a simple wrapper around DateTime that handles simple new(), set() and get methods. If you ask it for something it can't handle by itself, it reblesses your object into full DateTime and then calls the method on the DateTime object. If you're going for speed, there's more low-hanging fruit to be had. Check out my modifications in DateTime::Diet2 (attached). Yeah, I was thinking that we'd do something like this if we're going to turn this into an officially blessed implementation. I was assuming Rick just used AUTOLOAD for a quick proof of concept. I'll bet you could get another few percent by using string eval to generate the accessor methods... Or just hand-coding them. It's not like there's that many. -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
Re: DateTime.pm on a Diet
On 7/5/05, Rick Measham <[EMAIL PROTECTED]> wrote:
> DateTime::Diet (attached) is a simple wrapper around DateTime that
> handles simple new(), set() and get methods. If you ask it for something
> it can't handle by itself, it reblesses your object into full DateTime
> and then calls the method on the DateTime object.
If you're going for speed, there's more low-hanging fruit to be had. Check
out my modifications in DateTime::Diet2 (attached).
---
These tests compare DateTime::Diet[2] to DateTime for object creation, plus
simple accessors and mutators
RateDateTime DateTime::Diet DateTime::Diet2
DateTime 1754/s ---94%-98%
DateTime::Diet 28736/s 1538% ---66%
DateTime::Diet2 84746/s 4731%195% --
These tests compare DateTime::Diet[2] to DateTime for the same things, but
then call a non-diet function causing the object to become a full blown
DateTime object.
RateDateTime DateTime::Diet DateTime::Diet2
DateTime 1629/s ---47%-53%
DateTime::Diet 3049/s 87% ---11%
DateTime::Diet2 3436/s111% 13% --
---
I'll bet you could get another few percent by using string eval to generate
the accessor methods...
-John
package DateTime::Diet;
use strict;
use warnings;
use DateTime;
use vars qw/$AUTOLOAD/;
sub new {
my $class = shift;
my %args = @_;
return bless \%args, $class;
}
sub AUTOLOAD {
my $attr = $AUTOLOAD;
$attr =~ s/.*:://;
return unless $attr =~ /[^A-Z]/; # skip DESTROY and all-cap methods
if ($attr &&
$attr=~/^set_(year|month|hour|minute|second|nanosecond|locale|time_zone)$/) {
$_[0]->{$1} = $_[1];
return $_[0];
}
if ($attr && $attr=~/^(month|day|hour|min(ute)?|sec(ond)?)$/) {
return $_[0]->{$1} if $_[0]->{$1};
return 1 if $1=~/month|day/;
return 0;
}
$_[0] = bless \%{new DateTime(%{$_[0]})}, 'DateTime';
return $_[0]->$attr(@_);
}
package DateTime::Diet2;
use strict;
use warnings;
use DateTime;
our $AUTOLOAD;
sub new
{
my $class = shift;
return bless { @_ }, $class;
}
sub DESTROY { }
BEGIN
{
no strict 'refs';
foreach my $attr (qw(year month hour minute second nanosecond locale
time_zone))
{
my $method = "set_$attr";
*$method = sub { $_[0]->{$attr} = $_[1]; return $_[0] };
}
foreach my $method (qw(month day))
{
*$method = sub
{
return $_[0]->{$method} if($_[0]->{$method});
return 1;
}
}
foreach my $method (qw(hour minute second))
{
*$method = sub
{
return $_[0]->{$method} if($_[0]->{$method});
return 0;
}
}
*min = \&minute;
*sec = \&second;
}
sub AUTOLOAD
{
my $attr = $AUTOLOAD;
$attr =~ s/.*:://;
$_[0] = bless \%{new DateTime(%{$_[0]})}, 'DateTime';
return $_[0]->$attr(@_);
}
package main;
use DateTime;
use Benchmark q(cmpthese);
print "These tests compare DateTime::Diet[2] to DateTime for object creation,
plus simple accessors and mutators\n\n";
cmpthese(5, {
' DateTime' => sub { my $dt = new DateTime( year => 2004 ); my $day =
$dt->day; $dt->set_minute(22); },
' DateTime::Diet' => sub { my $dt = new DateTime::Diet( year => 2004 ); my
$day = $dt->day; $dt->set_minute(22); },
'DateTime::Diet2' => sub { my $dt = new DateTime::Diet2( year => 2004 ); my
$day = $dt->day; $dt->set_minute(22); },
});
print "\n\nThese tests compare DateTime::Diet[2] to DateTime for the same
things, but then call a non-diet function causing the object to become a full
blown DateTime object.\n\n";
cmpthese(1, {
' DateTime' => sub { my $dt = new DateTime( year => 2004 ); my $day =
$dt->day; $dt->set_minute(22); $dt->datetime; $day=$dt->day; },
' DateTime::Diet' => sub { my $dt = new DateTime::Diet( year => 2004 ); my
$day = $dt->day; $dt->set_minute(22); $dt->datetime; $day=$dt->day; },
'DateTime::Diet2' => sub { my $dt = new DateTime::Diet2( year => 2004 ); my
$day = $dt->day; $dt->set_minute(22); $dt->datetime; $day=$dt->day; },
});
Re: DateTime.pm on a Diet
I've used this approach in Date::Set. It runs the same test suite of DateTime::Event::ICal, in one third of the time. - Flavio S. Glock 2005/7/6, Rick Measham <[EMAIL PROTECTED]>: > > DateTime::Diet (attached) is a simple wrapper around DateTime that > handles simple new(), set() and get methods. If you ask it for something > it can't handle by itself, it reblesses your object into full DateTime > and then calls the method on the DateTime object.
Re: DateTime.pm on a Diet
Le 5 juil. 05 à 15:15, John Siracusa a écrit : On 7/5/05 7:04 AM, Eric Cholet wrote: A while ago we did some profiling of an app that uses DateTime extensively and found out that most of the time is spent in Params::Validate. I understand this is somewhat of a religious issue, but in this case it's a net loss for us: the params have been validated zillions of times, and we lose performance in production. I would love a DateTime sans param validation. Have you tried turning off the validation by setting the PERL_NO_VALIDATION environment variable? John, thanks for the pointer. I see this is documented in Params::Validate. I'd never thought of looking in Params::Validate docs to find out how to disable DateTime's use of it. It might be useful to include a pointer to this "feature" in DateTime's docs. -- Eric Cholet
Re: DateTime.pm on a Diet
> Geoffrey, if you're reading this, I'd love your comments on how useful > this would be in your case. I am :) ok, I haven't looked at the code yet, but from your explanations it sounds attractive, especially in our situation. just like you, we don't necessarily need data validation since the data is coming mostly from database date/time fields. also similarly we would be creating objects for sql fields that may be infrequently manipulated in the codebase itself. so, take the overhead out of those kinds of things and you've made half my case for me :) the only sticking point would be the date math "exceptions" dave mentions. in my particular situation we deal with slices of time - the exact time of an event, when that event is visible on the web, what that time looks like to our servers on the west coast versus data entry on the east coast versus an attendee in australia, etc - so it's the date math that is driving force behind using DateTime in the first place. in essence, optimize away, but exploding object size when you need to do something real with it isn't much of a win. anyway, I hope you guys are taking my opinions appropriately - DateTime is clearly the right choice for perl DT logic IMHO for a variety of reasons, but I wasn't able to convince others to adopt it since their criteria for "right choice" was significantly different. so it's their arguments and mindset that I'm trying to convey :) --Geoff
Re: DateTime.pm on a Diet
On 7/5/05 11:09 PM, Rick Measham wrote: > I have written a module I tentatively call DateTime::Diet that helps > with this. I use DateTime in all sort of projects including hooks in > Class::DBI where many many table records all become DateTime objects, > even if I don't need the column that represents a DateTime. After bumping into the cost (CPU and memory) of creating DateTime objects for each date-like column in a database row, my solution was to have my OO/DB mapper module defer the "inflation" process until I actually ask for a DateTime object. I'm not using Class::DBI, but I believe it has a similar ability (although I'm not sure if it can "round-trip" from and back into the database without any inflation). Anyway, just chalk this up to another example of working around the overhead of creating many, many DateTime objects. I looked at speeding up DateTime->new a while ago (over a year I think) and a few things were done (by Dave, I think) to make it a lot faster, but I've stuck with my deferred inflation system because, as fast as new() gets, it's still faster to do nothing at all. -John
Re: DateTime.pm on a Diet
Dave Rolsky wrote: I'm not sure what you mean. It's much quicker for operations that occur before the rebless. If you were to keep using that object for further operations, the speed increase given from calling one "diet" method would fade into background noise. True. I expected a quicker degeneration, but was quite surprised. The point of DateTime::Diet was to be able to quickly create lots of DateTime-able objects. The implementation means that they transparently convert to being full DateTime object whenever you need to do anything with them. But this might be useful for cases where people have to create lots of objects, they don't care about time zones, and they rarely have to use the "advanced" DT.pm methods (math, time zones, etc). Almost agree. It's a subtle difference to me between creating lots of object and doing things with them. When I link to a database, the link turns dates into object. However I don't need to do anything with most of them. For example, I don't need to compare them, as I'd have done that in SQL. From there, I can still access the objects as if they were full blown DateTime objects because they will become such objects when they need to. I suspect people would want to be able to use this in conjunction with a parser like DT::Format::MySQL or something, where they know the data in the DB represents real datetimes. Exactly. My thought is that this sort of module is ideally suited here. We know the data is a real datetime so the validation isn't needed. We're (potentially) creating lots of objects and so the time/memory overhead saving is useful. Geoffrey, if you're reading this, I'd love your comments on how useful this would be in your case. Of course at the moment I'm just handling the simplest of setters and getters, but there's no reason why we couldn't include some other things like strptime, ymd, datetime etc. (Strptime would be interesting as it would need to failover to DateTime on some patterns) The main thing would be not to increase the amount of time it takes to create the object, so there'd be no math (that relies on month lengths, leap years, DST etc). The most data this will hold is that which can be passed to DateTime->new(). Cheers! Rick Measham
Re: DateTime.pm on a Diet
On Wed, 6 Jul 2005, Rick Measham wrote:
I've included the output of the attached script below. I was surprised to
note that even after the rebless was included in the tests, the Diet version
was still *much* quicker.
I'm not sure what you mean. It's much quicker for operations that occur
before the rebless. If you were to keep using that object for further
operations, the speed increase given from calling one "diet" method would
fade into background noise.
For example:
timethese(1,
{
' DateTime' =>
sub { my $dt = new DateTime( year => 2004 );
my $day = $dt->day;
$dt->set_minute(22);
$dt->datetime;
for ( qw( year month hour day minute second nanosecond ) ) {
my $foo = $dt->$_();
}
$dt->set_minute(30);
for ( qw( year month hour day minute second nanosecond ) ) {
my $foo = $dt->$_();
}
},
'DateTime::Diet' =>
sub { my $dt = new DateTime::Diet( year => 2004 );
my $day = $dt->day;
$dt->set_minute(22);
$dt->datetime;
for ( qw( year month hour day minute second nanosecond ) ) {
my $foo = $dt->$_();
}
$dt->set_minute(30);
for ( qw( year month hour day minute second nanosecond ) ) {
my $foo = $dt->$_();
}
},
});
I get these results:
Benchmark: timing 1 iterations of DateTime, DateTime::Diet...
DateTime: 8 wallclock secs ( 8.26 usr + 0.00 sys = 8.26 CPU) @
1210.65/s (n=1)
DateTime::Diet: 7 wallclock secs ( 6.44 usr + 0.00 sys = 6.44 CPU) @
1552.80/s (n=1)
So it still has an impact, but it's starting to become less and less
significant.
But this might be useful for cases where people have to create lots of
objects, they don't care about time zones, and they rarely have to use the
"advanced" DT.pm methods (math, time zones, etc).
I suspect people would want to be able to use this in conjunction with a
parser like DT::Format::MySQL or something, where they know the data in
the DB represents real datetimes.
That's not currently possible, but it could be added to the generic
DT::Format::* API spec.
-dave
/*===
VegGuide.Orgwww.BookIRead.com
Your guide to all that's veg. My book blog
===*/
Re: DateTime.pm on a Diet
Dave Rolsky asked: The first question to answer is what are people doing with these objects? Geoffrey Young wrote: in our case, creating many, many, many of them. more than you can possibly imagine. think one object with a few dozen "time" representations held within it, then a few thousand of those floating around at any given second. all within a mod_perl process where memory, not lookups, was the main (perceived) concern. I have written a module I tentatively call DateTime::Diet that helps with this. I use DateTime in all sort of projects including hooks in Class::DBI where many many table records all become DateTime objects, even if I don't need the column that represents a DateTime. DateTime::Diet (attached) is a simple wrapper around DateTime that handles simple new(), set() and get methods. If you ask it for something it can't handle by itself, it reblesses your object into full DateTime and then calls the method on the DateTime object. It does no parameter checking of its own, as it's designed to work with data that's already been checked in other places. I've not extensively tested it, but want comments before I go any further. I've included the output of the attached script below. I was surprised to note that even after the rebless was included in the tests, the Diet version was still *much* quicker. Cheers! Rick Measham These tests compare DateTime::Diet to DateTime for object creation, plus simple accessors and mutators Benchmark: timing 5 iterations of DateTime, DateTime::Diet... DateTime: 32 wallclock secs (28.33 usr + 0.01 sys = 28.34 CPU) @ 1764.29/s (n=5) DateTime::Diet: 3 wallclock secs ( 1.76 usr + 0.00 sys = 1.76 CPU) @ 28409.09/s (n=5) These tests compare DateTime::Diet to DateTime for the same things, but then call a non-diet function causing the object to become a full blown DateTime object. Benchmark: timing 1 iterations of DateTime, DateTime::Diet... DateTime: 6 wallclock secs ( 5.97 usr + 0.00 sys = 5.97 CPU) @ 1675.04/s (n=1) DateTime::Diet: 4 wallclock secs ( 3.07 usr + 0.00 sys = 3.07 CPU) @ 3257.33/s (n=1) datetime.test.pl Description: Perl program
Re: DateTime.pm on a Diet
On Tue, 5 Jul 2005, Geoffrey Young wrote: again, not arguing over which is better, but having _all_ the zone data cached had the appearance of being wasteful for us - for the most part we would be converting between _maybe_ a dozen time zones tops, more likely around the 6 or so surrounding the US and UK. so being able to cache only the data we know we needed would have gone a long way toward making people _feel_ like DT was lean. This is why I think the biggest win for memory would be having XS based time zones. This would also help speed somewhat, but I think the memory savings could be very, very substantial. If you look at the compiled Olson DB bits used by libc, they're fairly small binary files (<1-2k usually). I think it'd be possible to get the DT::TimeZone stuff down to around this size per zone if we could use C structs and XS, rather than simply using XS to manipulate Perl data structures (which is faster but saves no memory). -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
Re: DateTime.pm on a Diet
just FYI, but I tried to champion DateTime during a massive DT rewrite at work and lost. the biggest gripe was that the objects were "insanely large". large in terms of memory per object and (more important to them, apparently) large enough that it made frequent Data::Dumper dumps difficult to parse during debugging. after tracking down the "insanely large" issue, it seems that initially the objects weren't what _I_ might consider large (maybe half a screen with dumper) but that on any subsequent manipulations things spiraled out of control. IIRC it was all the zone data caching embedded within the object. now, I won't argue one way or the other about which is better (larger objects versus more lookups) especially since I have no benchmarks to back anything up :) but if you're interested in what prevented "us" from using DT across the board that was probably the top of the list. > The first question to answer is what are people doing with these > objects? in our case, creating many, many, many of them. more than you can possibly imagine. think one object with a few dozen "time" representations held within it, then a few thousand of those floating around at any given second. all within a mod_perl process where memory, not lookups, was the main (perceived) concern. again, not arguing over which is better, but having _all_ the zone data cached had the appearance of being wasteful for us - for the most part we would be converting between _maybe_ a dozen time zones tops, more likely around the 6 or so surrounding the US and UK. so being able to cache only the data we know we needed would have gone a long way toward making people _feel_ like DT was lean. just FYI, really - nothing to get in any kind of flame war about :) --Geoff
Re: DateTime.pm on a Diet
On 7/5/05 7:04 AM, Eric Cholet wrote: > A while ago we did some profiling of an app that uses DateTime > extensively and found out that most of the time is spent in > Params::Validate. I understand this is somewhat of a religious issue, > but in this case it's a net loss for us: the params have been > validated zillions of times, and we lose performance in production. I > would love a DateTime sans param validation. Have you tried turning off the validation by setting the PERL_NO_VALIDATION environment variable? I agree with Dave that moving big/numerous data structures from Perl to C will have the most bang for the buck: On 7/4/05 1:47 PM, Dave Rolsky wrote: > Implementing time zones in XS as native C data structures would almost > certainly be a huge win. Beyond that, the biggest culprits will probably be what they always are in non-compute-intensive Perl: function/method call overhead and memory management overhead. Reduce the number of subs that are called and reduce the size and number of allocations/deallocations and then you might start to get closer to the point where actual "calculations" start to become visible in the profile. -John
Re: DateTime.pm on a Diet
Le 4 juil. 05 à 19:47, Dave Rolsky a écrit : Well, I'd take Sam's message with a grain of salt. I use it in lots of projects, and I know it's being used by lots of other people for lots of things. The idea that it's "too slow" is bogus. It might be too slow/bulky if you're creating thousands or hundreds of thousands of objects at once, but even then I wonder if it is. A while ago we did some profiling of an app that uses DateTime extensively and found out that most of the time is spent in Params::Validate. I understand this is somewhat of a religious issue, but in this case it's a net loss for us: the params have been validated zillions of times, and we lose performance in production. I would love a DateTime sans param validation. -- Eric Cholet
Re: DateTime.pm on a Diet
Dave,
I agree about taking it with a grain of salt.
However,
a) it is true that DateTime is slow (compared to other alternatives), and
b) It's just not good that we give people excuse like this to
not to use DateTime or to make it look like DateTime is not up to par
So I think we should be taking the performance issue seriously.
> Anyway, before people rush off down various paths I'd suggest some
> profiling and benchmarking, rather than just "making it faster".
I lost the data along the way, but if I remember correctly, the
slowest bits were:
-> DateTime::new
-> Params::Validate
-> timezones or locales (I forget)
-> date math
(It took way too long to generate the profiling data, so I really
don't feel like taking it again...)
> The first question to answer is what are people doing with these
> objects? I suspect the biggest benefit would be simply to speed up
> object creation, rather than the datetime math bits. Secondly, I think
> slimming down time zones would be a big win (for memory savings), and
> speeding them up would be nice although not necessarily that
> noticeable. After that improving datetime math would be good, I think.
>From looking at the code, I think there are still much more that can
be done in XS. Let's take DateTime->new, for example. When called,
new() calls these functions from within:
_ymd2rd (XS)
_time_as_seconds (XS)
_normalize_nanoseconds
_calc_utc_rd
_handle_offset_modifier
_calc_local_rd
The last 4 functions can definitely be implemented in C -- they are
probably good candidates, too, because most of they are mostly
responsible for doing integer arithmatic that is much better handled
in C.
I also would like to suggest that it might make sense to put most of
the data in C structs, e.g.
struct dt {
long utc_rd_days;
long utc_rd_secs;
long local_rd_days;
long local_rd_secs;
...
};
# DateTime's blessed hash would look like
#
# { _xs_state => $c_struct, ... other fields }
Then we'd be manipulating the dt->utc_rd_days, dt->utc_rd_secs, etc
fields directly instead of SVs. This way we can do probably 90% of the
internal calculations in C, and we can also minimize the memory foot
print.
To summarize... I was already looking at writing Locales in C (looks
like that mail didn't get sent to the list...(*1)) and minimizing the
foot print for DT::Duration, but I suppose I can do TimeZones first?
(*1) http://www.nntp.perl.org/group/perl.datetime/5859
P.S. - attached is a patch for DT::Locale in XS (there are still test failures)
--d
xs-locale.patch
Description: Binary data
generate_xs_from_icu
Description: Binary data
Re: DateTime.pm on a Diet
On Mon, 4 Jul 2005, Daisuke Maki wrote: > I have found DateTime unsuitable to quite a few tasks for > various unfortunate reasons; which is a shame given the level of > following and amount of design time that's gone into it. And I quite agree. It's a shame if we don't fix this and make DateTime.pm useful in any project -- and damnit I want to see it survive in Perl6. Well, I'd take Sam's message with a grain of salt. I use it in lots of projects, and I know it's being used by lots of other people for lots of things. The idea that it's "too slow" is bogus. It might be too slow/bulky if you're creating thousands or hundreds of thousands of objects at once, but even then I wonder if it is. So without anymore adew, here's my braindump of things that needs reviewing, and possible solutions: DateTime.pm - Prioritize features in DateTime.pm, and figure out the most effective area to improve - Use XS, where feasible. Move more existing PurePerl stuff in *PP.pm modules I think most everything that can be XS already is. XS has its own overhead, and if you're just calling lots of Perl functions in XS, particularly OO stuff, it may not be that much faster. I tried rewriting duration addition in XS and it didn't help at all. Locale / TimeZone Given... - Locale and TimeZones are singletons - They are not *that* often used Possible Improvements: - Defer loading them until absolutely necessary That's already done. - Do not keep copy of a these objects in memory, and keep the ID only This would be _slow_. Going to disk every time you need to look up locale or time zone information would be a huge speed hit. - (collary: keep necessary flags like "is_floating" in memory as part of DateTime's blessed hash) This isn't checked all that often. Also implement in XS? Implementing time zones in XS as native C data structures would almost certainly be a huge win. Unfortunately I don't have the C chops for this. Locales are pretty small and I doubt they're a big speed hit. Frankly, I thought Sam's message was a bit hand-wavy. He says: I can see why you'd want to ship a copy of the locale information and timezone database, but IMHO there needs to be a way that these can be coded to be a little more space/file efficient. But it's not clear if he's looked at the locales. They're already pretty tiny, just a few small arrays or hashes for each one. It's not clear how you could make them any smaller, and they're only loaded as needed! Anyway, before people rush off down various paths I'd suggest some profiling and benchmarking, rather than just "making it faster". The first question to answer is what are people doing with these objects? I suspect the biggest benefit would be simply to speed up object creation, rather than the datetime math bits. Secondly, I think slimming down time zones would be a big win (for memory savings), and speeding them up would be nice although not necessarily that noticeable. After that improving datetime math would be good, I think. -dave /*=== VegGuide.Orgwww.BookIRead.com Your guide to all that's veg. My book blog ===*/
DateTime.pm on a Diet
[using my gmail account because for some reason I'm not seeing my mails go through to the list...] Okay, I've been meaning to do this for a while, but now I've just about had enough (plus, now I have some free time). As Sam Vilain wronte in a separete thread: > I have found DateTime unsuitable to quite a few tasks for > various unfortunate reasons; which is a shame given the level of > following and amount of design time that's gone into it. And I quite agree. It's a shame if we don't fix this and make DateTime.pm useful in any project -- and damnit I want to see it survive in Perl6. So without anymore adew, here's my braindump of things that needs reviewing, and possible solutions: Break Down of DateTime.pm: DateTime.pm - Accessors/Mutators - Formatters (ymd, mdy, stringify, etc) - Calculations (day_of_week, week, internal calculations, etc) - Date math (+DateTime::Duration) DateTime-TimeZone DateTime-Locale DateTime-Duration DateTime.pm - Prioritize features in DateTime.pm, and figure out the most effective area to improve - Use XS, where feasible. Move more existing PurePerl stuff in *PP.pm modules Locale / TimeZone Given... - Locale and TimeZones are singletons - They are not *that* often used Possible Improvements: - Defer loading them until absolutely necessary - Do not keep copy of a these objects in memory, and keep the ID only - (collary: keep necessary flags like "is_floating" in memory as part of DateTime's blessed hash) Also implement in XS? If you guys have any suggestions, critiques, or anything else, please let the list know. For one, I'm going to see where I can just go ahead and switch to XS and send a patch... Let's make this excellent framework leaner and meaner! --d
