Re: Potential DateTime DOS Attack
On Wed, 16 Dec 2009, Michael G Schwern wrote: Give me something to work with here. Some insight into what and why DateTime is doing what its doing. Is there a reason that DST info has to be generated linearly? Would it be difficult to hold off on generating time zone info until its needed? Are there instructions somewhere for dealing with the Olsen database? SOME sort of discussion about how to solve the problem rather than the ways to paper over it. I've thought about this a bit, and one solution I came up with was something like this ... When generating the time zone data, we know that after a certain point, there is either one rule in effect (zone without DST or with permanent DST), or two rules in fixed alternation. Either way, we generate a subroutine that determines the current time zone data (which is referred to as a "span" internally, as in a span of time during which a specific rule is in effect). For the one-rule zones, this is trivial. Just return the data for the last rule. For the two-rule zones, this should be calculable mathematically. In both cases, we can simply not store the generated data, ever. The generated files already include pre-calculated data for the next 10 years. We could increase that to 20, and simply leave it at that. -dave /* http://VegGuide.org http://blog.urth.org Your guide to all that's veg House Absolute(ly Pointless) */
Re: Potential DateTime DOS Attack
On Wed, Dec 16, 2009 at 4:16 PM, Michael G Schwern wrote: > I am, quite frankly, appalled at the response I've gotten to this report. > > No, this is not something the user should be guarding against. No, > documenting it does not make it go away. No, you shouldn't put an arbitrary > upper bound in. No, I should not have been using UTC. That is all > accepting low-quality. We don't do that in Perl. > > This isn't just an annoyance, its a wide spread security hole triggered by > totally innocent use. Its like finding out my doorbell will cause my house > to explode if somebody buzzes too long. I do not want to program in an > environment where the assumption is that everything is dangerous. This > isn't even a problem one should imagine encountering. > > The reaction I was expecting was more along the lines of "oh fuck, that's > really bad, let's figure out how to fix this". Yes, its ok to report a > problem without a ready solution. Instead, I'm told its documented and to > cough up a patch. > > This is not a simple problem to solve. I'm not even sure what the efficient > DST algorithm is, though I'm looking for it. I've looked into the > DateTime::TimeZone code before and this is not going to be a simple patch. > I'm willing to try and fix it, but not if the DateTime folks, the folks who > know the code, are reacting with something between lethargy and hostility. > It going to be hard enough without dragging everyone along. > > Give me something to work with here. Some insight into what and why > DateTime is doing what its doing. Is there a reason that DST info has to be > generated linearly? Would it be difficult to hold off on generating time > zone info until its needed? Are there instructions somewhere for dealing > with the Olsen database? SOME sort of discussion about how to solve the > problem rather than the ways to paper over it. > I don't want to be insensitive, but I think this response is a touch melodramatic. First, Perl has plenty of dangerous behaviors. CGI's param(...) returning a list can cause a lot of problems and exploits if a user isn't careful and sanitize their input properly. To claim that "Perl doesn't do that" doesn't make it true. Programming is hard. Second, I think you grossly misconstrued the responses to this. I think everybody suggesting things in the thread did it with the mentality of an intermediate fix rather than a long term one. Very few people are comfortable with the internals of DateTime to offer up proper solutions. Third, you aren't going to get an "oh fuck" reaction out of a problem that is documented any more than you would expect Congress recalling all vehicles because if you hold the accelerator down you could crash into a brickwall. Analogies suck, lets not use them. The point is that raising a known issue and expecting something different than, "Yes, we know about it what are other ways to handle it" seems a little on the unreasonable size. On a constructive note, DateTime::TimeZone has Olsen timezone information. I had no idea that it didn't suffer the same problem (like I said, I don't have a clue about the Internals, and I just extrapolated that to "Very few people"... busted). I think that at the very least an *intermediate* fix would be to update the pod warning to be more prominent and a mention that Olsen tz files do not suffer the same consequences. Thanks, -J PS., Schwern, you're the man. If you can't do it, who can!? I still have your tea, btw (probably long stale now). If you can get this fixed, I'll throw in a pitcher of beer with it.
Re: Potential DateTime DOS Attack
Schwern++. On Wed, Dec 16, 2009 at 4:16 PM, Michael G Schwern wrote: > I am, quite frankly, appalled at the response I've gotten to this report. > > No, this is not something the user should be guarding against. No, > documenting it does not make it go away. No, you shouldn't put an arbitrary > upper bound in. No, I should not have been using UTC. That is all > accepting low-quality. We don't do that in Perl. > > This isn't just an annoyance, its a wide spread security hole triggered by > totally innocent use. Its like finding out my doorbell will cause my house > to explode if somebody buzzes too long. I do not want to program in an > environment where the assumption is that everything is dangerous. This > isn't even a problem one should imagine encountering. > > The reaction I was expecting was more along the lines of "oh fuck, that's > really bad, let's figure out how to fix this". Yes, its ok to report a > problem without a ready solution. Instead, I'm told its documented and to > cough up a patch. > > This is not a simple problem to solve. I'm not even sure what the efficient > DST algorithm is, though I'm looking for it. I've looked into the > DateTime::TimeZone code before and this is not going to be a simple patch. > I'm willing to try and fix it, but not if the DateTime folks, the folks who > know the code, are reacting with something between lethargy and hostility. > It going to be hard enough without dragging everyone along. > > Give me something to work with here. Some insight into what and why > DateTime is doing what its doing. Is there a reason that DST info has to be > generated linearly? Would it be difficult to hold off on generating time > zone info until its needed? Are there instructions somewhere for dealing > with the Olsen database? SOME sort of discussion about how to solve the > problem rather than the ways to paper over it. >
Re: Potential DateTime DOS Attack
I am, quite frankly, appalled at the response I've gotten to this report. No, this is not something the user should be guarding against. No, documenting it does not make it go away. No, you shouldn't put an arbitrary upper bound in. No, I should not have been using UTC. That is all accepting low-quality. We don't do that in Perl. This isn't just an annoyance, its a wide spread security hole triggered by totally innocent use. Its like finding out my doorbell will cause my house to explode if somebody buzzes too long. I do not want to program in an environment where the assumption is that everything is dangerous. This isn't even a problem one should imagine encountering. The reaction I was expecting was more along the lines of "oh fuck, that's really bad, let's figure out how to fix this". Yes, its ok to report a problem without a ready solution. Instead, I'm told its documented and to cough up a patch. This is not a simple problem to solve. I'm not even sure what the efficient DST algorithm is, though I'm looking for it. I've looked into the DateTime::TimeZone code before and this is not going to be a simple patch. I'm willing to try and fix it, but not if the DateTime folks, the folks who know the code, are reacting with something between lethargy and hostility. It going to be hard enough without dragging everyone along. Give me something to work with here. Some insight into what and why DateTime is doing what its doing. Is there a reason that DST info has to be generated linearly? Would it be difficult to hold off on generating time zone info until its needed? Are there instructions somewhere for dealing with the Olsen database? SOME sort of discussion about how to solve the problem rather than the ways to paper over it.
DateTime.pm and Bugzilla
Hello, I am trying to update our Bugzilla system which required a Perl upgrade as well. IFor each module error I fix, a new one pops up. Hopefully I am getting close to squashing them all. I am currently stuck on a DateTime error: Software error: Can't locate object method "_calc_utc_rd" via package "DateTime::Infinite::Future" at C:/Perl/lib/DateTime/Infinite.pm line 54. Compilation failed in require at DateTime.pm line 69. Compilation failed in require at Bugzilla/Util.pm line 53. BEGIN failed--compilation aborted at Bugzilla/Util.pm line 53. Compilation failed in require at Bugzilla/Hook.pm line 26. BEGIN failed--compilation aborted at Bugzilla/Hook.pm line 26. Compilation failed in require at Bugzilla/Config.pm line 37. BEGIN failed--compilation aborted at Bugzilla/Config.pm line 37. Compilation failed in require at Bugzilla.pm line 38. BEGIN failed--compilation aborted at Bugzilla.pm line 38. Compilation failed in require at C:/Bugzilla/index.cgi line 34. BEGIN failed--compilation aborted at C:/Bugzilla/index.cgi line 34. I have downloaded datetime.pm several times but I can only seem to find your version. (I was hopeful that an older version might work better as my new ActivePerl on Windows is 589 build 826 (not 10). I do not find an infinite folder or future and past pm files no matter where I look. Or maybe, I'm on the wrong track altogether. I do not have enough experience with Perl to start trying to rewrite Bugzilla dependencies. Do you have any advice? Thank you for your time, Ginger Ginger L. Smith School Zone Interactive Quality Control & Technical Support ging...@schoolzone.com 616-846-5030 x127
Re: time_zone and DateTime::Format::Strptime
There is nothing I want to do with parse_datetime. parse_datetime works
great and nothing I have said suggests that anything different should happen
when parsing a string to a datetime object. Everything about
parse_datetime, reading of a string and setting of a time_zone works exactly
how anyone would expect. So again parse_datetime is 100% OK, nothing should
change about it at all.
My suggestion was simply if parse_datetime sets the value of your string to
a specific time_zone why can't format_datetime. I showed earlier that the
time_zone parameter when passed to the constructor of Format:: already
overrides any time zone that may be pulled from the string. What is
returned from parse_datetime is a datetime object that is constructed with
the time_zone parameter provided.
When format_datetime is called it uses the pattern and the locale from the
Format:: constructor to do it's job:
sub format_datetime {
my ( $self, $dt ) = @_;
my $pattern = $self->pattern;
$pattern =~ s/%O/$dt->time_zone->name/eg;
return $dt->clone->set_locale($self->
locale)->strftime($pattern);
}
The $dt object you pass in currently has it's locale changed. Thats a
parameter passed to the constructor of Format:: used for both constructing a
datetime object and for formatting a datetime object. If I pass in a $dt to
format_datetime with a locale of en_AS and the constructor of Format:: has
locale set to en_US format_datetime clones my object sets the locale then
returns the pattern.
All I'm purposing is something like this:
sub format_datetime {
my ( $self, $dt ) = @_;
my $pattern = $self->pattern;
$pattern =~ s/%O/$dt->time_zone->name/eg;
my $clone_dt = $dt->clone;
if ($self->format_with_time_zone) {
return
$clone_dt->set_locale($self->locale)->set_time_zone($self->time_zone)->strftime($pattern);
}
return $clone_dt->set_locale($self->locale)->strftime($pattern);
}
No crazy math. Just setting the time_zone of the object to be the time_zone
of the Format:: constructor in the same way locale is done.
If you have a DateTime based on epoch 1260967863 then all of the following
are representations of that epoch time:
12/16/09 12:51:03 UTC
12/16/09 07:51:03 EST
12/16/09 06:51:03 CST
12/16/09 13:51:03 CET
They all reference the same point in time. They are just formatted
different for the convenience of those who wish to read times based on a
particular time_zone. Nothing is being done to change the point in time of
the object. All you are doing is formatting datetime object so it makes
sense as a string. Nothing that produces the formatted string should change
the point in time the object represented originally. Neither locale or
time_zone alter the point in time that is being represented by a string.
-Kevin
On Wed, Dec 16, 2009 at 2:40 AM, Evan Carroll wrote:
> > Just the format_datetime function when it comes to the value which is
> > assigned to the time_zone attribue. It's already an attribute of the
> > module.
>
> I simply /do not/ believe you understand the difference between being
> 1:30 "read-in the context of" America/Chicago, and "converting it with
> date-math" to the timezone of America/Chicago. The constructor
> argument has /nothing at all/ to do with what your wanting (date-math
> convenience).
>
> >> Here is better API idea, why not just make the hash ref to
> >> format_datetime, a huge method wrapper
> >
> > Taken to the point of absurdity.
>
> But why not? If your goal is simply to provide a convenience wrapper
> that is capable of doing the date-math through DateTime, then why not
> permit you to say, add => '1 hour', to the current time, to "format"
> the time for a clock that is moving one hour too slow? I agree it is
> rather absurd and silly, but I'm not seeing why it is /more/ silly.
>
> Maybe it would be better if for your understanding you read all of
> these arguments as implicitly suffixed with
> "as_used_in_the_constructor_to_DateTime", so you don't confuse the
> DT::Format::* with something that can or should manipulate the
> supplied DateTime in the call to parse_datetime.
>
> --
> Evan Carroll
> System Lord of the Internets
> http://www.evancarroll.com
>
Reply
On Wed, Dec 16, 2009 at 2:40 AM, Evan Carroll wrote:
> > Just the format_datetime function when it comes to the value which is
> > assigned to the time_zone attribue. It's already an attribute of the
> > module.
>
> I simply /do not/ believe you understand the difference between being
> 1:30 "read-in the context of" America/Chicago, and "converting it with
> date-math" to the timezone of America/Chicago. The constructor
> argument has /nothing at all/ to do with what your wanting (date-math
> convenience).
>
> >> Here is better API idea, why not just make the hash ref to
> >> format_datetime, a huge method wrapper
> >
> > Taken to the point of absurdity.
>
> But why not? If your goal is simply to provide a convenience wrapper
> that is capable of
Re: Potential DateTime DOS Attack
J. Shirley wrote: >Do not try to use named time zones (like "America/Chicago") with dates >very far in the future (thousands of years). The current >implementation ofDateTime::TimeZone?will use a huge amount of memory >calculating all the DST changes from now until the future date. You could instead use DateTime::TimeZone::Tzfile, which does not suffer from this problem. Just requires that you have the compiled Olson files (which are freely available if you don't already have them). $zone = DateTime::TimeZone::Tzfile->new( filename => "/usr/share/zoneinfo/America/Chicago", ); $dt = DateTime->new( year => 3, month => 1, day => 1, time_zone => $zone, ); -zefram
