DateTime::Set patch

[Stephen Gowing]

Hi,

DateTime::Set 0.21 dies when as_list is asked to produce a list from
outside the range of the set.  It shows up in the tests included with
DateTime::Event::ICal.

 Can't call method "is_infinite" on an undefined value at 
/usr/lib/perl5/site_perl/5.8.3/DateTime/Set.pm line 508.

I've attached a patch for DateTime::Set.  Hope you find it useful.

sbg
diff -ruN DateTime-Set-0.21/lib/DateTime/Set.pm DateTime-Set-0.21-sbg/lib/DateTime/Set.pm
--- DateTime-Set-0.21/lib/DateTime/Set.pm	2005-04-08 00:40:53.0 +1000
+++ DateTime-Set-0.21-sbg/lib/DateTime/Set.pm	2005-05-06 15:16:52.299754149 +1000
@@ -505,8 +505,8 @@
 ## return undef if $set->{set}->is_too_complex;  # undef = no begin/end
  
 return undef
-if $set->max->is_infinite ||
-   $set->min->is_infinite;
+if ( $set->max && $set->max->is_infinite ) ||
+   ( $set->min && $set->min->is_infinite );
 
 # return if $set->{set}->is_null;  # nothing = empty
 my @result;
diff -ruN DateTime-Set-0.21/t/as_list_empty.t DateTime-Set-0.21-sbg/t/as_list_empty.t
--- DateTime-Set-0.21/t/as_list_empty.t	1970-01-01 10:00:00.0 +1000
+++ DateTime-Set-0.21-sbg/t/as_list_empty.t	2005-05-06 15:19:07.307484035 +1000
@@ -0,0 +1,24 @@
+#!/usr/bin/perl
+
+use strict;
+
+use Test::More tests => 2;
+
+use DateTime;
+use DateTime::Set;
+
+my $d1 = DateTime->new( year => 2002, month => 3, day => 11 );
+my $d2 = DateTime->new( year => 2002, month => 4, day => 11 );
+my $d3 = DateTime->new( year => 2002, month => 5, day => 11 );
+my( $set, $r, @dt );
+
+$set = DateTime::Set->from_datetimes( dates => [ $d1 ] );
[EMAIL PROTECTED] = $set->as_list;
+$r = join(' ', @dt);
+
+is($r, '2002-03-11T00:00:00', 'Single date set');
+
[EMAIL PROTECTED] = $set->as_list( start => $d2, end => $d3 );
+$r = join(' ', @dt);
+
+is($r, '', 'Out of range');

Re: Storing recurrences in a SQL DBMS?

[Dave Rolsky]

On Thu, 5 May 2005, Simon Perreault wrote:
In fact, your solution is a way to move the set logic from the app to the SQL
server. I don't think it's a good idea. You might as well have the SQL server
call a procedure using DateTime::Sets directly, it would be the same
computation. Nothing is gained, except that the logic is stored unserialized,
and IMHO it's more appropriate to have the application server, not the DB
server, do the application logic.
The set _is_ data!  The fact that it requires some involved procedural 
logic doesn't change the fact that the at its core, it is data, and as 
such properly belongs in the DBMS.  Unfortunately, there is no DBMS that i 
know with good support for this sort of thing as a first class data type. 
But generally speaking, storing sets as scalar data in the DBMS is totally 
legit.  The fact that these sets can be infinite certainly makes things 
harder, but it doesn't mean that it shouldn't be done.

Well, not really. Suppose I want to store N sets. That would create N views
(not counting the sub-views). Now that the sets are stores, I want to know
"which sets intersect with today?" That's equivalent to "which views contain
a row that intersects with today?" I don't think that's possible to know
without iterating on the view names.
Yeah, this is definitely a smell.  I think the smell here is that the 
storage mechanism for a single piece of scalar data is being blown up into 
multiple tables.  What you really want is a native data type with the 
appropriate operators, so you can do:

 SELECT ... FROM RecurringEvent WHERE INTERSECTS(SET, $datetime)
That'd be sweet.
I like to think my method is pragmatic. I deal with finite and infinite sets
differently. I store the recursion rule as an ICal string in a column and in
another column I use a bool to indicate whether the set is finite or
infinite. If it's finite, I expand the recursion to all its occurrences in a
table of occurrences. Easy. If it's infinite, then there is simply nothing
else to do. It's up to the application to select all infinite sets, parse the
rule and do the calculation. As others mentioned, I cache occurrences of
infinite recursions in a separate table, so the calculation needs not be done
every time.
Yeah, this has been discussed a bit earlier in the thread.  This is 
probably the sanest way to do without support for these sets as a first 
class data type.

I do wonder how hard it would be to at least implement a _finite_ set as a 
first class data type.  In Postgres, Perl is one of the languages you can 
use for server side programming.  I haven't tried implementing a data type 
in Perl, but now I'm quite curious.

It seems like for many applications, you could get away with never using 
an infinite recurrence.  There are a couple options:

- Tell the user up front that they can only store X (say, 25) years worth, 
then make it easy to extend a soon-to-expire recurrence.

- Bound the set at some ridiculously far future date (y3k) and store that 
bounded set.

Either of these would be acceptable for things like a group web calendar, 
but if you're looking at something on an astronomic scale (storing 
something like rotations of the galaxy ;) it might be a problem.

-dave
/*===
VegGuide.Orgwww.BookIRead.com
Your guide to all that's veg.   My book blog
===*/

Re: Storing recurrences in a SQL DBMS?

[Flavio S. Glock]

Simon Perreault wrote:
"which views contain a row that intersects with today?" 
You could use a structure like:
create view year_month as
  select n, 'dt_year'  as tbl_name  from dt_year   union
  select n, 'dt_month' as tbl_name  from dt_month;
  n  | tbl_name
-+--
 1970-01-01 00:00:00 | dt_month
 1970-01-01 00:00:00 | dt_year
 1970-02-01 00:00:00 | dt_month
 1970-03-01 00:00:00 | dt_month
 1970-04-01 00:00:00 | dt_month
...
However, I don't know if you can modify a view to add or remove 
recurrences (is it possible?).

Here's my way of doing it. 
I guess I agree with you:
http://www.nntp.perl.org/group/perl.datetime/5749
- Flavio S. Glock

Re: Storing recurrences in a SQL DBMS?

[Simon Perreault]

On Thursday 05 May 2005 15:46, Flavio S. Glock wrote:
> CREATE VIEW MY_RECURRENCE_28403 ( N ) AS
>  SELECT ( N + INTERVAL '7 MONTH' ) FROM DT_YEAR;
> CREATE VIEW MY_RECURRENCE_83554 ( N ) AS
>  SELECT ( N + INTERVAL '5 DAY' ) FROM MY_RECURRENCE_28403;
> CREATE VIEW MY_RECURRENCE ( N ) AS
>  SELECT ( N ) FROM MY_RECURRENCE_83554;

Hmmm... Took me some time to get it. It's really imaginative, congratulations.

> You are responsible for maintaining the "DT_YEAR" table - it may be as
> big as needed.

That's one nugget that I don't like. Your solution gets close, but it really 
doesn't store an infinite set. It's a way to select subsets of what's already 
there. I've been thinking that it just wasn't possible to store infinite 
sets, and that's one more indication of that, although it gets really close.

In fact, your solution is a way to move the set logic from the app to the SQL 
server. I don't think it's a good idea. You might as well have the SQL server 
call a procedure using DateTime::Sets directly, it would be the same 
computation. Nothing is gained, except that the logic is stored unserialized, 
and IMHO it's more appropriate to have the application server, not the DB 
server, do the application logic.

>  > Is it possible to do a
>  > query such as "which sets intersect with today?" on the SQL
>  > data resulting from that serialization?
>
> Yes, this is just a normal table.

Well, not really. Suppose I want to store N sets. That would create N views 
(not counting the sub-views). Now that the sets are stores, I want to know 
"which sets intersect with today?" That's equivalent to "which views contain 
a row that intersects with today?" I don't think that's possible to know 
without iterating on the view names.

Here's my way of doing it. I'm sure I won't be telling you anything new, I'm 
just writing because maybe others might enjoy it.

I like to think my method is pragmatic. I deal with finite and infinite sets 
differently. I store the recursion rule as an ICal string in a column and in 
another column I use a bool to indicate whether the set is finite or 
infinite. If it's finite, I expand the recursion to all its occurrences in a 
table of occurrences. Easy. If it's infinite, then there is simply nothing 
else to do. It's up to the application to select all infinite sets, parse the 
rule and do the calculation. As others mentioned, I cache occurrences of 
infinite recursions in a separate table, so the calculation needs not be done 
every time.

This was an efficient way of doing it because:

- I postulate that it's not possible to store infinite sets in an SQL 
database. You may get close, but you'll never really have it. The best you 
can achieve is some sort of cache.

- In most applications, there are not many infinite sets to deal with. I 
noticed this when using a calendaring application. If I had many infinite 
sets, then my calendar would be much too *crowded* for me to use it. I can't 
imagine someone needing more than, let's say, a hundred infinite sets *in his 
whole lifetime*. Throw in a cache and you don't need to worry about 
performance.

Re: Storing recurrences in a SQL DBMS?

[Flavio S. Glock]

Simon Perreault wrote:
On Thursday 05 May 2005 13:31, Flavio S. Glock wrote:
Flavio S. Glock wrote:
I'm working on a module that translates datetime sets into SQL
statements.

Can you explain how you can serialize infinite sets? 

$ perl -Ilib -MDateTime::Format::SQL -e ' print 
DateTime::Format::SQL->format_set( set => 
DateTime::Event::Recurrence->yearly( months => 7, days => 5 ), 
table_name => 'MY_RECURRENCE', lazy => 1 ) '

Yields (this still needs some optimization):
CREATE VIEW MY_RECURRENCE_28403 ( N ) AS
SELECT ( N + INTERVAL '7 MONTH' ) FROM DT_YEAR;
CREATE VIEW MY_RECURRENCE_83554 ( N ) AS
SELECT ( N + INTERVAL '5 DAY' ) FROM MY_RECURRENCE_28403;
CREATE VIEW MY_RECURRENCE ( N ) AS
SELECT ( N ) FROM MY_RECURRENCE_83554;
You are responsible for maintaining the "DT_YEAR" table - it may be as 
big as needed.

> Is it possible to do a
> query such as "which sets intersect with today?" on the SQL
> data resulting from that serialization?
Yes, this is just a normal table.
- Flavio S. Glock

Re: Storing recurrences in a SQL DBMS?

[Simon Perreault]

On Thursday 05 May 2005 13:31, Flavio S. Glock wrote:
> Flavio S. Glock wrote:
> > I'm working on a module that translates datetime sets into SQL
> > statements.

Can you explain how you can serialize infinite sets? Is it possible to do a 
query such as "which sets intersect with today?" on the SQL data resulting 
from that serialization?

Thank you

Re: Storing recurrences in a SQL DBMS?

[Flavio S. Glock]

Flavio S. Glock wrote:
>
I'm working on a module that translates datetime sets into SQL 
statements.
Dave Rolsky wrote:
I think the basic idea is good, but the parameter & method names could 
be clearer.
>
Also, I'm wondering if that's the best namespace.  DT::F::SQL implies 
some sort of standard SQL formatting to me.  I wonder if it should even 
be in DT::F.  Maybe it should be something like DT::E::Recurrence::AsSQL?

Anyone else have a good idea for the module name?
How about DT::Set::SQL ?
You could use it to serialize other types of recurrences (crontab, 
sunrise).

It can be extended to serialize spansets too.
- Flavio

Re: Storing recurrences in a SQL DBMS?

[Dave Rolsky]

On Wed, 4 May 2005, Flavio S. Glock wrote:
Flavio S. Glock wrote:
I'm working on a module that translates datetime sets into SQL statements.
I'd appreciate to have your feedback on it.
Here is a preliminary version:
http://www.ipct.pucrs.br/flavio/perl/DateTime-Format-SQL-0.00_07.tar.gz
This is the example in the synopsis:
 use DateTime::Format::SQL;
 use DateTime::Event::Recurrence;
 my $set = DateTime::Event::Recurrence->monthly;
 my $formatter = DateTime::Format::SQL->factory;
 print $formatter->format_set( set => $set,
   table_name => 'MY_RECURRENCE',
   lazy => 1 );
I think the basic idea is good, but the parameter & method names could be 
clearer.

Also, I'm wondering if that's the best namespace.  DT::F::SQL implies some 
sort of standard SQL formatting to me.  I wonder if it should even be in 
DT::F.  Maybe it should be something like DT::E::Recurrence::AsSQL?

Anyone else have a good idea for the module name?
-dave
/*===
VegGuide.Orgwww.BookIRead.com
Your guide to all that's veg.   My book blog
===*/

[PATCH] DT::Locale doesn't handle RFC 3066 language tags

[Simon Perreault]

Hi,

First, let me congratulate everyone for such a nice toolkit. This is much 
better than what I've been using previously.

I was a bit surprised though when I noticed that DateTime::Locale::load() 
doesn't accept standard RFC 3066 language tags. This is it expects 
underscores while the RFC uses dashes. So 'fr-CA' doesn't work while 'fr_CA' 
does.

You'll find a small patch attached that simply converts dashes to underscores 
in the parameter that load() receives.
--- Locale.pm.bak	2005-05-05 11:17:14.0 -0400
+++ Locale.pm	2005-05-05 11:21:17.0 -0400
@@ -197,6 +197,9 @@
 my $class = shift;
 my $name = shift;
 
+# Support RFC 3066 language tags, which use '-' instead of '_'.
+$name =~ tr/-/_/;
+
 my $key = $name;
 
 return $LoadCache{$key} if exists $LoadCache{$key};