On Tue, Jun 19, 2012 at 11:07 AM, Aquil H. Abdullah <
aquil.abdul...@gmail.com> wrote:
> Hello Anthony,
>
> Thanks for your reply. I did a little bit more searching and found the
> discussion datetime objects on tables (
> http://comments.gmane.org/gmane.comp.python.pytables.user/2469) ,which is
> a bit more recent. In it you list your order of preference for storing
> timestamps:
>
> 1. Create a numpy dtype or PyTables description that matches the structure
> of datetime to your desired precision and save them in a Table. If you want
> to save timezone information as well, I might add an extra length-3 string
> column and save the str representation of the tzinfo field ('UTC', 'KST',
> 'EDT').
>
> [Could you give an example?]
>
> The best I could come up with so far is to create a column of type
> tables.Time32Col() and then transform my datetime object into a timestamp
> and store it. For example,
>
> In [1]: import tables, pytz
> In [2]: from datetime import datetime, timedelta
> In [3]: from time import mktime
> In [4]: import calendar
> In [5]: dt = datetime.utcnow().replace(tzinfo=pytz.UTC)
> In [6]: dt
> Out[6]: datetime.datetime(2012, 6, 19, 15, 22, 19, 892159, tzinfo=<UTC>)
> In [7]: ts = calendar.timegm(dt.timetuple())
> In [8]: ts
> Out[8]: 1340119339
>
> I can then store ts in my table, and retrieve it with
>
> In [35]: datetime.utcfromtimestamp(tbl.cols.timestamp[0])
> Out[35]: datetime.datetime(2012, 6, 19, 15, 22, 19)
>
> although, at this point I've lost the timezone.
>
> Anyway, I am a little bit confused, because I don't understand the
> difference between a Time32Col() and a Int32Col() in the example that I've
> written. Do you have a better example?
>
So in terms of how they are stored on disk, Time32Col and Int32Col are
basically the same thing. The difference is that they are flagged as being
times so that other processes will know to interpret them as time stamps
rather than plain old ints. The same thing goes for Time64Col and
Float64Col.
Now as an example, to capture the full information you want, you will
effectively have to store two columns: one for the time stamp and one for
the time zone. So you description will look like:
desc = {'timestamp': Time64Col(pos=1), 'tz': StringCol(3, pos=2)}
Then the entries will look like tuples which match the desc like the
following:
In [35]: n = datetime.utcnow().replace(tzinfo=pytz.timezone('US/Central'))
In [36]: time.mktime(n.timetuple()) + n.microsecond * 1e-6, str(n.tzname())
Out[36]: (1340147092.873432, 'CST')
Basically, you have to store the tz info next to the timestamp. I hope
this helps!
Be Well
Anthony
>
> Thanks!
>
>
>
> On Tue, Jun 19, 2012 at 2:16 AM, Anthony Scopatz <scop...@gmail.com>wrote:
>
>> Hey Aquil,
>>
>> Yes, the string method certainly works. The other thing you could do
>> that isn't mentioned in that post is have a table or 2D array whose first
>> column is the float timestamp [1] and whose second column is a string
>> repr of just the timezone name (or a int or float of the offset in sec of
>> the timezone). This will likely be faster.
>>
>> All of the strategies mentioned will work, but will have varying speeds.
>> I personally prefer anything with a timestamp since it relies on the
>> canonical form.
>>
>> Be Well
>> Anthony
>>
>> PS I am sorry that you have to deal with timezones. They are a real pain!
>>
>> 1. http://pytables.github.com/usersguide/libref.html#tables.Time64Col
>>
>> On Mon, Jun 18, 2012 at 10:05 PM, Aquil H. Abdullah <
>> aquil.abdul...@gmail.com> wrote:
>>
>>> I need to store dates and timezone aware datetimes in a PyTable.
>>> Currently, I am storing values of those types as ISO 8601 strings. I ran
>>> across the thread pytables for timeseries data (
>>> http://osdir.com/ml/python.pytables.user/2007-11/msg00036.html) which
>>> leads me to believe that what I am doing is the best way to store these
>>> types of values, but I just wanted to check in case I am missing something.
>>>
>>> Regards,
>>>
>>> --
>>> Aquil H. Abdullah
>>> aquil.abdul...@gmail.com
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. Discussions
>>> will include endpoint security, mobile security and the latest in malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> Pytables-users mailing list
>>> Pytables-users@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/pytables-users
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Pytables-users mailing list
>> Pytables-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/pytables-users
>>
>>
>
>
> --
> Aquil H. Abdullah
> aquil.abdul...@gmail.com
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Pytables-users mailing list
> Pytables-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>
>
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
