[
https://issues.apache.org/jira/browse/PIG-1314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12873382#action_12873382
]
Russell Jurney commented on PIG-1314:
-------------------------------------
Ok, thinking about really doing this soon, after Boolean. I'd like to add two
new primitives to Pig - DateTime and Duration.
I'd do this on the wiki, but I don't have edit access. Can someone please
grant the ability to make a new page to user RussellJurney on the Pig wiki?
Design Notes:
1) I'd like to use Jodatime for this, as I did in the DateTime UDFs. It is
possible to use the Java date libs, but it would be painful to do so. Jodatime
also performs better than Java's native date classes. It is Apache 2.0
licensed and is already pulled in via ivy in the DateTime UDFs - see PIG-1310
2) Date Format for text/dumps: ISO8601. Looks like: [YYYY][MM][DD]T[hh][mm]Z
It is a human readable, sortable/comparable, international standard. See
http://en.wikipedia.org/wiki/ISO_8601#Dates
2.5) In memory type: org.joda.time.DateTime. See
http://joda-time.sourceforge.net/apidocs/org/joda/time/DateTime.html
The internal format of jodatime is a Long epoch/Unix/POSIX time. See
http://joda-time.sourceforge.net/faq.html#internalstorage
3) Duration Format for text/dumps: ISO8601. Looks like:
P[n]Y[n]M[n]DT[n]H[n]M[n]S It is a human readable, sortable/comparable,
international standard. See http://en.wikipedia.org/wiki/ISO_8601#Durations
3.5) In-memory format: org.joda.time.Duration. See
http://joda-time.sourceforge.net/apidocs/org/joda/time/Duration.html
4) All date functions in PIG-1310 should be included, except those replaced by
the use of operators on datetimes and durations. Adding/subtracting datetimes
should result in a duration. Durations can be
added/subtracted/divided/multiplied/negated.
Date/Duration truncation, date differences, date parsing/conversion should be
included. Conversion from int/long POSIX, SQL and datemonth should be
included. Conversion from any string with a DateFormat string should be
included.
5) Casting to and from Integer and Long should be supported, as a Unix/POSIX
time. Casting to/from chararray in ISO8601 format should be supported.
Comments? Suggestions?
> Add DateTime Support to Pig
> ---------------------------
>
> Key: PIG-1314
> URL: https://issues.apache.org/jira/browse/PIG-1314
> Project: Pig
> Issue Type: Bug
> Components: data
> Affects Versions: 0.7.0
> Reporter: Russell Jurney
> Fix For: 0.8.0
>
> Original Estimate: 672h
> Remaining Estimate: 672h
>
> Hadoop/Pig are primarily used to parse log data, and most logs have a
> timestamp component. Therefore Pig should support dates as a primitive.
> Can someone familiar with adding types to pig comment on how hard this is?
> We're looking at doing this, rather than use UDFs. Is this a patch that
> would be accepted?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
