RE: Date / time stuff with spark.

2016-01-22 Thread Mohammed Guller
..@gmail.com] Sent: Friday, January 22, 2016 8:01 AM To: Spencer, Alex (Santander) Cc: Andrew Holway; user@spark.apache.org Subject: Re: Date / time stuff with spark. Related thread: http://search-hadoop.com/m/q3RTtSfi342nveex1&subj=RE+NPE+when+using+Joda+DateTime FYI On Fri, Jan 22, 2016 at 6:50 AM, Spe

Re: Date / time stuff with spark.

2016-01-22 Thread Ted Yu
ormat.forPattern("-MM-dd"); > val tranDate = dateFormat.parseDateTime(someDateString) > > > Alex > > -Original Message- > From: Andrew Holway [mailto:andrew.hol...@otternetworks.de] > Sent: 21 January 2016 19:25 > To: user@spark.apache.org > Subje

Re: Date / time stuff with spark.

2016-01-22 Thread Durgesh Verma
anDate = dateFormat.parseDateTime(someDateString) > > > Alex > Â > -Original Message- > From: Andrew Holway [mailto:andrew.hol...@otternetworks.de] > Sent: 21 January 2016 19:25 > To: user@spark.apache.org > Subject: Date / time stuff with spark. > >

RE: Date / time stuff with spark.

2016-01-22 Thread Spencer, Alex (Santander)
e val dateFormat = format.DateTimeFormat.forPattern("-MM-dd"); val tranDate = dateFormat.parseDateTime(someDateString) Alex   -Original Message- From: Andrew Holway [mailto:andrew.hol...@otternetworks.de] Sent: 21 January 2016 19:25 To: user@spark.apache.org Subject: Da

Re: Date / time stuff with spark.

2016-01-21 Thread Andrew Holway
P.S. We are working with Python. On Thu, Jan 21, 2016 at 8:24 PM, Andrew Holway wrote: > Hello, > > I am importing this data from HDFS into a data frame with > sqlContext.read.json(). > > {“a": 42, “a": 56, "Id": "621368e2f829f230", “smunkId": > "CKm26sDMucoCFReRGwodbHAAgw", “popsicleRange": "176

Date / time stuff with spark.

2016-01-21 Thread Andrew Holway
Hello, I am importing this data from HDFS into a data frame with sqlContext.read.json(). {“a": 42, “a": 56, "Id": "621368e2f829f230", “smunkId": "CKm26sDMucoCFReRGwodbHAAgw", “popsicleRange": "17610", "time": "2016-01-20T23:59:53+00:00”} I want to do some date/time operations on this json data b