Re: calculate time difference between consecutive rows

[ayan guha]

Please post your code and results. Lag will be null for the first record.
Also, what data type you are using? Are you using cast?
On 21 Jul 2016 14:28, "Divya Gehlot"  wrote:

> I have a dataset of time as shown below :
> Time1
> 07:30:23
> 07:34:34
> 07:38:23
> 07:39:12
> 07:45:20
>
> I need to find the diff between two consecutive rows
> I googled and found the *lag *function in *spark *helps in finding it .
> but its  giving me *null *in the result set.
>
> Would really appreciate the help.
>
>
> Thanks,
> Divya
>
>

Re: calculate time difference between consecutive rows

[Jacek Laskowski]

Hi,

What was the code you tried? You should use the built-in window
aggregates (windows) functions or create one yourself. I haven't tried
lag before (and don't think it's what you need really).

Pozdrawiam,
Jacek Laskowski

https://medium.com/@jaceklaskowski/
Mastering Apache Spark http://bit.ly/mastering-apache-spark
Follow me at https://twitter.com/jaceklaskowski


On Thu, Jul 21, 2016 at 6:27 AM, Divya Gehlot  wrote:
> I have a dataset of time as shown below :
> Time1
> 07:30:23
> 07:34:34
> 07:38:23
> 07:39:12
> 07:45:20
>
> I need to find the diff between two consecutive rows
> I googled and found the lag function in spark helps in finding it .
> but its  giving me null in the result set.
>
> Would really appreciate the help.
>
>
> Thanks,
> Divya
>

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

calculate time difference between consecutive rows

[Divya Gehlot]

I have a dataset of time as shown below :
Time1
07:30:23
07:34:34
07:38:23
07:39:12
07:45:20

I need to find the diff between two consecutive rows
I googled and found the *lag *function in *spark *helps in finding it .
but its  giving me *null *in the result set.

Would really appreciate the help.


Thanks,
Divya