Hi, I'm new to spark. And i needed some help in understanding how reduceByKey works.
I have the following data: col1 col2 col3 1/11/2014 12:18:40 AM 123 143 1/11/2014 12:18:45 AM 123 143 1/11/2014 12:18:49 AM 123 143 the output i need is col2 col3 totaltime(currect value of col1 - prev val of col1) 123 143 9 I'm doing the following: map((col2,col3),col1).reduceByKey( *<here i don't know how to perform the subtraction of dates > *) How to perform subtraction of dates ? How does reduceByKey work when my map emits as follows ((col2,col3),(col1,col4))? Thanks in advance.
