1/ you mean like reshape in R?
2/ Or you mean by windowing the stream on a period basis?
1/ if you have RDD[Seq[Any]], you can have an RDD[Seq[Seq[Any]]] using
`transform` then `sliding(6,6)` on the passed Seq
2/ > If the period is time you may check the method ending with `window` in
DStream
> otherwise... I don't see the use case, so I'd have some difficulties
helping you ^ ^
Also, if you're building (row, col) pairs along with each cell data when
the data is coming along, the PairedDStreamFunctions could help you if your
put this pair as a key. But I'm just guessing...
HTH (a bit :D)
andy
On Wed, Nov 20, 2013 at 1:01 AM, Michael Kun Yang <[email protected]>wrote:
> Hi spark-enthusiasts,
>
> I am new to spark streaming. I need to convert streaming data into table.
>
> How to convert a data stream
> {x_1, x_2, x_3, ..., x_n, ...}
> into a table with the format:
> x_1, x_2, x_3, x_4, x_5, x_6
> x_2, x_3, x_4, x_5, x_6, x_7
> ...
> x_{n + 1}, x_{n + 2}, ..., x_{n + 7}
> ...
>
> Thank you very much!
> -Kun
>
