[
https://issues.apache.org/jira/browse/SPARK-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14652407#comment-14652407
]
Joseph K. Bradley commented on SPARK-7316:
------------------------------------------
Retargeting for 1.6
> Add step capability to RDD sliding window
> -----------------------------------------
>
> Key: SPARK-7316
> URL: https://issues.apache.org/jira/browse/SPARK-7316
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Affects Versions: 1.3.0
> Reporter: Alexander Ulanov
> Assignee: Alexander Ulanov
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> RDDFunctions in MLlib contains sliding window implementation with step 1.
> User should be able to define step. This capability should be implemented.
> Although one can generate sliding windows with step 1 and then filter every
> Nth window, it might take much more time and disk space depending on the step
> size. For example, if your window is 1000 then you will generate the amount
> of data thousand times bigger than your initial dataset. It does not make
> sense if you need just every Nth window, so the data generated will be 1000/N
> smaller.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
