[ https://issues.apache.org/jira/browse/SPARK-3660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tathagata Das updated SPARK-3660: --------------------------------- Priority: Major (was: Minor) > Initial RDD for updateStateByKey transformation > ----------------------------------------------- > > Key: SPARK-3660 > URL: https://issues.apache.org/jira/browse/SPARK-3660 > Project: Spark > Issue Type: New Feature > Components: Streaming > Reporter: Soumitra Kumar > Fix For: 1.3.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > How to initialize state tranformation updateStateByKey? > I have word counts from previous spark-submit run, and want to load that in > next spark-submit job to start counting over that. > One proposal is to add following argument to updateStateByKey methods. > initial : Option [RDD [(K, S)]] = None > This will maintain the backward compatibility as well. > I have a working code as well. > This thread started on spark-user list at: > http://apache-spark-user-list.1001560.n3.nabble.com/How-to-initialize-updateStateByKey-operation-td14772.html -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org