I don't think persist is meant for end-user usage. You might want to call saveAsTextFiles, for example, if you're saving to the file system as strings. You can also dump the DStream to a DB -- there are samples on this list (you'd have to do a combo of foreachRDD and mapPartitions, likely)
On Wed, Oct 1, 2014 at 3:49 AM, Chia-Chun Shih <chiachun.s...@gmail.com> wrote: > Hi, > > My application is to digest user logs and deduct user quotas. I need to > maintain latest states of user quotas persistently, so that latest user > quotas will not be lost. > > I have tried *updateStateByKey* to generate and a DStream for user quotas > and called *persist(StorageLevel.MEMORY_AND_DISK())*, but it didn't work. > > Are there better approaches to persist states for spark streaming? > > Thanks. > > > > > > >
