Hi Yana, So, user quotas need another data store, which can guarantee persistence and afford frequent data updates/access. Is it correct?
Thanks, Chia-Chun 2014-10-01 21:48 GMT+08:00 Yana Kadiyska <yana.kadiy...@gmail.com>: > I don't think persist is meant for end-user usage. You might want to call > saveAsTextFiles, for example, if you're saving to the file system as > strings. You can also dump the DStream to a DB -- there are samples on this > list (you'd have to do a combo of foreachRDD and mapPartitions, likely) > > On Wed, Oct 1, 2014 at 3:49 AM, Chia-Chun Shih <chiachun.s...@gmail.com> > wrote: > >> Hi, >> >> My application is to digest user logs and deduct user quotas. I need to >> maintain latest states of user quotas persistently, so that latest user >> quotas will not be lost. >> >> I have tried *updateStateByKey* to generate and a DStream for user >> quotas and called *persist(StorageLevel.MEMORY_AND_DISK())*, but it >> didn't work. >> >> Are there better approaches to persist states for spark streaming? >> >> Thanks. >> >> >> >> >> >> >> > >
