The receiver-based kafka createStream in spark 1.2 uses zookeeper to store offsets. If you want finer-grained control over offsets, you can update the values in zookeeper yourself before starting the job.
createDirectStream in spark 1.3 is still marked as experimental, and subject to change. That being said, it works better for me in production than the receiver based api. On Fri, Jun 26, 2015 at 6:43 AM, Shushant Arora <[email protected]> wrote: > I am using spark streaming 1.2. > > If processing executors get crashed will receiver rest the offset back to > last processed offset? > > If receiver itself got crashed is there a way to reset the offset without > restarting streaming application other than smallest or largest. > > > Is spark streaming 1.3 which uses low level consumer api, stabe? And > which is recommended for handling data loss 1.2 or 1.3 . > > > > > > >
