Re: How to disable write ahead logs?

2016-11-28 Thread Takeshi Yamamuro
Hi, If you disable tracker-side WAL, you unset a checkpoint dir by using streamingContext.checkpoint(). http://spark.apache.org/docs/latest/streaming-programming-guide.html#how-to-configure-checkpointing // maropu On Tue, Nov 29, 2016 at 9:04 AM, Tim Harsch wrote: > Hi all, > > I set `spark.s

How to disable write ahead logs?

2016-11-28 Thread Tim Harsch
Hi all, I set `spark.streaming.receiver.writeAheadLog.enable=false` and my history server confirms the property has been set. Yet, I continue to see the error: 16/11/28 15:47:04 ERROR util.FileBasedWriteAheadLog_ReceivedBlockTracker: Failed to write to write ahead log after 3 failures I su