Others feel free to chime in here, the recommendation I've seen is to not reduce the batch size below 500ms since the Spark machinery gets in the way below that point.
If you're looking for latency at the millisecond scale then I'd recommend checking out Apache Apex, Storm, and Flink. All of those are capable of single digit millisecond processing. On Mon, Oct 12, 2015 at 3:35 PM xweb <ashish8...@gmail.com> wrote: > What kind of latency are people in achieving in production using spark > streaming? Is it in 1 second+ range. or have people been able to achieve > latency in say 250 ms range. > > Any best practices on achieving sub second latency if even possible? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Latency-in-practice-tp25038.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
