Thanks for sharing Ian! Guozhang
On Wed, May 24, 2017 at 3:49 AM, Damian Guy <damian....@gmail.com> wrote: > Thanks for reporting back Ian. Very useful information. > > Cheers, > Damian > > On Wed, 24 May 2017 at 10:40 Ian Duffy <i...@ianduffy.ie> wrote: > > > Hi All, > > > > In the past, we experienced lots of problems with running Kafka Stream > > Applications on AWS. > > > > We've seen issues with state locking, memory spiking to 100% and the > > instance dying, very > > slow startup on pulling down initial rocksdbs and so on.... > > > > Today we realised that the instances were experiencing such high > throughput > > that the IOPS > > credits on our EBSs storing the rocks EBS were completely used up > resulting > > in very slow IO > > and most of our issues. > > > > Sharing this as others might have the same problems. > > > > We were running on EBSs of 100GB which give 300 "provisioned" IOPS (3 > > iops per 1gb). > > Most cost effective solution for us is to just increase the EBS size to > get > > more "provisioned" > > IOPS. Alternative solutions are to use i3.* instances or IO1 EBS volumes > > but they are expensive. > > > > Hope this helps somebody else, > > Ian. > > > -- -- Guozhang
