Thanks for reporting back Ian. Very useful information. Cheers, Damian
On Wed, 24 May 2017 at 10:40 Ian Duffy <i...@ianduffy.ie> wrote: > Hi All, > > In the past, we experienced lots of problems with running Kafka Stream > Applications on AWS. > > We've seen issues with state locking, memory spiking to 100% and the > instance dying, very > slow startup on pulling down initial rocksdbs and so on.... > > Today we realised that the instances were experiencing such high throughput > that the IOPS > credits on our EBSs storing the rocks EBS were completely used up resulting > in very slow IO > and most of our issues. > > Sharing this as others might have the same problems. > > We were running on EBSs of 100GB which give 300 "provisioned" IOPS (3 > iops per 1gb). > Most cost effective solution for us is to just increase the EBS size to get > more "provisioned" > IOPS. Alternative solutions are to use i3.* instances or IO1 EBS volumes > but they are expensive. > > Hope this helps somebody else, > Ian. >
