I have build spray service which would run in 2 8GB Boxes . It receives
Json every 5 seconds which would get converted to MyJsonMessage .Each
MyJsonMessage will contain 3000 MyObjects. So 3000 MyObjects would get
created every 5 seconds.
Internally I am using batching to process these 3000 objects in batch of
1000s which will be sent to an consumerActor.
My heap size is set to 5GB.
I am emitting metrics. I have seen YGC time increasing at a very high rate
, even heap size grows and frequently touches 5 GB. I am new to Akka so I
am not sure if there is any memory leak here or only option is to add more
boxes. Solutions/Suggestions?
case class MyJsonMessage(
inputString1:String,
inputString2:String,
objectList:List[MyObject])
case class MyObject(
objectName : String,
objectValue : String,
data : Map[String,String])
class MyHttpService (implicit val context: akka.actor.ActorRefFactory
.....) extends MyJsonProtocol{
def worker = MyHttpServiceWorker
val multicastRoute = path("service" / "task" / segment) {
(configName: String) => {
post { ctx =>
var payload = try {
Left(JsonParser(ctx.request.entity.asString).convertTo[MyJsonMessage])
} catch {
case ex: Exception =>
log.error("Error converting message payload: ", ex)
Right(ex)
}
worker.process(payload.left.get)
}
object MyHttpServiceWorker{
def process[T](request: T) = {
request match {
case request : MyJsonMessage =>
val objectListCount = request.objectList.size
val batches = objectListCount > 1000 match {
case true => ceil(objectListCount * 1.0 / 1000).toInt
case false => 1
}
List.range[Long](0, batches).foreach(batch => {
val split = MyBulkObjectRequest(request, batch.toInt *
batchSize, limit * batchSize)
MyObjectRequestConsumer ! split
limit += 1
})
MyObjectSuccessResponse(objectListCount, batches, requestId , UUID))
}}
Following is the dispatcher for MyObjectRequestConsumer in conf
myobject-dispatcher {
type = Dispatcher
executor = "fork-join-executor"
fork-join-executor {
parallelism-min = 16
parallelism-factor = 4.0
parallelism-max = 16
}
throughput = 1
}
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
