[Lift] Re: Nested 'react'

2009-10-19 Thread Philipp 

 so when I'm running load test I see that number of threads that used
 by dispatcher is growing (so far up to 200) and performance is going
 down. Most of them are always in blocked state
 Dispatcher's Actor queue never should be more that few messages as I'm
 using QoS = 5 on channel and sending ack only when worker picks up
 message, also I have just a few workers
 When I'm looking to heap, I can see that more than 80% (up to 95%) are
 scala.actors.FJTaskRunner$VolatileTaskRef objects and number of
 those is quickly growing (have 1'000'000 right now).

 What can be a cause of spawning so many threads and large number of
 'scala.actors.FJTaskRunner$VolatileTaskRef' objects (GC doesn't kill
 them)?

The pool resizing logic was creating new threads too eagerly. This has
been fixed in Scala 2.7.7.RC1 (it also replaces `FJTaskRunner` with
`java.util.concurrent.ThreadPoolExecutor`).

Philipp

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Lift group.
To post to this group, send email to liftweb@googlegroups.com
To unsubscribe from this group, send email to 
liftweb+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/liftweb?hl=en
-~--~~~~--~~--~--~---

[Lift] Re: Nested 'react'

2009-10-02 Thread David Pollak 
On Mon, Sep 28, 2009 at 1:57 PM, ph pkirsa...@38studios.com wrote:


 I don't know if this group is right place to post it, but I couldn't
 find any general Scala or Scala/Actor groups, and this is most active
 Scala group..

 I have server that consumes AMQP messages, handles messages in
 dedicated workers(Actors). So design is this:

 1. One Dispatcher which is event-based actor (nested reacts)
   a. First it listens for event from worker that worker is ready
   b. when worker is ready it uses nested react to get AMQP message
 and forward it to worker
 react {
   case WorkerReady(worker:Actor) = react {
  case m...@message = worker ! msg; channelHandler ! 'ack; act
   }
 }
 2. Few workers that are thread-based actors (sends 'WorkerReady' to
 dispatcher)
 3. RabbitMQ Java lib, handleDelivery (baseConsume) in Java lib
 Connection thread (sends 'message' to dispatcher)
 4. There is also one per dispatcher thread-based actor that handles
 AMQP channel, but it's not important here (channelHandler ! 'ack)

 so when I'm running load test I see that number of threads that used
 by dispatcher is growing (so far up to 200) and performance is going
 down. Most of them are always in blocked state
 Dispatcher's Actor queue never should be more that few messages as I'm
 using QoS = 5 on channel and sending ack only when worker picks up
 message, also I have just a few workers
 When I'm looking to heap, I can see that more than 80% (up to 95%) are
 scala.actors.FJTaskRunner$VolatileTaskRef objects and number of
 those is quickly growing (have 1'000'000 right now).

 What can be a cause of spawning so many threads and large number of
 'scala.actors.FJTaskRunner$VolatileTaskRef' objects (GC doesn't kill
 them)? And is there a way to limit size of thread pool used by even
 Actors? Is there a way to monitor size of Actor queue?


Please see this thread for more information on tuning Actors
http://groups.google.com/group/liftweb/browse_thread/thread/b3783e24b8417521/f89548ba1fa70319?hl=enlnk=gstq=oome#f89548ba1fa70319

You might consider using akkasource.org Actors.



 



-- 
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Lift group.
To post to this group, send email to liftweb@googlegroups.com
To unsubscribe from this group, send email to 
liftweb+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/liftweb?hl=en
-~--~~~~--~~--~--~---

[Lift] Re: Nested 'react'

2009-10-02 Thread Jonas Bonér 

FWIW, here is the documentation for Akka's AMQP module:
http://wiki.github.com/jboner/akka/reference-amqp

Let me know if you feel that you would like to have things behave
differently or is missing some stuff in the API (it's currently
work-in-progress).

/Jonas

2009/10/2 David Pollak feeder.of.the.be...@gmail.com:


 On Mon, Sep 28, 2009 at 1:57 PM, ph pkirsa...@38studios.com wrote:

 I don't know if this group is right place to post it, but I couldn't
 find any general Scala or Scala/Actor groups, and this is most active
 Scala group..

 I have server that consumes AMQP messages, handles messages in
 dedicated workers(Actors). So design is this:

 1. One Dispatcher which is event-based actor (nested reacts)
   a. First it listens for event from worker that worker is ready
   b. when worker is ready it uses nested react to get AMQP message
 and forward it to worker
 react {
   case WorkerReady(worker:Actor) = react {
      case m...@message = worker ! msg; channelHandler ! 'ack; act
   }
 }
 2. Few workers that are thread-based actors (sends 'WorkerReady' to
 dispatcher)
 3. RabbitMQ Java lib, handleDelivery (baseConsume) in Java lib
 Connection thread (sends 'message' to dispatcher)
 4. There is also one per dispatcher thread-based actor that handles
 AMQP channel, but it's not important here (channelHandler ! 'ack)

 so when I'm running load test I see that number of threads that used
 by dispatcher is growing (so far up to 200) and performance is going
 down. Most of them are always in blocked state
 Dispatcher's Actor queue never should be more that few messages as I'm
 using QoS = 5 on channel and sending ack only when worker picks up
 message, also I have just a few workers
 When I'm looking to heap, I can see that more than 80% (up to 95%) are
 scala.actors.FJTaskRunner$VolatileTaskRef objects and number of
 those is quickly growing (have 1'000'000 right now).

 What can be a cause of spawning so many threads and large number of
 'scala.actors.FJTaskRunner$VolatileTaskRef' objects (GC doesn't kill
 them)? And is there a way to limit size of thread pool used by even
 Actors? Is there a way to monitor size of Actor queue?

 Please see this thread for more information on tuning Actors
 http://groups.google.com/group/liftweb/browse_thread/thread/b3783e24b8417521/f89548ba1fa70319?hl=enlnk=gstq=oome#f89548ba1fa70319

 You might consider using akkasource.org Actors.





 --
 Lift, the simply functional web framework http://liftweb.net
 Beginning Scala http://www.apress.com/book/view/1430219890
 Follow me: http://twitter.com/dpp
 Surf the harmonics

 




-- 
Jonas Bonér

twitter: @jboner
blog:http://jonasboner.com
work:   http://crisp.se
work:   http://scalablesolutions.se
code:   http://github.com/jboner
code:   http://akkasource.org

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Lift group.
To post to this group, send email to liftweb@googlegroups.com
To unsubscribe from this group, send email to 
liftweb+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/liftweb?hl=en
-~--~~~~--~~--~--~---