[ 
https://issues.apache.org/jira/browse/AMQ-5486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15046777#comment-15046777
 ] 

Gary Tully commented on AMQ-5486:
---------------------------------

that looks like a sensible enhancement. my only comment would be the current 
MAX_INT default  appears to be being replaced with 10, as the default value 
when the property is referenced. I guess it would be best to retain the current 
defaults.

> Thread synchronization overhead is unexpectedly high
> ----------------------------------------------------
>
>                 Key: AMQ-5486
>                 URL: https://issues.apache.org/jira/browse/AMQ-5486
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.9.1
>         Environment: UbuntuServer 12.04 x86_64, Linux Kernel 3.20.23, OpenJDK 
> 1.7.0_51 64-Bit Server VM, Xms 2G, Xms 8G, CPU: E5-2620 v2 X 2, 64 GB RAM
>            Reporter: Benjamin  Huang
>             Fix For: NEEDS_REVIEW
>
>         Attachments: Method_Statistics.html, Monitor_History.zip, 
> Monitor_Usage_Statistics.html, monitor_usage_stats.png, 
> nio_worker_blocked_1.png, nio_worker_blocked_2.png, 
> nio_worker_blocked_frequently.png, tunable_threadpool_options.patch
>
>
> There's about 20 topics with virtual topic enabled, hundreds of 
> comsumers/producers connected to MQ on NIO transport connector. During the 
> run there're about 12000 msg flow in per second, not a very high rate, but 
> ActiveMQ consumes a lot of CPU resource (about 600%~1000%). To find out 
> what's the most CPU consuming code path, I use JProfiler to dig into the 
> process. 
> Among all the NIO worker threads, most of them were frequently blocked and 
> did a little job between the 'unblocked' time. While they're expected spend 
> most of their time slices on waiting for work item and processed them.
> !nio_worker_blocked_frequently.png!
> After reviewing the monitor usage history and stats, I think these NIO 
> workers were competing fiercely with each other on executing a synchronized 
> method (DestinationMap::get), which is also the most hot spot in the program 
> . I also notice that the caller AbstractRegion::getDestinations acquires a 
> read lock before calling it, so I guess this could be a left out, read lock 
> is the actual lock type required here.
> !monitor_usage_stats.png!
> !nio_worker_blocked_1.png!
> !nio_worker_blocked_2.png!
> It's too difficult for me to list all critical sections between NIO workers, 
> or between NIO workers and BrokerService which adds up to the overall 
> synchronize overhead. So I attach the relevant info, with the hope of finding 
> a complete solution to this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to