May or may not be your issue but make sure you don't have something installed that uses Error.prepareStackTrace and friends to grab the callsites, there's a few such as "longjohn" (or most of the long stack trace modules), and the sentry client, both of those produce really horrid cpu usage
On Monday, 9 December 2013 22:13:43 UTC-8, Aaron Boyd wrote: > > Some quick tests seemed to corroborate what Ben was saying about > epoll_wait(), but I haven't had enough time to really confirm that. > > Regarding my overall performance problems, I threw hardware at them for > now... > > Wish I could be of more help. I will at some point return to profiling my > nodejs app in more detail and will be more than happy to share my findings. > > > On Tuesday, December 3, 2013 1:33:08 AM UTC-8, Roman Podlinov wrote: >> >> Aaron, >> >> can you please share your progress? >> We have the same issue with libc and node js >> >> http://stackoverflow.com/questions/20260544/node-js-lower-performance-on-ubuntu-12-04-3-because-of-libc >> >> >> On Wednesday, October 2, 2013 10:41:09 PM UTC+4, Aaron Boyd wrote: >>> >>> Thanks Ben. We're making some progress (I've been working with Ken on >>> this). Installing binutils helped, as did slamming the CPU harder. >>> >>> We're banging around trying to get a top-down tree (tweaking >>> linux-tick-processor). >>> >>> +1 for filtering out epoll, or anything else that makes it so you don't >>> have to slam the cpu as a prerequisite to profiling. >>> >>> A general challenge has been getting confident that we're seeing the >>> whole picture (e.g., now i'm often seeing "syscall" at 47% with no break >>> down) so we don't waste time optimizing a small bit. >>> >>> -aaron >>> >>> >>> >>> >>> On Tuesday, October 1, 2013 9:29:30 PM UTC-7, Ben Noordhuis wrote: >>>> >>>> On Wed, Oct 2, 2013 at 5:34 AM, Ben Noordhuis <[email protected]> >>>> wrote: >>>> > On Wed, Oct 2, 2013 at 12:48 AM, Kenneth Gunn <[email protected]> >>>> wrote: >>>> >> Hi! >>>> >> >>>> >> >>>> >> My team is developing a service in node. We are experiencing high >>>> CPU >>>> >> utilization and are attempting to profile, but are having a hard >>>> time >>>> >> getting a sufficient picture of what’s going on. We have experience >>>> >> profiling in various other environments, but this is our first crack >>>> at >>>> >> node. >>>> >> >>>> >> >>>> >> We've tried a few different tools (including nodetime.com, which >>>> has been >>>> >> useful for some things), and have spent most of our time with the v8 >>>> >> profiler. The main problem is that our viewable results only cover a >>>> small >>>> >> portion of the program runtime. More than 80% of the time is spent >>>> in >>>> >> libc.so, and that time isn't rolled up by function or caller in the >>>> node >>>> >> program. Also, the C++ section, which I would expect to contain >>>> events in >>>> >> the v8 interpreter itself, is empty. (Below, I'm including an >>>> abbreviated >>>> >> output from the v8 tick processor.) >>>> > >>>> > You need to have the binutils package installed. The tick processor >>>> > uses `nm` to map addresses to symbol. >>>> > >>>> > Small nomenclature nit: V8 is a just-in-time compiler, not an >>>> interpreter. >>>> > >>>> >> We're aware that the v8 profiling output changes frequently, and >>>> we've >>>> >> managed to figure out how to get the right tick processor version >>>> that >>>> >> corresponds to the node version we are using. (Our steps are here: >>>> >> https://gist.github.com/kennethgunn/6770664 ) We've seen very >>>> similar >>>> >> results with versions of node ranging from v0.8.9 to v0.10.18. >>>> >> >>>> >> >>>> >> Is libc actually responsible for 80+% of the CPU time? If so, how >>>> do we >>>> >> roll that up to the the higher level code leading to those calls? >>>> Does it >>>> >> sound like we're missing something here, or is there another set of >>>> tools we >>>> >> should consider using? Your help is greatly appreciated! >>>> > >>>> > That's probably node.js sleeping in the epoll_wait() system call. >>>> > Future versions of node.js will filter out such ticks but right now >>>> > that's not possible, you have to keep your application busy when >>>> > profiling. >>>> >>>> Forgot to mention, you can get a reasonable approximation of non-idle >>>> time by passing -j or --js to the tick processor. That filters out >>>> samples that aren't accountable to JS land. >>>> >>> -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
