Some quick tests seemed to corroborate what Ben was saying about epoll_wait(), but I haven't had enough time to really confirm that.
Regarding my overall performance problems, I threw hardware at them for now... Wish I could be of more help. I will at some point return to profiling my nodejs app in more detail and will be more than happy to share my findings. On Tuesday, December 3, 2013 1:33:08 AM UTC-8, Roman Podlinov wrote: > > Aaron, > > can you please share your progress? > We have the same issue with libc and node js > > http://stackoverflow.com/questions/20260544/node-js-lower-performance-on-ubuntu-12-04-3-because-of-libc > > > On Wednesday, October 2, 2013 10:41:09 PM UTC+4, Aaron Boyd wrote: >> >> Thanks Ben. We're making some progress (I've been working with Ken on >> this). Installing binutils helped, as did slamming the CPU harder. >> >> We're banging around trying to get a top-down tree (tweaking >> linux-tick-processor). >> >> +1 for filtering out epoll, or anything else that makes it so you don't >> have to slam the cpu as a prerequisite to profiling. >> >> A general challenge has been getting confident that we're seeing the >> whole picture (e.g., now i'm often seeing "syscall" at 47% with no break >> down) so we don't waste time optimizing a small bit. >> >> -aaron >> >> >> >> >> On Tuesday, October 1, 2013 9:29:30 PM UTC-7, Ben Noordhuis wrote: >>> >>> On Wed, Oct 2, 2013 at 5:34 AM, Ben Noordhuis <[email protected]> >>> wrote: >>> > On Wed, Oct 2, 2013 at 12:48 AM, Kenneth Gunn <[email protected]> >>> wrote: >>> >> Hi! >>> >> >>> >> >>> >> My team is developing a service in node. We are experiencing high CPU >>> >> utilization and are attempting to profile, but are having a hard time >>> >> getting a sufficient picture of what’s going on. We have experience >>> >> profiling in various other environments, but this is our first crack >>> at >>> >> node. >>> >> >>> >> >>> >> We've tried a few different tools (including nodetime.com, which has >>> been >>> >> useful for some things), and have spent most of our time with the v8 >>> >> profiler. The main problem is that our viewable results only cover a >>> small >>> >> portion of the program runtime. More than 80% of the time is spent in >>> >> libc.so, and that time isn't rolled up by function or caller in the >>> node >>> >> program. Also, the C++ section, which I would expect to contain >>> events in >>> >> the v8 interpreter itself, is empty. (Below, I'm including an >>> abbreviated >>> >> output from the v8 tick processor.) >>> > >>> > You need to have the binutils package installed. The tick processor >>> > uses `nm` to map addresses to symbol. >>> > >>> > Small nomenclature nit: V8 is a just-in-time compiler, not an >>> interpreter. >>> > >>> >> We're aware that the v8 profiling output changes frequently, and >>> we've >>> >> managed to figure out how to get the right tick processor version >>> that >>> >> corresponds to the node version we are using. (Our steps are here: >>> >> https://gist.github.com/kennethgunn/6770664 ) We've seen very >>> similar >>> >> results with versions of node ranging from v0.8.9 to v0.10.18. >>> >> >>> >> >>> >> Is libc actually responsible for 80+% of the CPU time? If so, how do >>> we >>> >> roll that up to the the higher level code leading to those calls? >>> Does it >>> >> sound like we're missing something here, or is there another set of >>> tools we >>> >> should consider using? Your help is greatly appreciated! >>> > >>> > That's probably node.js sleeping in the epoll_wait() system call. >>> > Future versions of node.js will filter out such ticks but right now >>> > that's not possible, you have to keep your application busy when >>> > profiling. >>> >>> Forgot to mention, you can get a reasonable approximation of non-idle >>> time by passing -j or --js to the tick processor. That filters out >>> samples that aren't accountable to JS land. >>> >> -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
