Hi Hui -- The fifo tasking layer uses the pthreads threading layer, and that in turn uses pthreads_mutex_lock(). It's here: runtime/src/threads/pthreads/*
greg On Wed, 21 Oct 2015, Hui Zhang wrote:
Hi, Greg Thanks ! I'm currently using CHPL_TASKS=fifo, but I grep around and don't see any pthread-mutex_lock() calls under runtime/src/tasks/fifo....all that calls come from the massivethreads folder, so what do you mean by fifo tasking uses pthread-mutex-lock() quite a bit ? sorry I'm not familiar with tasking layer yet... thanks! On Wed, Oct 21, 2015 at 11:11 AM, Greg Titus <[email protected]> wrote: Hello Hui -- I went and read the README to better understand the question, since it's actually been quite a while since I used the gperftools CPU profiler. According to the README, the problem has to do with pthread_mutex_lock() that happen to be in progress when the CPU profiler signal fires. I would put the ProfilerStart before the forall-stmt and the ProfilerStop after its body. The rest of the discussion will be a bit technical, but there's no avoiding that. Since you're going to try to draw conclusions about Chapel task behavior by looking at profiling data from the pthreads that host the tasks, you have to understand something about how that hosting works in order to be successful. The issues vary depending on which tasking layer implementation you are using. Assuming you're using CHPL_TASKS=qthreads, the default, all the pthreads are created at program startup time. I don't know if the qthreads library uses pthread_mutex_lock() calls. I suspect that it does, but perhaps only under certain circumstances. My guess is that you won't encounter profiler crashes with CHPL_TASKS=qthreads. One difficulty you might run into with CHPL_TASKS=qthreads is that it does lightweight Chapel task switching in user space, that is, it can run more than one Chapel task per pthread (time-multiplexed, basically). If this happens it will confuse your results. If you want to profile the Chapel tasks then you need to make sure you don't run more Chapel tasks in a parallel construct than there are Qthreads worker pthreads to run them on. By default this will be true, so you should be okay. But don't set dataParTasksPerLocale greater than its default value, or you'll get confusing results. (When I used the gperftools profiler I was doing internal studies of a different tasking layer, which behaved more or less like qthreads in that it did lightweight task switching at user level. But I was using the profiler to study the tasking layer itself, not the Chapel tasks it was running.) If you're using CHPL_TASKS=fifo, the pthreads are created as needed. Specifically, they are created whenever we have tasks queued to run and no threads to run them on. But once created, they stick around, so if you run one parallel construct and then another over the same iterator, typically the second parallel construct will not create more pthreads. Fifo tasking uses pthread-mutex_lock() quite a bit, so you might have trouble with this one. Hope this helps! greg On Tue, 20 Oct 2015, Hui Zhang wrote: Hello, Greg I'm not sure if I complete follow your explanation. So if I have the following block of code, where would you put ProfilerStart and ProfilerStop ? var arrayA: [1..numMessages] real; forall msg in 1..numMessages do { arrayA[msg] = 0.1; for i in 1..100000 { CPUheavy(arrayA[msg]); CPUlight(arrayA[msg]); } writeln("Hello, world! (from iteration ", msg, " of ", numMessages, ") has arrayA[msg] ",arrayA[msg]); } the thread creating calls should appear within the red line, but if I want to see which function is spending most time among all the threads, then shall I put ProfStart and ProfStop outside or inside the red line ? Thanks On Tue, Oct 20, 2015 at 6:26 PM, Greg Titus <[email protected]> wrote: I haven't had trouble with it crashing. This might be because I always put the start/stop calls in serial code, before or after any parallel constructs but not within them. Since parallel constructs other than begin-stmts have implicit syncs afterward, and all the runtime tasking layers only create threads (if they have to at all) in order to execute parallel constructs, this means that the start/stop calls and the thread creation calls are well separated. That's just a hypothesis, but personally I don't recall having any trouble with gperftools crashing. greg On Tue, 20 Oct 2015, Hui Zhang wrote: Ah... sorry I missed that. It works now , thanks ! One more question: from gperftools' README, it seems like it sometimes crashes when threads get created, and it suggested to avoid putting ProfilerStart and ProfilerStop among the places where new threads get forked, but in this way, how can I profile the multi-threading feature in Chapel ? How did you normally use it to profile a Chapel code with forall/coforall clauses ? Thanks ! On Tue, Oct 20, 2015 at 6:06 PM, Greg Titus <[email protected]> wrote: I think it's complaining about ProfilerStop(), perhaps that you need: void ProfilerStop(void); to make it a complete prototype by saying explicitly that it takes no arguments? greg On Tue, 20 Oct 2015, Hui Zhang wrote: Hello, Greg So I create a prof-decls.h and put these two lines in it: int ProfilerStart(const char* fname); void ProfilerStop(); which I get from the gperftools/profiler.h chpl -g myCode.chpl prof-decls.h -I../include/ -o myCode -L../lib -ltest it reports less errors but still this one: In file included from /chpl__header.h:5:0, from /_main.c:1: ./prof-decls.h:2:1: error: function declaration isn’t a prototype [-Werror=strict- prototypes] why the ProfilerStart declaration isn't the function prototype ? Thanks On Tue, Oct 20, 2015 at 5:07 PM, Greg Titus <[email protected]> wrote: Hello -- I believe you can solve this problem by putting C-style declarations for ProfilerStart() and ProfilerStop() in a .h file (prof-decls.h, say) and then adding that .h file to the chpl command line: chpl -g myCode.chpl prof-decls.h -I../include/ -o myCode -L../lib -ltest (Sorry I didn't yet respond to your earlier message. I needed time to dig into the cause of the problem, but had not found that time yet.) greg On Tue, 20 Oct 2015, Hui Zhang wrote: Hello, I'm trying to use gperftools to profile the Chapel code, but simply link the profiler library doesn't produce the profile file, so I need to explicitly call two external C functions within my Chapel code, here's what I did: 1. in my chapel module: extern proc ProfilerStart(name: c_string); **it suggested me to use c_string instead of string exetern proc ProfilerStop(); then in proc main, I called this two functions 2. recompile my chapel program: chpl -g myCode.chpl -I../include/ -o myCode -L../lib -ltest But I got this error : /_main.c:30:0: /Hui.c: In function ‘chpl_user_main’: /Hui.c:160:1: error: implicit declaration of function ‘ProfilerStart’ [-Werror=implicit-function-declaration] /Hui.c:160:1: error: nested extern declaration of ‘ProfilerStart’ [-Werror=nested-externs] /Hui.c:163:1: error: implicit declaration of function ‘ProfilerStop’ [-Werror=implicit-function-declaration] /Hui.c:163:1: error: nested extern declaration of ‘ProfilerStop’ [-Werror=nested-externs] cc1: all warnings being treated as errors make: *** [/tmp/chpl-hzhang86-15966.deleteme/experiment.work.funh.tmp] Error 1 error: compiling generated source [mysystem.cpp:43] Should I just ignore these warnings ? but how to disable this warning as error option? Or am I doing it wrong ? Thanks -- Best regards Hui Zhang -- Best regards Hui Zhang -- Best regards Hui Zhang -- Best regards Hui Zhang -- Best regards Hui Zhang
------------------------------------------------------------------------------
_______________________________________________ Chapel-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/chapel-developers
