Hello Ashwin -- I would very much prefer that for prototype work we not modify the runtime interface. Since we have multiple implementations of all of the “big 3” runtime layers (memory, tasking, network communication), interface changes typically have to be implemented multiple times (once per layer implementation) and the effects of interface thrash are magnified.
With respect to this one detail, it’s intentional that the tasking layer interface specifies task body functions by means of an index into the node-local function table. This frees us from having to worry about architectures and/or systems where the same function is at different virtual addresses on different nodes, for example when the OS does address space layout randomization on user executables. If you can possibly arrange for the additional information you need to be put somewhere independent of the runtime interface proper, say in a tasking-layer-specific part of the bundled args, that would be the preferable way to proceed. Then the interface as a whole and the other tasking implementations can remain stable while the ATMI-based prototype evolves toward stability itself. greg > On Jul 25, 2017, at 4:28 PM, Aji, Ashwin <[email protected]> wrote: > > Hi Michael, > > I am aware of the function table, but am struggling to retrieve the function > ID in the runtime based on lambda/proc that is passed in the Futures module. > I see that the only time that the lambda/proc is converted to its function > table ID is at the call site of the lambda/proc. But in my case, I don’t want > to actually call the lambda/proc -- I only want to pass it along to the > runtime. In essence, I want to either convert the lambda/proc directly to a C > function pointer in the runtime (ideal case), or somehow retrieve the > function table ID so that I can look up the function table for the function > pointer. > > Ideally, I would like the 'begin' statements to act like a task that returns > a Future object, so the Future object pointer would need to be added to > either the bundledArgs arguments or passed as a separate argument to > chpl_task_addToTaskList. The Future class/record may contain, at a minimum: > 1) void *retVal: memory region to store the return value. > 2) uint64_t handle: task handle that can be used to create a task > graph/chain. > 3) state: optional field to query the current running state of the task. > > I think that it will also be necessary to pass a task handle parameter to > chpl_task_addToTaskList to denote the parent task that we need to wait for > before launching the current task. (We don’t need multiple parent handles to > create a chain with async/andThen primitives). > > If I can figure out how to pass lambda/proc to the runtime properly, then I > can have a partially running prototype up and running and we can then try to > add the compiler features that you were suggesting. > > Regards, > Ashwin > > -----Original Message----- > From: Michael Ferguson [mailto:[email protected]] > Sent: Tuesday, July 25, 2017 8:16 AM > To: Aji, Ashwin <[email protected]>; [email protected] > Cc: Hamidouche, Khaled <[email protected]> > Subject: Re: [Chapel-developers] Passing Proc pointer/ID and arg list from > the Chapel module to the C runtime > > Hi - > > The existing runtime code puts task functions into a table and assigns each > an ID (a chpl_fn_int_t). I think if it were me, I'd try to arrange for the > Futures module to create task functions similarly to what happens for > 'begin'. But doing that might amount to doing a bunch of the Futures TODOs to > make them more integrated with the language/compiler. > > The code you showed has a problem in that the Chapel compiler doesn't > guarantee that tuple elements are stored consecutively in memory. (I don't > want to guarantee that about records either, so that things like Apan's SoA > <-> AoS can be legal transformations). > As a result, it might work some of the time, but other times you might not > see the arguments you expected hanging off of the pointers. > > Anyway, the shortest path here might be to add some minimum amount of extra > information/tracking to the compilation process for the tasks that the > Futures module creates. What information/arguments do you want the > chpl_task_addToTaskList call (or related thing) to use that it doesn't right > now? > -michael > > >Hi, > > > >I am trying to write a custom tasking layer in Chapel that uses our > >ATMI runtime (tasking layer on top of AMD’s ROCm runtime stack); one of > >the use cases being able to re-implement the Futures package using ATMI. > >Eventually, I want to be > > able to translate Futures.async to atmi_task_launch and translate > >Futures.andThen to atmi_task_launch with the dependent tasks as > >arguments. ATMI is better equipped to build task graphs, and hence this > >exploration. > > > > > >externproc chpl_task_launch(fn: c_fn_ptr, num_args: > >int, args_sizes: [] int(64), > >args...?numvals); > > > >proc async(taskFn, args...) { > > > >var f: Future; > > > >//get fn ptr to taskFn, which itself can be taking variable arguments > > > >//get fn args list > > f = chpl_task_launch(c_ptrTo(taskFn), n, sizes, (...args)); > > > >return f; > >} > > > >To this end, I would need to pass custom Chapel procedures and lambdas > >as {function id, arg1, arg2, …} to the ATMI C runtime layer. How can I > >use the existing Chapel-to-C interop to do the following: > >1) > >Pass a user-written proc/lambda with any number arguments. > >2) > >Pass the incoming variadic arguments as an argument region or any other > >way in which it can be retrieved by the runtime. > > > >It would have been easier if there was a C++ interoperability layer to > >Chapel, but we only have a C interoperability layer for now, so we will > >have to make do with that. Right now, I am trying to get to the above > >in a somewhat convoluted way like this: > > > >pragma"no doc" > >proc getArgs(arg) { > > > >return (arg,); > >} > > > >pragma"no doc" > >proc getArgs(arg, args...) { > > > >return (arg, (...getArgs((...args)))); > >} > > > >externproc chpl_task_launch(fn: c_fn_ptr, num_args: > >int, args_sizes: [] int(64), > >args: c_void_ptr); //args...?numvals); > > > >proc async(taskFn, args...) { > > > >var args_list = getArgs((...args)); > > > >var f = chpl_task_launch(c_ptrTo(taskFn), n, sizes, > >c_ptrTo(args_list));// (...args)); > > > >return f; > >} > > > >But we are not sure if the above is the cleanest approach or if there > >are holes in this approach. Any thoughts/suggestions? > > > > > >Regards, > >Ashwin > > > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! > http://sdm.link/slashdot_______________________________________________ > Chapel-developers mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/chapel-developers ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Chapel-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/chapel-developers
