I think suggested approach should work. Any idea on how to add dynamic jobs loading (I would like it is added in one of nearest releases)? I.e. I copy new library to each server then load it with Ignite API and have new jobs available without restarting cluster.
--Yakov 2015-10-20 18:46 GMT+03:00 Igor Sapego <isap...@gridgain.com>: > Hello Guys, > > Recently I've been working on the IGNITE-1680: CPP: Implement basic API for > remote job execution https://issues.apache.org/jira/browse/IGNITE-1680. > > I would like to discuss the approach I came up with. > > In my current approach I use existing portable mechanism to serialize the > job object so we can send it to the remote node. However in C++ unlike in > Java or C# we do not have any out-of-box way to instantiate objects of type > which is unknown on the compile time. Still we have some runtime type > information provided by the portable layer (e.g. we have type id). So we > need to implement solution to deserialize and invoke a job object and > serialize result of this invocation. To put it simple we are going to need > something like this: > > bool InvokeRemoteJob(int32_t jobTypeId, > PortableReaderImpl& jobReader, > PortableWriterImpl& resultWriter); > > As we can not operate with type during runtime in C++ it is proposed to > make user to provide us with such kind of function. We can define three > simple macros so user can to provide us with the list of the jobs like > this: > > IGNITE_REMOTE_JOB_LIST_BEGIN > IGNITE_REMOTE_JOB_DECLARE(JobType1, ReturnType1) > IGNITE_REMOTE_JOB_DECLARE(JobType2, ReturnType2) > ... > IGNITE_REMOTE_JOB_DECLARE(JobTypeN, ReturnTypeN) > IGNITE_REMOTE_JOB_LIST_END > > This will expand to something like that: > > extern "C" IGNITE_IMPORT_EXPORT > bool InvokeRemoteJob(int32_t jobTypeId, > PortableReaderImpl& jobReader, > PortableWriterImpl& resultWriter) > { > if (jobTypeId == PortableType<JobType1>::GetTypeId()) > { > JobType1 job(jobReader.ReadTopObject<JobType1>()); > ReturnType1 call_res = job.call(); > resultWriter.WriteTopObject(call_res); > return true; > } > ... > return false; > } > > Every module (dynamic library or executable) that has jobs that could be > invoked remotely is going to need to have list of the remote jobs defined. > Then while we are loading modules we can lookup InvokeRemoteJob function > for every module and store them so we can invoke jobs later. > > I have implemented proof of concept for this approach and proved it > working. Now I would like to hear what do you people think about it so > please provide your comments if any. > > Thanks > > Best Regards, > Igor >
