Re: Commons sub project for parallel method execution

t; > >> sandbox, I >>> > >> > intend to find some time in the next days to try Apache Commons >>> > Javaflow >>> > >> > with this library. >>> > >> > >>> > >> > Jenkins implemented pipelines +

Re: Commons sub project for parallel method execution

rted >> > >> > it looked a lot like Javaflow. The execution in parallel is taken >> care >> > >> in >> > >> > some internal modules in Jenkins, but I would like to see how if >> > simpler >> > >> > implementation like

Re: Commons sub project for parallel method execution

ut I would like to see how if > > simpler > > >> > implementation like this one would work. > > >> > > > >> > Ideally, this utility would execute in parallel, say, 20 tasks each > > >> taking > > >> > 5 minutes (haven't looked if it supports fork/join). Then I would be >

Re: Commons sub project for parallel method execution

ints during the execution and if the whole workflow > >> fails, I > >> > would be able to restart it from the last checkpoint. > >> > > >> > > >> > I use Java7+ concurrent classes when I need to execute tasks in > parallel > >> >

Re: Commons sub project for parallel method execution

any way to have persistentable (?) > > continuation workflows as in Jenkins, but with simple Java code. > > > > Cheers > > Bruno > > > > ____ > > From: Gary Gregory <garydgreg...@gmail.com> > > To: Commons Developers

Re: Commons sub project for parallel method execution

ssage in this thread to give >> > GPars a try too!), but I am unaware of any way to have persistentable >> (?) >> > continuation workflows as in Jenkins, but with simple Java code. >> > >> > Cheers >> > Bruno >> > >> >

Re: Commons sub project for parallel method execution

nuation workflows as in Jenkins, but with simple Java code. > > Cheers > Bruno > > > From: Gary Gregory <garydgreg...@gmail.com> > To: Commons Developers List <dev@commons.apache.org> > Sent: Tuesday, 13 June 2017 2:08 PM > Subject: Re: Commons sub project

Re: Commons sub project for parallel method execution

workflows as in Jenkins, but with simple Java code. Cheers Bruno From: Gary Gregory <garydgreg...@gmail.com> To: Commons Developers List <dev@commons.apache.org> Sent: Tuesday, 13 June 2017 2:08 PM Subject: Re: Commons sub project for parallel method executi

Re: Commons sub project for parallel method execution

A lot of useful and interesting suggestions here. Usage of annotations instead of hardcoding method names is definitely a very good idea. Decorating the domain classes with annotations will be much more clean and maintainable. And I think we can check out the usage of MethodHandle's

Re: Commons sub project for parallel method execution

A java CUDA jni library would be excellent. On Jun 12, 2017 7:08 PM, "Gary Gregory" wrote: > On Mon, Jun 12, 2017 at 6:56 PM, Matt Sicker wrote: > > > So wouldn't something like ASM or Javassist or one of the zillion other > > bytecode libraries be a

Re: Commons sub project for parallel method execution

On Mon, Jun 12, 2017 at 6:56 PM, Matt Sicker wrote: > So wouldn't something like ASM or Javassist or one of the zillion other > bytecode libraries be a better alternative to using reflection for > performance? Also, using the Java 7 reflections API improvements helps > speed

Re: Commons sub project for parallel method execution

So wouldn't something like ASM or Javassist or one of the zillion other bytecode libraries be a better alternative to using reflection for performance? Also, using the Java 7 reflections API improvements helps speed things up quite a bit. On 12 June 2017 at 20:37, Paul King

Re: Commons sub project for parallel method execution

My goto library for such tasks would be GPars. It has both Java and Groovy support for most things (actors/dataflow) but less so for asynchronous task execution. It's one of the things that would be good to explore in light of Java 8. Groovy is now Apache, GPars not at this stage. So with adding

Re: Commons sub project for parallel method execution

On Mon, Jun 12, 2017 at 6:21 PM, Matt Sicker wrote: > Last time I used Spring, they had an @Async annotation you could use which > would automatically execute in an executor service (all handled via bean > proxies as usual). > Yeah, for higher-level stuff like Spring Batch,

Re: Commons sub project for parallel method execution

Last time I used Spring, they had an @Async annotation you could use which would automatically execute in an executor service (all handled via bean proxies as usual). On 12 June 2017 at 19:22, Gary Gregory wrote: > Hi All, > > I think it would be most helpful to note the

Re: Commons sub project for parallel method execution

Hi All, I think it would be most helpful to note the distinction between the parallelism aspect and the bridge to domain classes aspect (currently done with reflection in the proposed github repo.) It seems (to me) that in between the ForkJoin framework already in Java (a low-level library) and

Re: Commons sub project for parallel method execution

I'd be interested to see where this leads to. It could end up as a sort of Commons Parallel library. Besides providing an execution API, there could be plenty of support utilities that tend to be found in all the *Util(s)/*Helper classes in projects like all the ones I mentioned earlier (basically

Re: Commons sub project for parallel method execution

The upshot is that there has to be a way to do this with some custom code to at least have the ability to 'fast path' the code without reflection. Using lambdas should make this fairly syntactically unobtrusive. On Mon, Jun 12, 2017 at 4:02 PM, Arun Mohan wrote: > Yes,

Re: Commons sub project for parallel method execution

Yes, reflection is not very performant but I don't think I have any other choice since the library has to inspect the object supplied by the client at runtime to pick out the methods to be invoked using CompletableFuture. But the performance penalty paid for using reflection will be more than

Re: Commons sub project for parallel method execution

On a lower-level, if you want to use this for lower-level services (where there is no network latency for example), you will need to avoid using reflection to get the best performance. Gary On Mon, Jun 12, 2017 at 3:15 PM, Arun Mohan wrote: > Hi Gary, > > Thanks for

Re: Commons sub project for parallel method execution

Hi Gary, Thanks for your response. You have some valid and interesting points :-) Of course you are right that Spark is much more mature. Thanks for your insight. It will be interesting indeed to find out if the core parallelization engine of Spark can be isolated like you suggest. I started

Re: Commons sub project for parallel method execution

Hi Arun, Sure, and that is to be expected, Spark is more mature than a four class prototype. What I am trying to get to is that in order for the library to be useful, you will end up with more in a first release, and after a couple more releases, there will be more and more. Would Spark not have

Re: Commons sub project for parallel method execution

Hi, This project is a pure java project with no dependencies on any library outside the JDK. It can be used with any java based project or java frameworks like Spring as just a jar file. Given some objects and a couple of methods on these objects, the library can execute these methods in

Re: Commons sub project for parallel method execution

I already see a huge difference here: Spark requires a bunch of infrastructure to be set up, while this library is just a library. Similar to Kafka Streams versus Spark Streaming or Flink or Storm or Samza or the others. On 12 June 2017 at 16:28, Gary Gregory wrote: > On

Re: Commons sub project for parallel method execution

On Mon, Jun 12, 2017 at 2:26 PM, Arun Mohan wrote: > Hi All, > > Good afternoon. > > I have been working on a java generic parallel execution library which will > allow clients to execute methods in parallel irrespective of the number of > method arguments, type of

Commons sub project for parallel method execution

Hi All, Good afternoon. I have been working on a java generic parallel execution library which will allow clients to execute methods in parallel irrespective of the number of method arguments, type of method arguments, return type of the method etc. Here is the link to the source code: