npeltier commented on issue #4: SLING-7707 Create Executor Pipe URL: https://github.com/apache/sling-org-apache-sling-pipes/pull/4#issuecomment-396522714 thanks a lot for this @spopescu ! Some remarks though, as discussed offline: - i'll push soon a superpipe abstract class soon that should get extended for any pipe which output is depending on subpipe(s). I'd extend anyway this pipe from container pipe, as it should not differ from it, and thus we could reuse the same code for initialisation e.g. - we can't assume executor pipe's sub pipes are all read only, and a JCR session is not thread safe, hence the implementation unfortunately has to be slightly different: you'd need to clone the resolver for each subpipe, and use plumber execute for each one of it. super pipe's output would compute the resultoutputs with its own resolver. This is bad news for your code, but good news as well, as you don't have to handle concurrency at your level, the repo will do that for you, - i'd add a write test, you can reuse existing one, just referring them from your pipe, - talking of that, pipebuilder like this is not usable. I'd rather propose an alternative approach to ```run``` like ```runParallel(numThreads)``` (for lack of a better name) that builds up your pipe instead of the container. Then you could do something like ```.ref("pipe0").ref("pipe1").ref("pipe2").ref("pipe4).runParallel(2)``` - minor remark: 10 threads as a default is a lot considering writing pipes, i'd rather set it to 3 or 4 from my experience of CPU time (even if my laptop is not the same than a prod cpu)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
