Taskbot has been available for almost a year now. As often happens I noticed I was writing repetitive code, and tried to abstract it. Hopefully it's useful for others too.
Using $tb:OPTIONS-SYNC doesn't avoid timeouts. It just means that the caller will wait for any results, instead of yielding empty sequence after the initial tasks are spawned. But timeouts can still happen. So as usual you have to batch the work in chunks that finish in time, or increase the timeout, or both. With a seriously long-running task you wouldn't want to count on the network connection either, so you'd probably save the results to a database. I'm not sure if turning a list function parameter into a function reference would help. It sounds like creating the list takes a long time, and doing that by calling a function would take just as long. Ideally you'd break up that list generation into tasks, too. But some problems are amenable to parallelization and some aren't. More details about the use-case might be revealing. -- Mike > On 16 Jan 2015, at 01:35 , David Lee <[email protected]> wrote: > > https://github.com/mblakele/taskbot > > Mike, why didn't you read my mind when I needed this for [insert recent > project] ? > This is really nice. > Of course I can immediately see some additions I would have needed ... > (replace the list with a function, check pointing state for restart > persistence ,...) > and a billion features I don't need but would be potentially useful ( > resource capping by querying the meters DB occasionally, cross server spawns > - may need > to serialize function items for that ... hmmm, and of course a GUI ! ) > > if only it were open source, on GitHub, had a license that allowed > commercial use without the worrying about the lawyers ... had logging and > exception support > and written by a nice person that wouldn't mind the pull requests ... > ... oh wait ... it is ! wow. > > One question before I put this on my infinite queue of jobs for my clones to > work on in parallel universes > > How does the using $tb:OPTIONS-SYNC avoid the problem of the calling task > timing out if the job takes too long ? > > That and the problem of the original list itself being too expensive to > create were my big stumbling blocks on a recent project. > ( the rest was just a PITA of repetitive work this would have eliminated) > _______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
