For task submission, I have a slice of tasks. The easy way to submit them is to spin up a goroutine that writes them sequentially to a channel and then closes the channel when there are no more tasks.
On Friday, January 25, 2019 at 9:29:43 PM UTC+1, robert engels wrote: > > I think the reason you haven’t found a library, is that given the > specifications, it is probably less than 100 lines of Go code to accomplish. > > The important elements lacking in your specification though are “how are > the tasks submitted” (e.g. http? in code ?, etc.) , , and “how is the > output presented” (e.g. to the console, each task decides, etc .) > > On Jan 25, 2019, at 1:04 PM, twp...@gmail.com <javascript:> wrote: > > Hi, > > I have a number of slow tasks that I want to run concurrently across > runtime.NumCPU() workers in a single process. The tasks have a specific > input order, but they are completely independent of each other and can > execute in any order. I would like to print the output of each task in the > same order as the input order of tasks. > > This can be implemented by including each task's index in the input order > as it is distributed via a channel to the workers, and the final collection > of results assembled using these task indexes before the results are > printed. > > Assumptions: > - Small number of tasks (~10,000 max), i.e. this easily fits in memory. > - Single Go process, i.e. I don't want/need a distributed system. > > This feels like it should be common problem and there's probably either a > library or a standard Go pattern out there which can do it. My web search > skills didn't find such a library though. Do you know of one? > > Cheers, > Tom > > > Background info to avoid the XY problem <http://xyproblem.info/>: this is > to make chezmoi <https://github.com/twpayne/chezmoi> run faster. I want > to run the doctor checks > <https://github.com/twpayne/chezmoi/blob/ed27b49f9ca4cd3662e6a59908dee24b0d295b79/cmd/doctor.go#L102-L163> > > (basically os.Exec'ing a whole load of binaries to get their versions) > concurrently in the short term. In the long term I want to make chezmoi's > apply concurrent, so it runs faster too. In the first case, the order > requirement is because I want all users to see the output in the same order > so that it's easy to compare. In the second case, the order requirement > comes because I need to ensure that parent directories are in the correct > state before checking their children. > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts...@googlegroups.com <javascript:>. > For more options, visit https://groups.google.com/d/optout. > > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
