> (the task was to limit the whole thing to about 10% of cores) I still don't think you needed a worker pool here. Like OP mentioned above, you could just limit the number of goroutines executed to 10% of total cores.
On Saturday, 28 December 2019 18:02:08 UTC+5:30, Chris Burkert wrote: > > There are Pros and Cons for everything in life. Some time ago I wrote a > database tool which does something per table where the runtime largely > depends on the table size. I started with one goroutine per table because > it was easy. But that put a high load on the database (the task was to > limit the whole thing to about 10% of cores) with out of memory situations. > So I switched to a worker pool which solved that. However now the overall > runtime was unpredictable. When the large tables were handled at the end > their runtime defined the overall runtime. So I to feed the pool with large > tables first. This again lead to out of memory situations so I reordered > the tables such that large tables are mixed with smaller tables. > > Brian Candler <b.ca...@pobox.com <javascript:>> schrieb am Sa. 28. Dez. > 2019 um 11:09: > >> On Friday, 27 December 2019 16:30:48 UTC, Bruno Albuquerque wrote: >>> >>> This might be useful too you, in any case: >>> >>> https://git.bug-br.org.br/bga/workerpool >>> >>> >> I think the point from Bryan Mills' video is, "worker pool" is something >> of an anti-pattern in go. goroutines are so cheap that you might as well >> start a goroutine for each piece of work you have to do, and let it >> terminate when that piece of work is done. >> >> Apart from the startup cost, the other reason for having a "worker pool" >> is to limit the number of concurrent tasks being executed, and there are >> better ways of doing that in go (also shown in the video). >> >> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golan...@googlegroups.com <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/f840beee-748f-42b6-809f-4c7505208aee%40googlegroups.com >> >> <https://groups.google.com/d/msgid/golang-nuts/f840beee-748f-42b6-809f-4c7505208aee%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/ecddafbc-eb73-45e1-8a5a-f738e88c6821%40googlegroups.com.
