Hi Nick, I have just checked all the links you posted, they are indeed very interesting and very efficient. However, I think those are very complicate in terms of installation and setup, and I still see a lot of usages for a multi-process scheduler.
2016-08-28 20:32 GMT-07:00 Nick Coghlan <ncogh...@gmail.com>: > On 29 August 2016 at 11:50, Thales filizola costa <thale...@gmail.com> > wrote: > > What do you guys think? How to improve it? Is it relevant enough to be > > incorporated to std python ? > > There are actually quite a few distributed schedulers out there (which > can expand beyond a single machine), but "python multiprocess > scheduler" isn't likely to bring them up in a web search (as when > you're limited to a single machine, multiprocessing.Pool or > concurrent.futures.ProcessPoolExecutor is generally already good > enough). > > At a Python level, Celery is probably the most popular option for > that: http://www.celeryproject.org/ > > Another well-established option is Kamaelia: http://www.kamaelia.org/Home. > html > > Dask is a more recent alternative specifically focused on > computational tasks: http://dask.pydata.org/en/latest/ > > Once you move outside Python specific tooling, there are even more > language independent options to play with, including the likes of > Mesos and Kubernetes. > > Cheers, > Nick. > > P.S. It's a fairly sad indictment of our industry that people think > this is a sensible question to ask in developer interviews - the > correct answer from a business efficiency perspective is "I wouldn't, > I would use an existing open source task scheduler rather than > inventing my own", just as the correct answer to "How would you > implement a sort algorithm?" from that perspective is "I wouldn't, as > the Python standard library's sorting implementation is vastly > superior to anything I could come up with in 5 minutes on a > whiteboard, and the native sorting capabilities of databases are also > excellent". Reimplementing existing software from first principles is > a great learning exercise, but it's not particularly relevant to the > task of day-to-day software construction in most organisations. > > (Alternatively, if the answer the interviewer is looking for is "I > wouldn't, I would use...", then it may be an unfair "Gotcha!" > question, and those aren't cool either, since they expect the > interviewee to be able to read the interviewer's mind, rather than > just answering the specific question they were asked) > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia >
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
