Re: [Interest] [External] Re: How to properly terminate QtConcurrent run
Den tors 18 aug. 2022 16:44Sean Murphy via Interest skrev: > > > I'll look into this idea, but some of what my processChunk() does are > > > calls into an external library that I can't control so my options are > > > limited there. As far as I can tell, this is an issue for any code > that just takes > > a long time. > > > > Yes, if this it's a call into an external API that provides no mechanism > to cancel > > the call prematurely, you have no option for graceful shutdown and have > to > > kill the threads forcefully. > > > > I guess in the code handling your application exit, you can wait a > little bit for > > all threads in the pool, and after an "unreasonable" > > amount of time has passed (half a second?), kill them all forcefully. > > Bit of a big hammer, but I don't know of any way to query for the threads > > currently used by a specific ::map call (maybe someone else does though). > > Yeah, and that seems to be the rub with the QtConcurrent functions, I don't > get any sort of handle to the threads it is spawning off, unless maybe I > can get > at them via the global thread pool? Yes, it uses the global threadpool by default I think. I guess the problem with eg an API to get the list of threads used by a ::map call is that it would be inherently racy - by the time you kill a thread it could have been put back into and out of the pool already and be running something you don't want to kill? So maybe better with a custom pool. Elvis I'm just starting to look at that. Other than > trying to call cancel() on the associated QFutureWatcher, which does work > eventually as long as your function called by map() actually returns, > there doesn't > seem to be a way to stop those concurrent threads. > > With the way my code exists at the moment, I've got the main application > running > in the UI thread. I intentionally spawn off Thread A and move my > processingManager > object over to that thread, and when that object gets a processing > request, it calls > QtConcurrent::map() which spawns off Threads B, C, D, etc... and I don't > seem to have > any control over those threads. I know when I had the infinite loop bug in > my > processChunk() call, killing off Thread A (the processingManager's thread) > and deleting > the processingManager itself didn't seem to forcefully terminate Threads B > through > whatever - they just kept on running forever. I'm going to try writing up > a small test app to > double check this. I was doing all this in the main application, which has > a whole lot of > other stuff going on, so maybe my issue was somewhere else... > > Sean > This e-mail, including any attached files, may contain confidential > information, privileged information and/or trade secrets for the sole use > of the intended recipient. Any review, use, distribution, or disclosure by > others is strictly prohibited. If you are not the intended recipient (or > authorized to receive information for the intended recipient), please > contact the sender by reply e-mail and delete all copies of this message. > ___ > Interest mailing list > Interest@qt-project.org > https://lists.qt-project.org/listinfo/interest > ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] [External] Re: How to properly terminate QtConcurrent run
> > I'll look into this idea, but some of what my processChunk() does are > > calls into an external library that I can't control so my options are > > limited there. As far as I can tell, this is an issue for any code that > > just takes > a long time. > > Yes, if this it's a call into an external API that provides no mechanism to > cancel > the call prematurely, you have no option for graceful shutdown and have to > kill the threads forcefully. > > I guess in the code handling your application exit, you can wait a little bit > for > all threads in the pool, and after an "unreasonable" > amount of time has passed (half a second?), kill them all forcefully. > Bit of a big hammer, but I don't know of any way to query for the threads > currently used by a specific ::map call (maybe someone else does though). Yeah, and that seems to be the rub with the QtConcurrent functions, I don't get any sort of handle to the threads it is spawning off, unless maybe I can get at them via the global thread pool? I'm just starting to look at that. Other than trying to call cancel() on the associated QFutureWatcher, which does work eventually as long as your function called by map() actually returns, there doesn't seem to be a way to stop those concurrent threads. With the way my code exists at the moment, I've got the main application running in the UI thread. I intentionally spawn off Thread A and move my processingManager object over to that thread, and when that object gets a processing request, it calls QtConcurrent::map() which spawns off Threads B, C, D, etc... and I don't seem to have any control over those threads. I know when I had the infinite loop bug in my processChunk() call, killing off Thread A (the processingManager's thread) and deleting the processingManager itself didn't seem to forcefully terminate Threads B through whatever - they just kept on running forever. I'm going to try writing up a small test app to double check this. I was doing all this in the main application, which has a whole lot of other stuff going on, so maybe my issue was somewhere else... Sean This e-mail, including any attached files, may contain confidential information, privileged information and/or trade secrets for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message. ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
Re: [Interest] [External] Re: How to properly terminate QtConcurrent run
Den ons 17 aug. 2022 kl 17:12 skrev Sean Murphy via Interest : > > Thanks for the response! > > > You should signal processChunk to stop processing early. You can for > > example do this through an atomic variable which is periodically checked, > > e.g. > > as part of the stop condition in that loop you mentioned. > > I'll look into this idea, but some of what my processChunk() does are calls > into an > external library that I can't control so my options are limited there. As far > as I can > tell, this is an issue for any code that just takes a long time. Yes, if this it's a call into an external API that provides no mechanism to cancel the call prematurely, you have no option for graceful shutdown and have to kill the threads forcefully. I guess in the code handling your application exit, you can wait a little bit for all threads in the pool, and after an "unreasonable" amount of time has passed (half a second?), kill them all forcefully. Bit of a big hammer, but I don't know of any way to query for the threads currently used by a specific ::map call (maybe someone else does though). Elvis > > Take this hypothetical scenario for example: > - we have some processing task that would take hours to run if run in a > single thread, > but it can be broken down into chunks > - once broken into chunks, each chunk takes maybe a minute to run, but most > of > that minute is consumed in an external library that I can't do anything > about > - user starts one of these tasks, but before it finishes they decide they > don't care > about the results and want to completely quit the application > - I warn the user there's a job in progress, are they sure they want to > quit? They say Yes > > So at this point, the user really wants to quit the application. From what > I've gleaned so > far, and I could be wrong so feel free to correct me, I can tell the > QFutureWatcher to > cancel the job. But this only prevents any chunks that hadn't been started > from getting > started, it doesn't immediately terminate the chunks that are currently in > progress, > those continue until they all finish, at which point the QFutureWatcher > signals both > canceled() and finished(), and now I can safely quit the application. But if > it were me as > the user, I'd be annoyed that I have to wait up to a minute for the > application to quit > after I asked it to quit. > > > I may be wrong, but I think terminating a thread forcefully is considered a > > poor design. > > In general I agree, but I'm not sure I do in the special case of the user > quitting the application > under the conditions I've described above. At that point to me (this is me > wearing my user hat > now, not my developer hat), I would want the application to quit immediately > and I don't really > care what happens under the hood - as long as it doesn't crash. > > (Developer hat back on) I don't see a way to easily accomplish this under the > QtConcurrent > API. Most of the examples I can find online show how to do the set up part, > but they rarely > give any time to showing how to cleanly abort a run in progress. The only > actual solution I > see that can do this cleanly is to ditch QtConcurrent, and write my own > chunkProcessor > objects that spawn and manage their own worker threads, which when signaled, > can forcefully > terminate regardless of what is being done at the time. > > I'm open to suggestions/corrections/etc. Also, if anyone has any references > they like that talk > more about quitting/cleaning up threads under all sorts of different > scenarios, I'd love to read > more about it - I seem to be struggling to find those. > > Sean > This e-mail, including any attached files, may contain confidential > information, privileged information and/or trade secrets for the sole use of > the intended recipient. Any review, use, distribution, or disclosure by > others is strictly prohibited. If you are not the intended recipient (or > authorized to receive information for the intended recipient), please contact > the sender by reply e-mail and delete all copies of this message. > ___ > Interest mailing list > Interest@qt-project.org > https://lists.qt-project.org/listinfo/interest ___ Interest mailing list Interest@qt-project.org https://lists.qt-project.org/listinfo/interest
