[dpdk-dev] [dpdk-dev, 1/3] rte_interrupts: add rte_eal_intr_exit to shut down IRQ thread
Hi Thomas, Base on the previous conversation, at least it requires v2 to reword some comments. > > >> 2. I propose to add addition comments on rte_epoll_wait() API > > >> description. For any signal, it causes an error return, user needs > > >> to handle. > > > Agreed. In addition, one conversion is not close. > > >> the default termination handler > > > I am not so experienced with this "default termination handler". Can > someone > > > clarify what it is so I could comment better about it? > > For example, you're handling SIGINT. After finishing your necessary app > > cleanup, then 'signal(SIGINT, SIG_DFL); raise(SIGINT);'. > > The default signal handler can terminate the interrupt thread. > > >> 1. Can you explain and add patch comments why default signal handler > > >> is not good enough to terminate app. > > > Yes if someone call tell me more about what it is so I can check it. Thanks, Cunming > -Original Message- > From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com] > Sent: Saturday, July 09, 2016 1:36 AM > To: Liang, Cunming > Cc: Matthew Hall ; dev at dpdk.org > Subject: Re: [dpdk-dev,1/3] rte_interrupts: add rte_eal_intr_exit to shut down > IRQ thread > > Cunming, what is the status of this patchset, please? > > 2016-03-23 11:24, Liang, Cunming: > > Hi Mattew, > > > > Thank you for your time. > > > > On 3/22/2016 3:39 PM, Matthew Hall wrote: > > > On Mon, Mar 21, 2016 at 03:58:44PM +0800, Liang, Cunming wrote: > > >> the default termination handler > > > I am not so experienced with this "default termination handler". Can > someone > > > clarify what it is so I could comment better about it? > > For example, you're handling SIGINT. After finishing your necessary app > > cleanup, then 'signal(SIGINT, SIG_DFL); raise(SIGINT);'. > > The default signal handler can terminate the interrupt thread. > > > > > > > >> If EINTR is caused by some non-term purpose signals, are you going > > >> to exit the interrupt thread any way? > > > We should discuss what makes sense here. I'm just trying to get some > > > things > > > working and finding EINTR was getting eaten and causing infinite looping. > > SIGINT/SIGTERM causes EINTR return, while SIGUSR1 also can cause the > > EINTR return. For the dedicated EAL interrupt thread, it won't be > > expected to exit for all kinds of the cause. > > On this view, I'm in favor of your patch which cancel the interrupt > > thread, but don't directly return by the EINTR. > > > > > > > >> Without setting 'PTHREAD_CREATE_DETACHED' won't cause the infinite > > >> loop. However by using pthread_cancel to terminate the thread, > > >> indeed it's necessary to set 'PTHREAD_CREATE_DETACHED'. > > > My general understanding is that PTHREAD_CREATE_DETACHED should be > used for > > > any thread, which should not keep a process open by itself if it is > > > executing, > > > i.e. a "daemon thread". I believe the interrupt thread qualifies as such a > > > thread if I have understood everything right (which is hard to promise > > > when > > > you only work in DPDK in spare time). > > > > > >> It looks like 'pthread_cancel' is the right way and I saw it > > >> continue keeps current EINTR handling in EAL interrupt thread. > > > It is one option. Depending what makes the most sense. > > > > > >> 1. Can you explain and add patch comments why default signal handler > > >> is not good enough to terminate app. > > > Yes if someone call tell me more about what it is so I can check it. > > > > > >> 2. I propose to add addition comments on rte_epoll_wait() API > > >> description. For any signal, it causes an error return, user needs > > >> to handle. > > > Agreed. > > > > > >> 3. Will you do a favorite to add 'PTHREAD_CREATE_DETACHED' to all > > >> EAL pthread too. > > > As a spare time developer I am a bit conservative about too large of a > > > scope > > > and messing with code for other threads or features I didn't personally > > > use or > > > test. This is because I don't have the same QA resources as Intel / 6WIND > > > / > > > etc.. Some help from a full time developer would be great here. > > All right, reasonable to me. > > > > > > > >> Cunming > > > Matthew. > > >
[dpdk-dev] [dpdk-dev, 1/3] rte_interrupts: add rte_eal_intr_exit to shut down IRQ thread
Cunming, what is the status of this patchset, please? 2016-03-23 11:24, Liang, Cunming: > Hi Mattew, > > Thank you for your time. > > On 3/22/2016 3:39 PM, Matthew Hall wrote: > > On Mon, Mar 21, 2016 at 03:58:44PM +0800, Liang, Cunming wrote: > >> the default termination handler > > I am not so experienced with this "default termination handler". Can someone > > clarify what it is so I could comment better about it? > For example, you're handling SIGINT. After finishing your necessary app > cleanup, then 'signal(SIGINT, SIG_DFL); raise(SIGINT);'. > The default signal handler can terminate the interrupt thread. > > > > >> If EINTR is caused by some non-term purpose signals, are you going > >> to exit the interrupt thread any way? > > We should discuss what makes sense here. I'm just trying to get some things > > working and finding EINTR was getting eaten and causing infinite looping. > SIGINT/SIGTERM causes EINTR return, while SIGUSR1 also can cause the > EINTR return. For the dedicated EAL interrupt thread, it won't be > expected to exit for all kinds of the cause. > On this view, I'm in favor of your patch which cancel the interrupt > thread, but don't directly return by the EINTR. > > > > >> Without setting 'PTHREAD_CREATE_DETACHED' won't cause the infinite > >> loop. However by using pthread_cancel to terminate the thread, > >> indeed it's necessary to set 'PTHREAD_CREATE_DETACHED'. > > My general understanding is that PTHREAD_CREATE_DETACHED should be used for > > any thread, which should not keep a process open by itself if it is > > executing, > > i.e. a "daemon thread". I believe the interrupt thread qualifies as such a > > thread if I have understood everything right (which is hard to promise when > > you only work in DPDK in spare time). > > > >> It looks like 'pthread_cancel' is the right way and I saw it > >> continue keeps current EINTR handling in EAL interrupt thread. > > It is one option. Depending what makes the most sense. > > > >> 1. Can you explain and add patch comments why default signal handler > >> is not good enough to terminate app. > > Yes if someone call tell me more about what it is so I can check it. > > > >> 2. I propose to add addition comments on rte_epoll_wait() API > >> description. For any signal, it causes an error return, user needs > >> to handle. > > Agreed. > > > >> 3. Will you do a favorite to add 'PTHREAD_CREATE_DETACHED' to all > >> EAL pthread too. > > As a spare time developer I am a bit conservative about too large of a scope > > and messing with code for other threads or features I didn't personally use > > or > > test. This is because I don't have the same QA resources as Intel / 6WIND / > > etc.. Some help from a full time developer would be great here. > All right, reasonable to me. > > > > >> Cunming > > Matthew. >
[dpdk-dev] [dpdk-dev, 1/3] rte_interrupts: add rte_eal_intr_exit to shut down IRQ thread
Hi Mattew, Thank you for your time. On 3/22/2016 3:39 PM, Matthew Hall wrote: > On Mon, Mar 21, 2016 at 03:58:44PM +0800, Liang, Cunming wrote: >> the default termination handler > I am not so experienced with this "default termination handler". Can someone > clarify what it is so I could comment better about it? For example, you're handling SIGINT. After finishing your necessary app cleanup, then 'signal(SIGINT, SIG_DFL); raise(SIGINT);'. The default signal handler can terminate the interrupt thread. > >> If EINTR is caused by some non-term purpose signals, are you going >> to exit the interrupt thread any way? > We should discuss what makes sense here. I'm just trying to get some things > working and finding EINTR was getting eaten and causing infinite looping. SIGINT/SIGTERM causes EINTR return, while SIGUSR1 also can cause the EINTR return. For the dedicated EAL interrupt thread, it won't be expected to exit for all kinds of the cause. On this view, I'm in favor of your patch which cancel the interrupt thread, but don't directly return by the EINTR. > >> Without setting 'PTHREAD_CREATE_DETACHED' won't cause the infinite >> loop. However by using pthread_cancel to terminate the thread, >> indeed it's necessary to set 'PTHREAD_CREATE_DETACHED'. > My general understanding is that PTHREAD_CREATE_DETACHED should be used for > any thread, which should not keep a process open by itself if it is executing, > i.e. a "daemon thread". I believe the interrupt thread qualifies as such a > thread if I have understood everything right (which is hard to promise when > you only work in DPDK in spare time). > >> It looks like 'pthread_cancel' is the right way and I saw it >> continue keeps current EINTR handling in EAL interrupt thread. > It is one option. Depending what makes the most sense. > >> 1. Can you explain and add patch comments why default signal handler >> is not good enough to terminate app. > Yes if someone call tell me more about what it is so I can check it. > >> 2. I propose to add addition comments on rte_epoll_wait() API >> description. For any signal, it causes an error return, user needs >> to handle. > Agreed. > >> 3. Will you do a favorite to add 'PTHREAD_CREATE_DETACHED' to all >> EAL pthread too. > As a spare time developer I am a bit conservative about too large of a scope > and messing with code for other threads or features I didn't personally use or > test. This is because I don't have the same QA resources as Intel / 6WIND / > etc.. Some help from a full time developer would be great here. All right, reasonable to me. > >> Cunming > Matthew.
[dpdk-dev] [dpdk-dev, 1/3] rte_interrupts: add rte_eal_intr_exit to shut down IRQ thread
On Mon, Mar 21, 2016 at 03:58:44PM +0800, Liang, Cunming wrote: > the default termination handler I am not so experienced with this "default termination handler". Can someone clarify what it is so I could comment better about it? > If EINTR is caused by some non-term purpose signals, are you going > to exit the interrupt thread any way? We should discuss what makes sense here. I'm just trying to get some things working and finding EINTR was getting eaten and causing infinite looping. > Without setting 'PTHREAD_CREATE_DETACHED' won't cause the infinite > loop. However by using pthread_cancel to terminate the thread, > indeed it's necessary to set 'PTHREAD_CREATE_DETACHED'. My general understanding is that PTHREAD_CREATE_DETACHED should be used for any thread, which should not keep a process open by itself if it is executing, i.e. a "daemon thread". I believe the interrupt thread qualifies as such a thread if I have understood everything right (which is hard to promise when you only work in DPDK in spare time). > It looks like 'pthread_cancel' is the right way and I saw it > continue keeps current EINTR handling in EAL interrupt thread. It is one option. Depending what makes the most sense. > 1. Can you explain and add patch comments why default signal handler > is not good enough to terminate app. Yes if someone call tell me more about what it is so I can check it. > 2. I propose to add addition comments on rte_epoll_wait() API > description. For any signal, it causes an error return, user needs > to handle. Agreed. > 3. Will you do a favorite to add 'PTHREAD_CREATE_DETACHED' to all > EAL pthread too. As a spare time developer I am a bit conservative about too large of a scope and messing with code for other threads or features I didn't personally use or test. This is because I don't have the same QA resources as Intel / 6WIND / etc.. Some help from a full time developer would be great here. > Cunming Matthew.
[dpdk-dev] [dpdk-dev, 1/3] rte_interrupts: add rte_eal_intr_exit to shut down IRQ thread
Hi Matthew, On 3/18/2016 6:55 AM, Matthew Hall wrote: > From Cunming: >> I'm trying to understand the motivation. >> >> I don't think you're going to gracefully exit intr thread but leave all >> other eal threads live. We don't have API to new launch intr thread again. > The doc comment added for rte_eal_intr_exit already explains this. According > to the doc I wrote, use of the function is limited to shutting everything > down. > >> So I guess your app is using own pthread(none EAL thread), you're trying to >> safely shutdown the whole application by your signal handler. > No, the app is using DPDK pthreads, and trying to shutdown everything safely > and cleanly w/ its signal handler, across DPDK and many other services in the > app. Get you. You don't satisfy with the default termination signal handler(SIG_DEL). The purpose is to safely clean everything by self-defined signal handler. Can you share us more of your observation on why the default termination handler is not enough/safe? As some of the samples are using it to terminate app, your concern may be necessary to apply on them as well. > Unfortunately, right now from my experience it is impossible to get > everything to > cleanly shutdown, one an interrupt thread is activated. Because interrupt > threads violate violate POSIX semantics: > > 1) It ignores EINTR and immediately forcibly restarts a poll() syscall. If the > signal is delivered to the interrupt thread of the process by the kernel, this > makes the thread uninterruptible to process the signal. Stuck running forever. If EINTR is caused by some non-term purpose signals, are you going to exit the interrupt thread any way? > 2) It does not properly set PTHREAD_CREATE_DETACHED for a background thread. > So it holds the process open for its infinite loop of poll(). Stuck running > forever. Without setting 'PTHREAD_CREATE_DETACHED' won't cause the infinite loop. However by using pthread_cancel to terminate the thread, indeed it's necessary to set 'PTHREAD_CREATE_DETACHED'. > 3) There is no way to access the thread_id from intr_thread. So then you can't > call pthread_cancel on it to shut it down. Stuck running forever. It looks like 'pthread_cancel' is the right way and I saw it continue keeps current EINTR handling in EAL interrupt thread. >> For this purpose, the device shall close safely(turn off intr) during the >> time, intr thread still wait but no event will be raised. > In theory yes. In practice no. Because the intr thread violated POSIX rules > for background processing threads per above. > >> In this view, it seems not necessary to have this new. Can you explain more >> detail for the purpose? > Based on my testing, I disagree. I could not get reliable shutdowns without > this, or I wouldn't have coded it. (: Now it's clear to me, overall it's fine. Three additional comments. 1. Can you explain and add patch comments why default signal handler is not good enough to terminate app. 2. I propose to add addition comments on rte_epoll_wait() API description. For any signal, it causes an error return, user needs to handle. 3. Will you do a favorite to add 'PTHREAD_CREATE_DETACHED' to all EAL pthread too. Cunming > Matthew. > >
[dpdk-dev] [dpdk-dev, 1/3] rte_interrupts: add rte_eal_intr_exit to shut down IRQ thread
>From Cunming: > I'm trying to understand the motivation. > > I don't think you're going to gracefully exit intr thread but leave all > other eal threads live. We don't have API to new launch intr thread again. The doc comment added for rte_eal_intr_exit already explains this. According to the doc I wrote, use of the function is limited to shutting everything down. > So I guess your app is using own pthread(none EAL thread), you're trying to > safely shutdown the whole application by your signal handler. No, the app is using DPDK pthreads, and trying to shutdown everything safely and cleanly w/ its signal handler, across DPDK and many other services in the app. Unfortunately, right now from my experience it is impossible to get everything to cleanly shutdown, one an interrupt thread is activated. Because interrupt threads violate violate POSIX semantics: 1) It ignores EINTR and immediately forcibly restarts a poll() syscall. If the signal is delivered to the interrupt thread of the process by the kernel, this makes the thread uninterruptible to process the signal. Stuck running forever. 2) It does not properly set PTHREAD_CREATE_DETACHED for a background thread. So it holds the process open for its infinite loop of poll(). Stuck running forever. 3) There is no way to access the thread_id from intr_thread. So then you can't call pthread_cancel on it to shut it down. Stuck running forever. > For this purpose, the device shall close safely(turn off intr) during the > time, intr thread still wait but no event will be raised. In theory yes. In practice no. Because the intr thread violated POSIX rules for background processing threads per above. > In this view, it seems not necessary to have this new. Can you explain more > detail for the purpose? Based on my testing, I disagree. I could not get reliable shutdowns without this, or I wouldn't have coded it. (: Matthew.
