Re: [go-nuts] Ensure context timeout behaviour from caller, or callee ?
>
> I find it awkward
> and risky if I cannot pass down a cancelable context down the stack
> where things may not return.
Absolutely, we always want to be able to pass that down regardless.
Those that are
> lower the stack are not enforcing timeouts or cancellations, they are
> merely getting a notification of it.
>
Right, this is how I understand it as well. In this case the database lib
methods takes the context and act on the
cancellation (rightly so), but also seemed to suggest to not bother
enforcing that timeout at the higher level. Perhaps I'm
reading too much into their example though.
I am a bit confused about what you are asking here.
>
Apologies. I think you've already answered my question with your previous
statement ('Those that are lower down the stack are not enforcing
timeouts...').
Thanks for your patience, this has been illuminating.
On Thu, 21 Sept 2023 at 12:40, burak serdar wrote:
> On Wed, Sep 20, 2023 at 5:04 PM Jerry Londergaard
> wrote:
> >
> > Are you comfortable though, *relying* on this behaviour down the stack
> to enforce the timeout that was declared
> > further up the stack (leaving potentially leaked goroutines out of
> it...)?
>
> That's the nature of handling requests using this layered approach. As
> you move down layers in a concurrent program, those that are at a
> higher layer set the context for the lower layers. Those that are
> lower the stack are not enforcing timeouts or cancellations, they are
> merely getting a notification of it.
>
> >
> >> If the new goroutine cannot check context cancellation, then the
> >> goroutine starting that new goroutine can deal with cancellation like
> >> you did, hoping that the new goroutine eventually ends.
> >
> >
> > Ok, I think you're referring to this example of mine
> https://go.dev/play/p/EGNlJqo3hY5
> >
> >> I usually use
> >> the former, but the latter has its uses if you are calling a
> >> third-party library or some operation you cannot cancel.
> >
> >
> > This is close to the heart of the question. My example of that
> third-party library is the database one,
> > and it seems to say (based on its suggested usage) "You shouldn't need
> to handle your context timeout, we'll make sure we do".
> > Which, maybe works fine in that specific case, but in general feels a
> bit awkward. It seems you would agree?
>
> I am a bit confused about what you are asking here. I find it awkward
> and risky if I cannot pass down a cancelable context down the stack
> where things may not return. But then, that's the only way to deal
> with such third-party code. If goroutines leak, they leak, and
> eventually you run out of memory, crash, and restart. Sometimes
> failing and recovering is cheaper than fixing.
>
> >
> >>
> >> If that operation never ends, the goroutine leaks.
> >
> >
> > That's a good point. If you don't run code in a goroutine you can never
> leak goroutines :-p
> > It seems to me though, that if we are running things in goroutines, and
> passing down the context,
> > and the code being run in those goroutines isn't listening for
> cancellation signals,
> > which results in the running goroutine becoming leaked, then that feels
> like a bug in that code (i.e not the callers responsibility).
>
> >
> >
> >
> >
> > On Thu, 21 Sept 2023 at 05:34, burak serdar
> wrote:
> >>
> >> On Wed, Sep 20, 2023 at 7:47 AM Jerry Londergaard
> >> wrote:
> >> >
> >> > When using a context.WithTimeout, I always felt that it should be the
> function where the context was created should be the one that ensures that
> it (the current function) does not block longer than intended. That is to
> say, it should call context.WithTimeout(), and then run the subsequent code
> in a goroutine (passing in the context most likely), whilst using a select
> multiplexer to listen to when the context is cancelled (by the timeout),
> and then resume. Here's an example of this
> https://go.dev/play/p/EGNlJqo3hY5
> >> >
> >> > However, when I was looking at this on the go database docs, it seems
> to say to push that logic further down the stack to the 'child' (or callee)
> https://go.dev/doc/database/cancel-operations#:~:text=Canceling%20database%20operations%20after%20a%20timeout
> >> >
> >> > I think this is an alternative implementation of my first example,
> that is analogous to what the database package is suggesting
> https://go.dev/play/p/DABt-Z36T-F
> >> >
> >> > Whilst I understand that the child should listen for the signal and
> clean-up what it's doing in the event of the cancellation, that's doesn't
> feel like the same thing as the caller *relying* on that behaviour for the
> caller function to resume running.
> >> >
> >> > How should I think about this? Am I missing something here?
> >>
> >> Both patterns have their uses. If the new goroutine can periodically
> >> check context cancellation and needs to do some cleanup, then it is
> >> best to pass the context down, clean up, and return on cancellation.
> >> If the new goroutine cannot
Re: [go-nuts] Ensure context timeout behaviour from caller, or callee ?
On Wed, Sep 20, 2023 at 5:04 PM Jerry Londergaard wrote: > > Are you comfortable though, *relying* on this behaviour down the stack to > enforce the timeout that was declared > further up the stack (leaving potentially leaked goroutines out of it...)? That's the nature of handling requests using this layered approach. As you move down layers in a concurrent program, those that are at a higher layer set the context for the lower layers. Those that are lower the stack are not enforcing timeouts or cancellations, they are merely getting a notification of it. > >> If the new goroutine cannot check context cancellation, then the >> goroutine starting that new goroutine can deal with cancellation like >> you did, hoping that the new goroutine eventually ends. > > > Ok, I think you're referring to this example of mine > https://go.dev/play/p/EGNlJqo3hY5 > >> I usually use >> the former, but the latter has its uses if you are calling a >> third-party library or some operation you cannot cancel. > > > This is close to the heart of the question. My example of that third-party > library is the database one, > and it seems to say (based on its suggested usage) "You shouldn't need to > handle your context timeout, we'll make sure we do". > Which, maybe works fine in that specific case, but in general feels a bit > awkward. It seems you would agree? I am a bit confused about what you are asking here. I find it awkward and risky if I cannot pass down a cancelable context down the stack where things may not return. But then, that's the only way to deal with such third-party code. If goroutines leak, they leak, and eventually you run out of memory, crash, and restart. Sometimes failing and recovering is cheaper than fixing. > >> >> If that operation never ends, the goroutine leaks. > > > That's a good point. If you don't run code in a goroutine you can never leak > goroutines :-p > It seems to me though, that if we are running things in goroutines, and > passing down the context, > and the code being run in those goroutines isn't listening for cancellation > signals, > which results in the running goroutine becoming leaked, then that feels like > a bug in that code (i.e not the callers responsibility). > > > > > On Thu, 21 Sept 2023 at 05:34, burak serdar wrote: >> >> On Wed, Sep 20, 2023 at 7:47 AM Jerry Londergaard >> wrote: >> > >> > When using a context.WithTimeout, I always felt that it should be the >> > function where the context was created should be the one that ensures that >> > it (the current function) does not block longer than intended. That is to >> > say, it should call context.WithTimeout(), and then run the subsequent >> > code in a goroutine (passing in the context most likely), whilst using a >> > select multiplexer to listen to when the context is cancelled (by the >> > timeout), and then resume. Here's an example of this >> > https://go.dev/play/p/EGNlJqo3hY5 >> > >> > However, when I was looking at this on the go database docs, it seems to >> > say to push that logic further down the stack to the 'child' (or callee) >> > https://go.dev/doc/database/cancel-operations#:~:text=Canceling%20database%20operations%20after%20a%20timeout >> > >> > I think this is an alternative implementation of my first example, that is >> > analogous to what the database package is suggesting >> > https://go.dev/play/p/DABt-Z36T-F >> > >> > Whilst I understand that the child should listen for the signal and >> > clean-up what it's doing in the event of the cancellation, that's doesn't >> > feel like the same thing as the caller *relying* on that behaviour for the >> > caller function to resume running. >> > >> > How should I think about this? Am I missing something here? >> >> Both patterns have their uses. If the new goroutine can periodically >> check context cancellation and needs to do some cleanup, then it is >> best to pass the context down, clean up, and return on cancellation. >> If the new goroutine cannot check context cancellation, then the >> goroutine starting that new goroutine can deal with cancellation like >> you did, hoping that the new goroutine eventually ends. I usually use >> the former, but the latter has its uses if you are calling a >> third-party library or some operation you cannot cancel. If that >> operation never ends, the goroutine leaks. >> >> > >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "golang-nuts" group. >> > To unsubscribe from this group and stop receiving emails from it, send an >> > email to golang-nuts+unsubscr...@googlegroups.com. >> > To view this discussion on the web visit >> > https://groups.google.com/d/msgid/golang-nuts/f45936f2-36b3-46cb-aa93-9bbbf9438843n%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to
Re: [go-nuts] Ensure context timeout behaviour from caller, or callee ?
> > If the new goroutine can periodically > check context cancellation and needs to do some cleanup, then it is > best to pass the context down, clean up, and return on cancellation. Understood. I would always pass the context down anyway, letting that callee decide whether or not it wants to listen for a context cancellation (regardless of what the caller is doing with the context). Are you comfortable though, *relying* on this behaviour down the stack to enforce the timeout that was declared further up the stack (leaving potentially leaked goroutines out of it...)? If the new goroutine cannot check context cancellation, then the > goroutine starting that new goroutine can deal with cancellation like > you did, hoping that the new goroutine eventually ends. > Ok, I think you're referring to this example of mine https://go.dev/play/p/EGNlJqo3hY5 I usually use > the former, but the latter has its uses if you are calling a > third-party library or some operation you cannot cancel. > This is close to the heart of the question. My example of that third-party library is the database one, and it seems to say (based on its suggested usage) "You shouldn't need to handle your context timeout, we'll make sure we do". Which, maybe works fine in that specific case, but in general feels a bit awkward. It seems you would agree? > If that operation never ends, the goroutine leaks. > That's a good point. If you don't run code in a goroutine you can never leak goroutines :-p It seems to me though, that if we are running things in goroutines, and passing down the context, and the code being run in those goroutines isn't listening for cancellation signals, which results in the running goroutine becoming leaked, then that feels like a bug in that code (i.e not the callers responsibility). On Thu, 21 Sept 2023 at 05:34, burak serdar wrote: > On Wed, Sep 20, 2023 at 7:47 AM Jerry Londergaard > wrote: > > > > When using a context.WithTimeout, I always felt that it should be the > function where the context was created should be the one that ensures that > it (the current function) does not block longer than intended. That is to > say, it should call context.WithTimeout(), and then run the subsequent code > in a goroutine (passing in the context most likely), whilst using a select > multiplexer to listen to when the context is cancelled (by the timeout), > and then resume. Here's an example of this > https://go.dev/play/p/EGNlJqo3hY5 > > > > However, when I was looking at this on the go database docs, it seems to > say to push that logic further down the stack to the 'child' (or callee) > https://go.dev/doc/database/cancel-operations#:~:text=Canceling%20database%20operations%20after%20a%20timeout > > > > I think this is an alternative implementation of my first example, that > is analogous to what the database package is suggesting > https://go.dev/play/p/DABt-Z36T-F > > > > Whilst I understand that the child should listen for the signal and > clean-up what it's doing in the event of the cancellation, that's doesn't > feel like the same thing as the caller *relying* on that behaviour for the > caller function to resume running. > > > > How should I think about this? Am I missing something here? > > Both patterns have their uses. If the new goroutine can periodically > check context cancellation and needs to do some cleanup, then it is > best to pass the context down, clean up, and return on cancellation. > If the new goroutine cannot check context cancellation, then the > goroutine starting that new goroutine can deal with cancellation like > you did, hoping that the new goroutine eventually ends. I usually use > the former, but the latter has its uses if you are calling a > third-party library or some operation you cannot cancel. If that > operation never ends, the goroutine leaks. > > > > > -- > > You received this message because you are subscribed to the Google > Groups "golang-nuts" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to golang-nuts+unsubscr...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/f45936f2-36b3-46cb-aa93-9bbbf9438843n%40googlegroups.com > . > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CANBXCctphX6rxfH1mHcLedVf7gd3qc2UJ6QVD4qRz9usZAex7w%40mail.gmail.com.
Re: [go-nuts] Ensure context timeout behaviour from caller, or callee ?
On Wed, Sep 20, 2023 at 7:47 AM Jerry Londergaard wrote: > > When using a context.WithTimeout, I always felt that it should be the > function where the context was created should be the one that ensures that it > (the current function) does not block longer than intended. That is to say, > it should call context.WithTimeout(), and then run the subsequent code in a > goroutine (passing in the context most likely), whilst using a select > multiplexer to listen to when the context is cancelled (by the timeout), and > then resume. Here's an example of this https://go.dev/play/p/EGNlJqo3hY5 > > However, when I was looking at this on the go database docs, it seems to say > to push that logic further down the stack to the 'child' (or callee) > https://go.dev/doc/database/cancel-operations#:~:text=Canceling%20database%20operations%20after%20a%20timeout > > I think this is an alternative implementation of my first example, that is > analogous to what the database package is suggesting > https://go.dev/play/p/DABt-Z36T-F > > Whilst I understand that the child should listen for the signal and clean-up > what it's doing in the event of the cancellation, that's doesn't feel like > the same thing as the caller *relying* on that behaviour for the caller > function to resume running. > > How should I think about this? Am I missing something here? Both patterns have their uses. If the new goroutine can periodically check context cancellation and needs to do some cleanup, then it is best to pass the context down, clean up, and return on cancellation. If the new goroutine cannot check context cancellation, then the goroutine starting that new goroutine can deal with cancellation like you did, hoping that the new goroutine eventually ends. I usually use the former, but the latter has its uses if you are calling a third-party library or some operation you cannot cancel. If that operation never ends, the goroutine leaks. > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/f45936f2-36b3-46cb-aa93-9bbbf9438843n%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAMV2Rqr%2BsMQk8ho5gJToqoY%3DnqdDpFUt5t_S8cZzhgDHLJTNSQ%40mail.gmail.com.
[go-nuts] Ensure context timeout behaviour from caller, or callee ?
When using a context.WithTimeout, I always felt that it should be the function where the context was created should be the one that ensures that it (the current function) does not block longer than intended. That is to say, it should call context.WithTimeout(), and then run the subsequent code in a goroutine (passing in the context most likely), whilst using a select multiplexer to listen to when the context is cancelled (by the timeout), and then resume. Here's an example of this https://go.dev/play/p/EGNlJqo3hY5 However, when I was looking at this on the go database docs, it seems to say to push that logic further down the stack to the 'child' (or callee) https://go.dev/doc/database/cancel-operations#:~:text=Canceling%20database%20operations%20after%20a%20timeout I think this is an alternative implementation of my first example, that is analogous to what the database package is suggesting https://go.dev/play/p/DABt-Z36T-F Whilst I understand that the child should listen for the signal and clean-up what it's doing in the event of the cancellation, that's doesn't feel like the same thing as the caller *relying* on that behaviour for the caller function to resume running. How should I think about this? Am I missing something here? -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/f45936f2-36b3-46cb-aa93-9bbbf9438843n%40googlegroups.com.
