Re: [go-nuts] How to drop old value in a channel salety
If you read more than write, then you can use a sync.RWMutex an RLock when reading, Lock when writing. Kurtis Rader a következőt írta (2020. november 15., vasárnap, 6:13:03 UTC+1): > On Sat, Nov 14, 2020 at 8:34 PM Robert Engels > wrote: > >> That wasn’t my take on the OPs need. He said the consumer is very >> expensive - implying to me they only want to process the latest. If >> dropping the oldest is viable you might as well drop all old entries and >> use the latest. >> > > The O.P. wrote two days ago: > > > Thanks. I want the receiver always get the relately new vaule, I don't > want the sender blocked and I either choose drop the current value or the > first value of the channel. But I don't find a way to safely drop the first > value from the channel. > > In other words, they want the consumer to always fetch, and process, the > oldest available value. They want the producer(s) to always enqueue their > value; even if doing so requires dropping the oldest value from the queue > to make room for the new value. Whether that is sensible behavior, thus > requiring something other than a simple Go channel, is not obvious given > what the O.P. has written so far. > -- > Kurtis Rader > Caretaker of the exceptional canines Junior and Hank > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/4554312b-e4ea-4f5a-9782-7e2be4154e77n%40googlegroups.com.
Re: [go-nuts] How to detect HTTP client stop polling at the server side
> On 14 Nov 2020, at 20.06, Afriyie Abraham Kwabena > wrote: > > Hi, > > My question is about multiple HTTP client polling an HTTP server randomly > with a PATCH request to update a resource at the server running in front of > MongoDB. The clients poll with their different ids and a valid time in their > request. At the server, I can keep their ids and their different times. > How can detect at the server if a specific client stop polling. I would like > to write a function that detect at the HTTP server if a specific client stop > polling and then remove its resources after some time from a database. Am > using gorilla/mux package for the server. Any help, am new to Golang > programming. > I think this is not Go specific, but more like engineering in general. My suggestion is you can store the last time when user request to PATCH endpoint in some storage or internal memory (like map[userID]timestamp) and then have some worker that check the timestamp to clean it up every X seconds or minutes. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/EA52034E-DFED-4853-8720-402BF5E8967F%40gmail.com.
Re: [go-nuts] How to drop old value in a channel salety
On Sat, Nov 14, 2020 at 8:34 PM Robert Engels wrote: > That wasn’t my take on the OPs need. He said the consumer is very > expensive - implying to me they only want to process the latest. If > dropping the oldest is viable you might as well drop all old entries and > use the latest. > The O.P. wrote two days ago: > Thanks. I want the receiver always get the relately new vaule, I don't want the sender blocked and I either choose drop the current value or the first value of the channel. But I don't find a way to safely drop the first value from the channel. In other words, they want the consumer to always fetch, and process, the oldest available value. They want the producer(s) to always enqueue their value; even if doing so requires dropping the oldest value from the queue to make room for the new value. Whether that is sensible behavior, thus requiring something other than a simple Go channel, is not obvious given what the O.P. has written so far. -- Kurtis Rader Caretaker of the exceptional canines Junior and Hank -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CABx2%3DD-KBQhYV1J3o4s%2Ba1JnpXDz%2BprG4g5N_KjKo1Cp7WHS%2Bg%40mail.gmail.com.
Re: [go-nuts] How to drop old value in a channel salety
That wasn’t my take on the OPs need. He said the consumer is very expensive - implying to me they only want to process the latest. If dropping the oldest is viable you might as well drop all old entries and use the latest. Pretty common in trading systems with a price feed - in many cases only the latest price matters. Not a big deal either way - only pointing it out. >> On Nov 14, 2020, at 10:18 PM, Kurtis Rader wrote: > >> On Sat, Nov 14, 2020 at 7:54 PM Robert Engels wrote: > >> I don’t think a ring buffer works. At least in a traditional ring buffer the >> producer cannot pass the consumer - if the buffer is full the producer >> blocks or drops. Trying to ensure the consumer always reads the most recent >> available item is better (and more simply) served with a shared hand off >> queue. > > It is trivial to create a ring buffer type that drops the oldest queued item > when appending a new item if the ring is full. Which would seem to be the > behavior the O.P. wants. The important question in this scenario is whether > such an object must be useful in a `select` statement. If not the > implementation is borderline trivial. > > -- > Kurtis Rader > Caretaker of the exceptional canines Junior and Hank > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/CABx2%3DD82zaBDcyuCPGayqLmf0oHA7SYi2uNaZ2WYUOAsrAzVag%40mail.gmail.com. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/21BC31A3-D9DC-4E2A-A8A1-7A0B5F078392%40ix.netcom.com.
Re: [go-nuts] How to drop old value in a channel salety
On Sat, Nov 14, 2020 at 7:54 PM Robert Engels wrote: > I don’t think a ring buffer works. At least in a traditional ring buffer > the producer cannot pass the consumer - if the buffer is full the producer > blocks or drops. Trying to ensure the consumer always reads the most recent > available item is better (and more simply) served with a shared hand off > queue. > It is trivial to create a ring buffer type that drops the oldest queued item when appending a new item if the ring is full. Which would seem to be the behavior the O.P. wants. The important question in this scenario is whether such an object must be useful in a `select` statement. If not the implementation is borderline trivial. -- Kurtis Rader Caretaker of the exceptional canines Junior and Hank -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CABx2%3DD82zaBDcyuCPGayqLmf0oHA7SYi2uNaZ2WYUOAsrAzVag%40mail.gmail.com.
Re: [go-nuts] How to drop old value in a channel salety
I don’t think a ring buffer works. At least in a traditional ring buffer the producer cannot pass the consumer - if the buffer is full the producer blocks or drops. Trying to ensure the consumer always reads the most recent available item is better (and more simply) served with a shared hand off queue. > On Nov 14, 2020, at 8:48 PM, Kurtis Rader wrote: > > > On Sat, Nov 14, 2020 at 5:43 PM 陶青云 wrote: >>> Don't use channels for this. It's not what they are for. >>> >>> Ian >> I see some examples using the buffered channel as a job queue. Could you >> give me some advise on this. > > If you search for terms like "go channel job queue" you'll find examples that > at first glance appear relevant to your problem. Such as > https://www.opsdash.com/blog/job-queues-in-go.html. But none of them, AFAICT, > implement the behavior you want. If you don't need compatibility with Go's > `select` statement probably the simplest solution is a ring (circular) > buffer. Search for "go ring buffer" for some examples; although, implementing > a ring buffer is simple enough it can be used as an interview question. So I > leave the details as an exercise for the student. > > You could use a Go channel if you added an explicit mutex that has to be > acquired by both the channel consumer(s) and producer(s) before popping > and/or inserting a value into the channel. But that is suboptimal since the > channel send/receive ops already use an implicit mutex to make those > individual operations concurrent safe. The only purpose of the explicit mutex > is to make it possible for the producer(s) in your example to atomically drop > an "old" entry before inserting a "new" entry. As Ian said a Go channel is > the wrong mechanism for your problem. > > -- > Kurtis Rader > Caretaker of the exceptional canines Junior and Hank > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/CABx2%3DD-PBF_LCEWy_BGfG-bu1v%2BTNP5m_s_dM9Ld8ducwnY3%3Dg%40mail.gmail.com. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/D276E597-D379-48AD-AA1D-A1154D34739E%40ix.netcom.com.
Re: [go-nuts] How to drop old value in a channel salety
On Sat, Nov 14, 2020 at 5:43 PM 陶青云 wrote: > Don't use channels for this. It's not what they are for. >> >> Ian > > I see some examples using the buffered channel as a job queue. Could you > give me some advise on this. > If you search for terms like "go channel job queue" you'll find examples that at first glance appear relevant to your problem. Such as https://www.opsdash.com/blog/job-queues-in-go.html. But none of them, AFAICT, implement the behavior you want. If you don't need compatibility with Go's `select` statement probably the simplest solution is a ring (circular) buffer. Search for "go ring buffer" for some examples; although, implementing a ring buffer is simple enough it can be used as an interview question. So I leave the details as an exercise for the student. You could use a Go channel if you added an explicit mutex that has to be acquired by both the channel consumer(s) and producer(s) before popping and/or inserting a value into the channel. But that is suboptimal since the channel send/receive ops already use an implicit mutex to make those individual operations concurrent safe. The only purpose of the explicit mutex is to make it possible for the producer(s) in your example to atomically drop an "old" entry before inserting a "new" entry. As Ian said a Go channel is the wrong mechanism for your problem. -- Kurtis Rader Caretaker of the exceptional canines Junior and Hank -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CABx2%3DD-PBF_LCEWy_BGfG-bu1v%2BTNP5m_s_dM9Ld8ducwnY3%3Dg%40mail.gmail.com.
[go-nuts] Re: Testing HTTP client timeout behaviour
> On 14 Nov 2020, at 10:28 pm, Amit Saha wrote:
>
> Hi all, I am attempting to write a test for http client timeout behaviour.
> Here’s my bad test server:
>
> func startBadTestHTTPServer() *httptest.Server {
> ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r
> *http.Request) {
> time.Sleep(60 * time.Second)
> fmt.Fprint(w, "Hello World")
> }))
> return ts
> }
>
> func TestFetchBadRemoteResource(t *testing.T) {
> ts := startBadTestHTTPServer()
> defer ts.Close()
>
> client := http.Client{Timeout: 15 * time.Second}
> data, err := fetchRemoteResource(client, ts.URL)
> if err != nil {
> t.Fatal(err)
> }
>
> expected := "Hello World"
> got := string(data)
>
> if expected != got {
> t.Errorf("Expected response to be: %s, Got: %s", expected, got)
> }
> }
>
> When I run the test, I get:
>
> RUN TestFetchBadRemoteResource
>fetch_remote_resource_bad_server_test.go:27: Get "http://127.0.0.1:62721":
> context deadline exceeded (Client.Timeout exceeded while awaiting headers)
>
>
> This is expected. But I also get:
>
> 2020/11/14 22:24:58 httptest.Server blocked in Close after 5 seconds, waiting
> for connections:
> *net.TCPConn 0xc000124040 127.0.0.1:62722 in state active
> --- FAIL: TestFetchBadRemoteResource (60.00s)
>
>
> I understand why this is happening. But, my test still waits for 60 seconds
> for the server to shutdown.
>
> Is there a way around this behaviour where the test server can be forcibly
> terminated and just carry on?
I didn’t succeed in finding a way for exactly what I wanted. However looking at
https://golang.org/src/net/http/client_test.go I found a much better way to
simulate a bad server. Instead a time.Sleep(), have it block on a channel and
then write to the channel in the test. So my bad server now is:
func startBadTestHTTPServer(shutdownServer chan struct{}) *httptest.Server {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r
*http.Request) {
<-shutdownServer
fmt.Fprint(w, "Hello World")
}))
return ts
}
In the test I then do:
data, err := fetchRemoteResource(client, ts.URL)
if err != nil {
shutdownServer <- struct{}{}
t.Fatal(err)
}
>
> Thanks,
> Amit
>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/66CC9BF3-EB1F-4ED0-878E-680D74077AA9%40gmail.com.
Re: [go-nuts] How to drop old value in a channel salety
在2020年11月13日星期五 UTC+8 下午11:19:46 写道： > On Thu, Nov 12, 2020 at 11:13 PM 陶青云 wrote: > > > > > > It is not a one-length buffered channel. I need to drop because the > recveiver do heavy work that can't process as quickly as sender. > > Don't use channels for this. It's not what they are for. > > Ian > > I see some examples using the buffered channel as a job queue. Could you give me some advise on this. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/8bfcf776-2739-45a9-8394-977dc3186e2bn%40googlegroups.com.
Re: [go-nuts] How to drop old value in a channel salety
在2020年11月14日星期六 UTC+8 上午3:09:24 写道：
> If I understand what you're trying to do, I'd approach it this way, using
> a generously buffered channel and discarding the extras at the consumer, as
> shown below, instead of at the producer:
>
> result <- c // wait for result to appear
> for len(c) > 0 {
> // there is a newer result available
> result <- c
> }
> // process latest available result
>
> In normal case I don't want to discarding. Even consumer find the channel
> is full, it should not to discard.
>
>
>>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/dfb924b6-6eb0-406d-9c51-b396d1181078n%40googlegroups.com.
Re: [go-nuts] detecting cyclic references
Or github.com/kortschak/utter . This package does an arguably better job at dealing with self- referencing structures. On Sat, 2020-11-14 at 03:52 -0800, twp...@gmail.com wrote: > > My use case is that I want to pretty print in-memory objects for > debug purposes during testing, and one of the features would be to > point out such cycles as this. > > Consider using an existing library for this, for example: > https://github.com/sanity-io/litter > https://github.com/davecgh/go-spew -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/e4c5f7bbc6a6390c004bf20fcef77db816cf4215.camel%40kortschak.io.
Re: [go-nuts] Re: How does the Golang compiler handle dependencies?
On Sat, Nov 14, 2020 at 2:54 AM kev kev wrote: > Oh right, I seem to not understand why golang is faster in that respect. > If you can include the precompiled headers and or have an object file > > The key point is that in C++, declarations in header files tend to be leaky, especially in larger projects. That is, your class includes some private declarations, but those are listed in the (public) header file. This happens transitively/recursively through your whole header file import hierarchy. This means the header file needs to pull in dependent header files in order to satisfy declarations in the private area of the class, yet since they are not publicly facing, this creates a need for parsing more code for a given compilation, slowing down the compiler: you still have to look at every byte to parse, and you can only throw information away once you have analysed which parts you are actually using in the compilation unit. To boot, C++ is a rather complex language to parse, where you may need more than a single pass over the declarations in order to figure out what is going on. It also creates a situation where changing some of the foundational header files in the project results in large recompilations of everything because there is a potential for things to have changed. The traditional way of solving this has been to cache heavily and only recompile if need be. However, caching only helps you so much if you are leaking in header files all over the place. In contrast, Go's design plugs this hole. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAGrdgiWMSdGfZ1%3D%2Bh2%3D0%2BLBCwsVcmh1LVCbAD%2By%3DcN%3D5YPxbww%40mail.gmail.com.
[go-nuts] How to detect HTTP client stop polling at the server side
Hi, My question is about multiple HTTP client polling an HTTP server randomly with a PATCH request to update a resource at the server running in front of MongoDB. The clients poll with their different ids and a valid time in their request. At the server, I can keep their ids and their different times. How can detect at the server if a specific client stop polling. I would like to write a function that detect at the HTTP server if a specific client stop polling and then remove its resources after some time from a database. Am using gorilla/mux package for the server. Any help, am new to Golang programming. Thanks in advance -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/8658410c-5cac-4dff-80e3-ebb1b8bbb5een%40googlegroups.com.
Re: [go-nuts] detecting cyclic references
> My use case is that I want to pretty print in-memory objects for debug
purposes during testing, and one of the features would be to point out such
cycles as this.
Consider using an existing library for this, for example:
https://github.com/sanity-io/litter
https://github.com/davecgh/go-spew
On Wednesday, November 11, 2020 at 1:59:18 PM UTC+1 arpad@gmail.com
wrote:
> `l[0] = l` would be fine for me, indeed. Though I am not sure I understand
> the suggested solution. Notice that the type of the slice is
> `[]interface{}`. This...
>
> ll := l[0].([]interface{})
> println( == )
>
> ...would print `false`.
>
> I think the main challenge is that l and ll, or l[0] for that matter, are
> not the same values. I tried with reflection, too, but couldn't figure a
> way.
>
> PS: I am quite aware how unearthy this problem is, and how it should not
> ever happen in case of actual programs. My use case is that I want to
> pretty print in-memory objects for debug purposes during testing, and one
> of the features would be to point out such cycles as this.
>
>
> On Wednesday, November 11, 2020 at 8:13:55 AM UTC+1
> axel.wa...@googlemail.com wrote:
>
>> The question was about detecting cyclic references. For that, the `l[0] =
>> l` case is enough.
>>
>> On Wed, Nov 11, 2020 at 2:31 AM jake...@gmail.com
>> wrote:
>>
>>> FYI, that does detect the simple case of l[0] = l, but not more
>>> complicated circular cases like l[1] = l[1:]
>>>
>>> On Tuesday, November 10, 2020 at 2:38:26 AM UTC-5
>>> axel.wa...@googlemail.com wrote:
>>>
If the slice is empty, it doesn't reference anything.
If it is not empty, [0] can be used to identify the slice
(potentially also using len/cap, if it's interesting).
On Tue, Nov 10, 2020 at 4:11 AM arpad ryszka
wrote:
> Hi,
>
> is there a way to detect the cyclic reference in the following example
> somehow? Either via reflection or by other means? My understanding is
> that
> slices cannot be compared with == (unless to nil), or used as keys in
> maps.
> Is there a different way?
>
> l := []interface{}{"foo"}
> l[0] = l
> fmt.Println(l)
>
> or here it is as a playground link:
> https://play.golang.org/p/T0qZlF8m-vi
>
> Cheers,
> Arpad
>
> --
> You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/b2cb6b5e-febc-407f-b5b3-d9ca196ce68bn%40googlegroups.com
>
>
> .
>
--
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts...@googlegroups.com.
>>>
>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/250107e9-f688-4205-ae52-728221eb2e4cn%40googlegroups.com
>>>
>>>
>>> .
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/cc25d4f0-51fa-4851-989b-fab4527bc709n%40googlegroups.com.
[go-nuts] Testing HTTP client timeout behaviour
Hi all, I am attempting to write a test for http client timeout behaviour.
Here’s my bad test server:
func startBadTestHTTPServer() *httptest.Server {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r
*http.Request) {
time.Sleep(60 * time.Second)
fmt.Fprint(w, "Hello World")
}))
return ts
}
func TestFetchBadRemoteResource(t *testing.T) {
ts := startBadTestHTTPServer()
defer ts.Close()
client := http.Client{Timeout: 15 * time.Second}
data, err := fetchRemoteResource(client, ts.URL)
if err != nil {
t.Fatal(err)
}
expected := "Hello World"
got := string(data)
if expected != got {
t.Errorf("Expected response to be: %s, Got: %s", expected, got)
}
}
When I run the test, I get:
RUN TestFetchBadRemoteResource
fetch_remote_resource_bad_server_test.go:27: Get "http://127.0.0.1:62721":
context deadline exceeded (Client.Timeout exceeded while awaiting headers)
This is expected. But I also get:
2020/11/14 22:24:58 httptest.Server blocked in Close after 5 seconds, waiting
for connections:
*net.TCPConn 0xc000124040 127.0.0.1:62722 in state active
--- FAIL: TestFetchBadRemoteResource (60.00s)
I understand why this is happening. But, my test still waits for 60 seconds for
the server to shutdown.
Is there a way around this behaviour where the test server can be forcibly
terminated and just carry on?
Thanks,
Amit
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/94AE404A-6CB1-44EF-AE10-35CC6F87B3D6%40gmail.com.
