Re: [go-nuts] Re: Why not tuples?
Thanks Brian, that's actually similar to what I'm already doing.
Cheers!
On Wednesday, December 7, 2022 at 8:16:53 AM UTC-3 Brian Candler wrote:
> On Tuesday, 6 December 2022 at 22:27:38 UTC dple...@google.com wrote:
>
>> It'd be great if, for example, you could tag an entire type like:
>>
>> type Row struct { `json:tuple`
>> date int64
>> score float64
>> city string
>> }
>>
>> so that
>>
>> var v []Row
>> json.Unmarshal(data, )
>>
>> would automatically parse the triples into usable structs (and
>> json.Marshal would turn them back into lists).
>>
>
> You can wrap that pattern yourself though, and it's not too much work.
> https://go.dev/play/p/JtUxQUQdd92
>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/0d101208-c021-4fb2-add4-ca498437e04fn%40googlegroups.com.
Re: [go-nuts] Re: Why not tuples?
That was perfectly put, Daniel, thanks! :-)
And, indeed, what I've been doing is to unmarshal as a "[][]any" and then
coercing to the right types.
I also agree tuples have a place in the world. They're quite common when
dealing with tabular data - for example, to reduce the amount of repetition
when describing "rows" of data. But I don't really have to agree or
disagree - they just exist, and that itself begs for an action: either
ignore them, or find a workaround (like
https://github.com/barweiss/go-tuple ), or implement the syntax, type etc
for it (which I totally understand if it's too hard or undesirable).
Some approaches that I find interesting are Python's type hints (e.g.
"tuple[int, float, str]", which is well checked with tools like mypy) or
even the cleaner approach from Rust (e.g. "(i32, f64, String)"). But
anyway, I digress.
I like the approach with struct tags too; While not being a real
implementation of tuples, they at least help with unmarshalling and
marshalling of data to cope with other existing services when they need to
follow that structure.
Cheers!
On Tuesday, December 6, 2022 at 7:27:38 PM UTC-3 dple...@google.com wrote:
> On Sat, Dec 3, 2022 at 11:34 PM burak serdar wrote:
>
>>
>>
>> On Sat, Dec 3, 2022 at 8:47 PM Diogo Baeder wrote:
>>
>>> Now, imagine this scenario: I have a web application which has to access
>>> a webservice that responds with JSON payloads; These payloads are a list of
>>> values, where each value is a smaller list like '[20220101, 1.234, "New
>>> York"]'. And these smaller lists follow the same type sequence: int64,
>>> float64, string. Suppose that I want to filter those values and send a
>>> response to the client, with the data structure unchanged (same format and
>>> types). Today, it doesn't seem to be possible to do that in Go, unless I do
>>> some dirty hack like decoding to '[]any' and then cast to the other types,
>>> and then hack again to put these values in the response to the client.
>>>
>>
>> What you described above is a struct.
>>
>
> I think the problem Diogo is pointing out is a reasonable one - you
> specify the type of the object you're unpacking by e.g.
> json.Unmarshal(data, ), but right now there's no way (AFAIK)
> for the typed object to capture "a list of [int, float, string] triples"
> except by capturing it as a [][]any and then manually typechecking that
> each one is indeed an int, a float, and a string.
>
> It'd be great if, for example, you could tag an entire type like:
>
> type Row struct { `json:tuple`
> date int64
> score float64
> city string
> }
>
> so that
>
> var v []Row
> json.Unmarshal(data, )
>
> would automatically parse the triples into usable structs (and
> json.Marshal would turn them back into lists).
>
> *
>
> I also don't think arguments that Go doesn't need tuples really hold
> water, given that Go already *has* tuples, just only for return types. We
> could all be writing
>
> func Foo() struct{int, error} {
>return struct{int, error}{3, nil}
> }
> v := Foo()
> i, err := v.int, v.error
> ...
>
> but I think everyone agrees that tuples make this code much more readable.
> Given that we all do agree that there are cases where having tuples makes
> for better code, I don't actually know why Go doesn't support them in
> general. Is it out of fear that people will use them poorly, or is it
> actually pretty complex to implement and not worthwhile?
>
> --
> Dan Lepage
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/64d5942a-1723-4578-84f0-d7d3083b1947n%40googlegroups.com.
Re: [go-nuts] Re: Why not tuples?
Hi folks, Thanks for all the inputs, I really appreciate the effort and help :-) Regardless of the whole discussion of whether or not tuples are a good or bad data structure, they exist out there in the wild, and we have to deal with them in many situations - of course, in our own code we can opt by continuing to use them or not, depending on the language we're using. It would be nice if Go could support them as first-class citizens - I know it's not the only language that doesn't, though, Java for example doesn't support them either (BTW there's a nice Java library, javatuples, which gives nice names like Pair, Triplet, Quartet etc, to n-sized "tuples" - I find that enjoyable). Anyways, thanks for the discussion, that helps me get a better grasp of the language! Cheers! On Sunday, December 4, 2022 at 11:39:41 PM UTC-3 Kevin Chowski wrote: > If you really need anonymous tuple types that support decoding that sort > of JSON, it isn't too hard to write one: https://go.dev/play/p/Fn_wUXh2drs > > Go's generics don't support varargs types (...yet? who knows) so there'd > be a little copypasta if you needed many different tuple lengths, but Java > has been doing that for years ;) > > (IMO, using these anonymous tuple types across a whole codebase is not > great: data should be labeled with useful names as it is passed around a > program. But if you really are just using this to make json parsing code > easier, that seems reasonable to me.) > > On Saturday, December 3, 2022 at 8:47:03 PM UTC-7 diogo...@gmail.com > wrote: > >> Hi there, sorry for weighting in so late in the game, but I just started >> again to learn Go and was thinking why the language still doesn't have a >> tuple type. >> >> Now, imagine this scenario: I have a web application which has to access >> a webservice that responds with JSON payloads; These payloads are a list of >> values, where each value is a smaller list like '[20220101, 1.234, "New >> York"]'. And these smaller lists follow the same type sequence: int64, >> float64, string. Suppose that I want to filter those values and send a >> response to the client, with the data structure unchanged (same format and >> types). Today, it doesn't seem to be possible to do that in Go, unless I do >> some dirty hack like decoding to '[]any' and then cast to the other types, >> and then hack again to put these values in the response to the client. >> >> I totally understand the reasoning for preferring the usage of structs >> for heterogeneous data (and I myself do prefer them, they're much more >> powerful in general), but there's real world data that's available like in >> the example above, and we just can't go on changing them at their sources. >> I might be mistaken (please let me know if it's the case), but it seems >> like Go is missing an opportunity to interoperate with what's a fundamental >> data structure in many other languages (Python, Rust etc). I'm having a lot >> of fun learning to use the language, and would be happy to see this feature >> being implemented at the core. >> >> (Maybe what I said above is total BS, I acknowledge that since I'm an >> almost complete ignorant in the language) >> >> Cheers! >> >> On Thursday, April 19, 2018 at 1:03:55 PM UTC-3 Louki Sumirniy wrote: >> >>> Multiple return values. They do kinda exist in a declarative form of >>> sorts, in the type signature, this sets the number and sequence and types >>> of return values. You could even make functions accept them as also input >>> values, I think, but I don't think it works exactly like this. I'm not a >>> fan of these things because of how you have to nominate variables or _ and >>> type inference will make these new variables, if you := into whatever the >>> return was. >>> >>> I'm not sure what the correct word is for them. Untyped in the same way >>> that literals can be multiple types (especially integers) but singular in >>> their literal form. >>> >>> >>> On Thursday, 19 April 2018 16:06:42 UTC+3, Jan Mercl wrote: On Thu, Apr 19, 2018 at 2:51 PM Louki Sumirniy < louki.sumir...@gmail.com> wrote: > Sorry for the self-promotion but it was relevant in that I was working on how to tidy up the readability of my code and needed multiple returns and simple untyped tuples were really not nearly as convenient as using a type struct. I have no idea what you mean by 'untyped tuples' because Go does not have tuples, or at least not as a well defined thing. I can only guess if you're trying to implement tuples in Go with an array, slice or a struct, ...? To add to my confusion, Go functions can have as many return values as one wishes just fine, ie. I obviously do not even understand what problem you're trying to solve. Sorry. -- -j >>> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To
Re: [go-nuts] Re: Why not tuples?
Hi there, sorry for weighting in so late in the game, but I just started again to learn Go and was thinking why the language still doesn't have a tuple type. Now, imagine this scenario: I have a web application which has to access a webservice that responds with JSON payloads; These payloads are a list of values, where each value is a smaller list like '[20220101, 1.234, "New York"]'. And these smaller lists follow the same type sequence: int64, float64, string. Suppose that I want to filter those values and send a response to the client, with the data structure unchanged (same format and types). Today, it doesn't seem to be possible to do that in Go, unless I do some dirty hack like decoding to '[]any' and then cast to the other types, and then hack again to put these values in the response to the client. I totally understand the reasoning for preferring the usage of structs for heterogeneous data (and I myself do prefer them, they're much more powerful in general), but there's real world data that's available like in the example above, and we just can't go on changing them at their sources. I might be mistaken (please let me know if it's the case), but it seems like Go is missing an opportunity to interoperate with what's a fundamental data structure in many other languages (Python, Rust etc). I'm having a lot of fun learning to use the language, and would be happy to see this feature being implemented at the core. (Maybe what I said above is total BS, I acknowledge that since I'm an almost complete ignorant in the language) Cheers! On Thursday, April 19, 2018 at 1:03:55 PM UTC-3 Louki Sumirniy wrote: > Multiple return values. They do kinda exist in a declarative form of > sorts, in the type signature, this sets the number and sequence and types > of return values. You could even make functions accept them as also input > values, I think, but I don't think it works exactly like this. I'm not a > fan of these things because of how you have to nominate variables or _ and > type inference will make these new variables, if you := into whatever the > return was. > > I'm not sure what the correct word is for them. Untyped in the same way > that literals can be multiple types (especially integers) but singular in > their literal form. > > > On Thursday, 19 April 2018 16:06:42 UTC+3, Jan Mercl wrote: >> >> On Thu, Apr 19, 2018 at 2:51 PM Louki Sumirniy >> wrote: >> >> > Sorry for the self-promotion but it was relevant in that I was working >> on how to tidy up the readability of my code and needed multiple returns >> and simple untyped tuples were really not nearly as convenient as using a >> type struct. >> >> I have no idea what you mean by 'untyped tuples' because Go does not have >> tuples, or at least not as a well defined thing. I can only guess if you're >> trying to implement tuples in Go with an array, slice or a struct, ...? To >> add to my confusion, Go functions can have as many return values as one >> wishes just fine, ie. I obviously do not even understand what problem >> you're trying to solve. Sorry. >> >> >> -- >> >> -j >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/8e728e0f-341d-4340-a868-aac028dfc443n%40googlegroups.com.
Re: [go-nuts] Performance for concurrent requests
Hey guys! Turns out that one of the biggest bottlenecks was the type guessing when parsing the JSON content to a "map[string]any"; As soon as I implemented more appropriate structs to unmarshall the bytes I immediately got faster responses. Another big improvement was when changing from the standard JSON library to go-json ( https://github.com/goccy/go-json ). Right now, I get response times close enough to Rust (just a tad slower, not much), which is good enough for me! :-) The experiment project is up-to-date with my findings now. Thank you all for the help! Cheers! On Saturday, December 3, 2022 at 9:21:10 AM UTC-3 Diogo Baeder wrote: > Hi all! > > Thanks for the inputs, I'll do some profiling here and update about my > findings. > > I don't want to change anything on Nginx because the comparison I'm doing > between different stacks is on the same basis. If I try to tune Nginx or > anything like that I'll be comparing apples to oranges, so that's not the > point of my experiment. > > Cheers! > > On Saturday, December 3, 2022 at 6:07:42 AM UTC-3 Brian Candler wrote: > >> Have you tried a tcpdump of the packets between the Go program and >> nginx? Is it using HTTP/1.1 or HTTP/2? If it's HTTP/1.1, does tcpdump >> show that it actually starts all 50 HTTP client requests simultaneously? >> >> You are making all these concurrent requests to the same host. The >> default of MaxConnsPerHost I believe is 0 (unlimited), but >> DefaultMaxIdleConnsPerHost is 2. It could be worth cranking that up. I >> wonder if it's being forced to close down 48 of those connections >> immediately because it can't return them to the pool. >> >> I also wonder whether there's tuning required at the Nginx side, e.g. for >> the backlog queue. >> >> On Saturday, 3 December 2022 at 05:08:40 UTC harr...@spu.edu wrote: >> >>> I wonder a bit about io.ReadAll versus constructing a JSON Decoder. In >>> general, though, using pprof is the best way to start to break down a >>> question like this. Would the actual workload involve more structured JSON, >>> or more computation with decoded values? >>> >>> On Friday, December 2, 2022 at 7:31:50 PM UTC-8 bse...@computer.org >>> wrote: >>> >>>> On Fri, Dec 2, 2022 at 8:13 PM Diogo Baeder wrote: >>>> >>>>> Hi guys, >>>>> >>>>> I've been working on some experiments with different web application >>>>> stacks to check their performances under a specific scenario: one in >>>>> which >>>>> I have to make several concurrent requests and then gather the results >>>>> together (in order) and throw them out as JSON in the response body. >>>>> (This >>>>> project is only an experiment, but it's informing me for decisions that >>>>> have to be made for a real-world project where we have a similar >>>>> scenario.) >>>>> >>>>> However, probably due to my ignorance in Go, I cannot make it perform >>>>> as well as I expected - actually the best I'm getting are results that >>>>> are >>>>> even slower than Python, which was a surprise to me. Here they are: >>>>> https://github.com/yougov/concurrency-tests#edit-13-added-golang-with-gin >>>>> >>>>> So, looking at the code here: >>>>> https://github.com/yougov/concurrency-tests/blob/master/stacks/goapp/main.go >>>>> >>>>> - does anybody see any problem in the implementation that could be >>>>> hurting >>>>> performance? I tried using a WaitGroup, tried sharing memory (nasty, I >>>>> know, but just for the sake of experimentation), tried multiple JSON >>>>> codecs, different web frameworks, and nothing worked so far. I have a >>>>> feeling that I'm doing something fundamentally wrong and stupid, and that >>>>> somehow I can make a small change to make the experiment much faster. >>>>> >>>> >>>> Have you measured how much time is spent on the http.Get calls? It is >>>> likely that the 50 concurrent http.Get calls is the bottleneck. >>>> >>>> Also note that you don't need a channel there. You can simply use a >>>> waitgroup and set the results from inside the goroutine, because each >>>> goroutine knows the index. But that is unlikely to change anything >>>> measurable when compared to the Get calls. >>>> >>>> >>>&
Re: [go-nuts] Performance for concurrent requests
Hi all! Thanks for the inputs, I'll do some profiling here and update about my findings. I don't want to change anything on Nginx because the comparison I'm doing between different stacks is on the same basis. If I try to tune Nginx or anything like that I'll be comparing apples to oranges, so that's not the point of my experiment. Cheers! On Saturday, December 3, 2022 at 6:07:42 AM UTC-3 Brian Candler wrote: > Have you tried a tcpdump of the packets between the Go program and nginx? > Is it using HTTP/1.1 or HTTP/2? If it's HTTP/1.1, does tcpdump show that > it actually starts all 50 HTTP client requests simultaneously? > > You are making all these concurrent requests to the same host. The default > of MaxConnsPerHost I believe is 0 (unlimited), but > DefaultMaxIdleConnsPerHost is 2. It could be worth cranking that up. I > wonder if it's being forced to close down 48 of those connections > immediately because it can't return them to the pool. > > I also wonder whether there's tuning required at the Nginx side, e.g. for > the backlog queue. > > On Saturday, 3 December 2022 at 05:08:40 UTC harr...@spu.edu wrote: > >> I wonder a bit about io.ReadAll versus constructing a JSON Decoder. In >> general, though, using pprof is the best way to start to break down a >> question like this. Would the actual workload involve more structured JSON, >> or more computation with decoded values? >> >> On Friday, December 2, 2022 at 7:31:50 PM UTC-8 bse...@computer.org >> wrote: >> >>> On Fri, Dec 2, 2022 at 8:13 PM Diogo Baeder wrote: >>> >>>> Hi guys, >>>> >>>> I've been working on some experiments with different web application >>>> stacks to check their performances under a specific scenario: one in which >>>> I have to make several concurrent requests and then gather the results >>>> together (in order) and throw them out as JSON in the response body. (This >>>> project is only an experiment, but it's informing me for decisions that >>>> have to be made for a real-world project where we have a similar scenario.) >>>> >>>> However, probably due to my ignorance in Go, I cannot make it perform >>>> as well as I expected - actually the best I'm getting are results that are >>>> even slower than Python, which was a surprise to me. Here they are: >>>> https://github.com/yougov/concurrency-tests#edit-13-added-golang-with-gin >>>> >>>> So, looking at the code here: >>>> https://github.com/yougov/concurrency-tests/blob/master/stacks/goapp/main.go >>>> >>>> - does anybody see any problem in the implementation that could be hurting >>>> performance? I tried using a WaitGroup, tried sharing memory (nasty, I >>>> know, but just for the sake of experimentation), tried multiple JSON >>>> codecs, different web frameworks, and nothing worked so far. I have a >>>> feeling that I'm doing something fundamentally wrong and stupid, and that >>>> somehow I can make a small change to make the experiment much faster. >>>> >>> >>> Have you measured how much time is spent on the http.Get calls? It is >>> likely that the 50 concurrent http.Get calls is the bottleneck. >>> >>> Also note that you don't need a channel there. You can simply use a >>> waitgroup and set the results from inside the goroutine, because each >>> goroutine knows the index. But that is unlikely to change anything >>> measurable when compared to the Get calls. >>> >>> >>> >>>> >>>> Thanks in advance, I'm sure this will help me learning more about the >>>> language! :-) >>>> >>>> Cheers! >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "golang-nuts" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to golang-nuts...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/golang-nuts/42d7d04f-f6d8-4d96-bcdf-bcf32b99a73cn%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/golang-nuts/42d7d04f-f6d8-4d96-bcdf-bcf32b99a73cn%40googlegroups.com?utm_medium=email_source=footer> >>>> . >>>> >>> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/66d94fac-a7f2-4fd3-a809-b1be1f668181n%40googlegroups.com.
[go-nuts] Performance for concurrent requests
Hi guys, I've been working on some experiments with different web application stacks to check their performances under a specific scenario: one in which I have to make several concurrent requests and then gather the results together (in order) and throw them out as JSON in the response body. (This project is only an experiment, but it's informing me for decisions that have to be made for a real-world project where we have a similar scenario.) However, probably due to my ignorance in Go, I cannot make it perform as well as I expected - actually the best I'm getting are results that are even slower than Python, which was a surprise to me. Here they are: https://github.com/yougov/concurrency-tests#edit-13-added-golang-with-gin So, looking at the code here: https://github.com/yougov/concurrency-tests/blob/master/stacks/goapp/main.go - does anybody see any problem in the implementation that could be hurting performance? I tried using a WaitGroup, tried sharing memory (nasty, I know, but just for the sake of experimentation), tried multiple JSON codecs, different web frameworks, and nothing worked so far. I have a feeling that I'm doing something fundamentally wrong and stupid, and that somehow I can make a small change to make the experiment much faster. Thanks in advance, I'm sure this will help me learning more about the language! :-) Cheers! -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/42d7d04f-f6d8-4d96-bcdf-bcf32b99a73cn%40googlegroups.com.
