Re: [go-nuts] Re: Speed up png.Decode

[Lee Armstrong]

Thanks Rob,

Yes you are right these images are small. <100kb each with an example
below. And as I mentioned I have millions of these.

I am able to max the CPUs out with a worker pool but that is when I noticed
a lot of the work was actually decoding the PNG images which made we wonder
if if was able to reuse decoders at all and even if that would help!


On Sat, 27 Jun 2020 at 01:30, Rob Pike  wrote:

> Without information about the particular images - the data set you're
> working with - it's hard to provide any accurate advice. If the images are
> small, maybe it's all overhead. If the images are of a particular form of
> PNG, maybe a fast path is missing. And so on.
>
> To get meaningful help for problems like this, the more information you
> can provide, the better.
>
> -rob
>
>
> On Fri, Jun 26, 2020 at 10:59 PM Lee Armstrong  wrote:
>
>> Thanks, I am already maxing out some servers but wondered if it could be
>> sped up.
>>
>> I will give the bimg package a go!
>>
>> On Friday, June 26, 2020 at 1:55:40 PM UTC+1 ren...@ix.netcom.com wrote:
>>
>>> Just parallelize in the cloud. At a minimum parallelize locally with
>>> multiple Go routines.
>>>
>>> On Jun 26, 2020, at 7:51 AM, howar...@gmail.com wrote:
>>>
>>> 
>>>
>>> I don't know if the CGo transitions for 16 million images will
>>> completely swamp the speed gains, or how much post-processing you need to
>>> do to these images, but you might try https://github.com/h2non/bimg and
>>> see if it gives you any wins. It claims 'typically 4x faster' then the Go
>>> Image package. It is a CGO interface to libvips, which is an experimental
>>> branch of libspng, which is a performance focused alternative to libpng.
>>>
>>> Howard
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/9f8d3086-75f4-47e1-b49a-4dce057258cco%40googlegroups.com
>>> 
>>> .
>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>
>
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/d694c7d7-8438-4d7c-93ab-46cccaf09b65n%40googlegroups.com
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAC5LoL_yWxgfDUCs5UwStep-dOMoWovpF_RK0K_MuPNzsG6Unw%40mail.gmail.com.

Re: [go-nuts] Re: [generics] default type parameter

[Robert Engels]

Yes I did. Sorry all. 

> On Jun 26, 2020, at 7:42 PM, Sebastien Rosset  wrote:
> 
> 
> Did you reply to the wrong thread?
> 
>> On Friday, June 26, 2020 at 3:09:42 PM UTC-7, Robert Engels wrote:
>> What about an option to disable the signal on the thread as it crosses the 
>> CGo boundary? This seems better than disabling the async preemption entirely?
>> 
 On Jun 26, 2020, at 4:45 PM, Sebastien Rosset  wrote:
 
>>> 
>>> 
>>> Below are the publicly exposed asn1.ObjectIdentifier fields in the 
>>> golang/go repo. It has fairly limited exposure,
>>> but I'm sure some people will argue it's too much (maybe ok in go 2.x but 
>>> not 1.x?).
>>> encoding/asn1:
>>> My guess is this would be primarily used by SNMP applications such as 
>>> https://github.com/soniah/gosnmp/search?q=asn1_q=asn1
>>> crypto/x509/pkix package has 4 exported fields using asn1.ObjectIdentifier
>>> crypto/x509 package:
>>>  ECDSA keys
>>>  Used in the x509.Certificate struct:
>>> // A Certificate represents an X.509 certificate.
>>> type Certificate struct {
>>>   ...
>>>   UnhandledCriticalExtensions []asn1.ObjectIdentifier
>>>   UnknownExtKeyUsage []asn1.ObjectIdentifier
>>>   PolicyIdentifiers []asn1.ObjectIdentifier
>>> }
>>> 
>>> I doubt there are many applications that actually inspect these 3 fields.
>>> 
>>> 
>>> 
 On Friday, June 26, 2020 at 12:53:43 PM UTC-7, Sebastien Rosset wrote:
 As an aside, the most common use of the encoding/asn1 package is most 
 likely crypto/x509. x509. Certificate exposes public fields that use the 
 asn1.ObjectIdentifier, so asn1 ends up being exposed in a lot of 
 applications, such as for TLS connection management.
 
> On Friday, June 26, 2020 at 12:04:09 PM UTC-7, Sebastien Rosset wrote:
> sure, thank you. I will go through the PR review process for asn1 and 
> x509, maybe some good ideas will come up. 
> Sebastien 
> 
>> On Friday, June 26, 2020 at 11:51:05 AM UTC-7, Ian Lance Taylor wrote:
>> On Fri, Jun 26, 2020 at 11:03 AM Sebastien Rosset  
>> wrote: 
>> > 
>> > @ianlancetaylor , thank you for the quick reply. The reason I was 
>> > asking is because potentially this could have been used to fix `type 
>> > ObjectIdentifier []int` in the `encoding/asn1` package and the 
>> > `crypto/x509` package. Currently these package are not fully compliant 
>> > with the ASN.1 specification, which means in practice some 
>> > certificates cannot be parsed. 
>> > 
>> > 
>> > I am trying to fix the encoding/asn1 and crypto/x509 package by adding 
>> > support for OID values that are greater than 2^31. There are multiple 
>> > ways to fix the issues, and unfortunately it won't be possible to 
>> > simply change the ObjectIdentifier type because that would break too 
>> > many applications. If it's not possible to change the type, then most 
>> > alternatives seem to be somewhat cumbersome. For reference the PR is 
>> > https://github.com/golang/go/pull/39795. 
>> 
>> Thanks, understood. 
>> 
>> Generics don't solve all problems.  I agree that there seems to be a 
>> way that we could modify generics to solve this particular problem. 
>> But it means introducing an idea that the rest of the language has 
>> decided to reject: default values for arguments.  I don't think it 
>> would be consistent with the language to permit default values for 
>> type arguments when we do not permit default values for non-type 
>> arguments.  While we don't have to be strictly consistent here, I 
>> think we need a good reason to break consistency.  And in the larger 
>> scheme of things I don't think that making it easier to make a 
>> backward compatible change to one specific package, a package that is 
>> not all that widely used, is a good enough reason. 
>> 
>> I'm not claiming to have the final word, but that is my opinion. 
>> 
>> Ian 
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golan...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/39bf626a-ed31-4c89-bc0f-a685927e7046o%40googlegroups.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/8a0157a1-aea2-489e-99ce-42e5c135a61do%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 

Re: [go-nuts] Re: [generics] default type parameter

[Sebastien Rosset]

Did you reply to the wrong thread?

On Friday, June 26, 2020 at 3:09:42 PM UTC-7, Robert Engels wrote:
>
> What about an option to disable the signal on the thread as it crosses the 
> CGo boundary? This seems better than disabling the async preemption 
> entirely?
>
> On Jun 26, 2020, at 4:45 PM, Sebastien Rosset  > wrote:
>
> 
>
> Below are the publicly exposed asn1.ObjectIdentifier fields in the 
> golang/go repo. It has fairly limited exposure,
> but I'm sure some people will argue it's too much (maybe ok in go 2.x but 
> not 1.x?).
>
>1. encoding/asn1:
>   1. My guess is this would be primarily used by SNMP applications 
>   such as 
>   https://github.com/soniah/gosnmp/search?q=asn1_q=asn1
>   2. crypto/x509/pkix package has 4 exported fields using asn1.
>ObjectIdentifier
>3. crypto/x509 package:
>   1. ECDSA keys
>   2. Used in the x509.Certificate struct:
>
> // A Certificate represents an X.509 certificate.
>
> type Certificate struct {
>
> ...
>
> UnhandledCriticalExtensions []asn1.ObjectIdentifier
>
> UnknownExtKeyUsage []asn1.ObjectIdentifier
>
> PolicyIdentifiers []asn1.ObjectIdentifier
>
> }
>
>
> I doubt there are many applications that actually inspect these 3 fields.
>
>
>
> On Friday, June 26, 2020 at 12:53:43 PM UTC-7, Sebastien Rosset wrote:
>>
>> As an aside, the most common use of the encoding/asn1 package is most 
>> likely crypto/x509. x509. Certificate exposes public fields that use the 
>> asn1.ObjectIdentifier, so asn1 ends up being exposed in a lot of 
>> applications, such as for TLS connection management.
>>
>> On Friday, June 26, 2020 at 12:04:09 PM UTC-7, Sebastien Rosset wrote:
>>>
>>> sure, thank you. I will go through the PR review process for asn1 and 
>>> x509, maybe some good ideas will come up. 
>>> Sebastien 
>>>
>>> On Friday, June 26, 2020 at 11:51:05 AM UTC-7, Ian Lance Taylor wrote:

 On Fri, Jun 26, 2020 at 11:03 AM Sebastien Rosset  
 wrote: 
 > 
 > @ianlancetaylor , thank you for the quick reply. The reason I was 
 asking is because potentially this could have been used to fix `type 
 ObjectIdentifier []int` in the `encoding/asn1` package and the 
 `crypto/x509` package. Currently these package are not fully compliant 
 with 
 the ASN.1 specification, which means in practice some certificates cannot 
 be parsed. 
 > 
 > 
 > I am trying to fix the encoding/asn1 and crypto/x509 package by 
 adding support for OID values that are greater than 2^31. There are 
 multiple ways to fix the issues, and unfortunately it won't be possible to 
 simply change the ObjectIdentifier type because that would break too many 
 applications. If it's not possible to change the type, then most 
 alternatives seem to be somewhat cumbersome. For reference the PR is 
 https://github.com/golang/go/pull/39795. 

 Thanks, understood. 

 Generics don't solve all problems.  I agree that there seems to be a 
 way that we could modify generics to solve this particular problem. 
 But it means introducing an idea that the rest of the language has 
 decided to reject: default values for arguments.  I don't think it 
 would be consistent with the language to permit default values for 
 type arguments when we do not permit default values for non-type 
 arguments.  While we don't have to be strictly consistent here, I 
 think we need a good reason to break consistency.  And in the larger 
 scheme of things I don't think that making it easier to make a 
 backward compatible change to one specific package, a package that is 
 not all that widely used, is a good enough reason. 

 I'm not claiming to have the final word, but that is my opinion. 

 Ian 

>>> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golan...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/39bf626a-ed31-4c89-bc0f-a685927e7046o%40googlegroups.com
>  
> 
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8a0157a1-aea2-489e-99ce-42e5c135a61do%40googlegroups.com.

Re: [go-nuts] Re: Speed up png.Decode

[Rob Pike]

Without information about the particular images - the data set you're
working with - it's hard to provide any accurate advice. If the images are
small, maybe it's all overhead. If the images are of a particular form of
PNG, maybe a fast path is missing. And so on.

To get meaningful help for problems like this, the more information you can
provide, the better.

-rob


On Fri, Jun 26, 2020 at 10:59 PM Lee Armstrong  wrote:

> Thanks, I am already maxing out some servers but wondered if it could be
> sped up.
>
> I will give the bimg package a go!
>
> On Friday, June 26, 2020 at 1:55:40 PM UTC+1 ren...@ix.netcom.com wrote:
>
>> Just parallelize in the cloud. At a minimum parallelize locally with
>> multiple Go routines.
>>
>> On Jun 26, 2020, at 7:51 AM, howar...@gmail.com wrote:
>>
>> 
>>
>> I don't know if the CGo transitions for 16 million images will completely
>> swamp the speed gains, or how much post-processing you need to do to these
>> images, but you might try https://github.com/h2non/bimg and see if it
>> gives you any wins. It claims 'typically 4x faster' then the Go Image
>> package. It is a CGO interface to libvips, which is an experimental branch
>> of libspng, which is a performance focused alternative to libpng.
>>
>> Howard
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/9f8d3086-75f4-47e1-b49a-4dce057258cco%40googlegroups.com
>> 
>> .
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/d694c7d7-8438-4d7c-93ab-46cccaf09b65n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOXNBZTc6nOw3DimkEjMRpvy_HO8K%3D9xqPhFCt2oAYhr%3D0kZTQ%40mail.gmail.com.

Re: [go-nuts] Slices and inconsistency

[David Riley]

On Jun 26, 2020, at 2:59 PM, Ian Lance Taylor  wrote:
> 
> I honestly don't think that append is an example of worse-is-better.
> I think it's an example of preferring to pass and return values rather
> than pointers.  The language could easily have made append take a
> pointer to a slice as its first argument, which would eliminate a
> certain class of bugs.  But it would also mean that you couldn't
> create a slice by simply appending to nil, as in s :=
> append([]int(nil), 1, 2, 3).  You would instead have to declare the
> variable first.

I should clarify that I don't consider it a dig; I think it's a practical 
solution to the problem even if it has its warts, because it is both performant 
and flexible.  Other languages choose either in-place operations (e.g. Java) or 
copies (e.g. Erlang), but you can do a lot more with what's offered in Go.  It 
just takes some getting used to.

> While append does confuse people from time to time, I think it's
> clearly documented, as pointed out upthread.  It's not really all that
> hard to understand.  And once you understand it, it's easy to use.

The "once you understand it" part is really the issue here, because it's not 
immediately obvious if you're not reading the manual.

> I don't think any particular approach is obviously better here, so I
> don't think this is an example of worse-is-better.

The canonical example in Worse Is Better is Unix's solution to the problem 
where you get an interrupt in the middle of a syscall; Unix just said "hey, 
this failed, do it again" instead of going to great lengths to try to resume 
the operation, which was the pragmatic choice even if it led to a somewhat 
quirky interface.

"Worse Is Better" is often characterized more by quirkiness rather than 
outright badness, and Richard Gabriel (LISP purist that he was) was pretty 
upfront about being cheekily hyperbolic about the properties that Unix and C 
had that made them perhaps occasionally inelegant but ultimately more effective 
and survivable.  I think a lot of aspects of Go embody that particular spirit, 
again for good reason considering its progenitors.


- Dave

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/46EE15DC-9728-4DD3-BAAA-F794A78DE123%40gmail.com.

Re: [go-nuts] Slices and inconsistency

[David Riley]

> On Jun 26, 2020, at 2:52 PM, Tyler Compton  wrote:
> 
> On Fri, Jun 26, 2020 at 10:52 AM David Riley  wrote:
>> Also, to this specific point: this exact approach, as with much of Go, 
>> embodies the Bell Labs approach to design (for better or for worse, and with 
>> good reason).  Sometimes we have to live with the artifacts of evolution.
>> 
> One interesting counterexample here is the GC and scheduler, which take on a 
> huge amount of complexity in the implementation to create a dead-simple 
> interface. It seems like Go is willing to take a worse-is-better approach 
> when the amount of interface complexity is relatively small.

Agreed, and this is a good point! Similar for even early Unix and the memory 
allocator.  Some things you have to make a simple interface for or no one will 
use them.


- Dave

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/AB926BC1-A29E-4D77-8DCD-BA022463797E%40gmail.com.

Re: [go-nuts] Re: [generics] default type parameter

[Robert Engels]

What about an option to disable the signal on the thread as it crosses the CGo 
boundary? This seems better than disabling the async preemption entirely?

> On Jun 26, 2020, at 4:45 PM, Sebastien Rosset  wrote:
> 
> 
> 
> Below are the publicly exposed asn1.ObjectIdentifier fields in the golang/go 
> repo. It has fairly limited exposure,
> but I'm sure some people will argue it's too much (maybe ok in go 2.x but not 
> 1.x?).
> encoding/asn1:
> My guess is this would be primarily used by SNMP applications such as 
> https://github.com/soniah/gosnmp/search?q=asn1_q=asn1
> crypto/x509/pkix package has 4 exported fields using asn1.ObjectIdentifier
> crypto/x509 package:
>  ECDSA keys
>  Used in the x509.Certificate struct:
> // A Certificate represents an X.509 certificate.
> type Certificate struct {
>   ...
>   UnhandledCriticalExtensions []asn1.ObjectIdentifier
>   UnknownExtKeyUsage []asn1.ObjectIdentifier
>   PolicyIdentifiers []asn1.ObjectIdentifier
> }
> 
> I doubt there are many applications that actually inspect these 3 fields.
> 
> 
> 
>> On Friday, June 26, 2020 at 12:53:43 PM UTC-7, Sebastien Rosset wrote:
>> As an aside, the most common use of the encoding/asn1 package is most likely 
>> crypto/x509. x509. Certificate exposes public fields that use the 
>> asn1.ObjectIdentifier, so asn1 ends up being exposed in a lot of 
>> applications, such as for TLS connection management.
>> 
>>> On Friday, June 26, 2020 at 12:04:09 PM UTC-7, Sebastien Rosset wrote:
>>> sure, thank you. I will go through the PR review process for asn1 and x509, 
>>> maybe some good ideas will come up. 
>>> Sebastien 
>>> 
 On Friday, June 26, 2020 at 11:51:05 AM UTC-7, Ian Lance Taylor wrote:
 On Fri, Jun 26, 2020 at 11:03 AM Sebastien Rosset  
 wrote: 
 > 
 > @ianlancetaylor , thank you for the quick reply. The reason I was asking 
 > is because potentially this could have been used to fix `type 
 > ObjectIdentifier []int` in the `encoding/asn1` package and the 
 > `crypto/x509` package. Currently these package are not fully compliant 
 > with the ASN.1 specification, which means in practice some certificates 
 > cannot be parsed. 
 > 
 > 
 > I am trying to fix the encoding/asn1 and crypto/x509 package by adding 
 > support for OID values that are greater than 2^31. There are multiple 
 > ways to fix the issues, and unfortunately it won't be possible to simply 
 > change the ObjectIdentifier type because that would break too many 
 > applications. If it's not possible to change the type, then most 
 > alternatives seem to be somewhat cumbersome. For reference the PR is 
 > https://github.com/golang/go/pull/39795. 
 
 Thanks, understood. 
 
 Generics don't solve all problems.  I agree that there seems to be a 
 way that we could modify generics to solve this particular problem. 
 But it means introducing an idea that the rest of the language has 
 decided to reject: default values for arguments.  I don't think it 
 would be consistent with the language to permit default values for 
 type arguments when we do not permit default values for non-type 
 arguments.  While we don't have to be strictly consistent here, I 
 think we need a good reason to break consistency.  And in the larger 
 scheme of things I don't think that making it easier to make a 
 backward compatible change to one specific package, a package that is 
 not all that widely used, is a good enough reason. 
 
 I'm not claiming to have the final word, but that is my opinion. 
 
 Ian 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/39bf626a-ed31-4c89-bc0f-a685927e7046o%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/A8750105-438D-415C-965B-B74578135D53%40ix.netcom.com.

Re: [go-nuts] Re: [generics] default type parameter

[Sebastien Rosset]

Below are the publicly exposed asn1.ObjectIdentifier fields in the 
golang/go repo. It has fairly limited exposure,
but I'm sure some people will argue it's too much (maybe ok in go 2.x but 
not 1.x?).

   1. encoding/asn1:
  1. My guess is this would be primarily used by SNMP applications such 
  as https://github.com/soniah/gosnmp/search?q=asn1_q=asn1
  2. crypto/x509/pkix package has 4 exported fields using asn1.
   ObjectIdentifier
   3. crypto/x509 package:
  1. ECDSA keys
  2. Used in the x509.Certificate struct:
   
// A Certificate represents an X.509 certificate.

type Certificate struct {

...

UnhandledCriticalExtensions []asn1.ObjectIdentifier

UnknownExtKeyUsage []asn1.ObjectIdentifier

PolicyIdentifiers []asn1.ObjectIdentifier

}


I doubt there are many applications that actually inspect these 3 fields.



On Friday, June 26, 2020 at 12:53:43 PM UTC-7, Sebastien Rosset wrote:
>
> As an aside, the most common use of the encoding/asn1 package is most 
> likely crypto/x509. x509. Certificate exposes public fields that use the 
> asn1.ObjectIdentifier, so asn1 ends up being exposed in a lot of 
> applications, such as for TLS connection management.
>
> On Friday, June 26, 2020 at 12:04:09 PM UTC-7, Sebastien Rosset wrote:
>>
>> sure, thank you. I will go through the PR review process for asn1 and 
>> x509, maybe some good ideas will come up. 
>> Sebastien 
>>
>> On Friday, June 26, 2020 at 11:51:05 AM UTC-7, Ian Lance Taylor wrote:
>>>
>>> On Fri, Jun 26, 2020 at 11:03 AM Sebastien Rosset  
>>> wrote: 
>>> > 
>>> > @ianlancetaylor , thank you for the quick reply. The reason I was 
>>> asking is because potentially this could have been used to fix `type 
>>> ObjectIdentifier []int` in the `encoding/asn1` package and the 
>>> `crypto/x509` package. Currently these package are not fully compliant with 
>>> the ASN.1 specification, which means in practice some certificates cannot 
>>> be parsed. 
>>> > 
>>> > 
>>> > I am trying to fix the encoding/asn1 and crypto/x509 package by adding 
>>> support for OID values that are greater than 2^31. There are multiple ways 
>>> to fix the issues, and unfortunately it won't be possible to simply change 
>>> the ObjectIdentifier type because that would break too many applications. 
>>> If it's not possible to change the type, then most alternatives seem to be 
>>> somewhat cumbersome. For reference the PR is 
>>> https://github.com/golang/go/pull/39795. 
>>>
>>> Thanks, understood. 
>>>
>>> Generics don't solve all problems.  I agree that there seems to be a 
>>> way that we could modify generics to solve this particular problem. 
>>> But it means introducing an idea that the rest of the language has 
>>> decided to reject: default values for arguments.  I don't think it 
>>> would be consistent with the language to permit default values for 
>>> type arguments when we do not permit default values for non-type 
>>> arguments.  While we don't have to be strictly consistent here, I 
>>> think we need a good reason to break consistency.  And in the larger 
>>> scheme of things I don't think that making it easier to make a 
>>> backward compatible change to one specific package, a package that is 
>>> not all that widely used, is a good enough reason. 
>>>
>>> I'm not claiming to have the final word, but that is my opinion. 
>>>
>>> Ian 
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/39bf626a-ed31-4c89-bc0f-a685927e7046o%40googlegroups.com.

Re: [go-nuts] Re: [generics] default type parameter

[Ian Lance Taylor]

On Fri, Jun 26, 2020 at 12:54 PM Sebastien Rosset  wrote:
>
> As an aside, the most common use of the encoding/asn1 package is most likely 
> crypto/x509. x509. Certificate exposes public fields that use the 
> asn1.ObjectIdentifier, so asn1 ends up being exposed in a lot of 
> applications, such as for TLS connection management.

True, but will those packages be affected by a change in the encoding/asn1 API?

When I said that the package was not widely used I should have said
that not many other packages import it, and thus not many other
packages are affected by any changes.  I shouldn't have implied that
only a few Go programs import it at all.

Ian


> On Friday, June 26, 2020 at 12:04:09 PM UTC-7, Sebastien Rosset wrote:
>>
>> sure, thank you. I will go through the PR review process for asn1 and x509, 
>> maybe some good ideas will come up.
>> Sebastien
>>
>> On Friday, June 26, 2020 at 11:51:05 AM UTC-7, Ian Lance Taylor wrote:
>>>
>>> On Fri, Jun 26, 2020 at 11:03 AM Sebastien Rosset  wrote:
>>> >
>>> > @ianlancetaylor , thank you for the quick reply. The reason I was asking 
>>> > is because potentially this could have been used to fix `type 
>>> > ObjectIdentifier []int` in the `encoding/asn1` package and the 
>>> > `crypto/x509` package. Currently these package are not fully compliant 
>>> > with the ASN.1 specification, which means in practice some certificates 
>>> > cannot be parsed.
>>> >
>>> >
>>> > I am trying to fix the encoding/asn1 and crypto/x509 package by adding 
>>> > support for OID values that are greater than 2^31. There are multiple 
>>> > ways to fix the issues, and unfortunately it won't be possible to simply 
>>> > change the ObjectIdentifier type because that would break too many 
>>> > applications. If it's not possible to change the type, then most 
>>> > alternatives seem to be somewhat cumbersome. For reference the PR is 
>>> > https://github.com/golang/go/pull/39795.
>>>
>>> Thanks, understood.
>>>
>>> Generics don't solve all problems.  I agree that there seems to be a
>>> way that we could modify generics to solve this particular problem.
>>> But it means introducing an idea that the rest of the language has
>>> decided to reject: default values for arguments.  I don't think it
>>> would be consistent with the language to permit default values for
>>> type arguments when we do not permit default values for non-type
>>> arguments.  While we don't have to be strictly consistent here, I
>>> think we need a good reason to break consistency.  And in the larger
>>> scheme of things I don't think that making it easier to make a
>>> backward compatible change to one specific package, a package that is
>>> not all that widely used, is a good enough reason.
>>>
>>> I'm not claiming to have the final word, but that is my opinion.
>>>
>>> Ian
>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/a34c936f-b68e-4f63-aad2-47c0869f71e0o%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcUzN4S%3Dm2Hr8kmStcLNe7HsDA62mJkcdg8-%2B-Xk7dHG8w%40mail.gmail.com.

[go-nuts] Re: go.pkg.dev: blitznote.com/src/semver doesn't show up

['Carla Pfaff' via golang-nuts]

For me it is in the search results. Currently on page 5, fifth entry:
https://pkg.go.dev/search?page=5=semver

On Friday, 26 June 2020 at 21:47:30 UTC+2 Mark Kubacki wrote:

> Hi,
>
> I've noticed my package, blitznote.com/src/semver, doesn't show up in 
> pkg.go.dev despite having been licensed under one of the blessed 
> licenses, being some months old, and having been pulled several times using 
> the default proxy.
>
> Specifically, searching for “semver” won't produce it, even though after 
> some pages search results are no longer original code (but forks, 
> vendorings).
>
> Going directly through 
> https://pkg.go.dev/blitznote.com/src/semver/v3?tab=doc will work.
>
> Where can I find the requirements for having it (and other modules of 
> mine) included in search results?
>
> Thanks, Mark
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/50fa635e-b029-4a48-94f1-29cd705f080fn%40googlegroups.com.

Re: [go-nuts] Re: [generics] default type parameter

[Sebastien Rosset]

As an aside, the most common use of the encoding/asn1 package is most 
likely crypto/x509. x509. Certificate exposes public fields that use the 
asn1.ObjectIdentifier, so asn1 ends up being exposed in a lot of 
applications, such as for TLS connection management.

On Friday, June 26, 2020 at 12:04:09 PM UTC-7, Sebastien Rosset wrote:
>
> sure, thank you. I will go through the PR review process for asn1 and 
> x509, maybe some good ideas will come up. 
> Sebastien 
>
> On Friday, June 26, 2020 at 11:51:05 AM UTC-7, Ian Lance Taylor wrote:
>>
>> On Fri, Jun 26, 2020 at 11:03 AM Sebastien Rosset  
>> wrote: 
>> > 
>> > @ianlancetaylor , thank you for the quick reply. The reason I was 
>> asking is because potentially this could have been used to fix `type 
>> ObjectIdentifier []int` in the `encoding/asn1` package and the 
>> `crypto/x509` package. Currently these package are not fully compliant with 
>> the ASN.1 specification, which means in practice some certificates cannot 
>> be parsed. 
>> > 
>> > 
>> > I am trying to fix the encoding/asn1 and crypto/x509 package by adding 
>> support for OID values that are greater than 2^31. There are multiple ways 
>> to fix the issues, and unfortunately it won't be possible to simply change 
>> the ObjectIdentifier type because that would break too many applications. 
>> If it's not possible to change the type, then most alternatives seem to be 
>> somewhat cumbersome. For reference the PR is 
>> https://github.com/golang/go/pull/39795. 
>>
>> Thanks, understood. 
>>
>> Generics don't solve all problems.  I agree that there seems to be a 
>> way that we could modify generics to solve this particular problem. 
>> But it means introducing an idea that the rest of the language has 
>> decided to reject: default values for arguments.  I don't think it 
>> would be consistent with the language to permit default values for 
>> type arguments when we do not permit default values for non-type 
>> arguments.  While we don't have to be strictly consistent here, I 
>> think we need a good reason to break consistency.  And in the larger 
>> scheme of things I don't think that making it easier to make a 
>> backward compatible change to one specific package, a package that is 
>> not all that widely used, is a good enough reason. 
>>
>> I'm not claiming to have the final word, but that is my opinion. 
>>
>> Ian 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/a34c936f-b68e-4f63-aad2-47c0869f71e0o%40googlegroups.com.

[go-nuts] go.pkg.dev: blitznote.com/src/semver doesn't show up

[Mark Kubacki]

Hi,

I've noticed my package, blitznote.com/src/semver, doesn't show up in 
pkg.go.dev despite having been licensed under one of the blessed licenses, 
being some months old, and having been pulled several times using the 
default proxy.

Specifically, searching for “semver” won't produce it, even though after 
some pages search results are no longer original code (but forks, 
vendorings).

Going directly through 
https://pkg.go.dev/blitznote.com/src/semver/v3?tab=doc will work.

Where can I find the requirements for having it (and other modules of mine) 
included in search results?

Thanks, Mark

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/2faffdcd-3e5b-4f6d-a00a-55fc872d5296n%40googlegroups.com.

Re: [go-nuts] Re: [generics] default type parameter

[Sebastien Rosset]

sure, thank you. I will go through the PR review process for asn1 and x509, 
maybe some good ideas will come up. 
Sebastien 

On Friday, June 26, 2020 at 11:51:05 AM UTC-7, Ian Lance Taylor wrote:
>
> On Fri, Jun 26, 2020 at 11:03 AM Sebastien Rosset  > wrote: 
> > 
> > @ianlancetaylor , thank you for the quick reply. The reason I was asking 
> is because potentially this could have been used to fix `type 
> ObjectIdentifier []int` in the `encoding/asn1` package and the 
> `crypto/x509` package. Currently these package are not fully compliant with 
> the ASN.1 specification, which means in practice some certificates cannot 
> be parsed. 
> > 
> > 
> > I am trying to fix the encoding/asn1 and crypto/x509 package by adding 
> support for OID values that are greater than 2^31. There are multiple ways 
> to fix the issues, and unfortunately it won't be possible to simply change 
> the ObjectIdentifier type because that would break too many applications. 
> If it's not possible to change the type, then most alternatives seem to be 
> somewhat cumbersome. For reference the PR is 
> https://github.com/golang/go/pull/39795. 
>
> Thanks, understood. 
>
> Generics don't solve all problems.  I agree that there seems to be a 
> way that we could modify generics to solve this particular problem. 
> But it means introducing an idea that the rest of the language has 
> decided to reject: default values for arguments.  I don't think it 
> would be consistent with the language to permit default values for 
> type arguments when we do not permit default values for non-type 
> arguments.  While we don't have to be strictly consistent here, I 
> think we need a good reason to break consistency.  And in the larger 
> scheme of things I don't think that making it easier to make a 
> backward compatible change to one specific package, a package that is 
> not all that widely used, is a good enough reason. 
>
> I'm not claiming to have the final word, but that is my opinion. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/551cacb5-8bef-439d-bb17-2fc8d6b8c357o%40googlegroups.com.

Re: [go-nuts] Slices and inconsistency

[Ian Lance Taylor]

On Fri, Jun 26, 2020 at 11:52 AM Tyler Compton  wrote:
>
> On Fri, Jun 26, 2020 at 10:52 AM David Riley  wrote:
>>
>> Also, to this specific point: this exact approach, as with much of Go, 
>> embodies the Bell Labs approach to design (for better or for worse, and with 
>> good reason).  Sometimes we have to live with the artifacts of evolution.
>
>
> One interesting counterexample here is the GC and scheduler, which take on a 
> huge amount of complexity in the implementation to create a dead-simple 
> interface. It seems like Go is willing to take a worse-is-better approach 
> when the amount of interface complexity is relatively small.

I honestly don't think that append is an example of worse-is-better.
I think it's an example of preferring to pass and return values rather
than pointers.  The language could easily have made append take a
pointer to a slice as its first argument, which would eliminate a
certain class of bugs.  But it would also mean that you couldn't
create a slice by simply appending to nil, as in s :=
append([]int(nil), 1, 2, 3).  You would instead have to declare the
variable first.

While append does confuse people from time to time, I think it's
clearly documented, as pointed out upthread.  It's not really all that
hard to understand.  And once you understand it, it's easy to use.

I don't think any particular approach is obviously better here, so I
don't think this is an example of worse-is-better.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcVCGBdcA%2BHe28UqycJJb_GiG7LcV%2BP4WjHjvcAEDMrFJw%40mail.gmail.com.

Re: [go-nuts] How to do "frame" based profiling in GO?

[Michael Nischt]

Hi, thanks for the quick reply.

I tried that but I fear I don't really understand the API/tools for that.

What I did was wrapping the update function called every frame like that:
func profUpdate(update func(time.Duration), frame *uint64) 
func(time.Duration) {
  return func(time.Duration) {
frameID := *frame
*frame++
labels := pprof.Labels("frame", fmt.Sprintf("%v", frameID))
pprof.Do(context.Background(), labels, func(ctx context.Context) {
  update(dT)
})
  }
}

Maybe the background context is the problem but after running the server 
for 1 minute only three tags are listed for frames 998, 1724 and 1967:
go tool pprof -tags logs/game-server.prof 
frame: Total 30.0ms
 10.0ms (33.33%): 1724 
 10.0ms (33.33%): 1967 
 10.0ms (33.33%): 998 

and there appearance in the graph is not really helpful either: 
https://i.imgur.com/eo9qgfv.png

Even if it had all frames, wouldn't each function call would have each 
frame label instead of splitting the graph by frames, right?
(at least it looks like that in this example: 
https://rakyll.org/profiler-labels/ - and each function with let's say 100 
frame labels would already be impossible to read I fear).

Further, the labels don't even appear in the flame graph. let alone having 
it sliced by frame.

I really hope it's just that I simply don't understand how to use the API. 

However, I'm somewhat afraid frame based profiling is not really a common 
use case in GO and there's not good way to do it without writing an own 
solution.
Best,
 Michael
On Friday, June 26, 2020 at 8:45:44 PM UTC+2 ren...@ix.netcom.com wrote:

> You want to use pprof “labels” and mark the frames. 
>
> On Jun 26, 2020, at 1:35 PM, michael...@gmail.com wrote:
>
> 
>
> Hello,
>
> I'm trying to figure out how to do "frame" based profiling in GO and was 
> redirect to this mailing--list from the `r/golang` sub (topic 
> ).
>  
> Would be amazing if anyone here can help.
>
> In short: I have a game server in Go and need to profile CPU costs of 
> individual frames but sadly can't find a way to do that properly. 
>
> I thought that the `pprof` CPU profile should be the right tool. However, 
> at least the profile visualization tools always aggregate the methods 
> execution costs between `StartCPUProfile` and `StopCPUProfile`. While in 
> some scenarios this is great, we have currently have big spikes in for 
> frame duration and would love to see the difference in times per frame. 
> Ideally in a flame-graph like visualization but grouped by frames. So we'd 
> somehow need to mark the frame start/end and tell the visualization to use 
> that.
>
> Ideally, like the profiler in the Unity3D Editor does:
> [image: Post image] 
> 
>
> Unity3D Profiling - Per frame method execution times
>
> I tried use `Start/StopCPUProfile` at the beginning and end of a frame to 
> create an individual capture per frame. That's not ideal for comparison but 
> should be good enough and simple in concept. Sadly the cost of 
> `StartCPUProfile` is so high, we can't call that per frame.
>
> Looking at the code, the the 100 millisecond sleep in `profileWriter` 
> explains that it. So what we'd need is a way to mark the frame start/end 
> somehow for visual grouping. Is that possible with `pprof`?
> [image: Post image] 
> 
>
> pprof.go
>
> Further, I wonder if it's even possible to profile a game server with a 
> simulation framerate of 120Hz. Since the 100Hz CPU profile rate is hard 
> coded in `StartCPUProfile` I would guess we even per frame aggregation it 
> make not work of a frame duration is usually below 1s / 120 or is that not 
> relevant?
>
> Best,
>  Michael
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/d0e8b1ae-d992-44ed-bd9a-6a72e6526d4co%40googlegroups.com
>  
> 
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0a2cbe39-6246-4f6e-9add-a3644ad67017n%40googlegroups.com.

Re: [go-nuts] Slices and inconsistency

[Tyler Compton]

On Fri, Jun 26, 2020 at 10:52 AM David Riley  wrote:

> Also, to this specific point: this exact approach, as with much of Go,
> embodies the Bell Labs approach to design (for better or for worse, and
> with good reason).  Sometimes we have to live with the artifacts of
> evolution.
>

One interesting counterexample here is the GC and scheduler, which take on
a huge amount of complexity in the implementation to create a dead-simple
interface. It seems like Go is willing to take a worse-is-better approach
when the amount of interface complexity is relatively small.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAA%3DXfu3xWV94g5smX%3D6kk3QD9fzZN_nhwqF0XykVsmUBDKLTLw%40mail.gmail.com.

Re: [go-nuts] Re: [generics] default type parameter

[Ian Lance Taylor]

On Fri, Jun 26, 2020 at 11:03 AM Sebastien Rosset  wrote:
>
> @ianlancetaylor , thank you for the quick reply. The reason I was asking is 
> because potentially this could have been used to fix `type ObjectIdentifier 
> []int` in the `encoding/asn1` package and the `crypto/x509` package. 
> Currently these package are not fully compliant with the ASN.1 specification, 
> which means in practice some certificates cannot be parsed.
>
>
> I am trying to fix the encoding/asn1 and crypto/x509 package by adding 
> support for OID values that are greater than 2^31. There are multiple ways to 
> fix the issues, and unfortunately it won't be possible to simply change the 
> ObjectIdentifier type because that would break too many applications. If it's 
> not possible to change the type, then most alternatives seem to be somewhat 
> cumbersome. For reference the PR is https://github.com/golang/go/pull/39795.

Thanks, understood.

Generics don't solve all problems.  I agree that there seems to be a
way that we could modify generics to solve this particular problem.
But it means introducing an idea that the rest of the language has
decided to reject: default values for arguments.  I don't think it
would be consistent with the language to permit default values for
type arguments when we do not permit default values for non-type
arguments.  While we don't have to be strictly consistent here, I
think we need a good reason to break consistency.  And in the larger
scheme of things I don't think that making it easier to make a
backward compatible change to one specific package, a package that is
not all that widely used, is a good enough reason.

I'm not claiming to have the final word, but that is my opinion.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcWD3wYG13pNUociK6vwHOsCH5X_05b9wEhxKcDu8_o0MQ%40mail.gmail.com.

Re: [go-nuts] How to do "frame" based profiling in GO?

[Robert Engels]

You want to use pprof “labels” and mark the frames. 

> On Jun 26, 2020, at 1:35 PM, michael.nis...@gmail.com wrote:
> 
> 
> Hello,
> 
> I'm trying to figure out how to do "frame" based profiling in GO and was 
> redirect to this mailing--list from the `r/golang` sub (topic). Would be 
> amazing if anyone here can help.
> 
> In short: I have a game server in Go and need to profile CPU costs of 
> individual frames but sadly can't find a way to do that properly. 
> 
> I thought that the `pprof` CPU profile should be the right tool. However, at 
> least the profile visualization tools always aggregate the methods execution 
> costs between `StartCPUProfile` and `StopCPUProfile`. While in some scenarios 
> this is great, we have currently have big spikes in for frame duration and 
> would love to see the difference in times per frame. Ideally in a flame-graph 
> like visualization but grouped by frames. So we'd somehow need to mark the 
> frame start/end and tell the visualization to use that.
> 
> Ideally, like the profiler in the Unity3D Editor does:
> 
> 
> Unity3D Profiling - Per frame method execution times
> 
> I tried use `Start/StopCPUProfile` at the beginning and end of a frame to 
> create an individual capture per frame. That's not ideal for comparison but 
> should be good enough and simple in concept. Sadly the cost of 
> `StartCPUProfile` is so high, we can't call that per frame.
> 
> Looking at the code, the the 100 millisecond sleep in `profileWriter` 
> explains that it. So what we'd need is a way to mark the frame start/end 
> somehow for visual grouping. Is that possible with `pprof`?
> 
> 
> pprof.go
> 
> Further, I wonder if it's even possible to profile a game server with a 
> simulation framerate of 120Hz. Since the 100Hz CPU profile rate is hard coded 
> in `StartCPUProfile` I would guess we even per frame aggregation it make not 
> work of a frame duration is usually below 1s / 120 or is that not relevant?
> 
> Best,
>  Michael
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/d0e8b1ae-d992-44ed-bd9a-6a72e6526d4co%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/36D99174-9694-4664-9493-416087FBBA91%40ix.netcom.com.

[go-nuts] How to do "frame" based profiling in GO?

[michael . nischt]

Hello,

I'm trying to figure out how to do "frame" based profiling in GO and was 
redirect to this mailing--list from the `r/golang` sub (topic 
).
 
Would be amazing if anyone here can help.

In short: I have a game server in Go and need to profile CPU costs of 
individual frames but sadly can't find a way to do that properly. 

I thought that the `pprof` CPU profile should be the right tool. However, 
at least the profile visualization tools always aggregate the methods 
execution costs between `StartCPUProfile` and `StopCPUProfile`. While in 
some scenarios this is great, we have currently have big spikes in for 
frame duration and would love to see the difference in times per frame. 
Ideally in a flame-graph like visualization but grouped by frames. So we'd 
somehow need to mark the frame start/end and tell the visualization to use 
that.

Ideally, like the profiler in the Unity3D Editor does:
[image: Post image] 


Unity3D Profiling - Per frame method execution times

I tried use `Start/StopCPUProfile` at the beginning and end of a frame to 
create an individual capture per frame. That's not ideal for comparison but 
should be good enough and simple in concept. Sadly the cost of 
`StartCPUProfile` is so high, we can't call that per frame.

Looking at the code, the the 100 millisecond sleep in `profileWriter` 
explains that it. So what we'd need is a way to mark the frame start/end 
somehow for visual grouping. Is that possible with `pprof`?
[image: Post image] 


pprof.go

Further, I wonder if it's even possible to profile a game server with a 
simulation framerate of 120Hz. Since the 100Hz CPU profile rate is hard 
coded in `StartCPUProfile` I would guess we even per frame aggregation it 
make not work of a frame duration is usually below 1s / 120 or is that not 
relevant?

Best,
 Michael

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d0e8b1ae-d992-44ed-bd9a-6a72e6526d4co%40googlegroups.com.

[go-nuts] Re: [generics] default type parameter

[Sebastien Rosset]

@ianlancetaylor , thank you for the quick reply. The reason I was asking is 
because potentially this could have been used to fix `type ObjectIdentifier 
[]int` in the `encoding/asn1` package and the `crypto/x509` package. 
Currently these package are not fully compliant with the ASN.1 
specification, which means in practice some certificates cannot be parsed.

I am trying to fix the encoding/asn1 and crypto/x509 package by adding support 
for OID values that are greater than 2^31. There are multiple ways to fix 
the issues, and unfortunately it won't be possible to simply change the 
ObjectIdentifier type because that would break too many applications. If 
it's not possible to change the type, then most alternatives seem to be 
somewhat cumbersome. For reference the PR is 
https://github.com/golang/go/pull/39795.


Sebastien



On Friday, June 26, 2020 at 10:55:39 AM UTC-7, Sebastien Rosset wrote:
>
> First apologies for initially bringing up the topic as a github issue 
> rather than golang-nuts. I am copy-pasting my question here with 
> @ianlancetaylor 
> response.
>
>
> A default type value can be specified in C++ templates. I was asking if a 
> similar construct been considered for go generic type parameters. 
> Potentially this would make it possible to retrofit existing types without 
> breaking existing code.
>
> For example, consider the existing asn1.ObjectIdentifier type which is a 
> []int. One problem with this type is it's not compliant with the ASN.1 
> specification, which states each sub-oid may be an INTEGER of arbitrary 
> length (e.g. *big.Int). Potentially ObjectIdentifier could be modified to 
> accept a generic parameter, but that would break a lot of existing code. If 
> there was a way to specify int is the default parameter value, maybe that 
> would make it possible to retrofit existing code.
>
> type SignedInteger interface {
>   type int, int32, int64, *big.Int
> }type ObjectIdentifier(type T SignedInteger) []T// type ObjectIdentifier(type 
> T SignedInteger=int) []T  // `int` would be the default instantiation type.
> // New code with generic awareness would compile in go2.var oid1 
> ObjectIdentifier(int) = ObjectIdentifier(int){1, 2, 3}
> // But existing code would fail to compile:var oid1 ObjectIdentifier = 
> ObjectIdentifier{1, 2, 3}
>
> Just to be clear, the above asn1.ObjectIdentifier is just an example. I'm 
> not saying using generics is the only way or the best way to solve the 
> ASN.1 compliance issue.
>
>
>
> > @sebastien-rosset We have not considered default types for generic type 
> parameters. The language does not have default values for function 
> arguments, and it's not obvious why generics would be different. In my 
> opinion, the ability to make existing code compatible with a package that 
> adds generics is not a priority. If a package is rewritten to use generics, 
> it's OK to require existing code to change, or to simply introduce the 
> generic code using new names.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b29e342c-bcd1-4fc3-baf9-431d96120674o%40googlegroups.com.

[go-nuts] [generics] default type parameter

[Sebastien Rosset]

First apologies for initially bringing up the topic as a github issue 
rather than golang-nuts. I am copy-pasting my question here with 
@ianlancetaylor 
response.


A default type value can be specified in C++ templates. I was asking if a 
similar construct been considered for go generic type parameters. 
Potentially this would make it possible to retrofit existing types without 
breaking existing code.

For example, consider the existing asn1.ObjectIdentifier type which is a 
[]int. One problem with this type is it's not compliant with the ASN.1 
specification, which states each sub-oid may be an INTEGER of arbitrary 
length (e.g. *big.Int). Potentially ObjectIdentifier could be modified to 
accept a generic parameter, but that would break a lot of existing code. If 
there was a way to specify int is the default parameter value, maybe that 
would make it possible to retrofit existing code.

type SignedInteger interface {
type int, int32, int64, *big.Int
}type ObjectIdentifier(type T SignedInteger) []T// type ObjectIdentifier(type T 
SignedInteger=int) []T  // `int` would be the default instantiation type.
// New code with generic awareness would compile in go2.var oid1 
ObjectIdentifier(int) = ObjectIdentifier(int){1, 2, 3}
// But existing code would fail to compile:var oid1 ObjectIdentifier = 
ObjectIdentifier{1, 2, 3}

Just to be clear, the above asn1.ObjectIdentifier is just an example. I'm 
not saying using generics is the only way or the best way to solve the 
ASN.1 compliance issue.



> @sebastien-rosset We have not considered default types for generic type 
parameters. The language does not have default values for function 
arguments, and it's not obvious why generics would be different. In my 
opinion, the ability to make existing code compatible with a package that 
adds generics is not a priority. If a package is rewritten to use generics, 
it's OK to require existing code to change, or to simply introduce the 
generic code using new names.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/cfe946b9-2ce9-4b7a-acf7-0b45e3a4bf32o%40googlegroups.com.

Re: [go-nuts] Slices and inconsistency

[David Riley]

On Jun 26, 2020, at 1:41 PM, Robert Engels  wrote:
> 
> My opinion is that if RTFM is required more than once for a core concept 
> there may be a design problem. It clearly bites a lot of people. Slices are a 
> higher level struct, as the underlying array is the same as Java, but Java 
> doesn’t suffer these knowledge gaps. I’m guessing because it’s higher but not 
> high enough. Done for performance while forsaking some safety and clarity.

Also, to this specific point: this exact approach, as with much of Go, embodies 
the Bell Labs approach to design (for better or for worse, and with good 
reason).  Sometimes we have to live with the artifacts of evolution.

https://www.jwz.org/doc/worse-is-better.html


- Dave


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/6E11D888-40DA-4606-B698-8509CAE5502F%40gmail.com.

Re: [go-nuts] Slices and inconsistency

[David Riley]

On Jun 26, 2020, at 1:41 PM, Robert Engels  wrote:
> 
> I don’t think “how it works” is the confusion, more of “how to use it 
> properly”
> 
> My opinion is that if RTFM is required more than once for a core concept 
> there may be a design problem. It clearly bites a lot of people. Slices are a 
> higher level struct, as the underlying array is the same as Java, but Java 
> doesn’t suffer these knowledge gaps. I’m guessing because it’s higher but not 
> high enough. Done for performance while forsaking some safety and clarity.  

Frankly, I think the problem is with append().  I understand why it's 
structured the way it is, and yes, I agree the documentation is fairly clear 
about how it works *if you read it*, but we need to be honest with ourselves 
about how people read the documentation (or don't).  The problem with append() 
is that it returns a value, and if you only learn by example or quickly skim 
through the docs (as Howard pointed out), it's not going to be immediately 
apparently that it's not necessarily a new slice or array.

I don't say this because I think append() should be modified to be an in-place 
operator (that would be impractical and break a lot of things that rely on 
current behavior) but it shouldn't be mysterious to us that people coming from 
every language that has a variable-length vector think of it incorrectly 
because that's how the vast majority of them treat an append operation (either 
as an in-place or as a copy, and we've kind of split the difference).

I wish I had a more constructive answer to this, because I guess you can't make 
other people's tutorials call this out, and using the return value as a 
different slice is a valid thing to do even if it is inadvisable without doing 
more elaborate checking on the capacity, etc, so it's kind of hard to put in a 
linter.


- Dave

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7E5E5A28-4CAE-41DF-A354-320CC7F7ED87%40gmail.com.

Re: [go-nuts] Slices and inconsistency

[Robert Engels]

I don’t think “how it works” is the confusion, more of “how to use it properly”

My opinion is that if RTFM is required more than once for a core concept there 
may be a design problem. It clearly bites a lot of people. Slices are a higher 
level struct, as the underlying array is the same as Java, but Java doesn’t 
suffer these knowledge gaps. I’m guessing because it’s higher but not high 
enough. Done for performance while forsaking some safety and clarity.  

> On Jun 26, 2020, at 12:28 PM, howardcs...@gmail.com wrote:
> 
> 
> "If the capacity of s is not large enough to fit the additional values, 
> append allocates a new, sufficiently large underlying array that fits both 
> the existing slice elements and the additional values. Otherwise, append 
> re-uses the underlying array."
> 
> That seems pretty explicit to me? That's from the spec, the other link above, 
> the blog, is even more explicit, by straight up showing the implementation 
> before introducing the built-in.
> 
> The tour also says it: https://tour.golang.org/moretypes/15 
> "If the backing array of s is too small to fit all the given values a bigger 
> array will be allocated. The returned slice will point to the newly allocated 
> array."
> 
> It also links to this documentation: https://blog.golang.org/slices-intro 
> "The append function appends the elements x to the end of the slice s, and 
> grows the slice if a greater capacity is needed." - this is the only one that 
> seems less than explicit about the semantics, and in the sense that it does 
> not make explicit that 'grows the slice' means 'returns a copy of the slice 
> with greater capacity.' 
> 
> 
> The latter link then links further to this documentation: 
> https://golang.org/doc/effective_go.html#slices
> and just like the blog, it starts by introducing the implementation, where it 
> is clear that only one path includes a copy function, and it explicitly says 
> "If the data exceeds the capacity, the slice is reallocated." A bit later 
> where it covers append directly, it says "The result needs to be returned 
> because, as with our hand-written Append, the underlying array may change." 
> 
> 4 bits of official documentation, all of which mention the 'gotcha' more or 
> less directly. And in most of those cases, discussion of append directly 
> follows discussion of 'copy', which should make it clear how to get copies of 
> slices.
> 
> I think maybe where it gets missed is from people learning the language by 
> example, by coping code from StackOverflow, reading code in GitHub, etc. Go 
> is pretty surprisingly easy to pick up by example like that, but it does mean 
> those corner cases can bite - however, the actual documentation is pretty 
> clear and up-front about the gotchas.
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/f98d538e-55d5-4a61-b7d2-1ab9497d3cd5o%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/B7116805-340B-4F43-A7F3-69B3A8D71011%40ix.netcom.com.

Re: [go-nuts] Slices and inconsistency

[howardcshaw]

"If the capacity of s is not large enough to fit the additional values, 
append allocates a new, sufficiently large underlying array that fits both 
the existing slice elements and the additional values. Otherwise, append 
re-uses the underlying array."

That seems pretty explicit to me? That's from the spec, the other link 
above, the blog, is even more explicit, by straight up showing the 
implementation before introducing the built-in. 

The tour also says it: https://tour.golang.org/moretypes/15 
"If the backing array of s is too small to fit all the given values a 
bigger array will be allocated. The returned slice will point to the newly 
allocated array."

It also links to this documentation: https://blog.golang.org/slices-intro 
"The append function appends the elements x to the end of the slice s, and 
grows the slice if a greater capacity is needed." - this is the only one 
that seems less than explicit about the semantics, and in the sense that it 
does not make explicit that 'grows the slice' means 'returns a copy of the 
slice with greater capacity.' 


The latter link then links further to this documentation: 
https://golang.org/doc/effective_go.html#slices
and just like the blog, it starts by introducing the implementation, where 
it is clear that only one path includes a copy function, and it explicitly 
says "If the data exceeds the capacity, the slice is reallocated." A bit 
later where it covers append directly, it says "The result needs to be 
returned because, as with our hand-written Append, the underlying array may 
change." 

4 bits of official documentation, all of which mention the 'gotcha' more or 
less directly. And in most of those cases, discussion of append directly 
follows discussion of 'copy', which should make it clear how to get copies 
of slices.

I think maybe where it gets missed is from people learning the language by 
example, by coping code from StackOverflow, reading code in GitHub, etc. Go 
is pretty surprisingly easy to pick up by example like that, but it does 
mean those corner cases can bite - however, the actual documentation is 
pretty clear and up-front about the gotchas.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/f98d538e-55d5-4a61-b7d2-1ab9497d3cd5o%40googlegroups.com.

Re: [go-nuts] Re: accessibility of module version information in executable

[Marvin Renich]

* seank...@gmail.com  [200626 12:31]:
> go version -m $binary
> 
> runtime/debug.BuildInfo

Thanks much!  I looked through runtime, but did not think to look at
runtime/debug.  I was completely unaware of the -m option to go version.

...Marvin

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/20200626165309.t33llcl6h5klz2e7%40basil.wdw.

[go-nuts] Re: accessibility of module version information in executable

[seank...@gmail.com]

go version -m $binary

runtime/debug.BuildInfo

On Friday, June 26, 2020 at 6:13:56 PM UTC+2 Marvin Renich wrote:

> When building a command with go build that imports modules, does the
> resulting executable contain information about which modules are
> included as well as their versions?
>
> If so, is there a tool available to extract that information from the
> executable? Can the information be retrieved from within the program
> (e.g. through the runtime package)?
>
> If not, would a request for this be received favorably?
>
> ...Marvin
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/a2816324-6b9d-4e98-b2da-7c31c8de2d05n%40googlegroups.com.

[go-nuts] accessibility of module version information in executable

[Marvin Renich]

When building a command with go build that imports modules, does the
resulting executable contain information about which modules are
included as well as their versions?

If so, is there a tool available to extract that information from the
executable?  Can the information be retrieved from within the program
(e.g. through the runtime package)?

If not, would a request for this be received favorably?

...Marvin

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/20200626161324.epfevcf53q76ix4d%40basil.wdw.

Re: [go-nuts] Slices and inconsistency

[David Riley]

> On Jun 26, 2020, at 9:28 AM, Robert Engels  wrote:
> 
>> On Jun 26, 2020, at 8:23 AM, David Riley  wrote:
>> 
>> You are correct, the programmer needs to read the manual. Slices are 
>> "windows" into their underlying array, therefore assigning data into slices 
>> tied to a given array changes the data in other arrays as well.  The 
>> append() function merely extends the slice by n and assigns into those 
>> elements, returning the new slice header (the old one can remain unchanged, 
>> but they will both be slices of the same array).
>> 
>> Ian pointed to the canonical documentation on this: 
>> https://blog.golang.org/slices
>> 
>> Additionally, the language spec has this: 
>> https://golang.org/ref/spec#Appending_and_copying_slices
>> 
>> I do agree that it would be nicer if this were flagged a little more loudly, 
>> since the semantics of the append() builtin can lead the early developer to 
>> believe they are making a copy of the slice in question, but they aren't.
>> 
> That is not quite true. Once is it extended beyond the current capacity you 
> will have a new backing array and the slices will diverge.

Agreed, but the problem here is that unless you are checking the capacity 
beforehand, you cannot count on that behavior.  The general point is that if 
you want a modified copy of an array/slice, you need to be explicit about it or 
you will run into unpredictable and unpleasant to debug problems, but that is 
not immediately apparent given the superficial semantics of the append() 
builtin unless you understand slices at a deeper level.

As in all things, the proper fix is to RTFM, but I think TFM could afford to be 
a *tiny* bit more explicit about this because it's surprisingly easy to miss.


- Dave



-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/222003D0-79C2-46D8-B0CE-41CF0342BE78%40gmail.com.

Re: [go-nuts] Slices and inconsistency

[Robert Engels]

That is not quite true. Once is it extended beyond the current capacity you 
will have a new backing array and the slices will diverge. 

> On Jun 26, 2020, at 8:23 AM, David Riley  wrote:
> 
> On Jun 25, 2020, at 8:49 PM, chandrak13...@gmail.com wrote:
>> 
>> Essentially based on the existing capacity, the assignment of one slice 
>> effects other slices. These are stemming from the underlying pointer 
>> arithmetic and seems inconsistent. Looks like programmer needs to know the 
>> history of capacity before understanding the ramifications of slice 
>> assignments.
> 
> You are correct, the programmer needs to read the manual. Slices are 
> "windows" into their underlying array, therefore assigning data into slices 
> tied to a given array changes the data in other arrays as well.  The append() 
> function merely extends the slice by n and assigns into those elements, 
> returning the new slice header (the old one can remain unchanged, but they 
> will both be slices of the same array).
> 
> Ian pointed to the canonical documentation on this: 
> https://blog.golang.org/slices
> 
> Additionally, the language spec has this: 
> https://golang.org/ref/spec#Appending_and_copying_slices
> 
> I do agree that it would be nicer if this were flagged a little more loudly, 
> since the semantics of the append() builtin can lead the early developer to 
> believe they are making a copy of the slice in question, but they aren't.
> 
> 
> - Dave
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CA9014AE-527F-4D18-81A9-6E63FAD97B54%40gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/725CB0E1-4D82-41F7-9AC9-A3AD30ED1CF5%40ix.netcom.com.

Re: [go-nuts] Slices and inconsistency

[David Riley]

On Jun 25, 2020, at 8:49 PM, chandrak13...@gmail.com wrote:
> 
> Essentially based on the existing capacity, the assignment of one slice 
> effects other slices. These are stemming from the underlying pointer 
> arithmetic and seems inconsistent. Looks like programmer needs to know the 
> history of capacity before understanding the ramifications of slice 
> assignments.

You are correct, the programmer needs to read the manual. Slices are "windows" 
into their underlying array, therefore assigning data into slices tied to a 
given array changes the data in other arrays as well.  The append() function 
merely extends the slice by n and assigns into those elements, returning the 
new slice header (the old one can remain unchanged, but they will both be 
slices of the same array).

Ian pointed to the canonical documentation on this: 
https://blog.golang.org/slices

Additionally, the language spec has this: 
https://golang.org/ref/spec#Appending_and_copying_slices

I do agree that it would be nicer if this were flagged a little more loudly, 
since the semantics of the append() builtin can lead the early developer to 
believe they are making a copy of the slice in question, but they aren't.


- Dave


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CA9014AE-527F-4D18-81A9-6E63FAD97B54%40gmail.com.

Re: [go-nuts] Re: Speed up png.Decode

[Lee Armstrong]

Thanks, I am already maxing out some servers but wondered if it could be 
sped up.

I will give the bimg package a go!

On Friday, June 26, 2020 at 1:55:40 PM UTC+1 ren...@ix.netcom.com wrote:

> Just parallelize in the cloud. At a minimum parallelize locally with 
> multiple Go routines. 
>
> On Jun 26, 2020, at 7:51 AM, howar...@gmail.com wrote:
>
> 
>
> I don't know if the CGo transitions for 16 million images will completely 
> swamp the speed gains, or how much post-processing you need to do to these 
> images, but you might try https://github.com/h2non/bimg and see if it 
> gives you any wins. It claims 'typically 4x faster' then the Go Image 
> package. It is a CGO interface to libvips, which is an experimental branch 
> of libspng, which is a performance focused alternative to libpng.
>
> Howard
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/9f8d3086-75f4-47e1-b49a-4dce057258cco%40googlegroups.com
>  
> 
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d694c7d7-8438-4d7c-93ab-46cccaf09b65n%40googlegroups.com.

Re: [go-nuts] Re: Speed up png.Decode

[Robert Engels]

Just parallelize in the cloud. At a minimum parallelize locally with multiple 
Go routines. 

> On Jun 26, 2020, at 7:51 AM, howardcs...@gmail.com wrote:
> 
> 
> I don't know if the CGo transitions for 16 million images will completely 
> swamp the speed gains, or how much post-processing you need to do to these 
> images, but you might try https://github.com/h2non/bimg and see if it gives 
> you any wins. It claims 'typically 4x faster' then the Go Image package. It 
> is a CGO interface to libvips, which is an experimental branch of libspng, 
> which is a performance focused alternative to libpng.
> 
> Howard
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/9f8d3086-75f4-47e1-b49a-4dce057258cco%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/4602D226-D923-4F93-A256-429D8657F5FE%40ix.netcom.com.

[go-nuts] Re: Speed up png.Decode

[howardcshaw]

I don't know if the CGo transitions for 16 million images will completely 
swamp the speed gains, or how much post-processing you need to do to these 
images, but you might try https://github.com/h2non/bimg and see if it gives 
you any wins. It claims 'typically 4x faster' then the Go Image package. It 
is a CGO interface to libvips, which is an experimental branch of libspng, 
which is a performance focused alternative to libpng.

Howard

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9f8d3086-75f4-47e1-b49a-4dce057258cco%40googlegroups.com.