Re: [go-nuts] Golang and virustotal

2022-05-11 Thread Klaus Post
Hi! You can try 'garble' and see if that helps: https://github.com/burrowers/garble For me, it seems to reduce the number of false positives, but there are still some left. /Klaus -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To

[go-nuts] Re: Constant CPU usage on an idle HTTP server from Go 1.17

2022-04-28 Thread Klaus Post
This "smells" a lot like https://github.com/golang/go/issues/51654 - though the repro is with TLS. This is very disruptive for us (MinIO) as well, seeing this at several customers. Obviously downgrading to Go 1.16.6 is less than optimal for us. /Klaus On Tuesday, 12 April 2022 at 14:55:41

Re: [go-nuts] GOAMD64 build tags?

2022-02-02 Thread Klaus Post
On Tuesday, 1 February 2022 at 21:12:08 UTC+1 Ian Lance Taylor wrote: > On Tue, Feb 1, 2022 at 4:34 AM Klaus Post wrote: > > > > I looked through the source to see if `GOAMD64` version results in build > tags being set, but I was unsure if that is the case. > >

[go-nuts] GOAMD64 build tags?

2022-02-01 Thread Klaus Post
Hi! I looked through the source to see if `GOAMD64` version results in build tags being set, but I was unsure if that is the case. It would be really helpful for assembly code that can be assumed to work without cpuid checks, avoiding a bit of branching. For example adding

[go-nuts] Re: Go 1.14 Release Candidate 1 is released

2020-02-07 Thread Klaus Post
Hi! I am seeing a generally negative impact of Go 1.13 -> 1.14-RC1 in terms of speed. Running benchmarks in my deflate package - and removing the "no change" entries: nameold time/opnew time/opdelta DecodeDigitsSpeed1e5-12903µs ± 0% 940µs ± 1%

Re: [go-nuts] S2: High speed compression with upgrade path from Snappy.

2019-08-28 Thread Klaus Post
Also, while there was other opportunities for making it more effective (3 byte offsets, even shorter repeat codes), I choose to keep it as close to Snappy as feasible so the same decoder works for both formats. Re "better (compression)": Yeah, it does require to read the docs. Oh well.

Re: [go-nuts] S2: High speed compression with upgrade path from Snappy.

2019-08-28 Thread Klaus Post
On Wednesday, 28 August 2019 12:57:33 UTC+2, Nigel Tao wrote: > > On Wed, Aug 28, 2019 at 7:11 PM Klaus Post > wrote: > > TLDR; LZ4 is typically between the default and "better" mode of s2. > > Nice! > > Just a suggestion: rename "better" t

Re: [go-nuts] S2: High speed compression with upgrade path from Snappy.

2019-08-28 Thread Klaus Post
On Wednesday, 28 August 2019 02:37:25 UTC+2, Nigel Tao wrote: > > On Mon, Aug 26, 2019 at 8:29 PM Klaus Post > wrote: > > This package is aimed at replacing Snappy as a high speed compression > package. If you are mainly looking for better compression zstandard gives &g

Re: [go-nuts] S2: High speed compression with upgrade path from Snappy.

2019-08-27 Thread Klaus Post
On Tuesday, 27 August 2019 10:38:36 UTC+2, Nigel Tao wrote: > > Nice work, Klaus! > > > On Mon, Aug 26, 2019 at 8:29 PM Klaus Post > wrote: > > Concurrent stream compression - several GB/sec. > > Faster decompression > > A number of modern compressi

[go-nuts] Re: S2: High speed compression with upgrade path from Snappy.

2019-08-27 Thread Klaus Post
On Monday, 26 August 2019 13:49:23 UTC+2, Jason E. Aten wrote: > > Excellent! Thank you, Klaus! > > It sounds like it does, but just to be clear: does S2 also replace/upgrade > the snappy *streaming* format (the outer format that is different from > snappy itself; e.g.

[go-nuts] S2: High speed compression with upgrade path from Snappy.

2019-08-26 Thread Klaus Post
Hi! S2 is an extension of Snappy. It can decode Snappy content but S2 content cannot be decoded by Snappy. S2 is aimed for high throughput, so it features concurrent compression for bigger payloads (streams). Benefits over Snappy: - Better compression - Concurrent

Re: [go-nuts] Re: [ANN] VictoriaMetrics - fast open source time series database written in Go

2019-07-11 Thread Klaus Post
On Thursday, 11 July 2019 17:37:09 UTC+2, Aliaksandr Valialkin wrote: > > > > This is really great idea! Will try implementing it. > > Does github.com/klauspost/compress support all the levels for data > decompression? VictoriaMetrics varies compression level depending on the > data type. It

Re: [go-nuts] Re: [ANN] VictoriaMetrics - fast open source time series database written in Go

2019-07-11 Thread Klaus Post
On Sunday, 23 June 2019 20:09:48 UTC+2, Aliaksandr Valialkin wrote: > > > > On Sun, Jun 23, 2019 at 8:34 PM Jason E. Aten > wrote: > >> very nice. >> >> https://github.com/klauspost/compress just added zstd all in Go, might >> be a lovely way to remove the CGO dependency. >> > > I'm keeping eye

[go-nuts] ANN: Zstandard Go compression package

2019-06-02 Thread Klaus Post
I am happy to announce the availability of the first Zstandard compressor written in Go. This package will allow you to get very fast compression to the Zstandard format. The current compressor is focused mainly on speed and outperforms gzip/deflate typically by a factor of 3x for similar

[go-nuts] Re: How to monitor "Out of memory" crash

2018-10-07 Thread Klaus Post
On Friday, 5 October 2018 12:56:50 UTC+2, Thomas S wrote: > > > My service crash because "panic - out of memory", and I'm not sure how to > monitore it. > I just want to know, when it crashes, where are the main memory > allocations, to understand the problem. > > It is pretty sudden, I can

[go-nuts] Re: Zip file created by archive/zip can't be opened on Windows machine

2017-11-12 Thread Klaus Post
Hi! Windows cannot extract password protected ZIP files without third party software. Sorry, but it a beef you need to take with Microsoft. ref:

[go-nuts] Re: How make fastest map[uint32]uint64 with fixed size(250m)?

2017-07-24 Thread Klaus Post
On Monday, 24 July 2017 17:32:48 UTC+2, il wrote: > > I have task: > > 1. Initialize map(250 million items) > 2. Fill all values > 3. Start http server. > 4. Process requests. Iterates all values. > > I need fast "for" and read access with reasonable memory consumption. This > is very important.

[go-nuts] Re: golang too many json unmarshal trigger gc frequency to many

2016-10-28 Thread Klaus Post
On Friday, 28 October 2016 07:36:20 UTC+2, Erik Dubbelboer wrote: > > Try https://github.com/pquerna/ffjson to improve speed and reduce garbage > generation. This in combination with sync.Pool will increase your > throughput by A LOT. > Also try out http://ugorji.net/blog/go-codec-primer -

Re: [go-nuts] Length of Channel Using len Function

2016-08-28 Thread Klaus Post
On Sunday, 28 August 2016 17:33:10 UTC+2, Jan Mercl wrote: > > Using len(ch) like this in a concurrency scenario is a big no because then > the value you get carries 0 bits of useful information. It's not a data > race, it's worse, the race is semantic and not fixable without removing the > use

Re: [go-nuts] Re: Who wants to use Go to process your camera's raw files?

2016-08-13 Thread Klaus Post
On Saturday, 13 August 2016 17:22:29 UTC+2, Peter Herth wrote: > > Having a parallel version of the raw converter could yield conversion > times far faster then the original C. And it would be a neat Go library to > have. > For the biggest part, I would not expect you to be able to beat C

[go-nuts] Re: Who wants to use Go to process your camera's raw files?

2016-08-13 Thread Klaus Post
On Saturday, 13 August 2016 17:18:16 UTC+2, Paul wrote: > > From what I gather even Adobe uses [dcraw]. > I am pretty sure it is the other way around. Observing for years, it seems like dc is reverse engineering the Adobe DNG Converter. His color conversion matrices are definitely from that,

[go-nuts] Re: Who wants to use Go to process your camera's raw files?

2016-08-13 Thread Klaus Post
On Wednesday, 27 July 2016 04:36:31 UTC+2, Jonathan Pittman wrote: > > Well me too! I am looking to see what level of interest there is in the > Go community to see this happen. I am also looking for people who are > interested in working on this. > (a bit late to the party) I am the author