Re: [go-nuts] cmd/vet: "possible misuse of unsafe.Pointer" on C pointer -> uintptr -> different package -> unsafe.Pointer -> C pointer

On Thu, Feb 13, 2020 at 8:42 AM Alex wrote: > > I have to pass C pointers between packages so I used uintptr like how syscall > does things. > However go vet gives the message "possible misuse of unsafe.Pointer". > > Is there something I could do to avoid vet complaining? > > Package A: > type Fo

[go-nuts] Re: cmd/vet: "possible misuse of unsafe.Pointer" on C pointer -> uintptr -> different package -> unsafe.Pointer -> C pointer

> > Why not just pass the C pointer from one package to another? > >From docs: > Cgo translates C types into equivalent unexported Go types. Because the > translations are unexported, a Go package should not expose C types in its > exported API: a C type used in one Go package is different fr

[go-nuts] Re: cmd/vet: "possible misuse of unsafe.Pointer" on C pointer -> uintptr -> different package -> unsafe.Pointer -> C pointer

Why all the conversions? Why not just pass the C pointer from one package to another? Or an unsafe.Pointer? You can use a placeholder type, even. Make procAddr a *byte, for example. The pointer conversion rules are intended to ensure safety when playing with Go pointers (by that, I mean pointer

[go-nuts] Linux, unix.Capset, and runtime.LockOSThread

Hi! I have a Linux-specific question about capabilities, threads, and syscalls. I have a Go program which runs other programs with specific capabilities set. It basically does the following: cmd := exec.Command(...) runtime.LockOSThread() unix.RawSyscall(unix.CAPSET, ...) cmd.Start() Empirically

[go-nuts] Re: Looking for some nicely broken image files (gif, png, and jpeg)

Ah, awesome, I hadn't done any real looking for fuzzing solutions yet. I'll grab that one and at least use it to start setting up some local testing while I'm hunting around for test data I can include. (I don't think I can grab apache 2 licensed data and toss it into a package with the standard

Re: [go-nuts] Go without garbage collector

For those that are still interested in this...An interesting point - when you run the Java or Go versions you will see CPU usage in excess of 100% - an indicator that the runtimes are parallelizing the work of GC with the application - something that (I don't believe) a malloc based application do

Re: [go-nuts] Go without garbage collector

I find it difficult to believe that"However, on a desktop CPU with 32 MB of LLC, x86-64 mode, std::unordered_map has (if I am reading it correctly) 2.0e6 misses and std::map has 2.2e6 misses."level of cache misses is going to improve the total performance by 2x, but I can't really test. I am runnin

Re: [go-nuts] Go without garbage collector

On Feb 13, 2020, at 1:45 PM, Robert Engels wrote: > > Swift uses GC - it uses a reference counting GC which has been proven to be > inferior compared to tracing collectors - especially in concurrent > environments. Not to mention cycles. Worth noting, too, that Swift does this largely because

Re: [go-nuts] Go without garbage collector

On Thursday, February 13, 2020 at 7:02:37 PM UTC+1, Robert Engels wrote: > > @atom ... I changed the cpp code to use unordered_map (cc and h) and it > made the code slower... (iteration over most hash map implementations is > slower than sorted maps since often you need to skip the unused key slo

Re: [go-nuts] Go without garbage collector

Reference counting is GC (at least if you are using automatic reference counting, which is what all of cited systems do). It was one of the earliest forms of GC as it is the easiest to implement.That Java code does not represent the C++ code. The C++ code is in no way concurrent. The suggested Java

[go-nuts] Re: Looking for some nicely broken image files (gif, png, and jpeg)

Hello Dan, Sorry, one quick clarification -- I had misremembered which image packages are already running in google/oss-fuzz. Of the three image packages you mentioned, I think it is just gif that is missing currently (and not both gif and jpeg missing): https://github.com/google/oss-fuzz/blo

Re: [go-nuts] Go without garbage collector

Then other languages should faithfully replicate C++ code. For example, iteration in Java should be done like this: for( AtomicReference iter = arr.iterate(); iter.get().hasNext(); iter.get().next()) { ... } And no, reference counting is NOT a GC. > On Feb 13, 2020, at 10:45, Robert Engels wr

Re: [go-nuts] Go without garbage collector

The C++ code is the original production code. The other languages were created from the C++ version.Swift uses GC - it uses a reference counting GC which has been proven to be inferior compared to tracing collectors - especially in concurrent environments. Not to mention cycles.Rust has multiple 'r

[go-nuts] Re: Looking for some nicely broken image files (gif, png, and jpeg)

Hello Dan, I would definitely echo the fuzzing suggestion from Jake. In addition, in your quest to find broken images, I would suggest grabbing the images from the dvyukov/go-fuzz corpus, which has a bunch of images that are already broken in "creative" ways for you based on the coverage-guide

[go-nuts] Re: Proposed additions to maphash documentation

I mailed a small CL for this. It now says "uniform distribution on 64-bit integers". That should imply reduction by masking (or other methods) is fine. https://go-review.googlesource.com/c/go/+/219340 On Sunday, February 9, 2020 at 10:52:22 AM UTC-8, Juliusz Chroboczek wrote: > > It would be hel

Re: [go-nuts] Go without garbage collector

No. It doesn’t prove anything. The C++ code is badly written, as it creates completely superfluous objects (probably optimized away) and does an indirect lookup (which will NOT be). This needs to be fixed even if the code is a faithful translation of Java code. So in practice all the supposed

Re: [go-nuts] Go without garbage collector

Just a thought: It would be an interesting experiment to count the memory usage in the C++ version of the benchmark to match Java's memory consumption (1.5-2.5 GB) and to deallocate only those C++ objects which increase memory consumption beyond this limit. At the end of the benchmark, C++ coul

[go-nuts] Re: Looking for some nicely broken image files (gif, png, and jpeg)

Also it's probably easiest if folks have these handy to just propose an addition to the test repository at https://github.com/drswork/image/tree/master/testdata since sending busted images via email is fraught with all sorts of exciting peril. On Thursday, February 13, 2020 at 1:05:50 PM UTC-5 D

[go-nuts] Re: Looking for some nicely broken image files (gif, png, and jpeg)

Fuzz testing is absolutely reasonable and something I want to set up once I've got the first pass of code done. On Thursday, February 13, 2020 at 11:42:16 AM UTC-5 jake...@gmail.com wrote: > This may be slightly tangential, but this seems like the kind of code that > would benefit greatly from

Re: [go-nuts] Go without garbage collector

@atom ... I changed the cpp code to use unordered_map (cc and h) and it made the code slower... (iteration over most hash map implementations is slower than sorted maps since often you need to skip the unused key slots).So I would be very curious to see your code changes where this brought the perf

[go-nuts] Re: cmd/vet: "possible misuse of unsafe.Pointer" on C pointer -> uintptr -> different package -> unsafe.Pointer -> C pointer

I have to use a massive third party API so it's not really a choice to use cgo. Do you have any safer options to pass C pointers between packages? The C pointers are all C allocated and package B needs to call C functions directly and pass those pointers. On Friday, 14 February 2020 01:02:24 UT

Re: [go-nuts] Go without garbage collector

I won't dispute that, but at least this particular case, it requires on-going maintenance by the developer (company). In this case of Java very few performance improvements have required code changes.Using a different structure entirely (hash map vs. tree map) is a different issue - it touches on t

[go-nuts] Re: cmd/vet: "possible misuse of unsafe.Pointer" on C pointer -> uintptr -> different package -> unsafe.Pointer -> C pointer

You need to read https://golang.org/pkg/unsafe/#Pointer *very, very, very* carefully before using unsafe.Pointer in any way. It spells out 6 conversions that are considered "valid". It says: "Code not using these patterns is likely to be invalid today or to become invalid in the future." AFAICT

[go-nuts] Re: Looking for some nicely broken image files (gif, png, and jpeg)

This may be slightly tangential, but this seems like the kind of code that would benefit greatly from fuzz testing, like go-fuzz . For this kind of code, go-fuzz can really help harden it against bad, malformed and malici

[go-nuts] cmd/vet: "possible misuse of unsafe.Pointer" on C pointer -> uintptr -> different package -> unsafe.Pointer -> C pointer

I have to pass C pointers between packages so I used uintptr like how syscall does things. However go vet gives the message "possible misuse of unsafe.Pointer". Is there something I could do to avoid vet complaining? Package A: type Foo struct { procAddr uintptr } func (f Foo) ProcAddr() uintp

Re: [go-nuts] Go without garbage collector

On Thursday, February 13, 2020 at 3:05:45 PM UTC+1, Robert Engels wrote: > > The code hasn’t been changed. The performance numbers have changed > dramatically with no developer intervention. No “hand optimizing” required. > C++ evolves over time also. Hashmaps have been added to C++ in C++11, whi

Re: [go-nuts] Go without garbage collector

One other minor point on the proposed optimization, if this were a concurrent collection you wouldn’t do this (eg a ConcurrentHashMap in Java) because the end point changes. So again, more cognitive load and maintenance issues in the C++ world as you need far deeper knowledge of the systems (i

Re: [go-nuts] Go without garbage collector

To reiterate, this demonstrates the advancements in GC and JVM tech. These types of tests aren’t well representative of Java’s current sweet spot - which is long running server processes with Gigs of memory, but it still perform admirably - even when including startup costs and runtime memory ov

Re: [go-nuts] Re: arm64 builder on raspberry pi 3B/3B+

As does Armbian (armbian.com). I dare say they're my favorite ARM distribution for getting stuff up and running on generic ARM boards; their default SD card installs mostly Just Do The Right Thing (including automatically expanding the filesystem on first boot to fit), and the hardware support i

Re: [go-nuts] Go without garbage collector

I understand that English may not be your primary language, but please reread the first sentence again. Beyond that you still miss the point. The code hasn’t been changed. The performance numbers have changed dramatically with no developer intervention. No “hand optimizing” required. The fa

[go-nuts] Re: arm64 builder on raspberry pi 3B/3B+

Note, Ubuntu Server 19.10 (arm64) works out of the box on Raspberry Pi 3 and 4, It might be an easier option. I've written about my experience of running 19.10 on Pi 4 [1]. But I didn't manage to run 18.04 — headless mode just didn't work and I didn't have opportunity to attach a keyboard and

Re: [go-nuts] Go without garbage collector

On 2/13/20, robert engels wrote: > See this paper/project https://github.com/hundt98847/multi-language-bench > that was done by Google > in 2011. > > ... > > Or Java is almost 2x faster than C++, and Go is nearly the same performance > as C++.

Re: [go-nuts] Go without garbage collector

On Wednesday, 12 February 2020 22:57:04 UTC, robert engels wrote: > > (Or even Go's GC performance progression - but as I mentioned, in this > particular test the lack of a generational collector is holding it back). > > This is discussed in great detail here: https://blog.golang.org/ismmkeynote