Re: [go-nuts] Clarification on unsafe conversion between string <-> []byte

2019-09-24 Thread khr
On Tuesday, September 24, 2019 at 2:44:46 AM UTC-7, fra...@adeven.com wrote: > > Robert Engels, I am not familiar with the two libraries you named. But > from your description I think (I'm not sure) that we have different uses in > mind. > > The escape analysis that would be required for us to

Re: [go-nuts] Clarification on unsafe conversion between string <-> []byte

2019-09-24 Thread francis
Robert Engels, I am not familiar with the two libraries you named. But from your description I think (I'm not sure) that we have different uses in mind. The escape analysis that would be required for us to avoid using unsafe is _possible_, but does not yet exist in the Go compiler. The compiler

Re: [go-nuts] Clarification on unsafe conversion between string <-> []byte

2019-09-24 Thread francis
Keith, I think you've cracked it. It seems very simple in retrospect. Thanks. On Tuesday, September 24, 2019 at 1:40:02 AM UTC+2, Keith Randall wrote: > > In the runtime we use structs like these, but with unsafe.Pointer data > fields (runtime.stringStruct and runtime.slice). They are much

Re: [go-nuts] Clarification on unsafe conversion between string <-> []byte

2019-09-23 Thread 'Keith Randall' via golang-nuts
In the runtime we use structs like these, but with unsafe.Pointer data fields (runtime.stringStruct and runtime.slice). They are much safer to use than reflect's types with uintptr Data fields. Unfortunately we can't change reflect's types because of the Go 1 compatibility guarantee. You can

Re: [go-nuts] Clarification on unsafe conversion between string <-> []byte

2019-09-23 Thread Robert Engels
As someone that has worked with a lot of similar libraries in the HFT space - things like UnsafeString or FastString in Java I would caution against doing this in Go - especially as proposed here. Taking an immutable object like string and making it mutable by accident is a recipe for disaster.

Re: [go-nuts] Clarification on unsafe conversion between string <-> []byte

2019-09-23 Thread Francis
So I think the current state of unsafe conversions of string <-> []byte is roughly 1. Use the reflect Slice/StringHeader struct. These structs give you clear fields to set and read from. If the runtime representation of a string or []byte ever changes then these structs should change to

Re: [go-nuts] Clarification on unsafe conversion between string <-> []byte

2019-09-23 Thread Dan Kortschak
Any particular reason for that? Neither is safer than the other and it's not clear to me that you can actually achieve the goal of having a compile-time check for the correctness of this type of conversion. On Mon, 2019-09-23 at 02:36 -0700, fran...@adeven.com wrote: > But this relies on a

Re: [go-nuts] Clarification on unsafe conversion between string <-> []byte

2019-09-23 Thread francis
That's super interesting. Thanks for the pointer Jan :bow: On Monday, September 23, 2019 at 11:42:55 AM UTC+2, Jan Mercl wrote: > > On Mon, Sep 23, 2019 at 11:37 AM > wrote: > > > ... and the problems with storing a uinptr in a variable are mostly > related to moving garbage collectors ... > >

Re: [go-nuts] Clarification on unsafe conversion between string <-> []byte

2019-09-23 Thread Jan Mercl
On Mon, Sep 23, 2019 at 11:37 AM wrote: > ... and the problems with storing a uinptr in a variable are mostly related > to moving garbage collectors ... Please note that even though, AFAIK, none of the Go compilers so far uses a moving GC, it's not the only source of objects being moved

Re: [go-nuts] Clarification on unsafe conversion between string <-> []byte

2019-09-23 Thread francis
That would work Kortschak. But this relies on a string's representation being the same as, but a bit smaller thabn, a []byte. I would prefer to use the Slice/StringHeader. It's worth noting that _most_ of the problems I described in my initial post are hypothetical at this stage. The issue

Re: [go-nuts] Clarification on unsafe conversion between string <-> []byte

2019-09-23 Thread francis
That would work Kortschak. But this relies on a string's representation being the same as, but a bit smaller thabn, a []byte. I would prefer to use the Slice/StringHeader. It's worth noting that _most_ of the problems I described in my initial post are hypothetical at this stage. The issue

Re: [go-nuts] Clarification on unsafe conversion between string <-> []byte

2019-09-21 Thread Dan Kortschak
func bytesToString(b []byte) string { return *(*string)(unsafe.Pointer()) } https://play.golang.org/p/azJPbl946zj On Fri, 2019-09-20 at 13:30 -0700, Francis wrote: > Thanks Ian, that's a very interesting solution. > > Is there a solution for going in the other direction? Although I >

Re: [go-nuts] Clarification on unsafe conversion between string <-> []byte

2019-09-20 Thread Francis
Thanks Ian, that's a very interesting solution. Is there a solution for going in the other direction? Although I excluded it from the initial post, it was only to reduce the size of the discussion. I would also like to implement func BytesToString(b []byte) string { I don't clearly see how

Re: [go-nuts] Clarification on unsafe conversion between string <-> []byte

2019-09-18 Thread Ian Lance Taylor
On Wed, Sep 18, 2019 at 2:42 AM Francis wrote: > > I am looking at the correct way to convert from a byte slice to a string and > back with no allocations. All very unsafe. > > I think these two cases are fairly symmetrical. So to simplify the discussion > below I will only talk about

[go-nuts] Clarification on unsafe conversion between string <-> []byte

2019-09-18 Thread Francis
I am looking at the correct way to convert from a byte slice to a string and back with no allocations. All very unsafe. I think these two cases are fairly symmetrical. So to simplify the discussion below I will only talk about converting from a string to []byte. func StringToBytes(s string) (b