Re: [go-nuts] Re: Possible Go Compiler or Runtime bug?

[Ian Lance Taylor]

On Thu, Jul 27, 2023 at 3:17 PM 'Michael Knyszek' via golang-nuts
 wrote:
>
> The number 32 for the capacity comes from this constant, and the returned 
> byte slice in that case is likely coming from a compiler-inserted call to 
> stringtoslicebyte. An empty string is correctly replaced by a zero-length 
> byte slice, it just so happens that the capacity of the array backing that 
> slice may or may not be zero. As Brian said, commenting or uncommenting that 
> line likely just leads to the byte slice's backing array being allocated 
> differently.
>
> I believe this is working as intended, because I don't think the spec makes 
> any guarantees about the capacity of the slice you get back from a 
> conversion. On the one hand, it is a little surprising given that the make 
> builtin very explicitly sets the capacity of the slice. On the other hand, 
> append is an example of a built-in that may return a slice with a larger 
> capacity than requested, so it's not like this case is the only outlier.

I agree that this is not a bug.  The language makes no promises about
the capacity of the slice returned by a string to []byte conversion.

The comment in the code

//this crashes, panics because capacity is 0, even though I can access
all 32 bytes in the loop above

is misleading.  In the case that crashes, the program can't access all
32 bytes in the loop above.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcWV%3D-g9m8A_EhLnTyuK0x2ffkgDP5wCJtPKG%3DWkM77FUg%40mail.gmail.com.

[go-nuts] Re: Possible Go Compiler or Runtime bug?

['Michael Knyszek' via golang-nuts]

On Thursday, July 27, 2023 at 6:16:57 PM UTC-4 Michael Knyszek wrote:

I believe this is working as intended, because I don't think the spec 
 makes any guarantees about the 
capacity of the slice you get back from a conversion.

 (Other than the fact that the capacity must always be >= the length.)


On Thursday, July 27, 2023 at 8:57:58 AM UTC-4 Brian Candler wrote:

Interesting:
https://play.golang.com/p/nSZ7tKSeot4

With the Printf commented out, it shows t has a cap of 32.  With Printf 
uncommented, the cap drops to 0.  Maybe the slice buffer is allocated on 
the stack in one case, and the heap on another?

The slice header of 24 bytes shouldn't have anything to do with this. The 
cap() of a slice relates to the amount of storage allocated for the data 
elements (which the header points to), not the space consumed by the header 
itself.

On Thursday, 27 July 2023 at 13:42:21 UTC+1 Brian Candler wrote:

That looks very weird. The panic is triggered if I uncomment line 17 (the 
final fmt.Printf) even though it never reaches there - it panics when 
getStrBytes is called from line 9 (in line 23).

On Thursday, 27 July 2023 at 13:12:40 UTC+1 Kyle Harrity wrote:

I first asked this on https://reddit.com/r/golang but the contribution 
guide on github recommends this forum, so I'll post here before finally 
raising an issue on github if this appears to be a real bug.

ORIGINAL POST:

I came across this issue in some code I was reviewing today where a string 
is converted into a []byte and then a 32 byte slice is taken from that and 
returned. It returns a 32 byte slice even if the string is empty or less 
than 32 bytes in length as long as its not a string literal (comes from a 
function or stored in variable). I can index the slice normally and iterate 
over its elements, but attempting to print it with fmt.Printf causes a 
runtime error where it realizes the capacity is not actually 32. Trying to 
get a slice larger than 32 fails though smaller slices are okay. I think 
that has something to do with the storage needed to describe a slice 8 
bytes for memory location, 8 bytes for size, 8 bytes for capacity, 8 for 
padding as explained here: 
https://stackoverflow.com/questions/67839752/why-does-an-empty-slice-have-24-bytes

Here's a playground demo: https://play.golang.com/p/yiLPvRYq8PJ 

Maybe this is a known issue and or expected behavior so I thought I'd ask 
here before raising an issue on github.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8fdee71a-c30a-443b-894e-a589b43ec33en%40googlegroups.com.

[go-nuts] Re: Possible Go Compiler or Runtime bug?

['Michael Knyszek' via golang-nuts]

The number 32 for the capacity comes from this constant 
,
 
and the returned byte slice in that case is likely coming from a 
compiler-inserted call to stringtoslicebyte 
.
 
An empty string is correctly replaced by a zero-length byte slice, it just 
so happens that the capacity of the array backing that slice may or may not 
be zero. As Brian said, commenting or uncommenting that line likely just 
leads to the byte slice's backing array being allocated differently.

I believe this is working as intended, because I don't think the spec 
 makes any guarantees about the 
capacity of the slice you get back from a conversion. On the one hand, it 
is a little surprising given that the make builtin very explicitly sets the 
capacity of the slice 
. On the other 
hand, append  is an 
example of a built-in that may return a slice with a larger capacity than 
requested, so it's not like this case is the only outlier.

On Thursday, July 27, 2023 at 8:57:58 AM UTC-4 Brian Candler wrote:

> Interesting:
> https://play.golang.com/p/nSZ7tKSeot4
>
> With the Printf commented out, it shows t has a cap of 32.  With Printf 
> uncommented, the cap drops to 0.  Maybe the slice buffer is allocated on 
> the stack in one case, and the heap on another?
>
> The slice header of 24 bytes shouldn't have anything to do with this. The 
> cap() of a slice relates to the amount of storage allocated for the data 
> elements (which the header points to), not the space consumed by the header 
> itself.
>
> On Thursday, 27 July 2023 at 13:42:21 UTC+1 Brian Candler wrote:
>
>> That looks very weird. The panic is triggered if I uncomment line 17 (the 
>> final fmt.Printf) even though it never reaches there - it panics when 
>> getStrBytes is called from line 9 (in line 23).
>>
>> On Thursday, 27 July 2023 at 13:12:40 UTC+1 Kyle Harrity wrote:
>>
>>> I first asked this on https://reddit.com/r/golang but the contribution 
>>> guide on github recommends this forum, so I'll post here before finally 
>>> raising an issue on github if this appears to be a real bug.
>>>
>>> ORIGINAL POST:
>>>
>>> I came across this issue in some code I was reviewing today where a 
>>> string is converted into a []byte and then a 32 byte slice is taken from 
>>> that and returned. It returns a 32 byte slice even if the string is empty 
>>> or less than 32 bytes in length as long as its not a string literal (comes 
>>> from a function or stored in variable). I can index the slice normally and 
>>> iterate over its elements, but attempting to print it with fmt.Printf 
>>> causes a runtime error where it realizes the capacity is not actually 32. 
>>> Trying to get a slice larger than 32 fails though smaller slices are okay. 
>>> I think that has something to do with the storage needed to describe a 
>>> slice 8 bytes for memory location, 8 bytes for size, 8 bytes for capacity, 
>>> 8 for padding as explained here: 
>>> https://stackoverflow.com/questions/67839752/why-does-an-empty-slice-have-24-bytes
>>>
>>> Here's a playground demo: https://play.golang.com/p/yiLPvRYq8PJ 
>>>
>>> Maybe this is a known issue and or expected behavior so I thought I'd 
>>> ask here before raising an issue on github.
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/a8445637-e17c-4b9d-b0bc-b2fe63519e6fn%40googlegroups.com.

[go-nuts] Re: Possible Go Compiler or Runtime bug?

[Brian Candler]

Interesting:
https://play.golang.com/p/nSZ7tKSeot4

With the Printf commented out, it shows t has a cap of 32.  With Printf 
uncommented, the cap drops to 0.  Maybe the slice buffer is allocated on 
the stack in one case, and the heap on another?

The slice header of 24 bytes shouldn't have anything to do with this. The 
cap() of a slice relates to the amount of storage allocated for the data 
elements (which the header points to), not the space consumed by the header 
itself.

On Thursday, 27 July 2023 at 13:42:21 UTC+1 Brian Candler wrote:

> That looks very weird. The panic is triggered if I uncomment line 17 (the 
> final fmt.Printf) even though it never reaches there - it panics when 
> getStrBytes is called from line 9 (in line 23).
>
> On Thursday, 27 July 2023 at 13:12:40 UTC+1 Kyle Harrity wrote:
>
>> I first asked this on https://reddit.com/r/golang but the contribution 
>> guide on github recommends this forum, so I'll post here before finally 
>> raising an issue on github if this appears to be a real bug.
>>
>> ORIGINAL POST:
>>
>> I came across this issue in some code I was reviewing today where a 
>> string is converted into a []byte and then a 32 byte slice is taken from 
>> that and returned. It returns a 32 byte slice even if the string is empty 
>> or less than 32 bytes in length as long as its not a string literal (comes 
>> from a function or stored in variable). I can index the slice normally and 
>> iterate over its elements, but attempting to print it with fmt.Printf 
>> causes a runtime error where it realizes the capacity is not actually 32. 
>> Trying to get a slice larger than 32 fails though smaller slices are okay. 
>> I think that has something to do with the storage needed to describe a 
>> slice 8 bytes for memory location, 8 bytes for size, 8 bytes for capacity, 
>> 8 for padding as explained here: 
>> https://stackoverflow.com/questions/67839752/why-does-an-empty-slice-have-24-bytes
>>
>> Here's a playground demo: https://play.golang.com/p/yiLPvRYq8PJ 
>>
>> Maybe this is a known issue and or expected behavior so I thought I'd ask 
>> here before raising an issue on github.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/a5654b4d-304e-4cf9-b2d3-70e72b54132cn%40googlegroups.com.

[go-nuts] Re: Possible Go Compiler or Runtime bug?

[Brian Candler]

That looks very weird. The panic is triggered if I uncomment line 17 (the 
final fmt.Printf) even though it never reaches there - it panics when 
getStrBytes is called from line 9 (in line 23).

On Thursday, 27 July 2023 at 13:12:40 UTC+1 Kyle Harrity wrote:

> I first asked this on https://reddit.com/r/golang but the contribution 
> guide on github recommends this forum, so I'll post here before finally 
> raising an issue on github if this appears to be a real bug.
>
> ORIGINAL POST:
>
> I came across this issue in some code I was reviewing today where a string 
> is converted into a []byte and then a 32 byte slice is taken from that and 
> returned. It returns a 32 byte slice even if the string is empty or less 
> than 32 bytes in length as long as its not a string literal (comes from a 
> function or stored in variable). I can index the slice normally and iterate 
> over its elements, but attempting to print it with fmt.Printf causes a 
> runtime error where it realizes the capacity is not actually 32. Trying to 
> get a slice larger than 32 fails though smaller slices are okay. I think 
> that has something to do with the storage needed to describe a slice 8 
> bytes for memory location, 8 bytes for size, 8 bytes for capacity, 8 for 
> padding as explained here: 
> https://stackoverflow.com/questions/67839752/why-does-an-empty-slice-have-24-bytes
>
> Here's a playground demo: https://play.golang.com/p/yiLPvRYq8PJ 
>
> Maybe this is a known issue and or expected behavior so I thought I'd ask 
> here before raising an issue on github.
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/deff8771-ab58-43ee-b479-16c41d74582dn%40googlegroups.com.